Analysis of expedient behavior of various types of automata
D.P. Dimitrichenko
Upload the full text
Abstract. Collective behavior of automata is one of the directions of development of machine learning methods. Such machines fulfil the function of goal-oriented behavior. The machine performs an action, in response to which the environment sends its output signal to the input of the machine. The machine, in accordance with its design, responds to this input signal with the next action. Thus, a closed loop of interaction is built between a certain environment and the machine operating in it. This environment itself in many cases allows for machine implementation. Effectiveness evaluation of the machine is defined as an optimization problem of maximizing the sum of positive signals (rewards), or minimizing negative signals (penalties), received from the environment, over the considered period of time. Formalization of both the properties of the environment and the actions of the machines, as well as processing of the obtained results is performed using the apparatus of game theory. In this case, signals from the environment are conveniently represented as the sums of the winnings and losses of the players-machines. In this paper, a comparison of machines of different designs is carried out, since the efficiency of machine reactions is determined not only by the properties of the environment, but also by such parameters as the type and depth of memory.
Keywords: automaton, expedient behavior, optimal strategy, memory depth, game theory, formalization of the environment, dynamic environment
For citation. Dimitrichenko D.P. Analysis of expedient behavior of various types of automata. News of the Kabardino-Balkarian Scientific Center of RAS. 2024. Vol. 26. No. 6. Pp. 165–174. DOI: 10.35330/1991-6639-2024-26-6-165-174
References
- Stefanyuk V.L. Lokal’naya organizatsiya intellektual’nykh sistem [Local organization of intelligent systems]. Moscow: FIZMATLIT, 2004. 328 p. (In Russian)
- Dimitrichenko D.P. Optimization of a recurrent neural network using automata with variable structure. Programmnyye sistemy i vychislitel’nyye metody [Software systems and computational methods]. 2023. No. 4. Pp. 30–43. DOI: 10.7256/2454-0714.2023.4.69011. (In Russian)
- Pospelov D.A. Igry i avtomaty [Games and automata]. Moscow: Energiya, 1966. 136 p. (In Russian)
- Tsetlin M.L. Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem [Studies in the theory of automata and modeling of biological systems]. Moscow, 1969. 316 p. (In Russian)
- Pospelov D.A. Veroyatnostnyye avtomaty [Probabilistic automata]. Moscow: Energiya, 88 p. (In Russian)
- Varshavskii V.I. Kollektivnoye povedeniye avtomatov [Collective behavior of automata]. Moscow: Nauka, 1973. 408 p. (In Russian)
- Varshavskii V.I., Pospelov D.A. Orkestr igrayet bez dirizhera: razmyshleniya ob evolyutsii nekotorykh tekhnicheskikh sistem i upravleniye imi [The orchestra plays without a conductor: reflections on the evolution of some technical systems and their control]. Moscow: Nauka, 1984. 208 p. (In Russian)
Information about the author
Dmitry P. Dimitrichenko, Candidate of Technical Sciences, Senior Researcher, the Department
of Neuroinformatics and Machine Learning, Institute of Applied Mathematics and Automation – branch of
Kabardino-Balkarian Scientific Center of the Russian Academy of Sciences;
360000, Russia, Nalchik, 89 A Shortanov street;
dimdp@rambler.ru, ORCID: https://orcid.org/0000-0003-2399-3538, SPIN-code: 3272-3520