Learning automata are stochastic finite state machines that attempt to learn the characteristic of a random environment with which they interact. The fundamental problem is that of learning, through feedback, the action which has the highest probability of being rewarded by the environment. The problem of designing automata for stationary environments has been extensively studied. When the environment is non-stationary, the question of modelling the non-stationarity is, in itself, a very interesting problem. In this paper, we generalize the model used in [14,15] to present three models of non-stationarity. In the first two, the non-stationarity is modelled by a homogeneous Markov chain governing the way in which the characteristics change. The final model considers the more general case when the transition matrix of this chain itself changes with time in a geometric manner. In each case we have analyzed the stochastic properties of the resultant switching environment. The question of analyzing various automata interacting with these environments is open.

Additional Metadata
Conference Proceedings of 1993 International Conference on Systems, Man and Cybernetics
Oommen, J, & Masum, Hassan (Hassan). (1993). On modelling non-stationary random environments using switching techniques. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (pp. 572–577).