The main contribution of this work is a novel learning algorithm for machine reinforcement learning when Poissonian stochastic time delays are present in the reinforcement signal. The novel approach can deal with rewards which may be received out of order in time or overlap with one another. A PID controller is simulated with and without a stochastic time delay to demonstrate the difficulties of the problem. Experimental results with mobile robots demonstrate that the proposed method improves the performance over that of traditional Q-learning for a learning agent in an environment with Poissonian-type stochastically delayed rewards.

cost, Jitter, Markov Decision Process, Reinforcement learning, reward, stochastic time delay
2015 28th IEEE Canadian Conference on Electrical and Computer Engineering, CCECE 2015
Department of Systems and Computer Engineering

Campbell, J.S. (Jeffrey S.), Givigi, S.N. (Sidney N.), & Schwartz, H.M. (2015). Handling stochastic reward delays in machine reinforcement learning. In Canadian Conference on Electrical and Computer Engineering (pp. 314–319). doi:10.1109/CCECE.2015.7129295