We propose a generic modular framework which will enable testing LFO in multiple domains, using multiple learning methods. The framework compares an extension on CBR, called Temporal backtracking with probabilistic graphical model based learning methods such as Bayesian networks, Input-Output Hidden Markov models, Dynamic Bayesian Networks, Neural Networks, and Time-Windowed Bayesian and Neural networks. The framework allows for cross validation and calculation of confusion matrices. The framework calculates multiple different performance measures such as F-measure and accuracy. The framework is used to test how well each learning method learns state-based and reactive behavior in the 2D simulated soccer playing domain RoboCup. It was found that the Temporal backtracking had the highest F-Measure and Time-Windowed Neural Network had the highest accuracy when reproducing state-based behavior in the RoboCup domain.

Additional Metadata
Conference 2017 AAAI Spring Symposium
Gunaratne, S. (Sacha), Esfandiari, B, & Chan, C. (Caleb). (2017). Towards a framework for testing learning from observation of state-based agents. In AAAI Spring Symposium - Technical Report (pp. 499–505).