Towards Model Repair by Human Opinion–Guided Reinforcement Learning

Abstract
Model repair often entails long sequences of model transformations. Finding the correct model repair sequence is challenging, and its complexity increases with the number of model transformations involved in the repair sequence. In realistic, longitudinally extensive modelling settings, the same model repair scenarios might be encountered repeatedly, providing an excellent opportunity to learn the most appropriate repair actions through reinforcement learning (RL). While such ideas have been explored before, the efficiency of RL-based methods in long repair sequences is still an open challenge. In this paper, we propose a method to improve learning performance by human opinions—cognitive constructs that are subject to uncertainty, but also emerge earlier than hard evidence. Our findings indicate that opinion-based guidance significantly improves the learning performance, even with moderately uncertain human opinions. To counter the uncertainty of individual human advisors, our method allows for collaborative guidance by experts of various expertise and skill levels.