caa a22 4500 605514607 CHVBK 20210128100705.0 cr unu---uuuuu 210128e20150101xx s 000 0 eng 10.1007/s10458-014-9252-6 doi (NATIONALLICENCE)springer-10.1007/s10458-014-9252-6 Strategies for simulating pedestrian navigation with multiple reinforcement learning agents [Elektronische Daten] [Francisco Martinez-Gil, Miguel Lozano, Fernando Fernández] In this paper, a new multi-agent reinforcement learning approach is introduced for the simulation of pedestrian groups. Unlike other solutions, where the behaviors of the pedestrians are coded in the system, in our approach the agents learn by interacting with the environment. The embodied agents must learn to control their velocity, avoiding obstacles and the other pedestrians, to reach a goal inside the scenario. The main contribution of this paper is to propose this new methodology that uses different iterative learning strategies, combining a vector quantization (state space generalization) with the Q-learning algorithm (VQQL). Two algorithmic schemas, Iterative VQQL and Incremental, which differ in the way of addressing the problems, have been designed and used with and without transfer of knowledge. These algorithms are tested and compared with the VQQL algorithm as a baseline in two scenarios where agents need to solve well-known problems in pedestrian modeling. In the first, agents in a closed room need to reach the unique exit producing and solving a bottleneck. In in the second, two groups of agents inside a corridor need to reach their goal that is placed in opposite sides (they need to solve the crossing). In the first scenario, we focus on scalability, use metrics from the pedestrian modeling field, and compare with the Helbing's social force model. The emergence of collective behaviors, that is, the shell-shaped clogging in front of the exit in the first scenario, and the lane formation as a solution to the problem of the crossing, have been obtained and analyzed. The results demonstrate that the proposed schemas find policies that carry out the tasks, suggesting that they are applicable and generalizable to the simulation of pedestrians groups. The Author(s), 2014 Pedestrian simulation nationallicence VQQL nationallicence Collective behaviors nationallicence MARL nationallicence Martinez-Gil Francisco Departament d'Informàtica, Escola Tècnica Superior d'Enginyeria (ETSE), Universitat de València, Avda. de la Universidad s/n, 46100, Burjassot, Valencia, Spain aut Lozano Miguel Departament d'Informàtica, Escola Tècnica Superior d'Enginyeria (ETSE), Universitat de València, Avda. de la Universidad s/n, 46100, Burjassot, Valencia, Spain aut Fernández Fernando Department of Computer Science, University Carlos III of Madrid, Avda. de la Universidad 30, 28911, Leganés, Madrid, Spain aut Autonomous Agents and Multi-Agent Systems Springer US; http://www.springer-ny.com 29/1(2015-01-01), 98-130 1387-2532 29:1<98 2015 29 10458 https://doi.org/10.1007/s10458-014-9252-6 text/html Onlinezugriff via DOI BK010053 XK010053 XK010000 Metadata rights reserved Springer special CC-BY-NC licence nationallicence 1 research-article jats NATIONALLICENCE NATIONALLICENCE NL-springer NATIONALLICENCE 856 40 https://doi.org/10.1007/s10458-014-9252-6 text/html Onlinezugriff via DOI NATIONALLICENCE 700 1- Martinez-Gil Francisco Departament d'Informàtica, Escola Tècnica Superior d'Enginyeria (ETSE), Universitat de València, Avda. de la Universidad s/n, 46100, Burjassot, Valencia, Spain aut NATIONALLICENCE 700 1- Lozano Miguel Departament d'Informàtica, Escola Tècnica Superior d'Enginyeria (ETSE), Universitat de València, Avda. de la Universidad s/n, 46100, Burjassot, Valencia, Spain aut NATIONALLICENCE 700 1- Fernández Fernando Department of Computer Science, University Carlos III of Madrid, Avda. de la Universidad 30, 28911, Leganés, Madrid, Spain aut NATIONALLICENCE 773 0- Autonomous Agents and Multi-Agent Systems Springer US; http://www.springer-ny.com 29/1(2015-01-01), 98-130 1387-2532 29:1<98 2015 29 10458