caa a22 4500 605478279 CHVBK 20210128100404.0 cr unu---uuuuu 210128e20150901xx s 000 0 eng 10.1007/s10994-015-5484-1 doi (NATIONALLICENCE)springer-10.1007/s10994-015-5484-1 Policy gradient in Lipschitz Markov Decision Processes [Elektronische Daten] [Matteo Pirotta, Marcello Restelli, Luca Bascetta] This paper is about the exploitation of Lipschitz continuity properties for Markov Decision Processes to safely speed up policy-gradient algorithms. Starting from assumptions about the Lipschitz continuity of the state-transition model, the reward function, and the policies considered in the learning process, we show that both the expected return of a policy and its gradient are Lipschitz continuous w.r.t. policy parameters. By leveraging such properties, we define policy-parameter updates that guarantee a performance improvement at each iteration. The proposed methods are empirically evaluated and compared to other related approaches using different configurations of three popular control scenarios: the linear quadratic regulator, the mass-spring-damper system and the ship-steering control. The Author(s), 2015 Reinforcement learning nationallicence Markov Decision Process nationallicence Lipschitz continuity nationallicence Policy gradient algorithm nationallicence Pirotta Matteo Department of Electronics, Information, and Bioengineering, Politecnico di Milano, Piazza Leonardo Da Vinci, 32, I-20133, Milan, Italy aut Restelli Marcello Department of Electronics, Information, and Bioengineering, Politecnico di Milano, Piazza Leonardo Da Vinci, 32, I-20133, Milan, Italy aut Bascetta Luca Department of Electronics, Information, and Bioengineering, Politecnico di Milano, Piazza Leonardo Da Vinci, 32, I-20133, Milan, Italy aut Machine Learning Springer US; http://www.springer-ny.com 100/2-3(2015-09-01), 255-283 0885-6125 100:2-3<255 2015 100 10994 https://doi.org/10.1007/s10994-015-5484-1 text/html Onlinezugriff via DOI BK010053 XK010053 XK010000 Metadata rights reserved Springer special CC-BY-NC licence nationallicence 1 research-article jats NATIONALLICENCE NATIONALLICENCE NL-springer NATIONALLICENCE 856 40 https://doi.org/10.1007/s10994-015-5484-1 text/html Onlinezugriff via DOI NATIONALLICENCE 700 1- Pirotta Matteo Department of Electronics, Information, and Bioengineering, Politecnico di Milano, Piazza Leonardo Da Vinci, 32, I-20133, Milan, Italy aut NATIONALLICENCE 700 1- Restelli Marcello Department of Electronics, Information, and Bioengineering, Politecnico di Milano, Piazza Leonardo Da Vinci, 32, I-20133, Milan, Italy aut NATIONALLICENCE 700 1- Bascetta Luca Department of Electronics, Information, and Bioengineering, Politecnico di Milano, Piazza Leonardo Da Vinci, 32, I-20133, Milan, Italy aut NATIONALLICENCE 773 0- Machine Learning Springer US; http://www.springer-ny.com 100/2-3(2015-09-01), 255-283 0885-6125 100:2-3<255 2015 100 10994