caa a22 4500 605514771 CHVBK 20210128100706.0 cr unu---uuuuu 210128e20150701xx s 000 0 eng 10.1007/s10458-014-9262-4 doi (NATIONALLICENCE)springer-10.1007/s10458-014-9262-4 Emergence of emotional appraisal signals in reinforcement learning agents [Elektronische Daten] [Pedro Sequeira, Francisco Melo, Ana Paiva] The positive impact of emotions in decision-making has long been established in both natural and artificial agents. In the perspective of appraisal theories, emotions complement perceptual information, coloring our sensations and guiding our decision-making. However, when designing autonomous agents, is emotional appraisal the best complement to the perceptions? Mechanisms investigated in affective neuroscience provide support for this hypothesis in biological agents. In this paper, we look for similar support in artificial systems. We adopt the intrinsically motivated reinforcement learning framework to investigate different sources of information that can guide decision-making in learning agents, and an evolutionary approach based on genetic programming to identify a small set of such sources that have the largest impact on the performance of the agent in different tasks, as measured by an external evaluation signal. We then show that these sources of information: (i) are applicable in a wider range of environments than those where the agents evolved; (ii) exhibit interesting correspondences to emotional appraisal-like signals previously proposed in the literature, pointing towards our departing hypothesis that the appraisal process might indeed provide essential information to complement perceptual capabilities and thus guide decision-making. The Author(s), 2014 Emotions nationallicence Appraisal theory nationallicence Intrinsic motivation nationallicence Genetic programming nationallicence Reinforcement learning nationallicence Sequeira Pedro INESC-ID/Instituto Superior Técnico, Universidade de Lisboa, TagusPark, Edifício IST, 2744-016, Porto Salvo, Portugal aut Melo Francisco INESC-ID/Instituto Superior Técnico, Universidade de Lisboa, TagusPark, Edifício IST, 2744-016, Porto Salvo, Portugal aut Paiva Ana INESC-ID/Instituto Superior Técnico, Universidade de Lisboa, TagusPark, Edifício IST, 2744-016, Porto Salvo, Portugal aut Autonomous Agents and Multi-Agent Systems Springer US; http://www.springer-ny.com 29/4(2015-07-01), 537-568 1387-2532 29:4<537 2015 29 10458 https://doi.org/10.1007/s10458-014-9262-4 text/html Onlinezugriff via DOI BK010053 XK010053 XK010000 Metadata rights reserved Springer special CC-BY-NC licence nationallicence 1 research-article jats NATIONALLICENCE NATIONALLICENCE NL-springer NATIONALLICENCE 856 40 https://doi.org/10.1007/s10458-014-9262-4 text/html Onlinezugriff via DOI NATIONALLICENCE 700 1- Sequeira Pedro INESC-ID/Instituto Superior Técnico, Universidade de Lisboa, TagusPark, Edifício IST, 2744-016, Porto Salvo, Portugal aut NATIONALLICENCE 700 1- Melo Francisco INESC-ID/Instituto Superior Técnico, Universidade de Lisboa, TagusPark, Edifício IST, 2744-016, Porto Salvo, Portugal aut NATIONALLICENCE 700 1- Paiva Ana INESC-ID/Instituto Superior Técnico, Universidade de Lisboa, TagusPark, Edifício IST, 2744-016, Porto Salvo, Portugal aut NATIONALLICENCE 773 0- Autonomous Agents and Multi-Agent Systems Springer US; http://www.springer-ny.com 29/4(2015-07-01), 537-568 1387-2532 29:4<537 2015 29 10458