Manifesto

Intrinsically Motivated Open-ended Learning (IMOL) is one of the most exciting areas of autonomous/developmental robotics and machine learning/developmental AI. IMOL aims at the development of artificial intelligent systems which are able to autonomously acquire knowledge and skills ‘for their own sake’, through the interaction with the environment. Although related to other approaches pursuing similar goals such as ‘lifelong learning’, ‘continual learning’, and ‘curriculum learning’, IMOL focuses on the role of intrinsic/extrinsic motivations, goals, and hierarchical architectures to support the autonomous development of a broad repertoire of knowledge and skills. This website is the major connecting portal for the community of researchers, practitioners and other people interested in these issues, promoting the exchange of ideas, interactions, tools and cumulative results.

The development of artificial agents with the capacity to learn how to interact with the environment from only little prior knowledge and without constant human supervision is a key step towards a wider deployment of intelligent machines and robots in highly unstructured or even unfamiliar scenarios, such as those in which humans live.

Without denying the important role of social mechanisms (such as imitation, teaching, etc.) in enhancing learning processes, the IMOL community leverages the power of intrinsic motivations in driving artificial agents to lead machines to autonomously acquire knowledge through direct interaction with the environment. Such motivations drive agents to interact with new objects, discover new ways to modify their environment and, similarly to children or scientists, acquire new knowledge and skills driven by 'pure curiosity' and a desire to understand the world around them.

These autonomous learning paradigms should substantially increase the ability of the agents to adapt to and manipulate their environment and thus ultimately successfully complete given, 'extrinsic' tasks and goals assigned to them by human users; outside of engineering, in real organisms, these tasks/goals are imposed by the requirements of survival and expressed as extrinsic motivations such as hunger and pain).

IMOL solutions may incorporate a wide spectrum of machine learning approaches, such as supervised and unsupervised learning. Hierarchical reinforcement learning plays a key role in these systems due to its potential to support autonomous learning when applied in synergy with intrinsic motivations. Shallow and deep neural networks, universal policies, end-to-end learning, and visual planning are other important components for IMOL systems.

The IMOL community first formed during the ‘IMOL Workshop’, a small but highly interactive workshop held the first time in Venice in 2009 (see the archive). The main feature of the community and of this Workshop is the aim to identify fresh approaches to learning, identify open problems for IMOL and innovative strategies for their solution inspired by natural intelligence (animal and human), rather than constraining oneself to only search for solutions to specific given tasks and to compare existing models.

As examples of the type of problems and strategies studied by the community and proposed in the literature we list the following: intrinsic motivations as an engine which can drive fully autonomous learning, the role of self-generation of goals for autonomous development, autonomous learning mechanisms based on complex dynamic systems and emergence, the interplay of intrinsic and extrinsic motivations, the development or discovery of cognitive architectures able to support cumulative open-ended learning, the discrete vs. continuous nature of autonomously acquired sensorimotor knowledge, and the integration of learned habitual behaviour and deliberative processes.

In the service of above research, the IMOL Community pursues a number of concrete objectives and invites new members to join and extend its activities. These include:

• Organising the biannual ‘IMOL Workshop’ (on odd years)
• Organising the biannual ‘IMOL Hands-on Workshop’ (on even years)
• Organising the development and run of IMOL benchmarks and competitions
• Offer resources for the IMOL community, such as models, lists of relevant publications, and job offers
• Foster the identification and implementation activities promoting IMOL research
• Manage the Community Board and other organs coordinating the Community

Examples of specific topics of interest for the IMOL community (non-exhaustive):

• Intrinsic motivations
• Autonomous open-ended learning machines and robots
• Architectures for open-ended learning
• Autonomous representation learning of states and skills
• Hierarchical and multi-task reinforcement learning
• Deep reinforcement learning
• Curriculum learning
• Goal self-generation
• Goal-based skill learning
• Compositionality and chunking
• Knowledge transfer and avoidance of catastrophic forgetting
• Neural/probabilistic representations and abstractions
• Open-ended learning of visual and multimodal representations
• Abstraction and hierarchies of goals and skills
• Visual planning and problem solving
• Software/hardware (servers, simulators, robots) supporting long learning processes
• Closing the simulation-to-reality gap in autonomous robots
• Principles underlying open-ended development in children
• Philosophical and ethical implications of IMOL research
• Mitigation of risks of real-world use of open-ended learning systems