The reinforcement learning team works on both fundamental and applied AI research, with a particular focus on reinforcement learning. Reinforcement learning is the study of sequential decision making under uncertainty, and hence encompasses many problems, from robotic manipulation to coordinating multiple goal-based agents. Our aim is to improve our understanding of biological and artificial intelligence, and use these insights to develop AI systems that can be applied to complex tasks in the real world.

Members

Kai Arulkumaran
Research Team Leader

Kai is a Research Team Leader at Araya and Visiting Researcher at Imperial College London. He received his B.A. in Computer Science at the University of Cambridge in 2012 and his Ph.D. in Bioengineering at Imperial College London in 2020. He has previously worked at DeepMind, Microsoft Research, Facebook AI Research, Twitter and NNAISENSE. His research interests are deep learning, reinforcement learning, evolutionary computation and theoretical neuroscience.

Google Scholar

WebsiteTwitterEmail
Manuel Baltieri
Researcher

Manuel is a Researcher at Araya and a Visiting Researcher at the University of Sussex. After graduating with a B.Eng. in Computer Engineering and Business Administration at the University of Trento, he received an M.Sc. in Evolutionary and Adaptive Systems and a Ph.D. in Computer Science and AI, both from the University of Sussex. Following that, he was awarded a JSPS/Royal Society postdoctoral fellowship, and worked in the Lab for Neural Computation and Adaptation at RIKEN CBS with Taro Toyoizumi, until he joined Araya at the end of 2021. His research interests include artificial intelligence and artificial life, theories of agency and individuality, origins of life, embodied cognition and decision making.

Google Scholar

WebsiteTwitterEmail
Dan Ogawa Lillrank
Junior Researcher

Dan is a Junior Researcher at Araya. He received his B.A. in Engineering Physics at Royal Institute of Technology Sweden in 2015, and his M.S. in Robotics at 2018. He has previously worked at Telexistence Inc, and as a Researcher at AIST. His research interests are mainly in robotics learning, the intersection of machine learning & robotics.

TwitterEmail
Roberto Gallotta
Junior Researcher

Roberto is a Junior Researcher in Araya. He obtained his B.S. in Computer and Electronic Engineering at Università di Pavia and his M.S. in Artificial Intelligence and Robotics at La Sapienza University of Rome. His topics of interest are artificial life, evolutionary strategies and open-endedness.

WebsiteTwitterEmail
Shogo Akiyama
Engineer

Shogo is an Engineer at Araya. He received his B.S. in Computer Science in 2019. He has previously worked as an AI and web application engineer. His research interests are reinforcement learning and natural language processing.

TwitterEmail
Rousslan Dossa
Intern
Rousslan is an Intern at Araya, and a doctoral student at the Graduate School of System Informatics at the Kobe University expected to graduate in 2023. His research interests span over the topics of deep reinforcement learning with an emphasis on self-supervised learning, human cognition-inspired decision-making, neuroscience, and evolutionary computing.
WebsiteTwitter
Roman Koshkin
Intern
Roman is an Intern at Araya. After earning his M.A. in Cognitive Psychology from the National Research University HSE (Russia), he joined the Center for Bioelectric Interfaces (Russia), where he used ML/DL models to infer mental states based on scalp-recorded EEG. More recently, Roman's interests have shifted towards continual and sample-efficient learning in biologically-inpired spiking neural networks.
WebsiteTwitter

Highlights

Procedural Content Generation for Space Engineers

Many video games rely on procedural content generation (PCG) in order to (semi-)autonomously create a large amount of game content. For some assets, such as vehicles, the generated content must be both functional and aesthetically-pleasing. In addition, the designer, or even player, typically wants a variety of options to choose from. To achieve this for the Space Engineers 3D sandbox game, we have been using a combination of techniques from evolutionary computation. In “Evolving Spaceships with a Hybrid L-system Constrained Optimisation Evolutionary Algorithm”, we developed a novel hybrid evolutionary algorithm to generate interesting and functional spaceships. We later improved upon this in “Surrogate Infeasible Fitness Acquirement FI-2Pop for Procedural Content Generation”, by developing a novel fitness function, and further applying it to the well-known MAP-Elites “quality-diversity” algorithm.

Benchmarking Imitation Learning

Researchers have developed many imitation learning method over the last few years, each claiming state-of-the-art performance on various tasks. However, results can be easily influenced by factors outside of the claimed contributions of a method, such as data preprocessing, extensive hyperparameter tuning, or simply more computation. In A Pragmatic Look at Deep Imitation Learning, we broke down the contributions of many methods, and developed a unified framework for imitation learning methods, allowing us to perform a fair comparison between algorithms. We have also released an open source library for other researchers to research or apply imitation learning methods.

Brain-Computer Interface Robot Control

Brain-computer interfaces (BCIs) provide a direct connection between our nervous signals and computers, enabling humans with neurological/physiological disorders to interact with the world in ways that would otherwise be impossible. However, BCI inputs are challenging to work with, requiring sophisticated signal processing and machine learning techniques. Coupling BCIs with robots through the shared autonomy paradigm would enable users to physically interact with the world, but likewise, real-world robots is challenging, requiring further state-of-the-art methods in artificial intelligence and control theory. We are currently working on integrating and developing research on all of these areas in order to achieve more generalisable BCI robot control.