The reinforcement learning team works on both fundamental and applied AI research, with a particular focus on reinforcement learning. Reinforcement learning is the study of sequential decision making under uncertainty, and hence encompasses many problems, from robotic manipulation to coordinating multiple goal-based agents. Our aim is to improve our understanding of biological and artificial intelligence, and use these insights to develop AI systems that can be applied to complex tasks in the real world.

KEY WORDS
#Deep Learning
#Reinforcement Learning
#Imitation Learning
#Evolutionary Computation
#Artificial Life

Highlights

Brain-Computer Interface Robot Control

Brain-computer interfaces (BCIs) provide a direct connection between our nervous signals and computers, enabling humans with neurological/physiological disorders to interact with the world in ways that would otherwise be impossible. However, BCI inputs are challenging to work with, requiring sophisticated signal processing and machine learning techniques. Coupling BCIs with robots through the paradigm of shared autonomy would enable users to physically interact with the world. Unfortunately, operating robots in the real world (outside of highly-constrained factory settings) is challenging, requiring further state-of-the-art methods in artificial intelligence and control theory. We are currently working on integrating and developing research on all of these areas in order to achieve more generalisable BCI robot control.

Benchmarking Imitation Learning

Researchers have developed many imitation learning method over the last few years, each claiming state-of-the-art performance on various tasks. However, results can be easily influenced by factors outside of the claimed contributions of a method, such as data preprocessing, extensive hyperparameter tuning, or simply more computation. In A Pragmatic Look at Deep Imitation Learning, we broke down the contributions of many methods, and developed a unified framework for imitation learning methods, allowing us to perform a fair comparison between algorithms. We have also released an open source library for other researchers to research or apply imitation learning methods.

Procedural Content Generation for Space Engineers

Many video games rely on procedural content generation (PCG) in order to (semi-)autonomously create a large amount of game content. For some assets, such as vehicles, the generated content must be both functional and aesthetically-pleasing. In addition, the designer, or even player, typically wants a variety of options to choose from. To achieve this for the Space Engineers 3D sandbox game, we have been using a combination of techniques from evolutionary computation. In Evolving Spaceships with a Hybrid L-system Constrained Optimisation Evolutionary Algorithm, we developed a novel hybrid evolutionary algorithm to generate interesting and functional spaceships. We later improved upon this in Surrogate Infeasible Fitness Acquirement FI-2Pop for Procedural Content Generation, by developing a novel fitness function, and further applying it to the well-known MAP-Elites "quality-diversity" algorithm.

Members

Kai Arulkumaran, Ph.D.
Research Team Lead
Kai is a Research Team Leader at Araya and Visiting Researcher at Imperial College London. He received his B.A. in Computer Science at the University of Cambridge in 2012 and his Ph.D. in Bioengineering at Imperial College London in 2020. He has previously worked at DeepMind, Microsoft Research, Facebook AI Research, Twitter and NNAISENSE. His research interests are deep learning, reinforcement learning, evolutionary computation and theoretical neuroscience.
Manuel Baltieri, Ph.D.
Chief Researcher
Manuel is a Senior Researcher at Araya and a Visiting Researcher at the University of Sussex. After graduating with a B.Eng. in Computer Engineering and Business Administration at the University of Trento, he received an M.Sc. in Evolutionary and Adaptive Systems and a Ph.D. in Computer Science and AI, both from the University of Sussex. Following that, he was awarded a JSPS/Royal Society postdoctoral fellowship, and worked in the Lab for Neural Computation and Adaptation at RIKEN CBS with Taro Toyoizumi, until he joined Araya at the end of 2021. His research interests include artificial intelligence and artificial life, theories of agency and individuality, origins of life, embodied cognition and decision making.
Dan Ogawa Lillrank
Researcher
Dan is a Researcher at Araya. He received his B.A. in Engineering Physics at Royal Institute of Technology Sweden in 2015, and his M.S. in Robotics at 2018. He has previously worked at Telexistence Inc, and as a Researcher at AIST. His research interests are mainly in robotics learning, the intersection of machine learning & robotics.
Akito Yoshida
Researcher
Akito is a Researcher in Araya. He received his Master's in Information Science and Technology at the University of Tokyo in 2022. He is interested in the process of human-like intelligence development through the interaction between the brain, body and environment.
Shogo Akiyama
Senior Researcher
Shogo is an Engineer at Araya. he received his B.S. in Computer Science in 2019. he has previously worked as an AI and web application engineer. His research interests are reinforcement learning and natural language processing.
Roman Koshkin
Intern
Roman is an Intern at Araya. After earning his MA in Cognitive Psychology from the National Research University HSE (Russia), he joined the Center for Bioelectric Interfaces (Russia), where he used ML / DL models to infer mental states based on scalp- recorded EEG. More recently, Roman's interests have led towards continual and sample-efficient learning in biologically-inpired spiking neural networks.