The Reinforcement Learning team works on the intersection of both biological and artificial intelligence - literally - with a focus on brain-robot interfaces. Our aim is to build brain-robot interfaces to allow humans to control external robot arms to perform a range of everyday tasks, such as cleaning or preparing food.
The team is participating in the Kanai Project for Moonshot Goal 1, known as the “Internet of Brains (IoB).”

Overview

We are a multidisciplinary team, working on areas spanning AI, robotics, human-computer interaction (HCI), and neuroscience.

AI

Reinforcement learning (RL) is the study of sequential decision making under uncertainty, and therefore forms a basis for developing autonomous agents. RL agents need to trade off exploring their environment, and exploiting it - achieving their tasks as best as possible. Exploration can be unsafe for robotics, so most of our work in AI instead focuses on imitation learning (IL), where agents learn from demonstration data instead of exploration. We’ve investigated the fundamentals of IL [1], and released an open-source library for IL research.

In prior work for robot control [2], we’ve used and improved upon the Perceiver-Actor IL algorithm. Its use of 3D voxels and traditional robot planners makes it particularly data efficient. With only ~30 demonstrations it is able to learn tasks such as opening and closing drawers, and picking and placing objects.

However, like most other robot learning teams, nowadays we’ve started using algorithms such as ACT and Diffusion Policy, which have the potential to learn more challenging dexterous manipulation tasks.

Robotics

One of the key requirements for IL is data collection, and so we also investigate teleoperation solutions. Currently, our favourite solution is GELLO, a miniature, kinematically-equivalent version of our Franka Panda robot, which we have augmented with force feedback to the user.

At a higher level, we are also investigating the use of vision-language models (VLMs) for robot planning, as a replacement for traditional symbolic robot planners. In particular, we are trying to improve open source VLMs for planning, using techniques such as fine-tuning and prompt engineering [3]. Our goal is to be able to query VLMs to inform robot controllers how to achieve tasks that we specify in natural language.

HCI

When studying human-computer interaction (HCI), it is important to be able to quantify results. To address this, we have developed a web-based user interface for controlling multiple robots simultaneously, including both qualitative and quantitative metrics [4]. Our software supports multiple input devices, including the mouse, keyboard, gamepads, eye trackers, and EMG.

Whilst most human-robot interfaces only provide visual feedback, we believe that multimodal interfaces can be even more effective. This is what we investigated in our prior work by developing a visual and auditory EEG interface for robot control, then conducting user studies [5].

Neuroscience

When it comes to decoding brain signals, we are currently focused on online decoding with minimal data collection for calibrating EEG decoders. We believe that traditional EEG pipelines with filtering, feature extraction (e.g., CSP), and “shallow” classifiers (e.g., SVMs), in combination with good machine learning practices, are still best in this regime.

Although the majority of our work revolves around brain-robot interfaces, we have also done some more fundamental research at the intersection of AI and neuroscience. In particular, we believe that AI can help formalise and investigate theories of consciousness [6]. We’ve also implemented the first AI agent that satisfies all 4 indicators of Global Workspace Theory [7].

Summary

Our goal is to create a future where humans can interact with robot agents directly, using their thoughts, enabling us to go beyond the limits of our own physical bodies. We believe that the key to this is to combine the latest research in AI and robotics with intuitive and intelligent human-robot interfaces.

Members

Kai Arulkumaran, Ph.D.
Research Team Lead
Kai is a Research Team Lead at Araya. He received his B.A. in Computer Science at the University of Cambridge in 2012 and his Ph.D. in Bioengineering at Imperial College London in 2020. He has previously worked at DeepMind, Microsoft Research, Facebook AI Research, Twitter Cortex and NNAISENSE. His research interests are deep learning, reinforcement learning, evolutionary computation and theoretical neuroscience.
Manuel Baltieri, Ph.D.
Chief Researcher
Manuel is a Chief Researcher at Araya and a Visiting Researcher at the University of Sussex. After graduating with a B.Eng. in Computer Engineering and Business Administration at the University of Trento, he received an M.Sc. in Evolutionary and Adaptive Systems and a Ph.D. in Computer Science and AI, both from the University of Sussex. Following that, he was awarded a JSPS/Royal Society postdoctoral fellowship, and worked in the Lab for Neural Computation and Adaptation at RIKEN CBS with Taro Toyoizumi, until he joined Araya at the end of 2021. His research interests include artificial intelligence and artificial life, theories of agency and individuality, origins of life, embodied cognition and decision making.
Rousslan Dossa, Ph.D.
Chief Researcher
Rousslan is a Chief Researcher at Araya. He received his Ph.D. from Kobe University in 2023. His research interests span over the topics of deep reinforcement learning with an emphasis on self-supervised learning, human cognition-inspired decision-making, neuroscience, and evolutionary computing.
Shivakanth Sujit
Senior Researcher
Shivakanth is a Senior Researcher at Araya. He received his M.Sc. in 2023 from Mila Quebec working with Prof Samira Ebrahimi Kahou. He is interested in deep reinforcement learning for robotics and LLMs. Before joining Mila he completed his undergraduate at NIT Trichy, India in Control Engineering, and this background drives his research in combining the insights from control theory and RL for building agents that can safely interact in the real world.
Shogo Akiyama
Senior Researcher
Shogo is a Senior Researcher at Araya. He received his B.S. in Computer Science in 2019. He has previously worked as an AI and web application engineer. His research interests are reinforcement learning and natural language processing.
Marina Di Vincenzo
Senior Researcher
Marina is a Senior Researcher at Araya. She received her B.A. in Psychology at the University of Urbino “Carlo Bo” in 2017; her M.S. in Neuroscience and Psychological Rehabilitation in 2021 at the University “La Sapienza” of Rome; and a Diploma in Artificial Intelligence, the same year, at the Institute of Sciences and Technologies of Cognition of the National Research Council (ISTC-CNR). Her research focuses on User-Centered Design in Assistive Neurotechnology.
Hannah Kodama Douglas
Hannah Kodama Douglas
Senior Researcher
Hannah is a Senior Researcher at Araya. She received her B.S. in Statistics at Carnegie Mellon University in 2020 before completing her postbac at the National Institutes of Health in the Unit of Neural Computation and Behavior. She then received her M.S. in Computational Neuroscience from Princeton University in 2024. She's interested in exploring ways to apply insights from neuroscience and machine learning to develop practical brain-machine interfaces.
Luca Nunziante
Luca Nunziante
Senior Researcher
Luca is a Senior Researcher at Araya. After receiving his bachelor's degree in Electronic and Computer Engineering at the University of Campania Luigi Vanvitelli, he completed his M.Sc. in Artificial Intelligence and Robotics at La Sapienza University of Rome in 2024. During his master he visited the Space Robotics Laboratory at Tohoku University, Japan. His research interest are robot control, artificial intelligence, and the intersection of the two.