X Communication Team is advancing research and development in "heart-connecting BMI (Brain-Machine Interface)." By integrating Araya's strengths in cutting-edge AI, neuroscience, and theoretical studies on consciousness, they are exploring the possibilities of new communication methods beyond speech. This endeavor aims to create a society where diverse people can understand each other better than ever before. Furthermore, this team's efforts are part of the Moonshot Goal 1 Kanai Project: Internet of Brains (IoB).

X communication
EXodus from limitations of body, brain, space, and time, realized by eXtra facility enabling eXpanded communication.

KEY WORDS

#Consciousness #InformationTheoreticResearch #IntegratedInformationTheory #GlobalWorkspaceTheory #GenerativeAI #BMI #NonInvasive #MoonshotResearchAndDevelopmentInitiative

INTRODUCTION

Development of Non-Invasive Speech-Decoding BMI

X Communication Team is currently working on the development of a non-invasive speech-decoding BMI (Brain-Machine Interface). Specifically, this involves decoding words a user wants to convey or select, using brainwaves and AI. The decoded information is then transcribed into text inputs for computers or smartphones or converted into speech through AI synthesis, enabling communication.

A BMI (Brain-Machine Interface) technology directly connects the brain and machines, utilizing brain information. It is achieved by combining brain activity measurement technologies, like EEG, with analysis technologies, such as artificial intelligence, to extract human intentions. For example, this includes technologies that interpret user intentions from brainwaves to control computers or devices that transmit information directly to the brain, providing sensory experiences like vision or taste without involving sensory organs. These BMIs are part of the broader application of Neurotech, which encompasses technologies based on neuroscience. Neurotech includes monitoring brain activity, stimulating the brain for treatment or enhancement, and supporting technologies.

There are two types of BMIs: invasive and non-invasive. Invasive BMIs involve physically damaging methods like implanting electrodes in the brain. The advantage is the ability to obtain high-precision signals from the brain directly. However, the drawbacks include the high physical and psychological barriers due to the need for surgical implantation and the difficulty in capturing information from many brain areas.

Non-invasive BMIs, on the other hand, involve methods that do not physically harm the body, like placing electrodes on the scalp. Their advantage is the lower physical and psychological barriers compared to invasive methods, as they do not require surgery and can measure brain activity simply by wearing headgear or entering a scanner. This opens up possibilities for general use in non-medical fields like healthcare. However, there are disadvantages, such as the need to wear equipment every time and the lower precision of the signals.

The X Communication Team aims to implement non-invasive BMI in society. The potential for communication using BMI is not limited to individuals with physical disabilities but is open to everyone. Through the development of devices that can be widely used by the general public, the team strives to contribute to solving societal challenges in communication.

HIGHLIGHTS

AI-Assisted BMI

At Araya, they are advocating for the practical implementation of non-invasive BMIs through what they call "AI-Assisted BMIs." An AI-Assisted BMI is a Brain-Machine Interface that actively utilizes AI technology to enhance the capability of analyzing brainwaves. With AI technology, even brainwave data of lower accuracy, which is a characteristic of non-invasive as compared to invasive BMIs, can be sufficiently utilized. Through this complementary relationship between AI and humans, they aim to address societal challenges. For specific examples, please refer to the two experimental contents mentioned later.

Trusted Brain Tech / BMI の実現に向けて①「身体的能力と知覚能力の拡張による身体の成約からの開放」の紹介」金井良太(プロジェクトマネージャー)

 

Development of Brainwave Measurement System

For brainwave measurement in the experiment, a developed system is being used. The aim is to overcome the challenges of non-invasive EEG as much as possible and to make the most of the electrical signals derived from brain activity.
There are mainly two challenges with non-invasive EEG:
Compared to invasive methods, the brainwaves are measured as very weak signals due to the interference of the skull and scalp.
Electrical signals originating from facial expressions and eye movements can be stronger than the brainwaves, potentially overshadowing them.
The current solution includes implementing a filter that removes noise signals from the brainwaves in real-time, ensuring the collection of cleaner brainwave data. Two types of electrodes are used in combination: the electrodes of the non-invasive ultra-high-density EEG, which are mounted on the scalp, and electrodes that measure electrical signals originating from facial muscles (see image for reference). Additionally, whispering is encouraged during speech to minimize the output of electromyography (muscle-generated electrical signals).

Experimental Example 1: ChatGPT empowered Brain Gmail Interface

The experiment involves a user with a non-invasive BMI operating Gmail through brainwaves and ChatGPT. Using a non-invasive ultra-high-density EEG, brainwave data was collected when the user spoke the names of specific colors. This data was then used to train an AI, enabling the identification of words from brainwaves. By integrating ChatGPT, it became possible to operate Gmail through brainwaves.

 

Experimental Example 2:Brain-Robot Interface with Imitation Learning

This experiment, conducted as the next step following the "Gmail Operation Experiment with Ultra-High-Density EEG," was carried out in collaboration with the Reinforcement Learning team. It involved a user in Akasaka wearing a non-invasive BMI and remotely operating a robotic arm located in Akihabara using brainwaves. The remote operation of the robotic arm was achieved through the combination of word identification from brainwaves (AI-assisted BMI) and imitation learning.

CONCLUSION

X Communication Team is committed to developing BMIs that can accomplish a wider range of tasks, exploring a broad spectrum of possibilities. Their goal is to help diverse individuals overcome societal barriers and to contribute to widening the options for participation and communication within society. This effort is directed towards creating a more inclusive environment where technology empowers individuals, particularly those facing physical or communicative challenges, to engage more fully with the world around them.

 

Related Links

MineRL Competitions

Members

Shuntaro Sasai, Ph.D.
Chief Research Officer
After graduating from the College of Arts and Sciences at the University of Tokyo in 2008, he received his Ph.D. in the Graduate School of Education at the University of Tokyo in 2013. Since 2013, he is a member of the Center for Sleep and Consciousness at the University of Wisconsin at Madison. His research interests are in understanding the brain-consciousness relationship and developing brain-machine interface techniques enabling cross-brain communications. Currently, he focuses on modeling the brain’s wiring architectures supporting conscious experience and cognitive functions.
Motoshige Sato
Senior Researcher
While attending graduate school at the University of Tokyo, Motoshige interned at Araya . After he received Ph.D (Pharmaceutical Sciences) in 2023, he joined Araya as a senior researcher. He is working on brain decoding and brain-machine interface (BMI) using deep learning, aiming to increase human degree of freedom and augment abilities. Google Scholar HP: https://motoshigesato.encoder.jp/
Masakazu Inoue
Masakazu Inoue
Senior Researcher
Masakazu is a Senior Researcher at Araya. He received his Master’s Degree in Information Science and Technology at the University of Tokyo and joined Araya in 2019. He worked on various projects as a machine learning engineer, focusing on image recognition and edge AI. In 2023, he moved to the R&D department to work on BMI research. He is a PhD candidate at the University of Tokyo.
Yasuo Kabe
Intern
Yasuo is a Ph.D. student in the Department of Mechano-Informatics at the University of Tokyo. His research interest is in developing design principles for human augmentation based on neuroscience.
Kenichi Tomeoka
Kenichi Tomeoka
Intern
He is a master's degree student at the Graduate School of Arts and Sciences, the University of Tokyo. He has started his career in Neuroscience/Neurotech fascinated by the question of how consciousness is generated in the brain. Currently, he is working on the research and development of Brain-Mahcine Interface (BMI), dreaming of solving human difficulties and creating new possibilities using the cutting-edge technology.