X Communication Team is advancing research and development in "heart-connecting BMI (Brain-Machine Interface)." By integrating Araya's strengths in cutting-edge AI, neuroscience, and theoretical studies on consciousness, they are exploring the possibilities of new communication methods beyond speech. This endeavor aims to create a society where diverse people can understand each other better than ever before. Furthermore, this team's efforts are part of the Moonshot Goal 1 Kanai Project: Internet of Brains (IoB).
EXodus from limitations of body, brain, space, and time, realized by eXtra facility enabling eXpanded communication.
Development of Non-Invasive Speech-Decoding BMI
X Communication Team is currently working on the development of a non-invasive speech-decoding BMI (Brain-Machine Interface). Specifically, this involves decoding words a user wants to convey or select, using brainwaves and AI. The decoded information is then transcribed into text inputs for computers or smartphones or converted into speech through AI synthesis, enabling communication.
A BMI (Brain-Machine Interface) technology directly connects the brain and machines, utilizing brain information. It is achieved by combining brain activity measurement technologies, like EEG, with analysis technologies, such as artificial intelligence, to extract human intentions. For example, this includes technologies that interpret user intentions from brainwaves to control computers or devices that transmit information directly to the brain, providing sensory experiences like vision or taste without involving sensory organs. These BMIs are part of the broader application of Neurotech, which encompasses technologies based on neuroscience. Neurotech includes monitoring brain activity, stimulating the brain for treatment or enhancement, and supporting technologies.
There are two types of BMIs: invasive and non-invasive. Invasive BMIs involve physically damaging methods like implanting electrodes in the brain. The advantage is the ability to obtain high-precision signals from the brain directly. However, the drawbacks include the high physical and psychological barriers due to the need for surgical implantation and the difficulty in capturing information from many brain areas.
Non-invasive BMIs, on the other hand, involve methods that do not physically harm the body, like placing electrodes on the scalp. Their advantage is the lower physical and psychological barriers compared to invasive methods, as they do not require surgery and can measure brain activity simply by wearing headgear or entering a scanner. This opens up possibilities for general use in non-medical fields like healthcare. However, there are disadvantages, such as the need to wear equipment every time and the lower precision of the signals.
The X Communication Team aims to implement non-invasive BMI in society. The potential for communication using BMI is not limited to individuals with physical disabilities but is open to everyone. Through the development of devices that can be widely used by the general public, the team strives to contribute to solving societal challenges in communication.
At Araya, they are advocating for the practical implementation of non-invasive BMIs through what they call "AI-Assisted BMIs." An AI-Assisted BMI is a Brain-Machine Interface that actively utilizes AI technology to enhance the capability of analyzing brainwaves. With AI technology, even brainwave data of lower accuracy, which is a characteristic of non-invasive as compared to invasive BMIs, can be sufficiently utilized. Through this complementary relationship between AI and humans, they aim to address societal challenges. For specific examples, please refer to the two experimental contents mentioned later.
Development of Brainwave Measurement System
For brainwave measurement in the experiment, a developed system is being used. The aim is to overcome the challenges of non-invasive EEG as much as possible and to make the most of the electrical signals derived from brain activity.
There are mainly two challenges with non-invasive EEG:
Compared to invasive methods, the brainwaves are measured as very weak signals due to the interference of the skull and scalp.
Electrical signals originating from facial expressions and eye movements can be stronger than the brainwaves, potentially overshadowing them.
The current solution includes implementing a filter that removes noise signals from the brainwaves in real-time, ensuring the collection of cleaner brainwave data. Two types of electrodes are used in combination: the electrodes of the non-invasive ultra-high-density EEG, which are mounted on the scalp, and electrodes that measure electrical signals originating from facial muscles (see image for reference). Additionally, whispering is encouraged during speech to minimize the output of electromyography (muscle-generated electrical signals).
Experimental Example 1: ChatGPT empowered Brain Gmail Interface
The experiment involves a user with a non-invasive BMI operating Gmail through brainwaves and ChatGPT. Using a non-invasive ultra-high-density EEG, brainwave data was collected when the user spoke the names of specific colors. This data was then used to train an AI, enabling the identification of words from brainwaves. By integrating ChatGPT, it became possible to operate Gmail through brainwaves.
Experimental Example 2:Brain-Robot Interface with Imitation Learning
This experiment, conducted as the next step following the "Gmail Operation Experiment with Ultra-High-Density EEG," was carried out in collaboration with the Reinforcement Learning team. It involved a user in Akasaka wearing a non-invasive BMI and remotely operating a robotic arm located in Akihabara using brainwaves. The remote operation of the robotic arm was achieved through the combination of word identification from brainwaves (AI-assisted BMI) and imitation learning.
X Communication Team is committed to developing BMIs that can accomplish a wider range of tasks, exploring a broad spectrum of possibilities. Their goal is to help diverse individuals overcome societal barriers and to contribute to widening the options for participation and communication within society. This effort is directed towards creating a more inclusive environment where technology empowers individuals, particularly those facing physical or communicative challenges, to engage more fully with the world around them.
- What is Neurotech (Braintech) (1): Basic technologies and issues supporting Neurotech (Braintech)
- Successfully conducted experiment on operating G-mail using ultra-high-density electroencephalograph
- ABEMA Hills [Weekday 12:00 noon - Live Broadcast] - Latest NEWS - Send mail with your mind! The Dream of EEG x ChatGPT (News)_JapaneseOnly
- ChatGPT is a dream that aims to send emails with your mind (News) (ABEMA TIMES) - Yahoo!_JapaneseOnly
- Successfully conducted experiment controlling a robot arm using ultra-high-density EEG
- Toward the Realization of Trusted Brain Tech / BMI (1) Introduction of "Freeing the body from its commitments by extending physical and perceptual capabilities" Ryota Kanai (Project Manager)