Various robots can be found inside the Future of Life signature pavilion, including Petra, Pangie, Punica, and aiai walkie. The engineering team from the Interaction Technology Bank, Deep Interaction Laboratory Group, Advanced Telecommunications Research Institute International (ATR) was in charge of system development for these robots as well as guidance and tours by these robots within the pavilion. This time, we spoke with NISHIMURA Shogo and Arne HITZMANN, full-time researchers and members of this team. They talked about the project’s system development and the current operation of robots within the pavilion.

The Expo as part of a real-world social field experiment for the Moonshot Research and Development Program

At ATR, Nishimura and Hitzmann are participating in the research and development project for “The Realization of an Avatar-Symbiotic Society where Everyone Can Perform Active Roles without Constraint” (Avatar-Symbiotic Society Project), where ISHIGURO Hiroshi serves as project manager. This is a project under Goal 1 “Realization of a society in which human beings can be free from limitations of body, brain, space, and time by 2050” of the Moonshot Research and Development Program (Moonshot), a national initiative being undertaken by Japan Science and Technology Agency (JST). The project is developing cybernetic avatars (CAs) to realize an avatar-symbiotic society. CAs are robot or computer graphics (CG) avatars that people can remotely control as their own avatars. The two researchers are carrying out research and development of mobile CAs capable of social activities such as security, tour guiding, and working as store clerks. Nishimura said that their involvement in the Future of Life signature pavilion was an extension of that work.

“As part of the real-world social field experiment of the Avatar-Symbiotic Society Project, CAs will provide information to and guide people who visit the Future of Life signature pavilion throughout the duration of the Expo. Our key mission this time is to demonstrate the technology through long-term operation of the CAs and the infrastructure system for tele-operating them, and to investigate the social acceptability of CAs among visitors who will have a simulated experience of a society that utilizes CAs.”

Many institutions are involved in Moonshot, including The University of Osaka, RIKEN, Keio University, Nagasaki University, The University of Electro-Communications, and Nagoya University. ATR is in charge of aspects that include robots, control systems, communication systems with the robots, and experimental operation. ATR has already developed the mobile CA called Teleco, upon which the robots operating within the Future of Life signature pavilion were developed based on.

Facing all kinds of issues with the robots’ movements, which only became known when working at the pavilion

In this project, Hitzmann was responsible for developing a navigation system tailored to the exhibits in the pavilion and creating the software to centrally manage all the robots in the pavilion. He revealed that he experienced many unprecedented difficulties even though development was based on the existing Teleco.

“Each robot is equipped with a large battery that allows it to operate all day on a single charge, making it heavier than a regular Teleco. The pavilion’s flooring material is also special, so it was extremely difficult to make the robots move smoothly. Professor Ishiguro was also very particular about creating robots that move naturally with minimal vibration. As such, we made various adjustments.”

Nishimura liaised with the relevant parties—both within and outside the team—about robot operations. He was also the engineer who played a central role in implementing the functions necessary for presenting the robots and making them operate more stably. He added that there were many unique issues that came with operating the robots inside the pavilion.

“It was a challenge to newly implement a function that allows all nine robots to be operated together on a schedule using signals from the pavilion’s system. As it was often difficult to get a proper feel of the situation through online meetings, the responsible members gathered on-site to deepen our understanding and advance the implementation of this function. We selected the best option at each point in time by sharing information, such as the latest changes and corrections as well as methods to deal with problems when they occurred.”

Hitzmann said that there were many things he only realized after actually working on-site at the pavilion.

“Often, the robots did not move as intended due to factors such as the swaying of curtains installed in the pavilion and the activation of sensors triggered by visitors’ movements. This was something that could not have been anticipated in the laboratory. Therefore, we went to the pavilion and kept making fine adjustments.”

Quickly solving robot malfunctions and other problems through the power of teamwork

Looking back, Nishimura said that the robots also malfunctioned many times.

“Even when a problem occurred in a robot’s operation, it was extremely difficult to identify the cause due to the unique environment and the custom design for the Expo. To solve a problem that had manifested in a single robot, we had to seek the opinions and support of team members who are knowledgeable in each field to trace the problem back to its root cause—such as the software inside the robot, the robot’s hardware, the wireless network to which each robot was connected, or the design of the external server accessed by the robot—and solve each problem one by one. Thanks to all these people, the robots are now operating stably with problems kept to a minimum.”

After the opening of the Expo, Hitzmann took the lead in dealing with on-site problems and managing the operation of the robots.

“We brought approximately 15 robots to the pavilion, including spare machines. We had to establish a system that can manage these many robots while maintaining both the software and hardware, and immediately replace a robot when a malfunction occurs. We also needed to adjust the robots’ movement remotely. Both were major issues. About 15 members exchanged opinions and worked to put in place a system for achieving these goals.”

They made all possible preparations, such as introducing additional software that allows tasks to be automated as much as possible to prevent human error, and developing training for the operational staff within the pavilion.

Developing a training program for operation staff and making daily adjustments even after the Expo’s opening

“Regarding training, some of the robot operators in the pavilion were handling robots for the first time. Therefore, we developed a two-week program so that everyone could carry out their work with confidence. Specifically, it was a hands-on process where, together with experienced people, the robot operators learned things such as the powering up and down of a robot when it needs to be replaced, and the tasks that need to be carried out after replacement. Even though we took these measures, unexpected problems still occur. At such times, the on-site engineers will address the issue and share the solutions they come up with in real time through a communication network between the people in charge.”

Adjustments continued to be made on a daily basis after the Expo opened and the public began visiting the pavilion. Hitzmann reflects on the first day that the public visited the pavilion.

“We were surprised that the behavior of the public was often different from what we had expected. For example, we thought that when a robot started speaking, people would listen until the end, but some people started moving without listening; or that people would move to the left when a robot moved to the right, but this was sometimes not the case, and some people even came into contact with the robot. We share these issues with the pavilion’s operation team, and we overcome them by having on-site staff guide visitors and call out to people early before any problems occur. I am grateful to the on-site staff for keeping things running smoothly.”

Diverse robot operations achieved through teamwork

Even now, various fine adjustments continue to be made at the pavilion every day, and there is on-going collaboration with the on-site staff so that the robots can welcome visitors without issues.

“Right now, it is hot at the venue of the Expo, but I remember the days in the middle of winter when the pavilion was still unfinished. We were wearing down jackets and making final adjustments while the north wind was blowing. At the press conference for the media, we operated and controlled the robots in the midst of a snowstorm. It was a truly harsh environment, but we were able to complete the project working united as one team, including the members of Vstone who worked with us on robot development. Now, it is a memory that remains with me,” said Nishimura.

Hitzmann added, “The long days of preparation before the Expo’s opening are undoubtedly unforgettable memories. I am filled with gratitude for the support of Vstone, the Expo team, and my colleagues, who always found solutions and tackled issues head on to achieve the goals.”

Look closely and find your favorite robot

Lastly, the two of them had these messages for future visitors to the Expo.

Nishimura said, “Some of the robots are so close you can almost touch them, and I hope visitors will try to observe them up close. It will also make me happy if you pay attention to the robots’ behavior and gestures as they guide you around and find your favorite robot. By the way, I am a fan of all the robots, but I have a special attachment to aiai walkie, as it was the one that required the most work.”

From Hitzmann, “A lot of effort has gone into this pavilion, and it is packed with all kinds of ingenious ideas. It would be a shame to rush through it. Please take your time to look around and experience something for yourself from the pavilion’s exhibits. I hope that visiting the pavilion will help you think more positively about the future and give you new perspectives about a society where we coexist with robots (i.e., an avatar-symbiotic society). I also hope that visitors will enjoy coming across various cultures and countries as they go around the venue of the Expo.”

(Interviewed: July 2025)

[Profile]

Interaction Technology Bank, Deep Interaction Laboratory Group, Advanced Telecommunications Research Institute International
https://dil.atr.jp/ITB/en/
The Interaction Technology Bank was established with the aim of transforming cutting-edge research outcomes on robots and avatars into practical technologies in collaboration with industry partners and research institutions. Just as a financial bank nurtures capital, the Technology Bank cultivates technological assets, advancing research and development that delivers value back to society.

Global Open Lab Strategy Office, Advanced Telecommunications Research Institute International
https://dil.atr.jp/GOL/en/
The Global Open Lab Strategy Office was established as a dedicated team to strategically promote international research activities aimed at realizing a future society where robots and avatars coexist with humans. Through international collaborative research, industry partnerships, and social field experiments, we work together to create practical models of future societies that address social challenges, with the ultimate goal of realizing an avatar symbiotic society.