A Digital Twin of Kabuki Actor Shido Nakamura Created with NTT’s Another Me Technology Debuts at “Cho Kabuki 2022 Powered by NTT”

TOKYO–(BUSINESS WIRE)–NTT Corporation (President: Akira Shimada, “NTT”) and Shochiku Co., Ltd. (President & CEO: Junichi Sakamoto) have announced “Cho Kabuki 2022 Powered by NTT” sponsored by Shochiku, with the first act starring Shido Nakamura’s digital twin, “Shido Twin.” The debut of Shido Twin showcases NTT’s research and development of a computer that reproduces people as digital twins and is the first step in the social implementation of this new technology. We hope you will come and witness the birth of Shido Nakamura reproduced as a digital twin.

Created in collaboration with Shochiku Co., Ltd., the Shido Twin autonomously reproduces the gestures of Shido Nakamura, who plays the leading role in Cho-kabuki, using the body motion generation technology of Another Me.?

Another Me is a technology born out of NTT’s Digital Twin Computing¹ concept. With Another Me, we aim to allow individuals to create an “alternative self” in the digital space, dramatically increasing the opportunities to flourish and grow by transcending the constraints of the real world, to play an active role in society, and to share the results of their own experiences with others.

“Cho Kabuki” has been co-produced by Shochiku Co., Ltd. and Dwango Co., Ltd. since 2016 and held annually as part of the Niconico Chokaigi festival. Cho Kabuki is a brand-new kabuki experience that combines kabuki?a traditional Japanese theatre form with a 400-year history?and NTT’s cutting-edge ICT technology. This year’s performance incorporates the latest ICT technology, including the “Telephone Shop”² NTT’s “Kirari!”, and the experiment has gained acclaim for expanding the possibilities of theater.

“Super Kabuki 2022 Powered by NTT” Performance Schedule https://chokabuki.jp/2022theatre/

“NTT has developed ‘Shido Twin,’ which learns my speech and mannerisms to behave and gesture like me, and it will debut in one act of the Cho Kabuki performance starting tomorrow. I believe that this cutting-edge technology will further expand the possibilities of the Cho Kabuki production. We will do our best so that our customers can enjoy Cho Kabuki again.” ?Shido Nakamura

How Another Me Technology Works

Another Me is a body motion generation technology that learns someone’s personality from their audio and video data, automatically generating human movement in response to the person’s speech. To reproduce a wide range of movements that capture even the minute mannerisms of a person, our researchers at NTT have incorporated a deep learning technique called Generative Adversarial Networks (GAN), which build a model that generates actions from speech. We have achieved the world’s highest performance in capturing the personality and naturalness of a person.

NTT has also developed cutting-edge speech synthesis technology, enabling the learning of the expressive power unique to a voice, such as individual inflection and intonation, from a small amount of speech. We could reproduce Shido Nakamura’s characteristic intonation and emotive speech using only a short recording of his voice. We will also try to reproduce his dynamic movements and gestures.

We hope you will look forward to the future research and development of Another Me technology and the new Shido Twin. You can access the full report about Shido Twin here.

¹ “What is Digital Twin Computing?”(https://www.rd.ntt/iown/0003.html) (Japanese)

² Telephone Shop: NTT’s “stage name” created by the Cho Kabuki Audience

Contacts

Media contact:
Research Planning Department, Production Division

NTT

nttrd-pr@ml.ntt.com

Development Planning Department

Shochiku Co., Ltd.

sh_kaihatsukikaku@shochiku.co.jp