Industry News

Canon & IBM Announce Business Partnership Utilizing Volumetric Video Technology

This site contains affiliate links to products and services. We may receive a commission for purchases made through these links.

MELVILLE, NY, July 8, 2021, – Canon Inc. (Headquarters: Ota-ku, Tokyo; CEO: Fujio Mitarai) and IBM Japan Ltd. (Headquarters: Chuo-ku, Tokyo; CEO: Akio Yamaguchi) announced today that the two companies have formed a business partnership to use volumetric video technology to promote the visual and performing arts.

Volumetric video technology involves the use of digitalized 3D spatial data, which is rendered by computers to create a virtual space in which video can be generated from any angle. This technology has attracted significant interest in the sporting world and is becoming increasingly widespread in the entertainment industry. In addition, Canon implemented this technology during an international rugby tournament in 2019 as well as other sporting events. In July 2020, the company established its Volumetric Video Studio – Kawasaki, to expand its range of activities in the entertainment industry.

Check out the videos of this technology

In this business partnership, both companies have leveraged their combined technologies, products, and networks to promote a volumetric video solution for the visual and performing arts community, with the aim of exploring new business opportunities and creating and growing new value for audiences. In particular, Canon is contributing its volumetric video technology, which makes possible nearly simultaneous video capture and generation of high-resolution video. Through this technology, the company provides a new method for video content production and audiovisual experiences, in addition to helping to pass on valuable cultural traditions.

Leveraging its know-how and experience in the field of art, IBM Japan is providing consulting services that include the development of business strategies and project execution support for companies involved in the arts in order to help the visual and performing arts achieve a digital transformation.

In addition, using its IBM Power System AC922 for accelerated computing and IBM Elastic Storage System broadband storage technology, IBM Japan will provide technical support through high-quality, high-speed, and secure visual data processing and streaming technology, thus increasing the value of services using volumetric data.

The first project created through this corporate partnership is the release today of a performance of the Noh theatre work “Lady Aoi,” produced by Hoshokai, utilizing volumetric video technology. Directed by Kazufusa Hosho, Head of the Hosho Noh Style, the performance gracefully flows between scenes of the Noh stage and the spiritual world. For the creation of the video, Canon obtained 3DCG data of a real-life Noh stage, generated dynamic backgrounds and CG animations, and captured video of the Noh actors’ performances at the Volumetric Video Studio – Kawasaki. These resources were then smoothly integrated into the completed product, right there at the studio. IBM Japan provided high-speed infrastructure and technological support through its IBM Power Systems AC922 advanced dedicated parallel processing server and IBM Elastic Storage System broadband storage technology, enabling the use of essential responsiveness required for real-time video rendering.

Through this partnership, both Canon and IBM Japan will aim to create new value within the visual and performing arts world while expanding the potential use cases of volumetric video technology.

Check out the videos of this technology

Quote from Kazufusa Hosho, Head of the Hosho Noh Style:
“The emotional state of Lady Rokujo, the main character in Lady Aoi, is beautifully expressed through the integration of volumetric visuals and 3D graphics that depict traversing back and forth from the Noh stage and the spiritual world. During the production process, I placed strong importance on how to convey the powerful impact of the story when it is finally ready for audiences to enjoy. Thanks to this volumetric video technology, I believe we have created a way of viewing performances unlike anything on the traditional stage.”

About Volumetric Studio Kawasaki:
Canon’s Volumetric Studio Kawasaki is a visual recording studio that can capture performers’ movements from every possible angle. More than 100 4K cameras are utilized to capture video that is subsequently rendered by computers into 3D data inside a virtual space. Inside this virtual space, content can be viewed from any angle, making possible new kinds of visual expression. Recording and content generation can be performed separately, helping contribute to more efficient workflows. The studio is located within Canon’s Kawasaki office (70-1 Yanagicho, Saiwai-ku, Kawasaki, Kanagawa 212-8602, Japan) and began operations in July 2020. To date, the studio has completed such projects as music-related content, variety TV programs, documentaries, TV commercials, promotional videos, and collaborations with the traditional Japanese performing art of Kabuki and Noh theatre. In addition, Canon uses its capabilities to generate volumetric video in real-time to create new forms of experiential visual entertainment including such events as live-streamed performances.

Hector1970

EOS R
CR Pro
Mar 22, 2012
1,354
609
It must be techology in its infancy. The usage during the international rugby tournament was a bit cartoonish. The demo video is also a little artificial looking. I'm sure it will get better and better.
 

entoman

wildlife photography
May 8, 2015
440
476
UK
The Star Trek holodeck is just around the corner. Amazing technology, but not a direction I like to see this world go in.
 

EOS 4 Life

EOS R
Sep 20, 2020
1,097
861
It must be techology in its infancy. The usage during the international rugby tournament was a bit cartoonish. The demo video is also a little artificial looking. I'm sure it will get better and better.
I thought it was meant for VR.
I must be missing something because what was in the demo used a green screen.
They must be doing something else for sports.
 

entoman

wildlife photography
May 8, 2015
440
476
UK
I thought it was meant for VR.
I must be missing something because what was in the demo used a green screen.
They must be doing something else for sports.
Only guessing, but let's imagine there are 20 cameras spaced regularly around the circumference of a sports stadium. Normally with such a situation the viewer would only have the choice of 20 different fixed angles of view. I think what is being hinted at here is that AI technology could enable all the in-between angles to be calculated and generated at a virtual image. And presumably could also automatically follow a player around and even provide the "player's view" of events. Taking it one step further, the viewer could "play the part" of the sportsman, as a game or activity, hence my reference above to the Star Trek holodeck.
 
  • Like
Reactions: pape2

Mt Spokane Photography

I post too Much on Here!!
CR Pro
Mar 25, 2011
16,714
1,686
After reading thru the Wikipedia article that describes the current state, it is a technology that is starting to be possible but its true potential is currently out of reach. The amount of data, computing power, and bandwidth required to view a high end volumetric video at home does not exist but some fairly basic uses are currently in use such as 360 cameras for sports games.

The future potential is only limited by the imagination, it will truly change the world, It will be used for things we don't even expect. Once a background is captured, it can be modified and reused over and over in ways that viewers may not recognize. I'm getting up there in age so I may not be around when its common, super high bandwidth is not coming to me soon. I'm lucky to get 30 Mbps and hoping that Starlink brings faster speeds to my rural location.

MPEG, the Motion Picture Experts Group is working toward a standard but if one does come out, it will be pretty basic. Gaming is one of the technologies that needs a standard now.
 

Antono Refa

EOS R
Mar 26, 2014
1,335
442
Only guessing, but let's imagine there are 20 cameras spaced regularly around the circumference of a sports stadium. Normally with such a situation the viewer would only have the choice of 20 different fixed angles of view. I think what is being hinted at here is that AI technology could enable all the in-between angles to be calculated and generated at a virtual image. And presumably could also automatically follow a player around and even provide the "player's view" of events.
AFAIK, that was already done, without AI, by an Israeli company called Replay Technologies, which was bought by Intel.
 

Antono Refa

EOS R
Mar 26, 2014
1,335
442
After reading thru the Wikipedia article that describes the current state, it is a technology that is starting to be possible but its true potential is currently out of reach. The amount of data, computing power, and bandwidth required to view a high end volumetric video at home does not exist but some fairly basic uses are currently in use such as 360 cameras for sports games.

The future potential is only limited by the imagination, it will truly change the world, It will be used for things we don't even expect. Once a background is captured, it can be modified and reused over and over in ways that viewers may not recognize.
I was shown a demo in which the ads displayed on the low wall around a soccer field were replaced in re-run. The system identified the wall, as well as the occlusions, and replaced the ads. I wouldn't be surprised if, eventually, it could be done in real time, and the broadcasting rights would include the right to replace them with localized ads.