Be it Iron Man in Marvel movies or Ana de Armas’s character in Blade Runner 2049, holograms in sci-fi movies are a treat to the viewer’s eyes. Though these human-like projections seem immensely futuristic, modern-day technology such as artificial intelligence has made hologram creation more accessible than ever.
Industry-level hologram creation has been in use for the last two decades, being utilized for creative and commercial purposes. Though it started with Microsoft’s Mixed Reality Studio, in recent years, many mid-sized companies, namely, 8i, Soar, and 4DR Studios, have offered cheaper and compact alternatives for hologram capture. By applying volumetric capture, a 3D scanning method to record human movement, these studios have successfully produced holograms for music videos, tech conferences, art installations, and VR movies.
Last year, Ukrainian President Volodymyr Zelensky’s hologram made an astounding appearance at the Viva Tech conference in Paris, thanks to Evercoast’s portable volumetric studio. 4DR Studios, a Dutch-based volumetric studio offered football fans the opportunity to interact with holograms of PSV players during the stadium tour in Eindhoven. Such instances of hologram creation are mostly limited to the industry level where money and production time play a significant factor.
Given the masses were deprived of this complex technology, Vologram, an Irish-based company has made significant progress in this field. By relying on artificial intelligence, the Trinity College spin-off created an app Volu, which let users create holograms just with their smartphone cameras.
3D model reconstruction using AI
Standard hologram creation usually requires the alignment of RGB depth cameras around a person within a green-screen studio, followed by heavy computational post-production. But Volu does the hologram construction in a quicker and faster way. The app allows users to record a video of a person and then upload it to their cloud server, where the 3D hologram production takes place.
Initially, Rafael Pagés and Jan Ondřej, founders of Vologram, started this project with the aim to reduce the number of cameras required to capture a human in volumetric video and study the progress for research purposes. But with rapid progress in the field of artificial intelligence, the duo was not only able to capture human movement in 3D with a help of a single camera but also produce the result within a short time.
“We trained our artificial neural network with hundreds of studio-captured 3D human models”, explained Pagés. “By studying these models we have successfully created a single algorithm that helps in the construction of the 3D hologram from the captured video.”
What seems like an easy process of turning a 2D video into a 3D volumetric model, in reality, requires a series of complex computational workflows. After receiving the metadata of the recorded video, the neural network understands the scene, the clothing, the background, the skeletal movement of the person, and the depth system. Based on the neutral networks’ memory, a reconstruction of the entire 3D model is done which then gets encoded and compressed into a compact format.
However, the quality of holograms created by AI assistance is not at par with the holograms produced by volumetric studios. Pagés said, “The AI needs more training to produce a sharper quality of reconstruction.” Unlike high-end volumetric studios, the Volu app is targeted at a consumer market and thus based on a different technology.
Natasja Paulssen, founder of 4DR Studios, agreed that the technology behind AI-assisted hologram creation is quite different when compared to traditional volumetric studio capture. While a solution like Volu allows users to capture just the front side of a person and recreates the remaining aspects of the human as well as the environment using AI, volumetric studio refrains from relying on AI. “We create high-quality 3D models using complex camera systems and time-consuming post-production,” she explained. “Instead of relying on AI to fill in gaps created by a single-point camera, our system is designed to create crystal clear holograms for museums, education, and commercial purpose.”
Endless possibilities
The option to create a 3D representation of a person opens up a lot of possibilities not only in the creative industry but also in the communication field. Initially, the app has been adapted mostly by tech-savvy users who are interested in XR development. But Rafael believes the technology is not just limited to the developer community, and can be used for a variety of purposes including campaigns, marketing, and education.
FarmVR, an agricultural company collaborated with entomologist, Molly Keck, to teach their users about beekeeping by using her hologram in virtual reality space.
“I am planning on using the Volu app for a course next semester on motion capture and AI” said Dr. Andrew Hogue, associate professor, Ontario Tech University. Being a program director of game development and media interaction, he shared his experience of testing volumetric capture in game creation. “We were attempting to build a game around volumetric video which was a challenge previously, but probably easier with Vologram given the tech that we have.”
Be it founders showcasing their businesses to potential customers, or a person sending holographic birthday wishes to loved loves, the possibilities are endless. “It can be regarded as a revolution,” exclaimed Hogue. “To be able to capture a person at the click of a button and view the 3D model in your surrounding is exciting from a consumers’ perspective.”
While hologram has mostly been limited to the imagination of film audience, technology has been successful in making human virtual models a reality. “We want to turn virtual and visit Mom and Dad on the other side of the world,” Paulssen exclaimed.
Users’ who wish to use holograms for personal communication on a regular basis will still have to wait. Though the idea of seeing your family, friends, and loved ones appearing as a 3D hologram in front of you seems fascinating, the technology needs to adapt itself for such use. But with rapid advancement in AI reconstruction and the increasing production of AR glasses, holographic communication for interpersonal conversation in the recent future seems possible.
Related Posts
