Monday, July 4, 2022
SCIENMAG: Latest Science and Health News
No Result
View All Result
  • Login
  • HOME PAGE
  • BIOLOGY
  • CHEMISTRY AND PHYSICS
  • MEDICINE
    • Cancer
    • Infectious Emerging Diseases
  • SPACE
  • TECHNOLOGY
  • CONTACT US
  • HOME PAGE
  • BIOLOGY
  • CHEMISTRY AND PHYSICS
  • MEDICINE
    • Cancer
    • Infectious Emerging Diseases
  • SPACE
  • TECHNOLOGY
  • CONTACT US
No Result
View All Result
Scienmag - Latest science news from science magazine
No Result
View All Result
Home SCIENCE NEWS Technology and Engineering

New system combines smartphone videos to create 4D visualizations

July 1, 2020
in Technology and Engineering
0
Share on FacebookShare on Twitter

Carnegie Mellon approach requires neither studio nor specialized cameras

IMAGE

Credit: Carnegie Mellon University

PITTSBURGH–Researchers at Carnegie Mellon University have demonstrated that they can combine iPhone videos shot “in the wild” by separate cameras to create 4D visualizations that allow viewers to watch action from various angles, or even erase people or objects that temporarily block sight lines.

Imagine a visualization of a wedding reception, where dancers can be seen from as many angles as there were cameras, and the tipsy guest who walked in front of the bridal party is nowhere to be seen.
The videos can be shot independently from variety of vantage points, as might occur at a wedding or birthday celebration, said Aayush Bansal, a Ph.D. student in CMU’s Robotics Institute. It also is possible to record actors in one setting and then insert them into another, he added.
“We are only limited by the number of cameras,” Bansal said, with no upper limit on how many video feeds can be used.
Bansal and his colleagues presented their 4D visualization method at the Computer Vision and Pattern Recognition virtual conference last month.
“Virtualized reality” is nothing new, but in the past it has been restricted to studio setups, such as CMU’s Panoptic Studio, which boasts more than 500 video cameras embedded in its geodesic walls. Fusing visual information of real-world scenes shot from multiple, independent, handheld cameras into a single comprehensive model that can reconstruct a dynamic 3D scene simply hasn’t been possible.
Bansal and his colleagues worked around that limitation by using convolutional neural nets (CNNs), a type of deep learning program that has proven adept at analyzing visual data. They found that scene-specific CNNs could be used to compose different parts of the scene.
The CMU researchers demonstrated their method using up to 15 iPhones to capture a variety of scenes — dances, martial arts demonstrations and even flamingos at the National Aviary in Pittsburgh.
“The point of using iPhones was to show that anyone can use this system,” Bansal said. “The world is our studio.”
The method also unlocks a host of potential applications in the movie industry and consumer devices, particularly as the popularity of virtual reality headsets continues to grow.
Though the method doesn’t necessarily capture scenes in full 3D detail, the system can limit playback angles so incompletely reconstructed areas are not visible and the illusion of 3D imagery is not shattered.

###

In addition to Bansal, the research team included Robotics Institute faculty members Yaser Sheikh, Deva Ramanan and Srinivasa Narasimhan. The team also included Minh Vo, a former Ph.D. student who now works at Facebook Reality Lab. The National Science Foundation, Office of Naval Research and Qualcomm supported this research.
Video: https://www.youtube.com/watch?v=quovnDPwL1k&feature=youtu.be

Media Contact
Byron Spice
[email protected]

Tags: Computer ScienceTechnology/Engineering/Computer Science
Share26Tweet16Share4ShareSendShare
  • Free-roaming dog

    Male dogs four times more likely to develop contagious cancer on nose or mouth than females

    65 shares
    Share 26 Tweet 16
  • COVID-19 fattens up our body’s cells to fuel its viral takeover

    93 shares
    Share 37 Tweet 23
  • Atezolizumab translates into survival benefit for bladder cancer patients with ctDNA positivity

    64 shares
    Share 26 Tweet 16
  • Fast and facile synthesis of antibacterial amino acid Schiff base copper complexes

    64 shares
    Share 26 Tweet 16
  • Only through international cooperation can AI improve patient lives

    65 shares
    Share 26 Tweet 16
  • Machine learning goes with the flow

    65 shares
    Share 26 Tweet 16
ADVERTISEMENT

About us

We bring you the latest science news from best research centers and universities around the world. Check our website.

Latest NEWS

COVID-19 fattens up our body’s cells to fuel its viral takeover

nTIDE May 2022 COVID Update: Uncertainty about inflation tempers good news for people with disabilities

Famous Sterkfontein Caves deposit 1 million years older than previously thought

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 190 other subscribers

© 2022 Scienmag- Science Magazine: Latest Science News.

No Result
View All Result
  • HOME PAGE
  • BIOLOGY
  • CHEMISTRY AND PHYSICS
  • MEDICINE
    • Cancer
    • Infectious Emerging Diseases
  • SPACE
  • TECHNOLOGY
  • CONTACT US

© 2022 Scienmag- Science Magazine: Latest Science News.

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In
Posting....