Left Menu
Development News Edition

New tech makes Harry Potter-like moving photos a reality

PTI | Washington DC | Updated: 13-06-2019 14:20 IST | Created: 13-06-2019 14:11 IST
New tech makes Harry Potter-like moving photos a reality
The researchers envision this could lead to a new way for gamers to create avatars that actually look like them, a method for visitors to interact with paintings in an art museum. Image Credit: Pixabay

Our photographs may soon resemble those from the world of Harry Potter, as scientists have developed a tool that can make a person from a 2D image run, walk or jump out of the frame. Their algorithm, called Photo Wake-Up, also allows users to view the animation in three dimensions using augmented reality tools.

"This is a very hard fundamental problem in computer vision," said Ira Kemelmacher-Shlizerman, an associate professor at the University of Washington in the US. "The big challenge here is that the input is only from a single camera position, so part of the person is invisible. Our work combines technical advancement on an open problem in the field with artistic creative visualisation," Kemelmacher-Shlizerman said.

Previously, researchers thought it would be impossible to animate a person running out of a single photo. "There is some previous work that tries to create a 3D character using multiple viewpoints," said Brian Curless, a professor at the University of Washington.

"But you still couldn't bring someone to life and have them run out of a scene, and you couldn't bring AR into it. It was really surprising that we could get some compelling results with using just one photo," said Curless. The applications of Photo Wake-Up are numerous, the team said.

The researchers envision this could lead to a new way for gamers to create avatars that actually look like them, a method for visitors to interact with paintings in an art museum -- say sitting down to have tea with Mona Lisa -- or something that lets children bring their drawings to life. Examples in the research paper include animating the Golden State Warriors' Stephen Curry to run off the court, Paul McCartney to leap off the cover of the "Help!" album and Matisse's "Icarus" to leave his frame.

To make the magic a reality, Photo Wake-Up starts by identifying a person in an image and making a mask of the body's outline. From there, it matches a 3D template to the subject's body position. Then the algorithm does something surprising: In order to warp the template so that it actually looks like the person in the photo, it projects the 3D person back into 2D.

Photo Wake-Up stores 3D information for each pixel: its distance from the camera or artist and how a person's joints are connected together. Once the template has been warped to match the person's shape, the algorithm pastes on the texture -- the colours from the image.

It also generates the back of the person by using information from the image and the 3D template. Then the tool stitches the two sides together to make a 3D person who will be able to turn around. Once the 3D character is ready to run, the algorithm needs to set up the background so that the character does not leave a blank space behind.

Photo Wake-Up fills in the hole behind the person by borrowing information from other parts of the image. Right now Photo Wake-Up works best with images of people facing forward and can animate both artistic creations and photographs of real people.

The algorithm can also handle some photos where people's arms are blocking part of their bodies, but it is not yet capable of animating people who have their legs crossed or who are blocking large parts of themselves.



Why COVID-19 is unstoppable in USA despite it being ranked at the top of GHS Index?

Several worst-hit countries such as Italy, France, Spain, the UK, Canada, and Russia have peaked COVID-19 cases in April. Almost all of them have gradually flattened the curve. However, the USA is setting new daily records of infections tha...

COVID-19 seems cooking biggest ever global scam

The increasing number of corruption cases on COVID-19 funds from throughout the world and involvement of high profile persons indicate that the countries cant ignore corruption in their pandemic response programs. This has generated the nee...

Health Management Information Systems lack holistic, integrated, and pandemic resilient character

Being a part of the United Nations system, the World Health Organization WHO deserves its share of rebuke for its alleged failure issue COVID-19 health emergency alerts on appropriate time. However, the pandemic has also exposed loopholes i...

Pride in the time of coronavirus: a welcome move online?

This year is different in many ways not least as celebrations are also taking place against the dramatic backdrop of a global health crisis and a resurgence in grassroots activism following the death of George Floyd in Minneapolis. ...


Latest News

It's a perilous time for multilateralism: Shashi Tharoor

By Sahil Pandey And Joymala Bagchi Parliamentarian Shashi Tharoor discussed the relevance of multilateral institutions in modern times and the handling of the global pandemic at the global virtual platform, India Global Week, on Friday.Thar...

Soccer-Bulgaria could delay next season's start after coronavirus surge

The Bulgarian Football Union BFU may delay the start of the next domestic season, scheduled for July 24, after coronavirus infections at several clubs surged, its medical commission said on Friday.More than half of top-flight clubs as well ...

LeT militant arrested in J-K's Bandipora

Security forces arrested a Lashkar-e-Toiba LeT militant from Bandipora district of Jammu and Kashmir and recovered ammunition from his possession, police said on Friday. Based on a specific input regarding movement of militants towards Haji...

India most attractive global market for clean energy: Modi

India has emerged as the most attractive global market for clean energy, Prime Minister Narendra Modi said on Friday. Launching a 750 MW solar project in Rewa in Madhya Pradesh through video-conferencing, Modi also said the state will emerg...

Give Feedback