Exposing Fake Videos

A team of researchers at UAlbany used eye blinking detection on an original video (top) and a DeepFake generated fake video (bottom) to determine if a video was counterfeit. In the top, blinking is detected within six seconds. (Graphic by Yuezun Li, Ming-Ching Chang and Siwei Lyu).

ALBANY, N.Y. (June 20, 2018) – It turns out seeing isn’t always believing. As technology continues its inevitable march at startling speed, the realism with which fake videos are now being produced is both “impressive and deeply unsettling,” according to Gizmodo.

But researchers at the University at Albany are providing the tools to fight back. Associate Professor of Computer Science Siwei Lyu, Assistant Professor of Electrical and Computer Engineering Ming-Ching Chang and Doctoral Student Yuezun Li have developed a method to expose fake videos by examining the blink rate of the subjects.

In their recently published paper “In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking,” the team was able to combine two neural networks to track differences in the eye movement of fake videos compared with legitimate sources.

“AI generated faces lack eye blinking function, as most training datasets do not contain faces with eyes closed,” said Lyu. “The lack of eye blinking is thus a telltale sign of a video coming from a different source than a video recorder.”

Their work has already caught the attention of some major media outlets. In addition to the article in Gizmodo, the research has been featured in The Register (U.K.), iNews, New Scientist and Sapo Tek in Portugal.

While the approach shows significant promise, there are several directions that remain unexplored.

“We need to explore other deep neural network architectures for more effective methods to detect closed eyes,” said Lyu. “Our current method only uses the lack of blinking as a cue for detection. However, the dynamic pattern of blinking should also be considered – too fast or frequent blinking that is deemed physiologically unlikely could also be a sign of tampering.”

Finally, eye blinking is a relatively easy cue in detecting fake face videos, and sophisticated forgers can still create realistic blinking effects with post-processing and more advanced models and more training data. So in the long run, the researchers are interested in exploring other types of physiological signals to detect fake videos.

The research was sponsored by by DARPA (Defense Advance Research Project Agency)’s Media Forensics (MediFor) program. Lyu serves as principal investigator of UAlbany’s MediFor team, and has been a leading scientist in the area of digital forensics for more than a decade.

Lyu has been the recipient of two NSF grants focused on digital media forensics, including a prestigious NSF CAREER Award in 2010, followed by a three-year $400,000 grant in 2013. In 2011, he earned the coveted Best Paper Award from IEEE Signal Processing Best Paper Award for his work in the field of digital forgery.

For more news, subscribe to UAlbany's RSS headline feeds