Oh wow, look at this video! It's Obama… well, I think it's Obama 🤔 I mean it looks and sounds like him but I'm not sure… Wait, why is Obama saying those things...
With election season approaching, the increase of misinformation has become a tool and weapon used to discredit public figures and polarize voters. DeepFakes are the new kids on the block with the potential to create misinformation. DeepFakes are nearly impossible to detect, which makes them an even more attractive weapon political candidates can employ.
DeepFakes are a class of counterfeit videos that superimpose face images and even vocals, allowing an attacker to realistically impersonate anyone they choose. DeepFake technology has progressed and it has become increasingly harder for a human to recognize these forgeries, much less a computer.
My hackathon team set out to combat this problem. For our first approach, we attempted to use regular face detection software; however, we were not seeing any results. After talking with the AWS rep at the hackathon we realized the shortcoming of face identification software: it looks for similarities in faces and ignores the minute differences. With DeepFake videos it is the minute differences that are going to identify an imposter from the actual subject.
Our hypothesis was that through the environment someone is brought up in and lives in, they develop their own speaking habits, something that is expressed through their lips and eyes. DeepFakes require the attacker's face to be slightly superimposed onto the victim's to control features like the lips or eyes, and doing so alters the speaking habits of the victim. By building a model of Obama, for example, we can train a computer to identify Obama's (or anyone's) natural speaking habits, and the program will be able to identify a DeepFake when this pattern is inconsistent.
Surprisingly, our model worked!!
To make the model accessible to users we attached it to a Chrome extension and a web application where users can enter a video URL and test the video's legitimacy. The Chrome extension, when enabled, automatically scans the web page for videos and runs them through the model. The second a video is flagged as a DeepFake it alerts the user in the browser. This feature is important to stopping the spread of misinformation because it informs social media users of counterfeit videos before they are able to share, like or comment.