Development of Algorithm to detact deepfakes images and videos

Now a days, roughly 3.5 billion people are connected to internet. Millions of people are uploading gigabytes of date on social platform for every minute of day and thousands of them share this information without checking any authenticity of data roughly. It is very difficult for casual consumers of

Project Title

Development of Algorithm to detact deepfakes images and videos

Project Area of Specialization

Artificial Intelligence

Project Summary

Now a days, roughly 3.5 billion people are connected to internet. Millions of people are uploading gigabytes of date on social platform for every minute of day and thousands of them share this information without checking any authenticity of data roughly. It is very difficult for casual consumers of images to authenticate digitally altered images

As Photoshop did for photographic alteration, so to have advances in artificial intelligence and computer graphics made image and video alteration seem real to the untrained eye. The term use to describe these fake images and videos are called “DeepFakes”: This technique is also being used for purpose like revenge porn, hoaxes, fake news, financial fraud, evidence and misrepresenting politicians during election time. DeepFakes is an imminent worldwide threat for advanced countries as well as undeveloped nations like Pakistan.

The project is entitled as “Development of Algorithm to Detect Deepfakes images and videos”. with sole purpose to develop artificial intelligence algorithm that can detect these DeepFake images and videos and check the authenticity of fake data. In December 2019, the first Deepfake Detection Challenge kicked off, backed by Microsoft, Facebook and Amazon. Main purpose of this challenge is to realize whole world about impact and threat of DeepFake worldwide and to invite researchers all around the world to participate and develop algorithms for detecting these DeepFake videos and images.

Concerning points for Deepfakes is that it is used to misrepresent well-known politicians in videos, Barak Obama deep fake with Jordan Peele’s voice, Argentine President Mauricio Macri has been replaced by the face of Adolf Hitler. Deepfakes is already at the point where you can't tell the difference between Deepfakes and the real thing and that’s why there is a need to develop a method in which we can differentiate Deepfakes and a real thing otherwise Deepfakes could be dangerous to many people lives.

Deepfakes are built by using Machine learning and Artificial Intelligence and our project is to build a method using similar techniques to detect real videos and images keeping different points in minds such as In 2018, US researchers discovered that Deepfake faces don’t blink normally. Similarly poor-quality Deepfakes are easier to spot. The lip-synching might be bad, or the skin tone patchy. There can be flickering around the edges of transposed faces and fine details, such as hair, are particularly hard for Deepfakes to render well, especially where strands are visible on the fringe.

It is very decisive moment for Pakistan as well to take this Deepfake threat seriously and contribute to develop detection algorithms for DeepFake images and videos and his project is committed to contribute in this research.

Project Objectives

Novel digital technologies make it increasingly difficult to distinguish between real and fake media. One of the main purposes of the project is to find an optimize way to differentiate what is right and wrong, what is real and fake.

The project is also based on techniques and architecture of deep learning so DeepFakes technique will be deeply explores with respect to their mathematical architecture and output results. Machine and deep learning models will be used to detect different facial features and then train those extracted features so using large data set. So basically, whole model will be designed on large data set using deep learning Algorithms.

On hardware level the software model will be tested using different equipment’s, analyzing different aspects, flaws, solution and implementation. Therefore, the purpose of this research is to analyze the current and future capabilities of DeepFake technologies to determine the threat it poses to national and personal security. No optimized algorithm is proposed for the detection of DeepFake till now. The already proposed algorithm also fails for high quality DeepFake

 videos and images. Therefore, the focus is to propose some sort of optimized and advanced solution that could classify the real manipulated videos and images. The designed algorithm will be tested using real and fake images and video to check its efficiency and accuracy.

Project Implementation Method

Literature proposes different implementations that can detect the DeepFake videos and images. Using convolution neural network features are extracted and then a recurrent neural network is trained using these features and then this recurrent neural network will classify if a video has been subject to some manipulation or not. This method is evaluated against a large data set.

It is well known that different deep learning techniques have been used to enhance the image compression especially autoencoders have been applied for dimensionality reduction, compact representations of images and generative models learning. So, autoencoders can extract more compressed representation of images and these compressed representations are used by recent convolution neural networks to create DeepFake by swapping the faces and features.

The second thing is the use of two sets of encoders-decoders with shared weights for the encoder networks. For this purpose, two sets of images and videos are required. The first set should have samples of original videos and images that should be replaced, and second set should contain the images that should be swapped in the targeted image or video. To ease the process of training of the autoencoders, the easiest face swap would have both the original and target face under similar viewing and illumination conditions. But this is not true for all cases. 

Due to difference in lightning conditions it is difficult for autoencoders to produce a realistic view in all conditions and this condition leads to swapped faces that are visually inconsistent with the rest of the frame. So, this can be our first feature that can exploit with our approach. At the frame level operation, a face detector only detects the face of subject. So, this is the second source of scene of inconsistency between the swapped face and rest of the scene.

 The third major weakness that is inherent to the generation process of the final video itself. Because the autoencoders is used frame-by-frame, it is completely unaware of any previous generated face that it may have created. The most prominent is an inconsistent choice of illuminants between scenes with frames, with leads to a flickering phenomenon in the face region common to most fake videos. So, using these different basic points it can be detected whether the video is real or not.

Basically, the proposed systems are composed by a convolutional LSTM structure for processing frame sequences. Convolutional LSTM includes two major steps. One is the CNN for the feature extraction. This can be done by using inception-v3. This will produce a sequence which will be used by LSTM as input. The key challenge that we need to address is the design of a model to recursively process a sequence in a meaningful manner. Then, use a SoftMax layer to compute the probabilities of the frame sequence being either pristine or DeepFake.

Benefits of the Project

Deepfakes — a technology originally used by Reddit perverts who wanted to superimpose their favorite actresses’ faces onto the bodies of porn stars – have come a long way since the original Reddit group was banned. Using computer technology to synthesize videos isn’t exactly new. Deep fakes are a giant failure and huge missed opportunity that idiots are globally ruining for cheap gags and illegal pornography.

Remember in Forrest Gump, how Tom Hanks kept popping up in the background of footage of important historical events, and got a laugh from President Kennedy? It wasn’t created using AI, but the result is the same. In other cases, such technology has been used to complete a film when an actor dies during production. Deepfake is majorly used for revenge porn to defame notable celebrities. As soon as fake videos go viral people believe initially and keep sharing with others makes the targeted person become embarrass watching such unusual acts. Until and unless an official statement of the targeted personality does not come, many people start believing making their life difficult, especially when they are criticized by their fans via social media platforms like Facebook, Twitter or Instagram.

 The project “Development of Algorithm to Detect Deepfakes images and videos” will detect these kinds of fake images and videos. As mention previously that Deepfake is a threat to our society ,many such type of content mislead people ,defame many celebrities, moreover it’s also effecting democracy .So there is severe need to development an algorithm and fix this problem.as in previous case the detection of these kind of videos may save these situation where someone just use it for revenge purposes.

Politicians are the most common victims of DeepFakes. Less than a year before the above video, University of Washington computer scientists had used neural network AI to model the shape of Obama’s mouth and the video got viral within days .so detection of such fakes videos ca save these kind of political situations.

Recently, Deepfake videos of notable figures like Mark Zuckerberg, and US House Speaker Nancy Pelosi have all made the news. And it’s becoming increasingly difficult to tell them apart from the real thing.

Technical Details of Final Deliverable

Deep learning has been successfully applied to solve various complex problems ranging from big data analytics to computer vision and human-level control. Deep learning advances however have also been employed to create software that can cause threats to privacy, democracy and national security. One of those deep learning-powered applications recently emerged is DeepFake". Deepfake algorithms can create fake images and videos that humans cannot distinguish them from authentic ones. The proposal of technologies that can automatically detect and assess the integrity sof digital visual media

is therefore indispensable.

 This project will develop an optimize algorithms used to detect DeepFakes images and videos. An extensive discussion on challenges, research trends and directions related to DeepFake technologies. By reviewing the background of DeepFakes and state-of-the-art Deepfake detection methods. Although, literature provides different ways to design the optimize Algorithm to detect these kinds of images and videos.

Firstly, Algorithm will be developed to detect fake images and later project will extend to detect fake videos. Using deep learning models, firstly features will be extracted and trained these features using a large data set of real and fake images and videos will classify whether a video should manipulate or not. So, by using some different techniques the project will be based on creation and detection of Deepfakes using optimize algorithm.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

IT

Other Industries

Education

Core Technology

Artificial Intelligence(AI)

Other Technologies

Sustainable Development Goals

Decent Work and Economic Growth, Sustainable Cities and Communities, Peace and Justice Strong Institutions

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com