HCiT: Deepfake Video Detection Using a Hybrid Model of CNN features and Vision Transformer

Archive ouverte : Communication dans un congrès

Kaddar, Bachir | Fezza, Sid Ahmed | Hamidouche, Wassim | Akhtar, Zahid | Hadid, Abdenour

Edité par HAL CCSD ; IEEE

International audience. The number of new falsified video contents is dramatically increasing, making the need to develop effective deepfake detection methods more urgent than ever. Even though many existing deepfake detection approaches show promising results, the majority of them still suffer from a number of critical limitations. In general, poor generalization results have been obtained under unseen or new deepfake generation methods. Consequently, in this paper, we propose a deepfake detection method called HCiT, which combines Convolutional Neural Network (CNN) with Vision Transformer (ViT). The HCiT hybrid architecture exploits the advantages of CNN to extract local information with the ViT's self-attention mechanism to improve the detection accuracy. In this hybrid architecture, the feature maps extracted from the CNN are feed into ViT model that determines whether a specific video is fake or real. Experiments were performed on Faceforensics++ and DeepFake Detection Challenge preview datasets, and the results show that the proposed method significantly outperforms the state-of-the-art methods. In addition, the HCiT method shows a great capacity for generalization on datasets covering various techniques of deepfake generation. The source code is available at: https://github.com/KADDAR-Bachir/HCiT

Consulter en ligne

Suggestions

Du même auteur

Automatic pain estimation from facial expressions: a comparative analysis using off-the-shelf CNN architectures | El Morabit, Safaa

Automatic pain estimation from facial expressions: a comparative analysis u...

Archive ouverte: Article de revue

El Morabit, Safaa | 2021-08-11

International audience. Automatic pain recognition from facial expressions is a challenging problem that has attracted a significant attention from the research community. This article provides a comprehensive analy...

Covid-19 recognition using ensemble-cnns in two new chest x-ray databases | Vantaggiato, Edoardo

Covid-19 recognition using ensemble-cnns in two new chest x-ray databases

Archive ouverte: Article de revue

Vantaggiato, Edoardo | 2021-03-03

The used datasets were obtained from publically open source datastes from: 1 ieee8023/covid-chestxray-dataset https://github.com/ieee8023/covid-chestxray-dataset (accessed on 2 March 2021); 2 Chest X-Ray Images (Pneumonia) from Ka...

Two Ensemble-CNN Approaches for Colorectal Cancer Tissue Type Classification | Paladini, Emanuela

Two Ensemble-CNN Approaches for Colorectal Cancer Tissue Type Classificatio...

Archive ouverte: Article de revue

Paladini, Emanuela | 2021

International audience. In recent years, automatic tissue phenotyping has attracted increasing interest in the Digital Pathology (DP) field. For Colorectal Cancer (CRC), tissue phenotyping can diagnose the cancer an...

Du même sujet

9. Incremental deep learning model for plant leaf diseases detection | Ouadfel, Salima

9. Incremental deep learning model for plant leaf diseases detection

Archive ouverte: Type de document indéfini

Ouadfel, Salima | 2022-09-30

International audience. In recent years, deep learning has revolutionized machine learning and has been used with great success in various engineering fields, such as transportation, agriculture, finance, and market...

EME-Net: A U-net-based Indoor EMF Exposure Map Reconstruction Method | Mallik, Mohammed

EME-Net: A U-net-based Indoor EMF Exposure Map Reconstruction Method

Archive ouverte: Communication dans un congrès

Mallik, Mohammed | 2022-03-27

International audience. In wireless communication systems, in order to respond to the perception of risks related to electromagnetic field exposure and allocate radio resources, the estimation of the received power ...

10. Incremental learning of convolutional neural networks in bioinformatics | Mousser, Wafa

10. Incremental learning of convolutional neural networks in bioinformatics

Archive ouverte: Type de document indéfini

Mousser, Wafa | 2022

International audience. In recent years, convolutional neural networks (CNNs) have been widely used in various computer visual recognition tasks and then extensively applied for medical images, particularly for comp...

Deep learning based face beauty prediction via dynamic robust losses and ensemble regression | Bougourzi, F.

Deep learning based face beauty prediction via dynamic robust losses and en...

Archive ouverte: Article de revue

Bougourzi, F. | 2022-04

International audience. In the last decade, several studies have shown that facial attractiveness can be learned by machines. In this paper, we address Facial Beauty Prediction from static images. The paper contains...

IDT: an incremental deep tree framework for biological image classification | Mousser, Wafa

IDT: an incremental deep tree framework for biological image classification

Archive ouverte: Article de revue

Mousser, Wafa | 2022-12

International audience. Nowadays, breast and cervical cancers are respectively the first and fourth most common causes of cancer death in females. It is believed that, automated systems based on artificial intellige...

Robust Automatic Modulation Recognition Through Joint Contribution of Hand-Crafted and Contextual Features | Jdid, Bachir

Robust Automatic Modulation Recognition Through Joint Contribution of Hand-...

Archive ouverte: Article de revue

Jdid, Bachir | 2021

International audience. Automatic modulation recognition (AMR) has become increasingly important in the field of signal processing, especially with the advancements of intelligent communication systems. Deep Learnin...

Chargement des enrichissements...