A deepfake detection framework developed as a project for the Fondamenti di Visione Artificiale e Biometria (Fundamentals of Computer Vision and Biometrics) course, part of the Computer Science Master's Degree program at the University of Salerno
| Name | GitHub profile |
|---|---|
| Aviello Raffaele | raffaeleav |
| Califano Alfonso | Ackermann32 |
| Capuano Rachele | raaacheeelc |
Detectwins was developed to explore Triplet Mining (and its variations) for deepfake detection while using Fourier-transformed images. The latter technique is used to leverage frequency domain representation of images, where visual signals are converted to highlight details of the images that are not visible in the spatial domain (e.g. ai artifacts). The framework is also capable of analyzing RGB images (with some success).
- Offline Semi-Hard Mining
- Offline Hard Mining
- Online Hard Mining
- One-Shot Learning
| Method | Accuracy | Precision | Recall | Specifity | F1 Score |
|---|---|---|---|---|---|
| Online Hard Mining | 88.10 | 80.65 | 84.60 | 89.85 | 82.57 |
| One Shot Learning (BigGan) | 90.93 | 79.59 | 97.89 | 87.45 | 86.67 |
| One Shot Learning (Latent Diffusion) | 78.36 | 69.35 | 62.90 | 86.10 | 65.71 |
- Artifact dataset
- Python dependencies are listed in the "requirements.txt" file
- Clone the repository
git clone https://github.com/raffaeleav/detectwins.git- Install dependencies (assuming conda is being used)
conda create -n "detectwins" python=3.10
conda activate detectwins
pip install -r detectwins/requirements.txt- Install Cuda toolkit (assuming you have Ubuntu 22.04)
wget https://developer.download.nvidia.com/compute/cuda/12.1.0/local_installers/cuda_12.1.0_530.30.02_linux.runsudo
sh cuda_12.1.0_530.30.02_linux.run
pip install torch torchvision torchaudio- Download Artifact dataset
kaggle datasets download -d awsaf49/artifact-dataset
unzip artifact-dataset.zip -d artifact- Switch to the project directory
cd detectwins- Start the Jupyter server and run testing
jupyter notebook notebooks/testing.ipynb- Python - used for the Triplet Mining algorithms
- Jupyter Notebooks - used for model training