A deep learning approach to face swap detection

Svetlana Volkova, Alexey Bogdanov

Abstract


We propose the method for detecting an incident at face authentication when an imposter falsifies the client's face using a digital technique named Face Swap to cheat the system. The method is based on a convolutional neural network to get facial features and classify them. The proposed method can work with faces obtained with low quality and heavy lighting conditions. It is confirmed by experiments on a big test dataset. Experiments show that the accuracy reaches values over 98% for low-quality images and over 99% for high-quality images. Classification results are congruent to the best results shown by the other known methods tested on the same test dataset. The proposed method can be applied to improve the quality of face authentication systems.


Full Text:

PDF

References


S. Salmani and D. Yadav, "Deepfake: A survey on facial forgery technique using generative adversarial network," 2019 International Conference on Intelligent Computing and Control Systems (ICCS). IEEE, 2019, doi: 10.1109/ICCS45141.2019.9065881.

Vooruit, Officiële Twitter-account van Vooruit, socialistische beweging “Trump heeft een boodschap voor alle Belgen” Twitter, 20 May 2018, Available: https://twitter.com/vooruit_nu/status/998089909369016325.

South China Morning Post, (2021, March 31), “Chinese government-run facial recognition system hacked by tax fraudsters: report,” South China Morning Post, Available: https://www.scmp.com/tech/tech-trends/article/3127645/chinese-government-run-facial-recognition-system-hacked-tax (16 June 2021).

B. Zeno, I. Kalinovskiy, Y. Matveev, and B. Alkhatib, "CtrlFaceNet: Framework for geometric-driven face image synthesis," Pattern Recognition Letters, vol. 138, pp 527-533, 2020, doi: 10.1016/j.patrec.2020.08.026.

J. Zhao, L. Xiong, J. Karlekar, J. Li, F. Zhao, Z. Wang, P. S. Pranata, P. S. Shen, S. Yan, and J. Feng, “Dual-Agent GANs for Photorealistic and Identity Preserving Profile Face Synthesis,” In NIPS, vol. 2, p. 3, Jan. 2017.

T. Karras, T. Aila, S. Laine, and J. Lehtinen, “Progressive growing of GANs for improved quality, stability, and variation,” International Conference on Learning Representations (ICLR), 2018.

J. Thies, M. Zollhöfer, M. Stamminger, C. Theobalt, and M. Nießner, “Face2Face: Real-Time Face Capture and Reenactment of RGB Videos,” In Conference on Computer Vision and Pattern Recognition (CVPR), 2387-2395, doi: 10.1109/CVPR.2016.262.

R. Tolosana, R. Vera-Rodriguez, J. Fierrez, A. Morales, and J. Ortega-Garcia, “DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection,” Information Fusion, vol. 64, 2020, pp. 131–148, doi: 10.1016/j.inffus.2020.06.014.

I. Korshunova, W. Shi, J. Dambre, and L. Theis, “Fast face-swap using convolutional neural networks,” In 2017 IEEE International Conference on Computer Vision (ICCV), Oct 2017, pp. 3697–3705, doi: 10.1109/ICCV.2017.397.

Y. Mirsky and W. Lee “The Creation and Detection of Deepfakes: A Survey,” ACM Computing Surveys, vol. 54, iss. 1, April 2021, pp 1–41, doi: 10.1145/3425780.

Y. Li and S. Lyu, “Exposing DeepFake Videos By Detecting Face Warping Artifacts,” In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2019.

A. Agarwal, R. Singh, M. Vatsa, and A. Noore. “SWAPPED! Digital face presentation attack detection via weighted local magnitude pattern,” In 2017 IEEE International Joint Conference on Biometrics (IJCB), IEEE, 2017, pp. 659–665.

R. Durall, M. Keuper, F-J Pfreundt, and J. Keuper, “Unmasking DeepFakes with simple Features,” arXiv preprint arXiv:1911.00686 (2019).

H. Mo, B. Chen, and W. Luo, “Fake faces identification via convolutional neural network,” In Proceedings of the 6th ACM Workshop on Information Hiding and Multimedia Security, ACM, 2018, pp. 43–47, doi: 10.1145/3206004.3206009.

L. Li, J. Bao, T. Zhang, H. Yang, D. Chen, F. Wen, and B. Guo, “Face x-ray for more general face forgery detection,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5001–5010, 2020, doi: 10.1109/CVPR42600.2020.00505.

X. Li, K. Yu, S. Ji, Y. Wang, C. Wu, and H. Xue, “Fighting Against Deepfake: Patch&Pair Convolutional Neural Networks (PPCNN),” In Companion Proceedings of the Web Conference 2020, 2020, pp. 88–89, doi: 10.1145/3366424.3382711.

Y. Nirkin, L. Wolf, Y. Keller, and T. Hassner, “DeepFake Detection Based on the Discrepancy Between the Face and its Context,” arXiv preprint arXiv:2008.12262 (2020).

S. Agarwal, H. Farid, Y. Gu, M. He, K. Nagano, and H. Li, “Protecting World Leaders Against Deep Fakes,” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2019, pp. 38–45.

T. Mittal, U. Bhattacharya, R. Chandra, A. Bera, and D. Manocha, “Emotions Don’t Lie: A Deepfake Detection Method using Audio-Visual Affective Cues,” arXiv preprint arXiv:2003.06711 (2020).

D. Guera and E. J. Delp, “Deepfake video detection using recurrent neural networks,” In IEEE Conference on Advanced Video and Signal Based Surveillance (AVSS), IEEE, 2018, pp. 1–6, doi: 10.1109/AVSS.2018.8639163

R. Wang, L. Ma, F. Juefei-Xu, X. Xie, J. Wang, and Y. Liu, “Fakespotter: A simple baseline for spoofing ai-synthesized fake faces,” arXiv preprint arXiv:1909.06122 (2019).

S. Fernandes, S. Raj, R. Ewetz, J. S. Pannu, S. K. Jha, E. Ortiz, I. Vintila, and M. Salter, “Detecting Deepfake Videos Using Attribution-Based Confidence Metric,” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020, pp. 308–309, doi: 10.1109/CVPRW50498.2020.00162.

A. Khodabakhsh and C. Busch “A generalizable Feepfake detector based on neural conditional distribution modeling,” 2020 International Conference of the Biometrics Special Interest Group (BIOSIG). IEEE, 2020.

A. Rossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner, “FaceForensics++: Learning to detect manipulated facial images,” In International Conference on Computer Vision (ICCV), 2019, doi: 10.1109/ICCV.2019.00009.

D. Afchar, V. Nozick, J. Yamagishi, and I. Echizen, “Mesonet: a compact facial video forgery detection network,” arXiv preprint arXiv:1809.00888 (2018).

M. Tarasiou and S. Zafeiriou, “Extracting deep local features to detect manipulated images of human faces,” In 2020 IEEE International Conference on Image Processing (ICIP), 2020, pp. 1821-1825, doi: 10.1109/ICIP40778.2020.9190714.

X. Ding, Z. Raziei, E. C. Larson, E. V. Olinick, P. Krueger, and M. Hahsler, “Swapped Face Detection using Deep Learning and Subjective Assessment,” arXiv preprint arXiv:1909.04217 (2019).

N.T. Do, I. S. Na, and S. H. Kim, “Forensics Face Detection From GANs Using Convolutional Neural Network,” In ISITC’2018, 2018, pp. 376-379.

C.C. Hsu, Y.X. Zhuang, and C.Y. Lee, “Deep fake image detection based on pairwise learning,” Applied Sciences, vol. 10, no. 1:370, 2020, doi: 10.3390/app10010370.

T. Fernando, C. Fookes, S. Denman, and S. Sridharan, “Exploiting Human Social Cognition for the Detection of Fake and Fraudulent Faces via Memory Networks,” arXiv preprint arXiv:1911.07844 (2019).

S. Zhang, X. Zhu, Z. Lei, H. Shi, X. Wang, and S. Z. Li, “S^3FD: Single shot scale-invariant face detector,” In ICCV, 2017, doi: 10.1109/ICCV.2017.30.

X. Tang, D. K. Du, Z. He, and J. Liu, “Pyramidbox: A context-assisted single shot face detector,” In ECCV, 2018, doi: 10.1007/978-3-030-01240-3_49.

J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, F. F. Li, “ImageNet: A Large-Scale Hierarchical Image Database”, In CVPR09, 2009, doi: 10.1109/CVPR.2009.5206848.

M. Sandler, A. Howard, M. Zhu A. Zhmoginov, and L. Chen, “MobileNetV2: inverted residuals and linear bottlenecks,” In Conference on Computer Vision and Pattern Recognition, 2018, pp. 4510–4520, doi: 10.1109/CVPR.2018.00474.

I. Loshchilov and F. Hutter, “Decoupled Weight Decay Regularization,” 7th International Conference on Learning Representations, ICLR, 2019.

D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” In ICLR, 2015.

I. Loshchilov and F. Hutter, “SGDR: Stochastic gradient descent with warm restarts,” In International Conference on Learning Representations, 2017.

G. Wang, J. Zhou, and Y. Wu Exposing Deep-faked Videos by Anomalous Co-motion Pattern Detection,” arXiv preprint arXiv:2008.04848 (2020)


Refbacks

  • There are currently no refbacks.


Abava  Кибербезопасность IT Congress 2024

ISSN: 2307-8162