这是indexloc提供的服务，不要输入任何密码
Skip to main content
Springer Nature Link
Log in

CG-FAS: Cross-label Generative Augmentation for Face Anti-Spoofing

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Face Anti-Spoofing (FAS) is essential to secure face recognition systems from various physical attacks. A sufficient and diverse training set helps to build robust FAS models. To exploit the potential of FAS datasets, we propose to generate high-quality data including live and diverse presentation attacks (PAs) faces, for data augmentation during the model training stage. Our method is called Cross-label Generative augmentation for Face Anti-Spoofing (CG-FAS), which could convert a live face into a 3D high-fidelity mask, replay, print, or other extra physical PAs. Correspondingly, CG-FAS can also restore a specific physical presentation attack into a live face. This function is realized by innovatively building an Interchange Bridge matrix, which stores disentangled spoof clues between PAs and live faces. To verify the effects of these generated data, we utilize them as augmentation data and conduct experiments on several typical FAS benchmarks. Extensive experimental results demonstrate the superior performance gain with CG-FAS for off-the-shelf data-driven FAS models. We hope the CG-FAS can shine a light on the deep FAS community to alleviate the data-hungry issue. The code will be released soon at: https://github.com/liuxingwt/CG-FAS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

Data Availability

The data from four public face anti-spoofing datasets and one human face dataset (i.e., OULU-NPU (Boulkenafet et al., 2017b), SiW (Liu et al., 2018a), HiFiMask (Liu et al., 2022), HKBU MARsV2 (Liu et al., 2016) and FFHQ (Karras et al., 2019)) that support the findings of this study are available from the third party institutions (including University of Oulu, Michigan State University, Institute of Automation, Chinese Academy of Sciences, Hong Kong Baptist University and NVIDIA) but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of the above-mentioned third-party institutions.

References

  • Abdal, R., Qin, Y., & Wonka, P. (2019). Image2StyleGAN: How to embed images into the StyleGAN latent space? In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4432–4441.

  • Abdal, R., Qin, Y., & Wonka, P. A. (2020). Image2StyleGAN++: How to edit the embedded images? In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8296–8305.

  • Arjovsky, M., Chintala, S., & Bottou, L. (2017). Wasserstein generative adversarial networks. In International conference on machine learning, PMLR. pp. 214–223.

  • Boulkenafet, Z., Komulainen, J., & Hadid, A. (2015). Face anti-spoofing based on color texture analysis. In International conference on image processing (ICIP).

  • Boulkenafet, Z., Komulainen, J., & Hadid, A. (2017). Face antispoofing using speeded-up robust features and fisher vector encoding. IEEE Signal Processing Letters, 24(2), 141–145.

    Google Scholar 

  • Boulkenafet, Z., Komulainen, J., Li, L., Feng, X., & Hadid, A. (2017b). OULU-NPU: A mobile face presentation attack database with real-world variations. In FGR, pp. 612–618.

  • Brock, A., Donahue, J., & Simonyan, K. (2018). Large scale GAN training for high fidelity natural image synthesis. In International conference on learning representations.

  • Deng, J., Guo, J., Xue, N., & Zafeiriou, S. (2019). ArcFace: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4690–4699.

  • Diederik, P, K. & Max, W. (2014). Auto-encoding variational bayes. International conference on learning representation.

  • Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. ICLR.

  • Fang, H., Liu, A., Wan, J., Escalera, S., Zhao, C., Zhang, X., Li, S. Z., & Lei, Z. (2024a). Surveillance face anti-spoofing. IEEE Transactions on Information Forensics and Security, 19, 1535–1546.

    Article  Google Scholar 

  • Fang, H., Liu, A., Yuan, H., Zheng, J., Zeng, D., Liu, Y., Deng, J., Escalera, S., Liu, X., Wan, J., & Lei, Z. (2024b). Unified physical-digital face attack detection.

  • Fang, M., Damer, N., Kirchbuchner, F., & Kuijper, A. (2022). Learnable multi-level frequency decomposition and hierarchical attention mechanism for generalized face presentation attack detection. In Proceedings of the IEEE/CVF winter conference on applications of computer vision.

  • Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. Advances in Neural Information Processing Systems, 27.

  • Härkönen, E., Hertzmann, A., Lehtinen, J., & Paris, S. (2020). GANSpace: Discovering interpretable GAN controls. Advances in Neural Information Processing Systems, 33, 9841–9850.

    Google Scholar 

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016a). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778.

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016b). Deep residual learning for image recognition. In CVPR.

  • Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017). GANs trained by a two time-scale update rule converge to a local nash equilibrium. Advances in Neural Information Processing Systems, 30.

  • Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017). Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125–1134.

  • Jourabloo, A., Liu, Y., & Liu, X. (2018). Face de-spoofing: Anti-spoofing via noise modeling. In ECCV.

  • Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2018). Progressive growing of GANs for improved quality, stability, and variation. In International conference on learning representations.

  • Karras, T., Aittala, M., Hellsten, J., Laine, S., Lehtinen, J., & Aila, T. (2020). Training generative adversarial networks with limited data. Advances in Neural Information Processing Systems, 33, 12104–12114.

    Google Scholar 

  • Karras, T., Aittala, M., Laine, S., Härkönen, E., Hellsten, J., Lehtinen, J., & Aila, T. (2021). Alias-free generative adversarial networks. Advances in Neural Information Processing Systems, 34.

  • Karras, T., Laine, S., & Aila, T. (2019). A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401–4410.

  • Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020b). Analyzing and improving the image quality of StyleGAN. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8110–8119.

  • Komulainen, J., Hadid, A., & Pietikäinen, M. (2013). Context based face anti-spoofing. In 2013 IEEE sixth international conference on biometrics: Theory, applications and systems (BTAS), pp. 1–8.

  • Ling, H., Kreis, K., Li, D., Kim, S. W., Torralba, A., & Fidler, S. (2021). EditGAN: High-precision semantic image editing. Advances in Neural Information Processing Systems, 34.

  • Liu, A., Tan, Z., Escalera, S., Guo, G., & Li, S. Z. (2021a). CASIA-SURF CeFA : A Benchmark for Multi-modal Cross-ethnicity Face Anti-spoofing. Proceedings of the IEEE/CVF winter conference on applications of computer vision (WACV),, pp. 1179–1187.

  • Liu, A., Zhao, C., Yu, Z., Su, A., Liu, X., Kong, Z., Wan, J., Escalera, S., Escalante, H. J., Lei, Z., et al. (2021b). 3D high-fidelity mask face presentation attack detection challenge. In Proceedings of the IEEE/CVF international conference on computer vision(ICCV) workshops, pp. 814–823.

  • Liu, A., Zhao, C., Yu, Z., Wan, J., Su, A., Liu, X., Tan, Z., Escalera, S., Xing, J., Liang, Y., et al. (2022). Contrastive context-aware learning for 3D high-fidelity mask face presentation attack detection. IEEE Transactions on Information Forensics and Security, 17, 2497–2507.

    Article  Google Scholar 

  • Liu, S., Yang, B., Yuen, P. C., & Zhao, G. (2016). A 3D mask face anti-spoofing database with real world variations. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 100–106.

  • Liu, Y., Jourabloo, A., & Liu, X. (2018a). Learning deep models for face anti-spoofing: Binary or auxiliary supervision. In CVPR.

  • Liu, Y., Jourabloo, A., & Liu, X. (2018b). Learning deep models for face anti-spoofing: Binary or auxiliary supervision. In CVPR.

  • Liu, Y., Stehouwer, J., Jourabloo, A., & Liu, X. (2019). Deep tree learning for zero-shot face anti-spoofing. In CVPR.

  • Liu, Y., Stehouwer, J., & Liu, X. (2020). On disentangling spoof trace for generic face anti-spoofing. In ECCV.

  • Lucena, O., Junior, A., Moia, V., Souza, R., Valle, E., & Lotufo, R. (2017). Transfer learning using convolutional neural networks for face anti-spoofing. In International conference image analysis and recognition, pp. 27–34.

  • Menotti, D., Chiachia, G., Pinto, A., Schwartz, W. R., Pedrini, H., Falcao, A. X., & Rocha, A. (2015). Deep representations for iris, face, and fingerprint spoofing detection. IEEE Transactions on Information Forensics and Security, 10(4), 864–879.

    Article  Google Scholar 

  • Miyato, T., Kataoka, T., Koyama, M., & Yoshida, Y. (2018). Spectral normalization for generative adversarial networks. In International conference on learning representations.

  • Nagpal, C. & Dubey, S. R. (2019). A performance evaluation of convolutional neural networks for face anti spoofing. In 2019 international joint conference on neural networks (IJCNN), pp. 1–8. IEEE.

  • Patashnik, O., Wu, Z., Shechtman, E., Cohen-Or, D., & Lischinski, D. (2021). StyleCLIP: Text-driven manipulation of StyleGAN imagery. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 2085–2094.

  • Patel, K., Han, H., & Jain, A. K. (2016). Secure face unlock: Spoof detection on smartphones. IEEE Transactions on Information Forensics & Security, 11(10), 2268–2283.

    Article  Google Scholar 

  • Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021). Learning transferable visual models from natural language supervision. In International conference on machine learning, PMLR. pp. 8748–8763.

  • Radford, A., Metz, L., & Chintala, S. (2016). Unsupervised representation learning with deep convolutional generative adversarial networks. In International conference on learning representations.

  • Richardson, E., Alaluf, Y., Patashnik, O., Nitzan, Y., Azar, Y., Shapiro, S., & Cohen-Or, D. (2021). Encoding in style: a StyleGAN encoder for image-to-image translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2287–2296.

  • Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684–10695.

  • Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In International conference on medical image computing and computer-assisted intervention, Springer. pp. 234–241.

  • Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., & Aberman, K. (2023). DreamBooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 22500–22510.

  • Shao, R., Lan, X., & Yuen, P. C. (2017). Deep convolutional dynamic texture learning with adaptive channel-discriminability for 3D mask face anti-spoofing. In IJCB, pp. 748–755.

  • Shen, Y., Gu, J., Tang, X., & Zhou, B. (2020a). Interpreting the latent space of GANs for semantic face editing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9243–9252.

  • Shen, Y., Yang, C., Tang, X., & Zhou, B. (2020b). InterFaceGAN: Interpreting the disentangled face representation learned by GANs. IEEE Transactions on Pattern Analysis and Machine Intelligence., 44(4), 2004–2018.

    Article  Google Scholar 

  • Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., & Ganguli, S. (2015). Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, PMLR. pp. 2256–2265.

  • Sun, Y., Liu, Y., Liu, X., Li, Y., & Chu, W.-S. (2023). Rethinking domain generalization for face anti-spoofing: Separability and alignment. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 24563–24574.

  • Tiago, D., Anjos, A., Martino, J. D., & Marcel, S. (2013). Can face anti-spoofing countermeasures work in a real world scenario? In International conference on biometrics, pp. 1–8.

  • Tov, O., Alaluf, Y., Nitzan, Y., Patashnik, O., & Cohen-Or, D. (2021). Designing an encoder for StyleGAN image manipulation. ACM Transactions on Graphics (TOG), 40(4), 1–14.

    Article  Google Scholar 

  • Van Oord, A., Kalchbrenner, N., & Kavukcuoglu, K. (2016). Pixel recurrent neural networks. In International conference on machine learning, PMLR. pp. 1747–1756.

  • Wang, C.-Y., Lu, Y.-D., Yang, S.-T., & Lai, S.-H. (2022a). PatchNet: A simple face anti-spoofing framework via fine-grained patch recognition. In CVPR.

  • Wang, Z., Wang, Z., Yu, Z., Deng, W., Li, J., Gao, T., & Wang, Z. (2022b). Domain generalization via shuffled style assembly for face anti-spoofing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4123–4133.

  • Wang, Z., Yu, Z., Wang, X., Qin, Y., Li, J., Zhao, C., Liu, X., & Lei, Z. (2023). Consistency regularization for deep face anti-spoofing. IEEE Transactions on Information Forensics and Security, 18, 1127–1140.

    Article  Google Scholar 

  • Wang, Z., Yu, Z., Zhao, C., Zhu, X., Qin, Y., Zhou, Q., Zhou, F., & Lei, Z. (2020). Deep spatial gradient and temporal depth learning for face anti-spoofing. In CVPR.

  • Wu, H., Zeng, D., Hu, Y., Shi, H., & Mei, T. (2021). Dual spoof disentanglement generation for face anti-spoofing with depth uncertainty learning. IEEE Transactions on Circuits and Systems for Video Technology, 32(7), 4626–4638.

    Article  Google Scholar 

  • Wu, Z., Lischinski, D., & Shechtman, E. (2021b). StyleSpace analysis: Disentangled controls for StyleGAN image generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12863–12872.

  • Xia, W., Zhang, Y., Yang, Y., Xue, J. -H., Zhou, B., & Yang, M. -H. (2021). GAN inversion: A survey. arXiv:2101.05278.

  • Xu, Z., Li, S., & Deng, W. (2015). Learning temporal features using LSTM-CNN architecture for face anti-spoofing. In ACPR, pp. 141–145.

  • Yang, X., Luo, W., Bao, L., Gao, Y., Gong, D., Zheng, S., Li, Z., & Liu, W. (2019). Face anti-spoofing: Model matters, so does data. In CVPR.

  • Yu, Z., Li, X., Niu, X., Shi, J., & Zhao, G. (2020). Face anti-spoofing with human material perception. In ECCV, Springer. pp. 557–575.

  • Yu, Z., Qin, Y., Li, X., Zhao, C., Lei, Z., & Zhao, G. (2022). Deep learning for face anti-spoofing: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5), 5609–5631.

    Google Scholar 

  • Yu, Z., Qin, Y., Zhao, H., Li, X., & Zhao, G. (2021). Dual-cross central difference network for face anti-spoofing. arXiv:2105.01290.

  • Yu, Z., Wan, J., Qin, Y., Li, X., Li, S. Z., & Zhao, G. (2020). NAS-FAS: Static-dynamic central difference network search for face anti-spoofing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(9), 3005–3023.

  • Yu, Z., Zhao, C., Wang, Z., Qin, Y., Su, Z., Li, X., Zhou, F., & Zhao, G. (2020). Searching central difference convolutional networks for face anti-spoofing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition.

  • Zhang, K.-Y., Yao, T., Zhang, J., Liu, S., Yin, B., Ding, S., & Li, J. (2021). Structure destruction and content combination for face anti-spoofing. In IJCB.

  • Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018). The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586–595.

  • Zhang, Y., Yin, Z., Li, Y., Yin, G., Yan, J., Shao, J., & Liu, Z. (2020). CelebA-Spoof: Large-scale face anti-spoofing dataset with rich annotations. In European conference on computer vision, Springer. pp. 70–85.

Download references

Acknowledgements

This work was supported by the Brain-like General Vision Model and Applications project (Grant No. 2022ZD0160402), Chinese National Natural Science Foundation Projects 62276254, U23B2054, and InnoHK program, Frontier Interdiscipline Project of Tsinghua University (20221080082), National Natural Science Foundation of China under Grant 62306061, and Guangdong Basic and Applied Basic Research Foundation (Grant No. 2023A1515140037).

Author information

Author notes

  1. Xing Liu, Anyang Su and Minghui Wu have contributed equally.

Authors and Affiliations

  1. Mininglamp Technology, Beijing, China

    Anyang Su, Minghui Wu, Jie Hao & Chenxu Zhao

  2. Zelos Technology, Beijing, China

    Xing Liu

  3. Great Bay University, Dongguan, China

    Zitong Yu

  4. Shanghai Artificial Intelligence Laboratory, Shanghai, China

    Anyang Su, Kangle Wu, Da An & Chenxu Zhao

  5. Department of Hydraulic Engineering, Tsinghua University, Beijing, China

    Mengzhen Xu

  6. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Zhen Lei

  7. School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS), Beijing, China

    Zhen Lei

  8. Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences, Hong Kong, China

    Zhen Lei

Authors
  1. Xing Liu
    View author publications

    Search author on:PubMed Google Scholar

  2. Anyang Su
    View author publications

    Search author on:PubMed Google Scholar

  3. Minghui Wu
    View author publications

    Search author on:PubMed Google Scholar

  4. Zitong Yu
    View author publications

    Search author on:PubMed Google Scholar

  5. Kangle Wu
    View author publications

    Search author on:PubMed Google Scholar

  6. Da An
    View author publications

    Search author on:PubMed Google Scholar

  7. Jie Hao
    View author publications

    Search author on:PubMed Google Scholar

  8. Mengzhen Xu
    View author publications

    Search author on:PubMed Google Scholar

  9. Chenxu Zhao
    View author publications

    Search author on:PubMed Google Scholar

  10. Zhen Lei
    View author publications

    Search author on:PubMed Google Scholar

Corresponding authors

Correspondence to Chenxu Zhao or Zhen Lei.

Additional information

Communicated by Segio Escalera.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Su, A., Wu, M. et al. CG-FAS: Cross-label Generative Augmentation for Face Anti-Spoofing. Int J Comput Vis 132, 5330–5345 (2024). https://doi.org/10.1007/s11263-024-02132-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1007/s11263-024-02132-5

Keywords