diff --git a/README.md b/README.md index 4ea2e90..b28f4da 100644 --- a/README.md +++ b/README.md @@ -1 +1,66 @@ -# awesome-ml-privacy-attacks \ No newline at end of file +# Awesome atacks on ML privacy [![Awesome](https://awesome.re/badge.svg)](https://awesome.re) + +## Table of Contents + + - [Surveys](#surveys) + - [Papers](#papers) + +# Surveys + + +# Papers and Code + +## Membership inference +- Shokri, R., Stronati, M., Song, C., and Shmatikov, V. (2017). **Membership inference attacks against machine learning models**. In 2017 IEEE Symposium on Security and Privacy (SP),pages 3–18. IEEE ([link](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7958568)) ([code](https://github.com/csong27/membership-inference)) +- Yeom, S., Giacomelli, I., Fredrikson, M., and Jha, S. (2018). **Privacy risk in machine learning:Analyzing the connection to overfitting**. In 2018 IEEE 31st Computer Security FoundationsSymposium (CSF), pages 268–282. IEEE ([link](https://ieeexplore.ieee.org/document/8429311)) ([code](https://github.com/samuel-yeom/ml-privacy-csf18)) +- Nasr, M., Shokri, R., and Houmansadr, A. (2019). **Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning.** In 2019 IEEE Symposium on Security and Privacy (SP), pages 739–753. IEEE ([link](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8835245)) ([code](https://github.com/privacytrustlab/ml_privacy_meter)) +- Hayes, J., Melis, L., Danezis, G., and De Cristofaro, E. (2019). **Logan: Membership inference attacks against generative models.** Proceedings on Privacy Enhancing Technologies,2019(1):133–152 (link) ([code](https://github.com/jhayes14/gen_mem_inf)) +- Jayaraman, B. and Evans, D., 2019. **Evaluating differentially private machine learning in practice**. In 28th USENIX Security Symposium USENIX Security 19) (pp. 1895-1912). +- Rahman, M. A., Rahman, T., Laganière, R., Mohammed, N., and Wang, Y. (2018). **Membership inference attack against differentially private deep learning model**.Transactionson Data Privacy, 11(1):61–79 ([link](http://www.tdp.cat/issues16/tdp.a289a17.pdf)) +- Salem, A., Zhang, Y., Humbert, M., Berrang, P., Fritz, M., and Backes, M. (2019). **Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models**. In 26th Annual Network and Distributed System Security Symposium,NDSS 2019, San Diego, California, USA, February 24-27, 2019 ([link](https://www.ndss-symposium.org/wp-content/uploads/2019/02/ndss2019_03A-1_Salem_paper.pdf)) ([code](https://github.com/AhmedSalem2/ML-Leaks)) +- Song, L., Shokri, R., and Mittal, P. (2019). **Privacy risks of securing machine learning models against adversarial examples**. In Proceedings of the 2019 ACM SIGSAC Conference onComputer and Communications Security, CCS ’19, page 241–257, New York, NY, USA.Association for Computing Machinery. ([link](https://dl.acm.org/doi/pdf/10.1145/3319535.3354211)) ([code](https://github.com/inspire-group/privacy-vs-robustness)) +- Sablayrolles, A., Douze, M., Schmid, C., Ollivier, Y. and Jegou, H.,(2019), May. **White-box vs Black-box: Bayes Optimal Strategies for Membership Inference**. In International Conference on Machine Learning (pp. 5558-5567). (link) ([link](http://proceedings.mlr.press/v97/sablayrolles19a.html)) +- Shokri, R., Strobel, M., and Zick, Y. (2019). **Privacy risks of explaining machine learning models**.arXiv preprint arXiv:1907.00164 ([link](https://arxiv.org/abs/1907.00164)) +- Truex, S., Liu, L., Gursoy, M.E., Yu, L. and Wei, W. (2019). **Demystifying membership inference attacks in machine learning as a service**. IEEE Transactions on Services Computing. ([link](https://ieeexplore.ieee.org/abstract/document/8634878)) +- Chen, D., Yu, N., Zhang, Y. and Fritz, M., 2019. **Gan-leaks: A taxonomy of membership inference attacks against gans**. arXiv preprint arXiv:1909.03935. ([link](https://arxiv.org/pdf/1909.03935.pdf)) +- Hilprecht, B., Härterich, M. and Bernau, D., 2019. **Monte carlo and reconstruction membership inference attacks against generative models**. Proceedings on Privacy Enhancing Technologies, 2019(4), pp.232-249. ([link](https://content.sciendo.com/view/journals/popets/2019/4/article-p232.xml)) +- Jia, J., Salem, A., Backes, M., Zhang, Y. and Gong, N.Z., 2019, November. **MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples**. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (pp. 259-274). ([link](https://arxiv.org/abs/1909.10594)) ([code](https://github.com/jjy1994/MemGuard)) +- Hisamoto, Sorami, Matt Post, and Kevin Duh. "**Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?**" Transactions of the Association for Computational Linguistics 8 (2020): 49-63. ([link](https://www.mitpressjournals.org/doi/full/10.1162/tacl_a_00299)) +- Long, Y., Bindschaedler, V., Wang, L., Bu, D., Wang, X., Tang, H., Chen, K. (2018). **Understanding membership inferences on well-generalized learning models**. arXiv preprint arXiv:1802.04889. ([link](https://arxiv.org/pdf/1802.04889)) + +## Reconstruction +Reconstruction attacks cover also attacks known as *model inversion* and *attribute inference*. +- Fredrikson, M., Lantz, E., Jha, S., Lin, S., Page, D., and Ristenpart, T. (2014). **Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing**. In 23rd USENIX Security Symposium (USENIX Security 14), pages 17–32, San Diego, CA.USENIX Association ([link](https://www.usenix.org/system/files/conference/usenixsecurity14/sec14-paper-fredrikson-privacy.pdf)) +- Fredrikson, M., Jha, S., and Ristenpart, T. (2015). **Model inversion attacks that exploit confidence information and basic countermeasures**. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1322–1333. ACM ([link](https://dl.acm.org/doi/pdf/10.1145/2810103.2813677)) ([code](https://github.com/yashkant/Model-Inversion-Attack)) +- Wu, X., Fredrikson, M., Jha, S. and Naughton, J.F., 2016, June. **A methodology for formalizing model-inversion attacks**. In 2016 IEEE 29th Computer Security Foundations Symposium (CSF) (pp. 355-370). IEEE. ([link](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7536387)) +- Hitaj, B., Ateniese, G., and Perez-Cruz, F. (2017). **Deep models under the gan: Information leakage from collaborative deep learning**. InProceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS ’17, page 603–618, New York,NY, USA. Association for Computing Machinery ([link](https://dl.acm.org/doi/pdf/10.1145/3133956.3134012)) +- Song, C., Ristenpart, T. and Shmatikov, V., 2017, October. **Machine learning models that remember too much**. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (pp. 587-601). ([link](https://dl.acm.org/doi/pdf/10.1145/3133956.3134077)) ([code](https://github.com/csong27/ml-model-remember)) +- Hidano, S., Murakami, T., Katsumata, S., Kiyomoto, S., & Hanaoka, G. (2017, August). **Model inversion attacks for prediction systems: Without knowledge of non-sensitive attributes**. In 2017 15th Annual Conference on Privacy, Security and Trust (PST) (pp. 115-11509). IEEE. ([link](https://ieeexplore.ieee.org/iel7/8476191/8476869/08476925.pdf?casa_token=VQ_s2jcJFp8AAAAA:Hg-wdpPcESm9UUsZHxCLzIvYqVEqW11_OCXEyjARxW5K2cFYi6EFNXlH8IKKjNSgv6oQoQJlsw)) +- Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., and Song, D. (2019). **The secret sharer: Evaluating and testing unintended memorization in neural networks**. In 28th USENIX Security Symposium (USENIX Security 19), pages 267–284, Santa Clara, CA. USENIX Association ([link](https://www.usenix.org/system/files/sec19-carlini.pdf)) +- Zhu, L., Liu, Z., , and Han, S. (2019). **Deep leakage from gradients**. In Annual Conferenceon Neural Information Processing Systems (NeurIPS). ([link](https://papers.nips.cc/paper/9617-deep-leakage-from-gradients.pdf)) ([code](https://github.com/mit-han-lab/dlg)) +- He, Z., Zhang, T. and Lee, R.B., 2019, December. **Model inversion attacks against collaborative inference**. In Proceedings of the 35th Annual Computer Security Applications Conference (pp. 148-162). ([link](https://dl.acm.org/doi/abs/10.1145/3359789.3359824)) ([code](https://github.com/zechenghe/Inverse_Collaborative_Inference)) +- Z. Wang, M. Song, Z. Zhang, Y. Song, Q. Wang and H. Qi, "**Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning**," IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, Paris, France, 2019, pp. 2512-2520. ([link](https://ieeexplore.ieee.org/document/8737416)) +- Zhao, B., Mopuri, K. R., & Bilen, H. (2020). **iDLG: Improved Deep Leakage from Gradients**. arXiv preprint arXiv:2001.02610. ([link](https://arxiv.org/pdf/2001.02610)) ([code](https://github.com/PatrickZH/Improved-Deep-Leakage-from-Gradients)) +- Pan, X., Zhang, M., Ji, S., & Yang, M. (2020) **Privacy Risks of General-Purpose Language Models**. ([link](https://www.researchgate.net/profile/Xudong_Pan3/publication/340965355_Privacy_Risks_of_General-Purpose_Language_Models/links/5ea7ca55a6fdccd7945b6a7d/Privacy-Risks-of-General-Purpose-Language-Models.pdf)) +- Yang, Z., Zhang, J., Chang, E. C., & Liang, Z. (2019, November). **Neural network inversion in adversarial setting via background knowledge alignment**. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (pp. 225-240). ([link](https://dl.acm.org/doi/pdf/10.1145/3319535.3354261?casa_token=lDNQ40-4Wa4AAAAA:p9olQ3qMdDZ0n2sl-nNIgk4sOuLRMBTGVTxycZ5wjGpnFPf5lTz-MYw0e8ISggSseHC9T46it5yX)) +- Zhang, Y., Jia, R., Pei, H., Wang, W., Li, B., & Song, D. (2020). The secret revealer: generative model-inversion attacks against deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 253-261). ([link](http://openaccess.thecvf.com/content_CVPR_2020/papers/Zhang_The_Secret_Revealer_Generative_Model-Inversion_Attacks_Against_Deep_Neural_Networks_CVPR_2020_paper.pdf)) ([link](http://openaccess.thecvf.com/content_CVPR_2020/papers/Zhang_The_Secret_Revealer_Generative_Model-Inversion_Attacks_Against_Deep_Neural_Networks_CVPR_2020_paper.pdf)) +- Geiping, Jonas, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. "**Inverting Gradients - How easy is it to break privacy in federated learning?**." arXiv preprint arXiv:2003.14053 (2020). ([link](https://arxiv.org/abs/2003.14053)) + + +## Property inference +- Ateniese, G., Mancini, L. V., Spognardi, A., Villani, A., Vitali, D., and Felici, G. (2015). **Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers**. International Journal of Security and Networks, 10(3):137–150. ([link](https://dl.acm.org/doi/10.1504/IJSN.2015.071829)) +- Ganju, K., Wang, Q., Yang, W., Gunter, C. A., and Borisov, N. (2018). **Property inference attacks on fully connected neural networks using permutation invariant representations**. InProceedings of the 2018 ACM SIGSAC Conference on Computer and CommunicationsSecurity, pages 619–633. ACM ([link](https://dl.acm.org/doi/pdf/10.1145/3243734.3243834)) +- Melis, L., Song, C., De Cristofaro, E., and Shmatikov, V. (2019). **Exploiting unintended feature leakage in collaborative learning**. In 2019 IEEE Symposium on Security and Privacy(SP), pages 691–706. IEEE ([link](https://ieeexplore.ieee.org/iel7/8826229/8835208/08835269.pdf)) ([code](https://github.com/csong27/property-inference-collaborative-ml)) +- Congzheng Song and Vitaly Shmatikov (2020). **Overlearning Reveals Sensitive Attributes**. In International Conference on Learning Representations. ICLR ([link](https://openreview.net/pdf?id=SJeNz04tDS)) ([code](https://drive.google.com/file/d/1hu0PhN3pWXe6LobxiPFeYBm8L-vQX2zJ/view?usp=sharing)) + +## Model extraction +- Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., and Ristenpart, T. (2016). **Stealing machine learning models via prediction apis**. In 25th USENIX Security Symposium (USENIX Security 16), pages 601–618, Austin, TX. USENIX Association ([link](https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_tramer.pdf)) ([code](https://github.com/ftramer/Steal-ML)) +- Wang, B. and Gong, N. Z. (2018). **Stealing hyperparameters in machine learning**. In 2018 IEEE Symposium on Security and Privacy (SP), pages 36–52. IEEE. ([link](https://ieeexplore.ieee.org/iel7/8418581/8418583/08418595.pdf)) +- J. R. Correia-Silva, R. F. Berriel, C. Badue, A. F. de Souza and T. Oliveira-Santos, (2018). **Copycat CNN: Stealing Knowledge by Persuading Confession with Random Non-Labeled Data** In International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, 2018, pp. 1-8, doi: 10.1109/IJCNN.2018.8489592. ([link](https://ieeexplore.ieee.org/document/8489592)) ([code](https://github.com/jeiks/Stealing_DL_Models)) +- Oh, S. J., Schiele, B., and Fritz, M. (2019). **Towards reverse-engineering black-box neural networks.** In Sixth InternationalConference on Learning Representations. ICLR, Vancouver, Canada ([link](https://openreview.net/forum?id=BydjJte0-)) ([code](https://github.com/coallaoh/WhitenBlackBox)) +- Orekondy, T., Schiele, B. and Fritz, M., 2019. **Knockoff nets: Stealing functionality of black-box models**. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4954-4963). ([link](http://openaccess.thecvf.com/content_CVPR_2019/papers/Orekondy_Knockoff_Nets_Stealing_Functionality_of_Black-Box_Models_CVPR_2019_paper.pdf)) ([code](https://github.com/tribhuvanesh/knockoffnets)) +- Juuti, M., Szyller, S., Marchal, S. and Asokan, N., 2019, June. **PRADA: protecting against DNN model stealing attacks**. In 2019 IEEE European Symposium on Security and Privacy (EuroS&P) (pp. 512-527). IEEE. ([link](https://ieeexplore.ieee.org/document/8806737)) ([code](https://github.com/SSGAalto/prada-protecting-against-dnn-model-stealing-attacks)) +- Smitha Milli, Ludwig Schmidt, Anca D. Dragan, and Moritz Hardt. 2019. **Model Reconstruction from Model Explanations**. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* ’19). Association for Computing Machinery, New York, NY, USA, 1–9. ([link](https://dl.acm.org/doi/abs/10.1145/3287560.3287562)) +- Chandrasekaran, Varun, Kamalika Chaudhuri, Irene Giacomelli, Somesh Jha, and Songbai Yan. "**Exploring connections between active learning and model extraction**." Usenix 2020 ([link](https://www.usenix.org/system/files/sec20summer_chandrasekaran_prepub.pdf)) +- Jagielski, Matthew, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot. (2020) "**High Accuracy and High Fidelity Extraction of Neural Networks**." In 29th USENIX Security Symposium (USENIX Security 20) ([link](https://www.usenix.org/conference/usenixsecurity20/presentation/jagielski)) +- Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh, Nicolas Papernot, and Mohit Iyyer (2020). **Thieves on Sesame Street! Model Extraction of BERT-based APIs**. In International Conference on Learning Representations. ICLR ([link](https://openreview.net/attachment?id=Byl5NREFDr&name=original_pdf)) ([code](https://github.com/google-research/language/tree/master/language/bert_extraction)) \ No newline at end of file