Awesome atacks on ML privacy

Table of Contents

• Surveys
• Papers

Surveys

Papers and Code

Membership inference

• Shokri, R., Stronati, M., Song, C., and Shmatikov, V. (2017). Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP),pages 3–18. IEEE (link) (code)
• Yeom, S., Giacomelli, I., Fredrikson, M., and Jha, S. (2018). Privacy risk in machine learning:Analyzing the connection to overfitting. In 2018 IEEE 31st Computer Security FoundationsSymposium (CSF), pages 268–282. IEEE (link) (code)
• Nasr, M., Shokri, R., and Houmansadr, A. (2019). Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE Symposium on Security and Privacy (SP), pages 739–753. IEEE (link) (code)
• Hayes, J., Melis, L., Danezis, G., and De Cristofaro, E. (2019). Logan: Membership inference attacks against generative models. Proceedings on Privacy Enhancing Technologies,2019(1):133–152 (link) (code)
• Jayaraman, B. and Evans, D., 2019. Evaluating differentially private machine learning in practice. In 28th USENIX Security Symposium USENIX Security 19) (pp. 1895-1912).
• Rahman, M. A., Rahman, T., Laganière, R., Mohammed, N., and Wang, Y. (2018). Membership inference attack against differentially private deep learning model.Transactionson Data Privacy, 11(1):61–79 (link)
• Salem, A., Zhang, Y., Humbert, M., Berrang, P., Fritz, M., and Backes, M. (2019). Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. In 26th Annual Network and Distributed System Security Symposium,NDSS 2019, San Diego, California, USA, February 24-27, 2019 (link) (code)
• Song, L., Shokri, R., and Mittal, P. (2019). Privacy risks of securing machine learning models against adversarial examples. In Proceedings of the 2019 ACM SIGSAC Conference onComputer and Communications Security, CCS ’19, page 241–257, New York, NY, USA.Association for Computing Machinery. (link) (code)
• Sablayrolles, A., Douze, M., Schmid, C., Ollivier, Y. and Jegou, H.,(2019), May. White-box vs Black-box: Bayes Optimal Strategies for Membership Inference. In International Conference on Machine Learning (pp. 5558-5567). (link) (link)
• Shokri, R., Strobel, M., and Zick, Y. (2019). Privacy risks of explaining machine learning models.arXiv preprint arXiv:1907.00164 (link)
• Truex, S., Liu, L., Gursoy, M.E., Yu, L. and Wei, W. (2019). Demystifying membership inference attacks in machine learning as a service. IEEE Transactions on Services Computing. (link)
• Chen, D., Yu, N., Zhang, Y. and Fritz, M., 2019. Gan-leaks: A taxonomy of membership inference attacks against gans. arXiv preprint arXiv:1909.03935. (link)
• Hilprecht, B., Härterich, M. and Bernau, D., 2019. Monte carlo and reconstruction membership inference attacks against generative models. Proceedings on Privacy Enhancing Technologies, 2019(4), pp.232-249. (link)
• Jia, J., Salem, A., Backes, M., Zhang, Y. and Gong, N.Z., 2019, November. MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (pp. 259-274). (link) (code)
• Hisamoto, Sorami, Matt Post, and Kevin Duh. "Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?" Transactions of the Association for Computational Linguistics 8 (2020): 49-63. (link)
• Long, Y., Bindschaedler, V., Wang, L., Bu, D., Wang, X., Tang, H., Chen, K. (2018). Understanding membership inferences on well-generalized learning models. arXiv preprint arXiv:1802.04889. (link)

Reconstruction

Reconstruction attacks cover also attacks known as model inversion and attribute inference.

• Fredrikson, M., Lantz, E., Jha, S., Lin, S., Page, D., and Ristenpart, T. (2014). Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing. In 23rd USENIX Security Symposium (USENIX Security 14), pages 17–32, San Diego, CA.USENIX Association (link)
• Fredrikson, M., Jha, S., and Ristenpart, T. (2015). Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1322–1333. ACM (link) (code)
• Wu, X., Fredrikson, M., Jha, S. and Naughton, J.F., 2016, June. A methodology for formalizing model-inversion attacks. In 2016 IEEE 29th Computer Security Foundations Symposium (CSF) (pp. 355-370). IEEE. (link)
• Hitaj, B., Ateniese, G., and Perez-Cruz, F. (2017). Deep models under the gan: Information leakage from collaborative deep learning. InProceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS ’17, page 603–618, New York,NY, USA. Association for Computing Machinery (link)
• Song, C., Ristenpart, T. and Shmatikov, V., 2017, October. Machine learning models that remember too much. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (pp. 587-601). (link) (code)
• Hidano, S., Murakami, T., Katsumata, S., Kiyomoto, S., & Hanaoka, G. (2017, August). Model inversion attacks for prediction systems: Without knowledge of non-sensitive attributes. In 2017 15th Annual Conference on Privacy, Security and Trust (PST) (pp. 115-11509). IEEE. (link)
• Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., and Song, D. (2019). The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX Security Symposium (USENIX Security 19), pages 267–284, Santa Clara, CA. USENIX Association (link)
• Zhu, L., Liu, Z., , and Han, S. (2019). Deep leakage from gradients. In Annual Conferenceon Neural Information Processing Systems (NeurIPS). (link) (code)
• He, Z., Zhang, T. and Lee, R.B., 2019, December. Model inversion attacks against collaborative inference. In Proceedings of the 35th Annual Computer Security Applications Conference (pp. 148-162). (link) (code)
• Z. Wang, M. Song, Z. Zhang, Y. Song, Q. Wang and H. Qi, "Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning," IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, Paris, France, 2019, pp. 2512-2520. (link)
• Zhao, B., Mopuri, K. R., & Bilen, H. (2020). iDLG: Improved Deep Leakage from Gradients. arXiv preprint arXiv:2001.02610. (link) (code)
• Pan, X., Zhang, M., Ji, S., & Yang, M. (2020) Privacy Risks of General-Purpose Language Models. (link)
• Yang, Z., Zhang, J., Chang, E. C., & Liang, Z. (2019, November). Neural network inversion in adversarial setting via background knowledge alignment. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (pp. 225-240). (link)
• Zhang, Y., Jia, R., Pei, H., Wang, W., Li, B., & Song, D. (2020). The secret revealer: generative model-inversion attacks against deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 253-261). (link) (link)
• Geiping, Jonas, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. "Inverting Gradients - How easy is it to break privacy in federated learning?." arXiv preprint arXiv:2003.14053 (2020). (link)

Property inference

• Ateniese, G., Mancini, L. V., Spognardi, A., Villani, A., Vitali, D., and Felici, G. (2015). Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers. International Journal of Security and Networks, 10(3):137–150. (link)
• Ganju, K., Wang, Q., Yang, W., Gunter, C. A., and Borisov, N. (2018). Property inference attacks on fully connected neural networks using permutation invariant representations. InProceedings of the 2018 ACM SIGSAC Conference on Computer and CommunicationsSecurity, pages 619–633. ACM (link)
• Melis, L., Song, C., De Cristofaro, E., and Shmatikov, V. (2019). Exploiting unintended feature leakage in collaborative learning. In 2019 IEEE Symposium on Security and Privacy(SP), pages 691–706. IEEE (link) (code)
• Congzheng Song and Vitaly Shmatikov (2020). Overlearning Reveals Sensitive Attributes. In International Conference on Learning Representations. ICLR (link) (code)

Model extraction

• Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., and Ristenpart, T. (2016). Stealing machine learning models via prediction apis. In 25th USENIX Security Symposium (USENIX Security 16), pages 601–618, Austin, TX. USENIX Association (link) (code)
• Wang, B. and Gong, N. Z. (2018). Stealing hyperparameters in machine learning. In 2018 IEEE Symposium on Security and Privacy (SP), pages 36–52. IEEE. (link)
• J. R. Correia-Silva, R. F. Berriel, C. Badue, A. F. de Souza and T. Oliveira-Santos, (2018). Copycat CNN: Stealing Knowledge by Persuading Confession with Random Non-Labeled Data In International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, 2018, pp. 1-8, doi: 10.1109/IJCNN.2018.8489592. (link) (code)
• Oh, S. J., Schiele, B., and Fritz, M. (2019). Towards reverse-engineering black-box neural networks. In Sixth InternationalConference on Learning Representations. ICLR, Vancouver, Canada (link) (code)
• Orekondy, T., Schiele, B. and Fritz, M., 2019. Knockoff nets: Stealing functionality of black-box models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4954-4963). (link) (code)
• Juuti, M., Szyller, S., Marchal, S. and Asokan, N., 2019, June. PRADA: protecting against DNN model stealing attacks. In 2019 IEEE European Symposium on Security and Privacy (EuroS&P) (pp. 512-527). IEEE. (link) (code)
• Smitha Milli, Ludwig Schmidt, Anca D. Dragan, and Moritz Hardt. 2019. Model Reconstruction from Model Explanations. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* ’19). Association for Computing Machinery, New York, NY, USA, 1–9. (link)
• Chandrasekaran, Varun, Kamalika Chaudhuri, Irene Giacomelli, Somesh Jha, and Songbai Yan. "Exploring connections between active learning and model extraction." Usenix 2020 (link)
• Jagielski, Matthew, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot. (2020) "High Accuracy and High Fidelity Extraction of Neural Networks." In 29th USENIX Security Symposium (USENIX Security 20) (link)
• Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh, Nicolas Papernot, and Mohit Iyyer (2020). Thieves on Sesame Street! Model Extraction of BERT-based APIs. In International Conference on Learning Representations. ICLR (link) (code)