Awesome-model-inversion-attack

A curated list of resources for model inversion attack (MIA). If some related papers are missing, please contact us via pull requests.

Outlines:

• What is the model inversion attack?
• Survey
• Computer vision domain
  • TODO
• Graph learning domain
  • TODO
• Natural language processing domain
• Tools
• Others
• Related repositories

What is the model inversion attack?

A model inversion attack is a privacy attack where the attacker is able to reconstruct the original samples that were used to train the synthetic model from the generated synthetic data set. (Mostly.ai)

The goal of model inversion attacks is to recreate training data or sensitive attributes. (Chen et al, 2021.)

In model inversion attacks, a malicious user attempts to recover the private dataset used to train a supervised neural network. A successful model inversion attack should generate realistic and diverse samples that accurately describe each of the classes in the private dataset. (Wang et al, 2021.)

Survey

Arxiv 2021 - A Survey of Privacy Attacks in Machine Learning. [paper]

Arxiv 2022 - A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability. [paper]

Arxiv 2022 - Trustworthy Graph Neural Networks: Aspects, Methods and Trends. [paper]

Arxiv 2022 - A Survey of Trustworthy Graph Learning: Reliability, Explainability, and Privacy Protection. [paper]

Philosophical Transactions of the Royal Society A 2018. Algorithms that remember: model inversion attacks and data protection law. [paper]

(Rigaki and Garcia, 2020) A Survey of Privacy Attacks in Machine Learning [paper]

(De Cristofaro, 2020) An Overview of Privacy in Machine Learning [paper]

(Fan et al., 2020) Rethinking Privacy Preserving Deep Learning: How to Evaluate and Thwart Privacy Attacks [paper]

(Liu et al., 2021) Privacy and Security Issues in Deep Learning: A Survey [paper]

(Liu et al., 2021) ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models [paper]

(Hu et al., 2021) Membership Inference Attacks on Machine Learning: A Survey [paper]

(Jegorova et al., 2021) Survey: Leakage and Privacy at Inference Time [paper]

(Joud et al., 2021) A Review of Confidentiality Threats Against Embedded Neural Network Models [paper]

(Wainakh et al., 2021) Federated Learning Attacks Revisited: A Critical Discussion of Gaps, Assumptions, and Evaluation Setups [paper]

(Oliynyk et al., 2022) I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences [paper]

Computer vision domain

Year	Title	Adversarial Knowledge	Venue	Paper Link	Code Link
2014	Privacy in Pharmacogenetics: An End-to-End Case Study of Personalized Warfarin Dosing	white-box	USENIX Security	paper
2015	Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures	white-box	CCS	paper	code1, code2, code3, code4
2015	Regression model fitting under differential privacy and model inversion attack	white-box	IJCAI	paper	code
2016	A Methodology for Formalizing Model-Inversion Attacks	white-box	CSF	paper
2017	Machine Learning Models that Remember Too Much	white-box	CCS	paper	code
2017	Model inversion attacks for prediction systems: Without knowledge of non-sensitive attributes	white-box	PST	paper
2018	Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting	white-box	CSF	paper
2019	MLPrivacyGuard: Defeating Confidence Information based Model Inversion Attacks on Machine Learning Systems	white-box	GLSVLSI	paper
2019	Model inversion attacks against collaborative inference	white-box
2019	Neural Network Inversion in Adversarial Setting via Background Knowledge Alignment	-	CCS	Paper	Code
2019	Exploiting Unintended Feature Leakage in Collaborative Learning	-	IEEE S&P	Paper	Code
2019	Adversarial Neural Network Inversion via Auxiliary Knowledge Alignment	-	Arxiv	Paper	-
2019	GAMIN: An Adversarial Approach to Black-Box Model Inversion	-	Arxiv	Paper	-
2020	The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks	-	CVPR	Paper	Code
2020	OVERLEARNING REVEALS SENSITIVE ATTRIBUTES	-	ICLR	Paper	-
2020	Deep Face Recognizer Privacy Attack: Model Inversion Initialization by a Deep Generative Adversarial Data Space Discriminator	-	APSIPA ASC	Paper	-
2020	Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning	-	USENIX Security	Paper	-
2020	Attacking and Protecting Data Privacy in Edge-Cloud Collaborative Inference Systems	-	IoTJ	Paper	Code
2020	Black-Box Face Recovery from Identity Features	-	ECCV Workshop	Paper	-
2020	Defending model inversion and membership inference attacks via prediction purification	-	Arxiv	Paper	-
2020	Generative model-inversion attacks against deep neural networks	-	CVPR	Paper	Code
2020	Privacy Preserving Facial Recognition Against Model Inversion Attacks	-	Globecom	Paper	-
2020	Broadening Differential Privacy for Deep Learning Against Model Inversion Attacks	-	Big Data	Paper	-
2021	Contrastive Model Inversion for Data-Free Knowledge Distillation	-	IJCAI	Paper	Code
2021	Membership Leakage in Label-Only Exposures	-	CCS	Paper	Code
2021	Black-box adversarial attacks on commercial speech platforms with minimal information	-	CCS	Paper	-
2021	Unleashing the tiger: Inference attacks on split learning	-	CCS	Paper	Code
2021	See through gradients: Image batch recovery via gradinversion	-	CVPR	Paper	-
2021	Soteria: Provable defense against privacy leakage in federated learning from representation perspective	-	CVPR	Paper	Code
2021	Imagine: Image synthesis by image-guided model inversion	-	CVPR	Paper	-
2021	Variational Model Inversion Attacks	-	NeurIPS	Paper	Code
2021	Exploiting Explanations for Model Inversion Attacks	-	ICCV	Paper	-
2021	Knowledge-Enriched Distributional Model Inversion Attacks	-	ICCV	Paper	Code
2021	Improving Robustness to Model Inversion Attacks via Mutual Information Regularization	-	AAAI	Paper	-
2021	Label-Only Membership Inference Attack	-	ICML	Paper	Code
2021	When Does Data Augmentation Help With Membership Inference Attacks?	-	ICML	Paper	-
2021	PRACTICAL DEFENCES AGAINST MODEL INVERSION ATTACKS FOR SPLIT NEURAL NETWORKS	-	ICLR workshop	Paper	Code
2021	Feature inference attack on model predictions in vertical federated learning	-	ICDE	Paper	Code
2021	PRID: Model Inversion Privacy Attacks in Hyperdimensional Learning Systems	-	DAC	Paper	-
2021	Robustness of on-device models: Adversarial attack to deep learning models on android apps	-	ICSE	Paper	-
2021	Defending Against Model Inversion Attack by Adversarial Examples	-	CSR Workshops	Paper	-
2021	Practical Black Box Model Inversion Attacks Against Neural Nets	-	ECML PKDD	Paper	-
2021	Model Inversion Attack against a Face Recognition System in a Black-Box Setting	-	APSIPA	Paper	-
2022	Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks	-	ICML	Paper	Code
2022	Label-Only Model Inversion Attacks via Boundary Repulsion	-	CVPR	Paper	Code
2022	ResSFL: A Resistance Transfer Framework for Defending Model Inversion Attack in Split Federated Learning	-	CVPR	Paper	Code
2022	Bilateral Dependency Optimization: Defending Against Model-inversion Attacks	-	KDD	Paper	Code
2022	ML-DOCTOR: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models	-	USENIX Security	Paper	Code
2022	An Approximate Memory based Defense against Model Inversion Attacks to Neural Networks	-	IEEE	Paper	Code
2022	Model Inversion Attack by Integration of Deep Generative Models: Privacy-Sensitive Face Generation From a Face Recognition System	-	TIFS	Paper	-
2022	Defending against Reconstruction Attacks through Differentially Private Federated Learning for Classification of Heterogeneous Chest X-Ray Data	-	Arxiv	Paper	-
2022	One Parameter Defense—Defending Against Data Inference Attacks via Differential Privacy	black-box	TIFS	Paper
2022	Reconstructing Training Data from Diverse ML Models by Ensemble Inversion	white-box	WACV	Paper
2022	SecretGen: Privacy Recovery on Pre-trained Models via Distribution Discrimination	white-box	ECCV	Paper
2022	UnSplit: Data-Oblivious Model Inversion, Model Stealing, andLabel Inference Attacks Against Split Learning	-	WPES	Paper	code
2022	MIRROR: Model Inversion for Deep LearningNetwork with High Fidelity	white-box	NDSS	Paper	code
2023	Sparse Black-Box Inversion Attack with Limited Information	black-box	ICASSP	Paper	code
2023	Breaching FedMD: Image Recovery via Paired-Logits Inversion Attack	black-box	CVPR	Paper	code
2023	Pseudo Label-Guided Model Inversion Attack via Conditional Generative Adversarial Network	white-box	AAAI	Paper	code
2023	C2FMI: Corse-to-Fine Black-box Model Inversion Attack	black-box	TDSC	Paper
2023	Boosting Model Inversion Attacks with Adversarial Examples	black-box	TDSC	Paper
2023	Reinforcement Learning-Based Black-Box Model Inversion Attacks	black-box	CVPR	Paper	code
2023	Re-thinking Model Inversion Attacks Against Deep Neural Networks	white-box	CVPR	Paper	code

TODO

Year	Title	Adversarial Knowledge	Venue	Paper Link	Code Link
2019	The secret sharer: Evaluating and testing unintended memorization in neural networks	white-box	USENIX	Paper
2019	Deep leakage from gradients	white-box	NIPS	Paper	code
2020	Inverting Gradients - How easy is it to break privacy in federated learning?	white-box	NIPS	Paper
2018	Reconstruction of training samples from loss functions	-	arXiv	Paper
2022	Reconstructing Training Data from Trained Neural Networks	white-box	NIPS	Paper
2020	A Framework for Evaluating Gradient Leakage Attacks in Federated Learning	white-box	CoRR	Paper
2017	Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning	both	CCS	Paper
2019	Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning	white-box	IEEE INFOCOM	Paper
2020	Evaluation Indicator for Model Inversion Attack	metric	AdvML	Paper
-	An Attack-Based Evaluation Method for Differentially Private Learning Against Model Inversion Attack	white-box	arXiv	Paper
2020	SAPAG: A Self-Adaptive Privacy Attack From Gradients	white-box	arXiv	Paper
2022	Exploring the Security Boundary of Data Reconstruction via Neuron Exclusivity Analysis	white-box	USENIX	Paper
2020	Knowledge-Enriched Distributional Model Inversion Attacks	white-box	arXiv	Paper
2020	MixCon: Adjusting the Separability of Data Representations for Harder Data Recovery	white-box	arXiv	Paper
2020	Evaluation of Inference Attack Models for Deep Learning on Medical Data	black-box	arXiv	Paper
2020	FaceLeaks: Inference Attacks against Transfer Learning Models via Black-box Queries	black-box	arXiv	Paper
-	Derivation of Constraints from Machine Learning Models and Applications to Security and Privacy	Theory	A	Paper
2021	On the (In)Feasibility of Attribute Inference Attacks on Machine Learning Models	both	IEEE EuroS&P	Paper
2021	R-GAP: Recursive Gradient Attack on Privacy	white-box	ICLR	Paper
2021	PRECODE - A Generic Model Extension to Prevent Deep Gradient Leakage	white-box	arXiv	Paper
2021	On the Importance of Encrypting Deep Features	black-box	arXiv	Paper
2022	Reconstructing Training Data with Informed Adversaries	white-box	arXiv	Paper
2022	Privacy Vulnerability of Split Computing to Data-Free Model Inversion Attacks	white-box	arXiv	Paper

Graph learning domain

Year	Title	Adversarial Knowledge	Venue	Paper Link	Code Link
2020	Stealing Links from Graph Neural Networks	-	USENIX Security	Paper	Code
2020	Improving Robustness to Model Inversion Attacks via Mutual Information Regularization	black & white-box	AAAI	Paper
2020	Reducing Risk of Model Inversion Using Privacy-Guided Training	black & white-box	Arxiv	Paper
2020	Quantifying Privacy Leakage in Graph Embedding	-	MobiQuitous	Paper	Code
2021	A Survey on Gradient Inversion: Attacks, Defenses and Future Directions	white-box	IJCAI	Paper
2021	NetFense: Adversarial Defenses against Privacy Attacks on Neural Networks for Graph Data	black-box	ICDE	Paper	code
2021	DeepWalking Backwards: From Node Embeddings Back to Graphs	-	ICML	Paper	Code
2021	GraphMI: Extracting Private Graph Data from Graph Neural Networks	white-box	IJCAI	Paper	code
2021	Node-Level Membership Inference Attacks Against Graph Neural Networks	-	Arxiv	Paper	-
2022	A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability	black & white-box	Arxiv	Paper
2022	Learning Privacy-Preserving Graph Convolutional Network with Partially Observed Sensitive Attributes	-	WWW	Paper	-
2022	Inference Attacks Against Graph Neural Networks	-	USENIX Security	Paper	Code
2022	Model Stealing Attacks Against Inductive Graph Neural Networks	-	IEEE S&P	Paper	Code
2022	DIFFERENTIALLY PRIVATE GRAPH CLASSIFICATION WITH GNNS	-	Arxiv	Paper	-
2022	GAP: Differentially Private Graph Neural Networks with Aggregation Perturbation	-	Arxiv	Paper	-
2022	SOK: DIFFERENTIAL PRIVACY ON GRAPH-STRUCTURED DATA	-	Arxiv	Paper	-
2022	Degree-Preserving Randomized Response for Graph Neural Networks under Local Differential Privacy	-	Arxiv	Paper	-
2022	Private Graph Extraction via Feature Explanations	-	Arxiv	Paper	-
2022	Privacy and Transparency in Graph Machine Learning: A Unified Perspective	-	Arxiv	Paper	-
2022	Finding MNEMON: Reviving Memories of Node Embeddings	-	CCS	Paper	-
2022	Defense against membership inference attack in graph neural networks through graph perturbation	-	IJIS	Paper	-
2022	Model Inversion Attacks against Graph Neural Networks	-	TKDE	Paper	-
2023	On Strengthening and Defending Graph Reconstruction Attack with Markov Chain Approximation	-	ICML
Paper	Code

TODO

Year	Title	Venue	Paper Link	Code Link
2021	Membership Inference Attack on Graph Neural Networks	International Conference on Trust, Privacy and Security in Intelligent Systems and Applications	Paper	-
2020	Model Extraction Attacks on Graph Neural Networks: Taxonomy and Realisation	ACM Asia Conference on Computer and Communications Security	Paper	-
2021	Adapting Membership Inference Attacks to GNN for Graph Classification: Approaches and Implications	Industrial Conference on Data Mining	Paper	-
2020	Locally Private Graph Neural Networks	Conference on Computer and Communications Security	Paper	-
2020	Backdoor Attacks to Graph Neural Networks	ACM Symposium on Access Control Models and Technologies	Paper	-
2019	Attacking Graph-based Classification via Manipulating the Graph Structure	Conference on Computer and Communications Security	Paper	-
2021	Private Graph Data Release: A Survey	ACM Computing Surveys	Paper
2022	Differentially Private Graph Neural Networks for Whole-Graph Classification	IEEE Transactions on Pattern Analysis and Machine Intelligence	Paper	-
----	Node-Differentially Private Estimation of the Number of Connected Components	arXiv	Paper	-
2022	LPGNet: Link Private Graph Networks for Node Classification	Conference on Computer and Communications Security	Paper	-
----	Releasing Graph Neural Networks with Differential Privacy Guarantees	arXiv	Paper	-
2021	DPGraph: A Benchmark Platform for Differentially Private Graph Analysis	SIGMOD Conference	Paper	-
2015	Private Release of Graph Statistics using Ladder Functions	SIGMOD Conference	Paper	-
2021	LINKTELLER: Recovering Private Edges from Graph Neural Networks via Influence Analysis	IEEE Symposium on Security and Privacy	Paper	-
----	GAP: Differentially Private Graph Neural Networks with Aggregation Perturbation	arXiv	Paper	-
----	GrOVe: Ownership Verification of Graph Neural Networks using Embeddings	arXiv	Paper	-

Natural language processing domain

Year	Title	Adversarial Knowledge	Venue	Paper Link	Code Link
2020	Extracting Training Data from Large Language Models	black-box	USENIX Security	Paper	code
2020	Privacy Risks of General-Purpose Language Models	black & white-box	S&P	Paper
2020	Information Leakage in Embedding Models	black & white-box	CCS	Paper
2020	KART: Privacy Leakage Framework of Language Models Pre-trained with Clinical Records	black-box	arXiv	Paper
2021	TAG: Gradient Attack on Transformer-based Language Models	white-box	EMNLP	Paper
2022	KART: Parameterization of Privacy Leakage Scenarios from Pre-trained Language Models	black-box	Arxiv	paper	code
2022	Text Revealer: Private Text Reconstruction via Model Inversion Attacks against Transformers	white-box	Arxiv	Paper
2022	Canary Extraction in Natural Language Understanding Models	white-box	ACL	paper
2022	Recovering Private Text in Federated Learning of Language Models	white-box	NeurIPS	paper	code
2023	Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence	black-box	ACL	paper	code
2023	Deconstructing Classifiers: Towards A Data Reconstruction Attack Against Text Classification Models	white-box	Arxiv	Paper

Tools

AIJack: Implementation of algorithms for AI security.

Privacy-Attacks-in-Machine-Learning: Membership Inference, Attribute Inference and Model Inversion attacks implemented using PyTorch.

ml-attack-framework: Universität des Saarlandes - Privacy Enhancing Technologies 2021 - Semester Project.

(Trail of Bits) PrivacyRaven [GitHub]

(TensorFlow) TensorFlow Privacy [GitHub]

(NUS Data Privacy and Trustworthy Machine Learning Lab) Machine Learning Privacy Meter [GitHub]

(IQT Labs/Lab 41) CypherCat (archive-only) [GitHub]

(IBM) Adversarial Robustness Toolbox (ART) [GitHub]

Others

2019 - Uncovering a model’s secrets. [blog1] [blog2]

2019 - Model Inversion Attacks Against Collaborative Inference. [slides]

2020 - Attacks against Machine Learning Privacy (Part 1): Model Inversion Attacks with the IBM-ART Framework. [blog]

2021 - ML and DP. [slides]

2023 - arXiv A Linear Reconstruction Approach for Attribute Inference Attacks against Synthetic Data [paper]

Related repositories

awesome-ml-privacy-attacks [repo]