Model Extraction & Stealing

IP Theft vs. Functional Cloning

Two related but distinct goals motivate extraction attacks. IP theft: stealing a proprietary model to deploy it without paying licensing fees, circumventing per-query API costs (models like GPT-4 cost cents per query; a clone costs nothing after the extraction investment). Functional cloning: producing a local copy of a target model primarily to enable further attacks — a cloned model provides white-box access, enabling adversarial example generation, membership inference, and defense bypass that would be impossible against the black-box original.

Cloud ML API Threat Model

MLaaS (Machine Learning as a Service) platforms — AWS SageMaker, Azure ML, Google Vertex AI, OpenAI API — expose model inference endpoints to paying customers. From an attacker's perspective, the API is a perfect oracle: it returns predictions for any input, often with confidence scores. The attacker's only constraints are query rate limits, query cost, and potentially detection. A typical extraction attack queries the target API, uses the input-output pairs as training data for a surrogate model, and iterates until the surrogate achieves acceptable fidelity.

Financial & Strategic Impact

A state-of-the-art LLM like GPT-4 reportedly cost over $100 million to train. Even smaller domain-specific models represent significant R&D investment. Model extraction can obtain functional equivalents for the cost of API queries — potentially millions of queries at fractions of a cent each, totaling thousands of dollars vs. millions to train from scratch. Beyond IP value, financial fraud detection models, credit scoring systems, and security classifiers are targets because knowing the model enables systematic evasion of the defenses it implements.

Equation-Solving Attacks

For simple models (linear classifiers, logistic regression, shallow decision trees), an attacker can analytically solve for model parameters. A linear model with d features has d+1 parameters; d+1 carefully chosen queries whose decision boundaries are known can exactly recover the model via linear algebra. Lowd & Meek (2005) showed exact reconstruction of linear SVMs. While impractical for deep neural networks, these attacks remain relevant for "simple model" API wrappers used in fraud detection and credit scoring.

Decision Boundary Attacks

For neural networks, boundary attacks reconstruct the model by finding where the model's prediction changes class. The attacker samples many inputs near estimated decision boundaries — regions that provide the most information about the model's internal structure. The surrogate is trained to match these boundary locations. Boundary attacks are query-intensive but work against hard-label APIs (only class prediction, no confidence scores), which are a common API hardening technique.

KnockoffNets & Active Learning

Orekondy et al. (2019) introduced KnockoffNets: train a surrogate model using samples from a natural image distribution, query the target API for labels, and train on the resulting dataset. Active learning strategies maximize information per query by selecting inputs that lie near the decision boundary or where the surrogate is most uncertain. Jacobian-based Dataset Augmentation (Papernot et al., 2017) uses the surrogate's Jacobian (gradient approximation) to synthesize informative training examples without querying the original training distribution.

Shadow Model Attack

Shokri et al. (2017) introduced the foundational attack. The adversary trains multiple "shadow models" on datasets drawn from the same distribution as the target's training set. For each shadow model, they know which examples were in-training and which were out. A meta-classifier is trained on (model confidence, in/out) pairs from shadow models. Applied to the target model, the meta-classifier predicts membership from confidence scores. Effective when the target model significantly overfits its training data.

LiRA: Likelihood Ratio Attack

Carlini et al. (2022) proposed LiRA as the state-of-the-art membership inference attack. It trains shadow models with and without the target example, then computes the likelihood ratio of the target model's output under both hypotheses. LiRA achieves near-perfect TPR at <0.1% FPR on strongly overfit models and outperforms all prior attacks significantly. The attack requires training multiple shadow models, making it computationally expensive but extremely accurate. LiRA is now the standard benchmark for evaluating differential privacy defenses.

Threshold-Based Inference

Simple threshold attacks exploit the observation that training examples typically receive higher confidence scores than non-training examples. The attacker sets a threshold on the target model's confidence for the correct class: examples above the threshold are classified as "members." While not as accurate as LiRA at low FPR, threshold attacks require zero shadow models — just a single query per example. They're effective against poorly regularized models and are the most practical for attackers with limited compute budgets.

Backdoor-Based Watermarks

Adi et al. (2018) proposed embedding watermarks as backdoors: the owner trains the model with a set of secret "key" inputs that always produce a specific output. The model behaves normally on all other inputs. To verify ownership, the owner queries the suspected stolen model with the secret keys — consistent correct outputs indicate the watermark is present. Implementations include: specific trigger phrases for text classifiers, special images for vision models, and specific API call sequences for LLMs.

Radioactive Data & DAWN

Radioactive data (Maini et al., 2021) marks training examples with imperceptible perturbations that cause the trained model to behave distinctively on specially crafted test queries — the "radiation" transfers from data to model. DAWN (Dataset Watermarking) embeds watermarks directly into the training dataset rather than the model; any model trained on the dataset inherits the watermark. Fingerprinting via decision boundary characteristics exploits the fact that two models with identical decision boundaries are almost certainly identical models — boundary-based fingerprinting queries both models near class boundaries to verify they match.

Limitations of Watermarking

• Model stealing removes backdoor watermarks: A surrogate trained to mimic the model's clean-data behavior won't replicate the backdoor behavior on the trigger keys
• Adaptive stripping attacks: An attacker who knows the watermarking scheme can train the surrogate to replicate all outputs except those that look like watermark queries
• Fine-tuning removal: Light fine-tuning on new data can overwrite backdoor-based watermarks while preserving most model capability
• Legal burden: Watermarks provide technical evidence but proving IP theft in court requires legal frameworks that are still developing in most jurisdictions

Rate Limiting & Query Monitoring

Track per-user query volumes and flag anomalously high rates. Extraction attacks require large numbers of diverse queries — statistical analysis of query distributions can distinguish legitimate use from extraction attempts. Features that indicate extraction: systematic input variation, queries near apparent decision boundaries, very diverse input distributions, unusual input formats. Combine rate limits with soft throttling (introduce increasing latency as query rate grows) to frustrate extraction attempts without hard blocking legitimate high-volume users.

Output Perturbation

Add noise to model outputs — returning "90.3% confidence" as "85–95%" or rounding confidence values — degrades the information available to the extraction algorithm. Returning top-k labels only (without scores) is even stronger. The tradeoff: every bit of output information removed slightly degrades legitimate API users' experience and downstream accuracy. Differential privacy at inference time provides formal guarantees on information leakage per query but requires careful privacy budget management.

Differential Privacy During Training

Training with DP-SGD (differentially private stochastic gradient descent) bounds the information about any individual training example in the model's weights. By extension, this limits how much information per query an adversary can extract about the model structure. DP-trained models are harder to extract to high fidelity because the model itself contains less precise information. The privacy-utility tradeoff means DP models sacrifice some accuracy — but for high-value proprietary models, the IP protection may justify the cost.