AI/ML security playbook. Use when assessing model supply chain attacks (pickle RCE, poisoned weights), adversarial examples, model poisoning, model stealing, data privacy attacks (membership inference, model inversion), and autonomous agent security risks.

SKILL: AI/ML Security — Expert Attack Playbook AI LOAD INSTRUCTION : Expert AI/ML security techniques. Covers model supply chain attacks (malicious serialization, Hugging Face model poisoning), adversarial examples (FGSM, PGD, C&W, physical world), training data poisoning, model extraction, data privacy attacks (membership inference, model inversion, gradient leakage), LLM specific threats, and autonomous agent security. Base models underestimate the severity of pickle deserialization RCE and the practicality of black box model extraction. 0. RELATED ROUTING [llm prompt injection](../llm prompt injection/SKILL.md) for LLM specific prompt injection, jailbreaking, and tool abuse techniques [deserialization insecure](../deserialization insecure/SKILL.md) for deeper coverage of Python pickle and general deserialization attack patterns [dependency confusion](../dependency confusion/SKILL.md) when the ML pipeline has supply chain risks via pip/npm package confusion 1. MODEL SUPPLY CHAIN ATTACKS 1.1 Malicious Model Files — Pickle RCE Python's pickle module executes arbitrary code during deserialization. PyTorch .pt / .pth files use pickle by default. Loading torch.load('model.pt') executes the embedded command. Applies to: Format Risk Mitigation .pt / .pth (PyTorch) Critical — pickle by default Use torch.load(..., weights only=True) (PyTorch ≥ 2.0) .pkl / .pickle Critical — raw pickle Never load untrusted pickles .joblib High — uses pickle internally Verify provenance .npy / .npz (NumPy) Medium — allow pickle=True enables RCE Use allow pickle=False .safetensors Safe — tensor only format, no code execution Preferred format .onnx Safe — graph definition only, no arbitrary code Preferred for inference 1.2 Hugging Face Model Poisoning Detection signals: Files with .pt / .pkl extension instead of .safetensors Custom Python code in the repository ( .py files outside standard config) Unusual config.json with trust remote code=True requirement Model card lacking provenance, training data description, or eval results 1.3 Dependency Confusion in ML Pipelines ML projects often have complex dependency chains: 2. ADVERSARIAL EXAMPLES 2.1 Attack Taxonomy Attack Type Knowledge Method White box Full model access (architecture + weights) Gradient based: FGSM, PGD, C&W Black box (transfer) Access to similar model Generate adversarial on surrogate, transfer to target Black box (query) API access only Estimate gradients via finite differences or evolutionary methods Physical world Camera/sensor input Adversarial patches, glasses, modified objects 2.2 FGSM (Fast Gradient Sign Method) Single step attack. Fast but less effective against robust models: Perturbation is imperceptible to humans but changes classification. 2.3 PGD (Projected Gradient Descent) Iterative version of FGSM. Stronger but slower: 2.4 C&W (Carlini & Wagner) Optimization based. Finds minimal perturbation to cause misclassification: Most effective for targeted attacks (force specific wrong class). 2.5 Physical World Adversarial Attack Method Impact Adversarial patch Printed sticker placed on object Misclassification of physical objects Adversarial glasses Special frames with adversarial pattern Face recognition evasion/impersonation Stop sign perturbation Small stickers on road signs Autonomous vehicle misreads sign Adversarial T shirts Printed pattern on clothing Person detection evasion Audio adversarial Imperceptible audio perturbation Voice assistant command injection 3. MODEL POISONING 3.1 Training Data Poisoning Inject malicious samples into the training set to create backdoored models: 3.2 Label Flipping Systematically flip labels for a subset of training data: Strategy Effect Random flip (5 10% of labels) Degrades overall model accuracy Targeted flip (specific class) Model fails on specific category Trigger based flip Backdoor: specific pattern → wrong class 3.3 Gradient Manipulation in Federated Learning Defenses : Robust aggregation (Krum, trimmed mean, median), anomaly detection on gradient updates, differential privacy. 4. MODEL STEALING / EXTRACTION 4.1 Query Based Extraction 4.2 Side Channel Attacks on ML APIs Side Channel Information Leaked Response timing Model architecture complexity, input dependent branching Prediction confidence scores Decision boundary proximity Top K class probabilities Full softmax output → better extraction Cache timing Whether input was seen before (membership inference) Power consumption (edge devices) Weight values during inference 4.3 Knowledge Distillation from Black Box Soft labels (probability distributions) leak far more information than hard labels. 5. DATA PRIVACY ATTACKS 5.1 Membership Inference Determine whether a specific data point was used in training: Privacy implications: medical data membership → reveals patient's condition. 5.2 Model Inversion Recover approximate training data from model access: 5.3 Gradient Leakage in Federated Learning Shared gradients reveal training data: 6. LLM SPECIFIC SECURITY (Cross ref) For detailed prompt injection techniques, see [llm prompt injection](../llm prompt injection/SKILL.md). 6.1 Training Data Extraction LLMs memorize training data, especially rare or repeated sequences: 6.2 System Prompt Extraction Covered in [llm prompt injection JAILBREAK PATTERNS.md](../llm prompt injection/JAILBREAK PATTERNS.md) Section 5. 6.3 Alignment Bypass Technique Method Fine tuning attack Fine tune on small harmful dataset → removes safety training Representation engineering Modify internal representations to suppress refusal Activation patching Identify and modify "refusal" neurons/directions Quantization degradation Aggressive quantization damages safety layers more than capability Key finding : Safety alignment is often a thin layer on top of base capabilities. A few hundred fine tuning examples can remove safety training while preserving general capability. 7. AGENT SECURITY 7.1 Permission Escalation 7.2 Multi Agent Trust Issues 7.3 Tool Use Without Confirmation Risk Level Tool Category Example Critical Code execution exec() , shell commands, script runners Critical Financial Payment APIs, trading, fund transfers High Data modification Database writes, file deletion, config changes High Communication Sending emails, posting messages, API calls Medium Data access File reads, database queries, search Low Computation Math, formatting, text processing Principle : Tools with side effects should require explicit user confirmation. Read only tools can be auto approved with logging. 8. TOOLS & FRAMEWORKS Tool Purpose Adversarial Robustness Toolbox (ART) Generate and defend against adversarial examples CleverHans Adversarial example generation library Fickling Static analysis of pickle files for malicious payloads ModelScan Scan ML model files for security issues NB Defense Jupyter notebook security scanner Garak LLM vulnerability scanner (probes for prompt injection, data leakage) PyRIT (Microsoft) Red teaming framework for generative AI Rebuff Prompt injection detection framework 9. DECISION TREE

ai-ml-security