Code obfuscation analysis and deobfuscation playbook. Use when reversing binaries protected by junk code, opaque predicates, self-modifying code, control flow flattening, VM protection, or string encryption.

SKILL: Code Obfuscation & Deobfuscation — Expert Analysis Playbook AI LOAD INSTRUCTION : Expert techniques for identifying, classifying, and defeating code obfuscation in native binaries. Covers junk code, opaque predicates, SMC, control flow flattening, movfuscator, VM protectors (VMProtect/Themida/Code Virtualizer), string encryption, import hiding, and anti disassembly tricks. Base models often conflate packing with obfuscation and miss the distinction between static and dynamic deobfuscation strategies. 0. RELATED ROUTING [anti debugging techniques](../anti debugging techniques/SKILL.md) when the obfuscated binary also has anti debug layers [symbolic execution tools](../symbolic execution tools/SKILL.md) when using angr/Z3 for automated deobfuscation [vm and bytecode reverse](../vm and bytecode reverse/SKILL.md) for deep VM protector bytecode analysis Quick identification picks Symptom in IDA/Ghidra Likely Obfuscation Start With Flat CFG, single giant switch Control flow flattening Symbolic execution to recover CFG Only mov instructions movfuscator demovfuscation / trace based lifting pushad/pushfd → VM entry VM protector Handler table extraction XOR loop before code execution SMC / string encryption Dynamic analysis, breakpoint after decode Impossible conditions (opaque predicates) Junk code insertion Pattern based removal All strings unreadable String encryption Hook decryption routine, or emulate No imports in IAT Import hiding Trace GetProcAddress / hash resolution 1. JUNK CODE & OPAQUE PREDICATES 1.1 Junk Code Insertion Dead code that never affects program output, added to increase analysis time. Identification : Instructions that write to registers/memory never read afterward Function calls whose return values are discarded and have no side effects Loops with invariant bounds that compute unused results Removal strategy : 1. Compute def use chains (IDA/Ghidra data flow analysis) 2. Mark instructions with no downstream use as dead 3. Verify removal doesn't change program behavior (trace comparison) 1.2 Opaque Predicates Conditional branches where the condition is always true or always false, but this is non obvious. Type Example Always Evaluates To Arithmetic x² ≥ 0 True Number theory x (x+1) % 2 == 0 True (product of consecutive ints) Pointer based ptr == ptr after aliasing True Hash based CRC32(constant) == known value True Deobfuscation : Abstract interpretation: prove the condition is constant Symbolic execution: Z3 proves ∀x: predicate(x) = True Pattern matching: recognize known opaque predicate families Dynamic: trace and observe the branch is never taken / always taken 2. SELF MODIFYING CODE (SMC) Runtime code patching: encrypted code is decrypted just before execution. 2.1 XOR Decryption Loop (Most Common) 2.2 Analysis Strategy 2.3 Automated Unpacking via Emulation 3. CONTROL FLOW FLATTENING (CFF) 3.1 Structure Original sequential blocks are transformed into a dispatcher loop: Each block sets state = next state before jumping back to the dispatcher. 3.2 Recovery Techniques Technique Tool Effectiveness Symbolic execution angr, Triton, miasm High — traces all state transitions Trace based recovery Pin/DynamoRIO trace → reconstruct CFG Medium — covers executed paths only Pattern matching Custom IDA/Ghidra script Medium — works for known flatteners D 810 (IDA plugin) IDA Pro High — specifically designed for CFF 3.3 Symbolic Deflattening (angr approach) 4. MOVFUSCATOR 4.1 Concept All computation reduced to mov instructions only (Turing complete via memory mapped computation tables). Created by Christopher Domas. 4.2 Identification Function contains only mov instructions (no add, sub, xor, jmp, call) Large lookup tables in data section Memory mapped flag registers 4.3 Demovfuscation Approach Description demovfuscator (tool) Static analysis, recovers original operations from mov patterns Trace + taint analysis Run with Pin/DynamoRIO, taint inputs, observe computation Symbolic execution Treat entire function as constraint system 5. VM PROTECTION (VMProtect / Themida / Code Virtualizer) 5.1 VM Architecture 5.2 VM Entry Point Identification 5.3 Handler Table Extraction 5.4 Devirtualization Approaches Method Description Tool Manual handler mapping Reverse each handler, build ISA spec IDA + scripting Trace recording Record all handler executions, reconstruct program REVEN, Pin Symbolic lifting Symbolically execute handlers, lift to IR Triton, miasm Pattern matching Match handler patterns to known VM families Custom scripts 5.5 VMProtect Specifics Uses opaque predicates in dispatcher Handler mutation: same opcode, different handler code per build Multiple VM layers (VM inside VM) Integrates anti debug and integrity checks 6. STRING ENCRYPTION 6.1 Common Patterns Pattern Example Recovery XOR loop for (i=0; i<len; i++) s[i] ^= key; Hook or emulate XOR function Stack strings mov [esp+0], 'H'; mov [esp+1], 'e'; ... IDA FLIRT / Ghidra script to reassemble RC4 encrypted Encrypted blob + RC4 key in binary Extract key, decrypt offline AES encrypted Encrypted blob + AES key derived at runtime Hook after decryption Custom encoding Base64 + XOR + reverse Trace the decode function, replicate 6.2 Automated String Decryption 7. IMPORT HIDING 7.1 GetProcAddress + Hash Lookup 7.2 Recovery 1. Identify the hash algorithm (common: CRC32, djb2, ROR13+ADD) 2. Compute hashes for all known API names 3. Build hash → API name lookup table 4. Annotate resolved calls in IDA/Ghidra 7.3 Common Hash Algorithms Name Algorithm Used By ROR13 hash = (hash 13 \ hash << 19) + char Metasploit shellcode djb2 hash = hash 33 + char Various malware CRC32 Standard CRC32 of function name Sophisticated packers FNV 1a hash = (hash ^ char) 0x01000193 Modern malware 8. ANTI DISASSEMBLY TRICKS 8.1 Techniques Trick Mechanism Fix Overlapping instructions jmp $+2; db 0xE8 (fake call prefix) Manual re analysis from correct offset Misaligned jumps Jump into middle of multi byte instruction Force IDA to re analyze at target Conditional jump pair jz $+5; jnz $+3 (always jumps, confuses linear disasm) Convert to unconditional jmp Return address manipulation push addr; ret instead of jmp addr Recognize push+ret as jump Exception based flow Trigger exception, real code in handler Analyze exception handler chain Call + add [esp] call $+5; add [esp], N; ret (computed jump) Calculate actual target 8.2 IDA Fixes 9. DECISION TREE 10. TOOLBOX Tool Purpose Best For IDA Pro + Hex Rays Disassembly, decompilation, scripting All around analysis Ghidra Free alternative with scripting (Java/Python) Budget friendly RE D 810 (IDA plugin) Automated CFF deflattening OLLVM style obfuscation miasm IR based analysis framework Symbolic deobfuscation Triton Dynamic symbolic execution Opaque predicate solving, CFF REVEN Full system trace recording and replay VM protector analysis demovfuscator movfuscator reversal mov only binaries x64dbg + plugins Dynamic analysis with scripting Windows RE Unicorn Engine CPU emulation SMC unpacking, shellcode Capstone Disassembly library Custom tooling IDA FLIRT Function signature matching Identify library code in stripped binaries Binary Ninja Alternative disassembler with MLIL/HLIL Automated analysis

code-obfuscation-deobfuscation