Anti-debugging detection and bypass playbook. Use when reversing protected binaries that detect debuggers via ptrace, PEB flags, timing checks, or signal/exception handlers on Linux and Windows.

SKILL: Anti Debugging Techniques — Detection & Bypass Playbook AI LOAD INSTRUCTION : Expert anti debug techniques across Linux and Windows. Covers ptrace, PEB flags, NtQueryInformationProcess, timing attacks, signal based detection, TLS callbacks, VEH tricks, and all corresponding bypass methods. Base models often miss the distinction between user mode and kernel mode detection and the correct patching strategy for each. 0. RELATED ROUTING [code obfuscation deobfuscation](../code obfuscation deobfuscation/SKILL.md) when the binary also uses control flow flattening, VM protection, or string encryption [vm and bytecode reverse](../vm and bytecode reverse/SKILL.md) when the anti debug sits inside a custom VM dispatcher [symbolic execution tools](../symbolic execution tools/SKILL.md) when you want to symbolically skip anti debug checks entirely Advanced Reference Also load [ANTI DEBUG MATRIX.md](./ANTI DEBUG MATRIX.md) when you need: Complete cross reference matrix of technique × OS × detection method × bypass method Per technique reliability ratings and false positive notes Tool compatibility chart (GDB, x64dbg, WinDbg, Frida, ScyllaHide) Quick bypass picks Detection Class First Bypass Backup ptrace based (Linux) LD PRELOAD hook ptrace() → return 0 Kernel module to hide tracer PEB.BeingDebugged (Windows) Patch PEB byte at fs:[0x30]+0x2 ScyllaHide auto patch Timing check (rdtsc) Conditional BP after rdtsc, fix registers Frida hook rdtsc return IsDebuggerPresent NOP the call / hook return 0 x64dbg built in hide INT 2D / UD2 exception Set VEH to handle gracefully TitanHide driver 1. LINUX ANTI DEBUG TECHNIQUES 1.1 ptrace(PTRACE TRACEME) The classic self attach: a process calls ptrace(PTRACE TRACEME, 0, 0, 0) . If a debugger is already attached, the call fails (returns 1). Bypass methods : Method How LD PRELOAD shim Compile shared lib: long ptrace(int r, ...) { return 0; } and set LD PRELOAD Binary patch NOP the ptrace call or patch return value check GDB catch catch syscall ptrace → modify $rax to 0 on return Kernel module Hook sys ptrace to allow multiple tracers 1.2 /proc/self/status — TracerPid Bypass : Mount a FUSE filesystem over /proc/self , or LD PRELOAD hook fopen / fread to filter TracerPid to 0. 1.3 Timing Checks (rdtsc / clock gettime) Measures elapsed time between two points; debugger single stepping causes noticeable delay. Bypass : Set hardware breakpoint after second rdtsc , modify eax to pass the comparison. Or use Frida to replace the timing function. 1.4 Signal Based Detection (SIGTRAP) When a debugger is attached, SIGTRAP is consumed by the debugger rather than delivered to the handler. Bypass : In GDB, use handle SIGTRAP nostop pass to forward the signal. 1.5 /proc/self/maps & LD PRELOAD Detection Checks for injected libraries or memory regions characteristic of debuggers/instrumentation. Bypass : Hook fopen("/proc/self/maps") to return a filtered version, or rename Frida's agent library. 1.6 Environment Variable Checks Some protections check for LD PRELOAD , LINES , COLUMNS (set by GDB's terminal), or debugger specific env vars. Bypass : Unset suspicious env vars before launch, or hook getenv() . 2. WINDOWS ANTI DEBUG TECHNIQUES 2.1 IsDebuggerPresent / CheckRemoteDebuggerPresent Bypass : Hook kernel32!IsDebuggerPresent to return 0, or patch PEB directly. 2.2 PEB Flags Field Offset (x64) Debugged Value Normal Value BeingDebugged PEB+0x02 1 0 NtGlobalFlag PEB+0xBC 0x70 (FLG HEAP ) 0 ProcessHeap.Flags Heap+0x40 0x40000062 0x00000002 ProcessHeap.ForceFlags Heap+0x44 0x40000060 0 Bypass : Zero all four fields. ScyllaHide does this automatically. 2.3 NtQueryInformationProcess InfoClass Value Debugged Return ProcessDebugPort 0x07 Non zero port ProcessDebugObjectHandle 0x1E Valid handle ProcessDebugFlags 0x1F 0 (inverted!) Bypass : Hook ntdll!NtQueryInformationProcess to return clean values per info class. 2.4 Hardware Breakpoint Detection Bypass : Hook GetThreadContext to zero DR0–DR3, or use NtSetInformationThread(ThreadHideFromDebugger) preemptively (ironically, the anti debug technique itself). 2.5 INT 2D / INT 3 / UD2 Exception Tricks INT 2D is the kernel debug service interrupt. Without a debugger, it raises STATUS BREAKPOINT ; with a debugger, behavior differs (byte skipping). Bypass : Handle in VEH or patch the interrupt instruction. 2.6 TLS Callbacks TLS callbacks execute before main() / WinMain() . Anti debug checks placed here run before the debugger's initial break. Bypass : In x64dbg, set "Break on TLS Callbacks" option. In WinDbg, use sxe ld to break on module load. 2.7 NtSetInformationThread(ThreadHideFromDebugger) After this call, the thread becomes invisible to the debugger — breakpoints and single stepping stop working silently. Bypass : Hook NtSetInformationThread to NOP when ThreadInfoClass == 0x11 . 2.8 VEH Based Detection Registers a Vectored Exception Handler that checks EXCEPTION RECORD for debugger specific behavior (single step flag, guard page violations with debugger semantics). Bypass : Understand the VEH logic and ensure the exception chain behaves identically to non debugged execution. 3. ADVANCED MULTI LAYER TECHNIQUES 3.1 Self Debugging (fork + ptrace) The process forks a child that attaches to the parent via ptrace. If an external debugger is already attached, the child's ptrace fails. Bypass : Patch the fork() return or kill/detach the watchdog child. 3.2 Multi Process Debugging Detection Parent and child cooperatively check each other's debug state, creating a mutual watch pattern. Bypass : Attach to both processes (GDB follow fork mode , or two debugger instances). 3.3 Timing Based with Multiple Checkpoints Distributes timing checks across multiple functions, comparing cumulative drift. Single patches fail because the total still exceeds threshold. Bypass : Frida Interceptor.replace all timing sources ( rdtsc , clock gettime , QueryPerformanceCounter ) to return controlled values. 3.4 Nanomite / INT3 Patching Original conditional jumps are replaced with INT3 (0xCC). A parent debugger process handles each INT3 , evaluates the condition, and sets the child's EIP accordingly. Bypass : Reconstruct the original jump table by tracing all INT3 handlers, then patch the binary. 4. COUNTERMEASURE TOOLS Tool Platform Capability ScyllaHide Windows (x64dbg/IDA/OllyDbg) Auto patches PEB, hooks NtQuery , hides threads, fixes timing TitanHide Windows (kernel driver) Kernel level hiding for all user mode checks Frida Cross platform Script based hooking of any function, timing spoofing LD PRELOAD shims Linux Replace ptrace, getenv, fopen at load time GDB scripts Linux catch syscall , conditional BP, register fixup Qiling Cross platform Full system emulation, bypass all hardware checks 5. SYSTEMATIC BYPASS METHODOLOGY 6. DECISION TREE 7. CTF & REAL WORLD PATTERNS Common CTF Anti Debug Patterns Pattern Frequency Quick Bypass Single ptrace(TRACEME) Very common LD PRELOAD one liner IsDebuggerPresent + NtGlobalFlag Common ScyllaHide rdtsc timing in loop Moderate Patch comparison threshold signal(SIGTRAP) + raise Moderate GDB signal forwarding fork + ptrace watchdog Rare but tricky Kill child or patch fork Nanomite INT3 replacement Rare (advanced) Reconstruct jump table Real World Protections Protector Primary Anti Debug Recommended Tool VMProtect PEB + timing + driver level TitanHide + ScyllaHide Themida Multi layer PEB + SEH + timing ScyllaHide + manual patches Enigma Protector IsDebuggerPresent + CRC checks x64dbg + ScyllaHide UPX (custom) Usually none (just packing) Standard unpack Custom (malware) Varies widely Frida + Qiling for analysis 8. QUICK REFERENCE — BYPASS CHEAT SHEET Linux One Liners Frida Anti Debug Bypass (Cross Platform) x64dbg ScyllaHide Quick Setup 1. Plugins → ScyllaHide → Options 2. Check: PEB BeingDebugged, NtGlobalFlag, HeapFlags 3. Check: NtQueryInformationProcess (all classes) 4. Check: NtSetInformationThread (HideFromDebugger) 5. Check: GetTickCount, QueryPerformanceCounter 6. Apply → restart debugging session

anti-debugging-techniques