differential-fuzzer
Information about the differential fuzzer tool, how to run it and use it catch bugs in Turso. Always load this skill when running this tool
By tursodatabase · 1,142 installs
npx skills add tursodatabase/turso --skill differential-fuzzer
Source repository · Upstream listing
Differential Fuzzer
Always load [Debugging skill for reference](../debugging/)
The differential fuzzer compares Turso results against SQLite for generated SQL statements to find correctness bugs.
Location
testing/differential oracle/fuzzer/
Running the Fuzzer
Single Run
Continuous Fuzzing (Loop Mode)
Docker Runner (CI/Production)
Environment variables for docker runner:
TIME LIMIT MINUTES Total runtime (default: 1440 = 24h)
PER RUN TIMEOUT SECONDS Per run timeout (default: 1200 = 20min)
NUM STATEMENTS Statements per run (default: 1000)
LOG TO STDOUT Print fuzzer output (default: false)
GITHUB TOKEN For auto filing issues
SLACK WEBHOOK URL For notifications
Output Files
All output goes to simulator output/ directory:
File Description
test.sql All executed SQL statements. Failed statements prefixed with FAILED: , errors with ERROR:
schema.json Database schema at end of run (or at failure)
test.db Turso database file (only with keep files )
test sqlite.db SQLite database file (only with keep files )
Reproducing Errors
Always follow these steps
1. Find the seed and profile in the error output:
2. Re run with that seed and profile (a seed only replays under the same profile):
3. Read the minimized reproduction first. On an oracle failure the fuzzer
writes these files to simulator output/ :
minimized.sql a shrunken state script plus the shrunken failing
statement, produced automatically. Start here.
turso state.sql / sqlite state.sql each engine's full state as a
replayable script, when you need more than the minimized version kept.
test.sql every executed statement (the failing one is marked
FAILED: ). The minimizer falls back to replaying this history when
the failure depends on how the state was built, not just its contents.
schema.json table structure at failure time.
4. Probe the reproduction with differential probe . It runs a
statement per line script on Turso and SQLite side by side, prints both
outcomes for every statement, marks divergences, and compares the final
table contents. Exit code 1 means something diverged.
Use it instead of piping SQL into the two shells: the tursodb shell cannot
ATTACH ':memory:' AS aux , so fuzzer reproductions with an aux schema
only run correctly through the probe. Reading from stdin also works:
echo "SELECT ~X'96';" cargo run q p differential fuzzer bin differential probe .
5. Bisect by editing the script. Copy minimized.sql , simplify one thing
at a time (replace an expression with a constant, drop a column, drop a
state line), and re run the probe after each edit. The divergence marker
tells you immediately whether the edit kept the bug. This loop usually
ends at a one line kernel you can hand to EXPLAIN on both engines.
6. Create a regression test in .sqltest (preferred) or .rs from the
kernel. Always load the [Debugging skill for reference](../debugging/).
Understanding Failures
Oracle Failure Types
1. Row set mismatch Turso returned different rows than SQLite
2. Turso errored but SQLite succeeded Turso rejected valid SQL
3. SQLite errored but Turso succeeded Turso accepted invalid SQL
4. Schema mismatch Tables/columns differ after DDL
Warning (non fatal)
Unordered LIMIT mismatch LIMIT without ORDER BY may return different valid rows
Key Source Files
File Purpose
main.rs CLI parsing, entry point
runner.rs Main simulation loop, executes statements on both DBs
oracle.rs Compares Turso vs SQLite results
schema.rs Introspects schema from both databases
memory/ In memory IO for deterministic simulation
Tracing
Set RUST LOG for more detailed output: