neo4j-import-skill

Import structured data into Neo4j — LOAD CSV, CALL IN TRANSACTIONS, neo4j-admin database import full (offline bulk), apoc.load.csv/json, apoc.periodic.iterate, driver batch writes. Covers method selection, header file format, type coercion, null handling, ON ERROR modes, CONCURRENT TRANSACTIONS, pre

By neo4j-contrib · 573 installs

npx skills add neo4j-contrib/neo4j-skills --skill neo4j-import-skill

Source repository · Upstream listing

Neo4j Import Skill When to Use Importing CSV, JSON, or Parquet files into Neo4j Batch upserting nodes and relationships (UNWIND + CALL IN TRANSACTIONS) Migrating relational data (SQL → graph) Bulk loading large datasets offline (neo4j admin import) Choosing between online (Cypher) and offline (admin) import methods Verifying import completeness (counts, constraints, index states) When NOT to Use Unstructured docs, PDFs, vector chunks → neo4j document import skill Live application writes (MERGE/CREATE in app code) → neo4j cypher skill neo4j admin backup/restore/config → neo4j cli tools skill GDS algorithm projection from existing graph → neo4j gds skill Method Decision Table Dataset size DB state Source Method Any size Online CSV (Aura or local) LOAD CSV + CALL IN TRANSACTIONS < 1M rows Online List/API response UNWIND + CALL IN TRANSACTIONS 10M rows Offline (local/self managed) CSV / Parquet neo4j admin database import full Any size Online APOC available apoc.periodic.iterate + apoc.load.csv Any size Online JSON/API apoc.load.json or driver batching Incremental delta Offline (Enterprise) CSV neo4j admin database import incremental Aura : only https:// URLs — no file:/// . Use neo4j admin import only on self managed. Pre Import Checklist Run in this exact order — skipping causes hard to debug duplicates or missed index usage: Constraints BEFORE import. Additional indexes AFTER import. Constraints create implicit RANGE indexes used by MERGE during load + enforce uniqueness Additional non unique indexes (TEXT, RANGE on non key props, FULLTEXT) created after load — Neo4j populates them async from the committed data; poll populationPercent until 100% Creating extra indexes before import slows every write during load with no benefit 1. Create uniqueness constraints (enables index used by MERGE): Neo4j 2026.06+ (Enterprise/Aura, GA): ALTER CURRENT GRAPH TYPE SET { … } replaces all individual constraint statements with a single declarative block. See neo4j cypher skill/references/graph type.md . Use individual CREATE CONSTRAINT on Community Edition or pre 2026.02. 2. Verify APOC if using apoc. procedures : If fails → APOC not installed. Use plain LOAD CSV instead. 3. Confirm target is PRIMARY (not replica): If role ≠ PRIMARY → stop. Redirect write to PRIMARY endpoint. 4. Count source file rows before import (catch encoding issues early): 5. Verify UTF 8 encoding — LOAD CSV requires UTF 8. Re encode if needed: LOAD CSV Patterns Basic node import with type coercion and null handling Null/empty string rules: CSV missing column → null (safe) CSV empty string "" → stored as "" not null — use nullIf(row.x, '') to convert toInteger(null) throws → always use toIntegerOrNull() toFloat(null) throws → always use toFloatOrNull() Neo4j never stores null properties — they are silently dropped on SET Relationship import (nodes must exist first) Always import ALL nodes before ANY relationships — MATCH fails on missing nodes. Tab separated or custom delimiter Compressed files (ZIP / gzip — local files only) Cloud storage (Enterprise Edition) Scheme Example AWS S3 s3://my bucket/data/persons.csv Google Cloud Storage gs://my bucket/persons.csv Azure Blob azb://account/container/persons.csv Useful built in functions inside LOAD CSV CALL IN TRANSACTIONS — Full Reference Syntax ON ERROR modes Mode Behavior Use when ON ERROR FAIL Default. Rolls back entire outer tx on first error All or nothing strict import ON ERROR CONTINUE Skips failed batch, continues remaining batches Resilient bulk load — track errors via REPORT STATUS ON ERROR BREAK Stops after first failed batch; keeps completed work Semi strict: stop early, keep successful batches ON ERROR RETRY Exponential backoff retry (default 30s) + fallback Concurrent writes with deadlock risk ON ERROR CONTINUE/BREAK → outer transaction succeeds even if inner batches fail. ON ERROR FAIL → cannot be combined with REPORT STATUS AS . CONCURRENT TRANSACTIONS (parallel batches) Use CONCURRENT for read heavy MERGE on non overlapping key spaces. Risk: deadlocks on overlapping writes → combine with ON ERROR RETRY . REPORT STATUS columns Column Type Meaning s.started BOOLEAN Batch transaction started s.committed BOOLEAN Batch committed successfully s.transactionId STRING Transaction ID s.errorMessage STRING or null Error detail if batch failed Batch size guidance Row count Recommended batch size Notes < 100k 10 000 Default is fine 100k – 1M 10 000 – 50 000 Monitor heap; increase if fast 1M – 10M 50 000 – 100 000 Enable CONCURRENT if CPUs available 10M online 50 000 Consider neo4j admin import instead Relationship import 5 000 Lower — each batch does 2x MATCH neo4j admin import (Offline Bulk Load) Fastest method: ~3 min for 31M nodes / 78M rels on SSD. DB must be stopped or non existent. Command structure Dry run (2026.02+) — validate without writing: Node header file format Field Meaning :ID Unique ID for relationship wiring (not stored as property by default) :ID(Group) Scoped ID space — use when node types share IDs :LABEL One or more labels; semicolon separated: Person;Employee prop:int Typed property; types: int long float double boolean byte short string prop:date Temporal: date localtime time localdatetime datetime duration ; Parquet INTERVAL columns import as DURATION [2026.07+] prop:int[] Array — semicolon separated values in cell: 1;2;3 prop:vector Float vector (2025.10+) — semicolon separated coordinates in CSV; imports directly from native Parquet list types [2026.06+] Relationship header file format :START ID / :END ID must reference the same :ID group as the node files. Key flags Flag Default Notes delimiter , Single byte UTF 8 char, TAB , \ID , or U+XXXX ; newline chars rejected [2026.07+] vector delimiter ; Separates prop:vector coordinates; must differ from delimiter and quote [enforced 2026.06+] id type STRING STRING \ INTEGER \ ACTUAL bad tolerance 1 (unlimited, changed 2025.12) Set 0 for strict prod imports threads CPU count Set explicitly on shared hosts max off heap memory 90% RAM Reduce if other services share host high parallel io off Set on for SSD/NVMe format standard block for 34B nodes/rels overwrite destination false Required if DB already exists dry run false 2026.02+ — validate without writing Schema file ( schema) [Enterprise, block format] Pass a Cypher file with CREATE CONSTRAINT / CREATE INDEX statements; executed automatically after import completes. Constraints are created first (correct order enforced). File paths can be local or remote ( s3:// , gs:// , https:// ). For incremental import, DROP CONSTRAINT / DROP INDEX are also supported [2025.02+] — used to remove indexes before the merge phase and recreate them after for faster writes. schema also accepts graph type DDL: ALTER CURRENT GRAPH TYPE SET {…} for full [2026.05+], ALTER CURRENT GRAPH TYPE ADD/DROP/ALTER {…} for incremental [2026.06+] — see neo4j cypher skill/references/graph type.md . Incremental import (Enterprise only) Three phase process — use when DB must stay online during import preparation: Requires Enterprise Edition + block store format. APOC Patterns (when APOC is available) Verify first: RETURN apoc.version() — if fails, use LOAD CSV or driver instead. apoc.periodic.iterate — batch process existing graph data Config key Default Notes batchSize 10000 Rows per inner transaction parallel false Enable for non overlapping writes; risk: deadlocks retries 0 Retry failed batches N times with 100ms delay Prefer CALL IN TRANSACTIONS (native Cypher) over apoc.periodic.iterate for new code — it has REPORT STATUS , CONCURRENT , and RETRY built in without APOC dependency. apoc.load.csv — load with config options apoc.load.json — load JSON from file or URL Driver Batch Write Pattern Use when source is not a file (API responses, DB migrations). Collect into BATCH SIZE (10 000) lists, call UNWIND $rows AS row MERGE ... per batch. ~10x faster than row at a time. → [Python + JS examples](references/driver batch write.md) MCP Tool Usage Operation MCP tool Notes SHOW CONSTRAINTS , SHOW INDEXES read cypher Always inspect before import CREATE CONSTRAINT , CREATE INDEX write cypher Gate: show planned constraint, confirm LOAD CSV / CALL IN TRANSACTIONS write cypher Gate: show row count + Cypher, confirm Verify counts read cypher Post import: MATCH (n:Label) RETURN count(n) Poll index state read cypher Poll until all state = 'ONLINE' Write gate — before any bulk write via MCP, show: 1. Query + affected labels 2. Estimated row count from source 3. EXPLAIN plan Wait for user confirmation. Never auto execute CALL IN TRANSACTIONS or CREATE CONSTRAINT without confirmation. Always pass database param if not default: {"code": "...", "database": "neo4j"} . Common Errors Error Cause Fix Couldn't load the external resource file:/// path not in Neo4j import dir Move file to $NEO4J HOME/import/ ; check dbms.security.allow csv import from file urls=true Cannot merge node using null property value MERGE key resolved to null Validate row.id IS NOT NULL before MERGE; add WHERE row.id IS NOT NULL toInteger() called on null Null column fed to non null safe fn Replace toInteger() → toIntegerOrNull() , toFloat() → toFloatOrNull() Node N already exists / constraint violation mid import Duplicate source IDs Dedup source CSV; use MERGE not CREATE ; add IF NOT EXISTS to constraint Heap overflow / OutOfMemoryError Batch too large or file too large Reduce batch size; switch to CALL IN TRANSACTIONS ; neo4j admin for offline Invalid input 'IN': expected...' PERIODIC COMMIT used Replace USING PERIODIC COMMIT → CALL IN TRANSACTIONS — PERIODIC COMMIT removed in Cypher 25 neo4j admin: Bad input data Wrong header format or type mismatch Check :ID , :START ID , :END ID present; check typed columns parse correctly neo4j admin: import fails silently bad tolerance default was unlimited pre 2025.12 Set bad tolerance=0 to surface all errors Index not used during MERGE Constraint not created before import Drop data, create constraint, re import Relationship import missing nodes Relationships imported before nodes Always import ALL node files before ANY relationship files Post Import Validation After import completes — run all: Do NOT run production queries until all indexes are ONLINE. References [LOAD CSV — Cypher Manual 25](https://neo4j.com/docs/cypher manual/25/clauses/load csv/) [CALL IN TRANSACTIONS — Cypher Manual](https://neo4j.com/docs/cypher manual/current/subqueries/subqueries in transactions/) [neo4j admin database import](https://neo4j.com/docs/operations manual/current/tools/neo4j admin/neo4j admin import/) [APOC periodic execution](https://neo4j.com/docs/apoc/current/graph updates/periodic execution/) [APOC load procedures](https://neo4j.com/docs/apoc/current/import/) [GraphAcademy: Importing CSV Data](https://graphacademy.neo4j.com/courses/impo