copilot-history-ingest

Ingest GitHub Copilot CLI session history into an Obsidian wiki as distilled knowledge pages. Use this skill when the user wants to capture their Copilot CLI sessions into a personal wiki — extracting architecture decisions, debug notes, and patterns into searchable Obsidian pages. Triggers on phras

By ar9av · 1,659 installs

npx skills add ar9av/obsidian-wiki --skill copilot-history-ingest

Source repository · Upstream listing

Copilot History Ingest — Conversation Mining You are extracting knowledge from the user's past GitHub Copilot CLI conversations and distilling it into the Obsidian wiki. Conversations are rich but messy — your job is to find the signal and compile it. This skill can be invoked directly or via the wiki history ingest router ( /wiki history ingest copilot ). Before You Start Writing profile: Before drafting or rewriting natural language Markdown, read and apply the Writing Profile Resolution section in llm wiki/SKILL.md . Framework schema, provenance, safety, and operation specific requirements take precedence. WRITING.md preferences apply only to newly drafted or rewritten natural language Markdown; preserve source content and structured records. 1. Resolve config — follow the Config Resolution Protocol in llm wiki/SKILL.md (inline @name override → walk up CWD for .env → global config → prompt setup). This gives OBSIDIAN VAULT PATH , COPILOT HISTORY PATH (defaults to ~/.copilot/session state ), and COPILOT VSCODE STORAGE PATH (VS Code workspaceStorage ; platform specific — ask the user if absent) 2. Read .manifest.json at the vault root to check what's already been ingested 3. Read index.md at the vault root to know what the wiki already contains Ingest Modes Append Mode (default) Check .manifest.json for each source file (events JSONL, transcript JSONL, checkpoint, session store DB). Only process: Sessions not in the manifest (new sessions) Sessions whose updated at is newer than their ingested at in the manifest This is usually what you want — the user ran a few new sessions and wants to capture the delta. Full Mode Process everything regardless of manifest. Use after a wiki rebuild or if the user explicitly asks. GitHub Copilot Data Layout Copilot stores data in three locations. Scan all three . Source 1: ~/.copilot/session state/ (CLI sessions) Source 2: ~/.copilot/session store.db (Global SQLite) The canonical cross session database. This is the highest value source: structured, queryable, and pre summarised. Source 3: VS Code Workspace Storage ( <workspaceStorage /<hash /GitHub.copilot chat/ ) VS Code extension data, keyed by workspace hash. The path is platform specific and must come from .env or user input. Key data sources ranked by value: 1. Checkpoints ( session store.db checkpoints table + per session checkpoints/ .json ) — Pre distilled summaries with overview , work done , technical details , important files , next steps . Gold. 2. Session summaries ( session store.db sessions.summary + index.md ) — One paragraph synopsis per session. 3. Turns ( session store.db turns table + events.jsonl / transcript JSONL) — Full conversation. Rich but verbose. 4. Memory artifacts ( memory tool/memories/<id /plan.md etc.) — Pre written plans and structured notes the user saved explicitly. Worth importing verbatim (or lightly summarised). 5. File access patterns ( session files table + tool.execution events) — Which files the agent repeatedly touched — reveals high value project files. 6. Session refs ( session refs table) — Commits, PRs, and issues linked to sessions. 7. vscode.metadata.json — Workspace folder path, branch, customTitle (user set session label). Useful for grouping and naming. Step 1: Survey and Compute Delta Scan all three data locations and compare against .manifest.json : Build a unified inventory — one entry per session UUID — and classify: New — not in manifest → needs ingesting Modified — in manifest but updated at is newer → needs re ingesting Unchanged — in manifest and not modified → skip in append mode Report to the user: "Found X sessions in session state, Y in session store.db, Z VS Code transcript files. Checkpoints: A. Delta: B new, C modified." Step 2: Ingest Checkpoints and Summaries First Checkpoints are already distilled — process them before touching raw turns. From session store.db : From per session checkpoints/ .json : Each checkpoint file has: title , overview , history , work done , technical details , important files , next steps . Read index.md (if present) as a session level summary — it's typically written at session end and is already concise. What to extract: overview → high level description of what the session accomplished work done → concrete tasks completed (good for skills / project pages) technical details → implementation specifics (good for concepts pages) important files → high value files in the project (good for project pages) next steps → open threads (good for linking to ongoing project work) Step 3: Parse Session Turns Read turns from session store.db (preferred — already parsed) or from events.jsonl / transcript JSONL. From session store.db : From events.jsonl / transcript JSONL: Each file is one session. Each line is a JSON event. See references/copilot data format.md for the full schema. Relevant event types: type What it is Worth reading? session.start Session metadata (cwd, branch, version) Yes — establishes project context user.message User turn Yes — data.content assistant.message Assistant turn Yes — data.content (text) + data.toolRequests tool.execution start Tool call Skim — reveals what files/commands were used tool.execution end Tool result No — usually noise Extraction strategy for assistant.message : data.content is the assistant's text response — extract this data.reasoningText is internal reasoning — skip (it's the unpacked reasoningOpaque field) data.toolRequests lists tool calls — skim tool names and arguments for file access patterns Skip type: "tool.execution end" entirely Step 3b: Process Memory Artifacts For each session that has a memory tool/memories/<base64 id / directory in VS Code workspace storage, read any markdown files saved there (typically plan.md ). These are documents the user explicitly saved — treat them as high quality, user authored content. Decode the base64 directory name to get the session UUID: Memory artifacts map to project skills/ or concepts/ pages, depending on content type. Step 3c: Extract File and Ref Patterns From session store.db : File access patterns reveal which files are architecturally important — note them on project pages. Session refs link Copilot sessions to git history — useful for connecting wiki knowledge to concrete code changes. Step 4: Cluster by Topic Don't create one wiki page per session. Instead: Group extracted knowledge by topic across sessions A single session about "debugging auth + setting up CI" → two separate topics Three sessions across different days about "React performance" → one merged topic cwd / repository give you a natural first level grouping; vscode.metadata.json 's customTitle gives a human readable session label Step 5: Distill into Wiki Pages Each Copilot project maps to a project directory in the vault. Derive the project name from cwd or repository : Prefer repository (e.g., owner/repo ) from session store.db over raw cwd when available. Project specific vs. global knowledge What you found Where it goes Example Project architecture decisions projects/<name /concepts/ projects/my project/concepts/main architecture.md Project specific debugging patterns projects/<name /skills/ projects/my project/skills/api rate limiting.md General concept the user learned concepts/ (global) concepts/react server components.md Recurring problem across projects skills/ (global) skills/debugging hydration errors.md A tool/service used entities/ (global) entities/vercel functions.md Patterns across many sessions synthesis/ (global) synthesis/common debugging patterns.md For each project with content, create or update the project overview page at projects/<name /<name .md — named after the project, not project.md . Obsidian's graph view uses the filename as the node label, so project.md makes every project show up as project in the graph. Naming it <name .md gives each project a distinct, readable node name. Important: Distill the knowledge , not the conversation. Don't write "In a session on March 15, the user asked about X." Write the knowledge itself, with the session as a source attribution. Write a summary: frontmatter field on every new/updated page — 1–2 sentences, ≤200 chars, answering "what is this page about?" for a reader who hasn't opened it. wiki query 's cheap retrieval path reads this field to avoid opening page bodies. Add confidence and lifecycle fields to every new page's frontmatter: Leave lifecycle unchanged on update. Mark provenance per the convention in llm wiki (Provenance Markers section): Checkpoints and index.md are pre distilled by the system — treat checkpoint derived claims as extracted (the system wrote them from observed actions). Memory artifacts are user authored — treat as extracted. Conversation turn distillation is mostly inferred. You're synthesizing a coherent claim from many turns. Apply ^[inferred] liberally to synthesized patterns, generalizations across sessions, and "what the user really meant" interpretations. Use ^[ambiguous] when the user changed direction mid session or when the session ended unresolved. Write a provenance: frontmatter block on every new/updated page summarizing the rough mix. Step 6: Update Manifest, Journal, and Special Files Update .manifest.json For each session processed, add/update its entry with: ingested at , session id , updated at source type : one of "copilot session" , "copilot checkpoint" , "copilot transcript" , "copilot memory artifact" project : the decoded project name pages created and pages updated lists Also update the projects section of the manifest: Create journal entry + update special files Update index.md , log.md , and hot.md with one locked call: Never hand edit index.md , log.md , or hot.md — the command takes the lock that keeps a parallel writer from dropping your update. takeaways is the one line conceptual summary that used to go in Recent Activity; omit it to leave the previous takeaways untouched. If an ongoing project is now better understood, record the thread so the next session picks it up: obsidian wiki memory todo add "<thread " origin projects/<name .md . See .skills/llm wiki/references/MEMORY.md for the full procedure. Privacy Distill and synthesize — don't copy raw conversation text verbatim Skip anything that looks like secrets, API keys, passwords, tokens data.reasoningOpaque / data.reasoningText in assistant events is internal reasoning — skip entirely, never copy to wiki If you encounter personal/sensitive content, ask the user before including it The user's conversations may reference other people — be thoughtful about what goes in the wiki Reference See references/copilot data format.md for detailed data structure documentation. QMD Refresh After Vault Writes QMD is a search index, not the source of truth. If $QMD WIKI COLLECTION is empty or unset, skip this step. Run it only after this skill has written or rewritten vault markdow