This skill should be used when the user requests to research topics using FireCrawl, enrich notes with web sources, search and scrape information, or write scientific/academic papers. It extracts research topics from markdown files, creates research documents with scraped sources, generates BibTeX b

FireCrawl Research Overview Enrich research documents by automatically searching and scraping web sources using the FireCrawl API. Extract research topics from markdown files and generate comprehensive research documents with source material. When to Use This Skill Use this skill when the user: Says "Research this topic using FireCrawl" Requests to enrich notes or documents with web sources Wants to gather information about topics listed in a markdown file Needs to search and scrape multiple topics systematically How It Works 1. Topic Extraction The script automatically extracts research topics from markdown files using two methods: Method 1: Headers Both Spatial Reasoning in AI and Computer Vision Applications become research topics. Method 2: Research Tags Both tagged items become research topics. 2. Search and Scrape For each topic: 1. Searches FireCrawl with the topic as query 2. Retrieves up to N results (default: 5) 3. Automatically scrapes full content from each result 4. Extracts markdown formatted content (main content only) 3. Output Generation Creates new markdown files in the specified output directory: One file per topic Filename: {topic} {timestamp}.md Contains: title, date, sources count, full scraped content Each source includes: title, URL, markdown content Usage Basic Usage Outputs to current directory. Specify Output Directory Creates files in ./output/ folder. Limit Results Per Topic Retrieves maximum 3 results per topic. Configuration API Key Setup 1. Copy .env.example to .env : 2. Add FireCrawl API key: The script automatically loads the API key from the skill's .env file. Rate Limiting The script includes automatic rate limiting for FireCrawl's free tier: Free tier limit: 5 requests/minute Built in delay: 12 seconds between topics Prevents API errors and credit exhaustion When processing multiple topics, expect: 5 topics: ~1 minute 10 topics: ~2 minutes 20 topics: ~4 minutes Workflow Example User request: "Research these AI topics using FireCrawl" Input file ( ai research.md ): Command: Output: Each file contains: Topic title Timestamp Source count Full scraped content from up to 5 sources Source URLs Common Patterns Pattern 1: Quick Research Extract topics from existing notes, research them, save to current folder: Pattern 2: Organized Research Create dedicated output folder for research results: Pattern 3: Deep Dive Increase results per topic for comprehensive coverage: Pattern 4: Obsidian Vault Integration Direct output to vault's research folder: Error Handling "API key not found" Create .env file in skill folder with FIRECRAWL API KEY=... "Rate limit exceeded" Free tier: 5 req/min Script has 12s delay built in If still hitting limit, reduce topics or wait between runs "Insufficient credits" Check FireCrawl account credits Upgrade plan or wait for credit reset "No topics found" Add topics to markdown using: Header format [research] Topic format [search] Topic format Script Details Location: scripts/firecrawl research.py Dependencies: python dotenv Environment variable management requests HTTP requests to FireCrawl API Install dependencies: FireCrawl Features Used: /v1/search endpoint Search with automatic scraping scrapeOptions.formats: ['markdown'] Markdown output scrapeOptions.onlyMainContent: true Filter noise Academic Writing Templates This skill includes templates for writing scientific papers in markdown format. Available Templates 1. Pandoc Scholarly Paper ( assets/templates/pandoc scholarly paper.md ) Standard academic paper format Compatible with Pandoc converter Supports citations via BibTeX Exports to PDF, DOCX, HTML 2. MyST Scientific Paper ( assets/templates/myst scientific paper.md ) MyST (Markedly Structured Text) format Advanced cross referencing Professional scientific publishing Multi format export (PDF, LaTeX, DOCX) Using Templates Copy template to your project: Edit content: Update YAML frontmatter (title, authors, affiliations) Write your content in sections Add citations using [@AuthorYear] (Pandoc) or {cite}\ AuthorYear\ (MyST) Convert to PDF/DOCX: Bibliography Generation Convert FireCrawl research results into BibTeX bibliography entries: What it does: Extracts URLs and titles from FireCrawl markdown files Generates BibTeX @misc entries Creates citation keys automatically Adds access dates Example workflow: Citation Examples Pandoc syntax: MyST syntax: Example Bibliography File An example bibliography is provided in assets/references.bib with common entry types: Journal articles ( @article ) Conference papers ( @inproceedings ) Books ( @book ) PhD theses ( @phdthesis ) Web resources ( @misc ) Preprints ( @article with arXiv) Tips 1. Organize topics hierarchically Use for main topics, for subtopics 2. Use descriptive names Topic text becomes filename, make it clear 3. Batch processing Group related topics in one file for efficiency 4. Output organization Create separate folders for different research projects 5. Content review Results are truncated at 3000 chars/source for readability 6. Academic workflow Use bibliography generator to cite research sources in papers 7. Template customization Modify templates for your field's citation style Limitations No summarization Returns raw scraped content, not summaries No deduplication Duplicate sources may appear across topics No quality ranking All results treated equally New files only Does not append to existing files Free tier constraints Rate limiting affects processing speed

firecrawl-research