Build production-ready Bright Data integrations with best practices baked in. Reference documentation for developers using coding assistants (Claude Code, Cursor, etc.) to implement web scraping, search, browser automation, and structured data extraction. Covers Web Unlocker API, SERP API, Web Scrap

CLI Setup Reference Install, authentication, and troubleshooting for the Bright Data CLI ( bdata ) are documented in a single canonical place: → [ references/cli setup.md ](references/cli setup.md) Consult it before any task that shells out to bdata . Bright Data APIs Bright Data provides infrastructure for web data extraction at scale. Four primary APIs cover different use cases — always pick the most specific tool for the job. Choosing the Right API Use Case API Why Scrape any webpage by URL (no interaction) Web Unlocker HTTP based, auto bypasses bot detection, cheapest Google / Bing / Yandex search results SERP API Specialized for SERP extraction, returns structured data Structured data from Amazon, LinkedIn, Instagram, TikTok, etc. Web Scraper API Pre built scrapers, no parsing needed Click, scroll, fill forms, run JS, intercept XHR Browser API Full browser automation Puppeteer / Playwright / Selenium automation Browser API Connects via CDP/WebDriver Route your own HTTP client through a raw proxy (DC/ISP/Residential/Mobile) Proxy networks When you need direct proxy access with your own request logic instead of a managed API — see the proxy.md skill Authentication Pattern (All APIs) All APIs share the same authentication model. The env vars below apply to direct REST API integrations — if you are using the bdata CLI, bdata login handles all of these automatically (see [ references/cli setup.md ](references/cli setup.md)). REST API authentication header for Web Unlocker and SERP API: Web Unlocker API HTTP based scraping proxy. Best for simple page fetches without browser interaction. Endpoint: POST https://api.brightdata.com/request Key Parameters Parameter Type Description zone string Zone name (required) url string Target URL with http:// or https:// (required) format string "raw" (HTML) or "json" (structured wrapper) (required) method string HTTP verb, default "GET" country string 2 letter ISO for geo targeting (e.g., "us" , "de" ) data format string Transform: "markdown" or "screenshot" async boolean true for async mode Quick Patterns Critical rule: Never use Web Unlocker with Puppeteer, Playwright, Selenium, or anti detect browsers. Use Browser API instead. See [references/web unlocker.md](references/web unlocker.md) for complete reference including proxy interface, special headers, async flow, features, and billing. SERP API Structured search engine result extraction for Google, Bing, Yandex, DuckDuckGo. Endpoint: POST https://api.brightdata.com/request (same as Web Unlocker) Essential Google URL Parameters Parameter Description Example q Search query q=python+web+scraping brd json Parsed JSON output brd json=1 (always use for data pipelines) gl Country for search gl=us hl Language hl=en start Pagination offset start=10 (page 2), start=20 (page 3) tbm Search type tbm=nws (news), tbm=isch (images), tbm=vid (videos) brd mobile Device brd mobile=1 (mobile), brd mobile=ios brd browser Browser brd browser=chrome brd ai overview Trigger AI Overview brd ai overview=2 uule Encoded geo location for precise location targeting Note: num parameter is deprecated as of September 2025. Use start for pagination. Parsed JSON Response Structure Bing Key Parameters Parameter Description q Search query setLang Language (prefer 4 letter: en US ) cc Country code first Pagination (increment by 10: 1, 11, 21...) safesearch off , moderate , strict brd mobile Device type Async for Bulk SERP Billing: Pay per 1,000 successful requests only. Async retrieve calls are not billed. See [references/serp api.md](references/serp api.md) for complete reference including Maps, Trends, Reviews, Lens, Hotels, Flights parameters. Web Scraper API Pre built scrapers for structured data extraction from 100+ platforms. No parsing logic needed. Sync Endpoint: POST https://api.brightdata.com/datasets/v3/scrape Async Endpoint: POST https://api.brightdata.com/datasets/v3/trigger Parameters Parameter Type Description dataset id string Scraper identifier from the Scraper Library (required) format string json (default), ndjson , jsonl , csv custom output fields string Pipe separated fields: url\ title\ price include errors boolean Include error info in results Request Body Poll for Async Results Progress status values: starting → running → ready failed Data retention: 30 days. Billing: Per delivered record. Invalid input URLs that fail are still billable. See [references/web scraper api.md](references/web scraper api.md) for complete reference including scraper types, output formats, delivery options, and billing details. Browser API (Scraping Browser) Full browser automation via CDP/WebDriver. Handles CAPTCHA, fingerprinting, and anti bot detection automatically. Connection: Playwright/Puppeteer: wss://${AUTH}@brd.superproxy.io:9222 Selenium: https://${AUTH}@brd.superproxy.io:9515 Custom CDP Functions Function Purpose Captcha.solve Manually trigger CAPTCHA solving Captcha.setAutoSolve Enable/disable auto CAPTCHA solving Proxy.setLocation Set precise geo location (call BEFORE goto) Proxy.useSession Maintain same IP across sessions Emulation.setDevice Apply device profile (iPhone 14, etc.) Emulation.getSupportedDevices List available device profiles Unblocker.enableAdBlock Block ads to save bandwidth Unblocker.disableAdBlock Re enable ads Input.type Fast text input for bulk form filling Browser.addCertificate Install client SSL cert for session Page.inspect Get DevTools debug URL for live session Session Rules One initial navigation per session — new URL = new session Idle timeout: 5 minutes Max duration: 30 minutes Geolocation Country level: append country us to credentials username EU wide: append country eu (routes through 29+ European countries) Precise: use Proxy.setLocation CDP command (before navigation) Error Codes Code Issue Fix 407 Wrong port Playwright/Puppeteer → 9222 , Selenium → 9515 403 Bad auth Check credentials format and zone type 503 Service scaling Wait 1 minute, reconnect Billing: Traffic based only. Block images/CSS/fonts to reduce costs. See [references/browser api.md](references/browser api.md) for complete reference including all CDP functions, bandwidth optimization, CAPTCHA patterns, and debugging. Detailed References [references/web unlocker.md](references/web unlocker.md) — Web Unlocker: full parameter list, proxy interface, special headers, async flow, features, billing, anti patterns [references/serp api.md](references/serp api.md) — SERP API: all Google params (Maps, Trends, Reviews, Lens, Hotels, Flights), Bing params, parsed JSON structure, async, billing [references/web scraper api.md](references/web scraper api.md) — Web Scraper API: sync vs async, all parameters, polling, scraper types, output formats, billing [references/browser api.md](references/browser api.md) — Browser API: connection strings, session rules, all CDP functions, geo targeting, bandwidth optimization, CAPTCHA, debugging, error codes Related Skills brightdata proxy — For routing requests through Bright Data's raw proxy networks (Datacenter, ISP, Residential, Mobile) with your own HTTP client instead of a managed API. Covers network/IP pool selection, the brd customer ... username format, targeting & sticky session params, SSL CA setup for Residential/Mobile, and integrations for cURL, Python (requests/httpx/aiohttp/Scrapy), Node (fetch/axios), Playwright, Puppeteer, and Selenium. Hand off to it whenever the task is raw proxy access rather than Web Unlocker / SERP / Web Scraper / Browser API. Escalation order when proxies hit consistent blocks: raw proxy → Web Unlocker → Browser API → Web Scraper API.

bright-data-best-practices