One API and one credential for frontier and open-source LLMs, built into your Neon branch and powered by Databricks. Use when a user wants to call an LLM, add AI/chat/an agent to their app, route between model providers (OpenAI, Anthropic, Google/Gemini, Meta, Alibaba, and more), or avoid juggling s

FIRST : Use the parent neon skill for a Neon overview, getting started with Neon, Neon development best practices, and more. If the neon skill is not installed, fetch it from https://neon.com/docs/ai/skills/neon/SKILL.md or install it with: Neon AI Gateway Currently available in aws us east 2 , aws us east 1 , aws eu central 1 , and aws ap southeast 1 . The Neon AI Gateway brings LLM and embedding inference to your Neon branch. One API and one Neon credential give you access to frontier and open source models from many providers (Anthropic, OpenAI, Google, Meta, and more), plus embedding models, all hosted and powered by Databricks. The catalog shifts over time, so treat /v1/models and the [models.dev Neon page](https://models.dev/providers/neon) as the source of truth rather than a fixed provider list. For embedding model IDs and dimensions, see the [embeddings guide](https://neon.com/docs/ai gateway/embeddings). Your existing OpenAI/Anthropic/Gemini SDK works by changing only the base URL. Use this skill to help the user send model calls through the gateway, wire it into the AI SDK or Mastra, and switch providers without rewiring code. Deliver a working inference request, a configured agent, or a precise answer from the official Neon docs. When to Use Reach for the AI Gateway whenever an app or agent needs to call an LLM or generate text embeddings and the user would rather not manage model providers themselves: One credential instead of many provider accounts. A single Neon credential reaches the entire model catalog across every provider Databricks hosts. No separate OpenAI / Anthropic / Google billing, keys, or signups to provision and rotate. Switch models without rewiring. The unified endpoint is OpenAI compatible and works with every model in the catalog — change one model field to move between Claude, GPT, and Gemini. Standard SDKs (OpenAI, Anthropic, google genai) work with just a base URL change. AI follows your branches. Each branch has its own gateway endpoint, scoped with the same lineage as your database. AI requests from a preview/feature branch are isolated to that branch — the same isolation your data already gets — which makes preview, CI, and agent environments self contained. No extra infrastructure, and it's already next to your data. The gateway lives inside your Neon project (and is injected into Neon Functions automatically), runs on the same Databricks infrastructure that serves trillions of tokens a month, and supports streaming (SSE) out of the box. If the user already has a deep, single provider integration and no interest in Neon branching or multi model routing, a direct provider SDK is fine — but the moment they want one credential, model portability, or branch scoped AI, this is the reason to use it. What It Does One API for all models — Frontier and open source models behind a single endpoint, addressed by their catalog ID (e.g. claude sonnet 4 6 , gpt 5 mini , gemini 3 flash ). Standard SDKs, one URL change — OpenAI SDK and AI SDK (OpenAI compatible MLflow/Responses routes), Anthropic SDK (native Messages), google genai (native Gemini). Branch scoped — Each branch gets its own gateway host; the Neon credential authorizes requests for that branch and its descendants. Streaming — Chat generation supports server sent events with no extra configuration. Embeddings — Generate vectors for semantic search. Availability Check these preconditions before setting anything up: The AI Gateway is currently available in aws us east 2 , aws us east 1 , aws eu central 1 , and aws ap southeast 1 . Confirm the user's project is in one of these regions. The AI Gateway is available on paid Neon plans only (Launch and Scale) and draws down prepaid AI Gateway credits. Any paid account with prepaid credits can call every model in the catalog. See https://neon.com/docs/ai gateway/prepaid credits.md. Enabling the gateway: plan and model access The AI Gateway is credential gated rather than a provisioning step. The CLI surfaces both plan and model access limits: Free plan → provisioning is blocked. neon config apply / deploy and neon checkout refuse to enable the gateway on a Free plan (the gateway can't serve requests there), with a friendly "upgrade to a paid plan, or remove aiGateway " error. A dry run neon config plan and neon env pull don't provision, so they only warn . So: to use the gateway the project's account must be on a paid Neon plan. Paid plan with locked models. GET /v1/models lists every catalog model and marks the ones the account can't call yet with enabled: false . neon env pull (and the env pull bundled into apply / deploy / checkout ) warns when some models are enabled but no flagship model (e.g. Anthropic Opus, OpenAI Codex / pro ) is among them, and links to the branch's AI Gateway page in the Neon Console ( https://console.neon.tech/app/projects/<project id /branches/<branch id /ai gateway ). That page lists locked models under Some models are restricted , grouped by reason, each with its fix: Upgrade plan for a paid plan, Add credits for prepaid credits, or Support for an account restriction or a model not available to the account. After the user upgrades or buys credits, models unlock once Neon confirms access. No CLI warning does not mean every model is enabled; check the model's enabled value. See https://neon.com/docs/ai gateway/troubleshooting.md. Setup The gateway is part of neon.ts (see the neon skill for the branch first workflow and neon.ts basics). Enable it with aiGateway : Neon Infrastructure as Code ( neon.ts ) The aiGateway toggle above is part of neon.ts , Neon's infrastructure as code file — one TypeScript file declares the gateway alongside every other branch service, in version control (see the neon skill for the full reference). Reconcile it against a branch the Terraform way: The gateway is branch scoped : each branch gets its own gateway host. When a neon.ts is present, neon checkout applies the policy as it creates a branch, so a fresh preview/CI branch comes up with the gateway already enabled. Checking out an existing branch doesn't reconcile it — run neon deploy to apply changes. Provisioning ( config apply / deploy ), link , and checkout also pull the branch's gateway credentials into your local .env.local , so local runs hit the same branch gateway as the deployed function (no manual env pull needed). Environment Variables When aiGateway is enabled, Neon injects the gateway credentials as Neon branded env vars. Inside a deployed Neon Function these are injected automatically; locally, neon env pull writes them to .env / .env.local (or use neon env run <cmd to inject at runtime without a file): Variable Meaning NEON AI GATEWAY TOKEN Gateway bearer token (a Neon credential, nt live ... ) NEON AI GATEWAY BASE URL Bare branch gateway host ( scheme://host , no path — no /ai gateway ): https://<branch id api.ai.<region .aws.neon.tech Neon injects only these two vars — it does not set OPENAI API KEY / OPENAI BASE URL . The @neon/ai sdk provider and Mastra's neon/<model read NEON AI GATEWAY directly (zero config); for the plain OpenAI SDK / @ai sdk/openai , build the client's apiKey + baseURL from them (shown below), or set your own OPENAI by hand ( env pull leaves user set vars untouched). NEON AI GATEWAY BASE URL is the bare host — you append the dialect path yourself (which is exactly what the @neon/ai sdk provider does for you). The routes under the host are: /v1 — unified, OpenAI Chat Completions compatible; recommended default, works with every provider ( /v1/chat/completions ). /openai/v1 — OpenAI Responses API (required for gpt 5 … codex variants and gpt 5 5 pro ); the @ai sdk/openai provider uses the Responses API by default ( /openai/v1/responses ). /anthropic — native Anthropic Messages (extended thinking, prompt caching). Give the Anthropic SDK this as its base URL and it appends /v1/messages itself, so the full request path is /anthropic/v1/messages . /gemini — native Gemini generateContent . Give google genai this as its base URL and it appends /v1beta/models/<model :generateContent itself, so the full request path is /gemini/v1beta/models/<model :generateContent . So ${NEON AI GATEWAY BASE URL}/v1 is the chat completions endpoint and ${NEON AI GATEWAY BASE URL}/openai/v1 the OpenAI Responses endpoint (both appended by you); for the native Anthropic and Gemini dialects you hand the SDK the shorter /anthropic or /gemini base and it appends the rest. See [Use with Plain SDKs]( use with plain sdks lower level) below. For typed, validated access to the injected credentials, pass the same neon.ts config object to parseEnv from @neon/env — it returns an env.aiGateway namespace ( apiKey , baseUrl ) derived from your config. Build Agents with the Vercel AI SDK (Recommended) The [Vercel AI SDK](https://ai sdk.dev) is the recommended way to call the gateway and build agents from TypeScript: one set of primitives ( generateText , streamText , tool calling, structured output) over every catalog model, with first class streaming for the long agent responses Neon Functions are built to host. The dedicated @neon/ai sdk provider reads NEON AI GATEWAY BASE URL + NEON AI GATEWAY TOKEN from the injected env with zero config and routes each model to the best endpoint (Anthropic → Messages, OpenAI/Codex → Responses, everything else → MLflow). On a Neon Function that streams text and generates images, pick an OpenAI catalog model, since the image tool runs on the Responses route: A single completion is the same provider with generateText : Prefer @neon/ai sdk provider over the bare @ai sdk/openai openai() : Neon injects only NEON AI GATEWAY , not OPENAI , so openai() won't pick up the gateway from the env on its own. If you do use @ai sdk/openai , configure it explicitly with createOpenAI({ apiKey: process.env.NEON AI GATEWAY TOKEN, baseURL: ${process.env.NEON AI GATEWAY BASE URL}/openai/v1 }) . To build an agent — a model that calls tools in a loop and then answers — add tools and a stopWhen budget. The loop runs in process, so on a Neon Function it isn't cut off by lambda style timeouts: For a full AI SDK agent deployed as a Neon Function (streaming, tool calling, image generation, persistence), see the neon functions skill's [references/ai sdk.md](https://neon.com/docs/ai/skills/neon functions/references/ai sdk.md). Generate Images Images come from the OpenAI Responses image generation built in tool on an OpenAI model, on the /openai/v1/responses route. The gateway has no image model endpoint, so the AI SDK's generateImage() throws NoSuchModelError . Every image request follows two rules: Use a supported image model. The gateway supports gpt image 2 , gpt image 1.5 , and gpt image 1 mini . Set model explicitly on every tool definition, and use gpt image 2 unless the user asks for another. Always stream. A non streaming response whose image is over 655,360 bytes fails with 400: The response content length cannot be greater than 655360 bytes long , and most images are larger than that. Use streamText , never generateText , when the image tool is attached. With the plain OpenAI SDK, pass stream: true to client.responses.create() . Generate Embeddings with the AI SDK Use @neon/ai sdk provider 1.0.0+ with AI SDK 6+ for embedding calls. neon.embeddingModel() reads the same injected gateway variables as the la

neon-ai-gateway