Databricks Model Serving endpoint lifecycle and ops. Use when asked to: CRUD serving endpoints (CLI or MLflow Deployments client); configure traffic routing for A/B / canary deploys and zero-downtime version swaps; retrieve OpenAPI schemas; inspect logs, metrics, or permissions; manage AI Gateway ra

Model Serving Endpoints FIRST : Use the parent databricks core skill for CLI basics, authentication, and profile selection. Model Serving provides managed endpoints for serving LLMs, custom ML models, and external models as scalable REST APIs. Endpoints are identified by name (unique per workspace). Endpoint Types Type When to Use Key Detail Pay per token Foundation Model APIs (Llama, GPT 5, Claude, Gemini, etc.) Uses system.ai. catalog models, pre provisioned in every workspace. Discover at runtime — see [Foundation Model API endpoints]( foundation model api endpoints) below. Provisioned throughput Dedicated GPU capacity Guaranteed throughput, higher cost Custom model Your own MLflow models or containers Deploy any model with an MLflow signature Endpoint Structure Served Entities : Each entity references a model (from Unity Catalog or MLflow) with scaling parameters. Get the entity name from served entities[].name in the get output — needed for build logs and logs commands. Traffic Config : Routes requests across served entities by percentage (for A/B testing, canary deployments). State : Endpoints transition NOT READY → READY after creation or config update. Poll via get to check state.ready . CLI Discovery — ALWAYS Do This First Do NOT guess command syntax. Discover available commands and their usage dynamically: Run databricks serving endpoints h before constructing any command. Run databricks serving endpoints <subcommand h to discover exact flags, positional arguments, and JSON spec fields for that subcommand. Create an Endpoint Do NOT list endpoints before creating. Discover available Foundation Models: see [Foundation Model API endpoints]( foundation model api endpoints) below for the runtime list snippet and default picking rules. You can also check the system.ai catalog in Unity Catalog, or run databricks serving endpoints list profile <PROFILE to see what's deployed in the workspace. Use databricks serving endpoints get open api <ENDPOINT NAME profile <PROFILE to inspect a specific endpoint's API schema. Long running operation; the CLI waits for completion by default. Use no wait to return immediately, then poll: For provisioned throughput or custom model endpoints, run databricks serving endpoints create h to discover the required JSON fields for your endpoint type. MLflow Deployments client (Python alternative) mlflow.deployments.get deploy client("databricks").create endpoint(name=..., config={...}) takes the same JSON shape as the CLI. Two gotchas: tags= is a top level kwarg , NOT a field inside config . Same [{key, value}] shape as serving endpoints patch add tags . traffic config.routes[].served model name = "<model <version " (e.g. "turbine failure 3" ). The API auto derives this from the entity, but you reference the exact string in traffic config — get the format wrong and the route silently doesn't match. Zero downtime version swap To roll an endpoint to a new model version: repoint the alias and call update endpoint with the new served entities + matching traffic config . Missing either half is the common bug — alias only doesn't update the endpoint; update endpoint only leaves the alias pointing at the old version. The CLI equivalent is databricks serving endpoints update config <NAME json '...' . Either way, poll both state.ready and state.config update afterward — see Endpoint Readiness below. Endpoint Readiness After create or update config , the endpoint provisions compute and loads the model. Do not query the endpoint until it is ready. Two state fields matter and they mean different things: state.ready — READY once the endpoint has any working config. Stays READY during a version swap. state.config update — NOT UPDATING once the current config update finishes; IN PROGRESS during a version swap. A loop watching only state.ready will say "ready" mid version swap while the old version is still serving. Poll both: Provisioning may take several minutes. Provisioned throughput endpoints take the longest (GPU allocation). Queries to endpoints that are not yet READY return 404 or 503. Query an Endpoint Chat / agent endpoints use the messages array: Classical ML endpoints use dataframe records (one record per row): Use stream for streaming responses on chat endpoints. For embeddings or other custom schemas: use get open api <ENDPOINT NAME first to discover the request/response shape. Get Endpoint Schema (OpenAPI) Returns the OpenAPI 3.1 JSON schema describing what each served model accepts and returns. Use this to understand an endpoint's input/output format before querying it. The schema shows paths per served model (e.g., /served models/<model name /invocations ) with full request/response definitions including parameter types, enums, and nullable fields. Other Commands Run databricks serving endpoints <subcommand h for usage details. Task Command Notes List all endpoints list Get endpoint details get <NAME Shows state, config, served entities Delete endpoint delete <NAME Update served entities or traffic update config <NAME json '...' Zero downtime: old config serves until new is ready Rate limits & usage tracking put ai gateway <NAME json '...' Update tags patch <NAME json '...' Build logs build logs <NAME <SERVED MODEL Get SERVED MODEL from get output: served entities[].name Runtime logs logs <NAME <SERVED MODEL Metrics (Prometheus format) export metrics <NAME Permissions get permissions <ENDPOINT ID ⚠️ Uses endpoint ID (hex string), not name. Find ID via get . What's Next Integrate with a Databricks App After creating a serving endpoint, wire it into a Databricks App. Step 1 — Check if the serving plugin is available in the AppKit template: If the output includes a serving plugin, scaffold with: Step 2 — If no serving plugin , add the endpoint resource manually to an existing app's databricks.yml : And inject the endpoint name as an environment variable in app.yaml : Then wire the endpoint into your app via the serving() plugin or a custom route in onPluginsReady . For the full app integration pattern, use the databricks apps skill and read the [Model Serving Guide](../databricks apps/references/appkit/model serving.md). Develop & deploy new models This skill is ops focused (manage existing endpoints). For the dev side flow — training, MLflow tracking, UC registration, custom PyFunc authoring, and hand rolled ResponsesAgent code — see [databricks ml training](../databricks ml training/SKILL.md) (experimental). Foundation Model API endpoints Pay per token, pre provisioned in every workspace. New models land regularly and a static skill list goes stale fast — always list at runtime instead of hard coding names . Filter by the databricks name prefix AND by the served entity being in system.ai. (other endpoints like databricks app template serving share the prefix but aren't FM API endpoints). Defaults when the user doesn't specify : pick the highest numbered Claude Sonnet for agents, the highest numbered codex max for code, databricks gte large en for embeddings — resolve actual names from the live list above. Off platform streaming For apps deployed outside Databricks Apps (Vercel, AWS, standalone Node.js) hitting Databricks AI Gateway with Vercel AI SDK v6, see [references/off platform streaming.md](references/off platform streaming.md). For AppKit based apps, use the databricks apps skill's built in serving plugin instead. Troubleshooting Error Solution cannot configure default credentials Use profile flag or authenticate first PERMISSION DENIED Check workspace permissions; for apps, ensure serving endpoint resource declared with CAN QUERY Endpoint stuck in NOT READY Wait up to 30 min for provisioned throughput. Check build logs: build logs <NAME <ENTITY NAME (get entity name from get output → served entities[].name ) RESOURCE DOES NOT EXIST Verify endpoint name with list Query returns 404 Endpoint may still be provisioning; check state.ready via get RATE LIMIT EXCEEDED (429) AI Gateway rate limit; check put ai gateway config or retry after backoff Endpoint missing from the Serving UI after deploy UI filter defaults to "Owned by me". Deploy jobs run as a service principal, so the endpoint is hidden until you switch to "All". databricks serving endpoints list always shows it.

databricks-model-serving