Configure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: semantic caching, token limit, content safety, load balancing, AI model governance, MCP rate limiting, jailbreak detection, add Azure OpenAI backend, add AI Foundry model, test AI gateway, LLM policies, confi

Azure AI Gateway Configure Azure API Management (APIM) as an AI Gateway for governing AI models, MCP tools, and agents. To deploy APIM , use the azure prepare skill. See [APIM deployment guide](https://learn.microsoft.com/azure/api management/get started create service instance). When to Use This Skill Category Triggers Model Governance "semantic caching", "token limits", "load balance AI", "track token usage" Tool Governance "rate limit MCP", "protect my tools", "configure my tool", "convert API to MCP" Agent Governance "content safety", "jailbreak detection", "filter harmful content" Configuration "add Azure OpenAI backend", "configure my model", "add AI Foundry model" Testing "test AI gateway", "call OpenAI through gateway" Quick Reference Policy Purpose Details azure openai token limit Cost control [Model Policies](references/policies.md token rate limiting) azure openai semantic cache lookup/store 60 80% cost savings [Model Policies](references/policies.md semantic caching) azure openai emit token metric Observability [Model Policies](references/policies.md token metrics) llm content safety Safety & compliance [Agent Policies](references/policies.md content safety) rate limit by key MCP/tool protection [Tool Policies](references/policies.md request rate limiting) Get Gateway Details Test AI Endpoint Common Tasks Add AI Backend See [references/patterns.md](references/patterns.md pattern 1 add ai model backend) for full steps. Apply AI Governance Policy Recommended policy order in <inbound : 1. Authentication Managed identity to backend 2. Semantic Cache Lookup Check cache before calling AI 3. Token Limits Cost control 4. Content Safety Filter harmful content 5. Backend Selection Load balancing 6. Metrics Token usage tracking See [references/policies.md](references/policies.md combining policies) for complete example. Troubleshooting Issue Solution Token limit 429 Increase tokens per minute or add load balancing No cache hits Lower score threshold to 0.7 Content false positives Increase category thresholds (5 6) Backend auth 401 Grant APIM "Cognitive Services User" role See [references/troubleshooting.md](references/troubleshooting.md) for details. References [ Detailed Policies ](references/policies.md) Full policy examples [ Configuration Patterns ](references/patterns.md) Step by step patterns [ Troubleshooting ](references/troubleshooting.md) Common issues [AI Gateway Samples](https://github.com/Azure Samples/AI Gateway) [GenAI Gateway Docs](https://learn.microsoft.com/azure/api management/genai gateway capabilities) SDK Quick References Content Safety : [Python](references/sdk/azure ai contentsafety py.md) [TypeScript](references/sdk/azure ai contentsafety ts.md) API Management : [Python](references/sdk/azure mgmt apimanagement py.md) [.NET](references/sdk/azure mgmt apimanagement dotnet.md)

azure-aigateway