Glossary
What is an AI gateway?
An AI gateway is a proxy that sits between an application and an AI model. Requests pass through it, so it can route them, log them, apply limits and check them against rules. It sees the traffic that goes through it. It does not see AI that runs on an employee device and never uses the gateway.
By the AIBound team · Updated 15 November 2026
Key takeaways
- An AI gateway sits between an application and an AI model and checks the traffic that passes through it.
- It cannot see AI that runs on a device and never touches the gateway, such as local models and coding agents.
- A gateway and control on the device cover different parts of the same problem.
Why AI gateways matter
A company that builds or buys AI needs one place to manage how its applications reach AI models. A gateway gives it that place. It can keep a record of every call, set limits and apply rules in one spot. The gap is that a gateway only governs the traffic sent through it, and employees run AI that never goes there.
How an AI gateway works
A gateway is a checkpoint on the road between an application and a model. Every request stops there before it moves on.
- 1 An application sends a request meant for an AI model.
- 2 The gateway checks it, applies the rules, records it and chooses where to send it.
- 3 The model answers, and the answer comes back through the gateway.
AI gateway vs endpoint AI control
An AI gateway governs the traffic that passes through it. Endpoint AI control governs the AI installed and running on the employee device. They look at different places.
| Question | AI gateway | Endpoint AI control |
|---|---|---|
| What does it see? | The traffic that passes through the gateway | The agents, MCP servers, skills and models on the device |
| Where does it sit? | Between an application and an AI model | On the employee device, next to the AI |
| What can it miss? | AI that never sends traffic through it | AI that runs only inside a hosted service |
Sources
- Kubernetes blog: Announcing the AI Gateway Working Group · kubernetes.io
Sources checked 19 November 2026
Examples
AI gateway examples
FAQ
Frequently asked questions
What is an AI gateway?
An AI gateway is a proxy between an application and an AI model. Requests go through it, so it can route them, log them, apply limits and check them against rules. It gives a company one place to manage the AI traffic its applications send, and it governs only what passes through it.
What does an AI gateway do?
A gateway routes requests to the right model, keeps a record of each call and applies limits and rules in one place. It can also check requests and answers against a policy. All of this works on the traffic sent through it. Anything that bypasses the gateway is outside its view.
Is an AI gateway the same as an AI firewall?
They overlap. Both sit in the path of AI traffic and apply rules to it. People tend to say gateway when they mean routing and control, and firewall when they mean filtering, but vendors use both words for similar products. Check what a product sees before you rely on the name.
Does an AI gateway cover AI agents on employee devices?
Only if the agent sends its traffic through the gateway. Local models, coding agents and MCP servers on a device often do not, so the gateway never sees them. Seeing and controlling that AI takes a control on the device itself, where the agents are installed and run.