What even is an MCP gateway?

There is a lot to learn in MCP land. There's a lot of new vocabulary, not everyone is sure what all the words mean, and in some cases people don't even agree on what the words mean. So on this episode of the Fast MCP podcast we tried something different and went back to school. Radhika Gulati (Product Marketing) sat down with Jeremiah Lowin (CEO) to draw out the confusing parts of MCP, starting with one of the squishiest terms of all: the MCP gateway.

What follows is an edited version of that conversation.

First, what even is a gateway?

Before you can talk about an MCP gateway, it helps to be clear about what a gateway is in the first place.

In an enterprise, a gateway is a way to manage how a lot of API endpoints are exposed: to the world, to internal consumers, to customers. It lets you control who can access what, when, and how. It's called a gateway because there's a fence that carves off the infinite multitude of potential APIs and endpoints you could hit, leaving only one blessed way to go access them.

A traditional API gateway might say Radhika is allowed to talk to this API, Jeremiah is allowed to talk to that one, and both of us can talk to a third. That's access control. A gateway also governs how APIs are exposed: under what URL you connect, and sometimes how two services living on different pieces of infrastructure get stitched together into a single endpoint.

So you can imagine why there's a desire to extend those concepts to MCP. If an MCP server is nothing more than an agent-native API, then the best practices for deploying APIs in an enterprise should carry over to deploying MCP servers in an enterprise. That's why we get this persistent request for MCP gateways, even though they're a little squishy to define once you get into it.

Why MCP needs one too

API gateways and MCP gateways are very similar, but when you talk to customers, "gateway" means a variety of things, to a degree that the list keeps growing.

Many customers start from the API gateway already deployed in their organization and reuse the same phrasing: IDP integrations, RBAC, access control, all the things that have to do with who is allowed to see something. Talk to another organization and the gateway is about what is being exposed, which is where the stitching together of APIs and a single endpoint for multiple things comes from. Talk to a third group and they're describing more advanced concepts: injecting middleware as a control point so the organization can change business logic, screen out certain payloads, or run requests through a PII filter to scrub personal information.

All of these are things you can do once you have a control point between the raw API landscape and the consumer. So when we talk about a gateway, we're really referring to everything you can do when you have centralized governance and control of the API surface area.

Trying to naively carry all of that over from the API world to MCP doesn't quite work, because MCP servers don't behave the way a REST endpoint does. There's baggage customers bring when they say "we want an MCP gateway." We actually started building the wrong thing months ago based on the vocabulary in some of those early requests.

The thing people mix up: gateway vs. registry

A lot of what people call an MCP gateway would more accurately be called an MCP registry. A registry is a list of a whole bunch of servers you can access. It's a real concept in MCP land, and plenty of MCP registries have none of the gateway functionality.

But a lot of the time when someone says "we need an MCP gateway," a registry is what they're actually asking for. So the MCP gateway has come to mean something broader than a traditional API gateway, which is better understood and has more established best practices.

What we've discovered is that you can't really have one without the other. Discovery is just as important as governance.

Mapping the players: servers, registry, gateway

Say you're going to deploy two MCP servers, X and Y, into your organization. There are two big questions a gateway helps answer: discovery and governance.

Discovery is how you find out a server exists. How do you register it into the consciousness of the organization? In MCP land the answer is the registry. You submit your server, the organization blesses your server and mine as two distinct known servers, and now the rest of the company can find out what capabilities they can add to their agents by connecting to them.

MCP discovery is genuinely hard, because you have to connect to the server to ask what it has. Unlike API discovery, where there are best practices for querying an OpenAPI spec, with MCP you handshake with the server and literally ask it what tools it has. So the registry can do more to cache or present that tool surface area, and knowing you'll deploy against a registry also shapes how you author servers so they comply with a known set of tool requirements.

Governance is the second part. Once a server shows up in the registry, I can find Radhika's server, see the URL, and add it to Claude (or my organization does that at an enterprise level). But now the question becomes: am I allowed to access that server? Ideally, if you don't have permission to access a server, the registry doesn't even show it to you. It's all well and good to show someone a thousand servers, but if they can't connect to them, you're just showing them garbage.

That's why you want a deep integration between access control, done at the gateway, and the registry. They should reflect each other.

Access control, down to the individual tool

The gateway is the place that determines whether you're allowed to connect. A well-designed gateway determines not just whether you can connect to a server, but whether you can connect to individual tools within it.

You might have one tool that reads data and another that writes it, and you're only permissioned for the reading tool because you can't write to the backing store. A gateway is a great place to manage that separately, without baking it into every server.

Imagine the alternative, where every server deployed in the organization is responsible for guarding its own access. Now every server needs complicated auth and IDP integrations, has to know who you are and what you can do, has to know how to cache and refresh tokens. It becomes untenable very quickly. A centralized gate lets you publish a server with a lot of functionality and manage who accesses it, and how, away from the server's deployment lifecycle.

Does this apply to third-party servers too?

Yes. One could argue that if you're accessing someone else's server, governed by someone else's auth, the gateway isn't solving a problem. But that's the wrong way to look at it.

We want the same identity to follow you across every server you access as part of the organization. Otherwise you get back doors into information that aren't centrally governed. Ideally both first-party and third-party servers live in the same registry and are governed by the same gateway. Could you hit a third party's hosted server directly? Of course. But the whole point of the gateway is to centrally manage and federate your identity and access, brokering it on behalf of the organization.

Horizon Remix: a virtual server from just the tools you need

Once first-party and third-party servers sit behind the same gateway, you can do really interesting things. In Horizon, our MCP gateway, there's a remix feature: take only the tools you need for a specific use case and build a virtual MCP server that bundles them together.

One tool from your server, one from mine, two from the Linear server, one from the GitHub server, all behind a single URL governed by the same gateway. The user connects to that one endpoint and doesn't have to log in three different times or tell their agent about three different servers. The gateway handles all the routing, identity brokering, and permissioning in one place, with full observability. An enormous headache just disappears.

MCP vs. CLI: individual habits vs. enterprise reality

To an individual, all of this can sound like a lot of oversight for talking to a few servers. That's where it's important to separate the role of MCP in the world from the role of MCP in the enterprise.

The most controversial thing in the MCP space has really been a misunderstanding of who benefits most from MCP, and that's where the "MCP versus CLI" debate comes from. As an individual with full control of your machine, that's a reasonable debate. CLIs are easy to install and useful, both for you and your agent. An MCP server can feel like more headache for less personal benefit.

In the enterprise, we are not even entertaining the idea of installing a bunch of CLIs on people's computers. An enterprise needs central management of what people can do and how. If you can do it remotely, at a distance, you don't even have to care what the machine is. The gateway is part of how you centrally administer business logic, information, and context for agents, without resorting to approaches that make sense for one person but are unworkable across 100, or 1,000, or 50,000 employees.

What FastMCP handles vs. what a gateway handles

So where does FastMCP fit? FastMCP itself doesn't have a ton of what you'd call gateway capabilities, because it's optimized for building a server, while a gateway is optimized for governing access to one or more servers.

If you boiled a gateway all the way down to governing access to a single server, FastMCP's auth primitives become helpful. FastMCP can say that someone with an admin role is allowed to use a given tool, which is a real form of access control. What we've made easy in FastMCP is tailoring business logic to the claims in a user's token without making you responsible for verifying that token. Once your auth is set up, you assume the claims you receive are representative, so you can say "if you're not an admin, you can't see this tool."

That's a perfectly reasonable thing for a server author to enforce. It's very different from saying "the engineering group can access this tool but the marketing group cannot." That starts to feel like a central concern, more appropriate for a gateway. And once you move into the multi-server landscape, FastMCP offers nothing, because it isn't designed to manage a fleet of servers. It's optimized for building one.

"We have MCP sprawl." What should teams do?

Teams asking these questions are asking the right ones. The glib answer is: get an MCP gateway in place. The more constructive answer starts with the end goal. A gateway is a tool, a means to an end, and usually the end is enhancing agent behavior through curated business logic deployed by internal teams.

If a company has MCP sprawl, it's running ahead of its own security and comfort posture. On one hand it's living on the cutting edge and getting real value from experimentation. On the other hand, it's probably doing it in a way that isn't observable, auditable, or controllable.

Consider a concrete failure mode. If I write an MCP server and naively give it an API key that can write to a database, then when you connect to it, by definition it's still using my key with my write access. We wouldn't build traditional APIs this way, not because it isn't possible, but because decades of best practices steer us away from it. With MCP, that pattern is extremely typical right now, because we're all so excited about the productivity gains that we can't wait to plug in more tools.

Pretty soon you've got an email MCP server, and so do I. Should we have two? What if one turns out to be compromised? Will you write a policy that says "only use this server" and hope everyone follows it? It's much easier to make a policy that, at the enterprise level, you access MCP servers through a blessed place. Knowing what's out there is the first step to sanity, which is where the registry comes in, and then you can actually do the governance.

Leaked keys and supply-chain attacks

The alternative to governance is waiting for someone to report that they were using a Joe Schmo MCP server they found on GitHub, it got hit by a supply-chain attack, and their credentials leaked. That's happening all the time now.

A gateway is not automatic immunity. It still needs monitoring. What it gives you is a single place to tell people where to find MCP servers, and a central point to take responsibility for whether a server meets the company's security posture. We wouldn't pretend the gateway magically protects you; if a gateway can host a server and a server can be compromised, no one can make that guarantee.

But because we're hosting servers, and for first-party servers that often means we have the source code, the gateway is a great place to deploy security scanning and bring a different set of best practices to bear. That may not fall under the narrow definition of an MCP gateway, but it's absolutely part of the mandate when you think of the gateway as the entire platform of MCP infrastructure in the organization.

How Prefect runs gateways on itself

We try to keep a generous posture at Prefect: effectively an unlimited budget for AI software, because we have a lot of responsible people who get nervous spending money on anything. If your card runs out, ask for more, go use tools, sign up for things. It's a high-agency, high-trust environment, and because everyone is excited and talking about this, we gravitate toward good tooling and best practices organically.

A lot of that happens by connecting our agents to MCP servers, which produces the same urge for a gateway any organization feels: what servers are we using, and who's using them? If we're paying for a service that supplies an MCP server and no one is using it, that's important to know from an observability standpoint.

The more interesting case is our internal, first-party servers. Radhika wrote one of our first ones, for our CMS, and then one for Gong before Gong had an official MCP server. Because she was worried about cost, she even wrote in code to report how many tokens were being used and how much money was being spent.

That's a great example of a responsibility graduating up the MCP supply chain. Was tracking tokens really the server author's job, or should it live higher up? Wouldn't you rather, as a server author, just focus on getting the business logic right, and take it on faith that when someone connects, the server knows who they are, forwards their credentials correctly, and won't leak conversations? The auth, the access, the observability: those are things you want to take for granted, but they require a service layer to deploy into. Which brings you right back to the utility of the gateway, even for an individual writing a first-party server for their colleagues.

It came up concretely at the time: the server used Radhika's API key, and when colleagues wanted to use the server internally, the question became "do they have to use my key?" That's exactly the kind of business logic that belongs in a gateway, not a server.

Where the spec is going: stateless

There's a new MCP spec coming, and the headline is that it's becoming stateless. That might sound like something nobody cares about, but people who build MCP software, and especially MCP gateways, care deeply.

MCP is inherently stateful today: it tracks a session ID and expects a long-lived connection. That makes it hard to horizontally distribute your servers or your gateway control plane. There are (unofficial) ways to run MCP statelessly, which we do, and then we fake the session. Our servers don't know they're running in stateful mode, because in a horizontally distributed architecture they can't run statefully without crashing when they try to access their own sessions. So we artificially recreate the session dynamics across the lifecycle of multiple requests against the Horizon gateway, so clients that expect a stateful connection still experience one, while the servers actually run statelessly. That's the only way to do a properly scalable, horizontally deployed architecture, and it's been a significant undertaking.

When the spec goes stateless as a whole, it simplifies all of this dramatically for us. If we've done our jobs right, no one will notice, which is the best kind of framework architecture: we took on the hard thing so others don't have to.

There is some nuance. FastMCP supports state in several ways, and one of them keys off a session ID to create persistent state for a user, so a tool used later in a session can reference state written earlier. That's very useful. Going stateless doesn't mean we throw out all server state. The one thing that goes away is the guarantee of a session ID, which is how you know multiple requests belong to the same session even when they hit different servers. So we'll do some work to let users opt back in to a virtual session: write information in one request, come back a day later under the same session ID, and still have your request satisfied, even though the raw spec wouldn't permit it. That's exactly where the gateway layer enables a class of business logic the spec won't make straightforward.

Interceptors and the future of gateways

The other thing to watch is a new working group, tentatively called interceptors, which is a pretty good name for what we'd call a gateway. The idea is to formalize practices around the layer that intercepts requests and responses going in and out of a server, with the opportunity to modify them, interact with them, or disallow them.

If that formalizes, it might produce a spec for what MCP gateways should look like or conform to as they modify things in flight. Honestly, there isn't a great mystery in how a gateway has to work. It's more a question of whether we standardize some useful metadata around it.

MCP vs. CLI vs. API, the verdict

So will it get clearer when to use MCP versus a CLI versus an API? MCP almost has to be the way agents connect to things, because of discovery. We don't want agents having raw access to APIs; learning to use them is hard, and there are attacks involved.

MCP as a protocol is driving real innovations: tool search, code mode, ways to make discovering business logic much more efficient. Could we deliver those over APIs? Of course, but we'd first invent some standard way of doing it, and we'd probably end up calling it MCP. Most critiques of MCP fall into two buckets. One is that it reimplements things that exist in API land and is more restricted in what it can do, which is true, and we think the restriction is a good thing. The other is about context pollution, and those critiques are dated; anyone deploying MCP today in a way that pollutes context is using a 2025-era version of the protocol, or a client out of step with the cutting edge.

APIs are great, companies have many of them, but they're built for programmatic consumption. Now we also have agentic consumption, so it makes sense to have something optimized for that discovery-and-iteration pattern. That's what MCP is for. CLIs, with the exception of a small number of extremely popular, well-vetted ones, have to be installed at the edge and break all the rules of observability and central governance, so it would be surprising if they gained currency as a distribution pattern in the enterprise.

Wrapping up

Our curriculum here was a little all over the place, but the goal was to give some useful soundbites on a genuinely confusing topic. The most honest thing we can say is that we're not always sure either. We're constantly learning, and this is our current best understanding.

If something in the MCP space is confusing you, send it in, and we'll use it as the basis for the next conversation.