Every team deploying agents hits the same procurement question: which gateway connects agents to our internal systems safely? Not just “does it support MCP” — does it enforce permissions, return typed errors, log audit events, and expose cost clearly? That is what a practical evaluation measures, not what a vendor’s landing page claims.

As of September 2026, the vocabulary is settling: MCP is the standard plug (https://modelcontextprotocol.io), but the gateway — Kong Gateway (docs.konghq.com/gateway, v3.9+), Envoy Gateway (envoyproxy.io/docs/gateway), Apache APISIX (apisix.apache.org) — is still the decision point. Anthropic’s evaluation framework (2026) separates graders by dimension (deterministic_tests, llm_rubric, state_check, transcript, latency); the same separation applies here. Enterprise fit (SSO, RBAC, audit) is a gate, integration effort (typed errors, docs) is a rubric, reliability (observed failures) is a contract, cost clarity is its own grader (cost per task, Artificial Analysis, Sep 2026 — see Intelligence vs Cost per Task chart), and documentation quality rounds out the rubric. No single score captures any gateway.

A builder in a community forum put it plainly: you are really evaluating two things at once — the protocol (does it connect?) and the gateway (does it enforce, verify, and explain?). This guide tests both.

What to check before buying (the 5 graders)

Use this rubric for any gateway you are considering — Kong, Envoy, APISIX, or a newer MCP-specific option. Score each dimension 1-5 based on what you can verify, not what the docs claim.

1. Enterprise fit (gate) — Does the gateway enforce auth_blocked before routing? Does it support your SSO provider (SAML, OIDC, or LDAP)? Is RBAC enforced at the gateway layer, or delegated to the upstream service? A gateway that skips the audit log (audit_event) is not a gateway — it is a protocol adapter that skips the harness.

2. Integration effort (rubric) — Are errors typed (not_found, rate_limited) or thrown exceptions? Does the schema reference cover all endpoints with versions? How long does “first working call” take with the docs alone? Kong’s docs (docs.konghq.com) are versioned and searchable; Envoy Gateway’s (envoyproxy.io) cover the core but leave some MCP-specific endpoints undocumented; APISIX (apisix.apache.org) requires plugin documentation for full coverage.

3. Reliability (contract) — After create_user(), does the audit log show the event? Does retry corrupt state? Does the gateway handle errors deterministically? This is the same state_check grader from the evaluation framework — verify it with a test script (generate_invoice() + audit_event), not a marketing benchmark.

4. Cost clarity (grader) — Not ”$ per query” but cost per task: the weighted average cost per verified outcome (artificialanalysis.ai/models, Sep 2026), including integration time and missing audit hooks. A gateway with $0 advertised price that requires a full-time engineer for audit integration is not free.

5. Documentation (rubric) — Versioned docs (v3.9, v2.3 etc.), searchable error reference, complete schema, and no missing endpoints. Check the docs for the version you will deploy — not the “latest” page, which may reference unreleased features.

Real gateway categories (as of Sep 2026)

These are the categories you will actually compare — not fictional scores, but real architectures with documented trade-offs:

  • Kong Gateway (konghq.com, v3.9+) — Full plugin ecosystem (SSO, RBAC, audit, rate limiting). Integration takes a senior engineer. Audit hooks (security_logs) exist but require custom plugin configuration. Cost: transparent licensing; integration cost is the hidden variable.
  • Envoy Gateway (envoyproxy.io) — Fast setup; typed errors (not_found) are first-class. RBAC is weaker at the gateway layer than Kong — filter at retrieval, not after routing. Documentation covers core well; MCP-specific endpoints need upstream confirmation.
  • Apache APISIX (apisix.apache.org) — Open-source; strong plugin support for auth and audit. Schema docs require plugin-level verification. Integration effort varies by plugin version — verify the exact plugin version (v2.14 etc.) against the gateway version.
  • Newer MCP-specific gateways — Emerging; often skip native SSO or audit. Treat them as protocol adapters until the harness contracts (auth_blocked, audit_event) pass deterministically.

No gateway covers all five dimensions well out of the box. That is not a vendor failure — it is the architecture: the harness (gateway + agent + permission model) must match your security review, not the vendor’s marketing page. A framework you can run in a week and apply continuously is more valuable than a perfect comparison that takes six months.

Note: This review applies the rubric above to the gateway category, not a specific tested version. Product-level scores with version stamps (v3.9, v2.3 etc.), observed failure modes, and last-verified dates require a separate T4 comparison — see site workflow.

Last verified: Sep 2026. References checked: Kong docs (konghq.com/gateway), Envoy Gateway docs (envoyproxy.io/docs/gateway), Apache APISIX docs (apisix.apache.org), Anthropic eval framework (2026), Artificial Analysis (artificialanalysis.ai/models).

Building the framework for your team

Start with the same three questions the evaluation framework asks:

  1. What hard failures must never pass? (enterprise fit gate — auth_blocked, SSO match)
  2. What state must exist after the agent acts? (state_check — audit_event in log, no duplicate records)
  3. How does behavior drift? (transcript — retry patterns, latency, error frequency over time)

Then aggregate: score each gateway against the 5 graders using the contracts you can actually test (auth_blocked event, create_user() state, not_found error). A gateway that passes contracts 1 and 3 but scores low on 2 or 4 is a governance success but an engineering burden. A gateway scoring low on 1 or 3 is not a gateway — it skips the harness.

A framework that takes one week to build and runs for months is worth more than a perfect vendor comparison that takes half a year. The rubric is the operating model.

Further reading