Skip to content
S&L Labs

Writeups

The LLM gateway is a secrets manager

To use someone else's model access, attackers do not need to attack a model. A key is enough, and keys are concentrated in gateways, platforms and logs. The paths attackers take to get there, and how I secure a gateway against them.

  • LLM gateway
  • Secrets
  • Supply chain
  • Simon Liedtke

My analysis, written up with AI.

An API key for a language model is worth money right away. Whoever has it gets compute on someone else’s bill and, depending on the configuration, access to prompts, responses and connected data. Attackers need neither access to the model nor a prompt injection for this.

The keys are rarely scattered. An LLM gateway bundles the keys of all of a team’s applications; a hosting platform or an agent console bundles the keys of many customers. In October 2026, there were several separately reported cases within a few days in which such places were attacked. I use them here as evidence for the individual attack paths. From them follows a rule I use to build and review gateways: an LLM gateway is a secrets manager and needs the same protection.

1. Five paths to the keys

The gateway’s supply chain

The gateway is a dependency like any other and enters the system via PyPI, npm or a container registry. In October 2026, a manipulated version of LiteLLM was distributed via PyPI, Docker Hub and GHCR. According to the incident report, it was available on PyPI for about 100 minutes and contained a program that harvests credentials. The source code in the repository was clean; what had been manipulated was the path from source code to package. The attackers used developers’ GitHub tokens, disabled branch protection and redirected the release workflow to a repository with a similar name (LiteLLM 2026a). According to a later report, it started with an admin key for an instance of the LiteLLM Agent Platform that had accidentally ended up in a public repository in May and was never revoked (LiteLLM 2026b).

Open instances and endpoints

A gateway or MCP server without authentication is a key that anyone can use. Using a honeypot, Symantec captured the instructions of an autonomous attack agent that searches the internet for unprotected model endpoints, LLM gateways and MCP servers and uses them for its own steps (More and Yenkar 2026).

Platforms and environment variables

If you store keys with a hosting provider or in an agent console, you inherit their incidents. Railway reported unauthorized access to customers’ environment variables and itself classifies it as part of a series of attacks on platform providers. According to the preliminary report, the attackers wanted to obtain API keys from AI providers for distillation attacks (Railway 2026a, 2026b). In the OpenCode console, according to press reports, the stored Anthropic, OpenAI and Google keys of some users were exposed (Toulas 2026).

Logs, API responses and CI

Secrets often leave their storage through paths nobody plans as output. In the LiteLLM case, the first GitHub token, that of an org owner, came from an API response of the company’s own Agent Platform. The attackers obtained two more through CI workflows that sent secrets to an external webhook (LiteLLM 2026b). Logs belong in the same category: a gateway that logs request and response bodies stores everything users and agents write into it, and with coding agents that includes tokens from terminal output, .env files and configurations. LiteLLM initially suspected this path as the source; the later report explicitly corrects that.

Account and billing at the provider

The key does not have to be stolen if the attackers take over the account. Wiz describes how attackers gained access to cloud accounts through phishing, enabled the Gemini API there and attached their own projects to the victims’ billing (Ayal and Maccarone 2026).

Mayadmin key of an instance of theLiteLLM Agent Platform ends upin a public repositoryCountermeasure: prepare rotationAPI response of the Agent Platformreturns the GitHub tokenof an org ownerCI workflows send secretsto an external webhook:two more GitHub tokensRelease chainbranch protection disabled,release workflow redirectedOctobermanipulated version viaPyPI, Docker Hub and GHCRCountermeasure: hash pinning,new versions with a delayProgram harvestscredentialsCountermeasure: egress allowlist
The LiteLLM case as a chain, based on LiteLLM's two reports. In italics, the countermeasures from section 2.

2. Building the gateway like a secrets manager

These are the points I look at first in a review. They correspond to the paths above.

No endpoint without authentication. This applies to the gateway’s API, the admin interface and every MCP server. Internal instances do not belong on the open internet, not even temporarily.

One key per application and environment, with a limit at the provider. The large providers offer projects or workspaces with their own keys and, depending on the provider, hard spending limits or budget alerts. A stolen key from the staging environment then runs into a low limit instead of the company credit card. The gateway itself gets its own limits per application, so that a compromised client cannot use up everyone else’s budget. The account at the provider follows the same rules as any cloud account: strong authentication, few administrators, notifications for new projects and keys.

Logs without content, or with redaction. The safest option is to log metadata only: application, model, token counts, latency, status. If content is needed for debugging, known token formats are replaced before writing:

import logging
import re

SECRET_PATTERNS = [
    re.compile(r"sk-ant-[A-Za-z0-9_\-]{20,}"),                # Anthropic
    re.compile(r"\bsk-(?:proj-)?[A-Za-z0-9_\-]{20,}"),          # OpenAI
    re.compile(r"AIza[A-Za-z0-9_\-]{35}"),                    # Google API
    re.compile(r"gh[opsur]_[A-Za-z0-9]{36,}"),                # GitHub tokens
    re.compile(r"github_pat_[A-Za-z0-9_]{40,}"),              # GitHub fine-grained
    re.compile(r"(?i)(\bbearer\s+)[A-Za-z0-9._~+/\-]+=*"),    # Bearer token
]


def redact(text: str) -> str:
    for pattern in SECRET_PATTERNS:
        if pattern.groups:
            text = pattern.sub(lambda m: m.group(1) + "[REDACTED]", text)
        else:
            text = pattern.sub("[REDACTED]", text)
    return text


class RedactingFormatter(logging.Formatter):
    def format(self, record: logging.LogRecord) -> str:
        return redact(super().format(record))


handler = logging.StreamHandler()
handler.setFormatter(RedactingFormatter("%(asctime)s %(name)s %(message)s"))
logging.getLogger("gateway").addHandler(handler)

The list is an excerpt. It has to cover the formats of your own providers and tools, and it does not catch secrets without a fixed prefix, such as database passwords. That is why it does not replace the decision not to log content. The formatter belongs on every handler that writes content. A filter on a logger would be the obvious alternative, but it does not apply to messages from child loggers and does not cover tracebacks. Existing logs can be searched with the same function: every line that redact changes probably contains a key that should be checked and rotated.

Restrict outbound connections. A gateway has to reach the providers and little else. An egress allowlist at the network level makes sure that an injected program cannot easily deliver its loot.

Update the gateway with a delay. If you pin dependencies by hash (pip install --require-hashes, a lockfile with hashes, Docker images by digest instead of tag) and only adopt new versions after a few days, you would not have installed the manipulated LiteLLM version. For the gateway, this delay is worth more than for any other library, because that is where all the keys are. Security updates then need their own path, deliberately fast. After an incident, a look into site-packages is part of the cleanup: unexpected .pth files and a sitecustomize.py run with every Python start, even long after the package has been replaced.

Prepare rotation before you need it. The admin key that started the LiteLLM case was public for five months without being revoked. After the Railway incident, all customers were advised to rotate all their secrets. That goes quickly when it is clear which keys exist, where they are stored and who can swap them. If you have to find that out during the incident, it takes days.

Platforms are part of the chain. Keys stored with a hosting provider, in an agent console or in a SaaS service get the same rules: their own key, their own limit, their own rotation.

3. The client holds keys too

The gateway is not the only place that holds keys. The machine on which developers work with a coding agent and MCP servers is one too. In October 2026, Pillar Security described a case in the coding agent OpenCode: a remote MCP server could execute code there as soon as the OAuth sign-in was started (Pillar Security 2026). Windows and WSL were affected up to version 1.18.32. There is no CVE, so scanners do not report the case.

The sequence matches the pattern I described in the writeup “MCP attack patterns, mapped to OWASP Agentic 2026” with CVE-2025-6514. During OAuth discovery, the server supplies an authorization_endpoint, and the client opens it. OpenCode did not check this address against the server it came from. The library it used inserted the address unescaped into a PowerShell command in which $(...) gets executed.

There I name two rules from the MCP Security Best Practices as countermeasures: only allow http and https, and never open URLs through a shell. The first rule would not have helped here, because the address was valid https://. The second would have. On top of that comes a third check I always look at in MCP clients: the endpoint has to match the authorization server whose metadata the client loaded.

import webbrowser
from urllib.parse import urlsplit


def origin(url: str) -> tuple[str, str, int]:
    parts = urlsplit(url)
    port = parts.port or {"https": 443, "http": 80}.get(parts.scheme, 0)
    return parts.scheme, (parts.hostname or "").lower(), port


def checked_authorization_url(endpoint: str, issuer: str) -> str:
    if not endpoint.isascii() or any(c <= " " or c in "\\\x7f" for c in endpoint):
        raise ValueError("Invalid character in URL")
    parts = urlsplit(endpoint)
    if parts.scheme != "https":
        raise ValueError("authorization_endpoint is not https")
    if parts.username is not None or parts.password is not None:
        raise ValueError("Userinfo in URL")
    if origin(endpoint) != origin(issuer):
        raise ValueError("authorization_endpoint does not match the issuer")
    return endpoint


def open_authorization(endpoint: str, issuer: str) -> None:
    webbrowser.open(checked_authorization_url(endpoint, issuer))

The character check rejects whitespace, control and non-ASCII characters, backslashes and userinfo, because browsers read such URLs differently from Python. From https://evil.com\@auth.example.com/, urlsplit reads the host auth.example.com, while a browser reads evil.com. The check does not replace avoiding the shell, because $ and parentheses are allowed in URLs. What matters is that the URL never becomes part of a composed shell command. webbrowser.open or a process call with an argument list meets that requirement. The origin comparison is also only as good as the issuer. Under RFC 8414, the issuer in the metadata must match the issuer value from which the client built the metadata URL. Against a malicious server that is its own authorization server, the comparison does not help; there, only avoiding the shell protects you. If an authorization server deliberately uses endpoints on a different host, that host belongs on an explicit allowlist, rather than loosening the check.

What I check here

I trace where provider keys are stored in a system: in the gateway, in environment variables, with hosting providers, in agent consoles and on developers’ machines. For each key, the review clarifies which application uses it, which limit applies, who can rotate it and whether it can show up in logs. On top of that come the gateway’s outbound network paths, the release chain through which the gateway itself enters the system, and, for MCP clients, how they handle OAuth metadata from third-party servers. The result is a prioritized list of findings, each with a proposed change.

Sources