Skip to content
S&L Labs

Go-live review

Go-live review for agents, RAG and MCP

Before your system goes live or gets more permissions, I review its architecture and code and run targeted tests against a staging environment. You get prioritized findings and a target architecture your team can implement.

Scope
One agent, RAG or MCP system with its tools, data sources and approvals. For MCP this includes tool poisoning, rug pulls and the supply chain of your MCP servers. We agree on the exact scope in writing during the intro call.
Effort
5 to 8 days at a fixed price. The effort depends mainly on how many tools the system has, how many of them can write and whether several tenants or agents are involved. You receive the proposal after the intro call.
Outcome
A report with prioritized findings, proposed changes and a target architecture, plus the attack tests as a test suite (evals) that your team runs in CI on every release, and a closing session with your team.

Common stacks

Apps and builders
  • Lovable
  • Bolt
  • v0
  • Cursor
  • Replit
  • Claude Code
  • GitHub Copilot
Agent platforms
  • Copilot Studio
  • Microsoft 365 Copilot
  • Salesforce Agentforce
  • n8n
  • Dify
Frameworks and protocols
  • MCP
  • LangGraph
  • LangChain
  • LlamaIndex
  • OpenAI Agents SDK
  • Claude Agent SDK
  • Vercel AI SDK
  • Haystack
  • CrewAI
Models and platforms
  • Claude
  • OpenAI
  • Azure OpenAI
  • Amazon Bedrock
  • Gemini
  • Mistral
  • Aleph Alpha
  • Llama
  • Ollama
  • vLLM
  • Langdock
Data and operations
  • Qdrant
  • pgvector
  • Pinecone
  • Weaviate
  • Elasticsearch
  • Langfuse
  • LiteLLM
Testing tools
  • promptfoo
  • garak
  • PyRIT

Product names are used for description only and imply no partnership.

Who the review is for

Your team builds an agent, a RAG application or an MCP server itself and runs it in your own product or in-house. The system reads untrusted content, accesses internal data or writes to other systems. It is about to go live or to get more permissions.

When another format fits better

  • The system is still at the design stage. Without code and a staging environment there is little to review. The threat modeling workshop fits better: we define the boundaries before they get built.
  • You use an off-the-shelf assistant without your own tools, such as Copilot or ChatGPT Enterprise. Then the work is about configuration and usage rules. I help with that on slworks.de (in German).
  • You are looking for a product. I do not sell a filter platform or any software. You get a report and a target architecture that your team implements.
  • You need a seal of approval for customers or an audit. That is the job of a testing firm or a certification body.
  • You want many agents reviewed in one go. A review covers one system. With several systems, we talk about the order.

Process

How a go-live review works

  1. Intro call and scope

    We clarify which system it is, what prompts the review and how far along it is. The proposal then sets out scope, rules of engagement and dates in writing.

  2. Reading

    I read the architecture, code, tool definitions, prompts and configuration. From this I build the threat model and the permission map per tool.

  3. Sessions with the team

    In two to three sessions I go through data flows, permissions and approvals with the engineers who built the system. That is where the questions get answered that the code alone cannot answer.

  4. Targeted tests against staging

    I attack the system where agents are typically vulnerable, with prompt injection, tool misuse and attempts to exfiltrate data, manually and automated, with variants of known attacks (fuzzing). Because language models do not always answer the same way, I run every test several times and record how each finding can be reproduced. Testing happens only in staging and only within the scope approved in writing.

  5. Report and target architecture

    You get the findings sorted by impact, each mapped to OWASP Agentic and with a concrete proposal for the change. In a closing session I answer your team’s questions.

  6. Follow-up check optional

    On request, I check the implementation again after a few weeks and repeat the tests that led to findings.

Where I check

  1. Person
  2. Checkpoint: Identity
  3. Agent
  4. Untrusted content Documents, web pages, emails, flows into the agent
  5. Checkpoint: Permissions per tool
  6. Tools and MCP servers
  7. Checkpoint: Approval with diff and log
  8. Data and systems
  • Trust boundary
  • Checkpoint in the review

Prerequisites

What you provide

  • Read access to the system’s repositories
  • A staging environment with test accounts whose roles match those in production
  • A written test authorization that sets scope, time window and excluded systems
  • A contact person on the team who answers questions about architecture and operations

I do not need production data. Access is limited to the duration of the review.

Methodology

What the review is based on

How it compares

Go-live review and classic pentest

The two complement each other. If your system also needs a penetration test, I describe in the report the scope a testing firm should cover.

Aspect Go-live review Classic pentest
Perspective From the inside. Architecture, code, permissions and data flows, plus targeted attacks on the weaknesses that are typical for agents. Mostly from the outside. Attacks on the application, interfaces and infrastructure.
Outcome Prioritized findings with proposed changes and a target architecture your team can implement. Vulnerabilities found, with evidence and rating, usually with remediation advice.
Scope One agent, RAG or MCP system with its tools, data sources and approvals. Network, servers, web applications and interfaces within the agreed scope.
Who A software architect who builds and runs such systems himself. A specialized testing firm with its own team.

Sample report

Excerpt from a report

The excerpt shows the structure and level of detail of the findings. System, findings and test results are invented.

Fictional example System: support agent with CRM and email tools
  1. F-01 Severity: High ASI03 Identity and Privilege Abuse

    The CRM tool runs under a service account

    Description
    The crm_search tool queries the CRM with a technical account that can read all customer records. The permissions of the support staff member using the agent play no part in the query.
    Impact
    Through the agent, records of other teams can be retrieved that the support staff member could not see in the CRM directly.
    Proposed change
    The tool calls the CRM with the identity of the requesting person, and the CRM checks the permissions. An isolation test per role covers this.
  2. F-02 Severity: High ASI01 Agent Goal Hijack

    Incoming emails, CRM data and sending share one context

    Description
    The agent reads incoming customer emails, has access to CRM data and can write to any address with send_email. In testing, a crafted email got the agent to send customer data to an outside address in 3 of 10 runs.
    Impact
    Anyone who sends an email from outside can exfiltrate CRM data without having access to the system.
    Proposed change
    The server only accepts the ticket’s address as recipient. Sending to other addresses goes through an approval that shows recipient and text in plain text.
  3. F-03 Severity: Medium ASI09 Human-Agent Trust Exploitation

    The approval shows the model’s reasoning instead of the email

    Description
    Before sending, the dialog shows the agent’s reasoning and a send button. Recipient and full text only become visible after another click.
    Impact
    The approver confirms emails without having seen their recipient or content. A manipulated reasoning makes approval more likely.
    Proposed change
    The dialog shows recipient, subject and text in plain text and highlights deviations from the ticket address. The model’s reasoning is removed.

Contact

Request a go-live review

In the intro call we clarify whether a review fits your system and what scope makes sense. You then receive a fixed-price proposal.