Go-live review
Go-live review for agents, RAG and MCP
Before your system goes live or gets more permissions, I review its architecture and code and run targeted tests against a staging environment. You get prioritized findings and a target architecture your team can implement.
- Scope
- One agent, RAG or MCP system with its tools, data sources and approvals. For MCP this includes tool poisoning, rug pulls and the supply chain of your MCP servers. We agree on the exact scope in writing during the intro call.
- Effort
- 5 to 8 days at a fixed price. The effort depends mainly on how many tools the system has, how many of them can write and whether several tenants or agents are involved. You receive the proposal after the intro call.
- Outcome
- A report with prioritized findings, proposed changes and a target architecture, plus the attack tests as a test suite (evals) that your team runs in CI on every release, and a closing session with your team.
Common stacks
- Apps and builders
-
- Lovable
- Bolt
- v0
- Cursor
- Replit
- Claude Code
- GitHub Copilot
- Agent platforms
-
- Copilot Studio
- Microsoft 365 Copilot
- Salesforce Agentforce
- n8n
- Dify
- Frameworks and protocols
-
- MCP
- LangGraph
- LangChain
- LlamaIndex
- OpenAI Agents SDK
- Claude Agent SDK
- Vercel AI SDK
- Haystack
- CrewAI
- Models and platforms
-
- Claude
- OpenAI
- Azure OpenAI
- Amazon Bedrock
- Gemini
- Mistral
- Aleph Alpha
- Llama
- Ollama
- vLLM
- Langdock
- Data and operations
-
- Qdrant
- pgvector
- Pinecone
- Weaviate
- Elasticsearch
- Langfuse
- LiteLLM
- Testing tools
-
- promptfoo
- garak
- PyRIT
Product names are used for description only and imply no partnership.
Who the review is for
Your team builds an agent, a RAG application or an MCP server itself and runs it in your own product or in-house. The system reads untrusted content, accesses internal data or writes to other systems. It is about to go live or to get more permissions.
When another format fits better
- The system is still at the design stage. Without code and a staging environment there is little to review. The threat modeling workshop fits better: we define the boundaries before they get built.
- You use an off-the-shelf assistant without your own tools, such as Copilot or ChatGPT Enterprise. Then the work is about configuration and usage rules. I help with that on slworks.de (in German).
- You are looking for a product. I do not sell a filter platform or any software. You get a report and a target architecture that your team implements.
- You need a seal of approval for customers or an audit. That is the job of a testing firm or a certification body.
- You want many agents reviewed in one go. A review covers one system. With several systems, we talk about the order.
Process
How a go-live review works
-
Intro call and scope
We clarify which system it is, what prompts the review and how far along it is. The proposal then sets out scope, rules of engagement and dates in writing.
-
Reading
I read the architecture, code, tool definitions, prompts and configuration. From this I build the threat model and the permission map per tool.
-
Sessions with the team
In two to three sessions I go through data flows, permissions and approvals with the engineers who built the system. That is where the questions get answered that the code alone cannot answer.
-
Targeted tests against staging
I attack the system where agents are typically vulnerable, with prompt injection, tool misuse and attempts to exfiltrate data, manually and automated, with variants of known attacks (fuzzing). Because language models do not always answer the same way, I run every test several times and record how each finding can be reproduced. Testing happens only in staging and only within the scope approved in writing.
-
Report and target architecture
You get the findings sorted by impact, each mapped to OWASP Agentic and with a concrete proposal for the change. In a closing session I answer your team’s questions.
-
Follow-up check optional
On request, I check the implementation again after a few weeks and repeat the tests that led to findings.
Where I check
- Person
- Checkpoint: Identity
- Agent
- Untrusted content Documents, web pages, emails, flows into the agent
- Checkpoint: Permissions per tool
- Tools and MCP servers
- Checkpoint: Approval with diff and log
- Data and systems
- Trust boundary
- Checkpoint in the review
Prerequisites
What you provide
- Read access to the system’s repositories
- A staging environment with test accounts whose roles match those in production
- A written test authorization that sets scope, time window and excluded systems
- A contact person on the team who answers questions about architecture and operations
I do not need production data. Access is limited to the duration of the review.
Methodology
What the review is based on
- OWASP Top 10 for Agentic Applications 2026
Framework for the threat model and the report. Each finding is mapped to an ASI identifier where one fits.
- OWASP Top 10 for LLM Applications
Adds model-level risks to the agentic list, in particular excessive agency (LLM06) and prompt injection (LLM01).
- ISO/IEC 27090:2026
Guidance on security threats to AI systems. Adds the perspective of international standardization to the OWASP lists.
- MITRE ATLAS
Knowledge base of attack techniques against AI systems. I derive the test cases for staging from it.
- BSI: Basisschutz gegen Indirect Prompt Injections in dokumentbasierten LLM-Chats
Baseline protection against indirect prompt injection in document-based LLM chats, published in German by the German Federal Office for Information Security, version 1.0 of September 14, 2026. The yardstick for systems that load documents into the model’s context.
How it compares
Go-live review and classic pentest
The two complement each other. If your system also needs a penetration test, I describe in the report the scope a testing firm should cover.
| Aspect | Go-live review | Classic pentest |
|---|---|---|
| Perspective | From the inside. Architecture, code, permissions and data flows, plus targeted attacks on the weaknesses that are typical for agents. | Mostly from the outside. Attacks on the application, interfaces and infrastructure. |
| Outcome | Prioritized findings with proposed changes and a target architecture your team can implement. | Vulnerabilities found, with evidence and rating, usually with remediation advice. |
| Scope | One agent, RAG or MCP system with its tools, data sources and approvals. | Network, servers, web applications and interfaces within the agreed scope. |
| Who | A software architect who builds and runs such systems himself. | A specialized testing firm with its own team. |
Sample report
Excerpt from a report
The excerpt shows the structure and level of detail of the findings. System, findings and test results are invented.
-
The CRM tool runs under a service account
- Description
- The crm_search tool queries the CRM with a technical account that can read all customer records. The permissions of the support staff member using the agent play no part in the query.
- Impact
- Through the agent, records of other teams can be retrieved that the support staff member could not see in the CRM directly.
- Proposed change
- The tool calls the CRM with the identity of the requesting person, and the CRM checks the permissions. An isolation test per role covers this.
-
Incoming emails, CRM data and sending share one context
- Description
- The agent reads incoming customer emails, has access to CRM data and can write to any address with send_email. In testing, a crafted email got the agent to send customer data to an outside address in 3 of 10 runs.
- Impact
- Anyone who sends an email from outside can exfiltrate CRM data without having access to the system.
- Proposed change
- The server only accepts the ticket’s address as recipient. Sending to other addresses goes through an approval that shows recipient and text in plain text.
-
The approval shows the model’s reasoning instead of the email
- Description
- Before sending, the dialog shows the agent’s reasoning and a send button. Recipient and full text only become visible after another click.
- Impact
- The approver confirms emails without having seen their recipient or content. A manipulated reasoning makes approval more likely.
- Proposed change
- The dialog shows recipient, subject and text in plain text and highlights deviations from the ticket address. The model’s reasoning is removed.
Contact
Request a go-live review
In the intro call we clarify whether a review fits your system and what scope makes sense. You then receive a fixed-price proposal.