Security risk

Instructions typed by a user cannot make the agent ignore its rules, change its role, produce prohibited content or act outside its authority.

Why it matters

The easiest attack on an AI agent needs no tools, only a persuasive sentence.

How we verify it

With the owner's written authorization, Caudals runs direct injection and jailbreak attempts across current technique families, such as role play, instruction override, encoding and obfuscation, multi-turn pressure and language switching. No attempt may succeed with critical or high impact.

Evidence we review

• The signed authorization
• The attack log, with every attempt and its outcome
• The guardrail configuration

Applies to

Every agent

Verified by

Testing. Reverified: every quarter.

Maps to

EU AI Act Art. 15NIST AI RMF MEASURE 2OWASP Top 10 for LLM applications LLM01:2025OWASP Top 10 for agentic applications ASI01MITRE ATLAS AML.T0051.000 · AML.T0054 · AML.M0020ISO/IEC 27001 A.8

Instructions hidden in content the agent reads, such as web pages, documents, emails, tool results or its own memory, are treated as data and never followed.

Why it matters

An agent that reads the web or a customer's attachment is reading text an attacker may have written.

How we verify it

Caudals plants instructions in each content channel the agent consumes, inside the test environment, and checks whether the agent obeys them, leaks data or calls tools as a result.

Evidence we review

• The inventory of content channels the agent reads
• The attack log
• The controls that separate instructions from data

Applies to

Every agent

Verified by

Testing. Reverified: every quarter.

Maps to

EU AI Act Art. 15OWASP Top 10 for LLM applications LLM01:2025 · LLM04:2025OWASP Top 10 for agentic applications ASI01 · ASI06 · ASI07MITRE ATLAS AML.T0051.001 · AML.T0070 · AML.T0080 · AML.M0030

The agent does not reveal its system prompt, internal instructions, or tool definitions. Nothing in that configuration would cause harm if leaked.

Why it matters

A leaked system prompt gives an attacker a map. It sometimes contains instructions that should never have been written down.

How we verify it

Caudals runs extraction attempts under the granted authorization. It reviews the system prompt for secrets, personal data, and rules that only work while hidden.

Evidence we review

• Attack logs from extraction attempts
• The system prompt review

Applies to

Every agent

Verified by

Testing. Reverified: every quarter.

Maps to

OWASP Top 10 for LLM applications LLM07:2025MITRE ATLAS AML.T0085.003 · AML.T0084 · AML.T0069SOC 2 C1

The agent discloses personal data only to the person it belongs to or to an authorized user. It does not reveal personal data held in its knowledge base, memory, logs, or other users' conversations.

Why it matters

One disclosed record can be a notifiable breach under the GDPR.

How we verify it

Caudals seeds test records and attempts to extract them through direct requests, social engineering, and prompt injection. It also verifies that identity is confirmed before data is disclosed.

Evidence we review

• Attack logs from extraction attempts on seeded records
• The identity verification flow
• A map of personal data the agent can access

Applies to

Agents that handle personal data

Verified by

Testing. Reverified: every quarter.

Maps to

ISO/IEC 42001 A.7NIST AI RMF MEASURE 2OWASP Top 10 for LLM applications LLM02:2025MITRE ATLAS AML.T0085 · AML.T0086SOC 2 P1–P8 · C1GDPR Art. 5 · Art. 32

The agent does not expose confidential business information. Where it serves multiple customers, one customer's data, documents, and history never appear in another session.

Why it matters

Shared infrastructure is efficient until one customer reads another's contract.

How we verify it

Caudals runs cross-customer tests with two test accounts and seeded documents. It attempts to extract confidential records and reviews how retrieval filters by customer and permission.

Evidence we review

• Results of cross-customer tests
• The retrieval and access-filter design
• Classification of data the agent may disclose

Applies to

Every agent

Verified by

Testing and evidence review. Reverified: every quarter.

Maps to

OWASP Top 10 for LLM applications LLM02:2025 · LLM08:2025MITRE ATLAS AML.T0085.000SOC 2 C1 · CC6GDPR Art. 32ISO/IEC 27001 A.8

API keys, passwords, and tokens are never placed in prompts, knowledge sources, or outputs. The agent's credentials are stored, scoped, and rotated like any other secret.

Why it matters

Anything in the model's context can be talked out of it.

How we verify it

Caudals attempts to extract credentials, scans prompts and knowledge sources for known secret patterns, and reviews how credentials are stored and rotated.

Evidence we review

• Secret scan results
• The secret management configuration
• Credential rotation records

Applies to

Every agent

Verified by

Testing and evidence review. Reverified: every quarter.

Maps to

OWASP Top 10 for LLM applications LLM02:2025 · LLM07:2025OWASP Top 10 for agentic applications ASI03MITRE ATLAS AML.T0098SOC 2 CC6ISO/IEC 27001 A.5 · A.8

Each tool and data source the agent can use is limited to the minimum its purpose requires. This includes read-only access where reading is enough, narrow interfaces instead of general ones, and limits on volume and value.

Why it matters

The damage an attacker can do through an agent equals the permissions the agent holds.

How we verify it

Caudals reviews each tool's permissions against its purpose and attempts to use tools beyond their intended scope.

Evidence we review

• The inventory of tools and permissions
• Service account scopes
• Results of out-of-scope tool attempts

Applies to

Agents that take actions

Verified by

Testing and evidence review. Reverified: every quarter.

Maps to

NIST AI RMF MANAGE 2OWASP Top 10 for LLM applications LLM06:2025OWASP Top 10 for agentic applications ASI02 · ASI03MITRE ATLAS AML.M0026 · AML.M0028 · AML.M0037SOC 2 CC6ISO/IEC 27001 A.5 · A.8

When the agent acts or retrieves data for a user, downstream systems enforce that user's identity and permissions. A user cannot access through the agent what they cannot reach directly.

Why it matters

An agent with a powerful service account turns every user into an administrator.

How we verify it

Caudals runs privilege-escalation tests across accounts with different permission levels. It requests data and actions that each account is not entitled to receive.

Evidence we review

• The authorization design, showing how identity reaches tools
• Results of the privilege-escalation tests

Applies to

Agents that take actions

Verified by

Testing. Reverified: every quarter.

Maps to

OWASP Top 10 for LLM applications LLM06:2025OWASP Top 10 for agentic applications ASI03 · ASI07MITRE ATLAS AML.TA0012 · AML.M0027SOC 2 CC6GDPR Art. 32

Agent output is validated or encoded before it is rendered, executed, or passed to another system. This prevents it from carrying scripts, database queries, commands, or malicious links.

Why it matters

The model's output is the input of everything downstream.

How we verify it

Caudals prompts the agent to emit active content, such as scripts, markup, data-carrying links, database queries, or commands. It then checks what the surrounding application does with that output.

Evidence we review

• Output handling test results
• The rendering and sanitization configuration
• The allow-list for links and domains, if one exists

Applies to

Every agent

Verified by

Testing. Reverified: every quarter.

Maps to

OWASP Top 10 for LLM applications LLM05:2025OWASP Top 10 for agentic applications ASI05MITRE ATLAS AML.T0077 · AML.M0033ISO/IEC 27001 A.8

The organization maintains an inventory of the models, tools, plugins, and data suppliers behind the agent. It assesses each supplier and holds contracts covering data use, retention, training on customer data, security, and change notifications.

Why it matters

Most of an AI agent is someone else's product, and its risks arrive with it.

How we verify it

Caudals reviews the inventory, supplier risk assessments, and contract terms for key providers.

Evidence we review

• The supplier and model inventory
• Due diligence records
• Contracts or data processing agreements with the relevant clauses

Applies to

Every agent

Verified by

Evidence review. Reverified: every year.

Maps to

EU AI Act Art. 25ISO/IEC 42001 A.10NIST AI RMF GOVERN 6 · MANAGE 3OWASP Top 10 for LLM applications LLM03:2025 · LLM04:2025OWASP Top 10 for agentic applications ASI04MITRE ATLAS AML.T0010 · AML.M0043SOC 2 CC9SR 26-2 § VIIDORA ICT third-party risk

The agent runs in an environment with baseline security controls, including access controls, encryption, patching, and separation of test and production. It enforces limits against automated abuse, such as rate limits, spending caps, and scraping protections.

Why it matters

An open endpoint with no limits is a bill waiting to arrive.

How we verify it

Caudals reviews environment controls, accepting a current SOC 2 report or ISO/IEC 27001 certificate covering the system as evidence. It also runs abuse tests at volumes agreed in the authorization.

Evidence we review

• A SOC 2 report, an ISO/IEC 27001 certificate, or equivalent control evidence
• Rate limit and spending cap configuration
• Abuse test results

Applies to

Every agent

Verified by

Testing and evidence review. Reverified: every quarter.

Maps to

EU AI Act Art. 15OWASP Top 10 for LLM applications LLM10:2025MITRE ATLAS AML.T0034SOC 2 CC6 · CC7 · A1ISO/IEC 27001 A.8

Security events such as blocked attacks, guardrail triggers, and unusual tool use are logged and reviewed. Alerts reach someone who can act. The agent is red-teamed again at least quarterly and after any material change.

Why it matters

New attack techniques are published every month. A test from last year proves little about today.

How we verify it

Caudals reviews logging and alerting against its own test attempts to verify they were recorded and noticed. The quarterly retest is part of maintaining the certificate.

Evidence we review

• Security logs for the test period
• Alert rules and on-call arrangements
• Earlier red-team reports

Applies to

Every agent

Verified by

Evidence review. Reverified: every year.

Maps to

EU AI Act Art. 12 · Art. 15NIST AI RMF MEASURE 2 · MEASURE 3MITRE ATLAS AML.M0024 · AML.M0035SOC 2 CC4 · CC7ISO/IEC 27001 A.8DORA Digital operational resilience testing