How certification works
Certification has six steps and usually takes four to eight weeks. This page describes each step, what Caudals does, what you provide, and what you receive.
The six steps
Scope
We define the agent to be certified, including its version, channels, languages, and capabilities. From that scope, we draw up the Statement of Applicability. It lists the requirements that apply and justifies any that do not. You sign the authorization for adversarial testing, which sets the environment, dates, and limits.
Build the tests
We build the operational test set from your documentation, real questions from your customers or staff, and a working session with your lead subject expert. Caudals domain experts validate every expected answer. We prepare the attack set for the security requirements that apply.
Test
We run the tests against the agent and grade every answer and every tool call. Operational answers are graded by their meaning against the source. Each attack attempt is recorded with its technique and its outcome.
Review evidence
For requirements verified by evidence, we review documents, configurations, logs, contracts, and records. We interview the people responsible for the agent. We then check the evidence against what we observed in testing.
Fix and re-test
You receive the findings, each with its evidence, severity, and the requirement it affects. You fix what blocks certification, and we test those requirements again. This step can repeat.
Decide and certify
A Caudals reviewer who did not run the tests reads the full file and makes the decision. If the decision is positive, we issue the certificate, publish the registry entry, and send you the badge and the certification report.
How long it takes
These are planning figures, not commitments. The time spent fixing findings depends on your team and varies the most.
| Phase | Typical duration |
|---|---|
| Scoping and authorization | 1 week |
| Building and validating the test set | 1 to 2 weeks |
| Testing and evidence review | 1 to 2 weeks |
| Fixes and re-tests | 1 to 3 weeks |
| Decision and issue | A few days |
Who does what
| Step | You | Caudals |
|---|---|---|
| Scope | Describe the agent and name its owner. Sign the authorization. | Draw up the Statement of Applicability. |
| Build the tests | Share documentation and real questions. Host one working session with your expert. | Write the test cases. Have domain experts validate the expected answers. |
| Test | Provide a test environment or test accounts. | Run the tests and grade the results. |
| Review evidence | Provide the requested evidence and time with the people responsible. | Review the evidence against the requirements and the test results. |
| Fix and re-test | Fix the findings that block certification. | Test the affected requirements again. |
| Decide and certify | Approve the public scope wording. | Make the decision. Issue the certificate and the registry entry. |
What you receive
- The certificate, with its number, scope, standard version, issue date, and expiry date.
- The registry entry, a public page that shows the current status of the certificate.
- The certification report, with the result of every requirement, each finding with its evidence, and fixes in order of priority.
- The test set, so you can run it as your own regression suite before each release.
- The badge, in the formats described in The badge.
After the certificate
You maintain the certificate by passing a quarterly re-test and reporting material changes. Keeping a certificate explains how.