Skip to main content

Document Conformance Toolkit

When an agent produces a document that has to follow a defined shape — a Safety Data Sheet, an ICD-10 prior-auth packet, a SOC 2 control assertion, a regulated invoice — "redact the PII" isn't enough. The document either conforms to the schema, or it doesn't.

The Document Toolkit is Rivaro's framework for declaring that schema, validating outputs against it, and emitting a cryptographically-signed receipt of conformance.

Where to find it in the app​

The toolkit is exposed in two pieces:

SurfaceWhere
Output Checks (the lightweight schema-validation case)Dashboard → Policies & Authority → RUNTIME → select an App Context → Security Policies → Output Checks sub-tab
Full Document Toolkit (schemas, catalogs, rule packs, evidence manifests, bindings, pack imports)Administered through the API

If your use case is "validate the shape of a tool-call response and reject if malformed," use Output Checks in the UI — described next.

If your use case requires the full toolkit (regulated forms, catalog lookups, cross-field rule packs, conformance receipts), see the sections below.

Output Checks​

For lighter-weight cases — "this tool returns a JSON blob with customer_id, amount, and status, and we want to validate the shape and reject malformed responses" — use Output Checks.

Output checks are document schemas + optional rule packs registered against an AppContext. They run on tool-call responses and policy-evaluated LLM outputs.

Configuring an output check​

  1. Open Dashboard → Policies & Authority → RUNTIME.
  2. Switch to the App Context tab and pick the AppContext.
  3. Inside that AppContext, open Security Policies → Output Checks.
  4. Click Add output check and fill in:
    • Name (e.g. refund_response_shape)
    • JSON Schema — the shape the output must conform to
    • Rule pack (optional) — cross-field predicates that pure JSON Schema can't express
    • On fail — what to do if validation fails: BLOCK (reject the response), LOG (record but allow), or REDACT (mask non-conformant fields)
  5. Click Test to dry-run the check against a sample payload before saving.

Example schema​

{
  "type": "object",
  "required": ["refund_id", "amount", "status"],
  "properties": {
    "refund_id": { "type": "string" },
    "amount": { "type": "number", "minimum": 0 },
    "status": { "type": "string", "enum": ["pending", "succeeded", "failed"] }
  }
}

Paste this into the JSON Schema field of the Output Check editor; the editor validates the schema itself before saving.

Output checks in policy bundles​

Output checks travel inside policy bundles under the outputChecks array, so the same prune-mode GitOps workflow that manages policy rules also manages output checks. See Policy Bundles.

When to use the full toolkit​

Use the toolkit any time an agent produces a structured artifact where downstream systems or regulators expect a specific shape:

  • Regulated forms (SDS, ICD prior-auth, SOC 2 SoA entries, model cards)
  • Tickets and records that feed audited systems (Jira issue templates, ServiceNow change requests)
  • Contracts, invoices, or legal documents with required fields
  • Any "the agent fills out this template" workflow

Use a plain policy rule if you only care about content properties (e.g. "no PII in egress"). Use Output Checks for shape validation on a tool response. Use the full toolkit when you need regulated catalog references, cross-field rule packs, and signed conformance receipts.

What the full toolkit provides​

Five composable resources you can mix and match:

ResourcePurpose
Document schemaThe structural contract — JSON Schema or schema fragment the document must satisfy
CatalogA regulated reference dataset (UN dangerous-goods numbers, ICD codes, NAICS codes, jurisdiction-specific control lists). Schemas can require values to come from a named catalog.
Rule packCross-field rules that pure JSON Schema can't express ("if shipping_method is air, then un_number must be in the IATA-approved list").
Evidence manifestDeclares the artifacts the agent must produce alongside the document — supporting PDFs, signatures, attestations.
BindingGlues a document schema + rule pack + evidence manifest to one or more DocumentArtifact types so the engine knows when to apply them.

How conformance works at runtime​

When an agent produces a candidate document:

  1. Rivaro identifies it as a DocumentArtifact of a known type.
  2. The toolkit looks up the binding for that artifact type.
  3. The candidate is validated:
    • Schema — does it satisfy the JSON Schema?
    • Rule pack — do the cross-field rules pass?
    • Catalog references — do all catalog-bound values exist in the named catalog?
    • Evidence manifest — are the required supporting artifacts present?
  4. The result is recorded as a ConformanceReceipt — pass, partial, or fail — and signed (Ed25519).
  5. If the binding's policy says BLOCK on non-conformance, the agent's output is rejected before it leaves Rivaro.

The receipt is verifiable offline by anyone holding Rivaro's Ed25519 public key. Use it as compliance evidence when an auditor asks "show me the agent produced a conformant SDS for shipment 12345."

Full toolkit​

Getting the toolkit configured

The schemas, catalogs, rule packs, evidence manifests, and bindings described below are administered through the API. Reach out to Rivaro and we can help configure them for your use case.

The full toolkit exposes these resource types via /api/document-toolkit/:

ResourceEndpoints
Document schemasGET / POST / DELETE /api/document-toolkit/schemas (name, schemaJson, version)
CatalogsGET / POST / DELETE /api/document-toolkit/catalogs (name, entries[])
Rule packsGET / POST / DELETE /api/document-toolkit/rule-packs (name, rules[])
Evidence manifestsGET / POST / DELETE /api/document-toolkit/evidence-manifests
BindingsGET / POST / DELETE /api/document-toolkit/bindings (artifactType, schemaId, rulePackId?, evidenceManifestId?, onFail)
Pack importPOST /api/document-toolkit/packs (multipart upload of a .zip containing schemas / catalogs / rule packs / bindings)
Receipt verifyGET /api/document-toolkit/receipts/{id}/verify (re-validate a persisted candidate)

Pre-built packs ship for common regulatory regimes — NERC CIP control catalog, ICD-10, GHS/UN dangerous-goods — and can be imported via the pack upload endpoint.

End-to-end example​

You're shipping a chemical-shipping agent that must produce SDS (Safety Data Sheets):

  1. Upload the GHS pack — ships an SDS schema, the UN dangerous-goods catalog, and the basic cross-field rules.
  2. Bind artifact type SDS_DOCUMENT → SDS schema + GHS rule pack, on-fail BLOCK.
  3. Agent produces an SDS in a tool call. The toolkit validates.
  4. Conformance receipt is signed and persisted. The agent's response succeeds only if every check passes.
  5. An auditor asks for evidence that SDS #4587 was conformant. You return the receipt and the public key — they verify offline.

Next steps​