Credo AI vs Kakunin: Pre-Deploy Scanners vs Runtime KYA

TL;DR

• Pre-deploy scanners like Credo AI evaluate AI models before deployment (bias testing, red-teaming, model cards) — they satisfy EU AI Act Article 9.
• Kakunin issues cryptographic X.509 identities and monitors agents in real time after deployment — it satisfies Articles 12 and 14.
• The two categories are complementary, not competing: pre-deploy scans catch design flaws, runtime KYA catches live behavioral drift a static scan can never see.
• A prompt injection three weeks post-launch can push a 'safe' model outside its approved scope — only runtime monitoring catches this.
• If budget forces a choice: not-yet-deployed systems should prioritize pre-deploy evaluation; already-live systems should prioritize runtime KYA.

Is Kakunin a replacement for Credo AI, or do they work together?

They work together. Credo AI-style tools evaluate an AI model's safety and fairness before it ships, satisfying the EU AI Act's Article 9 risk-management requirement. Kakunin issues a cryptographic identity to the deployed agent and continuously scores its live behavior, satisfying Articles 12 (logging) and 14 (human oversight and stop mechanism). A model can pass every pre-deployment check and still drift into unsafe behavior in production — that's the gap runtime KYA closes.

Architectural Comparison: Pre-Deploy Evaluation vs Runtime Defense

As enterprise adoption of artificial intelligence shifts from static predictive models to autonomous agent fleets, compliance and security officers face a fundamental vendor selection question: How should we govern AI agents?

Two distinct product categories have emerged in the market:

• Pre-Deployment Model Scanners (e.g., Credo AI, Arthur AI): Tools focused on offline model card evaluation, dataset fairness metrics, red-teaming benchmarks, and static prompt testing prior to deployment.
• Runtime KYA & Cryptographic Identity Infrastructure (Kakunin): Platforms that issue X.509 cryptographic identities (AWS KMS), enforce real-time scope guardrails, monitor live event telemetry, and execute automated credential revocation in production.

This technical comparison evaluates both approaches to help security teams choose the right architecture.

---

Why Both Primitives Are Complementary

Pre-deployment model governance and runtime agent KYA address different halves of regulatory compliance:

• Pre-Deployment Scanners (Credo AI) answer: 'Is this model trained safely and evaluated for bias before release?'
• Kakunin Runtime KYA answers: 'Is this specific deployed agent operating within its certified scope right now, and can we instantly halt it if it breaches risk limits?'

Explore complete comparison breakdowns on our AI Agent Compliance Vendor Comparison page, review the Know Your Agent (KYA) Framework, or generate a free compliance readiness report.

Mapping Vendor Categories to EU AI Act Obligations

The EU AI Act does not treat 'compliance' as a single deliverable — it splits obligations across the AI system's lifecycle, and pre-deploy scanners and runtime KYA infrastructure map to different articles almost by design. Article 9 requires a risk management system that identifies, evaluates, and mitigates known and foreseeable risks associated with a high-risk AI system, and it explicitly calls for this to happen before the system reaches the market and to be updated iteratively. This is Credo AI's home turf: dataset fairness testing, model card documentation, and red-teaming benchmarks produce exactly the evidentiary record Article 9 auditors want to see for design-stage risk identification.

Article 12, by contrast, requires automatic recording of events ('logs') throughout the system's operational lifetime, at a level that enables the provider to monitor for and identify situations that may result in the AI system presenting a risk, and to facilitate post-market monitoring. A static model card, however thorough, cannot satisfy this — logging has to be live, tamper-evident, and continuous. This is squarely runtime territory: Kakunin's Supabase-backed audit_log is append-only at the database trigger level, meaning UPDATE and DELETE operations are blocked outright rather than merely discouraged by application logic, which is the kind of WORM guarantee an Article 12 audit expects.

Article 14 sits between the two. It requires human oversight measures that allow a natural person to intervene, including the ability to decide not to use the system or to interrupt it through a 'stop' mechanism. Pre-deploy scanners inform the design of that oversight — they tell a compliance officer where the model is statistically likely to misbehave, which shapes the guardrails written into deployment policy. But the stop mechanism itself has to exist at runtime, with a bounded response time. Kakunin's auto-revocation at a risk score of 0.85 or higher, executed via CRL/OCSP within a 60-second SLA, is one concrete technical instantiation of that Article 14 requirement — a human-defined threshold enforced automatically, not a human clicking a button under time pressure.

• Article 9 (risk management, pre-market) — primarily Credo AI-class tooling
• Article 12 (automatic logging, continuous) — primarily Kakunin's WORM audit_log
• Article 14 (human oversight, stop mechanism) — designed with pre-deploy input, enforced by runtime revocation

When the Scan Passes but the Agent Drifts

Consider a concrete scenario common to MiCA-regulated crypto-asset service providers deploying an AI-driven trading assistant. Before release, the model passes a full pre-deployment evaluation: its training data is checked for label bias, its outputs are red-teamed against a battery of adversarial prompts, and its model card documents an approved scope limited to read-only market analysis and trade recommendations under a defined notional limit. Every artifact a Credo AI-style review would produce is in order, and the system is signed off for production.

Three weeks into live operation, a prompt injection embedded in an external data feed the agent consumes — a manipulated ticker comment field, for instance — induces the agent to reinterpret its own operating instructions and begin issuing trade instructions outside its approved notional limit, then attempts to query customer account data it was never scoped to read. None of this was visible to the pre-deployment scan, because the scan evaluated the model in isolation, not the live combination of model, tool access, external data, and adversarial input that only exists once the agent is running against production systems.

In a Kakunin-instrumented deployment, this drift shows up as a runtime signal, not a postmortem finding. The agent's behavioral risk score, computed continuously from live event telemetry, moves from a baseline near 0.12 toward 0.90+ within the same session as it attempts the out-of-scope calls — each unauthorized action registers as a scope violation against the certificate's encoded permissions. Once the score crosses the 0.85 auto-revocation threshold, the CRL/OCSP-backed process pulls the agent's X.509 credential within the 60-second SLA, and the mTLS gateway starts rejecting its requests before further trades can execute. The pre-deploy scan did its job at t=0; it was never designed to catch a failure mode that only exists at t=three-weeks-and-forty-seven-minutes.

Procurement Guidance for Compliance Teams

Evaluating these two categories side by side requires different diligence questions, because they are answering different audit requirements. Treating a runtime KYA RFP like a model-scanning RFP (or vice versa) produces a scorecard that doesn't actually predict regulatory readiness.

Questions for Pre-Deployment Scanner Vendors

The diligence here centers on reproducibility and dataset provenance, since Article 9's risk management system has to be demonstrably iterative, not a one-time checkbox.

• Can the vendor reproduce a prior bias/fairness finding against a re-run of the same model version, with the same result?
• What is the update cadence for the model card when the underlying model or fine-tune changes?
• Does the red-teaming methodology cover agentic/tool-use failure modes, or only single-turn text generation?
• How is dataset lineage documented for downstream Annex IV technical documentation requirements?

Questions for Runtime KYA Vendors

The diligence here centers on latency guarantees and tamper-evidence, since Article 12 and 14 both hinge on the runtime controls actually functioning under load, not just existing on paper.

• What is the measured (not advertised) revocation SLA under production load, and is it backed by CRL, OCSP, or both?
• Is the audit log append-only at the database layer (triggers) or only enforced in application code?
• How is the risk-scoring threshold configured — is 0.85-equivalent auto-revocation adjustable per tenant, per agent class, or fixed?
• Does certificate issuance encode the approved operational scope from the pre-deploy sign-off, or is scope tracked separately?

Integration Patterns: One Pipeline, Two Controls

Compliance teams evaluating these categories in isolation often miss that the strongest architecture doesn't pick one — it chains them. A practical pattern treats the pre-deployment scan as a promotion gate in the CI/CD pipeline: an agent or model version cannot be pushed to a production environment until its scan artifacts (bias metrics, red-team pass rate, model card) clear a defined threshold. That gate output then becomes an input to the runtime layer, rather than a document that gets filed and forgotten.

Specifically, the approved scope from the model card — trade limits, permitted data reads, allowed API surface — should be encoded directly into the X.509 certificate's extension fields at issuance time via AWS KMS, so the cryptographic identity the agent presents to the mTLS gateway is a direct, machine-enforced expression of what the pre-deployment review actually approved. A webhook from the scanning tool's completion event, routed through QStash with standard retry semantics, can trigger the certificate issuance job automatically, closing the gap between 'the scan says this is safe' and 'the runtime will only permit what the scan approved.'

From there, the two systems diverge in cadence but stay linked in data. Pre-deployment scans typically re-run on a fixed schedule or on model-version bump; runtime scoring runs continuously. When a runtime revocation event fires, feeding that event back into the next pre-deployment review cycle — as a new red-team scenario, for instance — closes the loop and satisfies the iterative-update language in Article 9 with actual production evidence rather than synthetic test cases alone. Teams building this integration should review the certificate scope model in the /docs section before wiring the webhook, since the extension field structure determines what granularity of scope violation the gateway can actually detect.

When You Only Have Budget for One

In an ideal budget, both categories are funded — they answer different audit questions and neither substitutes for the other. In practice, many mid-market CASPs and financial AI operators facing near-term MiCA or EU AI Act deadlines have to sequence spend, and the right sequencing depends on where the system sits in its lifecycle relative to the deadline, not on which vendor has the more compelling demo.

If the AI system has not yet been placed on the market — it is still in internal testing, has no live customer exposure, and the compliance deadline is measured in quarters rather than weeks — pre-deployment evaluation is the higher-leverage first purchase. Annex IV technical documentation and the Article 9 risk management system are gating requirements before the system can be deployed at all, and no amount of runtime monitoring substitutes for evidence that was supposed to exist before go-live.

If the AI system is already in production and processing live transactions — the far more common state for MiCA-regulated entities being asked to demonstrate compliance on an existing deployment — runtime KYA is the more urgent gap to close. A regulator or auditor examining an already-deployed agent will ask for continuous operational logs and evidence of an enforceable stop mechanism, which is exactly what Article 12 and Article 14 require and what pre-deployment artifacts, however good, cannot retroactively provide. For teams in this position, a free compliance readiness report against the current deployment is a faster way to identify the specific gap than reasoning about it in the abstract, and it avoids spending the single available budget line on the wrong half of the lifecycle.

Cost Structure and Pricing Model Differences

The two categories also diverge in how they price, which matters when a compliance team is building a multi-year budget rather than a one-time purchase. Pre-deployment scanning tools typically price on a per-model or per-evaluation-run basis — you pay to run a bias audit or a red-team pass against a specific model version, and cost scales with how often you re-evaluate, not with how much traffic the model serves once deployed. This makes pre-deploy tooling relatively predictable for teams that release model updates on a quarterly or slower cadence, but it can get expensive fast for teams doing continuous fine-tuning, since every meaningful model change technically warrants a fresh evaluation cycle.

Runtime KYA infrastructure, by contrast, typically prices on agent count or event volume, since the cost driver is certificate issuance, gateway verification, and continuous risk scoring rather than periodic evaluation runs. This means runtime costs scale with production usage — more agents, more transactions, more scored events — rather than with how often the underlying model changes. For a CASP running a small number of high-value trading agents, this pricing model is favorable; for an operator running thousands of low-value customer-facing bots, it's worth modeling total cost at expected scale before committing, since a per-agent pricing structure that looked reasonable at pilot scale can compound quickly across a large fleet.

Evaluating Vendor Lock-In Risk

A final consideration compliance teams often defer until too late is portability. Pre-deployment scan results — bias metrics, red-team transcripts, model cards — are generally exportable as standalone documents, since their whole purpose is to feed into a technical documentation file that has to survive independent of any particular vendor's tooling. Switching pre-deploy vendors mid-program is disruptive but not existential: the historical evaluation record can usually be retained and referenced even after migrating to a new scanning tool.

Runtime KYA infrastructure carries a different lock-in profile, because the certificate authority itself becomes a root of trust that downstream systems depend on. Migrating away from a runtime vendor means re-issuing every active agent's identity from a new CA, coordinating a cutover window across every integrated API gateway, and — critically — deciding what happens to the historical WORM audit trail generated under the old vendor's infrastructure. Compliance teams evaluating runtime KYA vendors should ask specifically how audit_log data is exported in the event of a vendor switch, and whether historical records remain independently verifiable (via the original signing certificate chain) after a migration, rather than becoming an opaque archive that's technically retained but practically unusable for a future audit.