Siegfried-Thor BolzHome · AI Audit for Enterprise CMS
AI Risk AuditingAI Audit for Enterprise CMS
Modern content platforms increasingly run on AI — personalisation, generative authoring, semantic search, RAG and agents. Each of those is a governance, compliance and security question. Here is how I audit the AI inside an enterprise CMS, end to end, and the methods and tools I use.
AI Governance Watch
The frameworks this audit measures against keep moving. So I maintain a public, continuously-watched reference library — AI Governance Watch — where every standard (NIST AI RMF, EU AI Act, ISO/IEC 42001 & 23894, DORA, CSA Agentic Profile, Berkeley CLTC) carries an audit-oriented summary, the clauses you would cite, and a PR-gated provenance trail for every change. It is how I make sure your audit is measured against the current rule — not last year's — with a defensible, evidence-backed trail.
How I run a CMS AI audit
A structured, framework-based workflow — adapted from the NIST AI RMF (Govern · Map · Measure · Manage) and the enterprise risk methodology from the University of Oxford "Managing Enterprise AI Risks" programme.
Scope & classify
Inventory every AI use case in the CMS and classify it — risk OF vs. risk DUE TO AI, level of autonomy, and EU AI Act risk category. This sets how deep the audit needs to go.
Map the system
Document data flows, models, prompts, integrations and access paths, and produce an AI-SBOM (the models, datasets and dependencies in play). Mirrors the NIST "Map" function.
Assess against frameworks
Test each use case against the EU AI Act, NIST AI RMF, ISO/IEC 42001 and ISO/IEC 27001, plus the OWASP Top 10 for LLM & Agentic Applications — referencing MITRE ATLAS for adversarial techniques.
Test the implementation
Because I read code: review prompts, RAG pipelines and APIs for prompt injection, data/model poisoning, PoisonedRAG, excessive agency and insecure access — with a real security toolchain and AI red-teaming.
Evidence & report
Deliver auditable artefacts — model & data cards, a living risk register and the AI-SBOM — and board-level findings argued with two-stage de-jure / de-facto reasoning: what the rule requires vs. what the system actually does.
Govern & monitor
Embed the Three Lines of Defence, post-deployment monitoring, an incident-response playbook and a decommissioning plan — so governance is continuous, not a one-off certificate.
Frameworks & standards
The reference set I assess against:
Methods & tools
Methodologies and the toolchain I use:
Signature methods
Beyond the framework checklist, these depth methods set the audit apart — grounded in the University of Oxford "Managing Enterprise AI Risks" programme and applied directly to your CMS.
Proportionate risk tiering
I don't audit every AI feature to the same depth. A seven-dimension risk score — harm severity, autonomy, data sensitivity, operational criticality, scale, robustness and human oversight — places each use case in a tier from 1 to 4. A low-risk meeting summariser gets a light touch; a high-stakes screening or scoring system gets full scrutiny. Proportionate governance — no sledgehammer on a fly.
Auditing agentic AI
An assistant suggests; an agent acts — it calls tools, writes to systems and chains steps before a human sees the result. I assess agents across four layers — model, orchestration, tool and memory — against the OWASP Top 10 for Agentic Applications, checking least-privilege tool access, a tested kill-switch and full traceability. Increasingly relevant as AEM / AEMaaCS adds agentic automation.
Guardrails vs. controls
A policy that says the AI should not do something is only an intention. I separate guardrails (the intent) from controls (the enforcing mechanism), and verify the control actually exists and is monitored — because a guardrail without a control is a wish.
Human oversight that isn't theatre
"Human-in-the-loop" on paper often means rubber-stamping. I test whether oversight is real: can the reviewer understand, intervene and override in time, backed by measured conditions and monitoring — as Article 14 of the EU AI Act requires? No oversight theatre.
This approach is grounded in the University of Oxford "Managing Enterprise AI Risks" programme (2026) and Packt's "Generative AI & Agentic AI for Finance" certification (2026), combined with 15+ years of secure enterprise AEM and Java delivery for government and telecommunications clients.
Frequently asked
What is an AI audit for a CMS?
An independent assessment of the AI features embedded in a CMS and its web platform — personalisation, generative authoring, semantic search, RAG and agents — against regulation and security frameworks, producing evidence such as risk classification, model cards, an AI-SBOM and a living risk register.
Which frameworks do you audit against?
The EU AI Act, NIST AI RMF, ISO/IEC 42001 and ISO/IEC 27001, plus the OWASP Top 10 for LLM and Agentic Applications and MITRE ATLAS.
How is this different from a normal compliance audit?
Most compliance auditors review documentation only. I also read the code — prompts, RAG pipelines, integrations and access paths — so findings reflect what the system actually does, not just what the paperwork claims.
Does this apply to Adobe Experience Manager (AEM / AEMaaCS)?
Yes — it is a core specialisation. AEM and AEM as a Cloud Service increasingly embed AI for authoring, search and personalisation, and that AI needs the same governance and security controls as any other high-value system.
Do you audit AI agents (agentic AI)?
Yes. Agents don't just suggest text — they act: calling tools, writing to systems and chaining steps autonomously. I assess them across four layers (model, orchestration, tool and memory) against the OWASP Top 10 for Agentic Applications, with least-privilege tool access, a tested kill-switch and full traceability. This is increasingly relevant as Adobe Experience Manager adds agentic automation.
Do we need a Fundamental Rights Impact Assessment (FRIA)?
Not every deployer does. Under the EU AI Act a FRIA is required mainly for public bodies and providers of public services, and for certain high-risk uses such as creditworthiness scoring and life/health insurance. For many private deployments a GDPR Data Protection Impact Assessment (DPIA) applies instead, with a FRIA as good practice. I help you determine which actually applies to your use case — professional guidance, not legal advice.
Need an AI audit for your CMS?
I deliver the assessment, the evidence and the remediation path. Available for projects in German 🇩🇪 and English 🇺🇸.