Agentic Risk Analyst

What could go wrong with this AI use case, and what should we do about it?

Chirag Dani · ARA v3.2.1
LLM: huggingface · meta-llama/Llama-3.3-70B-Instruct · Secure-RAG: connected

User guide

What each menu item does, what you put in, what you get out, and who it is for. ARA (Agentic Risk Analyst) turns a plain-language description of an AI use case into a risk assessment, then into the deliverables a client pays for.

Start in two minutes

  1. Open a worked example (Autonomous loan approval (Australia), AI-assisted AML transaction monitoring, Generative AI chatbot for internal HR policy, Autonomous incident-response agent (SOC)). They work without a language model and show every page.
  2. On an example, open Versions, sign-off and framework mapping, download the Board pack (Word) and the Risk register (Excel).
  3. Try ADM transparency pack and Vendor due diligence: neither needs a language model.
  4. When the server has a model configured, describe your own scenario on New assessment.

Menu items

MenuWhat it doesNeeds a language model?
New assessmentDescribe the AI use case (what it does, for whom, which data, how autonomous, where). Understand my scenario first shows how ARA read it, so you can correct jurisdiction, autonomy, oversight or industry. Run full assessment runs the Risk, Compliance and Security analysts, the Control Analyst and the Supervisor (1 to 4 minutes). Optional: client name, sector pack, red team. Clear empties the form.Yes
Assessment libraryEvery assessment (from this page, from comparisons, from the API and the worked examples), filterable by client. Signed-off assessments are marked.No
Assessment pageDecision, scenario brief, risk register and heat map, regulatory findings, security threats, controls and residual risk, remediation plan, red team, agent trace and downloads. The Next steps card links to everything below.No (to view)
Versions, sign-off and framework mappingEach completed run is a numbered version with a SHA-256 fingerprint; compare two versions to see what changed. Review each AI-drafted risk, threat, regulation and control (accept, accept with changes, reject). A reviewer then signs off, which locks the assessment until it is reopened with a reason. Controls are mapped (as suggestions to verify) to ISO/IEC 42001 Annex A, NIST AI RMF, ISO/IEC 27001:2022 and OWASP LLM 2025.No
Board pack and risk registerA Word board pack (decision, risks, obligations, controls, remediation, mapping, simulations, envelope, review and sign-off status) and an Excel risk register, carrying your branding.No
Failure simulatorThe model drafts how the system could fail step by step and which controls break each chain. You then remove capabilities, set each control to missing, planned, implemented or verified, add controls, or accept a risk (name, reason and review date required). Exposure is recalculated with a fixed rule, so the same choices always give the same numbers. "Verified" counts only with an evidence reference; reductions resting on controls without evidence are labelled "projection", and the page also shows exposure using evidenced controls only. Optional loss per event and events per year give an annual loss estimate.Yes for the draft; no for the recalculation
Operating envelopeThe limits an AI agent must act within: permitted actions with tool, amount limit and approval, prohibited actions, allowed and prohibited data, daily total, escalation triggers, stop conditions. Every saved edit is a new hashed version; push it to CALIBRE Governance Fabric, which then answers Permit, Deny or Escalate before each agent action.Yes for the draft; no for editing
ADM transparency packFor the Privacy Act automated-decision transparency obligation (APP 1.7 to 1.9, from 10 December 2026). List each system (form, CSV import or from an assessment), answer three questions (role of the program, effect on individuals, personal information), and get in scope, not in scope or needs legal review with reasons; draft APP 1.8 policy text; Excel inventory with change log; Word pack with an evidence file. Name the Secure-RAG workspace holding the vendor's documents to retrieve cited evidence and to be warned (daily, or on demand) when those documents change.No
Vendor due diligenceUpload a vendor's contract, terms and security or AI documentation (or point at a Secure-RAG workspace). Fourteen AI vendor controls are checked (training on your data, data location, sub-processors, incident notice, model change notice, audit rights, exit and others, with APRA CPS 230 and Victorian Government procurement notes). You get cited evidence or "not found" per control, a rule-based vendor rating, questions to ask and suggested clauses, in Word and Excel.No
Compare scenariosAssess two design options (for example fully automated against human-approved) and get a recommendation with the differences.Yes
Compare modelsRun the same scenario on two to four models and compare decisions and ratings.Yes
Consistency checkRun the same scenario two to five times and score agreement (decision, residual rating, risks). Shows a client how stable the analysis is.Yes
Red-team exportOn an assessment with a red team: download the tests as JSON or CSV and import them into a Secure-RAG Breach Lab target to run them against the live system.Yes for the red team
Audit trailEvery change and run in a hash chain; the page shows whether the chain is intact and its head value (also printed in each board pack).No
User guideThis page.No
How ARA worksThe agent pipeline, evidence and honesty rules, data handling.No
API and integrationEndpoints used by CALIBRE Governance Fabric and other systems.No
Branding and sector packsYour name, colour, footer and contact on reports; the sector packs (banking, insurance, superannuation, health, Victorian Government, EU) that add context for the Compliance agent.No

For the server administrator: enabling live assessments

ARA needs one language model provider. Set these in the hosting service's environment (Render: service, Environment), then redeploy:

ProviderVariablesNote
Hugging Face (default)HF_TOKENHugging Face Inference Providers are paid from account credits; a token with no credits fails with "402 Payment Required".
OpenAI or compatibleLLM_BACKEND=openai, OPENAI_API_KEY, LLM_MODELSet LLM_MODEL to a current model name from the provider.
AnthropicLLM_BACKEND=anthropic, ANTHROPIC_API_KEY, LLM_MODELAs above.
Google GeminiLLM_BACKEND=gemini, GEMINI_API_KEY, LLM_MODELCheck the provider's terms: free-tier content may be used by the provider; do not send client data on a free tier.

Also recommended: DEMO_ACCESS_CODE (protects paid model calls on a public site), ARA_API_KEY (for the Fabric), SECURE_RAG_URL and SECURE_RAG_TOKEN (evidence), FABRIC_URL and FABRIC_API_KEY (envelopes). On Render's free plan, stored data is lost when the service sleeps or redeploys; use a paid instance with a disk and set ARA_DB for client work.

Limits

  • Decision support, not legal or regulatory advice. A qualified person must review ratings, regulations and controls.
  • Rule-based parts (ADM test, vendor check, framework mapping, simulator arithmetic, consistency score) are transparent; their inputs still need review.
  • Worked examples are illustrations prepared in advance, not outputs of a live run.

Decision support only, not legal or regulatory advice. AI-generated ratings and citations must be reviewed by a qualified person before they are relied on.