Worked example: AI-assisted AML transaction monitoring
Scenario as entered
A Tier-2 bank in Australia proposes to replace its rules-based AML transaction-monitoring engine with an LLM-based agent that reads free-text payment memos, customer notes and adverse-media results to score suspicious-matter reports (SMRs). The agent will draft the SMR narrative and auto-lodge to AUSTRAC once a compliance analyst clicks approve. The model is hosted on a US cloud region.
Top risks
- Suspicious matters missed after replacing rules
- Analysts approve drafts without real review
Conditions for approval
- Do not retire the rules engine until detection parity is demonstrated
- Evidence-linked drafts and second-line sampling
Executive narrative
The idea has merit as a prioritisation and drafting aid, but replacing the rules engine without proven parity is not acceptable.
Agent consensus: All analysts flag detection parity as the deciding issue.
Scenario brief
A Tier-2 Australian bank wants an LLM agent to replace rules-based transaction monitoring, score alerts and draft SMRs, lodged after analyst approval; the model is hosted in the US.
Jurisdictions
AustraliaUnited States (hosting)Data types
Payment memosCustomer notesAdverse media resultsTransaction dataAffected parties
CustomersAML compliance teamAUSTRACRisk domains for the specialists
AML/CTF compliancePrivacy and cross-border dataModel riskAutomation biasMaterial information missing
- How the rules engine is retired or run in parallel
- Data residency controls in the US region
Alternative approaches
Run alongside the rules engine
Keep the existing rules; use the agent to prioritise and draft.
- + No loss of detection coverage
- − Two systems to maintain
Refine the interpretation and re-run (clears the current result)
Refine the interpretation
Your answers are sent to the agents as user-confirmed facts and override the AI's inference. Leave "(no change)" to keep the current value.
Risk register
Detection gaps and analyst over-reliance are the main risks.
| Insignificant | Minor | Moderate | Major | Severe | |
|---|---|---|---|---|---|
| Almost Certain | |||||
| Likely | 1 | 1 | |||
| Possible | 1 | 1 | |||
| Unlikely | |||||
| Rare | |||||
| Impact → (rows: likelihood). Cell number = count of risks; hover for IDs. | |||||
| ID | Category | Risk | Cause → consequence | Likelihood | Impact | Inherent |
|---|---|---|---|---|---|---|
| R-01 | Compliance | Suspicious matters missed after replacing rules The agent fails to flag patterns the rules caught. |
Unvalidated replacement → Failure to report; enforcement action | Possible | Severe | Critical |
| R-02 | Operational | Analysts approve drafts without real review Automation bias. |
Volume pressure; persuasive drafts → Inaccurate reports lodged | Likely | Major | High |
| R-03 | Privacy | Customer data processed offshore Data sent to a US-hosted model. |
Hosting choice → Cross-border disclosure obligations; confidentiality | Likely | Moderate | High |
| R-04 | Model | Hallucinated facts in SMR narratives Draft contains facts not in the records. |
Generative model → Misleading reports | Possible | Major | High |
Regulatory impact and compliance gaps
AML/CTF reporting duties are central; privacy and prudential obligations also apply. Confirm with legal counsel.
Anti-Money Laundering and Counter-Terrorism Financing Act 2006 (suspicious matter reporting) (Australia) applicable
Why it matters here: The agent supports SMR decisions and drafting.
Impact on this scenario: The program must cover the new monitoring method.
- Report suspicious matters to AUSTRAC within the statutory time frames
- Do not disclose ('tip off') that a report has been made
- Maintain an AML/CTF program
- Parallel run before retiring rules
- Tipping-off controls on outputs
Privacy Act 1988 (APP 8 cross-border disclosure) (Australia) applicable
Why it matters here: Personal information is sent to a US-hosted model.
Impact on this scenario: Contract and assessment of the US provider.
- Take reasonable steps so the overseas recipient handles the information consistently with the APPs
- Data processing agreement
- Data minimisation
APRA CPS 230 Operational Risk Management (Australia) applicable
Why it matters here: Material reliance on a third-party model provider for a compliance-critical process.
Impact on this scenario: Provider assessment and exit plan.
- Formal agreement with the material service provider
- Service provider due diligence
Compliance gaps
| Area | Observation | Gap | Risk created | Remediation |
|---|---|---|---|---|
| Validation | Plan replaces the rules engine. | No evidence the agent detects what the rules detect. | Critical | Run both in parallel and compare before any retirement. |
Security threats
Free text in memos is an injection path into the agent.
| ID | Threat | Attack vector | Assets at risk | L × I | Inherent | MITRE |
|---|---|---|---|---|---|---|
| T-01 | Prompt injection through payment memos ai_specific A memo instructs the agent to score the transaction as benign.
Detection and response requirements
|
Customer-controlled free text | Alert scores, SMR drafts | Possible × Major | High | OWASP LLM01:2025 |
| T-02 | Exposure of SMR content traditional Confidential report content leaks.
Detection and response requirements
|
Logs or provider retention | SMR data | Unlikely × Severe | High | - |
MITRE references are proposed by the model only when it is confident; verify them in ATLAS or ATT&CK.
Controls and residual risk
Parallel running, input sanitisation and evidence-linked drafts bring the residual risk to Medium.
Overall: inherent Critical → residual Medium
| ID | Control | Type | Addresses | Effectiveness | Priority |
|---|---|---|---|---|---|
| C-01 | Parallel run with the rules engine Keep the rules engine until detection parity is shown. |
Preventive | R-01 | High | Immediate |
| C-02 | Evidence-linked SMR drafts Every statement in a draft must cite a transaction or record; uncited text is flagged. |
Preventive | R-04, R-02 | Medium | Immediate |
| C-03 | Quality sampling of approvals Second-line review of a sample of approved SMRs. |
Detective | R-02 | Medium | Near-term |
| C-04 | Data minimisation and onshore option Send only necessary fields; assess an Australian region. |
Preventive | R-03 | Medium | Near-term |
Residual risk
| Risk | Inherent | Controls applied | Residual | Justification |
|---|---|---|---|---|
| R-01 Suspicious matters missed after replacing rules | Critical | C-01 | Medium | Parallel running prevents coverage loss. |
| R-02 Analysts approve drafts without real review | High | C-02, C-03 | Medium | Citations and sampling expose weak review. |
| R-03 Customer data processed offshore | High | C-04 | Medium | Minimisation reduces exposure. |
| R-04 Hallucinated facts in SMR narratives | High | C-02 | Low | Uncited statements are flagged. |
Residual ratings assume the controls are implemented and work as described. They are projections until the controls are verified.
Remediation plan
0-30 days
| ID | Action | Owner | Priority |
|---|---|---|---|
| A-01 | Redesign as parallel run alongside the rules engine | AML Compliance Lead | Critical |
30-90 days
| ID | Action | Owner | Priority |
|---|---|---|---|
| A-02 | Build citation checks into SMR drafts | AI Engineering | High |
| A-03 | Assess cross-border data handling of the US-hosted model | Privacy Officer | High |
90+ days
No actions.
Red team: attack the AI
The red team designs 8 adversarial tests for this scenario and judges whether the proposed controls would stop each one.
Agent trace
Pipeline: Scenario Intelligence → Risk, Compliance and Security analysts (in parallel) → Control Analyst → Supervisor. Model: Worked example: prepared in advance, not generated live.
- 06:37:15example: Worked example loaded (prepared in advance; no model was called).
Downloads
Decision support only, not legal or regulatory advice. AI-generated ratings and citations must be reviewed by a qualified person before they are relied on.