Worked example: Autonomous loan approval (Australia)
Scenario as entered
An Australian retail bank wants to deploy an autonomous AI agent that approves personal loans up to AUD 50,000 using customer transaction history, credit bureau data and open banking data. No human review is planned for approvals under AUD 30,000. The bank intends to use a third-party foundation model via API. Decisions will be delivered to customers in-app within 30 seconds.
Top risks
- Unfair or discriminatory credit outcomes
- Unsuitable loans approved
- Automated decisions without required transparency
Conditions for approval
- Human review of all declines and approvals above AUD 30,000
- Fairness and affordability testing before go-live
- Privacy policy updated for APP 1.7 to 1.9 before 10 December 2026
Executive narrative
Proceed only with a human gate for declines and larger loans, validated fairness and affordability, and updated transparency. Full automation of declines is not supported.
Agent consensus: Risk, Compliance and Security analysts agree the use case can proceed with conditions.
Scenario brief
An Australian bank wants an AI agent, built on a third-party foundation model, to approve personal loans up to AUD 50,000 in about 30 seconds, with no human review below AUD 30,000.
Jurisdictions
AustraliaData types
Transaction historyCredit bureau dataOpen banking (CDR) dataIdentity dataAffected parties
Retail loan applicantsThe bank's credit and conduct functionsRegulators (APRA, ASIC, OAIC)Risk domains for the specialists
CreditConduct and fairnessPrivacyModel riskThird-partyOperational resilienceMaterial information missing
- How declines are explained to applicants
- Model validation approach
- Which third-party model and where it is hosted
- Whether a human reviews declines
Alternative approaches
AI recommends, officer approves
The model scores and explains; a credit officer approves above a threshold and reviews all declines.
- + Lower conduct and fairness risk
- + Clearer accountability
- − Slower than full automation
Refine the interpretation and re-run (clears the current result)
Refine the interpretation
Your answers are sent to the agents as user-confirmed facts and override the AI's inference. Leave "(no change)" to keep the current value.
Risk register
Credit, conduct and privacy risks dominate because decisions are automated, consequential and hard to explain.
| Insignificant | Minor | Moderate | Major | Severe | |
|---|---|---|---|---|---|
| Almost Certain | |||||
| Likely | 2 | ||||
| Possible | 1 | 2 | |||
| Unlikely | |||||
| Rare | |||||
| Impact → (rows: likelihood). Cell number = count of risks; hover for IDs. | |||||
| ID | Category | Risk | Cause → consequence | Likelihood | Impact | Inherent |
|---|---|---|---|---|---|---|
| R-01 | Conduct | Unfair or discriminatory credit outcomes The model disadvantages groups of applicants. |
Bias in historical lending and proxy variables → Unfair treatment, remediation cost, regulatory action | Possible | Major | High |
| R-02 | Credit | Unsuitable loans approved Approvals that fail responsible-lending expectations. |
Model optimised for approval speed; weak affordability checks → Defaults and consumer harm | Possible | Major | High |
| R-03 | Privacy | Automated decisions without required transparency Applicants are not told how automated decisions are made. |
Privacy policy and notices not updated → Breach of APP 1.7 to 1.9 from 10 December 2026 | Likely | Moderate | High |
| R-04 | Third-party | Foundation model change or outage The provider changes or withdraws the model. |
Dependence on one external API → Silent change in decisions; service interruption | Possible | Moderate | Medium |
| R-05 | Model | Drift after deployment Performance degrades as conditions change. |
Economic change; data pipeline changes → Rising losses or unfair outcomes undetected | Likely | Moderate | High |
Regulatory impact and compliance gaps
Consumer credit, privacy and prudential obligations apply. Applicability statements are illustrative; confirm with legal counsel.
National Consumer Credit Protection Act 2009 (responsible lending) (Australia) applicable
Why it matters here: Personal loans to consumers are regulated credit.
Impact on this scenario: Automation must still meet responsible-lending obligations.
- Make reasonable inquiries and verify the consumer's financial situation
- Assess that the loan is not unsuitable
- Affordability rules
- Suitability assessment record
Privacy Act 1988 (APPs, including APP 1.7 to 1.9) (Australia) applicable
Why it matters here: Personal and credit information drives automated decisions.
Impact on this scenario: Policy and notice updates; data minimisation.
- Describe automated decisions in the privacy policy (from 10 December 2026)
- Handle credit information under the credit reporting rules
- Privacy impact assessment
- Updated privacy policy
APRA CPS 230 Operational Risk Management (Australia) applicable
Why it matters here: The bank is APRA-regulated and relies on a third-party model provider.
Impact on this scenario: The model provider may be a material service provider.
- Manage material service providers under a formal agreement
- Maintain critical operations within tolerance
- Service provider due diligence
- Business continuity plan
Anti-discrimination legislation (Commonwealth and state) (Australia) potentially applicable
Why it matters here: Credit decisions must not discriminate on protected attributes.
Impact on this scenario: Fairness testing is needed.
- Avoid direct and indirect discrimination
- Fairness testing across groups
Compliance gaps
| Area | Observation | Gap | Risk created | Remediation |
|---|---|---|---|---|
| Transparency | No plan to explain declines. | Applicants may not receive reasons. | High | Generate reason codes and review declines. |
Security threats
The decision API and the data pipeline are the main attack surfaces.
| ID | Threat | Attack vector | Assets at risk | L × I | Inherent | MITRE |
|---|---|---|---|---|---|---|
| T-01 | Adversarial manipulation of application data ai_specific Applicants game features to gain approval.
Detection and response requirements
|
Crafted transaction patterns or documents | Credit decisions | Possible × Major | High | MITRE ATLAS AML.T0043 |
| T-02 | Credential compromise of the model API traditional Attacker uses the bank's API key.
Detection and response requirements
|
Leaked secrets | Model access, Cost | Unlikely × Major | Medium | MITRE ATT&CK T1552 |
MITRE references are proposed by the model only when it is confident; verify them in ATLAS or ATT&CK.
Controls and residual risk
A human gate for declines and higher amounts, fairness testing and change control reduce residual risk to Medium.
Overall: inherent High → residual Medium
| ID | Control | Type | Addresses | Effectiveness | Priority |
|---|---|---|---|---|---|
| C-01 | Human review of declines and loans above AUD 30,000 A credit officer reviews every decline and every approval above the threshold. |
Preventive | R-01, R-02 | High | Immediate |
| C-02 | Fairness and affordability testing before release and monthly Test outcomes across groups and against affordability rules. |
Detective | R-01, R-02, R-05 | Medium | Immediate |
| C-03 | Automated-decision transparency Update the privacy policy and give applicants reasons for declines. |
Preventive | R-03 | High | Near-term |
| C-04 | Model change control and fallback Pin the model version, test before changes, keep a manual fallback. |
Preventive | R-04, R-05 | Medium | Near-term |
Residual risk
| Risk | Inherent | Controls applied | Residual | Justification |
|---|---|---|---|---|
| R-01 Unfair or discriminatory credit outcomes | High | C-01, C-02 | Medium | Human review and testing catch most unfair outcomes. |
| R-02 Unsuitable loans approved | High | C-01, C-02 | Medium | Affordability testing and review of large loans. |
| R-03 Automated decisions without required transparency | High | C-03 | Low | Policy and reasons address the obligation. |
| R-04 Foundation model change or outage | Medium | C-04 | Low | Version pinning and fallback. |
| R-05 Drift after deployment | High | C-02, C-04 | Medium | Monthly monitoring. |
Residual ratings assume the controls are implemented and work as described. They are projections until the controls are verified.
Remediation plan
0-30 days
| ID | Action | Owner | Priority |
|---|---|---|---|
| A-01 | Design the human review step and reason codes | Head of Credit | Critical |
30-90 days
| ID | Action | Owner | Priority |
|---|---|---|---|
| A-02 | Run fairness and affordability validation | Model Risk | High |
| A-03 | Update privacy policy and notices | Privacy Officer | High |
90+ days
| ID | Action | Owner | Priority |
|---|---|---|---|
| A-04 | Set up monthly drift and fairness monitoring | Model Risk | Medium |
Red team: attack the AI
The red team designs 8 adversarial tests for this scenario and judges whether the proposed controls would stop each one.
Agent trace
Pipeline: Scenario Intelligence → Risk, Compliance and Security analysts (in parallel) → Control Analyst → Supervisor. Model: Worked example: prepared in advance, not generated live.
- 06:37:15example: Worked example loaded (prepared in advance; no model was called).
Downloads
Decision support only, not legal or regulatory advice. AI-generated ratings and citations must be reviewed by a qualified person before they are relied on.