AI use-case risk assessment
Worked example: Generative AI chatbot for internal HR policy · ARA-261031-2D523F · 2026-10-31 06:37:15 UTC · Agentic Risk Analyst v3.2.1 · Worked example: prepared in advance, not generated live
EXECUTIVE DECISION
APPROVE WITH CONDITIONS
Inherent Critical → residual Low
Scenario
A global manufacturer wants a RAG chatbot over its HR policy library so 40,000 employees can ask policy questions in natural language in English, Mandarin and Portuguese. Personal data (salary, medical leave) may appear in retrieved documents. The chatbot is embedded in Microsoft Teams and uses an external commercial LLM.
Executive narrative
Valuable and low-autonomy, but only if personal records stay out of the index and permissions are enforced and tested.
Top risks
- Employees see colleagues' salary or medical data
- External LLM retains HR data
Conditions
- Exclude personal records from the index
- Permission-aware retrieval tested for every role
- Provider contract prevents training on prompts
- Confirm employee locations and complete privacy assessments
Remediation plan
| Horizon | ID | Action | Owner | Priority |
|---|
| 0-30 days | A-01 | Remove personal records from the library scope | HR Systems Owner | Critical |
| 0-30 days | A-02 | Run leakage tests per role | Security Testing | Critical |
| 30-90 days | A-03 | Complete privacy impact assessments per jurisdiction | Privacy Officer | High |
Risk register
| ID | Risk | L | I | Inherent |
|---|
| R-01 | Employees see colleagues' salary or medical data | Likely | Major | Critical |
| R-02 | Wrong policy answers | Possible | Moderate | Medium |
| R-03 | External LLM retains HR data | Possible | Major | High |
| R-04 | Translation errors change policy meaning | Possible | Moderate | Medium |
Regulatory findings
Evidence status: model knowledge only; verify against source texts.
| Regulation | Applicability | Why it matters |
|---|
| Privacy Act 1988 (Australia) | potentially applicable | Applies to Australian operations; the employee records exemption may cover some handling by a private-sector employer, but not necessarily disclosure to an external provider. |
| General Data Protection Regulation (EU) 2016/679 (European Union) | potentially applicable | Applies if employees are in the EU; health data is a special category. |
| Regulation (EU) 2024/1689 (AI Act), Article 50 transparency (European Union) | potentially applicable | AI systems that interact with people must make clear they are interacting with AI. |
| Personal Information Protection Law (China); LGPD (Brazil) (China; Brazil) | not enough information | Mandarin and Portuguese suggest employees in these countries; locations are not stated. |
Security threats
| ID | Threat | Inherent | MITRE |
|---|
| T-01 | Permission bypass through retrieval | Critical | OWASP LLM08:2025 |
| T-02 | Indirect prompt injection in a policy document | Medium | OWASP LLM01:2025 |
Controls and residual risk
| ID | Control | Priority | Addresses |
|---|
| C-01 | Permission-aware retrieval | Immediate | R-01 |
| C-02 | Exclude personal records from the index | Immediate | R-01, R-03 |
| C-03 | Provider terms: no training, limited retention | Immediate | R-03 |
| C-04 | Grounded answers with citations and version control | Near-term | R-02, R-04 |
| C-05 | Leakage testing before go-live | Immediate | R-01 |
| Risk | Inherent | Residual | Justification |
|---|
| R-01 Employees see colleagues' salary or medical data | Critical | Low | Personal records excluded; permissions enforced and tested. |
| R-02 Wrong policy answers | Medium | Low | Grounding and versions. |
| R-03 External LLM retains HR data | High | Low | Contract and minimisation. |
| R-04 Translation errors change policy meaning | Medium | Medium | Citations allow checking; translation review still needed. |
Red team (resilience Medium)
| ID | Test | Result |
|---|
| AT-01 | Salary fishing | Blocked |
| AT-02 | Instruction in a policy document | Partially Blocked |
| AT-03 | Medical leave of a named person | Blocked |
Decision support only, not legal or regulatory advice. AI-generated ratings and citations must be reviewed by a qualified person before they are relied on.