Test misuse scenarios and constrain unsupported inferences: What design decision best addresses the risk?
Evaluate foreseeable misuse and constrain sensitive inferences and unsupported promises rather than testing component accuracy alone.
The question
A customer-service team plans an AI assistant that summarizes conversations and drafts replies. Agents could use summaries to infer sensitive customer traits, and the assistant might draft unsupported promises about refunds. These uses are not part of the intended purpose, but are foreseeable in practice. What design decision best addresses the risk?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Train agents to follow the documented intended purpose.Training helps, but it does not test or constrain foreseeable misuse involving sensitive inferences and unauthorized commitments.
- Test misuse scenarios and constrain unsupported inferences and promises. ✓This addresses foreseeable use outside intended conditions by evaluating misuse and limiting outputs that could create privacy or integrity harms.
- Add a general disclaimer that the assistant may produce inaccurate content.A disclaimer may inform users but does not prevent sensitive inferences, unsupported promises, or monitor actual misuse.
- Test only whether summaries match the conversation text.Text matching does not assess foreseeable misuse for sensitive inferences or unsupported promises in customer-facing replies.
The trap
Assess how users may repurpose outputs, especially where privacy, integrity, or downstream commitments could be affected. How to remember it
Evaluate foreseeable misuse and constrain sensitive inferences and unsupported promises rather than testing component accuracy alone.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Use the smaller model: Which choice is most defensible? →
- Have an accountable education executive accept it: Which approach satisfies the requirement? →
- Select the lower-resource model and document: Which approach best satisfies the stated constraint? →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.
