Design against foreseeable misuse: What should designers address?
Designers must address foreseeable misuse, including repeated attempts to infer sensitive conditions.
The question
Under the organization's internal AI-use policy, a research-administration assistant summarizes participant records. Staff commonly ask it to identify people who may have undisclosed medical conditions, although that use is outside the stated purpose. What should designers address?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Publish a general reminder about the approved purpose.A reminder may clarify expectations, but it is insufficient when recurring misuse is already foreseeable in practice.
- Improve summary fluency for approved workflows.Better summaries support the intended task but do not address recurring requests to infer sensitive participant information.
- Design against foreseeable misuse. ✓Repeated requests to infer sensitive conditions make this misuse foreseeable and relevant to safeguards, access controls, and use boundaries during design.
- Evaluate only whether the model reproduces source records.Faithful reproduction does not resolve privacy risks created when users repurpose records to infer sensitive conditions.
The trap
Observed or predictable repurposing belongs in contextual risk analysis even when it violates the stated intended use. How to remember it
Designers must address foreseeable misuse, including repeated attempts to infer sensitive conditions.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Select the least resource-intensive model meeting safety: Which decision best reflects the tradeoff? →
- Escalate to the authorized accountable decision owner: Who should decide the next step? →
- Compare service benefits with energy impacts before: Which decision approach is most defensible? →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.