Constrain purpose and inputs: Which design response best addresses the foreseeable misuse?
Repurposed eligibility recommendations and unnecessary bank data require purpose and input controls plus realistic misuse testing.
The question
A public-benefit agency pilots a generative assistant that summarizes applicant interviews. Caseworkers begin asking it to recommend eligibility outcomes, while applicants upload complete bank statements after a helpful prompt. Evidence about privacy leakage and recommendation accuracy is incomplete. Which design response best addresses the foreseeable misuse?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Re-test summary quality using the original interview dataset before expansion.Original-task testing cannot establish safety for eligibility recommendations or reveal whether the new financial inputs create privacy leakage.
- Block eligibility-related prompts but retain unrestricted financial-document uploads for caseworker review.Prompt blocking addresses one misuse route but leaves unnecessary sensitive-input collection and possible privacy leakage unresolved.
- Issue a use notice directing caseworkers to keep the assistant within its summary role.A notice clarifies expectations but does not technically or procedurally address repurposed eligibility recommendations or unnecessary bank-statement collection.
- Constrain purpose and inputs; test foreseeable misuse. ✓This response limits repurposing for eligibility decisions, reduces unnecessary financial-data exposure, and evaluates realistic misuse before expansion.
The trap
When use changes and inputs become more sensitive, address both the new purpose and the new data exposure. How to remember it
Repurposed eligibility recommendations and unnecessary bank data require purpose and input controls plus realistic misuse testing.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Reassess the affected population and data assumptions: Before relying on the earlier validation, what should →
- Escalate documented residual risk to the authorized: What should happen next? →
- Reassess purpose and benefits against alternatives: What action is required? →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.
