It should be logged and managed as an identified risk: As a matter of good AI development governance, what
Risks surfaced during testing must be identified, documented, and managed to resolution before release.
The question
During model testing, an evaluator observes that accuracy drops sharply for a minority subgroup. As a matter of good AI development governance, what should happen with this finding?
Preparing for AIGP? Take the free 5-min readiness quiz →
- It should be set aside until after launch, when real production data can confirm the subgroup effect.Plausible-sounding but deferring a known fairness risk past release contradicts pre-deployment risk management.
- It should be resolved by relabeling the subgroup so the reported accuracy metric no longer shows it.Plausible-sounding but hiding the disparity by relabeling conceals rather than manages the underlying risk.
- It should be logged and managed as an identified risk, then tracked to resolution before release. ✓Issues found in testing must be identified, documented, and managed to resolution, which is the point of testing.
- It should be treated as normal variance and excluded, since no single metric can be perfect for all.Plausible-sounding but dismissing a material subgroup gap as noise skips the required identification and management.
The trap
Rationalizing or hiding a testing finding instead of logging and managing it as a risk. How to remember it
Risks surfaced during testing must be identified, documented, and managed to resolution before release.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Examining whether model outcomes differ unjustifiably: Which purpose is served specifically by the bias →
- To validate results: Which statement best captures the primary purpose of this documentation? →
- To summarize the model's intended use: What is the main function of a model card? →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.