Investigate the underrepresentation: Following good practice for managing issues and risks during training and
A remediable performance gap found in testing should be investigated, fixed and re-tested before the model progresses.
The question
During testing, the team discovers the model performs far worse on a subgroup that is underrepresented in the training data. Following good practice for managing issues and risks during training and testing, what is the most appropriate next step?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Proceed to release and add a disclaimer noting reduced accuracy for the subgroup, since documenting the limitation is sufficient here.Plausible but wrong: disclosure does not manage the risk; a known, remediable performance gap should be addressed before progressing.
- Adjust the evaluation metric to weight the subgroup less so the overall reported score meets the project's predefined release threshold.Plausible but wrong: reweighting the metric hides the problem rather than managing it, undermining the integrity of testing.
- Investigate the underrepresentation, remediate the data or training approach, and re-test before the model is allowed to progress. ✓Correct: identifying, remediating and re-testing the underlying data issue is the proper way to manage a risk surfaced during testing.
- Remove the underrepresented subgroup from the test set so results reflect only the population the model handles most reliably.Plausible but wrong: dropping the subgroup from testing conceals the risk and produces a misleadingly favorable evaluation.
The trap
Confusing disclosing or hiding a known issue with actually managing the underlying risk. How to remember it
A remediable performance gap found in testing should be investigated, fixed and re-tested before the model progresses.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Bias testing that measures whether model outcomes differ: Among the planned testing activities, which type →
- Recording datasets: Which practice best satisfies all three needs at once? →
- Delay the launch until conformity requirements are: What is the most defensible release-readiness decision? →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.
