Run disaggregated tests and document coverage: Under NIST MEASURE, what does the evidence most directly
MEASURE requires subgroup analysis, representative testing, and documented uncertainty rather than reliance on aggregate accuracy.
The question
A housing administrator reports strong overall accuracy for an application-screening model. A separate review finds materially different error patterns for two applicant groups, and the test dataset underrepresents one group. Governance, context mapping, and treatment planning are otherwise complete. Under NIST MEASURE, what does the evidence most directly require?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Approve the model using aggregate accuracy.Aggregate accuracy can conceal subgroup harms and does not resolve the underrepresentation identified in testing.
- Proceed with treatment while leaving testing limitations undocumented.MANAGE can address residual risk only on a credible evidentiary basis; omitting known testing limitations weakens risk prioritization and deployment decisions.
- Expand benchmarking before making a deployment decision.Additional benchmarking may help, but this option does not specifically require subgroup analysis, representativeness assessment, or documented uncertainty.
- Run disaggregated tests and document coverage and uncertainty. ✓MEASURE requires suitable quantitative and qualitative evaluation, representative testing, and documentation of uncertainty and testing limitations when subgroup error differences and underrepresentation appear.
The trap
When subgroup harms or data coverage are in question, select disaggregated measurement with explicit limitations. How to remember it
MEASURE requires subgroup analysis, representative testing, and documented uncertainty rather than reliance on aggregate accuracy.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How Laws, Standards and Frameworks Apply to AI questions
- Map intended purpose: Which NIST AI RMF MAP conclusion is most appropriate? →
- Prioritize treatment: Which MANAGE action is most direct? →
- Iterate among functions as evidence and context change: Which interpretation best reflects the relationship →
- All 394 Understanding How Laws, Standards and Frameworks Apply to AI questions →
Part of the Certsqill AIGP question bank · Understanding How Laws, Standards and Frameworks Apply to AI ·
Every answer, right and wrong, comes with its own explanation.
