Build an adjudicated: Which control most directly addresses both constraints before model training?
Incomplete outcomes and inconsistent definitions require a consistently defined, quality-reviewed label set before training.
The question
A housing administrator has many applications, but outcome labels are incomplete and different reviewers used inconsistent definitions of “approved.” Privacy, provenance, deployment representation, held-out testing, and monitoring are otherwise satisfied. Which control most directly addresses both constraints before model training?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Calculate approval rates by subgroupSubgroup rates may reveal disparities, but they do not repair incomplete labels or establish a consistent approval definition.
- Collect additional historical applicationsMore applications may preserve inconsistent or missing labels, increasing volume without making the target outcome reliable.
- Build an adjudicated, consistently defined label set ✓A documented labeling protocol and adjudication process address missing outcomes and inconsistent reviewer definitions before training.
- Reserve a separate test partition from existing recordsPartitioning supports evaluation independence, but existing records still contain incomplete and inconsistently defined target labels.
The trap
When labels are both incomplete and inconsistently defined, fix the labeling process before selecting metrics or partitions. How to remember it
Incomplete outcomes and inconsistent definitions require a consistently defined, quality-reviewed label set before training.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Use a deployment-representative test sample: Which control is most direct? →
- Define and validate an independent administrative target: Which control is most direct? →
- Test proxy effects and revise affected features: Which control most directly addresses the concern? →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.