Keep training and test records separate: Which missing control is most direct before relying on the reported
Training and test records must remain separate so evaluation measures generalization rather than memorization or leakage.
The question
A financial-document classifier was trained and evaluated using records drawn from one combined pool. Privacy, provenance, representative coverage, and acceptance metrics are otherwise satisfied. Which missing control is most direct before relying on the reported evaluation?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Report only the highest-performing document categorySelective reporting obscures performance variation and does not correct contamination between training and test records.
- Keep training and test records separate ✓Separating records prevents training information from contaminating evaluation, making reported performance more meaningful.
- Add more documents to the combined poolA larger combined pool does not prevent overlap or leakage between training and evaluation records.
- Replace the classifier with a rules-based workflowChanging system type avoids neither the stated evaluation problem nor the need for independent testing evidence.
The trap
If training and evaluation draw from one overlapping pool, fix separation before interpreting performance metrics. How to remember it
Training and test records must remain separate so evaluation measures generalization rather than memorization or leakage.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Test proxy effects and revise affected features: Which control most directly addresses the concern? →
- Use grouped, deduplicated partitions by employee or case: Which control is most direct? →
- Report error rates by subgroup: Which additional control is most direct? →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.
