Rebuild held-out tests with subgroup coverage and leakage | AIGP
7-day money-back guarantee — full refund within 7 days of purchase if you've completed under 20% of the questions. See pricing →
Certifications Tools Flashcards Career Paths Exam Guides Blog Pricing For Teams About

Language

✓ EnglishDeutschEspañolFrançaisPortuguês
Check readiness — free →

Rebuild held-out tests with subgroup coverage and leakage: Which approach satisfies both constraints?

AIGP Understanding How to Govern AI Development Hard

Leakage-free held-out testing with subgroup coverage addresses both incomplete representation and compromised evaluation validity.

The question

A retailer’s demand-planning model meets overall accuracy targets, but held-out data omitted a smaller regional subgroup and contains possible leakage from future sales. Release requires representative subgroup testing and leakage-free evaluation. Which approach satisfies both constraints?

Preparing for AIGP? Take the free 5-min readiness quiz →

  1. Rebuild held-out tests with subgroup coverage and leakage controls. ✓
    This approach supplies representative subgroup evidence while preventing leakage, satisfying both explicit release constraints.
  2. Release after confirming overall accuracy on the existing test set.
    Overall accuracy does not address the omitted subgroup, and suspected future-data leakage undermines confidence in the existing evaluation.
  3. Use a supplier benchmark covering similar retail demand patterns.
    A benchmark may offer contextual evidence, but it does not repair this model’s subgroup omission or possible leakage.
  4. Conduct red-team exercises focused on adversarial forecasting inputs.
    Red teaming can reveal unusual vulnerabilities, but it does not by itself establish representative subgroup coverage or leakage-free evaluation.
The trap
When acceptance criteria interact, select evidence that addresses every stated constraint, especially subgroup coverage and data leakage.

How to remember it

Leakage-free held-out testing with subgroup coverage addresses both incomplete representation and compromised evaluation validity.

How many of these would you get right?

One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.

Test your AIGP readiness — free

More Understanding How to Govern AI Development questions

Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development · Every answer, right and wrong, comes with its own explanation.