Require subgroup outcome testing: What is most defensible? | AIGP
7-day money-back guarantee — full refund within 7 days of purchase if you've completed under 20% of the questions. See pricing →
Certifications Tools Flashcards Career Paths Exam Guides Blog Pricing For Teams About

Language

✓ EnglishDeutschEspañolFrançaisPortuguês
Check readiness — free →

Require subgroup outcome testing: What is most defensible?

AIGP Understanding How to Govern AI Development Hard

Input drift is a signal, not proof; subgroup outcome testing is needed before deciding whether continued use remains acceptable.

The question

A school district uses an admissions-support model. Monitoring shows a changed applicant-data distribution, but subgroup outcome quality has not been measured since deployment. The vendor supplies only aggregate accuracy. The district’s governance lead must choose the next owner action before continued use. What is most defensible?

Preparing for AIGP? Take the free 5-min readiness quiz →

  1. Continue unchanged while collecting aggregate accuracy during the next annual review.
    Aggregate accuracy cannot resolve subgroup outcome uncertainty, and delaying measurement leaves the identified monitoring gap unmanaged.
  2. Pause all use permanently until a replacement model is procured.
    A permanent replacement is premature because the evidence gap calls for contextual testing and proportionate risk treatment first.
  3. Have the vendor confirm that distribution change explains the concern.
    A vendor explanation does not provide outcome validation, subgroup evidence, or assurance that changed inputs caused degraded performance.
  4. Require subgroup outcome testing, then reassess continuation with documented thresholds. ✓
    Changed inputs trigger investigation, while missing subgroup measurements require suitable validation before a continuation decision.
The trap
When drift and incomplete evidence coexist, identify the missing outcome measure rather than inferring harm from drift alone.

How to remember it

Input drift is a signal, not proof; subgroup outcome testing is needed before deciding whether continued use remains acceptable.

How many of these would you get right?

One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.

Test your AIGP readiness — free

More Understanding How to Govern AI Development questions

Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development · Every answer, right and wrong, comes with its own explanation.