Continuous monitoring of accuracy: Which design best fits all three needs?
A monitoring design covering accuracy, drift, and subgroup fairness with alerts and a retraining cadence meets all three needs.
The question
A model operates in a fast-changing environment, serves a diverse population, and is subject to reporting expectations on fairness. The team is designing its continuous-monitoring program. Which design best fits all three needs?
Preparing for AIGP? Take the free 5-min readiness quiz →
- A single annual accuracy check against the original test set, which is efficient and avoids the noise introduced by frequent production-performance measurements.Wrong because an annual check misses drift and provides no subgroup fairness visibility between reviews.
- Monitoring aggregate accuracy only, since subgroup and drift metrics add complexity without materially changing the overall system-level performance picture over time.Plausible but wrong because aggregate accuracy can hide both drift and disparate subgroup outcomes.
- Continuous monitoring of accuracy, data drift, and subgroup fairness against defined thresholds, with alerting and a scheduled retraining and revalidation cadence. ✓Correct because this design covers drift, fairness across subgroups, and a maintenance cadence together.
- Manual spot checks performed whenever staff happen to have spare capacity, providing flexible oversight without committing to any fixed measurement schedule at all.Plausible but wrong because ad hoc checks lack the consistency and coverage the three needs require.
The trap
Relying on aggregate accuracy or infrequent checks, which mask data drift and disparate subgroup outcomes. How to remember it
A monitoring design covering accuracy, drift, and subgroup fairness with alerts and a retraining cadence meets all three needs.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Deployment and Use questions
- Integrate the controls into deployment gates: Which best achieves this? →
- A recurring program combining red teaming: Which combination of periodic activities best provides that →
- Recording incidents: Which governance activity does this best describe? →
- All 424 Understanding How to Govern AI Deployment and Use questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Deployment and Use ·
Every answer, right and wrong, comes with its own explanation.
