Continuous monitoring of accuracy: Which design best fits all three needs?
A monitoring design covering accuracy, drift, and subgroup fairness with alerts and a retraining cadence meets all three needs.
The question
A model operates in a fast-changing environment, serves a diverse population, and is subject to reporting expectations on fairness. The team is designing its continuous-monitoring program. Which design best fits all three needs?
Preparing for AIGP? Take the free 5-min readiness quiz →
- A single annual accuracy check against the original test set, which is efficient and avoids the noise introduced by frequent production-performance measurements.Wrong because an annual check misses drift and provides no subgroup fairness visibility between reviews.
- Monitoring aggregate accuracy only, since subgroup and drift metrics add complexity without materially changing the overall system-level performance picture over time.Plausible but wrong because aggregate accuracy can hide both drift and disparate subgroup outcomes.
- Continuous monitoring of accuracy, data drift, and subgroup fairness against defined thresholds, with alerting and a scheduled retraining and revalidation cadence. ✓Correct because this design covers drift, fairness across subgroups, and a maintenance cadence together.
- Manual spot checks performed whenever staff happen to have spare capacity, providing flexible oversight without committing to any fixed measurement schedule at all.Plausible but wrong because ad hoc checks lack the consistency and coverage the three needs require.
The trap
Relying on aggregate accuracy or infrequent checks, which mask data drift and disparate subgroup outcomes. How to remember it
A monitoring design covering accuracy, drift, and subgroup fairness with alerts and a retraining cadence meets all three needs.
How many of these would you get right?
One of 499 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Deployment and Use questions
- On-premise deployment: Which deployment option most directly satisfies that constraint? →
- A control to deactivate or localize the AI system when: What does this policy primarily provide? →
- The data-use and training-rights clause governing customer: Which term in the vendor agreement should the →
- All 135 Understanding How to Govern AI Deployment and Use questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Deployment and Use ·
Every answer, right and wrong, comes with its own explanation.
