Ongoing outcome and prediction-quality metrics compared: To satisfy continuous-monitoring requirements, which
Tracking outcome and prediction-quality metrics against a baseline detects silent accuracy decay that uptime and volume signals miss.
The question
A deployed credit model shows no system errors and stable request volume, yet the fraction of its approvals that later default is slowly rising. To satisfy continuous-monitoring requirements, which signal should the team track to catch this kind of silent degradation?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Server CPU and memory utilization over time, since resource-usage trends are the earliest indicator that a deployed model's predictions are beginning to degrade.Resource metrics reflect infrastructure load, not prediction quality, so they will not reveal rising default rates among approvals.
- Ongoing outcome and prediction-quality metrics compared against a baseline, so gradual drops in real-world accuracy are detected despite no system errors. ✓Tracking outcome quality against a baseline surfaces silent accuracy decay that uptime and volume metrics cannot reveal.
- The count of API requests received per hour, since a stable request volume confirms the model is still performing as expected for its user base.Request volume shows usage, not correctness; the scenario already notes volume is stable while accuracy silently degrades.
- The application's error and exception logs, since a low error rate demonstrates the model continues to serve valid predictions to every incoming request.Error logs catch failures, not correct-but-wrong predictions; a model can return well-formed answers that are increasingly inaccurate.
The trap
Relying on infrastructure health, volume, or error logs to detect a model that silently loses accuracy while still responding. How to remember it
Tracking outcome and prediction-quality metrics against a baseline detects silent accuracy decay that uptime and volume signals miss.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- A model card summarizing intended use: Under III.C.1, which readiness artifact fits this need? →
- Threat modeling that systematically identifies potential: Which periodic activity fits this goal? →
- Log the incident in a central register with root cause: Which response best discharges the obligation to →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.
