Measure class errors and capacity before cost-sensitive: Which next step best supports threshold selection?
Measure class-specific errors and review capacity before selecting a cost-sensitive threshold.
The question
A financial document processor flags submissions for manual fraud review. Missing fraud has severe consequences, but the review team has fixed capacity. Overall accuracy is high; class-specific errors and available capacity are unknown. Which next step best supports threshold selection?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Measure class errors and capacity before cost-sensitive tuning ✓False-positive and false-negative rates, fraud severity, and review capacity are needed to choose a defensible cost-sensitive threshold.
- Lower the threshold immediatelyThis may reduce missed fraud but could exceed the team’s capacity; the relevant tradeoff has not been measured.
- Set equal weights for missed fraud and unnecessary reviewEqual weights contradict the stated severe consequence of missed fraud unless evidence establishes equivalent costs.
- Maximize validation-set accuracy before changing the thresholdOverall accuracy can conceal the asymmetric error costs and says nothing about whether review demand is feasible.
The trap
For asymmetric risks, obtain class-specific error rates and capacity constraints before changing a threshold. How to remember it
Measure class-specific errors and review capacity before selecting a cost-sensitive threshold.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Compare line-specific errors: What conclusion is most justified? →
- Conduct targeted security and privacy testing: What evidence gap is decisive? →
- Test re-identification and residual disclosure risks: What is the most defensible response? →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.