Set thresholds using documented error costs: Which control is most direct?
Different consequences require measuring and choosing thresholds according to documented false-positive and false-negative costs.
The question
An employment-screening model flags applicants for rejection. A false positive rejects a qualified applicant, while a false negative advances an unqualified applicant for further review; the consequences differ. Privacy, provenance, representative testing, and subgroup measurement are otherwise satisfied. Which control is most direct?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Optimize overall classification accuracyOverall accuracy treats errors alike and may choose a threshold misaligned with the distinct consequences of each error type.
- Report only the combined error rateA combined rate hides whether costly false positives or less costly false negatives dominate the screening outcomes.
- Use identical thresholds for every applicantA uniform threshold may be operationally simple, but it does not address the unequal consequences identified in the scenario.
- Set thresholds using documented error costs ✓Cost-sensitive threshold selection reflects the different consequences of false positives and false negatives in this workflow.
The trap
When error consequences differ, prioritize cost-sensitive error analysis over undifferentiated accuracy. How to remember it
Different consequences require measuring and choosing thresholds according to documented false-positive and false-negative costs.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Report error rates by subgroup: Which additional control is most direct? →
- Test the model and service against attacks: Which missing control most directly addresses the evidence gap? →
- Perform privacy-leakage and re-identification testing: Which control is most direct? →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.