Use targeted rare-case testing: Which evaluation response best | AIGP
7-day money-back guarantee — full refund within 7 days of purchase if you've completed under 20% of the questions. See pricing →
Certifications Tools Flashcards Career Paths Exam Guides Blog Pricing For Teams About

Language

✓ EnglishDeutschEspañolFrançaisPortuguês
Check readiness — free →

Use targeted rare-case testing: Which evaluation response best addresses both rarity and consequence?

AIGP Understanding How to Govern AI Development Hard

Rare severe errors need targeted evidence; aggregate accuracy and random expansion may leave the critical uncertainty unresolved.

The question

A public-benefit screening model shows 98% overall accuracy, but only a few severe eligibility errors appear in historical data. Those errors can wrongly deny essential support, and the general test set contains few such cases. Which evaluation response best addresses both rarity and consequence?

Preparing for AIGP? Take the free 5-min readiness quiz →

  1. Report overall accuracy with a warning about limited severe-error evidence
    A warning acknowledges uncertainty but does not generate evidence about rare, consequential errors or their operational impact.
  2. Lower the acceptance threshold to reduce potential denials
    Changing the threshold without measuring its effect on severe false negatives and workload is not evidence-based.
  3. Collect a much larger random test set and rely on its aggregate accuracy
    Random expansion may still contain too few severe cases and preserve the masking effect of aggregate accuracy.
  4. Use targeted rare-case testing, measure severe-error rates, and document uncertainty ✓
    Targeted testing increases evidence on rare cases, while severe-error measurement and uncertainty documentation address consequence and limited data.
The trap
Rare and severe does not call for aggregate accuracy alone; target the cases and report uncertainty around their error rates.

How to remember it

Rare severe errors need targeted evidence; aggregate accuracy and random expansion may leave the critical uncertainty unresolved.

How many of these would you get right?

One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.

Test your AIGP readiness — free

More Understanding How to Govern AI Development questions

Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development · Every answer, right and wrong, comes with its own explanation.