Report error rates by subgroup: Which additional control | AIGP
7-day money-back guarantee — full refund within 7 days of purchase if you've completed under 20% of the questions. See pricing →
Certifications Tools Flashcards Career Paths Exam Guides Blog Pricing For Teams About

Language

✓ EnglishDeutschEspañolFrançaisPortuguês
Check readiness — free →

Report error rates by subgroup: Which additional control is most direct?

AIGP Understanding How to Govern AI Development Medium

Aggregate accuracy can mask subgroup harms, so procurement should require disaggregated error-rate reporting.

The question

A procurement team sees 96% overall accuracy from a vendor’s classification system, but preliminary results suggest materially different error rates for two user groups. Privacy, provenance, representative testing, and acceptance criteria are otherwise satisfied. Which additional control is most direct?

Preparing for AIGP? Take the free 5-min readiness quiz →

  1. Extend the reporting period for aggregate accuracy
    Longer aggregate monitoring may improve trend visibility but still can hide materially different subgroup error rates.
  2. Report error rates by subgroup ✓
    Subgroup error rates reveal harms that aggregate accuracy can conceal and directly test the observed disparity concern.
  3. Average results across additional vendors
    Cross-vendor averaging can obscure variation and does not measure the current system’s group-specific error rates.
  4. Raise the overall accuracy threshold
    A stricter aggregate threshold may reject systems without identifying or measuring which groups experience different errors.
The trap
When group differences are suspected, choose disaggregated error metrics rather than another aggregate score.

How to remember it

Aggregate accuracy can mask subgroup harms, so procurement should require disaggregated error-rate reporting.

How many of these would you get right?

One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.

Test your AIGP readiness — free

More Understanding How to Govern AI Development questions

Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development · Every answer, right and wrong, comes with its own explanation.