Specify an operational human fallback: What should be specified during design?
A human fallback must be operationally defined so competent people can intervene when automated recommendations become unsafe or unreliable.
The question
A software procurement team is evaluating an AI system that may produce incorrect recommendations. The procurement requirement states that a competent person must be able to intervene when the system is uncertain or harmful. What should be specified during design?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Add a human reviewer’s name to the procurement record.Naming a reviewer does not establish time, information, authority, or an operational alternative for intervention.
- Require vendors to describe their general safety culture.Vendor culture information may be useful, but it does not create a concrete fallback for incorrect recommendations.
- Schedule periodic accuracy reviews after implementation.Periodic reviews support monitoring, but they do not ensure immediate intervention or an alternative process when harm is suspected.
- Specify an operational human fallback. ✓A defined fallback gives competent people a practical alternative when intervention is needed, satisfying the procurement requirement during design.
The trap
Human-in-the-loop language is insufficient unless intervention authority and an alternative process are operational. How to remember it
A human fallback must be operationally defined so competent people can intervene when automated recommendations become unsafe or unreliable.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- The business resolution metric adequately represents: Which prior assumption no longer holds? →
- Reassess the affected population and data assumptions: Before relying on the earlier validation, what should →
- Constrain purpose and inputs: Which design response best addresses the foreseeable misuse? →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.