Reproduce the disclosure across controlled model: Which evidence should be prioritized?
Controlled variants separate model, retrieval, and configuration effects, addressing the unresolved cause while containment remains active.
The question
An internal employee-assistance chatbot exposed a confidential fragment during an incident. Containment is underway, but the root cause is unknown and the chatbot uses a supplier model with changing retrieval data. Before approving resumed access, leaders need evidence distinguishing a model leakage issue from a retrieval or configuration issue. Which evidence should be prioritized?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Compare incident counts before and after access restrictions.Incident counts indicate whether containment changed observed events, but they do not identify the component causing confidential disclosure.
- Review the supplier’s latest security certification.A certification may address supplier controls generally, but it cannot distinguish leakage caused by this deployment’s retrieval or configuration.
- Reproduce the disclosure across controlled model, retrieval, and configuration variants. ✓Controlled variants isolate interacting causes while preserving evidence about the deployed context and supplier dependency before access resumes.
- Ask the supplier to provide a root-cause report and revised incident-response procedure.Supplier analysis and procedures help governance, but an unverified report may not distinguish local retrieval or configuration causes.
The trap
When responsibility is uncertain, vary one relevant component at a time and test the actual deployment path. How to remember it
Controlled variants separate model, retrieval, and configuration effects, addressing the unresolved cause while containment remains active.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Run stratified tests on multilingual workflow samples: Which evidence best resolves the uncertainty? →
- Test rollback with current records: Which evidence is most important? →
- Obtain use-case tests covering applicant subgroups: Which evidence should it require? →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.
