Preserve versioned prompts: Which evidence should the service preserve to support comparison and corrective
Versioned prompts, evidence, outputs, and actions permit reconstruction of what changed and how the incident affected customers.
The question
A customer-service model update is followed by incorrect refund guidance. The vendor has rolled back the update, and managers are investigating whether agents relied on retrieved policy text or generated content. Which evidence should the service preserve to support comparison and corrective action?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Keep the vendor’s incident summary and the aggregate refund count.Those summaries show broad impact but cannot reconstruct the model version, evidence supplied, generated answer, or individual action sequence.
- Preserve versioned prompts, retrieved context, outputs, and agent actions. ✓These linked records enable comparison across versions, separation of retrieval and generation causes, and reconstruction of consequential agent behavior.
- Record the final corrected policy and notify agents about the rollback.Corrective communication is useful, but it does not preserve historical evidence needed to analyze the incident or validate remediation.
- Save representative incorrect answers and the date of rollback.Examples and timing help illustrate the event but omit retrieved context, complete outputs, prompts, and actions needed for reliable diagnosis.
The trap
Preserve the model version and the full interaction chain, including retrieved evidence and consequential human actions. How to remember it
Versioned prompts, evidence, outputs, and actions permit reconstruction of what changed and how the incident affected customers.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Deployment and Use questions
- Apply a documented restart gate against those conditions: Which action is most defensible? →
- Require policy-source verification and escalation when: Which mitigation addresses the remaining risk most →
- Silent subgroup errors may remain undetected: Which specific risk remains insufficiently addressed by that →
- All 424 Understanding How to Govern AI Deployment and Use questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Deployment and Use ·
Every answer, right and wrong, comes with its own explanation.
