Record datasets and versions: Which documentation approach best satisfies all three aims under the
Only recording dataset versions, configurations, metrics, thresholds, and outcomes makes results reproducible, which is what validation, compliance, and risk defense require.
The question
An external auditor challenges a bank's claim that its credit model was tested fairly, and asks the bank to reproduce the reported test results months later. The team must document the training and testing process to validate results, establish compliance, and manage risk. Which documentation approach best satisfies all three aims under the reproducibility demand?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Archive only the final aggregate accuracy and fairness scores with sign-off dates, since headline metrics are what auditors and regulators ultimately rely upon.Final scores without the underlying data versions and configurations cannot be reproduced, so they fail the auditor's reproducibility and validation demand.
- Keep the reviewers' narrative sign-off memos describing that testing was thorough, since documented attestations by qualified staff carry the evidentiary weight needed.Narrative attestations assert diligence but provide no reproducible artifacts, so they cannot validate the specific results being challenged.
- Store the source code repository with commit history alone, since the pipeline code fully determines the outputs and therefore recreates any result on demand.Code alone omits the exact data versions, seeds, and thresholds; without them the same code can yield different results, defeating reproducibility.
- Record datasets and versions, test configurations, metrics, thresholds, and outcomes so results can be independently reproduced and defended against the challenge. ✓Capturing dataset versions, configurations, metrics, thresholds, and outcomes lets the results be reproduced and validated, which is what compliance and risk defense require.
The trap
Believing final metrics, sign-off memos, or code history alone make test results reproducible. How to remember it
Only recording dataset versions, configurations, metrics, thresholds, and outcomes makes results reproducible, which is what validation, compliance, and risk defense require.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Remove or correct the leaking feature: What is the most appropriate response? →
- Complete the applicable conformity assessment and prepare: Which combination is the correct precondition →
- Continuously monitor performance and drift metrics: Which approach best satisfies both the ongoing-monitoring →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.
