Version the model: Which documentation choice is most useful?
Reproducibility requires lineage across model, data, transformations, code, and evaluation artifacts, not merely a performance summary.
The question
A medical administration system, without giving clinical advice, routes incoming forms to processing queues. After an incident, reviewers need to reproduce the exact behavior, including preprocessing and training inputs. Which documentation choice is most useful?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Keep evaluation reports while allowing preprocessing changes without records.Unrecorded preprocessing changes can alter inputs and invalidate comparisons, even when evaluation reports remain available.
- Store only the latest training dataset and deployment configuration.Latest artifacts cannot reconstruct earlier behavior when prior data, preprocessing, model parameters, or evaluation splits differed.
- Record the model’s name and final accuracy.A model name and accuracy do not identify the exact inputs, transformations, or code needed to reproduce the incident behavior.
- Version the model, data, transformations, code, and evaluation split. ✓Versioning each behavior-relevant component enables reproducibility and connects observed results to specific training and testing conditions.
The trap
Include transformations and test splits, not just the model file and headline metric. How to remember it
Reproducibility requires lineage across model, data, transformations, code, and evaluation artifacts, not merely a performance summary.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Adjudicate sampled disagreements and revise the labeling: What should the team do first? →
- Build a targeted defect test set and require: Which evaluation change is most defensible? →
- Provenance documentation alone establishes suitability: Considering provenance documentation specifically, →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.
