The test estimate is likely inflated by record duplication: What interpretation is best supported?
Duplicates crossing splits create leakage, inflating test results; patient-level separation is needed to evaluate performance on new patients.
The question
A medical administration model predicts whether records need manual routing. The team randomly splits records, but many patients have multiple nearly identical entries appearing in both training and test sets. Test accuracy is 96%, while performance drops sharply on newly enrolled patients. What interpretation is best supported?
Preparing for AIGP? Take the free 5-min readiness quiz →
- The newly enrolled population necessarily requires a different target labelPerformance decline may reflect leakage or distribution differences; it does not establish that the target definition changed.
- The high test accuracy demonstrates adequate performance for deploymentDuplicated records undermine held-out independence, so high test accuracy cannot establish deployment readiness.
- The model needs additional duplicated records for stabilityAdding duplicates would reinforce leakage and may improve apparent scores without improving performance on new patients.
- The test estimate is likely inflated by record duplication ✓Near-duplicate patient records across splits allow memorization, making the reported test accuracy an unreliable estimate of generalization.
The trap
Check the unit of independence before trusting a random split; related records can invalidate an apparently held-out test. How to remember it
Duplicates crossing splits create leakage, inflating test results; patient-level separation is needed to evaluate performance on new patients.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Treat the gap as possible overfitting and investigate: Before approval, what does this contrast most directly →
- Compare line-specific errors: What conclusion is most justified? →
- Measure class errors and capacity before cost-sensitive: Which next step best supports threshold selection? →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.
