Remove or correct the leaking feature: What is the most appropriate response?
Target leakage is a training-phase defect that must be fixed at source by removing the feature, retraining, and documenting the remediation.
The question
While testing a fraud-detection model, engineers notice it achieves near-perfect accuracy because a feature effectively encodes the label, a form of target leakage. The governance process requires that issues found during training and testing be identified and managed. What is the most appropriate response?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Proceed to deployment and add a post-market monitoring alert, since production drift detection will catch the inflated accuracy once real traffic arrives.Deploying a leaking model and relying on monitoring ships a known defect; leakage is a build-time flaw that monitoring will not remediate.
- Keep the feature but lower the decision threshold, since recalibrating the cutoff will offset the optimistic accuracy the leakage introduced into results.Threshold tuning cannot cure leakage; the model still learns from information unavailable at inference, so the evaluation stays invalid.
- Remove or correct the leaking feature, retrain, and document the issue and remediation so the risk is managed before the model advances. ✓Target leakage is a training-phase defect; removing the feature, retraining, and documenting the fix manages the risk at its source before release.
- Accept the result and record it as a strength, since the exceptional accuracy demonstrates the training data was high quality and fit for the purpose.Near-perfect accuracy from leakage is a red flag, not a strength; recording it as quality ignores the identified risk.
The trap
Reading near-perfect accuracy as evidence of quality rather than a symptom of leakage. How to remember it
Target leakage is a training-phase defect that must be fixed at source by removing the feature, retraining, and documenting the remediation.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Bias testing to measure error-rate disparities across: Which pairing of test activities most directly →
- Record datasets and versions: Which documentation approach best satisfies all three aims under the →
- Complete the applicable conformity assessment and prepare: Which combination is the correct precondition →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.
