Use grouped, deduplicated partitions by employee or case: Which control is most direct?
Deduplicate records and split related requests by employee or case to prevent leakage.
The question
An employee-assistance model was evaluated after random row-level splitting. Many records are duplicates or repeated versions of the same employee request, so near-identical content appears in both partitions. Privacy, provenance, labels, and acceptance criteria are otherwise satisfied. Which control is most direct?
Preparing for AIGP? Take the free 5-min readiness quiz →
- Use grouped, deduplicated partitions by employee or case. ✓Deduplication and grouping prevent near-identical requests from crossing training and held-out evaluation partitions.
- Report uncertainty intervals and document duplicate prevalence alongside the current scores.These actions describe uncertainty and the problem but do not correct leakage from related records appearing across partitions.
- Repeat random splits and average the resulting performance estimates.Repeated random splitting can place the same employee’s related requests in both partitions and preserve the optimistic bias.
- Use a fresh holdout set.A fresh set may help, but it does not by itself remove related records from the existing partitions or establish group separation.
The trap
When records share an employee, case, or near-identical content, split by group rather than row. How to remember it
Deduplicate records and split related requests by employee or case to prevent leakage.
How many of these would you get right?
One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.
Test your AIGP readiness — freeMore Understanding How to Govern AI Development questions
- Keep training and test records separate: Which missing control is most direct before relying on the reported →
- Report error rates by subgroup: Which additional control is most direct? →
- Set thresholds using documented error costs: Which control is most direct? →
- All 426 Understanding How to Govern AI Development questions →
Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development ·
Every answer, right and wrong, comes with its own explanation.