Create a deduplicated held-out clip set: Which evidence should | AIGP
7-day money-back guarantee — full refund within 7 days of purchase if you've completed under 20% of the questions. See pricing →
Certifications Tools Flashcards Career Paths Exam Guides Blog Pricing For Teams About

Language

✓ EnglishDeutschEspañolFrançaisPortuguês
Check readiness — free →

Create a deduplicated held-out clip set: Which evidence should reviewers request?

AIGP Understanding How to Govern AI Development Medium

Create a deduplicated held-out clip set so overlapping content cannot inflate performance estimates.

The question

A media company reports excellent caption-quality performance using a test set assembled from production clips. An audit finds near-identical edits of several clips also appear in training data. Approval requires determining whether the reported performance is inflated by duplication. Which evidence should reviewers request?

Preparing for AIGP? Take the free 5-min readiness quiz →

  1. Add more near-identical clips to training
    Adding related clips may worsen overlap and provides no independent estimate of performance without duplication effects.
  2. Create a deduplicated held-out clip set ✓
    A deduplicated held-out set prevents near-identical training examples from inflating evaluation and measures performance on independent content.
  3. Recalculate scores using the existing test set
    Recalculation on the same duplicated test set preserves contamination and cannot determine whether performance was inflated.
  4. Ask editors to rate the current captions
    Editor ratings may offer useful qualitative insight, but do not by themselves remove duplication from the quantitative evaluation.
The trap
When records overlap across splits, rebuild the evaluation set rather than merely recomputing existing scores.

How to remember it

Create a deduplicated held-out clip set so overlapping content cannot inflate performance estimates.

How many of these would you get right?

One of 1581 AIGP questions on Certsqill. Take a free five-minute check and see your score per domain — not one number, but which section to open tonight.

Test your AIGP readiness — free

More Understanding How to Govern AI Development questions

Part of the Certsqill AIGP question bank · Understanding How to Govern AI Development · Every answer, right and wrong, comes with its own explanation.