File size: 933 Bytes
635f00b 4c84853 8e37fdc 4c84853 635f00b 81e7cac | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | ---
title: Dataset Integrity Auditor
emoji: 🔍
colorFrom: blue
colorTo: gray
sdk: gradio
sdk_version: 6.22.0
app_file: app.py
pinned: false
license: mit
datasets:
- xlangai/spider
- openai/gsm8k
- openai/openai_humaneval
- birdsql/bird-critic-1.0-open
- tatsu-lab/alpaca
tags:
- data-quality
- benchmark
- contamination
- deduplication
- evaluation
short_description: Audit any HF dataset for duplicates and leakage
---
# Dataset Integrity Auditor
How much of a benchmark is actually redundant? Paste a Hugging Face dataset id.
Separates records that are **identical across every evaluated field** from
records that merely **share a question stem**. Naive similarity lumps them
together and overstates duplication — by 9x on Spider's training split.
Six benchmarks audited so far, all clean. Code and write-up:
https://github.com/ashishsinha1602/dataset-integrity-audit
|