README.md

metadata

license: cc-by-4.0
extra_gated_heading: Access BioMysteryBench
extra_gated_prompt: >-
  BioMysteryBench is provided for **evaluation and benchmarking only**. By
  requesting access you agree that you will not use this benchmark (problem
  statements, answer rubrics, task formulation) to train, fine-tune, reinforce,
  or distill any machine-learning model. You may use it to construct evaluation
  prompts at inference time and to publish benchmark results with attribution to
  Anthropic.
extra_gated_fields:
  Affiliation: text
  I agree to use this benchmark (problem statements, answer rubrics, task formulation) for evaluation only and NOT for training, fine-tuning, or distillation of any model: checkbox
extra_gated_button_content: Agree and request access

BioMysteryBench (preview)

A 5-problem preview of BioMysteryBench, a bioinformatics research benchmark created by Anthropic. Each problem provides anonymized biological data files and asks a question that requires real analysis to answer — the source dataset cannot be looked up.

v11 (2026-07): preview refreshed — hb022 and hb053 were removed from the benchmark; hb024 and hb035 replace them here. See CHANGELOG.md.

Contents

• problems.csv / problems.parquet — one row per problem (id, question, answer_rubric, allowed_domains, human_solvable)
• data.zip — <id>/… data files for all 5 problems; extract into the working directory before solving

Rules

Looking up GEO/SRA/ENA/BioProject accession IDs to identify the original dataset, source publication, or study metadata is not allowed. Standard bioinformatics database use (gene ID lookup, sequence annotation, reference genome download, BLAST) is allowed.

Problems in this preview

id	human_solvable	question
hb020	yes	What organism does this crystal structure belong to? Provide the answer using binomial nomenclature …
hb002	yes	What bacteria is found in this sequenced dataset? Provide the bacteria's scientific name as the answ…
recqgsfxqqodhjens	yes	Identify the transcription factor whose binding sites are represented by the provided peaks data. Pr…
hb024	no	This dataset involves microbial communities from tissues of non-human species. How many tissue group…
hb035	no	What is the name of the most common microRNA associated with the genes given in this data? (Give the…

The full 90-problem set is at Anthropic/BioMysteryBench-full (gated, evaluation-only).