README_fastparquet_poc.md

metadata

tags:
  - security
  - vulnerability
  - poc
  - parquet
  - fastparquet
  - decompression-bomb
  - cwe-789
license: mit

fastparquet — Parquet PageHeader Decompression Bomb (PoC)

Repo: MBM7/fastparquet-parquet-decompression-bomb-poc
Status: Responsible disclosure — submitted to Huntr
Severity: High / CWE-789 (Uncontrolled Memory Allocation)

---

Summary

A crafted 492-byte .parquet file causes fastparquet to allocate gigabytes of memory before failing — enabling OOM-kill on any service that loads user-supplied Parquet files.

File size	Fake uncompressed_page_size	Peak allocation	Amplification
492 bytes	500,000,000	500 MB	1 : 1,016,260
492 bytes	2,000,000,000	2,000 MB	1 : 4,065,040

---

Root Cause

fastparquet/compression.py — decompress_data():

if algorithm.upper() in decom_into:
    x = np.empty(uncompressed_size, dtype='uint8')   # ← pre-allocates upfront
    decom_into[algorithm.upper()](
        np.frombuffer(data, dtype=np.uint8), x
    )
    return x

uncompressed_size is passed directly from ph.uncompressed_page_size (Thrift compact i32, PageHeader field 2) with no bound check at any point in the call chain:

fastparquet.ParquetFile.to_pandas()
  → read_col_group()          [core.py ~291]
    → _read_page()            [core.py ~20]
      → decompress_data(
            data,
            page_header.uncompressed_page_size,   ← from Thrift, unchecked
            codec
        )
          → np.empty(uncompressed_size)           ← GB allocation here

max_header_size or equivalent limit: does not exist in fastparquet.

---

Attack

Parquet format: PAR1 | PageHeader (Thrift compact) | compressed_data | ... | PAR1

PageHeader field 2 (uncompressed_page_size, type i32) is a zigzag-encoded varint at byte offset 7 in a minimal single-column file.

Replacing the 1-byte varint (value 4) with a 5-byte varint (value 500_000_000) produces a valid-looking file that triggers 500 MB of allocation on any codec in decom_into (GZIP, SNAPPY, ZSTD, BROTLI).

---

Reproduce

pip install fastparquet pyarrow
python poc_fastparquet_parquet_bomb.py

Expected output:

Crafted file size           : 492 bytes
Expected allocation         : 500,000,000 bytes
Amplification               : 1:1,016,260
Result                      : DecompressionError: unexpected end of file
Peak memory                 : 500 MB  ← allocation happened

---

Suggested Fix

fastparquet/compression.py, before np.empty():

MAX_UNCOMPRESSED = 256 * 1024 * 1024  # 256 MB configurable limit
if uncompressed_size > MAX_UNCOMPRESSED:
    raise ValueError(
        f"uncompressed_size {uncompressed_size} exceeds safety limit "
        f"{MAX_UNCOMPRESSED}. Possible decompression bomb."
    )

---

Environment

Package	Version
fastparquet	2026.5.0
pyarrow	25.0.0
Python	3.12

---

Files

File	Description
poc_fastparquet_parquet_bomb.py	PoC script (self-contained)
bomb_500mb.parquet	Crafted payload (492 bytes)

---

Discovered via empirical security research using UBDAF (Universal Bug Discovery & Analysis Framework) — Q2 CRITICAL pattern: arithmetic on externally-controlled value without bound check.