README_fastparquet_poc.md

---
tags:
  - security
  - vulnerability
  - poc
  - parquet
  - fastparquet
  - decompression-bomb
  - cwe-789
license: mit
---

# fastparquet — Parquet PageHeader Decompression Bomb (PoC)

**Repo:** `MBM7/fastparquet-parquet-decompression-bomb-poc`  
**Status:** Responsible disclosure — submitted to Huntr  
**Severity:** High / CWE-789 (Uncontrolled Memory Allocation)

---

## Summary

A crafted **492-byte** `.parquet` file causes `fastparquet` to allocate
**gigabytes of memory** before failing — enabling OOM-kill on any service
that loads user-supplied Parquet files.

| File size | Fake `uncompressed_page_size` | Peak allocation | Amplification |
|-----------|-------------------------------|-----------------|---------------|
| 492 bytes | 500,000,000                   | **500 MB**      | 1 : 1,016,260 |
| 492 bytes | 2,000,000,000                 | **2,000 MB**    | 1 : 4,065,040 |

---

## Root Cause

`fastparquet/compression.py` — `decompress_data()`:

```python
if algorithm.upper() in decom_into:
    x = np.empty(uncompressed_size, dtype='uint8')   # ← pre-allocates upfront
    decom_into[algorithm.upper()](
        np.frombuffer(data, dtype=np.uint8), x
    )
    return x
```

`uncompressed_size` is passed directly from `ph.uncompressed_page_size`
(Thrift compact i32, PageHeader field 2) **with no bound check** at any
point in the call chain:

```
fastparquet.ParquetFile.to_pandas()
  → read_col_group()          [core.py ~291]
    → _read_page()            [core.py ~20]
      → decompress_data(
            data,
            page_header.uncompressed_page_size,   ← from Thrift, unchecked
            codec
        )
          → np.empty(uncompressed_size)           ← GB allocation here
```

`max_header_size` or equivalent limit: **does not exist** in fastparquet.

---

## Attack

Parquet format: `PAR1 | PageHeader (Thrift compact) | compressed_data | ... | PAR1`

PageHeader field 2 (`uncompressed_page_size`, type i32) is a
zigzag-encoded varint at byte offset 7 in a minimal single-column file.

Replacing the 1-byte varint (value `4`) with a 5-byte varint
(value `500_000_000`) produces a valid-looking file that triggers 500 MB
of allocation on any codec in `decom_into` (GZIP, SNAPPY, ZSTD, BROTLI).

---

## Reproduce

```bash
pip install fastparquet pyarrow
python poc_fastparquet_parquet_bomb.py
```

Expected output:
```
Crafted file size           : 492 bytes
Expected allocation         : 500,000,000 bytes
Amplification               : 1:1,016,260
Result                      : DecompressionError: unexpected end of file
Peak memory                 : 500 MB  ← allocation happened
```

---

## Suggested Fix

`fastparquet/compression.py`, before `np.empty()`:

```python
MAX_UNCOMPRESSED = 256 * 1024 * 1024  # 256 MB configurable limit
if uncompressed_size > MAX_UNCOMPRESSED:
    raise ValueError(
        f"uncompressed_size {uncompressed_size} exceeds safety limit "
        f"{MAX_UNCOMPRESSED}. Possible decompression bomb."
    )
```

---

## Environment

| Package       | Version   |
|---------------|-----------|
| fastparquet   | 2026.5.0  |
| pyarrow       | 25.0.0    |
| Python        | 3.12      |

---

## Files

| File                                  | Description              |
|---------------------------------------|--------------------------|
| `poc_fastparquet_parquet_bomb.py`     | PoC script (self-contained) |
| `bomb_500mb.parquet`                  | Crafted payload (492 bytes) |

---

*Discovered via empirical security research using UBDAF (Universal Bug
Discovery & Analysis Framework) — Q2 CRITICAL pattern: arithmetic on
externally-controlled value without bound check.*