Hugging Face's logo

MBM7
/
fastparquet-parquet-decompression-bomb-poc

Model card Files
xet
 Community
fastparquet-parquet-decompression-bomb-poc
File size: 3,681 Bytes
c63619c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
---
tags:
  - security
  - vulnerability
  - poc
  - parquet
  - fastparquet
  - decompression-bomb
  - cwe-789
license: mit
---

# fastparquet — Parquet PageHeader Decompression Bomb (PoC)

**Repo:** `MBM7/fastparquet-parquet-decompression-bomb-poc`  
**Status:** Responsible disclosure — submitted to Huntr  
**Severity:** High / CWE-789 (Uncontrolled Memory Allocation)

---

## Summary

A crafted **492-byte** `.parquet` file causes `fastparquet` to allocate
**gigabytes of memory** before failing — enabling OOM-kill on any service
that loads user-supplied Parquet files.

| File size | Fake `uncompressed_page_size` | Peak allocation | Amplification |
|-----------|-------------------------------|-----------------|---------------|
| 492 bytes | 500,000,000                   | **500 MB**      | 1 : 1,016,260 |
| 492 bytes | 2,000,000,000                 | **2,000 MB**    | 1 : 4,065,040 |

---

## Root Cause

`fastparquet/compression.py` — `decompress_data()`:

```python
if algorithm.upper() in decom_into:
    x = np.empty(uncompressed_size, dtype='uint8')   # ← pre-allocates upfront
    decom_into[algorithm.upper()](
        np.frombuffer(data, dtype=np.uint8), x
    )
    return x
```

`uncompressed_size` is passed directly from `ph.uncompressed_page_size`
(Thrift compact i32, PageHeader field 2) **with no bound check** at any
point in the call chain:

```
fastparquet.ParquetFile.to_pandas()
  → read_col_group()          [core.py ~291]
    → _read_page()            [core.py ~20]
      → decompress_data(
            data,
            page_header.uncompressed_page_size,   ← from Thrift, unchecked
            codec
        )
          → np.empty(uncompressed_size)           ← GB allocation here
```

`max_header_size` or equivalent limit: **does not exist** in fastparquet.

---

## Attack

Parquet format: `PAR1 | PageHeader (Thrift compact) | compressed_data | ... | PAR1`

PageHeader field 2 (`uncompressed_page_size`, type i32) is a
zigzag-encoded varint at byte offset 7 in a minimal single-column file.

Replacing the 1-byte varint (value `4`) with a 5-byte varint
(value `500_000_000`) produces a valid-looking file that triggers 500 MB
of allocation on any codec in `decom_into` (GZIP, SNAPPY, ZSTD, BROTLI).

---

## Reproduce

```bash
pip install fastparquet pyarrow
python poc_fastparquet_parquet_bomb.py
```

Expected output:
```
Crafted file size           : 492 bytes
Expected allocation         : 500,000,000 bytes
Amplification               : 1:1,016,260
Result                      : DecompressionError: unexpected end of file
Peak memory                 : 500 MB  ← allocation happened
```

---

## Suggested Fix

`fastparquet/compression.py`, before `np.empty()`:

```python
MAX_UNCOMPRESSED = 256 * 1024 * 1024  # 256 MB configurable limit
if uncompressed_size > MAX_UNCOMPRESSED:
    raise ValueError(
        f"uncompressed_size {uncompressed_size} exceeds safety limit "
        f"{MAX_UNCOMPRESSED}. Possible decompression bomb."
    )
```

---

## Environment

| Package       | Version   |
|---------------|-----------|
| fastparquet   | 2026.5.0  |
| pyarrow       | 25.0.0    |
| Python        | 3.12      |

---

## Files

| File                                  | Description              |
|---------------------------------------|--------------------------|
| `poc_fastparquet_parquet_bomb.py`     | PoC script (self-contained) |
| `bomb_500mb.parquet`                  | Crafted payload (492 bytes) |

---

*Discovered via empirical security research using UBDAF (Universal Bug
Discovery & Analysis Framework) — Q2 CRITICAL pattern: arithmetic on
externally-controlled value without bound check.*