---
base_model:
- zai-org/GLM-4.5-Air
base_model_relation: quantized
quantized_by: ddh0
license: mit
---

# GLM-4.5-Air-GGUF

This repository contains several custom GGUF quantizations of [GLM-4.5-Air](https://huggingface.co/zai-org/GLM-4.5-Air), to be used with [llama.cpp](https://github.com/ggml-org/llama.cpp).

The naming scheme for these custom quantizations is as follows:

> **`ModelName-DefaultType-FFN-UpType-GateType-DownType.gguf`**

Where `DefaultType` refers to the default tensor type, and `UpType`, `GateType`, and `DownType` refer to the tensor types used for the `ffn_up_exps`,  `ffn_gate_exps`, and `ffn_down_exps` tensors respectively.

## Original quantizations

These quantizations use Q8_0 for all tensors by default - only the dense FFN block and conditional experts are downgraded. The shared expert is always kept in Q8_0. They were quantized using [bartowski's imatrix](https://huggingface.co/bartowski/zai-org_GLM-4.5-Air-GGUF/blob/main/zai-org_GLM-4.5-Air-imatrix.gguf).

| Filename                                     | Size (GB) | Size (GiB) | Average BPW | Direct link                                                                                                        |
| -------------------------------------------- | --------- | ---------- | ----------- | ------------------------------------------------------------------------------------------------------------------ |
| GLM-4.5-Air-Q8_0-FFN-IQ3_S-IQ3_S-Q5_0.gguf   | 61.66     | 57.43      | 4.47        | [Download](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/resolve/main/GLM-4.5-Air-Q8_0-FFN-IQ3_S-IQ3_S-Q5_0.gguf)   |
| GLM-4.5-Air-Q8_0-FFN-IQ4_XS-IQ4_XS-Q5_0.gguf | 68.56     | 63.86      | 4.97        | [Download](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/resolve/main/GLM-4.5-Air-Q8_0-FFN-IQ4_XS-IQ4_XS-Q5_0.gguf) |
| GLM-4.5-Air-Q8_0-FFN-Q4_K-Q4_K-Q5_1.gguf     | 72.82     | 67.82      | 5.27        | [Download](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/resolve/main/GLM-4.5-Air-Q8_0-FFN-Q4_K-Q4_K-Q5_1.gguf)     |
| GLM-4.5-Air-Q8_0-FFN-Q4_K-Q4_K-Q8_0.gguf     | 83.44     | 77.71      | 6.04        | [Download](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/resolve/main/GLM-4.5-Air-Q8_0-FFN-Q4_K-Q4_K-Q8_0.gguf)     |
| GLM-4.5-Air-Q8_0-FFN-Q5_K-Q5_K-Q8_0.gguf     | 91.94     | 85.63      | 6.66        | [Download](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/resolve/main/GLM-4.5-Air-Q8_0-FFN-Q5_K-Q5_K-Q8_0.gguf)     |
| GLM-4.5-Air-Q8_0-FFN-Q6_K-Q6_K-Q8_0.gguf     | 100.97    | 94.04      | 7.31        | [Download](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/resolve/main/GLM-4.5-Air-Q8_0-FFN-Q6_K-Q6_K-Q8_0.gguf)     |
| GLM-4.5-Air-Q8_0.gguf                        | 117.45    | 109.39     | 8.50        | [Download](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/resolve/main/GLM-4.5-Air-Q8_0.gguf)                        |
| GLM-4.5-Air-bf16.gguf                        | 220.98    | 205.81     | 16.00       | [Download](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/resolve/main/GLM-4.5-Air-bf16.gguf)                        |

## v2 quantizations

These quantizations use Q8_0 for all tensors by default, including the dense FFN block. Only the conditional experts are downgraded. The shared expert is always kept in Q8_0. They were quantized using [my own imatrix](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/blob/main/GLM-4.5-Air-ddh0_v2-imatrix.gguf) (the calibration text corpus can be found [here](https://huggingface.co/ddh0/imatrices/blob/main/ddh0_imat_calibration_data_v2.txt)).

| Filename                                          | Size (GB) | Size (GiB) | Average BPW | Direct link                                                                                                             |
| ------------------------------------------------- | --------- | ---------- | ----------- | ----------------------------------------------------------------------------------------------------------------------- |
| GLM-4.5-Air-Q8_0-FFN-IQ4_XS-IQ3_S-IQ4_NL-v2.gguf  | 60.94     | 56.76      | 4.41        | [Download](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/resolve/main/GLM-4.5-Air-Q8_0-FFN-IQ4_XS-IQ3_S-IQ4_NL-v2.gguf)  |
| GLM-4.5-Air-Q8_0-FFN-IQ4_XS-IQ4_XS-IQ4_NL-v2.gguf | 64.39     | 59.97      | 4.66        | [Download](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/resolve/main/GLM-4.5-Air-Q8_0-FFN-IQ4_XS-IQ4_XS-IQ4_NL-v2.gguf) |
| GLM-4.5-Air-Q8_0-FFN-IQ4_XS-IQ4_XS-Q5_0-v2.gguf   | 68.63     | 63.92      | 4.97        | [Download](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/resolve/main/GLM-4.5-Air-Q8_0-FFN-IQ4_XS-IQ4_XS-Q5_0-v2.gguf)   |
| GLM-4.5-Air-Q8_0-FFN-IQ4_XS-IQ4_XS-Q8_0-v2.gguf   | 81.36     | 75.78      | 5.89        | [Download](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/resolve/main/GLM-4.5-Air-Q8_0-FFN-IQ4_XS-IQ4_XS-Q8_0-v2.gguf)   |
| GLM-4.5-Air-Q8_0-FFN-Q5_K-Q5_K-Q8_0-v2.gguf       | 91.97     | 85.66      | 6.66        | [Download](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/resolve/main/GLM-4.5-Air-Q8_0-FFN-Q5_K-Q5_K-Q8_0-v2.gguf)       |
| GLM-4.5-Air-Q8_0-FFN-Q6_K-Q6_K-Q8_0-v2.gguf       | 100.99    | 94.06      | 7.31        | [Download](https://huggingface.co/ddh0/GLM-4.5-Air-GGUF/resolve/main/GLM-4.5-Air-Q8_0-FFN-Q6_K-Q6_K-Q8_0-v2.gguf)       |