---
language:
  - en
license: llama2
tags:
  - mlx
  - text-to-sql
  - duckdb
  - sql
  - text-generation
  - quantized
  - 4-bit
base_model: motherduckdb/DuckDB-NSQL-7B-v0.1
library_name: mlx
pipeline_tag: text-generation
---

# duckdb-nsql-7b-mlx-4bit

This repository contains an **MLX-optimized 4-bit quantized** variant of
[motherduckdb/DuckDB-NSQL-7B-v0.1](https://huggingface.co/motherduckdb/DuckDB-NSQL-7B-v0.1),
intended for maximum efficiency (lowest memory, fastest decoding) on **Apple Silicon (M1/M2/M3/M4)**.

## Model description

DuckDB-NSQL-7B is a 7B parameter language model fine-tuned to translate natural language questions into
**DuckDB SQL**. The **4-bit MLX** conversion targets minimal memory usage and high throughput, with a larger
quality trade-off compared to FP16/8-bit—especially for long schemas and complex queries.

## Conversion details

- **Base model**: motherduckdb/DuckDB-NSQL-7B-v0.1 (fine-tuned from Llama 2 7B)
- **Format**: MLX
- **Precision**: 4-bit quantized
- **Typical memory footprint**: ~4–5 GB (varies by MLX quantization / runtime)
- **Recommended for**: laptops / demos / constrained RAM; when speed matters more than perfect SQL fidelity

## Installation

```bash
pip install mlx-lm
```

## Usage

### Python

```python
from mlx_lm import load, generate

model, tokenizer = load("Nuxera/duckdb-nsql-7b-mlx-4bit")

schema = """
CREATE TABLE hospitals (
  hospital_id BIGINT,
  hospital_name VARCHAR,
  region VARCHAR,
  bed_capacity INTEGER
);

CREATE TABLE encounters (
  encounter_id BIGINT,
  hospital_id BIGINT,
  encounter_datetime TIMESTAMP,
  encounter_type VARCHAR
);
"""

question = "For each hospital region, how many encounters happened this month?"

prompt = f"""You are an assistant that writes valid DuckDB SQL queries.

### Schema:
{schema}

### Question:
{question}

### Response (DuckDB SQL only):"""

out = generate(model, tokenizer, prompt=prompt, max_tokens=256, temp=0.0)
print(out)
```

### Run as a local server

```bash
mlx_lm.server --model Nuxera/duckdb-nsql-7b-mlx-4bit --port 8080
```

## Prompt format

This model works best when you provide:

1) Clear schema (tables + columns)  
2) One question  
3) Explicit instruction to output **SQL only**

Example:

```
You are an assistant that writes valid DuckDB SQL queries.

### Schema:
CREATE TABLE ...

### Question:
...

### Response (DuckDB SQL only):
```

## Quality notes (4-bit)

4-bit can degrade more than 8-bit/FP16 when prompts include:

- very long schemas with many similarly named columns
- multi-join / nested subqueries
- ambiguous questions requiring stronger reasoning
- strict formatting constraints

If you need maximum reliability, prefer FP16 or 8-bit.

## License

This model inherits the **Llama 2** license from the base model.

## Citation

```bibtex
@misc{nuxera_duckdb_nsql_mlx_4bit,
  title={DuckDB-NSQL-7B MLX 4-bit Quantized Conversion},
  author={Nuxera AI},
  year={2025},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/Nuxera/duckdb-nsql-7b-mlx-4bit}}
}
```

Base model:

```bibtex
@misc{duckdb_nsql,
  title={DuckDB-NSQL-7B: Natural Language to SQL for DuckDB},
  author={MotherDuck},
  year={2024},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/motherduckdb/DuckDB-NSQL-7B-v0.1}}
}
```

## Acknowledgments

- Original model by [MotherDuck](https://huggingface.co/motherduckdb)
- MLX framework by [Apple ML Research](https://github.com/ml-explore/mlx)
- MLX-LM: [mlx-lm](https://github.com/ml-explore/mlx-examples/tree/main/llms)
- [Nuxera AI](https://nuxera.ai)