---
title: GGUF Splitter
emoji: ✂️
colorFrom: red
colorTo: gray
sdk: gradio
sdk_version: 6.5.1
hf_oauth: true
hf_oauth_scopes:
- read-repos
- write-repos
- manage-repos
pinned: true
short_description: Effortless GGUF Model Sharding for easier distribution
---

# GGUF Splitter

Effortless GGUF Model Sharding for easier distribution and deployment.

## Features

- **Automatic Splitting**: Calculates optimal split sizes using llama.cpp's dry-run mode
- **Hugging Face Integration**: Direct upload to new repositories with auto-generated names
- **Local Mode**: Save split files locally without uploading (for local development)
- **Smart Naming**: Automatically generates descriptive repository names based on model and quantization

## Running the Application

### Option 1: Hugging Face Spaces (Recommended)

The app is deployed as a Hugging Face Space. Just visit the Space URL and sign in with your HF account.

### Option 2: Running Locally with Docker

#### Prerequisites

- Docker and Docker Compose installed
- Hugging Face account with access token

#### Setup

1. **Login to Hugging Face CLI**:

   ```bash
   huggingface-cli login
   ```

2. **Run the application**:

   ```bash
   HF_TOKEN=$(cat ~/.cache/huggingface/token) docker compose up
   ```

3. **Access the app**: Open http://localhost:7860 in your browser

#### Local Mode (No Upload)

To run locally without uploading to Hugging Face:

```bash
RUN_LOCALLY=1 docker compose -f docker-compose.local.yml up
```

The app will save split files to the `./output` directory instead of uploading.

## How It Works

1. **Select Model**: Enter a Hugging Face model repository ID
2. **Choose GGUF File**: Select from available GGUF files in the repository
3. **Configure Output**: Set repository name and visibility (cloud mode only)
4. **Split & Upload/Save**: The app downloads, splits, and either uploads to HF or saves locally

The splitting uses `llama-gguf-split` from llama.cpp to ensure optimal tensor distribution and maintain model integrity.

## Example Output

A 100MB GGUF file typically splits into 4 parts:

- Part 1: ~31.7 MB
- Part 2: ~30.2 MB
- Part 3: ~30.1 MB
- Part 4: ~8.6 MB

Each part is a valid GGUF file that can be loaded independently.