Instructions to use mradermacher/LFM2-2.6B-Transcript-i1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mradermacher/LFM2-2.6B-Transcript-i1-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mradermacher/LFM2-2.6B-Transcript-i1-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use mradermacher/LFM2-2.6B-Transcript-i1-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use mradermacher/LFM2-2.6B-Transcript-i1-GGUF with Ollama:
```
ollama run hf.co/mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M
```
Unsloth Desktop

How to use mradermacher/LFM2-2.6B-Transcript-i1-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use mradermacher/LFM2-2.6B-Transcript-i1-GGUF with Docker Model Runner:
```
docker model run hf.co/mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M
```

Lemonade

How to use mradermacher/LFM2-2.6B-Transcript-i1-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.LFM2-2.6B-Transcript-i1-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use mradermacher/LFM2-2.6B-Transcript-i1-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use mradermacher/LFM2-2.6B-Transcript-i1-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "mradermacher/LFM2-2.6B-Transcript-i1-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

mradermacher commited on Jan 6

Commit

0abec76

verified ·

1 Parent(s): bf7ee43

uploaded from rich1

Browse files

Files changed (25) hide show

.gitattributes +24 -0
LFM2-2.6B-Transcript.i1-IQ1_M.gguf +3 -0
LFM2-2.6B-Transcript.i1-IQ1_S.gguf +3 -0
LFM2-2.6B-Transcript.i1-IQ2_M.gguf +3 -0
LFM2-2.6B-Transcript.i1-IQ2_S.gguf +3 -0
LFM2-2.6B-Transcript.i1-IQ2_XS.gguf +3 -0
LFM2-2.6B-Transcript.i1-IQ2_XXS.gguf +3 -0
LFM2-2.6B-Transcript.i1-IQ3_M.gguf +3 -0
LFM2-2.6B-Transcript.i1-IQ3_S.gguf +3 -0
LFM2-2.6B-Transcript.i1-IQ3_XS.gguf +3 -0
LFM2-2.6B-Transcript.i1-IQ3_XXS.gguf +3 -0
LFM2-2.6B-Transcript.i1-IQ4_NL.gguf +3 -0
LFM2-2.6B-Transcript.i1-IQ4_XS.gguf +3 -0
LFM2-2.6B-Transcript.i1-Q2_K.gguf +3 -0
LFM2-2.6B-Transcript.i1-Q2_K_S.gguf +3 -0
LFM2-2.6B-Transcript.i1-Q3_K_L.gguf +3 -0
LFM2-2.6B-Transcript.i1-Q3_K_M.gguf +3 -0
LFM2-2.6B-Transcript.i1-Q3_K_S.gguf +3 -0
LFM2-2.6B-Transcript.i1-Q4_0.gguf +3 -0
LFM2-2.6B-Transcript.i1-Q4_1.gguf +3 -0
LFM2-2.6B-Transcript.i1-Q4_K_M.gguf +3 -0
LFM2-2.6B-Transcript.i1-Q4_K_S.gguf +3 -0
LFM2-2.6B-Transcript.i1-Q5_K_M.gguf +3 -0
LFM2-2.6B-Transcript.i1-Q5_K_S.gguf +3 -0
LFM2-2.6B-Transcript.i1-Q6_K.gguf +3 -0

.gitattributes CHANGED Viewed

@@ -34,3 +34,27 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 LFM2-2.6B-Transcript.imatrix.gguf filter=lfs diff=lfs merge=lfs -text

 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 LFM2-2.6B-Transcript.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+LFM2-2.6B-Transcript.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text

LFM2-2.6B-Transcript.i1-IQ1_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a56549f4ec01fff4b8f652f81309ea2bdc7747d2f67e25f034fad16ad4513fd0
+size 637777344

LFM2-2.6B-Transcript.i1-IQ1_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5ef372974e01c82b999c8492ffbe6af840c3039605658e89776166cdcc048e03
+size 583243200

LFM2-2.6B-Transcript.i1-IQ2_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:87cef5a683ca7b235432d964bc20fcc51645af71b94a7d91d85b1ded440d99b6
+size 887420352

LFM2-2.6B-Transcript.i1-IQ2_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:584b0a3a2f1a359c64953d8f148da4c722157caa7bf0e28b7551a84d28eab59c
+size 814708160

LFM2-2.6B-Transcript.i1-IQ2_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8f49452f114ebcd9b1534d6c7133ad2b5562d30f6e7907fe09b96d27519877e1
+size 802428352

LFM2-2.6B-Transcript.i1-IQ2_XXS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ff32061a5d31d1370a448cfe58c295e0f588ec8611b2a1b89bf21a9fc75679f4
+size 728667584

LFM2-2.6B-Transcript.i1-IQ3_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:eec378f87dc4c817af5da13daec4bb42e1210101b320205380277a2b5c1657dd
+size 1174083008

LFM2-2.6B-Transcript.i1-IQ3_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1316163802ee2111ddf74d140eebb2255a92ed1719425cf7b9e33b502ebc3f1c
+size 1159738816

LFM2-2.6B-Transcript.i1-IQ3_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f5977b73fea32d7e2b472436c6d19165322442747f2ff4559dcab3d38a5445f6
+size 1110291904

LFM2-2.6B-Transcript.i1-IQ3_XXS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fc38dc51c75319655721b74dde5f27688a61ee68b6aa88cfcb2c227d6b5c40c1
+size 1026733504

LFM2-2.6B-Transcript.i1-IQ4_NL.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6feb82ac469bc5a62e12b8601da35783fe816323a5f343bb24067f43c94e50f6
+size 1483109824

LFM2-2.6B-Transcript.i1-IQ4_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:76e6d15d8ce95c6615ad1865f0e16b9b6ae1b0b5cbab48cf4f0ed1485703bb7d
+size 1407022528

LFM2-2.6B-Transcript.i1-Q2_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1b49f565ed47df050a69b50a7e9bb4e34536b9f35fa024aad092162864cd6dee
+size 983807424

LFM2-2.6B-Transcript.i1-Q2_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6dc70e55f9d4ce567dbdee9c8e43c4e7a0ea0b772029fead523db339f5d6d717
+size 927937984

LFM2-2.6B-Transcript.i1-Q3_K_L.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ffbb2ffc8c35e08eda3aa8062f28a2d116719552219bbb8447a8ade32bf5537f
+size 1340863936

LFM2-2.6B-Transcript.i1-Q3_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c2705c9d408e4b98456061bc7b5a0a88c54b7b37dcf0efa615394459b116ac8b
+size 1256060352

LFM2-2.6B-Transcript.i1-Q3_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bbc0a6a16ddf3c23195bc3204b1f478aba8470cfa395e489c76bb169f3800173
+size 1159738816

LFM2-2.6B-Transcript.i1-Q4_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d5f8ae637b15efc836a2204889fa1d1327a0fd2ac719706da5381f3c10ac98b5
+size 1487238592

LFM2-2.6B-Transcript.i1-Q4_1.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1ece352b4fa6f30017a0d65e4b81537ae1ed9bf154a874cdbee5703c8f28c60c
+size 1635284416

LFM2-2.6B-Transcript.i1-Q4_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:199b772a119405da040b701db0f26dfb424269ed7f50851a6f4c28b6a637ab0b
+size 1563669952

LFM2-2.6B-Transcript.i1-Q4_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4f300be155abf45d0359a9911d2055c1d327def454da58bae5a719a908afb3dc
+size 1491891648

LFM2-2.6B-Transcript.i1-Q5_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9db3b3988f8c2bb3f323f5ffe6432383f5ceb3addfdaed27e925198b2bf0a5d4
+size 1828959680

LFM2-2.6B-Transcript.i1-Q5_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2da698b968e07bbbaf226dbf400eacb807205a617e91aa2f00f0157d606b7d44
+size 1787459008

LFM2-2.6B-Transcript.i1-Q6_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9549ee2fd63a4a78ec764f8659aa26ad0e65dd00f835146aeae362531a06838f
+size 2110830016