Instructions to use eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-0.5B-Instruct")
model = PeftModel.from_pretrained(base_model, "eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M

Use Docker

docker model run hf.co/eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M

LM Studio
Jan

vLLM

How to use eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M

Ollama
How to use eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora with Ollama:
```
ollama run hf.co/eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M
```
Unsloth Desktop

How to use eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora with Docker Model Runner:
```
docker model run hf.co/eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M
```

Lemonade

How to use eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M

Run and chat with the model

lemonade run user.edgeai-docs-qwen2.5-coder-0.5b-lora-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

eoinedge commited on Jun 24

Commit

a735a51

verified ·

1 Parent(s): 18fc661

Remove generated documentation block

Browse files

Files changed (1) hide show

README.md +187 -210

README.md CHANGED Viewed

@@ -1,210 +1,187 @@
----
-license: apache-2.0
-base_model: Qwen/Qwen1.5-0.5B
-library_name: peft
-language:
-  - en
-tags:
-  - lora
-  - peft
-  - qwen
-  - edge-ai
-  - edge-impulse
-  - documentation
-  - code-generation
-  - conversational
-pipeline_tag: text-generation
-widget:
-  - text: "How can I deploy an Edge Impulse model to an Arduino device?"
-    example_title: "Edge AI deployment"
-space: eoinedge/edgeai-docs-embedding-qwen1.5-0.5b-instruct-space
----
-# edgeai-docs-embedding-qwen1.5-0.5b-instruct
-A lightweight LoRA adapter fine-tuned on **1,794 Edge Impulse / Edge AI MDX documentation files** from the [Edge Impulse documentation](https://docs.edgeimpulse.com), built on top of [`Qwen/Qwen1.5-0.5B`](https://huggingface.co/Qwen/Qwen1.5-0.5B).
-Optimized for:
-- answering developer questions about Edge Impulse Studio, SDKs, APIs, and tooling
-- summarizing technical documentation and tutorials
-- generating code snippets for edge ML workflows
-- lightweight local/edge deployment with PEFT adapters
-> **Larger variants in training:** [1.5B](https://huggingface.co/eoinedge/edgeai-qwen2.5coder-1.5b-lora) · [7B](https://huggingface.co/eoinedge/edgeai-qwen2.5coder-7b-lora) (Qwen2.5-Coder base)
----
-## Model Summary
-`edgeai-docs-embedding-qwen1.5-0.5b-instruct` is a PEFT LoRA adapter trained for documentation-focused text generation and conversational support over Edge Impulse / Edge AI knowledge.
-### Use cases
-- Documentation Q&A for Edge Impulse developers
-- Technical explanation of Studio workflows, SDK usage, and hardware deployment
-- Generating sample code for API, CLI, and Python SDK integrations
-- Retrieval-augmented generation (RAG) over Edge AI docs
----
-## Model Details
-| Property | Value |
-|---|---|
-| Base model | `Qwen/Qwen1.5-0.5B` |
-| Adapter type | LoRA (PEFT) |
-| LoRA rank (`r`) | 8 |
-| LoRA alpha | 32 |
-| Target modules | `q_proj`, `v_proj` |
-| Task type | CAUSAL_LM |
-| Trainable parameters | ~786K (0.17% of base) |
-| Training epochs | 3 |
-| Batch size | 4 (× grad accum 2 = effective 8) |
-| Learning rate | 3e-4 |
-| Max sequence length | 512 tokens |
-| Training hardware | Apple M1 Pro (MPS, fp16) |
-| Precision | float16 |
----
-## Training Data
-| Stat | Value |
-|---|---|
-| Source | [Edge Impulse documentation](https://docs.edgeimpulse.com) |
-| File format | MDX (Markdown + JSX components) |
-| Total files | 1,794 `.mdx` files |
-| Preprocessing | Stripped frontmatter, imports, JSX tags; unwrapped code fences; flattened links |
-| Chunk size | 512 tokens |
-Topics covered: Studio projects, datasets, data ingestion, DSP and transformation blocks, learning and processing blocks, model deployment, Python SDK, REST API, CLI tools, and edge inference.
----
-## Evaluation
-### QA evaluation
-- Dataset: 5 fixed developer-style prompts
-- Base avg keyword count: **8.2**
-- Adapter avg keyword count: **6.8**
-- Code snippet presence: **5/5** for both base and adapter
-### Perplexity on Edge AI samples
-- Test corpus: 30 sample Edge AI documentation files
-- Base mean perplexity: **11.53**
-- Adapter mean perplexity: **12.02**
-- Adapter wins: **4 / 30 documents**
-> These metrics are from small validation samples and should be interpreted as a lightweight benchmark rather than a full production evaluation.
----
-## Tutorials
-- [Offline SLMs for Edge AI Development — Part 1: Qwen LoRA Adapter Fine-Tuned on Edge Impulse Docs](https://docs.edgeimpulse.com/projects/expert-network/integrating-slms-on-linux)
-- [Offline SLMs for Edge AI Development — Part 2: RAG as an Enhancement for Fine-Tuned Models with FAISS](https://docs.edgeimpulse.com/projects/expert-network/rag-docs-assistant-faiss-qwen)
-- [Offline SLMs for Edge AI Development — Part 3: Agentic Coding with an Arduino Fine-Tuned Adapter via llama.cpp and OpenCode](https://docs.edgeimpulse.com/projects/expert-network/opencode-offline-coding-assistant)
----
-## Usage
-### Load with PEFT
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-from peft import PeftModel
-import torch
-BASE_MODEL = "Qwen/Qwen1.5-0.5B"
-ADAPTER = "eoinedge/edgeai-docs-embedding-qwen1.5-0.5b-instruct"
-device = "cuda" if torch.cuda.is_available() else ("mps" if torch.backends.mps.is_available() else "cpu")
-tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
-base_model = AutoModelForCausalLM.from_pretrained(
-    BASE_MODEL,
-    torch_dtype=torch.float16 if device != "cpu" else torch.float32,
-    device_map=device,
-)
-model = PeftModel.from_pretrained(base_model, ADAPTER)
-model.eval()
-```
-### Text generation pipeline
-```python
-from transformers import pipeline
-pipe = pipeline("text-generation", model="eoinedge/edgeai-docs-embedding-qwen1.5-0.5b-instruct")
-print(pipe([{"role": "user", "content": "How do I use the Edge Impulse Python SDK to upload data?"}]))
-```
----
-## Example prompts
-| Task | Prompt |
-|---|---|
-| Concept explanation | `What is a DSP block in Edge Impulse?` |
-| API usage | `How do I use the Edge Impulse Python SDK to upload data?` |
-| Deployment | `How do I deploy a model to an Arduino Nano 33 BLE Sense?` |
-| Code generation | `Write Python code to collect IMU data and upload it to Edge Impulse.` |
-| Troubleshooting | `Why is my Edge Impulse model showing high latency?` |
----
-## Limitations
-- Based on a 0.5B base model — may struggle with long multi-step reasoning
-- Training data covers Edge Impulse docs as of mid-2026; newer features may be missing
-- May hallucinate or fabricate undocumented APIs or block behavior
-- Not validated for safety-critical or production use
-- Validate generated code before deploying on hardware
----
-## Related models
-| Model | Base | Status |
-|---|---|---|
-| This model | Qwen/Qwen1.5-0.5B | ✅ Available |
-| [eoinedge/edgeai-qwen2.5coder-1.5b-lora](https://huggingface.co/eoinedge/edgeai-qwen2.5coder-1.5b-lora) | Qwen2.5-Coder-1.5B-Instruct | 🔄 Training |
-| [eoinedge/edgeai-qwen2.5coder-7b-lora](https://huggingface.co/eoinedge/edgeai-qwen2.5coder-7b-lora) | Qwen2.5-Coder-7B-Instruct | 🔄 Training |
-| [eoinedge/arduino-qwen0.5-lora](https://huggingface.co/eoinedge/arduino-qwen0.5-lora) | Qwen/Qwen1.5-0.5B | ✅ Available |
----
-## Citation
-```bibtex
-@misc{edgeai-docs-embedding-qwen1.5-0.5b-instruct,
-  author       = {Jordan, Eoin},
-  title        = {edgeai-docs-embedding-qwen1.5-0.5b-instruct},
-  year         = {2026},
-  publisher    = {Hugging Face},
-  howpublished = {\url{https://huggingface.co/eoinedge/edgeai-docs-embedding-qwen1.5-0.5b-instruct}}
-}
-```
-<!-- portfolio-card-standard:v1 -->
-## Provenance and maintenance
-This standardized block is maintained from a versioned local snapshot. It separates declared evidence from gaps that still require source documentation.
-| Field | Recorded value |
-| --- | --- |
-| Hugging Face asset | [eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora](https://huggingface.co/eoinedge/edgeai-docs-qwen2.5-coder-0.5b-lora) (model) |
-| Snapshot revision | `5d5d65a115937999aed51e771913721ed1aaab03` |
-| Snapshot location | `backups/huggingface/20260624T153725Z/models/eoinedge--edgeai-docs-qwen2.5-coder-0.5b-lora/` |
-| Last remote modification captured | `2026-05-16 17:21:57+00:00` |
-| Declared license | apache-2.0 |
-| Prior-card references | <https://docs.edgeimpulse.com>, <https://huggingface.co/Qwen/Qwen1.5-0.5B>, <https://huggingface.co/eoinedge/edgeai-qwen2.5coder-1.5b-lora>, <https://huggingface.co/eoinedge/edgeai-qwen2.5coder-7b-lora>, <https://docs.edgeimpulse.com/projects/expert-network/integrating-slms-on-linux>, <https://docs.edgeimpulse.com/projects/expert-network/rag-docs-assistant-faiss-qwen>, <https://docs.edgeimpulse.com/projects/expert-network/opencode-offline-coding-assistant>, <https://huggingface.co/eoinedge/arduino-qwen0.5-lora>, <https://huggingface.co/eoinedge/edgeai-docs-embedding-qwen1.5-0.5b-instruct}}> |
-### Model lineage
-| Field | Recorded value |
-| --- | --- |
-| Base model | Qwen/Qwen2.5-Coder-0.5B-Instruct |
-| Training data | Not declared in a machine-readable source — add exact dataset revision(s), split(s), and license(s). |
-| Training method and code | Not declared in a machine-readable source — add recipe, hyperparameters, hardware, and source commit. |
-| Evaluation | Not declared in a machine-readable source — add metrics, evaluation data, and limitations. |

+---
+license: apache-2.0
+base_model: Qwen/Qwen1.5-0.5B
+library_name: peft
+language:
+  - en
+tags:
+  - lora
+  - peft
+  - qwen
+  - edge-ai
+  - edge-impulse
+  - documentation
+  - code-generation
+  - conversational
+pipeline_tag: text-generation
+widget:
+  - text: "How can I deploy an Edge Impulse model to an Arduino device?"
+    example_title: "Edge AI deployment"
+space: eoinedge/edgeai-docs-embedding-qwen1.5-0.5b-instruct-space
+---
+# edgeai-docs-embedding-qwen1.5-0.5b-instruct
+A lightweight LoRA adapter fine-tuned on **1,794 Edge Impulse / Edge AI MDX documentation files** from the [Edge Impulse documentation](https://docs.edgeimpulse.com), built on top of [`Qwen/Qwen1.5-0.5B`](https://huggingface.co/Qwen/Qwen1.5-0.5B).
+Optimized for:
+- answering developer questions about Edge Impulse Studio, SDKs, APIs, and tooling
+- summarizing technical documentation and tutorials
+- generating code snippets for edge ML workflows
+- lightweight local/edge deployment with PEFT adapters
+> **Larger variants in training:** [1.5B](https://huggingface.co/eoinedge/edgeai-qwen2.5coder-1.5b-lora) · [7B](https://huggingface.co/eoinedge/edgeai-qwen2.5coder-7b-lora) (Qwen2.5-Coder base)
+---
+## Model Summary
+`edgeai-docs-embedding-qwen1.5-0.5b-instruct` is a PEFT LoRA adapter trained for documentation-focused text generation and conversational support over Edge Impulse / Edge AI knowledge.
+### Use cases
+- Documentation Q&A for Edge Impulse developers
+- Technical explanation of Studio workflows, SDK usage, and hardware deployment
+- Generating sample code for API, CLI, and Python SDK integrations
+- Retrieval-augmented generation (RAG) over Edge AI docs
+---
+## Model Details
+| Property | Value |
+|---|---|
+| Base model | `Qwen/Qwen1.5-0.5B` |
+| Adapter type | LoRA (PEFT) |
+| LoRA rank (`r`) | 8 |
+| LoRA alpha | 32 |
+| Target modules | `q_proj`, `v_proj` |
+| Task type | CAUSAL_LM |
+| Trainable parameters | ~786K (0.17% of base) |
+| Training epochs | 3 |
+| Batch size | 4 (× grad accum 2 = effective 8) |
+| Learning rate | 3e-4 |
+| Max sequence length | 512 tokens |
+| Training hardware | Apple M1 Pro (MPS, fp16) |
+| Precision | float16 |
+---
+## Training Data
+| Stat | Value |
+|---|---|
+| Source | [Edge Impulse documentation](https://docs.edgeimpulse.com) |
+| File format | MDX (Markdown + JSX components) |
+| Total files | 1,794 `.mdx` files |
+| Preprocessing | Stripped frontmatter, imports, JSX tags; unwrapped code fences; flattened links |
+| Chunk size | 512 tokens |
+Topics covered: Studio projects, datasets, data ingestion, DSP and transformation blocks, learning and processing blocks, model deployment, Python SDK, REST API, CLI tools, and edge inference.
+---
+## Evaluation
+### QA evaluation
+- Dataset: 5 fixed developer-style prompts
+- Base avg keyword count: **8.2**
+- Adapter avg keyword count: **6.8**
+- Code snippet presence: **5/5** for both base and adapter
+### Perplexity on Edge AI samples
+- Test corpus: 30 sample Edge AI documentation files
+- Base mean perplexity: **11.53**
+- Adapter mean perplexity: **12.02**
+- Adapter wins: **4 / 30 documents**
+> These metrics are from small validation samples and should be interpreted as a lightweight benchmark rather than a full production evaluation.
+---
+## Tutorials
+- [Offline SLMs for Edge AI Development — Part 1: Qwen LoRA Adapter Fine-Tuned on Edge Impulse Docs](https://docs.edgeimpulse.com/projects/expert-network/integrating-slms-on-linux)
+- [Offline SLMs for Edge AI Development — Part 2: RAG as an Enhancement for Fine-Tuned Models with FAISS](https://docs.edgeimpulse.com/projects/expert-network/rag-docs-assistant-faiss-qwen)
+- [Offline SLMs for Edge AI Development — Part 3: Agentic Coding with an Arduino Fine-Tuned Adapter via llama.cpp and OpenCode](https://docs.edgeimpulse.com/projects/expert-network/opencode-offline-coding-assistant)
+---
+## Usage
+### Load with PEFT
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel
+import torch
+BASE_MODEL = "Qwen/Qwen1.5-0.5B"
+ADAPTER = "eoinedge/edgeai-docs-embedding-qwen1.5-0.5b-instruct"
+device = "cuda" if torch.cuda.is_available() else ("mps" if torch.backends.mps.is_available() else "cpu")
+tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
+base_model = AutoModelForCausalLM.from_pretrained(
+    BASE_MODEL,
+    torch_dtype=torch.float16 if device != "cpu" else torch.float32,
+    device_map=device,
+)
+model = PeftModel.from_pretrained(base_model, ADAPTER)
+model.eval()
+```
+### Text generation pipeline
+```python
+from transformers import pipeline
+pipe = pipeline("text-generation", model="eoinedge/edgeai-docs-embedding-qwen1.5-0.5b-instruct")
+print(pipe([{"role": "user", "content": "How do I use the Edge Impulse Python SDK to upload data?"}]))
+```
+---
+## Example prompts
+| Task | Prompt |
+|---|---|
+| Concept explanation | `What is a DSP block in Edge Impulse?` |
+| API usage | `How do I use the Edge Impulse Python SDK to upload data?` |
+| Deployment | `How do I deploy a model to an Arduino Nano 33 BLE Sense?` |
+| Code generation | `Write Python code to collect IMU data and upload it to Edge Impulse.` |
+| Troubleshooting | `Why is my Edge Impulse model showing high latency?` |
+---
+## Limitations
+- Based on a 0.5B base model — may struggle with long multi-step reasoning
+- Training data covers Edge Impulse docs as of mid-2026; newer features may be missing
+- May hallucinate or fabricate undocumented APIs or block behavior
+- Not validated for safety-critical or production use
+- Validate generated code before deploying on hardware
+---
+## Related models
+| Model | Base | Status |
+|---|---|---|
+| This model | Qwen/Qwen1.5-0.5B | ✅ Available |
+| [eoinedge/edgeai-qwen2.5coder-1.5b-lora](https://huggingface.co/eoinedge/edgeai-qwen2.5coder-1.5b-lora) | Qwen2.5-Coder-1.5B-Instruct | 🔄 Training |
+| [eoinedge/edgeai-qwen2.5coder-7b-lora](https://huggingface.co/eoinedge/edgeai-qwen2.5coder-7b-lora) | Qwen2.5-Coder-7B-Instruct | 🔄 Training |
+| [eoinedge/arduino-qwen0.5-lora](https://huggingface.co/eoinedge/arduino-qwen0.5-lora) | Qwen/Qwen1.5-0.5B | ✅ Available |
+---
+## Citation
+```bibtex
+@misc{edgeai-docs-embedding-qwen1.5-0.5b-instruct,
+  author       = {Jordan, Eoin},
+  title        = {edgeai-docs-embedding-qwen1.5-0.5b-instruct},
+  year         = {2026},
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/eoinedge/edgeai-docs-embedding-qwen1.5-0.5b-instruct}}
+}
+```