Instructions to use bytess/zrah-model-1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries
PEFT
How to use bytess/zrah-model-1.0 with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use bytess/zrah-model-1.0 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf bytess/zrah-model-1.0
# Run inference directly in the terminal:
llama cli -hf bytess/zrah-model-1.0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf bytess/zrah-model-1.0
# Run inference directly in the terminal:
llama cli -hf bytess/zrah-model-1.0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf bytess/zrah-model-1.0
# Run inference directly in the terminal:
./llama-cli -hf bytess/zrah-model-1.0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf bytess/zrah-model-1.0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf bytess/zrah-model-1.0

Use Docker

docker model run hf.co/bytess/zrah-model-1.0

LM Studio
Jan
Ollama
How to use bytess/zrah-model-1.0 with Ollama:
```
ollama run hf.co/bytess/zrah-model-1.0
```
Unsloth Desktop

How to use bytess/zrah-model-1.0 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf bytess/zrah-model-1.0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "bytess/zrah-model-1.0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use bytess/zrah-model-1.0 with Docker Model Runner:
```
docker model run hf.co/bytess/zrah-model-1.0
```

Lemonade

How to use bytess/zrah-model-1.0 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull bytess/zrah-model-1.0

Run and chat with the model

lemonade run user.zrah-model-1.0-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use bytess/zrah-model-1.0 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf bytess/zrah-model-1.0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default bytess/zrah-model-1.0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use bytess/zrah-model-1.0 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf bytess/zrah-model-1.0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "bytess/zrah-model-1.0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Click to expand full config

Axolotl version: 0.8.0.dev0

adapter: lora
base_model: mistralai/Mistral-7B-Instruct-v0.3
model_type: MistralForCausalLM
tokenizer_type: AutoTokenizer
bf16: true
dataset_processes: 32
datasets:
  - path: bytess/zrah-personal-ai
    type: alpaca
gradient_accumulation_steps: 4
gradient_checkpointing: false
learning_rate: 0.0002
lora_alpha: 32
lora_dropout: 0.05
lora_r: 16           # Or try 8 for smaller size later
lora_target_modules:
  - q_proj
  - v_proj
  - k_proj
  - o_proj
  - gate_proj
  - down_proj
  - up_proj
loraplus_lr_embedding: 1.0e-06
lr_scheduler: cosine
max_prompt_len: 512
micro_batch_size: 4  # Increase from 2 if GPU allows
num_epochs: 3
optimizer: adamw_torch
output_dir: ./outputs/zrah_model
pretrain_multipack_attn: true
sample_packing_bin_size: 200
sample_packing_group_size: 100000
save_only_model: true
save_safetensors: true
sequence_len: 2048
shuffle_merged_datasets: true
train_on_inputs: false
trl:
  use_vllm: false
val_set_size: 0.0
weight_decay: 0.0

Zrah Model 1.0

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.3, trained using Axolotl on a RunPod instance with H100 PCIe GPU. The training was conducted for personal/self-aligned instruction tuning using a custom dataset: bytess/zrah-personal-ai.

⚠️ This model is intended only for demonstration and trial purposes as part of a personal portfolio. Use is allowed, but modification, fine-tuning, or redistribution is prohibited due to the private nature of the data.

Model Description

This model uses LoRA adapters to fine-tune Mistral-7B on a dataset of personal prompts and completions designed to align the model to the author's style, tone, and preferences. The model is intended for self-use in chat, writing assistance, and experimentation with instruction-following behavior.

Intended Uses & Limitations

Intended Uses:

Personal assistant/chatbot
Writing/code helper tailored to author's style
Experimentation with LoRA fine-tuning and self-alignment

Limitations:

Trained only on a small personal dataset; generalization to broader tasks is limited
Not evaluated on standardized benchmarks
May reflect idiosyncratic styles or biases present in the training data

Training and Evaluation Data

The model was trained on the bytess/zrah-personal-ai dataset. This dataset is structured in Alpaca-style (instruction, input, output) and consists of hand-curated personal instructions and completions.

Number of examples: 130 rows of data
Validation set: None used (val_set_size: 0.0)
Data shuffled and packed for efficiency using Axolotl's sample packing

Training Procedure

Hyperparameters

Base model: mistralai/Mistral-7B-v0.1
Adapter: LoRA (q_proj, k_proj, v_proj, o_proj, gate_proj, down_proj, up_proj)
LoRA Rank: 16, Alpha: 32, Dropout: 0.05
Batch size: 2 (micro) × 4 (gradient accumulation) = 8 total
Learning rate: 2e-4
Optimizer: AdamW
Scheduler: Cosine
Sequence length: 2048
Epochs: 3
Train on inputs: False (only outputs used for loss) Hardware
Trained on RunPod using an H100 PCIe GPU

Framework Versions

PEFT: 0.14.0
Transformers: 4.49.0
PyTorch: 2.5.1+cu124
Datasets: 3.2.0
Tokenizers: 0.21.0
Axolotl: 0.8.0.dev0

License

This model is provided under a custom license:

✅ Use allowed for testing and evaluation

❌ No fine-tuning

❌ No redistribution

❌ No use in commercial or production applications

Contact

For questions or portfolio purposes, contact project@ezrahernowo.com.

Model Card Authors

Ezra H

Downloads last month: 14

Safetensors

Model size

7B params

Tensor type

BF16

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for bytess/zrah-model-1.0

Base model

mistralai/Mistral-7B-v0.3

Finetuned

mistralai/Mistral-7B-Instruct-v0.3

Adapter

(888)

this model