Not-For-All-Audiences

Instructions to use lemonilia/Limamono-Mistral-7B-v0.50 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lemonilia/Limamono-Mistral-7B-v0.50 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lemonilia/Limamono-Mistral-7B-v0.50")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lemonilia/Limamono-Mistral-7B-v0.50")
model = AutoModelForCausalLM.from_pretrained("lemonilia/Limamono-Mistral-7B-v0.50", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use lemonilia/Limamono-Mistral-7B-v0.50 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf lemonilia/Limamono-Mistral-7B-v0.50:Q2_K
# Run inference directly in the terminal:
llama cli -hf lemonilia/Limamono-Mistral-7B-v0.50:Q2_K

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf lemonilia/Limamono-Mistral-7B-v0.50:Q2_K
# Run inference directly in the terminal:
llama cli -hf lemonilia/Limamono-Mistral-7B-v0.50:Q2_K

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf lemonilia/Limamono-Mistral-7B-v0.50:Q2_K
# Run inference directly in the terminal:
./llama-cli -hf lemonilia/Limamono-Mistral-7B-v0.50:Q2_K

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf lemonilia/Limamono-Mistral-7B-v0.50:Q2_K
# Run inference directly in the terminal:
./build/bin/llama-cli -hf lemonilia/Limamono-Mistral-7B-v0.50:Q2_K

Use Docker

docker model run hf.co/lemonilia/Limamono-Mistral-7B-v0.50:Q2_K

LM Studio
Jan

vLLM

How to use lemonilia/Limamono-Mistral-7B-v0.50 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lemonilia/Limamono-Mistral-7B-v0.50"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lemonilia/Limamono-Mistral-7B-v0.50",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/lemonilia/Limamono-Mistral-7B-v0.50:Q2_K

SGLang

How to use lemonilia/Limamono-Mistral-7B-v0.50 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lemonilia/Limamono-Mistral-7B-v0.50" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lemonilia/Limamono-Mistral-7B-v0.50",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lemonilia/Limamono-Mistral-7B-v0.50" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lemonilia/Limamono-Mistral-7B-v0.50",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Ollama
How to use lemonilia/Limamono-Mistral-7B-v0.50 with Ollama:
```
ollama run hf.co/lemonilia/Limamono-Mistral-7B-v0.50:Q2_K
```
Unsloth Desktop
Docker Model Runner
How to use lemonilia/Limamono-Mistral-7B-v0.50 with Docker Model Runner:
```
docker model run hf.co/lemonilia/Limamono-Mistral-7B-v0.50:Q2_K
```

Lemonade

How to use lemonilia/Limamono-Mistral-7B-v0.50 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull lemonilia/Limamono-Mistral-7B-v0.50:Q2_K

Run and chat with the model

lemonade run user.Limamono-Mistral-7B-v0.50-Q2_K

List all available models

lemonade list

Atomic Chat

lemonilia commited on Dec 13, 2023

Commit

7616ab2

1 Parent(s): 4cb8902

Update README.md

Browse files

Files changed (1) hide show

README.md +6 -10

README.md CHANGED Viewed

@@ -7,8 +7,8 @@ tags:
 - not-for-all-audiences
 ---
-# Limamono-7B (Mistral) v0.43
-This is an **early version** (43% completed) of a strongly NSFW roleplaying model trained with
 _extremely limited_ amounts of almost entirely synthetic data of hopefully higher quality than typical
 human conversations. The intended target audience is straight men and lesbians.
@@ -154,7 +154,7 @@ in the repository.
 ## Text generation settings
 For testing I use these settings:
 - Temperature: 1.0
-- Tail-Free Sampling: 0.85–0.89
 - Repetition Penalty: 1.11
 - Repetition Penalty range: 2048
 - Top-p: 1 (disabled), Top-k: 0 (disabled)
@@ -163,7 +163,7 @@ For testing I use these settings:
 [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) was used for training
 on one NVidia RTX3090.
-The training data consisted of **43** conversations (171k tokens / 963 messages)
 of roughly 4k tokens length. The learning rate is the one that about minimizes the
 eval loss on one epoch with a constant learning schedule. For the following two epochs
 what would be normally considered overfitting occurs, but at the same time output
@@ -182,7 +182,7 @@ quality also improves.
 - micro_batch_size: 1
 - num_epochs: 3
 - optimizer: adamw_torch
-- lr_scheduler: constant
 - learning_rate: 0.0002
 - weight_decay: 0.1
 - train_on_inputs: false
@@ -192,8 +192,4 @@ quality also improves.
 - tf32: true
 ### Train loss graph
-This one was obtained by experimentally repeating the data 3 times and finetuning for 1 epoch,
-with similar end results but a smoother graph without sudden jumps compared to finetuning
-unique data for 3 epochs.
-![Train loss](https://files.catbox.moe/hiu9ah.png)

 - not-for-all-audiences
 ---
+# Limamono-7B (Mistral) v0.50
+This is an **early version** (50% completed) of a strongly NSFW roleplaying model trained with
 _extremely limited_ amounts of almost entirely synthetic data of hopefully higher quality than typical
 human conversations. The intended target audience is straight men and lesbians.
 ## Text generation settings
 For testing I use these settings:
 - Temperature: 1.0
+- Tail-Free Sampling: 0.85
 - Repetition Penalty: 1.11
 - Repetition Penalty range: 2048
 - Top-p: 1 (disabled), Top-k: 0 (disabled)
 [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) was used for training
 on one NVidia RTX3090.
+The training data consisted of **50** conversations (199k tokens / 1117 messages)
 of roughly 4k tokens length. The learning rate is the one that about minimizes the
 eval loss on one epoch with a constant learning schedule. For the following two epochs
 what would be normally considered overfitting occurs, but at the same time output
 - micro_batch_size: 1
 - num_epochs: 3
 - optimizer: adamw_torch
+- lr_scheduler: cosine
 - learning_rate: 0.0002
 - weight_decay: 0.1
 - train_on_inputs: false
 - tf32: true
 ### Train loss graph
+![Train loss](https://files.catbox.moe/dg4qww.png)