Instructions to use RozGrov/NemoDori-v0.2-12B-MN-BT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use RozGrov/NemoDori-v0.2-12B-MN-BT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="RozGrov/NemoDori-v0.2-12B-MN-BT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("RozGrov/NemoDori-v0.2-12B-MN-BT")
model = AutoModelForCausalLM.from_pretrained("RozGrov/NemoDori-v0.2-12B-MN-BT", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use RozGrov/NemoDori-v0.2-12B-MN-BT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "RozGrov/NemoDori-v0.2-12B-MN-BT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RozGrov/NemoDori-v0.2-12B-MN-BT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/RozGrov/NemoDori-v0.2-12B-MN-BT

SGLang

How to use RozGrov/NemoDori-v0.2-12B-MN-BT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "RozGrov/NemoDori-v0.2-12B-MN-BT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RozGrov/NemoDori-v0.2-12B-MN-BT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "RozGrov/NemoDori-v0.2-12B-MN-BT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RozGrov/NemoDori-v0.2-12B-MN-BT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use RozGrov/NemoDori-v0.2-12B-MN-BT with Docker Model Runner:
```
docker model run hf.co/RozGrov/NemoDori-v0.2-12B-MN-BT
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

NemoDori-v0.2-12B-MN-BT

NemoDori-v0.2-12B-MN-BT is a merge of the following models using LazyMergekit:

Experimental 'sequel' of NemoDori (v0.1), an ERP-focused model, just for testing purpose. I still don't know what I've done...

My short experience using this:

When you instruct it to roleplay, it generates short chat-like response and to-the-point. (sometimes really short)
It advances the story slowly (even slower than v0.1 i think), responding to the last roleplay message quite nicely.
Creativity is maybe good(?).
Can follow instructions quite well (even on depth-0).

Update 1:
I was playing around with Talemate and... this model is pretty good. It was able to follow Talemate's instructions well, which Talemate then parsed it into it's format.
It can become all of Talemate agents (mostly maybe). So far i've tested, it is best when having a Conversation, capable of generating character's attributes (Creator) and the world state. When it's used for narrating (Narrator), sometimes it speaks for you and includes the Talemate's conversation (for this one, maybe I just didn't have a good instruction for it).

Those are my experiences using default presets from Talemate. I did tweaked them and played a bit near the end, and it does affect the results in a good way.
I'll try to test it again some more later.

🧩 Configuration

models:
  - model: crestf411/nemo-sunfall-v0.6.1
    parameters:
      weight: 0.5
  - model: unsloth/Mistral-Nemo-Instruct-2407
    parameters:
      weight: 0.3
  - model: RozGrov/NemoDori-v0.1-12B-MS
    parameters:
      weight: 1.0
merge_method: breadcrumbs_ties
base_model: RozGrov/NemoDori-v0.1-12B-MS
parameters:
    density: 0.95
    gamma: 0.01
dtype: float16

💻 Usage

!pip install -qU transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "RozGrov/NemoDori-v0.2-12B-MN-BT"
messages = [{"role": "user", "content": "What is a large language model?"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])

Downloads last month: 14

Safetensors

Model size

12B params

Tensor type

F16

Model tree for RozGrov/NemoDori-v0.2-12B-MN-BT

RozGrov/NemoDori-v0.1-12B-MS

crestf411/nemo-sunfall-v0.6.1

unsloth/Mistral-Nemo-Instruct-2407

Merge model

this model

Finetunes

2 models

Merges

2 models

Quantizations

2 models