README.md

---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
base_model: BananaMind/BananaMind-2-Nano
datasets:
- HuggingFaceTB/smol-smoltalk
tags:
- causal-lm
- chat
- instruction-tuned
- full-finetune
- small-language-model
- bananamind
- bananamind2
- bananamind2-nano
- smoltalk
- pytorch
- safetensors
- custom-code
- trust-remote-code
---

![BananaMind 2 Nano Chat](banner.png)

# BananaMind-2-Nano-Chat

BananaMind-2-Nano-Chat is the instruction-tuned version of [BananaMind-2-Nano](https://huggingface.co/BananaMind/BananaMind-2-Nano). It was fully fine-tuned on [HuggingFaceTB/smol-smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk) with loss applied only to assistant tokens and the assistant-ending EOS token.

The model has **9,968,128 parameters**, a **4,096-token context window**, and a custom **8,192-token digit-aware byte-level BPE tokenizer**. It supports system prompts, multi-turn conversations, and KV-cached generation.

## Model Details

| Field | Value |
|---|---:|
| Parameters | 9,968,128 |
| Base model | `BananaMind/BananaMind-2-Nano` |
| Architecture | BananaMind2Nano decoder-only Transformer |
| Layers | 10 |
| Hidden size | 256 |
| Intermediate size | 768 |
| Attention heads | 4 |
| KV heads | 2 |
| Head dimension | 64 |
| Attention | Grouped-query attention with QK norm |
| MLP | SwiGLU |
| Position embeddings | RoPE, theta 100,000 |
| Vocabulary size | 8,192 |
| Context length | 4,096 |
| Embeddings | Tied input/output embeddings |
| Generation cache | KV cache supported |
| Weight format | safetensors |
| Training type | Full-parameter supervised fine-tuning |

## Benchmarks

### BananaMind Instruct Bench 1.1

Self-reported results from the official [BananaMind Instruct Bench 1.1](https://huggingface.co/datasets/BananaMind/BananaMind-Instruct-Bench-1.1) script. This is a 300-example, difficulty- and category-weighted instruction benchmark. Generation was deterministic with `repetition_penalty=1.1` and seed 42.

| Model | Overall ELO | General | Multi-turn | System Prompts | Context Recall | Code |
|---|---:|---:|---:|---:|---:|---:|
| BananaMind-2-Medium-Chat | 787 | 534 | 872 | 666 | 1,085 | 1,291 |
| BananaMind-2-Mini-Chat | 654 | 389 | 812 | 743 | 924 | 667 |
| Supra 1.5 50M Instruct* | 647 | 520 | 804 | 590 | 860 | 780 |
| **BananaMind-2-Nano-Chat** | **643** | **353** | **796** | **600** | **978** | **885** |
| Supra 1.0 50M Instruct* | 520 | 531 | 405 | 590 | 794 | 667 |

**Detailed BananaMind-2-Nano-Chat result**

| Scope | Examples | Passed | Weighted Score | ELO |
|---|---:|---:|---:|---:|
| **Overall** | **300** | **35** | **9.15%** | **643** |
| General | 120 | 4 | 2.63% | 353 |
| Multi-turn | 75 | 18 | 15.58% | 796 |
| System Prompts | 60 | 2 | 3.42% | 600 |
| Context Recall | 30 | 9 | 23.16% | 978 |
| Code | 15 | 2 | 10.53% | 885 |

| Difficulty | Examples | Passed | Weighted Score | ELO |
|---|---:|---:|---:|---:|
| Easy | 100 | 26 | 27.70% | 763 |
| Medium | 100 | 8 | 8.82% | 622 |
| Hard | 100 | 1 | 1.12% | 459 |

\* Supra 1.5 used the benchmark's Alpaca-style fallback, with conversation context supplied through its `### Input` field. Supra 1.0 used the same fallback and a copied checkpoint with 3.0x linear RoPE scaling, extending its context from 1,024 to 3,072 tokens without additional long-context training so it could fit the benchmark inputs.

These scores are self-evaluated and may vary with the benchmark revision, Transformers version, dtype, hardware, and generation settings.

## Instruction Tuning

| Field | Value |
|---|---:|
| Dataset | `HuggingFaceTB/smol-smoltalk` |
| Dataset split | `train` |
| Dataset rows | 460,341 |
| Epochs | 1 |
| Final optimizer step | 3,193 |
| Packed tokens processed | 470,782,108 |
| Supervised assistant tokens | 360,967,001 |
| Sequence length | 4,096 |
| Micro batch | 18 |
| Gradient accumulation | 2 |
| Effective batch | 36 sequences |
| Peak learning rate | 2e-4 |
| Warmup | 100 steps |
| LR schedule | Constant after warmup |
| Optimizer | AdamW |
| Betas | 0.9, 0.95 |
| Weight decay | 0.1 |
| Gradient clipping | 1.0 |
| Seed | 1337 |

System and user messages were retained as context but masked from the loss. All parameters were trainable; no LoRA modules or adapters were used.

## Chat Template

The tokenizer includes a Jinja template for `system`, `user`, and `assistant` messages:

```text
<BOS><|system|>
{system message}
<|user|>
{user message}
<|assistant|>
{assistant response}<EOS>
```

The role markers use the existing tokenizer vocabulary. Fine-tuning did not add new tokens.

## Usage

This model uses custom architecture code and must be loaded with `trust_remote_code=True`.

```bash
pip install -U transformers safetensors torch
```

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "BananaMind/BananaMind-2-Nano-Chat"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" and torch.cuda.is_bf16_supported() else torch.float32

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=dtype,
).to(device).eval()

messages = [
    {"role": "system", "content": "You are a concise and helpful assistant."},
    {"role": "user", "content": "Write one Python line that prints 2 + 3."},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
)
inputs = {name: tensor.to(device) for name, tensor in inputs.items()}

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=192,
        do_sample=False,
        repetition_penalty=1.1,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.eos_token_id,
        use_cache=True,
    )

new_tokens = output[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))
```

For multi-turn chat, append the generated assistant response and the next user message to `messages`, then render the chat template again.

## Suggested Generation Settings

- `do_sample=False`
- `repetition_penalty=1.1`
- `max_new_tokens=128` to `256`
- `use_cache=True`

Small models can hallucinate facts, fail arithmetic, produce invalid code, or enter repetitive continuations. Keep a finite generation limit and do not use the model for high-stakes decisions.

## Repository Files

| File | Description |
|---|---|
| `config.json` | Transformers model configuration |
| `model.safetensors` | Fine-tuned model weights |
| `tokenizer.json` | Custom 8,192-token tokenizer |
| `tokenizer_config.json` | Tokenizer metadata |
| `chat_template.jinja` | System/user/assistant chat template |
| `generation_config.json` | Generation configuration |
| `configuration_bananamind2nano.py` | Custom Transformers config class |
| `modeling_bananamind2nano.py` | Custom Transformers model class |
| `sft_metadata.json` | Fine-tuning provenance and token counts |
| `banner.png` | Model-card banner |

## Intended Use

This model is intended for compact-model research, local chat experiments, educational demonstrations, and instruction-tuning comparisons.

## License

Apache 2.0