--- language: - en - th license: apache-2.0 tags: - gemma-4 - unsloth - lora - qlora - sft - text-generation - heretic - uncensored - llmfan46 - reasoning - claude-distill base_model: llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic pipeline_tag: text-generation inference: false --- # ๐Ÿง  Gemma 4 E4B Ultra Uncensored Heretic โ€” Unsloth QLoRA (r=16) **Model ID:** `hotdogs/gemma4-E4B-heretic_claude4.7-reasoning_lora-r16-step1290` A lightweight **LoRA adapter** (rank 16) fine-tuned with **QLoRA** on [llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic](https://huggingface.co/llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic) using [Unsloth](https://github.com/unslothai/unsloth). Trained on reasoning traces distilled from Claude Opus for 3 epochs (1,335 steps) โ€” just **73 MB** (F16 GGUF) / **162 MB** (safetensors). --- ## ๐Ÿ“Š Training Summary | Metric | Value | |--------|-------| | **Base Model** | [llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic](https://huggingface.co/llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic) | | **Dataset** | [lordx64/reasoning-distill-claude-opus-4-7-max](https://huggingface.co/datasets/lordx64/reasoning-distill-claude-opus-4-7-max) | | **Training Type** | QLoRA (`load_in_4bit: true`) | | **Epochs** | 3.0 (1,335 steps) | | **Train Loss** | 13.32 โ†’ **2.22** (โ†“ 83%) | | **Final Step Loss** | **1.28** (step 1,335) | | **Eval Loss** | 2.88 | | **Learning Rate** | `2e-4` โ†’ cosine decay โ†’ `1.5e-7` | | **Total Tokens Seen** | 1,608,612 | | **Training Time** | ~7.2 hours | | **Hardware** | NVIDIA RTX 4060 Ti 16GB | | **CUDA / Driver** | 13.0 / 580.126.09 | --- ## ๐Ÿ› ๏ธ LoRA Configuration | Parameter | Value | |-----------|-------| | **Rank (`r`)** | 16 | | **Alpha** | 16 (`lora_alpha / r = 1.0`) | | **Dropout** | 0.0 | | **Target Modules** | `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj` | | **Bias** | `none` | | **PEFT Version** | 0.18.1 | ### Additional Training Settings | Parameter | Value | |-----------|-------| | **Batch Size** | 1 (effective 18 with gradient accumulation) | | **Max Seq Length** | 512 | | **Optimizer** | `adamw_bnb_8bit` | | **LR Scheduler** | `linear` | | **Warmup Steps** | 5 | | **Weight Decay** | 0.001 | | **Random Seed** | 3407 | | **Sequence Packing** | โœ… enabled | | **Gradient Checkpointing** | `unsloth` | --- ## ๐Ÿ“ˆ Loss Curve ``` Step 0: 13.32 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ Step 300: 2.13 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ– Step 600: 1.64 โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ Step 900: 1.59 โ–ˆโ–ˆโ–ˆโ–ˆโ–Š Step 1200: 1.61 โ–ˆโ–ˆโ–ˆโ–ˆโ–‰ Step 1335: 1.28 โ–ˆโ–ˆโ–ˆโ–‰ โ† FINAL ``` Training converged smoothly from initial loss `~13.3` down to `2.22` (average). The final training step achieved **1.28** loss. Eval loss at 2.88 suggests moderate overfitting common with small LoRA adapters โ€” expected and acceptable for the adapter size (73 MB). --- ## ๐Ÿš€ How to Use ### Option 1: PEFT (PyTorch) ```python from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel base_model = "llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic" lora_path = "hotdogs/gemma4-E4B-heretic_claude4.7-reasoning_lora-r16-step1290" model = AutoModelForCausalLM.from_pretrained( base_model, torch_dtype="auto", device_map="auto" ) tokenizer = AutoTokenizer.from_pretrained(base_model) model.load_adapter(lora_path, adapter_name="lora") model.set_active_adapter("lora") messages = [{"role": "user", "content": "Explain the theory of relativity simply."}] text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = tokenizer(text, return_tensors="pt").to(model.device) outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ``` ### Option 2: Unsloth (Recommended โ€” 2ร— faster, uses less VRAM) ```python from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( model_name="hotdogs/gemma4-E4B-heretic_claude4.7-reasoning_lora-r16-step1290", max_seq_length=2048, load_in_4bit=True, # or False for BF16 ) FastLanguageModel.for_inference(model) messages = [{"role": "user", "content": "Write a poem about AI in Thai."}] text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = tokenizer(text, return_tensors="pt").to("cuda") output = model.generate(**inputs, max_new_tokens=512, temperature=0.7) print(tokenizer.decode(output[0], skip_special_tokens=True)) ``` ### Option 3: GGUF (llama.cpp) Download `gemma-4-E4B-uncensored-heretic-lora-r16.f16.gguf` (73 MB) from the `gguf/` directory. Then use with your existing base model GGUF: ```bash # Serve with llama.cpp LoRA support (llama-server with --lora) llama-server \ -m gemma-4-E4B-it-ultra-uncensored-heretic-Q6_K.gguf \ --lora gemma-4-E4B-uncensored-heretic-lora-r16.f16.gguf \ --lora-scaled gemma-4-E4B-uncensored-heretic-lora-r16.f16.gguf 1.0 ``` --- ## ๐Ÿ“ฆ Model Files | File | Format | Size | |------|--------|------| | `adapter_model.safetensors` | PEFT safetensors | 162 MB | | `adapter_config.json` | PEFT config | 1.3 KB | | `gguf/gemma-4-E4B-uncensored-heretic-lora-r16.f16.gguf` | GGUF LoRA (F16) | **73.4 MB** | | `tokenizer.json` | Tokenizer | 31 MB | | `trainer_state.json` | Training log | 394 KB | --- ## โš ๏ธ Limitations & Bias - **LoRA Adapter only** โ€” you need the base model [`llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic`](https://huggingface.co/llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic) loaded separately (no merged weights included). - **Eval gap** โ€” train loss 1.28 vs eval loss 2.88 indicates some overfitting on the Claude reasoning dataset. - **Limited context** โ€” trained with `max_seq_length=512` and sequence packing. Performance on very long reasoning chains may degrade. - **Uncensored** โ€” the base model has minimal alignment filtering, so outputs may be more creative/unfiltered than standard models. - **No formal benchmarks** โ€” MMLU, GSM8K, etc. not evaluated. Loss-based convergence suggests improved reasoning over the base model. - **Single GPU** โ€” trained on one RTX 4060 Ti 16GB with `batch_size=1`. Larger-scale generalization may vary. --- ## ๐Ÿ“š Citation If you use this model in research or production, please credit: ```bibtex @misc{gemma4-e4b-heretic-lora-2025, author = {UKA (Hermes Agent)}, title = {Gemma 4 E4B Ultra Uncensored Heretic โ€” Unsloth QLoRA Fine-tuned on Claude Reasoning Distill}, year = {2025}, publisher = {Hugging Face}, howpublished = {\\url{https://huggingface.co/hotdogs/gemma4-E4B-heretic_claude4.7-reasoning_lora-r16-step1290}}, note = {Trained with Unsloth on RTX 4060 Ti. Base model by llmfan46.} } ``` --- ## ๐Ÿ”— Links | Resource | URL | |----------|-----| | **Base Model** | [llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic](https://huggingface.co/llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic) | | **Dataset** | [lordx64/reasoning-distill-claude-opus-4-7-max](https://huggingface.co/datasets/lordx64/reasoning-distill-claude-opus-4-7-max) | | **Unsloth** | [github.com/unslothai/unsloth](https://github.com/unslothai/unsloth) | | **PEFT** | [huggingface.co/docs/peft](https://huggingface.co/docs/peft) | | **GGUF LoRA Guide** | [llama.cpp LoRA](https://github.com/ggml-org/llama.cpp/discussions/11379) | --- ## ๐Ÿ“ Changelog | Date | Event | |------|-------| | 2026-05-05 09:33 | Training started on `llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic` | | 2026-05-05 16:46 | Training completed โ€” checkpoint-1335 (3 epochs, 1,608,612 tokens) | | 2026-05-05 22:43 | GGUF LoRA exported โ€” 73.4 MB F16 | --- *Made with ๐Ÿ’œ by UKA ยท Powered by Unsloth & NVIDIA RTX 4060 Ti*