Gemma-E2B-Chimera-v4-MLX-4bit

Small body. Borrowed brains. Local first.

Gemma-E2B-Chimera is an experimental MLX-first text adaptation of Gemma 4 E2B focused on compact reasoning, instruction following, coding help, structured output, and useful local inference.

This repository contains the text v4 fused MLX 4-bit release candidate. It is intended for local Mac/MLX use and lightweight assistant workflows, not as a broad leaderboard claim.

The later multimodal sibling will be released separately as Gemma-E2B-Chimera-Multimodal.

Parameter Count Note

The Hugging Face parameter badge for this repository is expected to show about 738M parameters/stored tensor values. This is the text-only fused MLX 4-bit export, not the full upstream multimodal Gemma 4 E2B checkpoint.

Item	Count / Scope
This uploaded Chimera text-only MLX export	~738M stored tensor values
Upstream mlx-community/gemma-4-e2b-it-4bit package	~1.21B parameters reported by HF
Upstream google/gemma-4-E2B-it full multimodal checkpoint	~5.12B parameters reported by HF

In short: this release keeps the Gemma 4 E2B family/name lineage, but the uploaded artifact is a compact text-only MLX export.

---

Model Overview and Design

Gemma-E2B-Chimera v4 was built to make a very small Gemma-family model feel sharper in practical local workflows:

• Reasoning and planning: concise, useful answers without long rambles.
• Coding assistance: small Python/SQL/debugging tasks and test-writing prompts.
• Instruction following: exact line counts, short answers, and format constraints.
• Structured output: improved raw JSON behavior compared with stock E2B.
• Local-first deployment: MLX format, small enough for fast laptop inference.

This release is the survivor of many small ablations. Later experiments sometimes improved one slice, but none beat v4 on the overall keeper rule: better issue-chart behavior without regressing the broader local benchmark.

---

Training and Release Details

Field	Value
Release name	Gemma-E2B-Chimera
Variant	text v4
Format	fused MLX 4-bit
Base model	mlx-community/gemma-4-e2b-it-4bit
Uploaded parameter badge	~738M stored tensor values
Keeper adapter before fusion	chimera-phase4-json-6k-v1
Frozen date	2026-05-01
Runtime target	Apple Silicon / MLX

Training was done as an iterative local MLX LoRA workflow with small, targeted phases. Candidate phases covered reasoning, JSON repair, false-premise correction, code/tests, hard-negative preference data, and tool-call shape.

The final keeper is v4, because later SFT/DPO/ORPO experiments did not beat it under the promotion rules.

---

Evaluation

These are local project evals, not official public leaderboard submissions.

Full-40 Chimera Suite

The full-40 suite covers math/reasoning, coding, instruction following, exact JSON/tool-call formatting, false-premise correction, no-loop/verbosity control, and multiturn-style prompts.

Model	Score	Avg / 10	Hard fails
Gemma-E2B-Chimera v4	318 / 400	7.95	9
Stock Gemma 4 E2B	282 / 400	7.05	13
Stock Gemma 4 31B	261 / 400	6.53	15
Stock Gemma 4 E4B OptiQ	253 / 400	6.33	17
Phi-4-mini instruct 4bit	223 / 400	5.58	20

Category scores:

Model	Math	Coding	Instruction	JSON/tool	False premise	Control	Multiturn
Gemma-E2B-Chimera v4	48	34	54	35	27	60	60
Stock Gemma 4 E2B	51	38	54	10	26	60	43
Stock Gemma 4 31B	43	39	50	10	26	50	43
Stock Gemma 4 E4B OptiQ	34	18	54	10	34	60	43
Phi-4-mini instruct 4bit	34	36	48	0	21	48	36

Big-Boy Smoke v1

Compact local smoke suite inspired by IFEval, BFCL, GSM/MATH, LiveCodeBench, and SimpleQA. This is a practical sanity check, not an official leaderboard result.

Generation: raw no-harness, temperature 0, top_p 0, max_tokens 768, disable-thinking where supported.

Rank	Model	Total	Avg / 10	Hard fails	IFEval-lite	BFCL-lite	GSM/MATH-lite	LiveCodeBench-lite	SimpleQA-lite
1	Stock Gemma 4 E4B OptiQ	194 / 300	6.47	12	44 / 60	2 / 60	44 / 60	44 / 60	60 / 60
2	Gemma-E2B-Chimera v4	184 / 300	6.13	13	50 / 60	2 / 60	44 / 60	28 / 60	60 / 60
3	Stock Gemma 4 E2B	154 / 300	5.13	17	44 / 60	2 / 60	36 / 60	12 / 60	60 / 60
4	Phi-4-mini instruct 4bit	152 / 300	5.07	17	36 / 60	0 / 60	28 / 60	28 / 60	60 / 60

JSON and Tool-Call Note

Raw structured output improved versus stock E2B in the full-40 suite, but strict tool-call shape is still brittle.

For deployment, an optional local JSON harness was tested with oMLX:

Mode	Scope	Score	Readout
Chimera v4 raw no-thinking	BFCL-lite only	2 / 60	raw model often uses near-miss tool shapes
Chimera v4 JSON harness	BFCL-lite only	34 / 60	parser validation + retry improves reliability

The harness is a deployment wrapper, not a different model score.

This repo includes the optional harness here:

• tools/omlx_json_harness.py
• tools/README.md

---

Best Practices

Recommended deterministic settings for exact tasks:

temperature = 0.0
top_p = 0.0
max_tokens = 768 to 8192 depending on task length
enable_thinking = false when the runtime supports it

For normal chat:

• Use the model's bundled chat template.
• Keep global system prompts light.
• Prefer task-local instructions over a heavy permanent system prompt.
• For code, ask for runnable code only when that is what you want.
• For JSON/tool output, use parser validation and retry in production.

Suggested light system/profile prompt:

You are Gemma-E2B-Chimera, a careful local assistant. Answer the user's request directly and follow requested formats exactly. Be concise unless the task needs detail. For code, prefer runnable, minimal code. For math, logic, and planning, check the final answer before responding. If a request is impossible or underspecified, say so briefly and explain the blocker. Do not expose hidden reasoning; provide only the useful final answer and concise reasoning when helpful.

Do not force this prompt for strict JSON/tool tasks; use a JSON harness instead.

---

MLX Usage

Install MLX-LM, then run:

python -m mlx_lm.generate \
  --model stamsam/Gemma-E2B-Chimera-v4-MLX-4bit \
  --prompt "Explain why local-first AI matters in one paragraph." \
  --max-tokens 512 \
  --temp 0.0

For local folder inference:

python -m mlx_lm.generate \
  --model ./Gemma-E2B-Chimera-v4-MLX-4bit \
  --prompt "Write a Python function that reverses a string." \
  --max-tokens 512 \
  --temp 0.0

---

oMLX Compatibility

The canonical upload is the fused MLX model.

For oMLX, use the separate local compatibility export generated in the project as:

Gemma-E2B-Chimera-v4-text-fused-mlx-omlx

That export removes 140 shared-KV tensors that oMLX reports as extra parameters. It is a runtime compatibility package and should be published separately if needed.

---

Limitations

• This is a text-only release. Multimodal preservation is planned for a separate sibling model.
• It is still a very small model; long code synthesis can fail even when the architecture idea is right.
• Raw tool-call JSON is not fully reliable without a parser/retry harness.
• Some false-premise correction and tests-only prompts remain brittle.
• Local benchmark results are project-specific and should not be interpreted as broad public leaderboard wins.
• Verify important answers, especially for medical, legal, financial, safety-critical, or high-impact decisions.

---

Acknowledgements

Special thanks and credit to:

• Google and the Gemma team for the base model family.
• MLX and MLX-LM for making local Apple Silicon training and inference practical.
• Jackrong for the public distillation datasets and small-model release inspiration.
• Jiunsong and the SuperGemma model-card/release-hardening style that helped shape the eval discipline.
• The open-source dataset and local-LLM communities exploring reasoning distillation, tool calling, structured output, and small-model post-training.

Datasets and references used or studied during the project included:

• Jackrong/Qwen3.5-reasoning-700x
• Jackrong/DeepSeek-V4-Distill-8000x
• Jackrong/Kimi-K2.5-Reasoning-1M-Cleaned
• Jackrong/GLM-5.1-Reasoning-1M-Cleaned
• Jackrong/Qwen3.5-9B-DeepSeek-V4-Flash
• public tool-calling and structured-output datasets used as inspection/reference pools

Not every inspected dataset was bulk-trained into the final keeper. The keeper was selected by eval outcome, not by maximum data volume.

---

Citation

@misc{stamsam_gemma_e2b_chimera_v4_mlx_4bit,
  title        = {Gemma-E2B-Chimera-v4-MLX-4bit},
  author       = {Stamsam Labs},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {https://huggingface.co/stamsam/Gemma-E2B-Chimera-v4-MLX-4bit}
}