sraivante/Housy-Gemma3-4B-Instruct-GGUF

Housy Gemma 3 4B Instruct — GGUF

An experimental English, Hindi and Hinglish model for Indian household inventory commands. Housy was fine-tuned from Unsloth's Gemma 3 4B Instruct, which derives from Google's Gemma 3 4B Instruct. This release contains the merged text-only Q4_K_M GGUF, ready for local experimentation with llama.cpp or an Ollama import.

The main training tasks were adding an item, increasing or decreasing its quantity, deleting an item, and answering household-item or food-knowledge questions. Commands are intended to produce {"intent", "tool", "arguments"} JSON for a separate application to validate and interpret.

Release status: experimental. Four basic add/increase/decrease/delete smoke tests passed. Broader operations, absolute quantity updates and factual knowledge showed failures. A greeting even produced an unrelated inventory command. See the complete results below before connecting this model to an application.

Property	Value
Publisher	sraivante
Base loaded for training	unsloth/gemma-3-4b-it
Method	Supervised LoRA fine-tuning with a 4-bit-loaded base
Family	Gemma 3 4B Instruct; approximately 3.88B text parameters
Input/output	Text only
Languages targeted	English, Hindi and Hinglish
Download	housy-gemma3-q4_k_m.gguf
Quantization	Q4_K_M, mixed GGUF tensor types
File size	2,498,096,512 bytes (2.50 GB / 2.33 GiB)
Training sequence length	2,048 tokens
Included Ollama context setting	4,096 tokens
License	Gemma Terms of Use

This package contains merged weights, not a standalone LoRA adapter. The vision tower was removed during export, so image input is not supported. Transformers Safetensors are not included. The associated training-era data snapshot is included under training_data/ and published separately as Housy Household Instructions.

Quick start with Ollama

Download the repository into a local folder using the Hugging Face CLI, then import its Modelfile. Requires hf and Ollama to be installed.

hf download sraivante/Housy-Gemma3-4B-Instruct-GGUF --local-dir housy-gemma3
cd housy-gemma3
ollama create housy-gemma3 -f Modelfile
ollama run housy-gemma3 "Please add 3 kilograms of basmati rice to my inventory."

The included Modelfile preserves the original command-oriented prompt, Gemma turn delimiters, temperature 0, top-p 0.1, repetition penalty 1.1, context 4096 and <end_of_turn> stop token. It asks for one JSON object, but it does not guarantee schema compliance or a correct command. Start a fresh request for each independent command.

The 2.50 GB file size is not the total runtime RAM requirement. Loading, working buffers and the KV cache need additional memory. On the release workstation, Ollama 0.34.3 failed to allocate memory with its default loading configuration; verification therefore used the lower-memory llama.cpp configuration below. Ollama generation was not successfully verified in this release audit.

Verified llama.cpp configuration

The release tests used a locally available llama.cpp development build b11096-c550d2f60, CPU inference, 2,048 context tokens and disabled weight repacking. No experimental prefill/decode quantization switching was enabled.

With a recent build that supports these options, run:

llama-server --model housy-gemma3-q4_k_m.gguf --host 127.0.0.1 --port 18081 --ctx-size 2048 --batch-size 128 --ubatch-size 64 --threads 6 --gpu-layers 0 --no-repack --no-warmup --parallel 1 --no-webui

In another terminal, run the included example:

python example_llamacpp.py "Please add 3 kilograms of basmati rice to my inventory."

The example requires only Python's standard library. It uses the same single-turn prompt format as the release tests and prints the response; it does not execute any generated tool call.

One observed response to that prompt was:

{
  "intent": "ADD_ITEM",
  "tool": "addItemTool",
  "arguments": {
    "itemName": "Basmati Rice",
    "hindiName": "बासमती चावल",
    "category": "GROCERY",
    "quantity": 3,
    "unit": "kg"
  }
}

The exact output can differ across runtime versions and settings.

Prompt and application contract

The default system_prompt.txt is:

You are Housy, an Indian household inventory assistant.
Always respond with a SINGLE strict JSON object: {intent, tool, arguments}.
No markdown, no commentary.

The original notebook also used a hybrid inventory/food-knowledge prompt, included as system_prompt_hybrid.txt for reproducibility. That mode can produce free text, and factual correctness has not been established.

The intended core command mappings are:

User request	Intent	Tool
Add a household item	ADD_ITEM	addItemTool
Increase/decrease a quantity	UPDATE_QUANTITY	updateQuantityTool
Delete an item	DELETE_ITEM	deleteItemTool

The application schema lists a wider tool vocabulary, but that list is not a capability guarantee. The model failed the tested inventory-listing, refill-attention, reminder and deleted-item-lookup requests.

The model generates text. It does not read an inventory database, schedule reminders, write reports or send email. A consuming application must check user intent, allowed intent/tool pairs, item identity, required fields, units and quantity semantics before considering an action. Unsupported or ambiguous requests should remain non-executable. JSON syntax alone is insufficient: the tests observed syntactically valid but incorrect commands.

Training evidence

The saved notebook records a completed run on one NVIDIA A100-SXM4-80GB, finishing 4,764 optimizer steps over two epochs. The progress display reports 3 hours 40 minutes 18 seconds.

Setting	Recorded value
Training examples	38,100
Evaluation examples	1,200
Notebook-described train mix	10,800 command + 27,300 knowledge records
Notebook-described evaluation scope	Commands only
LoRA rank / alpha / dropout	16 / 32 / 0
Trainable parameters	29,802,496
Per-device batch size	4
Gradient accumulation	4
Effective batch size	16
Learning rate	2e-4
Warmup ratio	0.03
Seed	42
Maximum sequence length	2,048
Precision	BF16 when supported; notebook reports BF16 support
Trained modules	Language attention and MLP modules; vision layers frozen

Messages were formatted with the Gemma 3 chat template. The saved training cell does not show explicit assistant-only loss masking.

Epoch	Displayed training loss	Validation loss
1	0.119747	0.159844
2	0.099951	0.144737

These are recorded fine-tuning losses, not task accuracy, and they precede GGUF quantization. No broad benchmark, independent generalization estimate or quantization-quality comparison is available. The newly published dataset audit found 24 distinct exact records and 47 command item names shared between the train and evaluation splits, so the recorded evaluation loss is not an uncontaminated estimate of generalization.

The notebook identifies the local datasets as housy_all_train.jsonl and housy_all_eval.jsonl. Their original Colab hashes and the exact base-model revision were not recorded. The original notebook is not distributed here. training_summary.json preserves the available configuration and result evidence.

Training and evaluation data

The included data package and standalone dataset contain 38,100 training records and 1,200 evaluation records, with schemas, loading examples, provenance, validation results and SHA256 checksums. Original dataset contributions are licensed under Apache 2.0 with Copyright © 2026 sraivante; the model weights remain under the Gemma terms.

The release preserves Git source revision 892cd2560bd18ca07324dd308ca6d6f66e8502a3 from June 14, 2026. Its counts and saved first training example match the completed notebook. No original Colab data checksum was saved, so exact historical byte-for-byte identity cannot be proven. Three later training additions and one later evaluation addition are excluded because they postdate the run.

The training split contains 10,800 command records and 27,300 knowledge records; evaluation contains commands only. Command targets cover creation, positive quantity changes, negative quantity changes and deletion. Broader tools mentioned in the system prompt and absolute quantity assignment are not represented in this snapshot's command labels.

The audit found 173 duplicate training records, two duplicate evaluation records and 24 distinct exact records shared across splits. Knowledge answers contain category-based templates and were not factually validated. These limits and the incomplete upstream provenance for item names are documented in the dataset card and audit.

Release smoke tests — 2026-09-25

Nine newly authored command prompts and four unscored probes were run on the exact released GGUF, without constrained JSON decoding. They are a small smoke test, not a verified held-out benchmark.

• 9/9 command responses were valid JSON objects with the three expected top-level keys and a dictionary of arguments. This checks only the envelope, not the full tool schema.
• 5/9 matched the expected intent and tool.
• 4/9 matched the expected intent/tool and all selected argument checks.
• The four basic add/increase/decrease/delete examples passed those checks.
• Required-field completeness and full semantic correctness were not exhaustively scored.

Case	Observed result
Add 3 kg of basmati rice, English	Expected command and selected arguments
Increase milk by 2 liters, Hindi	Expected command and selected arguments
Decrease rice by 250 grams, Hinglish	Expected command and selected arguments
Delete turmeric, English	Expected command
List groceries, Hindi	Invented LIST_ITEMS/listItemTool and an unrelated item
Find items needing a refill, English	Invented CHECK_INVENTORY/checkInventory
Remind about a bill in 5 days	Wrong tool and a fabricated calendar date
Find a deleted milk entry	Produced a deletion command instead of a lookup
Set flour stock to 4 kg, Hinglish	Wrongly set isDelta: true

Unscored probes also exposed important limitations:

• Hello Housy! produced an unrelated toothpaste-addition command.
• A question about France returned free text despite the command-only instruction.
• A paneer question returned a short dairy/storage answer; its factual content was not systematically scored.
• दाल क्या है? incorrectly described dal as a spice and made unsupported storage/quantity claims.

All prompts, expected fields, raw outputs and settings are in verification.json, including the failures. Food, nutrition and storage answers should not be treated as authoritative.

Limitations

This checkpoint is suitable for research, inspection and controlled inventory-parser experiments. Its observed command confusion makes it unsuitable for unattended inventory mutations or other automatic actions without an independently validated application layer.

Quality across paraphrases, regional language varieties, quantities, units, ambiguous requests and multi-turn conversations is not established. Hindi and Hinglish support describe the intended task languages, not uniform reliability.

The GGUF metadata retains the base model's 131,072-token context setting. This fine-tuning run used only 2,048 tokens, and long-context behavior was not evaluated. No image, audio or video capability is included. Raspberry Pi speed and memory use were not measured in this release audit.

Export and integrity

The saved notebook records this export path:

1. Save the trained LoRA adapter.
2. Reload the base in FP16 and merge the adapter with PEFT.
3. Extract the language backbone into Gemma3ForCausalLM and tie the output head to the token embeddings. The notebook reports zero missing and zero unexpected backbone keys.
4. Convert the text-only model to FP16 GGUF with llama.cpp.
5. Quantize to Q4_K_M.

The original local GGUF is published byte for byte. Its embedded name is Housy Text.

SHA256  c6e4d48d92649535a3d97c25af18a8150344cf8b8ea47aa5a2eff0151caa1ee0
Bytes   2498096512

SHA256SUMS contains checksums for the release files. The selected training notebook's SHA256 is 0d10c7194f92962700e2d174f4ac441b731f07a062192c32885e6ff2974fc818.

License and attribution

The files under training_data/ are a separate dataset release: the publisher's original contributions and selection/arrangement are Copyright © 2026 sraivante, licensed under Apache License 2.0, with scope and third-party attribution explained in COPYRIGHT.md. This data license does not relicense the Gemma weights.

This is an independent Gemma derivative, distributed under the Gemma Terms of Use. The use restrictions in Section 3.2 and the Gemma Prohibited Use Policy apply to use and further distribution. See the included LICENSE, NOTICE, use policy and model modification notice. Retain the required notices when redistributing.

Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms

Base-model credit: Google and the Gemma team. Fine-tuning tools: Unsloth, Hugging Face Transformers/TRL and PEFT. GGUF conversion and runtime: llama.cpp. This release is not endorsed by Google.