OmTheLast/taj-mahal-history-tutor-4b-experimental

Taj Mahal History Tutor 4B — experimental pilot

This is the selected step-100 checkpoint of a small, English-language Taj Mahal teaching experiment. It merges a LoRA adapter into Qwen3-4B-Instruct-2507, so the files are standard Transformers safetensors weights. The project is testing a training and evaluation recipe before attempting a broader Indian-history model. This checkpoint has not been certified for NCERT, ICSE, state-board coverage, or general Indian history.

Intended use

Use it for research and prototyping short Taj Mahal history questions. Check factual answers against primary or authoritative sources before using them as teaching material. The model can omit requested facts, invent details, and mishandle instructions to answer only from a supplied note.

Example with Transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "OmTheLast/taj-mahal-history-tutor-4b-experimental"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
messages = [
    {"role": "system", "content": "You are a history tutor. Answer each question directly using the requested format. Give accurate information and do not invent facts."},
    {"role": "user", "content": "Where is the Taj Mahal?"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
output = model.generate(inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(output[0, inputs.shape[-1]:], skip_special_tokens=True))

Training

The base model was Qwen/Qwen3-4B-Instruct-2507, revision cdbee75f17c01a7cc42f958dc650907174af0554. LoRA used rank 16, scale 32, the last 16 layers, learning rate 2e-5, batch size 4, and assistant-only loss. The source-audited Taj dataset contained 499 training examples and 88 validation examples spanning architecture, construction, economics, the Mughal court and succession, conservation, and material culture. The selected checkpoint was chosen by minimum validation loss before its release benchmark: at update 100 it had seen 400 unique training examples once. Later checkpoints showed rising validation loss and were not used for this release. Training questions and source passages are not included in this model repository because source redistribution rights were not established.

The exact adapter SHA-256 and export checks are in MODEL_PROVENANCE.json and EXPORT_VERIFICATION.json. The license file includes the base model's Apache 2.0 license.

Evaluation

On the frozen 96-question Taj pilot benchmark, the original base model answered 50/96 correctly and this checkpoint answered 60/96 correctly. For short and reworded open factual questions, the scores were 10/48 and 22/48 respectively. On 16 questions requiring an answer only from a supplied note, the scores were 16/16 and 12/16. Thus improved Taj recall came with regressions in grounded answering. The trained model also lost 9 answers the base had got right. These are small pilot results, graded under a fixed rubric by the project coordinator; the benchmark was inspected during development and is not an independent measure of general history knowledge. Exact answer wording can change results.

The merged Transformers export was reloaded on Apple MPS with no missing, unexpected, or mismatched weights. The chat template and one deterministic probe matched the MLX evaluation. That is an export smoke test, not a full benchmark of every runtime. A larger fresh comparison of later Taj checkpoints is being graded separately; this release does not claim those checkpoints passed.

GGUF runtime variant

The repository also contains gguf/taj-step100-Q6_K.gguf, a Q6_K quantization of this checkpoint for llama.cpp-compatible runtimes. It was converted from the merged weights with llama.cpp converter build 7650. A local 8-question runtime smoke test found exact text agreement with the frozen MLX answers on 3/8 prompts for both full-precision GGUF and Q6_K, versus 1/8 for Q4_K_M. Exact wording agreement is not a factual-correctness score; the Q6_K variant has not undergone the 96-question or fresh 192-question benchmark. Its output can differ from the merged Transformers model. GGUF_RUNTIME_SUMMARY.json records only file hashes, sizes and these aggregate counts; held-out questions and answers are not included.

Example:

llama-cli -m taj-step100-Q6_K.gguf -p "Where is the Taj Mahal?" -st -n 128 --temp 0

Limitations

• This is a Taj-only experimental model, not a trustworthy standalone school-history tutor.
• Some source-specific dates, counts, names, relationships, and attributions remain unstable.
• It can answer the central fact correctly while omitting parts of a multi-part question or adding unsupported details.
• The 96-question benchmark repeats related facts and was seen by the research process, though benchmark questions were excluded from training.
• Q6_K GGUF received only a small runtime smoke test. Browser performance and ONNX behavior remain untested.