license: apache-2.0
base_model: google/gemma-3-4b-it
language:
- en
library_name: peft
pipeline_tag: text-generation
tags:
- date-arithmetic
- temporal-reasoning
- business-days
- timezones
- iso-week
- lora
- adaption-autoscientist
ChronoCalc-4B: Calendar Arithmetic Where Calendars Actually Break
ChronoCalc-4B is a LoRA adaptation of google/gemma-3-4b-it for the date and time computations
that real systems get wrong: business days across federal holidays, rolling a date off a weekend,
elapsed time across a daylight saving transition, the next occurrence of a recurring schedule, and
ISO week numbers including the years that have a week 53.
Built for the Adaption AutoScientist Challenge, Part 2 (Math and Code); the training corpus was co-optimized with Adaptive Data (Adaption Labs).
Read this before the numbers
This model does not claim to take the base from zero. On plain day-walking with no holiday in
the window, gemma-3-4b-it already scores 69 to 86% at every chain depth we tested. Saying
otherwise would be easy and false.
What the base cannot do is handle the exceptions:
| the base model, measured | score |
|---|---|
| plain day-walking, no holiday in the window | 69 to 86% |
| windows where a holiday changes the answer | 7% (2 of 30) |
| elapsed time across a DST transition | 2% |
| cross-year ISO week numbers | 0% |
That is the whole thesis. Deadlines do not go wrong because someone cannot add seven days. They go wrong because Veterans Day fell in the window, or the clocks moved, or the year had 53 weeks.
So the claim here is scoped: improvement on calendar exceptions, and no regression on plain day-walking. Both are reported.
Headline results
Base and tuned generated in one process under identical greedy decoding, scored by the released
score_chrono.py, which re-derives all 60,000 gold answers from row parameters in stdlib Python.
| slice | rows | base | ChronoCalc-4B |
|---|---|---|---|
| headline held-out | 500 | 11.0% | 91.0% |
| hard shard | 300 | 14.7% | 93.3% |
| enumerated calendar | 200 | 23.5% | 90.5% |
| range slice, 2031 to 2035 | 150 | 14.7% | 76.7% |
The gate that nearly killed this entry, and the rule written before it ran
The pre-registered Day-0 gate measured a blended F1 floor of 34%, which sat between the pass and
kill bands. The decomposition explains why: the blend is a property of the pilot mix, not of the
model. The pilot was 60% holiday-affected, and share x 7% + (1 - share) x 70% reproduces the
measured 34% to within half a point.
Because "between the bands after a rerun" was undefined in the spec, an adjudication rule was
written and committed to git before the rerun was generated or scored
(docs/eval/gcp/chrono_gate_adjudication.md). An adjudication rule invented after seeing the number
it adjudicates is worthless. The rule set four conditions, all of which had to hold or the entry
died with no further branches. It passed.
This is on the card as measurement transparency rather than buried, in the same way the DataViz entry shipped its disclosed regression with a diagnosis.
Every answer was computed twice
solver_a produces the answer and the working. solver_b, written from scratch with different
primitives, produces the answer again. A row whose solvers disagree is dropped and logged, never
repaired. There are 0 disagreements across 60,000 rows, and no language model appears anywhere
in the label path.
A reviewer can recompute every published number on a laptop with no GPU.
Scope and operating notes
- The base is not bad at dates in general, and this card refuses to imply otherwise. The improvement is scoped to calendar exceptions.
- 90% of rows name the holiday calendar by statute rather than enumerating it, because knowing the US federal calendar is part of the trained behaviour. The 200-row enumerated held-out slice prints the calendar in full, so the headline can be rechecked without any statute knowledge. Report both.
- Scope. US federal holidays, Gregorian dates, the IANA timezone database. Not other national calendars, not historical calendar reforms, not leap seconds.
- Completions are long on purpose. Every one enumerates its computation line by line before the
<answer>tag. A terser format would score the same on exact match and be worth less, and terseness has twice generalized out of a shard on this project and cost win rate on untouched tasks.
Training details
| base | google/gemma-3-4b-it |
| method | LoRA r32 alpha64 on the language-model linears, completion-only masking |
| corpus | 60,000 rows, 5 families, 51 distinct templates |
| epochs | 3 |
| max_len | 1024, measured (total p99 833, max 960, 0 of 60,000 over budget) |
| hardware | one A100-40GB, Spot |
max_len is measured because a 768 window would truncate roughly the top decile, and a truncated
completion loses its <answer> tag entirely, so the row would teach nothing under completion-only
masking.
Evaluation protocol
Base and tuned generated in one process under identical greedy decoding (do_sample=False).
Output length compared base against tuned, with a tuned median below 0.6x the base median failing the
run regardless of accuracy. Every gate was proven able to fail by injecting deliberately corrupted
rows before the corpus was trusted.
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
BASE = "google/gemma-3-4b-it"
tok = AutoTokenizer.from_pretrained(BASE)
# AutoModelForCausalLM resolves gemma-3-4b-it to its multimodal wrapper, which is correct.
# Do NOT load it through a text-only class: the decoder is then randomly initialized and
# the model emits whitespace, with no error raised.
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype="bfloat16",
attn_implementation="eager", device_map="cuda")
model = PeftModel.from_pretrained(model, "Jainamshahhh/chronocalc-4b").eval()
Reproducibility and license
The corpus regenerates byte for byte from a single seed, because a row is a pure function of its integer id. The scorer, every held-out slice and both solvers are published. Apache-2.0, matching the base model. Built with Adaptive Data by Adaption Labs, whose enhancement pass on this corpus was run, measured, and refused when it rewrote dates inside the questions; that decision is documented rather than omitted.