SargeDev/Solar-JEV-Gemma-embed-2-GGUF

Solar-JEV-Gemma-embed-2 — GGUFs

GGUF exports of SargeDev/Solar-JEV-Gemma-embed-2 — the frozen-EmbeddingGemma-2 probe that decodes calibrated decision judgments (noul / choice / score, distilled from Solar Decide Flash on SargeDev's decision corpora).

Files

File	Size	Notes
solar-jev-gemma-embed-2-F16.gguf	542 MB	bf16→F16 base export, 413 tensors
solar-jev-gemma-embed-2-IQ4_XS.gguf	170 MB	dynamic-style 4-bit (smaller)
solar-jev-gemma-embed-2-Q4_K_M.gguf	182 MB	4-bit K-quants
solar-jev-gemma-embed-2-Q5_K_M.gguf	213 MB	5-bit
solar-jev-gemma-embed-2-Q6_K.gguf	246 MB	near-lossless 6-bit
solar-jev-gemma-embed-2.imatrix	1.3 MB	importance matrix, 4,000 states sampled from corpus-v4 calibration split — for custom requantizing

Serve embeddings

Build llama.cpp with EmbeddingGemma-2 support (recent master), then:

llama-server -m solar-jev-gemma-embed-2-IQ4_XS.gguf \
  --embeddings --pooling mean \
  --ctx-size 2048 --batch-size 2048 --ubatch-size 2048 \
  --host 127.0.0.1 --port 8811

Get embeddings:

curl http://127.0.0.1:8811/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"model":"m","input":["task: classification | query: Is immediate intervention required? The cooling unit fails during peak load."],"encoding_format":"float"}'

Returns a 768-dim, L2-normalized vector (llama.cpp normalizes automatically).

Decoding judgments (heads)

The trained linear heads ship separately as heads.pt in SargeDev/Solar-JEV-Gemma-embed-2 (+ HF-format checkpoints there).

⚠️ Normalization contract: the published heads were trained on raw (un-normalized) mean-pooled EmbeddingGemma-2 vectors. llama.cpp outputs L2-normalized embeddings. To use llama.cpp embeddings with the published heads, rescale: heads expect vectors with norm ~300–600 — simplest correct path is the HF pipeline (see the model repo README) or multiply the normalized vector back by the raw norm for that text computed in HF.

Kind	Head	Output
noul	heads["noul"] Linear(768→2)	softmax → [P(no), P(yes)]
choice	heads["choice"] Linear(768→16)	softmax over first len(options) logits
score	heads["score"] Linear(768→6)	softmax over 0–5 rubric

Training dataset

GGUF exports of SargeDev/Solar-JEV-Gemma-embed-2, which was trained on SargeDev/solar-decisions-corpus-v4 and, via the v3 lineage, SargeDev/jev-distill-corpus-v3.

License

Apache-2.0. Base encoder: Google Gemma terms for the upstream weights.