Caelum-G4-38B-A12.5B-GGUF

CPU/RAM quantizations of the four-expert Caelum-G4-38B-A12.5B sparse MoE.

Caelum contains 38.01B total parameters and activates approximately 12.53B per token through top-1 routing. Sparse activation can reduce compute, but all four experts remain in the GGUF file and contribute to storage and memory-mapping requirements.

Which file should I download?

Quantization	Positioning	Validation state
Q4_K_M	Recommended default	23.16 GB; llama.cpp smoke and complete AirGapBench v1 matrix passed
Q5_K_M	Recommended quality option	27.04 GB; llama.cpp functional smoke passed
Q6_K	Higher fidelity	Planned; publish only after smoke and regression tests
Q8_0	Near-lossless reference	Planned; large and usually unnecessary for ordinary use
Q3_K_M	Low-RAM compromise	Planned; must pass quality and routing-regression tests
IQ2_M	Smallest experimental release	Planned; highest quality risk and not the recommended first download

Q4_K_M is the sensible headline quant. Keep IQ2_M available for machines that otherwise could not run the model, but label it experimental rather than promoting it as equivalent quality.

llama.cpp

Use a recent llama.cpp build with Gemma 4 Unified MoE support.

llama-cli \
  -hf EryriLabs/Caelum-G4-38B-A12.5B-GGUF:Q4_K_M \
  -cnv \
  -c 2048 \
  -n 512 \
  --temp 0

OpenAI-compatible local server:

llama-server \
  -hf EryriLabs/Caelum-G4-38B-A12.5B-GGUF:Q4_K_M \
  -c 2048 \
  --host 127.0.0.1 \
  --port 8080

The validated artifacts used stock llama.cpp commit:

69bf6437914596fbbc4caf09a7ac16f2acdd1a94

Measured internal result

The Q4_K_M build scored 20/28 (71.43%) on frozen AirGapBench v1, compared with 18/28 (64.29%) for the strongest dense donor and 17/28 (60.71%) for the official Gemma 4 12B IT base. All models used Q4_K_M, CPU-only llama.cpp, context 2,048, temperature 0, a 256-token output limit, and thinking disabled.

This is a small internal selection suite, not an external leaderboard. See the main model card for the complete category matrix, source lineage, limitations, and reproducibility details.

Quantization policy

• Quantize every file directly from the 75,931,245,984-byte F16 GGUF.
• Never requantize an existing Q4/Q5/etc. file.
• Prefer an importance matrix that exercises all four expert domains.
• Run the same functional smoke and AirGapBench v1 suite on every public quant.
• Check routing/expert utilization as well as aggregate quality; low-bit quantization can alter router decisions.
• Publish exact file sizes, SHA-256 hashes, llama.cpp commit, command lines, memory, and speed after measurement.

Current runtime evidence

• Q4_K_M file: 23,157,726,624 bytes.
• Q5_K_M file: 27,036,802,464 bytes.
• Both loaded and produced the expected arithmetic completion in stock llama.cpp.
• A short Q4 smoke measured 10.82 prompt tokens/s and 5.01 generation tokens/s; a short Q5 smoke measured 3.68 and 4.33 respectively. These tiny samples are functional checks, not release throughput claims.
• Q4 reached a 17.2 GiB process working set with --no-repack, one 256-token slot, and prompts spanning the four routing domains. A physical 32 GB machine has not yet been validated.

Limitations

• Only Q4_K_M has completed the current 28-case quantized benchmark matrix.
• Expected quality ranking is not a substitute for testing this architecture.
• Longer contexts increase KV-cache and working-memory requirements.
• The combined llama.cpp vision/audio projector is currently blocked during audio frontend initialization, so this repository is text-only for the validated release path.
• This model can produce incorrect, biased, unsafe, or fabricated output. Tool calls and consequential actions require validation and human oversight.

License

Apache-2.0, subject to the licenses and notices of the main Caelum model and all its source checkpoints. Caelum is an unofficial derivative and is not endorsed by Google, the Gemma team, or the donor authors.