Instructions to use Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained("Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4

SGLang

How to use Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4 with Docker Model Runner:
```
docker model run hf.co/Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4
```

DiffusionGemma 26B E38 — Abliterated NVFP4

A deployment-oriented NVFP4 release of E38 DiffusionGemma 26B, optimized for fast complete-response generation on NVIDIA Blackwell.

E38 NVFP4 reduces the E38 BF16 checkpoint from:

51.68 GB → 18.86 GB

while preserving the measured E38 capability profile and enabling native packed NVFP4 inference.

This release focuses on:

fast complete-response generation
native NVFP4 execution
Blackwell optimization
short-response and voice-assistant use
multimodal capability retention
memory efficiency
high serving throughput

Hands-on use also exposed a practical weakness in DiffusionGemma:

occasional grammar, lexical, repetition, and finalization errors

Controlled testing later showed that the same broad error class also exists in the original Base DiffusionGemma.

Current research is testing whether those errors can be corrected at inference time with minimal additional latency.

The experimental grammar-recovery system described below is NOT included in this downloadable checkpoint.

If the work continues to validate, it may become a separate E38 finalization / grammar-recovery branch.

Highlights

Metric	Result
Checkpoint size	18.86 GB
BF16 parent size	51.68 GB
Size reduction	63.5%
Relative size	2.74× smaller
48-step Quality-Max	660.44 tok/s
16-step single stream	827.28 tok/s
Concurrency-8 aggregate	1,053.64 tok/s
Recommended short mode	128 canvas / 16-step cap
Recommended short range	1–64 output tokens
Aligned NVFP4 validation	137/200 — 68.5%
Target refusals	0/402
Benign false refusals	0/249
Matched multimodal validation	20/20
Matched multi-turn generations	24/24

Recommended Inference Modes

Instant / Short Responses

canvas_length = 128
max_denoising_steps = 16

t_max = 0.80
t_min = 0.40
entropy_bound = 0.1
confidence_threshold = 0.005
stability_threshold = 1
adaptive_stopping = true

Recommended for:

short conversational replies
voice assistants
Instant Full-Text TTS
acknowledgements
short factual answers
concise explanations

Recommended output range:

approximately 1–64 tokens

Quality / General Responses

canvas_length = 256
max_denoising_steps = 48

t_max = 0.80
t_min = 0.40
entropy_bound = 0.1
confidence_threshold = 0.005
stability_threshold = 1
adaptive_stopping = true
long_form_capacity = 1280

Recommended for:

general chat
longer responses
reasoning
coding
detailed explanations
creative writing
long-form generation

Short-Response Validation

A large matched study compared 128/16 with 256/48 using:

500 frozen prompts
15 voice-assistant categories
4 matched seeds
2,000 matched pairs
4,000 total generations

Blind manual review completed:

1,850 matched A/B pairs

representing:

3,700 manually reviewed responses

Within the intended 1–64-token range:

Result	128/16	256/48
In-range responses reviewed	1,705	1,690
Strictly clean responses	1,678	1,638
Strictly clean rate	98.42%	96.92%
Confirmed language issues	27	52
Severe degeneration	0	0
Truncation	0	0

Among 1,633 fully paired in-range responses:

128/16: 98.65% strictly clean

256/48: 97.00% strictly clean

Difference:

+1.65 percentage points in favor of 128/16

Paired bootstrap 95% CI:

approximately +0.67 to +2.63 percentage points

Exact McNemar test:

p ≈ 0.0016

This was a separate short-response validation and should not be confused with the newer grammar-recovery experiment described below.

Current recommendation:

Use Case	Recommended Mode
Short complete responses / voice / 1–64 tokens	128/16
Longer responses / reasoning / general chat	256/48

128/16 is not currently recommended as the general long-form mode.

Grammar and Lexical Reliability

Hands-on testing made occasional language-finalization problems noticeable during normal use.

Observed issues include:

grammar mistakes
malformed words
duplicated words
agreement errors
broken sentence endings
lexical artifacts
rare local degeneration

A controlled 432-generation language study compared:

Base DiffusionGemma BF16
E38 BF16
E38 NVFP4

Model	Grammar Errors /10k ↓	Lexical Artifacts /10k ↓
Base BF16	4.059	2.243
E38 BF16	5.479	1.865
E38 NVFP4	3.236	1.387

The important finding is not that E38 was proven better or worse than Base.

It is that:

the same broad class of grammar and lexical errors also exists in the original Base DiffusionGemma.

The measured E38 BF16 grammar difference versus Base was not statistically established:

difference: +1.420 errors /10k words
95% CI: −0.674 to +3.541
McNemar p: 0.560

The evidence therefore does not support a simple conclusion that E38 or NVFP4 introduced the grammar problem.

Diffusion-Finalization Findings

Error analysis found that language failures were approximately:

3.01× more concentrated in the final quarter of the 256-token canvas

than in the first quarter.

Error-prone positions also showed approximately:

+0.0504 higher final entropy
−0.0418 lower top-1 / top-2 confidence margin

relative to matched clean positions.

Simply increasing the denoising budget did not reliably remove the problem.

For short responses, 128/16 actually performed better than 256/48 in blind testing.

For longer responses, however, reducing the diffusion budget too far increased repetition and degeneration.

The current working hypothesis is:

DiffusionGemma may sometimes contain the correct language capability but fail to cleanly finalize it during the original diffusion trajectory.

This remains a research hypothesis and is not claimed as a proven causal mechanism.

Current Grammar-Recovery Research

Experimental — Not Included in This Model

The following work is being developed separately from the currently downloadable E38 NVFP4 checkpoint.

The current E38 NVFP4 model does not include this proofreading/finalization system.

No model weights are changed.

The experimental inference path is intentionally simple:

Normal E38 generation
        ↓
Complete response
        ↓
One quick proofreading canvas
        ↓
Final response

The review pass is not asked to answer the original prompt again.

It only checks the completed response for obvious:

grammar mistakes
malformed wording
repetition
agreement errors
broken sentence construction
malformed endings
local finalization corruption

The objective is to correct these issues while adding as little latency as possible.

Current Experimental Testing

The latest grammar-recovery study used:

250 natural short responses

plus:

25 semantic-preservation stress cases

Results:

generation/review failures: 0
one-canvas reviews: 250/250 — 100%
maximum reviewed length: 139 tokens

Quality

Metric	Original 128/16	Experimental Review
Strictly clean	94.0%	99.2%
Grammar error	3.6%	0%
Lexical error	0.8%	0%
Repetition	0.4%	0%

99.2% is a current experimental testing result.

It is not a quality claim for the E38 NVFP4 checkpoint available on this page.

Among 15 original responses containing confirmed language errors:

13/15 — 86.7% were fully repaired

Total improvement:

13/15 — 86.7%

Paired clean-rate improvement:

+5.2 percentage points

95% CI:

+2.8 to +8.0 pp

McNemar exact:

p = 0.000244

Clean-Response Preservation

Among 235 original responses already judged clean:

96.2% were exactly unchanged
1.3% received harmless equivalent wording changes
2.6% received useful polish
0% unnecessary rewrite
0% minor downgrade
0% language damage
0% semantic/factual drift

Observed quality-preservation rate:

235/235 — 100%

Observed true-downgrade rate:

0%

Natural-set semantic drift:

0/250 observed

Separate semantic-stress result:

25/25 exactly preserved

These results are encouraging, but the current study used one primary adjudicator and broader independent evaluation is still warranted.

Blind Comparison

Across the 250 natural responses:

review wins: 19
ties: 229
original wins: 0
both bad: 2

Exact sign-test:

p = 3.81e-6

Again, these results belong to the experimental review system, not the currently released model.

Current Latency Cost

The quality improvement is not free.

Median additional review latency:

249.9 ms

P90:

326.4 ms

Median actual review iterations:

3

The established generation-only short-response reference is approximately:

139.2 ms median

The current production-reference estimate for generation plus review is approximately:

389.1 ms median

So the experimental review currently adds approximately:

+250 ms median

for the observed improvement from:

94.0% → 99.2% strictly clean

Current Optimization Goal

The goal now is not simply to prove that proofreading works.

It is:

correct grammar and lexical finalization errors with the smallest possible latency increase

Current work is focused on:

integrating review directly into the inference runtime
reusing tokenizer state
reusing GPU-resident tensors and buffers
reusing compiled execution paths
reusing CUDA graphs where possible
reducing review setup overhead
reducing review canvas size where safe
reducing review iterations where quality allows
improving adaptive stopping
conditionally reviewing only responses likely to contain an error

Because most responses are already clean, conditional review may provide the largest reduction in average added latency.

A possible future architecture is:

Normal generation
      ↓
Cheap confidence / anomaly check
      ├── likely clean → return response
      │
      └── suspicious
             ↓
        quick review canvas
             ↓
        final response

The target is to preserve as much of the current experimental 99.2% strictly-clean result as possible while reducing the current ~250 ms review overhead.

Possible E38 Finalization Branch

If integration, latency optimization, and further validation continue to succeed, this work may become a separate E38 inference branch.

Conceptually:

E38 NVFP4
│
├── Current release
│   └── normal inference
│
└── Possible E38 Finalization branch
    └── generation
          ↓
        quick review
          ↓
        final response

Important:

The current downloadable E38 NVFP4 model does not contain this review system.

No separate grammar-recovery checkpoint has been released yet.

This work currently changes the inference path, not the E38 weights.

Performance

Measured on one:

NVIDIA RTX PRO 6000 Blackwell Workstation Edition

Operating Point	Throughput
48-step Quality-Max	660.44 tok/s
16-step single stream	827.28 tok/s
16-step concurrency-8 aggregate	1,053.64 tok/s

Runtime stack:

native packed NVFP4 routed experts
vLLM V2 / 0.27.1
compiled execution
CUDA graphs
FULL_AND_PIECEWISE
FlashInfer CUTLASS NVFP4 MoE
FlashInfer autotuning
Triton attention
FP8 e4m3 KV
CUDA 13
SM120

Important:

1,053.64 tok/s is concurrency-8 aggregate serving throughput.

It is not single-stream throughput.

NVFP4 Quality Preservation

Aligned validation:

Evaluation	Base BF16	E38 BF16	E38 NVFP4
Objective — 200 prompts	128/200 — 64.0%	134/200 — 67.0%	137/200 — 68.5%
Regression subset — 100	62/100	58/100	64/100
Multimodal	20/20	20/20	20/20
Multi-turn generations	24/24	24/24	24/24
Target refusal	383/402	0/402	0/402
Benign false refusal	0/249	0/249	0/249

E38 BF16 versus E38 NVFP4:

67.0% → 68.5%

Difference:

+1.5 percentage points

95% CI:

−2.0 to +5.0 pp

p-value:

0.5811

The difference was not statistically significant.

NVFP4 preserved measured E38 BF16 quality.

The higher NVFP4 point estimate is not claimed as evidence that quantization inherently improved capability.

Public Benchmark Reference

The larger public benchmark suite was run on Base BF16 and E38 BF16.

Benchmark	Base BF16	E38 BF16	Delta
IFEval	67.10%	64.70%	−2.40 pp
BBH	71.62%	73.96%	+2.34 pp
MuSR	41.80%	50.00%	+8.20 pp
MMLU-Pro	49.61%	51.57%	+1.96 pp
MATH Level 5	84.06%	80.51%	−3.55 pp

E38 is best described as:

a capability redistribution rather than a universally stronger checkpoint

The complete public benchmark suite has not been rerun directly on NVFP4.

MATH Level 5

Evaluation size:

1,324 problems

Model	Correct
Base BF16	1,113 / 1,324 — 84.06%
E38 BF16	1,066 / 1,324 — 80.51%

Difference:

−3.55 percentage points

Statistics:

95% CI: −5.59 to −1.44 pp
McNemar p: 0.00119

This is a real measured E38 BF16 regression.

The full benchmark has not been rerun directly on E38 NVFP4.

Memory Efficiency

Model	Checkpoint Size
E38 BF16	51.68 GB
E38 NVFP4	18.86 GB

Reduction:

32.82 GB

Percentage reduction:

63.5%

Relative size:

2.74× smaller

Precision Layout

NVFP4

Routed expert weights:

NVFP4 W4A4
group size 16

BF16 Retained

attention
dense non-expert MLP
routers
embeddings
LM head
vision components
all 20 E38-modified tensors

Runtime KV

FP8 e4m3

All 20 E38-modified tensors remain exact BF16.

Native NVFP4 Execution

Runtime validation confirmed native packed NVFP4 execution.

No repeated:

BF16 routed-expert expansion
FP4 expert repacking
hidden full-precision expert conversion

was found in the hot inference path.

The checkpoint therefore provides real low-precision runtime execution rather than only reduced storage size.

E38 Architecture

E38 was selected from more than:

80 controlled candidate configurations

derived from:

google/diffusiongemma-26B-A4B-it

E38 modifies:

language layers 7–16
attn.o_proj
mlp.down_proj
20 language tensors
0 vision tensors

Measured modification magnitude:

relative Frobenius difference: 0.0250308802854
maximum absolute difference: 0.201904296875

E38 overlay SHA256:

9fe83b2fa2a3b7643f6dcc4a8360feb2cd96dcb9f70d215bd2e8f8f9d9d3044b

Frozen E38 selection hash:

cd69540ec547f2f5cf181d527e8366fe60b49df7cafde38b434c5e42eeeb6175

Current Status

Area	Status
E38 BF16	Complete
E38 NVFP4	Validated
NVFP4 integrity	PASS
Native packed NVFP4 inference	Validated
128/16 short mode	Validated
256/48 general mode	Validated
Grammar/finalization issue	Active research target
Same broad error class in Base	Observed
Experimental review result	Current testing: 99.2% strictly clean
Experimental full-fix rate	13/15 — 86.7%
Experimental natural semantic drift	0/250 observed
Experimental semantic stress	25/25 preserved
Experimental one-canvas rate	250/250
Experimental added review latency	249.9 ms median
Grammar-recovery system in this model	No
Possible separate E38 finalization branch	Under consideration
Review latency optimization	In progress

Known Limitations

This model remains experimental.

E38 intentionally has substantially reduced refusal behavior relative to upstream.
E38 BF16 showed a statistically significant MATH Level 5 regression.
The complete public benchmark suite has not been rerun directly on NVFP4.
Diffusion inference remains sensitive to canvas size and denoising configuration.
Occasional grammar and lexical errors still occur in the currently released model.
The diffusion-finalization explanation remains a working hypothesis.
The 99.2% strictly-clean result is from current experimental review testing, not the downloadable model by itself.
The experimental review/finalization path is not included in this checkpoint.
The current expanded review study used one primary adjudicator.
The experimental review currently adds approximately 250 ms median latency.
The review path is currently being integrated and optimized.
A separate E38 finalization / grammar-recovery branch may be created if further testing continues to succeed.
128/16 is intended for short complete responses and is not recommended as the general long-form mode.

Safety and Behavior

E38 intentionally retains substantially reduced refusal behavior.

Model	Target Refusal	Prompt-Majority Refusal	Benign False Refusal
Base BF16	383/402 — 95.27%	127/134 — 94.78%	0/249
E38 BF16	0/402	0/134	0/249
E38 NVFP4	0/402	0/134	0/249

This checkpoint should not be interpreted as preserving the original refusal behavior of the upstream model.

Users should evaluate safeguards appropriate for their intended application.

Reproducibility

Original Upstream

google/diffusiongemma-26B-A4B-it

Pinned revision:

f7f5b7f5fa82ffc52addd066915886d497f5517b

E38 BF16 Parent

Goodoldjam/DiffusionGemma-26B-E38-Abliterated-BF16

E38 Overlay SHA256

9fe83b2fa2a3b7643f6dcc4a8360feb2cd96dcb9f70d215bd2e8f8f9d9d3044b

Frozen E38 Selection Hash

cd69540ec547f2f5cf181d527e8366fe60b49df7cafde38b434c5e42eeeb6175

Integrity

Validated:

12/12 artifacts PASS
E38 overlay hash PASS
all 20 E38-modified tensors remain BF16
quantization layout unchanged
vision tensors unchanged
protected BF16 paths unchanged

Intended Use

This checkpoint is intended for experimentation with:

low-latency local chat
short complete-response generation
Instant Full-Text TTS
voice assistants
diffusion-language-model research
diffusion-budget optimization
NVFP4 deployment
NVIDIA Blackwell inference
multimodal experimentation
quantization research
abliteration research
serving-throughput research
memory-efficiency research

Grammar/finalization research is ongoing, but the experimental review system is not part of this checkpoint.

Support

If this model or the research is useful, support helps fund:

GPU compute
controlled evaluation
grammar and lexical testing
manual review
inference optimization
model hosting
continued public releases

The goal is to report both positive and negative results and avoid presenting experimental findings as released-model capabilities until they survive further validation.

License

Apache License 2.0.

This model is a quantized derivative of:

Goodoldjam/DiffusionGemma-26B-E38-Abliterated-BF16

which is derived from:

google/diffusiongemma-26B-A4B-it

Downloads last month: 625

Safetensors

Model size

14B params

Tensor type

BF16

Model tree for Goodoldjam/DiffusionGemma-26B-E38-Abliterated-NVFP4

Base model

google/diffusiongemma-26B-A4B-it

Finetuned

Goodoldjam/DiffusionGemma-26B-E38-Abliterated-BF16

Quantized

(1)

this model