Update README.md

README.md

@@ -93,6 +93,81 @@ The
 
 =================================================================
 
+```
+
+```python
+# Run once, then RESTART RUNTIME, then run this cell:
+#   !pip install -q "compressed-tensors>=0.15.0"
+
+import warnings, logging, os
+warnings.filterwarnings("ignore")
+logging.getLogger("transformers").setLevel(logging.ERROR)
+os.environ["TRANSFORMERS_VERBOSITY"] = "error"
+os.environ["TRANSFORMERS_NO_ADVISORY_WARNINGS"] = "1"
+try:
+    from transformers.utils import logging as _hf_log
+    _hf_log.set_verbosity_error()
+except Exception:
+    pass
+
+import transformers.utils.import_utils as _iu
+import transformers.utils as _tu
+for _m in (_iu, _tu):
+    if not hasattr(_m, "is_torch_fx_available"):
+        _m.is_torch_fx_available = lambda: False
+
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import transformers.modeling_utils as _mu
+
+# FP8 fix: skip re-init of already-loaded FP8 weights.
+_mu.PreTrainedModel._initialize_weights = lambda self, *a, **k: None
+_mu.PreTrainedModel.initialize_weights  = lambda self, *a, **k: None
+
+REPO = "frankmorales2020/deepseek-v2-lite-fp8-topo2026"
+PROMPTS = ["What is the capital of Japan?",
+           "What is the Spanish word for water?",
+           "What is Einstein's mass-energy equivalence?"]
+
+try:
+    tok = AutoTokenizer.from_pretrained(REPO, trust_remote_code=True)
+except Exception:
+    tok = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V2-Lite", trust_remote_code=True)
+if tok.pad_token is None:
+    tok.pad_token = tok.eos_token
+
+model = AutoModelForCausalLM.from_pretrained(
+    REPO, dtype=torch.float16, device_map="auto",
+    trust_remote_code=True, attn_implementation="eager").eval()
+
+nan = torch.isnan(model.get_input_embeddings().weight.float()).any().item()
+print("Embedding has NaN?", nan)
+
+if not nan:
+    for p in PROMPTS:
+        ins = tok(p, return_tensors="pt").to(model.device)
+        out = model.generate(**ins, max_new_tokens=40, do_sample=False,
+                             repetition_penalty=1.2, no_repeat_ngram_size=3,
+                             use_cache=False,
+                             pad_token_id=tok.pad_token_id)
+        txt = tok.decode(out[0], skip_special_tokens=True)
+        print(f"\nQ: {p}\nA: {txt[len(p):].strip() if txt.startswith(p) else txt.strip()}")
+
+```
+OUTPUT-EXPECTED 
+
+```text
+Compressing model: 100%|██████████| 3463/3463 [00:01<00:00, 1977.29it/s]
+Loading weights: 100% 12217/12217 [00:01<00:00, 8114.97it/s]Embedding has NaN? False
+Decompressing model: 100%|██████████| 3463/3463 [00:00<00:00, 13103.20it/s]
+
+Q: What is the capital of Japan?
+A: The answer to this question may seem obvious, but it’s actually a bit more complicated than you might think. The country has two capitals: Tokyo and Kyoto. Both cities are important in Japanese
+
+Q: What is the Spanish word for water?
+A: The English translation of “agua” in a sentence. Agua means Water, and it’s pronounced like this: ah-gwah (like how you say ‘gwa’
 
+Q: What is Einstein's mass-energy equivalence?
+A: Einstein’s Mass Energy Equivalence states that the energy of a body at rest (E) equals its mass times the speed of light squared. This equation can be written as: E = mc
 
 ```