Missing lm head weights?
#1
by maybeMayank - opened
I tried running the inference code, but got following error:
Materializing param=gpt.transformer.wte.weight
BaseGPTForCausalLM LOAD REPORT from: armenjeddi/NanoGPT-Base24-FineWeb300K
Key | Status |
-------------------+---------+-
gpt.lm_head.weight | MISSING |
Notes:
- MISSING :those params were newly initialized because missing from the checkpoint. Consider training on your downstream task.