params.json missing
I downloaded the model form "https://huggingface.co/mlx-llama" and ran python llama.py <path_to_tokenizer.model> "hello". I get params.json missing error, and in the weights the "layers.0.feed_forward.w1.weight" and "output.weight" keys are missing.
Hello @Farzamfan !
The original inference script llama.py has changed in a number of ways:
- The weights filename is now hardcoded.
- It expects a configuration file as you saw.
- Some configurations are taken from the weights themselves.
We need to re-export these models accordingly. Meanwhile, can you please use revision 0bf5d0e3bc1e51b82cf8083be422dc61dc88455b of https://github.com/ml-explore/mlx-examples.git? That revision runs inference fine using this model and the instructions in the model card.
Btw, the HF models should be updated to work with the latest mlx-examples. If you still have issues let me know and/or file an issue in https://github.com/ml-explore/mlx-examples
Thanks a lot @awni , I saw you updated this repo a couple days ago 🙌. I just changed the example to use the --prompt option, so the snippet now works with the latest main. Could you please verify if that's the case for you as well @Farzamfan ?
I'll now go through the other repos and verify whether they work.