| --- |
| license: llama3.1 |
| --- |
| More updates. Going to be attempting a imatrix quant at Q3 so hopefully if that goes well and runs right on latest master will upload asap! |
|
|
| Not uploading Q6. This Quant may not work with your current llama.cpp build as there were breaking changes in the latest master. |
|
|
| Q4_K_M IS UP! |
|
|
| For those of you who do not know how to reassamble the parts `cat Meta-Llama-3.1-405B-Instruct-Q4_K_M.gguf.part_* > Meta-Llama-3.1-405B-Instruct-Q4_K_M.gguf` |
|
|
| Am amusing generation to kill the time. Is the instruct tuned to think it can ONLY be run on the cloud? |
|
|
|  |
|
|
|  |
|
|
| Why download these GGUF's? I made some modifications to the llama.cpp quantification process to for SURE use the right tokenizer vs the Smaug BPE GGUF's that are out now. |
