GGUF for using it in common inference engines

#4
by christianweyer - opened

Hi,

thanks for this model @dousery !

Do you think you could create a GGUF from it so that we can run it on the many available inference engines and tools?
See https://docs.unsloth.ai/models/gpt-oss-how-to-run-and-fine-tune#new-saving-to-gguf-vllm-after-gpt-oss-training

Thanks!

I think I can do it here on my side if you would provide the LORA adapters.

I opened a new issue with more details. Closing this.
https://huggingface.co/dousery/medical-reasoning-gpt-oss-20b/discussions/5

christianweyer changed discussion status to closed

Sign up or log in to comment