Instructions to use baa-ai/Llama-3.1-70B-Instruct-SWAN-5bit-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use baa-ai/Llama-3.1-70B-Instruct-SWAN-5bit-MLX with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir Llama-3.1-70B-Instruct-SWAN-5bit-MLX baa-ai/Llama-3.1-70B-Instruct-SWAN-5bit-MLX
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Trevor Kennedy
SWAN adaptive mixed-precision quantization (5.51 avg bits, MAD bounds)
abf7841 verified - Xet hash:
- 2e56cad9ac2a38f4b7de211953881b613c19e51838c05adf8f7984c139c2e8f5
- Size of remote file:
- 17.2 MB
- SHA256:
- 6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
·
Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.