Instructions to use baa-ai/Llama-3.1-70B-Instruct-SWAN-5bit-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use baa-ai/Llama-3.1-70B-Instruct-SWAN-5bit-MLX with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir Llama-3.1-70B-Instruct-SWAN-5bit-MLX baa-ai/Llama-3.1-70B-Instruct-SWAN-5bit-MLX
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Commit History
Fix: normalise line endings (CR removal) in README eb52b3e verified
Upload README.md with huggingface_hub 805b502 verified
Upload README.md with huggingface_hub cbfbdff verified
Update model card: remove MINT/SWAN branding, optimised by baa.ai e892cc1 verified
Update model card: use generic language for allocation method 76932f9 verified
Update model card: add MINT-UI link and custom quantization guide fcf96d8 verified
Update base_model format to include relation: quantized 51c1684 verified
Add model card with metrics and usage 0c3be1b verified
Upload README.md with huggingface_hub 972fb37 verified
Trevor Kennedy commited on
Upload README.md with huggingface_hub cf0ae25 verified
Trevor Kennedy commited on
SWAN adaptive mixed-precision quantization (5.51 avg bits, MAD bounds) abf7841 verified
Trevor Kennedy commited on