Instructions to use infosave/Bonsai-8B_2bit_cmf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- cortiq
How to use infosave/Bonsai-8B_2bit_cmf with cortiq:
# one Rust binary, no additional dependencies cargo install cortiq-cli # or a prebuilt binary from github.com/infosave2007/cmf/releases hf download infosave/Bonsai-8B_2bit_cmf --include "*.cmf" --local-dir . ls *.cmf # some repos ship more than one quantization
cortiq run FILE.cmf --prompt "What is the capital of France?"
cortiq serve FILE.cmf --port 8080 # OpenAI-compatible server
- Notebooks
- Google Colab
- Kaggle
card: name the library that reads this file
Browse files
README.md
CHANGED
|
@@ -1,7 +1,9 @@
|
|
| 1 |
---
|
|
|
|
| 2 |
license: apache-2.0
|
| 3 |
base_model: prism-ml/Bonsai-8B
|
| 4 |
tags:
|
|
|
|
| 5 |
- cmf
|
| 6 |
- 2-bit
|
| 7 |
- 1-bit
|
|
|
|
| 1 |
---
|
| 2 |
+
library_name: cortiq
|
| 3 |
license: apache-2.0
|
| 4 |
base_model: prism-ml/Bonsai-8B
|
| 5 |
tags:
|
| 6 |
+
- cortiq
|
| 7 |
- cmf
|
| 8 |
- 2-bit
|
| 9 |
- 1-bit
|