Instructions to use mlx-community/Llama-2-7b-chat-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/Llama-2-7b-chat-mlx with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # if on a CUDA device, also pip install mlx[cuda] # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("mlx-community/Llama-2-7b-chat-mlx") prompt = "Once upon a time in" text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use mlx-community/Llama-2-7b-chat-mlx with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Generate some text mlx_lm.generate --model "mlx-community/Llama-2-7b-chat-mlx" --prompt "Once upon a time"
- Atomic Chat
update to new format
Browse files- README.md +1 -1
- params.json +1 -0
- Llama-2-7b-chat.npz → weights.npz +2 -2
README.md
CHANGED
|
@@ -28,7 +28,7 @@ export HF_HUB_ENABLE_HF_TRANSFER=1
|
|
| 28 |
huggingface-cli download --local-dir Llama-2-7b-chat-mlx mlx-llama/Llama-2-7b-chat-mlx
|
| 29 |
|
| 30 |
# Run example
|
| 31 |
-
python mlx-examples/llama/llama.py Llama-2-7b-chat-mlx/
|
| 32 |
```
|
| 33 |
|
| 34 |
Please, refer to the [original model card](https://huggingface.co/meta-llama/Llama-2-7b-chat) for details on Llama 2.
|
|
|
|
| 28 |
huggingface-cli download --local-dir Llama-2-7b-chat-mlx mlx-llama/Llama-2-7b-chat-mlx
|
| 29 |
|
| 30 |
# Run example
|
| 31 |
+
python mlx-examples/llama/llama.py Llama-2-7b-chat-mlx/ Llama-2-7b-chat-mlx/tokenizer.model "My name is "
|
| 32 |
```
|
| 33 |
|
| 34 |
Please, refer to the [original model card](https://huggingface.co/meta-llama/Llama-2-7b-chat) for details on Llama 2.
|
params.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"dim": 4096, "multiple_of": 256, "n_heads": 32, "n_layers": 32, "norm_eps": 1e-05, "vocab_size": -1}
|
Llama-2-7b-chat.npz → weights.npz
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:68122068194be1051286a36314577f8b1910e4d36c8f280f3852bff9609d90de
|
| 3 |
+
size 13476919308
|