Instructions to use ajibawa-2023/carl-33b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ajibawa-2023/carl-33b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ajibawa-2023/carl-33b")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("ajibawa-2023/carl-33b") model = AutoModelForCausalLM.from_pretrained("ajibawa-2023/carl-33b", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ajibawa-2023/carl-33b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ajibawa-2023/carl-33b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ajibawa-2023/carl-33b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/ajibawa-2023/carl-33b
- SGLang
How to use ajibawa-2023/carl-33b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ajibawa-2023/carl-33b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ajibawa-2023/carl-33b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ajibawa-2023/carl-33b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ajibawa-2023/carl-33b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use ajibawa-2023/carl-33b with Docker Model Runner:
docker model run hf.co/ajibawa-2023/carl-33b
Commit ·
602da03
1
Parent(s): 8468207
Upload folder using huggingface_hub
Browse files- .gitattributes +14 -0
- config.json +24 -0
- generation_config.json +7 -0
- pytorch_model.bin-aa +3 -0
- pytorch_model.bin-ab +3 -0
- pytorch_model.bin-ac +3 -0
- pytorch_model.bin-ad +3 -0
- pytorch_model.bin-ae +3 -0
- pytorch_model.bin-af +3 -0
- pytorch_model.bin-ag +3 -0
- pytorch_model.bin-ah +3 -0
- pytorch_model.bin-ai +3 -0
- pytorch_model.bin-aj +3 -0
- pytorch_model.bin-ak +3 -0
- pytorch_model.bin-al +3 -0
- pytorch_model.bin-am +3 -0
- pytorch_model.bin-an +3 -0
- special_tokens_map.json +24 -0
- tokenizer.model +3 -0
- tokenizer_config.json +34 -0
- trainer_state.json +0 -0
- training_args.bin +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,17 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
pytorch_model.bin-aa filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
pytorch_model.bin-ab filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
pytorch_model.bin-ac filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
pytorch_model.bin-ad filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
pytorch_model.bin-ae filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
pytorch_model.bin-af filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
pytorch_model.bin-ag filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
pytorch_model.bin-ah filter=lfs diff=lfs merge=lfs -text
|
| 44 |
+
pytorch_model.bin-ai filter=lfs diff=lfs merge=lfs -text
|
| 45 |
+
pytorch_model.bin-aj filter=lfs diff=lfs merge=lfs -text
|
| 46 |
+
pytorch_model.bin-ak filter=lfs diff=lfs merge=lfs -text
|
| 47 |
+
pytorch_model.bin-al filter=lfs diff=lfs merge=lfs -text
|
| 48 |
+
pytorch_model.bin-am filter=lfs diff=lfs merge=lfs -text
|
| 49 |
+
pytorch_model.bin-an filter=lfs diff=lfs merge=lfs -text
|
config.json
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_name_or_path": "/media/mla100eus/Projects_2/llama-30b",
|
| 3 |
+
"architectures": [
|
| 4 |
+
"LlamaForCausalLM"
|
| 5 |
+
],
|
| 6 |
+
"bos_token_id": 1,
|
| 7 |
+
"eos_token_id": 2,
|
| 8 |
+
"hidden_act": "silu",
|
| 9 |
+
"hidden_size": 6656,
|
| 10 |
+
"initializer_range": 0.02,
|
| 11 |
+
"intermediate_size": 17920,
|
| 12 |
+
"max_position_embeddings": 2048,
|
| 13 |
+
"max_sequence_length": 2048,
|
| 14 |
+
"model_type": "llama",
|
| 15 |
+
"num_attention_heads": 52,
|
| 16 |
+
"num_hidden_layers": 60,
|
| 17 |
+
"pad_token_id": 0,
|
| 18 |
+
"rms_norm_eps": 1e-06,
|
| 19 |
+
"tie_word_embeddings": false,
|
| 20 |
+
"torch_dtype": "bfloat16",
|
| 21 |
+
"transformers_version": "4.28.1",
|
| 22 |
+
"use_cache": false,
|
| 23 |
+
"vocab_size": 32000
|
| 24 |
+
}
|
generation_config.json
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_from_model_config": true,
|
| 3 |
+
"bos_token_id": 1,
|
| 4 |
+
"eos_token_id": 2,
|
| 5 |
+
"pad_token_id": 0,
|
| 6 |
+
"transformers_version": "4.28.1"
|
| 7 |
+
}
|
pytorch_model.bin-aa
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:31581c3a3bb4574995a36e8caa265c0462d93957e94de78d6575d4e9eafcbe80
|
| 3 |
+
size 9663676416
|
pytorch_model.bin-ab
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3727bddbc4d02dddc3079241cd1b9f7e9e53c61bb8bf8c385f497f7a39d56579
|
| 3 |
+
size 9663676416
|
pytorch_model.bin-ac
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2a53d5615d00eeab0af0084ab3923777b8818fa2c76c046a56d05e5c6490e982
|
| 3 |
+
size 9663676416
|
pytorch_model.bin-ad
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:671b55f86d08d865e6cfedf1b36c4e397a88ad3acbe58847a839e4f1a808ba49
|
| 3 |
+
size 9663676416
|
pytorch_model.bin-ae
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1a13c7363ebfaf4c01a9139a3bdad83968c7b1d06bacd24edbc76eb30f10cf4b
|
| 3 |
+
size 9663676416
|
pytorch_model.bin-af
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c1d8bdb5870de44bc35d297f246962f07d28bd47d66cd1d46a9cd0ef23dce868
|
| 3 |
+
size 9663676416
|
pytorch_model.bin-ag
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7748f5cc3f73a29ad6566309b1dc10f7bcc423be9f2e825850437719c5f0c588
|
| 3 |
+
size 9663676416
|
pytorch_model.bin-ah
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f80612770bd205e00c332883272de88396aba73d9b2c8f08bc83d22c2f2c6609
|
| 3 |
+
size 9663676416
|
pytorch_model.bin-ai
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8a679bfc1eb208e20c0696ced9c9ce74e033dcfd9e78cf321ace6b41869835d4
|
| 3 |
+
size 9663676416
|
pytorch_model.bin-aj
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5b14fdbe5dcca986dc5287146e804ab3d12ba22d3deb01f50e7edeade8fa109b
|
| 3 |
+
size 9663676416
|
pytorch_model.bin-ak
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7a8dd4950a6d7605ce275df0e2113a34b6dc6f6e6db49b3f2abc793521a46313
|
| 3 |
+
size 9663676416
|
pytorch_model.bin-al
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8bea7b87e4ebc62919bb482703974094175e502639d18a62646a2e2ff5008682
|
| 3 |
+
size 9663676416
|
pytorch_model.bin-am
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:965b3babe5aa76825d70dd1be48b2debb6e95da2a9716cd2a87c5f3456b9a921
|
| 3 |
+
size 9663676416
|
pytorch_model.bin-an
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d2565c8d24bdf675029a8232acce9fe7ad832f52edf6df5572f2d5385a06385b
|
| 3 |
+
size 4488198525
|
special_tokens_map.json
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"bos_token": {
|
| 3 |
+
"content": "<s>",
|
| 4 |
+
"lstrip": false,
|
| 5 |
+
"normalized": true,
|
| 6 |
+
"rstrip": false,
|
| 7 |
+
"single_word": false
|
| 8 |
+
},
|
| 9 |
+
"eos_token": {
|
| 10 |
+
"content": "</s>",
|
| 11 |
+
"lstrip": false,
|
| 12 |
+
"normalized": true,
|
| 13 |
+
"rstrip": false,
|
| 14 |
+
"single_word": false
|
| 15 |
+
},
|
| 16 |
+
"pad_token": "<unk>",
|
| 17 |
+
"unk_token": {
|
| 18 |
+
"content": "<unk>",
|
| 19 |
+
"lstrip": false,
|
| 20 |
+
"normalized": true,
|
| 21 |
+
"rstrip": false,
|
| 22 |
+
"single_word": false
|
| 23 |
+
}
|
| 24 |
+
}
|
tokenizer.model
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
|
| 3 |
+
size 499723
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_bos_token": true,
|
| 3 |
+
"add_eos_token": false,
|
| 4 |
+
"bos_token": {
|
| 5 |
+
"__type": "AddedToken",
|
| 6 |
+
"content": "<s>",
|
| 7 |
+
"lstrip": false,
|
| 8 |
+
"normalized": true,
|
| 9 |
+
"rstrip": false,
|
| 10 |
+
"single_word": false
|
| 11 |
+
},
|
| 12 |
+
"clean_up_tokenization_spaces": false,
|
| 13 |
+
"eos_token": {
|
| 14 |
+
"__type": "AddedToken",
|
| 15 |
+
"content": "</s>",
|
| 16 |
+
"lstrip": false,
|
| 17 |
+
"normalized": true,
|
| 18 |
+
"rstrip": false,
|
| 19 |
+
"single_word": false
|
| 20 |
+
},
|
| 21 |
+
"model_max_length": 2048,
|
| 22 |
+
"pad_token": null,
|
| 23 |
+
"padding_side": "right",
|
| 24 |
+
"sp_model_kwargs": {},
|
| 25 |
+
"tokenizer_class": "LlamaTokenizer",
|
| 26 |
+
"unk_token": {
|
| 27 |
+
"__type": "AddedToken",
|
| 28 |
+
"content": "<unk>",
|
| 29 |
+
"lstrip": false,
|
| 30 |
+
"normalized": true,
|
| 31 |
+
"rstrip": false,
|
| 32 |
+
"single_word": false
|
| 33 |
+
}
|
| 34 |
+
}
|
trainer_state.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
training_args.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d4de430603901a17e430abe49c30eda1021bc73fed19a06e3b090e159ce8799c
|
| 3 |
+
size 4731
|