Instructions to use jbenbudd/ADPrLlama with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jbenbudd/ADPrLlama with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jbenbudd/ADPrLlama")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("jbenbudd/ADPrLlama")
model = AutoModelForCausalLM.from_pretrained("jbenbudd/ADPrLlama", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

PEFT
How to use jbenbudd/ADPrLlama with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jbenbudd/ADPrLlama with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jbenbudd/ADPrLlama"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jbenbudd/ADPrLlama",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jbenbudd/ADPrLlama

SGLang

How to use jbenbudd/ADPrLlama with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jbenbudd/ADPrLlama" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jbenbudd/ADPrLlama",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jbenbudd/ADPrLlama" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jbenbudd/ADPrLlama",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use jbenbudd/ADPrLlama with Docker Model Runner:
```
docker model run hf.co/jbenbudd/ADPrLlama
```

Ben Buddendorff commited on Apr 17, 2025

Commit

d52da83

1 Parent(s): 020945a

train_residue_list_lr_3e-4_5_epochs_neg_1

Browse files

Files changed (11) hide show

README.md +19 -34
adapter_config.json +5 -5
adapter_model.safetensors +1 -1
all_results.json +11 -11
eval_results.json +6 -6
model_eval_results.csv +0 -0
trainer_log.jsonl +0 -0
trainer_state.json +0 -0
training_args.bin +1 -1
training_eval_loss.png +2 -2
training_loss.png +2 -2

README.md CHANGED Viewed

@@ -7,19 +7,19 @@ tags:
 - lora
 - generated_from_trainer
 model-index:
-- name: train_residue_list_lr_3e-4_5_epochs_neg_5
   results: []
 ---
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
-# train_residue_list_lr_3e-4_5_epochs_neg_5
 This model is a fine-tuned version of [GreatCaptainNemo/ProLLaMA_Stage_1](https://huggingface.co/GreatCaptainNemo/ProLLaMA_Stage_1) on the adpr_train dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.1675
-- Num Input Tokens Seen: 36312064
 ## Model description
@@ -53,36 +53,21 @@ The following hyperparameters were used during training:
 | Training Loss | Epoch  | Step | Validation Loss | Input Tokens Seen |
 |:-------------:|:------:|:----:|:---------------:|:-----------------:|
-| 0.314         | 0.1617 | 100  | 0.3099          | 1177472           |
-| 1.0912        | 0.3235 | 200  | 0.2833          | 2353792           |
-| 0.2867        | 0.4852 | 300  | 0.2665          | 3529856           |
-| 0.2527        | 0.6470 | 400  | 0.2551          | 4704512           |
-| 0.251         | 0.8087 | 500  | 0.2453          | 5881088           |
-| 0.2157        | 0.9705 | 600  | 0.2286          | 7056896           |
-| 0.2341        | 1.1310 | 700  | 0.2221          | 8223104           |
-| 0.2017        | 1.2928 | 800  | 0.2096          | 9403904           |
-| 0.1898        | 1.4545 | 900  | 0.2038          | 10580480          |
-| 0.2013        | 1.6163 | 1000 | 0.2014          | 11757056          |
-| 0.2059        | 1.7780 | 1100 | 0.1935          | 12932352          |
-| 0.1938        | 1.9397 | 1200 | 0.1979          | 14108928          |
-| 0.1803        | 2.1003 | 1300 | 0.1866          | 15276800          |
-| 0.1628        | 2.2620 | 1400 | 0.1774          | 16452224          |
-| 0.178         | 2.4238 | 1500 | 0.1787          | 17630080          |
-| 0.1751        | 2.5855 | 1600 | 0.1708          | 18805632          |
-| 0.1678        | 2.7473 | 1700 | 0.1674          | 19982720          |
-| 0.1581        | 2.9090 | 1800 | 0.1618          | 21154816          |
-| 0.1348        | 3.0696 | 1900 | 0.1682          | 22320640          |
-| 0.1192        | 3.2313 | 2000 | 0.1588          | 23493120          |
-| 0.1323        | 3.3930 | 2100 | 0.1559          | 24670976          |
-| 0.1263        | 3.5548 | 2200 | 0.1524          | 25849728          |
-| 0.1155        | 3.7165 | 2300 | 0.1491          | 27029120          |
-| 0.1162        | 3.8783 | 2400 | 0.1464          | 28204928          |
-| 0.0687        | 4.0388 | 2500 | 0.1594          | 29370240          |
-| 0.0695        | 4.2006 | 2600 | 0.1708          | 30547712          |
-| 0.0631        | 4.3623 | 2700 | 0.1689          | 31720576          |
-| 0.0614        | 4.5241 | 2800 | 0.1685          | 32897024          |
-| 0.0692        | 4.6858 | 2900 | 0.1681          | 34075520          |
-| 0.0651        | 4.8476 | 3000 | 0.1680          | 35252352          |
 ### Framework versions

 - lora
 - generated_from_trainer
 model-index:
+- name: train_residue_list_lr_3e-4_5_epochs_neg_1
   results: []
 ---
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
+# train_residue_list_lr_3e-4_5_epochs_neg_1
 This model is a fine-tuned version of [GreatCaptainNemo/ProLLaMA_Stage_1](https://huggingface.co/GreatCaptainNemo/ProLLaMA_Stage_1) on the adpr_train dataset.
 It achieves the following results on the evaluation set:
+- Loss: 0.2438
+- Num Input Tokens Seen: 18724952
 ## Model description
 | Training Loss | Epoch  | Step | Validation Loss | Input Tokens Seen |
 |:-------------:|:------:|:----:|:---------------:|:-----------------:|
+| 0.4277        | 0.3225 | 100  | 0.4275          | 1212416           |
+| 0.3824        | 0.6449 | 200  | 0.3592          | 2422016           |
+| 0.3314        | 0.9674 | 300  | 0.3160          | 3633408           |
+| 0.2962        | 1.2870 | 400  | 0.2967          | 4832344           |
+| 0.2829        | 1.6094 | 500  | 0.2788          | 6041304           |
+| 0.2524        | 1.9319 | 600  | 0.2397          | 7252056           |
+| 0.2185        | 2.2515 | 700  | 0.2294          | 8452392           |
+| 0.2162        | 2.5740 | 800  | 0.2240          | 9665576           |
+| 0.1905        | 2.8964 | 900  | 0.2098          | 10875816          |
+| 0.1481        | 3.2160 | 1000 | 0.2189          | 12076160          |
+| 0.15          | 3.5385 | 1100 | 0.2087          | 13288832          |
+| 0.1387        | 3.8609 | 1200 | 0.1994          | 14501632          |
+| 0.0735        | 4.1806 | 1300 | 0.2397          | 15699672          |
+| 0.0701        | 4.5030 | 1400 | 0.2403          | 16908632          |
+| 0.0681        | 4.8255 | 1500 | 0.2435          | 18117848          |
 ### Framework versions

adapter_config.json CHANGED Viewed

@@ -23,13 +23,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "o_proj",
-    "q_proj",
-    "gate_proj",
-    "up_proj",
     "k_proj",
     "v_proj",
-    "down_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "k_proj",
+    "gate_proj",
     "v_proj",
+    "o_proj",
+    "q_proj",
+    "down_proj",
+    "up_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ee1fd50f64de9fd8d7d8430bffb36a78e5e7892844e990686b97457a5e61c5a3
 size 639691872

 version https://git-lfs.github.com/spec/v1
+oid sha256:66eb45d88a880716d3f14ec5b62744622e04f5cff153b98ebd4c0ad10cc5b59e
 size 639691872

all_results.json CHANGED Viewed

@@ -1,13 +1,13 @@
 {
-    "epoch": 4.993125758188435,
-    "eval_loss": 0.16754263639450073,
-    "eval_runtime": 91.5328,
-    "eval_samples_per_second": 96.064,
-    "eval_steps_per_second": 6.009,
-    "num_input_tokens_seen": 36312064,
-    "total_flos": 1.474397207846191e+18,
-    "train_loss": 0.2081744998500571,
-    "train_runtime": 16798.8058,
-    "train_samples_per_second": 23.554,
-    "train_steps_per_second": 0.184
 }

 {
+    "epoch": 4.986698911729142,
+    "eval_loss": 0.243755504488945,
+    "eval_runtime": 46.4843,
+    "eval_samples_per_second": 94.871,
+    "eval_steps_per_second": 5.937,
+    "num_input_tokens_seen": 18724952,
+    "total_flos": 7.602987521131807e+17,
+    "train_loss": 0.2932651937200177,
+    "train_runtime": 7879.3287,
+    "train_samples_per_second": 25.18,
+    "train_steps_per_second": 0.197
 }

eval_results.json CHANGED Viewed

@@ -1,8 +1,8 @@
 {
-    "epoch": 4.993125758188435,
-    "eval_loss": 0.16754263639450073,
-    "eval_runtime": 91.5328,
-    "eval_samples_per_second": 96.064,
-    "eval_steps_per_second": 6.009,
-    "num_input_tokens_seen": 36312064
 }

 {
+    "epoch": 4.986698911729142,
+    "eval_loss": 0.243755504488945,
+    "eval_runtime": 46.4843,
+    "eval_samples_per_second": 94.871,
+    "eval_steps_per_second": 5.937,
+    "num_input_tokens_seen": 18724952
 }