Instructions to use JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1 with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-7B")
model = PeftModel.from_pretrained(base_model, "JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1")

Transformers

How to use JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1

SGLang

How to use JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1 with Docker Model Runner:
```
docker model run hf.co/JinNian0072/Qwen2.5-Coder-7B-lora-cpp-adiff-v1
```

Qwen2.5-Coder-7B-lora-cpp-adiff-v1 / checkpoint-383 /trainer_state.json

JinNian0072

Upload folder using huggingface_hub

988c6c0 verified 8 months ago

Raw History Blame Contribute Delete

7.63 kB

	{
	"best_global_step": 383,
	"best_metric": 0.37941670417785645,
	"best_model_checkpoint": "./lora_qwen7b_cpp_adiff_v1/checkpoint-383",
	"epoch": 1.0,
	"eval_steps": 500,
	"global_step": 383,
	"is_hyper_param_search": false,
	"is_local_process_zero": true,
	"is_world_process_zero": true,
	"log_history": [
	{
	"epoch": 0.02611818478615736,
	"grad_norm": 0.3439924716949463,
	"learning_rate": 2.347826086956522e-06,
	"loss": 0.9253,
	"step": 10
	},
	{
	"epoch": 0.05223636957231472,
	"grad_norm": 0.37593239545822144,
	"learning_rate": 4.956521739130435e-06,
	"loss": 0.8942,
	"step": 20
	},
	{
	"epoch": 0.07835455435847209,
	"grad_norm": 0.4532669186592102,
	"learning_rate": 7.5652173913043475e-06,
	"loss": 0.9621,
	"step": 30
	},
	{
	"epoch": 0.10447273914462944,
	"grad_norm": 0.30326446890830994,
	"learning_rate": 1.017391304347826e-05,
	"loss": 0.8363,
	"step": 40
	},
	{
	"epoch": 0.1305909239307868,
	"grad_norm": 0.4317711591720581,
	"learning_rate": 1.2782608695652173e-05,
	"loss": 0.7811,
	"step": 50
	},
	{
	"epoch": 0.15670910871694418,
	"grad_norm": 0.3774474263191223,
	"learning_rate": 1.5391304347826088e-05,
	"loss": 0.5888,
	"step": 60
	},
	{
	"epoch": 0.18282729350310153,
	"grad_norm": 0.1916833221912384,
	"learning_rate": 1.8e-05,
	"loss": 0.4563,
	"step": 70
	},
	{
	"epoch": 0.20894547828925888,
	"grad_norm": 0.13702069222927094,
	"learning_rate": 2.0608695652173913e-05,
	"loss": 0.4354,
	"step": 80
	},
	{
	"epoch": 0.23506366307541626,
	"grad_norm": 0.10257239639759064,
	"learning_rate": 2.3217391304347826e-05,
	"loss": 0.408,
	"step": 90
	},
	{
	"epoch": 0.2611818478615736,
	"grad_norm": 0.15386034548282623,
	"learning_rate": 2.582608695652174e-05,
	"loss": 0.3986,
	"step": 100
	},
	{
	"epoch": 0.28730003264773096,
	"grad_norm": 0.12275378406047821,
	"learning_rate": 2.8434782608695652e-05,
	"loss": 0.3773,
	"step": 110
	},
	{
	"epoch": 0.31341821743388837,
	"grad_norm": 0.13668131828308105,
	"learning_rate": 2.988394584139265e-05,
	"loss": 0.3579,
	"step": 120
	},
	{
	"epoch": 0.3395364022200457,
	"grad_norm": 0.11877596378326416,
	"learning_rate": 2.9593810444874276e-05,
	"loss": 0.3622,
	"step": 130
	},
	{
	"epoch": 0.36565458700620307,
	"grad_norm": 0.15862952172756195,
	"learning_rate": 2.93036750483559e-05,
	"loss": 0.352,
	"step": 140
	},
	{
	"epoch": 0.3917727717923604,
	"grad_norm": 0.14640262722969055,
	"learning_rate": 2.9013539651837528e-05,
	"loss": 0.3597,
	"step": 150
	},
	{
	"epoch": 0.41789095657851777,
	"grad_norm": 0.11437654495239258,
	"learning_rate": 2.872340425531915e-05,
	"loss": 0.3456,
	"step": 160
	},
	{
	"epoch": 0.4440091413646752,
	"grad_norm": 0.1443624049425125,
	"learning_rate": 2.8433268858800773e-05,
	"loss": 0.3513,
	"step": 170
	},
	{
	"epoch": 0.4701273261508325,
	"grad_norm": 0.11668717861175537,
	"learning_rate": 2.81431334622824e-05,
	"loss": 0.3246,
	"step": 180
	},
	{
	"epoch": 0.4962455109369899,
	"grad_norm": 0.1281779706478119,
	"learning_rate": 2.785299806576402e-05,
	"loss": 0.3504,
	"step": 190
	},
	{
	"epoch": 0.5223636957231472,
	"grad_norm": 0.130177304148674,
	"learning_rate": 2.7562862669245647e-05,
	"loss": 0.3487,
	"step": 200
	},
	{
	"epoch": 0.5484818805093046,
	"grad_norm": 0.1486888825893402,
	"learning_rate": 2.7272727272727273e-05,
	"loss": 0.3305,
	"step": 210
	},
	{
	"epoch": 0.5746000652954619,
	"grad_norm": 0.12134828418493271,
	"learning_rate": 2.69825918762089e-05,
	"loss": 0.3451,
	"step": 220
	},
	{
	"epoch": 0.6007182500816193,
	"grad_norm": 0.12231825292110443,
	"learning_rate": 2.669245647969052e-05,
	"loss": 0.3233,
	"step": 230
	},
	{
	"epoch": 0.6268364348677767,
	"grad_norm": 0.13645341992378235,
	"learning_rate": 2.6402321083172148e-05,
	"loss": 0.3365,
	"step": 240
	},
	{
	"epoch": 0.6529546196539341,
	"grad_norm": 0.14500953257083893,
	"learning_rate": 2.6112185686653773e-05,
	"loss": 0.3329,
	"step": 250
	},
	{
	"epoch": 0.6790728044400914,
	"grad_norm": 0.16711661219596863,
	"learning_rate": 2.5822050290135396e-05,
	"loss": 0.3305,
	"step": 260
	},
	{
	"epoch": 0.7051909892262488,
	"grad_norm": 0.14286471903324127,
	"learning_rate": 2.5531914893617022e-05,
	"loss": 0.3159,
	"step": 270
	},
	{
	"epoch": 0.7313091740124061,
	"grad_norm": 0.17056842148303986,
	"learning_rate": 2.5241779497098648e-05,
	"loss": 0.335,
	"step": 280
	},
	{
	"epoch": 0.7574273587985635,
	"grad_norm": 0.16313998401165009,
	"learning_rate": 2.495164410058027e-05,
	"loss": 0.318,
	"step": 290
	},
	{
	"epoch": 0.7835455435847208,
	"grad_norm": 0.16147395968437195,
	"learning_rate": 2.4661508704061896e-05,
	"loss": 0.2996,
	"step": 300
	},
	{
	"epoch": 0.8096637283708782,
	"grad_norm": 0.1918044090270996,
	"learning_rate": 2.4371373307543522e-05,
	"loss": 0.3172,
	"step": 310
	},
	{
	"epoch": 0.8357819131570355,
	"grad_norm": 0.15215300023555756,
	"learning_rate": 2.408123791102515e-05,
	"loss": 0.3258,
	"step": 320
	},
	{
	"epoch": 0.861900097943193,
	"grad_norm": 0.15154816210269928,
	"learning_rate": 2.379110251450677e-05,
	"loss": 0.3206,
	"step": 330
	},
	{
	"epoch": 0.8880182827293504,
	"grad_norm": 0.19857144355773926,
	"learning_rate": 2.3500967117988397e-05,
	"loss": 0.3038,
	"step": 340
	},
	{
	"epoch": 0.9141364675155077,
	"grad_norm": 0.1814548224210739,
	"learning_rate": 2.321083172147002e-05,
	"loss": 0.3157,
	"step": 350
	},
	{
	"epoch": 0.940254652301665,
	"grad_norm": 0.17991890013217926,
	"learning_rate": 2.2920696324951642e-05,
	"loss": 0.3044,
	"step": 360
	},
	{
	"epoch": 0.9663728370878224,
	"grad_norm": 0.17508666217327118,
	"learning_rate": 2.2630560928433268e-05,
	"loss": 0.3081,
	"step": 370
	},
	{
	"epoch": 0.9924910218739798,
	"grad_norm": 0.17764495313167572,
	"learning_rate": 2.2340425531914894e-05,
	"loss": 0.2957,
	"step": 380
	},
	{
	"epoch": 1.0,
	"eval_loss": 0.37941670417785645,
	"eval_runtime": 41.5299,
	"eval_samples_per_second": 22.105,
	"eval_steps_per_second": 11.052,
	"step": 383
	}
	],
	"logging_steps": 10,
	"max_steps": 1149,
	"num_input_tokens_seen": 0,
	"num_train_epochs": 3,
	"save_steps": 500,
	"stateful_callbacks": {
	"TrainerControl": {
	"args": {
	"should_epoch_stop": false,
	"should_evaluate": false,
	"should_log": false,
	"should_save": true,
	"should_training_stop": false
	},
	"attributes": {}
	}
	},
	"total_flos": 3.089357644575928e+17,
	"train_batch_size": 2,
	"trial_name": null,
	"trial_params": null
	}