metadata
base_model:
- Qwen/Qwen3-32B
- qihoo360/Light-IF-32B
library_name: peft
tags:
- mergekit
- peft
- lora
- vllm
NEXS Qwen3-32B IF LoRA (vLLM-ready)
Rank-128 LoRA adapter (bf16) extracted with mergekit from qihoo360/Light-IF-32B against the base model Qwen/Qwen3-32B, then sanitized for vLLM serving.
Sanitization applied
The raw mergekit extraction included full-rank modules_to_save tensors
(embed_tokens, lm_head, and norm layers) that vLLM's LoRA runtime does not
support. This upload contains only the pure low-rank lora_A/lora_B weights
(448 pairs: 64 layers x q/k/v/o/gate/up/down projections), with
modules_to_save: null in adapter_config.json. No resize_token_embeddings()
call is needed to load this adapter.
Serving with vLLM
python -m vllm.entrypoints.openai.api_server \
--model Qwen/Qwen3-32B \
--enable-lora \
--lora-modules IF=anjohn0077/NEXS-qwen3-32b-IF-lora \
--port 8000 \
--max-lora-rank 128 \
--gpu-memory-utilization 0.85
Evaluation (ifeval)
| Variant | Accuracy |
|---|---|
| Base model | 0.8336 |
| This LoRA on base (via vLLM) | 0.2737 |
| Original full fine-tune | 0.8669 |
Evaluated with lm-evaluation-harness against a local vLLM OpenAI-compatible endpoint:
lm_eval --model local-completions \
--model_args model=IF,base_url=http://localhost:8000/v1/completions,tokenizer=Qwen/Qwen3-32B,num_concurrent=10 \
--tasks ifeval \
--output_path results/vllm_IF