Update README for sanitized vLLM-ready toxicity adapter

README.md

@@ -6,23 +6,52 @@ library_name: peft
 tags:
 - mergekit
 - peft
-
+- lora
+- vllm
 ---
-# toxicity
-
-This is a LoRA extracted from a language model. It was extracted using [mergekit](https://github.com/arcee-ai/mergekit).
-
-## LoRA Details
-
-This LoRA adapter was extracted from [K-intelligence/Llama-SafetyGuard-Content-Binary](https://huggingface.co/K-intelligence/Llama-SafetyGuard-Content-Binary) and uses [meta-llama/Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B) as a base.
+# NEXS toxicity LoRA (vLLM-ready)
+
+Rank-128 LoRA adapter for the **toxicity** domain, extracted with
+[mergekit](https://github.com/arcee-ai/mergekit) from
+[K-intelligence/Llama-SafetyGuard-Content-Binary](https://huggingface.co/K-intelligence/Llama-SafetyGuard-Content-Binary) against the base model
+[meta-llama/Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B), then sanitized for vLLM
+serving.
+
+## Sanitization applied
+
+The raw mergekit extraction included full-rank `modules_to_save` tensors
+(`embed_tokens`, `lm_head`, and RMSNorm layers) that vLLM's LoRA runtime does
+not support. This upload contains only the pure low-rank `lora_A`/`lora_B`
+weights (224 pairs: 32 layers x q/k/v/o/gate/up/down projections, bf16), with
+`modules_to_save: null` in `adapter_config.json`. No `resize_token_embeddings()`
+call is needed to load this adapter.
+
+## Serving with vLLM
+
+```bash
+python -m vllm.entrypoints.openai.api_server \
+    --model meta-llama/Llama-3.1-8B \
+    --enable-lora \
+    --lora-modules toxicity=anjohn0077/NEXS-toxicity-lora \
+    --port 8000 \
+    --max-lora-rank 128 \
+    --gpu-memory-utilization 0.85
+```
 
- [!WARNING]
-> The vocabulary size has been extended from the base model's 128256 to 128258. To load this adapter, you must first call `model.resize_token_embeddings(128258)`.
+## Evaluation (sst2)
 
-### Parameters
+| Variant | Accuracy |
+|---|---|
+| Base model | 0.6732 |
+| **This LoRA on base (via vLLM)** | **0.8991** |
+| Original full fine-tune | 0.8899 |
 
-The following command was used to extract this LoRA adapter:
+Evaluated with [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness)
+against a local vLLM OpenAI-compatible endpoint:
 
-```sh
-/nethome/ian6/ian6-myenv/bin/mergekit-extract-lora --model K-intelligence/Llama-SafetyGuard-Content-Binary --base-model meta-llama/Llama-3.1-8B --out-path ./lora_adapters/toxicity --cuda
+```bash
+lm_eval --model local-completions \
+    --model_args model=toxicity,base_url=http://localhost:8000/v1/completions,tokenizer=meta-llama/Llama-3.1-8B,num_concurrent=10 \
+    --tasks sst2 \
+    --output_path results/vllm_toxicity
 ```