metadata
license: apache-2.0
base_model: Qwen/Qwen3.5-2B
library_name: peft
pipeline_tag: text-generation
language:
- en
tags:
- lora
- peft
- qwen
- qwen3
- guardrails
- code-detection
- code-classification
- content-moderation
- quantization
- 8-bit
metrics:
- accuracy
- f1
- precision
- recall
model-index:
- name: Code-Qwen3.5-2B-LoRA-8bit
results:
- task:
type: text-classification
name: Embedded Code Identification Guard
dataset:
name: Code Guard Held-out Test Set
type: custom
metrics:
- type: accuracy
name: is_valid accuracy
value: 0.95
- type: accuracy
name: violation-type-set exact match
value: 0.81
- type: f1
name: binary F1 (positive=invalid)
value: 0.9474
- type: f1
name: macro F1 over violation types
value: 0.7545
- type: precision
name: binary precision (positive=invalid)
value: 1
- type: recall
name: binary recall (positive=invalid)
value: 0.9
PII-Qwen3.5-2B-LoRA-8bit
LoRA adapter for Qwen/Qwen3.5-2B that detects embedded code snippets in user prompts across 10 programming languages (bash, c, go, java, javascript, php, python, ruby, rust, sql) plus a generic Code pattern. Trained on the LLM Guard code and ban_code scanner outputs.
The model is fine-tuned to emit a strict JSON object describing every code-snippet location found in the user prompt:
{"is_valid": false, "violations": {"python": [[5, 25]], "sql": [[40, 60]]}}
Quick start
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch, json, re
BASE = "Qwen/Qwen3.5-2B"
ADAPTER = "Yash1005/PII-Qwen3.5-2B-LoRA-8bit"
tokenizer = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
bnb = BitsAndBytesConfig(load_in_8bit=True)
model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(model, ADAPTER); model.eval()
def guard(prompt: str) -> dict:
chat = tokenizer.apply_chat_template(
[{"role":"system","content":SYSTEM_MSG},
{"role":"user","content":prompt}],
tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = tokenizer(chat, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=768, do_sample=False)
text = tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
return json.loads(re.search(r'\{.*\}', text, re.DOTALL).group(0))
Evaluation
Evaluated on 100 held-out prompts drawn from test_dataset_code.csv
- Source adapter:
Yash1005/Code-Qwen3.5-2B-LoRA-8bit - JSON parse errors:
0/100(0.0%)
Top-level metrics
| Metric | Value |
|---|---|
is_valid accuracy |
0.9500 |
| Violation-type-set exact match | 0.8100 |
| Binary F1 (positive = invalid) | 0.9474 |
| Binary precision | 1.0000 |
| Binary recall | 0.9000 |
| Macro F1 across violation types | 0.7545 |
Confusion matrix — binary is_valid decision
Positive class = the prompt contains a violation (is_valid=False).
| predicted invalid | predicted valid | |
|---|---|---|
| actual invalid | TP = 45 | FN = 5 |
| actual valid | FP = 0 | TN = 50 |
Per violation-type metrics
Only types that appear in either the actual or predicted labels are listed.
| Type | support | precision | recall | F1 |
|---|---|---|---|---|
python |
12 | 0.917 | 0.917 | 0.917 |
sql |
10 | 1.000 | 0.500 | 0.667 |
bash |
8 | 1.000 | 0.250 | 0.400 |
javascript |
8 | 0.778 | 0.875 | 0.824 |
rust |
7 | 1.000 | 0.714 | 0.833 |
java |
6 | 1.000 | 0.833 | 0.909 |
php |
5 | 1.000 | 1.000 | 1.000 |
ruby |
5 | 1.000 | 0.600 | 0.750 |
Code |
5 | 0.000 | 0.000 | 0.000 |
c |
4 | 1.000 | 1.000 | 1.000 |
go |
4 | 1.000 | 1.000 | 1.000 |
Supported violation types
The model emits one or more of these TYPE keys in the violations map of its JSON output:
bash, c, go, java, javascript, php, python, ruby, rust, sql, Code