SASVAAI/Qwen-3.5-9B-CVE-to-CWE

Qwen3.5-9B CVE-to-CWE Classifier (LoRA)

Given the free-text description of a published CVE, emits the single CWE identifier that best characterises the root-cause weakness, as one line of text (CWE-79, CWE-787, ...), over the 117-class label space of exploitintel/cve-cwe-consensus.

This is a LoRA adapter for Qwen/Qwen3.5-9B, trained with QLoRA (4-bit NF4 base, bf16 compute) via TRL SFT.

The headline number is one draw from a noisy measurement. This checkpoint scored macro-F1 0.473 on 300 held-out rows, but the identical configuration re-run later in the same search scored 0.395. Read How these values were chosen and Evaluation before quoting anything, and do not report 0.473 as the model's expected performance.

Model details

Developed by	SASVA AI Model Cognition Labs (MCL) Team
Base model	Qwen/Qwen3.5-9B
Base revision	c202236235762e1c871ad0ccb60c8ee5ba337b9a
Base parameters	9.65B (dense; 9,653,104,368 in the safetensors index, ~19.3 GB bf16)
Architecture family	qwen3_5 (Qwen3_5ForConditionalGeneration, text tower qwen3_5_text)
Adaptation	LoRA (r=128, alpha=256, dropout=0.15, rsLoRA off, DoRA off)
Trainable modules	in_proj_qkv, in_proj_z, out_proj (Gated DeltaNet layers); q_proj, k_proj, v_proj, o_proj (attention layers); gate_proj, up_proj, down_proj (every layer)
Excluded modules	none (the base is text-only in practice; no adapter tensor touches the vision tower)
Training method	qlora (--load-in-4bit, run 3 / trial 23)
Refinement	none
Precision	4-bit NF4 base with double quantisation, bf16 compute; adapter stored in float32
Language	English
License	Apache-2.0 (inherited from the base model)

Trainable parameters: 320,864,256 across 200 modules, 3.2975% of the 9,730,678,000 parameters PEFT counted with the adapter attached. The adapter file is 1,283,518,408 bytes (400 tensors, lora_A + lora_B per module, all float32). Every tensor sits under base_model.model.model.language_model.

One Qwen 3.5 structural fact shapes the module list. Confirmed against the base model's config.json: the 32 layers follow a 3-linear / 1-full pattern (full_attention_interval: 4), so 24 layers are Gated DeltaNet linear attention and expose in_proj_qkv / in_proj_z / out_proj, while the 8 full-attention layers (3, 7, 11, ..., 31) are grouped-query attention with 16 heads over 4 KV heads and expose q_proj / k_proj / v_proj / o_proj. The adapter's tensor counts match exactly: 48 per linear-attention projection (24 layers x A/B), 16 per attention projection (8 layers x A/B), 64 per MLP projection (32 layers x A/B). There are no MoE experts in this base.

Intended use

Direct use. Map a CVE description to one CWE id, for triage and labelling pipelines that already consume CWE ids. The 117-class label space is the dataset's; ids outside it were never seen in training.

The model was trained on a specific prompt shape and that shape is part of the contract:

• System prompt (verbatim): "You are a CVE-to-CWE classifier. Given a CVE vulnerability description, identify the single root-cause CWE weakness class that best characterizes the flaw. Output exactly one CWE identifier (e.g., CWE-79) on a single line with no explanation or additional text."
• User turn: the instruction "Classify the following CVE description into exactly one CWE weakness class. Reply with the CWE ID only, for example CWE-79.", a blank line, then the CVE description inside a bare ``` fence. This is the exact string the evaluator rendered (instruction + "\n\n```\n" + description + "\n```").
• Applied through the tokenizer's chat template (chat_template.jinja, shipped in this repo) with add_generation_prompt=True and enable_thinking=False. Do not concatenate strings by hand.
• The output is one line, CWE-<n>. In evaluation all 300 generations were a single well-formed id; the first line of the output is the prediction.
• Decode greedily (do_sample=False) with a small budget; 64 new tokens is what the metric was scored with.

How to get started

import torch
from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoTokenizer

BASE = "Qwen/Qwen3.5-9B"
ADAPTER = "SASVAAI/Qwen-3.5-9B-CVE-to-CWE"

# NOTE: AutoModelForImageTextToText, not AutoModelForCausalLM. Qwen 3.5's
# CausalLM mapping raises AttributeError: 'Qwen3_5Config' object has no
# attribute 'vocab_size' on the transformers build used here; the
# image-text-to-text mapping resolves to Qwen3_5ForConditionalGeneration and
# loads the text model correctly. Training used the same class.
tokenizer = AutoTokenizer.from_pretrained(ADAPTER, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
    BASE, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True
)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

# Verbatim from the training invocation. Do not paraphrase.
SYSTEM = (
    "You are a CVE-to-CWE classifier. Given a CVE vulnerability description, "
    "identify the single root-cause CWE weakness class that best characterizes "
    "the flaw. Output exactly one CWE identifier (e.g., CWE-79) on a single line "
    "with no explanation or additional text."
)
INSTRUCTION = (
    "Classify the following CVE description into exactly one CWE weakness class. "
    "Reply with the CWE ID only, for example CWE-79."
)
description = (
    "A stored cross-site scripting flaw in the FAQ page lets an attacker inject "
    "script that runs in other users' browsers."
)
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": f"{INSTRUCTION}\n\n```\n{description}\n```"},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, enable_thinking=False,
    return_tensors="pt", return_dict=True,
).to(model.device)

out = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True).strip())
# -> CWE-79

The base model is ~19 GB in bfloat16; one 24 GB-class GPU is enough for inference with the adapter. Loading the base in 4-bit with bitsandbytes (load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16) reproduces the training-time numerics and needs about 7 GB.

Decoding matters. The metric was scored greedily with max_new_tokens=64 through the chat template with thinking disabled. No sampling setting was validated, and enabling thinking changes the prompt the model sees.

Training details

Data. 700 training rows and 300 validation rows built from exploitintel/cve-cwe-consensus at revision 606cef101302fc2e0f69fd298f0de28b20a2aacf by a deterministic script (autocatalyst.datagen.cve_cwe_consensus, seed 0). No model generated any training content.

Row selection, as recorded in the builder's manifest:

• Source rows are {"messages": [system, user, assistant]}; the user turn is the CVE description, the assistant turn the label.
• Rows whose label is not exactly one CWE-<n> id were dropped (about 9% of the dataset are comma-separated multi-label lists). Rows whose id is outside the dataset's 117-class set, or whose description is empty, were dropped. Duplicate descriptions within a split were collapsed to the first.
• Train: from the dataset's train split (50,074 raw rows, 44,337 kept after dropping 4,390 multi-label or malformed and 1,347 duplicate rows), 700 rows were sampled with one row guaranteed per class and the remainder weighted by the square root of class frequency, so the head classes do not crowd out the tail that macro-F1 scores. All 117 classes are present.
• Validation: from the dataset's own validation split (11,052 raw rows, 8,878 kept), 300 rows sampled at the natural class distribution. 68 classes appear; every one of them is in the training set; no description text is shared with the 700 training rows (0 overlaps measured).

Train samples	700 rows / 117 classes
Validation samples	300 rows / 68 classes
Text overlap	0 rows
Prompt format	chat template + system prompt + instruction/fenced-description user turn (see Intended use)
Loss masking	answer tokens only; prompt tokens set to -100
Truncation	prompt left-truncated to fit max_seq_len 2048; answer never truncated

The dataset's head is heavy: in the 300 validation rows CWE-79 appears 49 times, CWE-862 22, CWE-89 17, CWE-200 and CWE-22 13 each, and 68 classes share the rest.

An LLM (Claude Opus 4.6, claude-opus-4-6, via an internal inference gateway) proposed the hyperparameters the search tried. It generated no training content and computed no metric.

Method

SFT method	qlora
Base quantisation during training	4-bit NF4, double quantisation, bf16 compute (bitsandbytes)
Refinement stage	none
Attention implementation	flash_attention_2 for the 8 GQA layers; the 24 Gated DeltaNet layers ran the PyTorch fallback because flash-linear-attention was not installed
Auto class	AutoModelForImageTextToText (see How to get started)
Hardware	8x NVIDIA H100 80GB HBM3, torchrun --nproc_per_node=8

The project allowed two methods for this run (bf16_lora, qlora) and the search tried both; see the trial table below.

No refinement stage ran; the published weights are the SFT adapter.

Final hyperparameters

Hyperparameter	Value	Source
learning_rate	0.0002	[TRAIN] cmdline
lr_scheduler_type	cosine	[TRAIN] cmdline
num_train_epochs	5	[TRAIN] cmdline, trainer_state.json
per_device_train_batch_size	2	[TRAIN] cmdline, trainer_state.json
gradient_accumulation_steps	4	[TRAIN] cmdline
max_seq_length	2048	[TRAIN] cmdline
warmup_ratio	0.05	[TRAIN] cmdline
weight_decay	0.05	[TRAIN] cmdline
loraplus_lr_ratio	2.0 (LoRA B matrices at 2x the learning rate)	[TRAIN] cmdline
lora_r / lora_alpha / lora_dropout	128 / 256 / 0.15	adapter_config.json
use_rslora / use_dora	false / false	adapter_config.json
target_modules	the 10 listed in Model details	adapter_config.json
load_in_4bit	true (NF4, double quant, bf16 compute)	[TRAIN] cmdline

Effective batch size: 64 (2 x 4 x 8). Optimizer steps: 55 (11 per epoch).

neftune_noise_alpha (0.0), use_liger_kernel, use_sample_packing and lora_init (default) were left at their no-op defaults. KD parameters are omitted deliberately: this is a qlora run, not a distillation run.

Provenance note: every value above was recovered from the platform database (runs, experiments, events tables for run 3) and cross-checked against the literal [TRAIN] command line recorded in the run log and against the shipped adapter_config.json.

How these values were chosen

These hyperparameters were selected by an automated search (autocatalyst.cli.run_autoresearch): an agent proposes one change at a time, runs train then eval, and keeps or discards on f1_macro (higher is better).

Run 3 ran 46 trials in 7 h 17 m (2026-09-08 20:29 to 2026-09-09 03:46); 35 scored and 11 errored. This checkpoint is trial 23, the run's best.

Every trial is comparable. The search's five "data strategies" all re-adapted the same fixed 700-row training file (the train_v1 ... train_v5 files below hold the same 700 rows) and every trial was scored on the same 300 validation rows, so the whole table is one comparison.

#	Method	rsLoRA	r	Dropout	LR	Epochs	Grad accum	Warmup	Weight decay	LoRA+ ratio	Data	f1_macro	Kept
1	bf16_lora	no	16	0.05	2e-4	2	4	0.05	0.01	1.0	train_v1	0.271028	yes
2	qlora	no	16	0.05	2e-4	2	4	0.05	0.01	1.0	train_v1	0.323589	yes
3	bf16_lora	no	16	0.05	2e-4	2	4	0.05	0.01	1.0	train_v1	0.271161	no
4	qlora	no	16	0.05	1e-4	2	4	0.05	0.01	1.0	train_v1	0.224824	no
5	qlora	no	16	0.05	2e-4	5	4	0.05	0.01	1.0	train_v1	0.328347	yes
6	qlora	no	16	0.05	2e-4	8	4	0.05	0.01	1.0	train_v1	0.315655	no
7	qlora	no	32	0.05	2e-4	5	4	0.05	0.01	1.0	train_v1	0.339796	yes
8	qlora	no	32	0.15	2e-4	5	4	0.05	0.01	1.0	train_v1	0.358649	yes
9	qlora	no	32	0.15	2e-4	5	4	0.05	0.01	2.0	train_v1	0.379137	yes
10	qlora	no	64	0.15	2e-4	5	4	0.05	0.01	2.0	train_v1	0.411651	yes
11	qlora	yes	64	0.15	2e-4	5	4	0.05	0.01	1.0	train_v1	0.385129	no
12	qlora	no	64	0.05	2e-4	5	4	0.05	0.01	2.0	train_v1	0.405714	no
13	qlora	no	64	0.15	2e-4	5	8	0.05	0.01	2.0	train_v1	0.377237	no
14	qlora	no	128	0.15	2e-4	5	4	0.05	0.01	2.0	train_v1	0.440878	yes
15	qlora	no	128	0.15	2e-4	8	4	0.05	0.01	2.0	train_v1	0.377763	no
16	qlora	yes	128	0.15	2e-4	5	4	0.05	0.01	1.0	train_v1	error	no
17	qlora	no	128	0.15	1.5e-4	5	4	0.05	0.01	2.0	train_v1	0.386976	no
18	qlora	yes	128	0.15	2e-4	5	4	0.05	0.01	1.0	train_v1	error	no
19	qlora	no	128	0.15	2e-4	5	4	0.05	0.01	2.0	train_v1	0.396817	no
20	qlora	no	128	0.15	2e-4	8	4	0.05	0.01	2.0	train_v2	0.407741	no
21	bf16_lora	no	128	0.15	2e-4	5	4	0.05	0.01	2.0	train_v2	0.403981	no
22	qlora	yes	128	0.15	2e-4	5	4	0.05	0.01	1.0	train_v2	error	no
23	qlora	no	128	0.15	2e-4	5	4	0.05	0.05	2.0	train_v2	0.473288	yes
24	qlora	no	128	0.10	2e-4	5	4	0.05	0.05	2.0	train_v2	0.444999	no
25	qlora	yes	128	0.15	2e-4	5	4	0.05	0.05	1.0	train_v2	error	no
26	qlora	no	128	0.15	2.5e-4	5	4	0.05	0.05	2.0	train_v2	0.427152	no
27	qlora	yes	128	0.15	2e-4	5	4	0.05	0.05	1.0	train_v2	error	no
28	qlora	yes	128	0.15	2e-4	5	4	0.05	0.05	1.0	train_v2	error	no
29	qlora	yes	128	0.15	2e-4	5	4	0.05	0.05	1.0	train_v3	error	no
30	qlora	no	128	0.15	2e-4	8	4	0.05	0.05	2.0	train_v3	0.384342	no
31	bf16_lora	no	128	0.15	2e-4	5	4	0.05	0.05	2.0	train_v3	0.403688	no
32	qlora	no	128	0.05	2e-4	5	4	0.05	0.05	2.0	train_v3	0.415561	no
33	qlora	no	128	0.15	1.75e-4	5	4	0.05	0.05	2.0	train_v3	0.435022	no
34	qlora	no	128	0.15	2e-4	5	4	0.03	0.05	2.0	train_v3	0.416758	no
35	qlora	no	128	0.15	2e-4	8	4	0.05	0.05	2.0	train_v4	0.406259	no
36	bf16_lora	no	128	0.15	2e-4	5	4	0.05	0.05	2.0	train_v4	0.408250	no
37	qlora	yes	128	0.15	2e-4	5	4	0.05	0.05	1.0	train_v4	error	no
38	qlora	no	128	0.15	2e-4	5	4	0.05	0.05	2.0	train_v4	0.395110	no
39	qlora	no	128	0.15	2e-4	5	4	0.05	0.01	2.0	train_v4	0.437202	no
40	qlora	yes	128	0.15	2e-4	5	4	0.05	0.05	1.0	train_v4	error	no
41	qlora	no	128	0.15	2e-4	8	4	0.05	0.05	2.0	train_v5	0.391961	no
42	bf16_lora	no	128	0.15	2e-4	5	4	0.05	0.05	2.0	train_v5	0.428970	no
43	qlora	no	128	0.05	2e-4	5	4	0.05	0.05	2.0	train_v5	0.423615	no
44	qlora	yes	128	0.15	2e-4	5	4	0.05	0.05	1.0	train_v5	error	no
45	qlora	yes	128	0.15	2e-4	5	4	0.05	0.05	1.0	train_v5	error	no
46	qlora	no	128	0.15	2e-4	5	8	0.05	0.05	2.0	train_v5	0.402578	no

All trials: lora_alpha = 2 x r, cosine schedule, batch 2 per device, max_seq_len 2048.

What the search actually established.

• Rank dominates, up to a point. Holding QLoRA, 5 epochs, dropout 0.15, LR 2e-4, weight decay 0.01, LoRA+ 2.0: r 32 → 64 → 128 moved macro-F1 0.379137 → 0.411651 → 0.440878 (#9, #10, #14).
• LoRA+ helped. The one clean A/B at r=32 (#8 → #9) moved 0.358649 → 0.379137 by putting the B matrices at 2x the learning rate.
• More epochs hurt. 8 epochs lost to 5 every time it was tried at r=128 (#15, #20, #30, #35, #41), by 0.01 to 0.06.
• 4-bit versus bf16 base: no measurable difference. At r=16 the 4-bit base beat bf16 (#2 at 0.323589 against #1 and #3 at 0.271); at r=128 with weight decay 0.05 the three bf16 runs (#31, #36, #42: 0.404 to 0.429) sit inside the two 4-bit runs of the same configuration (#38 at 0.395, #23 at 0.473). Quantising the base cost nothing this search could detect.
• rsLoRA at r=128 diverges. All 11 rsLoRA trials at r=128 (#16, #18, #22, #25, #27, #28, #29, #37, #40, #44, #45) were aborted by the training watchdog at the third logging step with gradient norm above 10 (the configured limit; observed 12.3 in the first). The likely cause is the scaling rule: rsLoRA scales updates by alpha / sqrt(r) instead of alpha / r, which with alpha = 2r is 2 sqrt(r), about 22.6 at r=128 against 2.0 without it, an 11x larger effective update. The one rsLoRA trial that finished (#11, r=64) scored below its non-rsLoRA sibling (#10). The 11 aborted trials are why the search shows 46 trials but 35 scores.

What it did not establish: the winning margin. The search's decision "weight decay 0.01 → 0.05 improved 0.440878 → 0.473288" (#14 → #23) does not survive the re-runs. Identical configurations were run more than once because each "data strategy" restarted the inner loop on the same data:

Configuration (all QLoRA, r=128, dropout 0.15, LR 2e-4, 5 epochs, grad accum 4, warmup 0.05, LoRA+ 2.0)	Trials	f1_macro	Spread
weight decay 0.05 (this checkpoint's config)	#23, #38	0.473288, 0.395110	0.078
weight decay 0.01	#14, #19, #39	0.440878, 0.396817, 0.437202	0.044
weight decay 0.05, bf16 base instead of 4-bit	#31, #36, #42	0.403688, 0.408250, 0.428970	0.025
weight decay 0.05, 8 epochs	#30, #35, #41	0.384342, 0.406259, 0.391961	0.022
weight decay 0.05, dropout 0.05	#32, #43	0.415561, 0.423615	0.008

Seeds were not pinned, so these differ only in initialisation and data order. The 0.078 gap between #23 and #38 is larger than the 0.032 "improvement" the search kept, and larger than most differences in the table. Read 0.473 as the high draw of a configuration whose expected score is somewhere in the low-to-mid 0.4s, and treat any two trials within about 0.05 of each other as tied.

Search space. Six knobs were varied (LORA_R, LORA_DROPOUT, LEARNING_RATE, EPOCHS, WEIGHT_DECAY, LORAPLUS_LR_RATIO) plus one GRAD_ACCUM probe (#13, #46), one warmup probe (#34), the bf16-vs-4-bit method switch, and the rsLoRA attempts. LR_SCHEDULER, MAX_SEQ_LEN, BATCH_SIZE, NEFTUNE_NOISE_ALPHA, USE_DORA, LORA_INIT, USE_LIGER_KERNEL and USE_SAMPLE_PACKING were never moved.

Observed training metrics (this checkpoint).

Final train loss (mean over the run)	0.6052066683769226
Final eval loss (teacher-forced, answer tokens)	0.9717274904251099
Eval mean token accuracy	0.8136
Train runtime	387.7625 s
Total FLOPs	4.311845712166912e+16
Throughput	9.026 samples/s, 0.142 steps/s

55 optimizer steps ran. The logged train loss fell from 1.2962 at step 10 (grad norm 1.06) to the run mean of 0.6052; the reported train loss is the mean over the run, not a converged value.

Evaluation

Protocol. All 300 validation rows, greedy decoding, max_new_tokens=64, prompts rendered through the chat template with enable_thinking=False. The project's classification evaluator takes the first line of the generation as the predicted label and compares it to the gold id. Generation ran through Hugging Face generate in batches of 4 (max_input_len 4096) after vLLM 0.19.1 refused the LoRA on this architecture and the evaluator fell back; the whole pass took 63 s on 8 GPUs. Every one of the 300 outputs was a single well-formed CWE-<n> id, so no prediction was lost to formatting, and the 64-token budget cut nothing (the longest output is one id).

Metric	Value
Macro-F1, union of gold and predicted classes (the search metric)	0.473288
Macro-F1, gold classes only	0.563769
Accuracy (= micro-F1)	0.71 (213 / 300)
Classes in gold / predicted / union	68 / 72 / 81

Two macro-F1 numbers, one convention. The evaluator averages F1 over the union of gold and predicted classes, so the 13 classes the model predicted that never occur in the 300 gold rows each contribute an F1 of 0 and pull the macro average down from 0.5638 to 0.4733. Both are reported; the union convention is the one the search optimised and the one the published baselines below appear to use, but check before comparing.

Published comparison points. The dataset authors report, on their own evaluation of the same dataset (a much larger split than these 300 rows):

Model	Method	Micro-F1	Macro-F1
Qwen3-32B (exploitintel/cve-cwe-qwen3-32b)	QLoRA r=16	0.729	0.595
Qwen3-8B (same author, same recipe)	QLoRA r=16	0.702	0.511
This adapter (Qwen3.5-9B, QLoRA r=128, 700 training rows)		0.71	0.473

Those models trained on the full ~44K-row training split; this adapter saw 700 rows. The micro-F1 is in the same range; the macro-F1 is 0.04 to 0.12 lower, which is the long tail this adapter had one to twelve examples of per class to learn from. The evaluation sets also differ in size and composition, so read this as context, not a controlled comparison.

Sibling run on the same 700 / 300 split. A Gemma 4 E4B adapter trained by the same platform on this split reached accuracy 0.697 and union macro-F1 0.4587, essentially tied with this checkpoint given the spread above.

Baseline for comparison. Not measured. The untuned Qwen/Qwen3.5-9B was never scored on these 300 rows, so nothing here quantifies how much of the score the fine-tuning is responsible for. This is the most important gap in this card.

This is a validation split the search selected against. 35 trials were scored on these same 300 rows and the best was kept, so expect optimistic bias on top of the re-run spread already described. The rows are drawn from the dataset authors' own validation split, so they are unseen CVEs from the same period as training, not future CVEs.

The evaluation set is reproducible. predictions.jsonl in this repo holds every one of the 300 rows: instruction, description, prediction and gold. The builder's manifest (dataset revision, seed, drop counts, per-class counts) is summarised under Training details.

Limitations and bias

One number, wide error bars. The same configuration scored 0.473 and 0.395 in two runs. Anyone deploying this should re-evaluate on their own data rather than trust either figure.

No baseline, so no established gain. See Evaluation.

Head classes dominate what accuracy measures. CWE-79 alone is 16% of the validation rows. A model that got only the top ten classes right would post a respectable accuracy and a poor macro-F1; the two numbers here disagree by 0.24 for that reason.

Tail classes were barely trained. 117 classes over 700 rows means many classes had a single training example. Expect the model to fall back to a frequent neighbour (CWE-20 for input validation issues, CWE-200 for disclosure) when the description is ambiguous, and to emit ids it saw rarely with low reliability. It also produced 13 ids in evaluation that never occur in the 300 gold rows; some may be reasonable alternative labels, some are wrong.

Only one label. Real CVEs often carry two CWE ids (about 9% of the source dataset). The training data dropped those rows, so the model always commits to one.

Prompt shape is the contract. Change the system prompt, the instruction sentence, the code fence, or enable thinking, and you are evaluating a model nobody measured.

Domain narrowness. English CVE descriptions in NVD / CNA style. Advisories in other formats, other languages, source-code inputs and exploit write-ups are unmeasured.

Inherits all biases and limitations of the base model. This adapter changes 3.3% of the parameters and was not evaluated for safety or fairness. The base model's own card governs those properties.

Environmental impact

Hardware	8x NVIDIA H100 80GB HBM3
Training time	6.46 minutes (387.7625 s)
Cloud provider / region	on-premise

Covers this trial only. The full 46-trial search that selected it took 7 h 17 m on the same hardware.

Framework versions

• PEFT 0.18.1
• TRL: 1.0.0
• Transformers: 5.7.0.dev0 (git main)
• Pytorch: 2.5.1+cu121
• bitsandbytes: 0.49.2
• flash-attn: 2.8.3
• Python: 3.12

PEFT's version is the one recorded in adapter_config.json at save time; the rest are the pinned versions of the training environment. transformers is a git-main build: the qwen3_5 architecture is not in the stable PyPI release.

Citation

@misc{qwen35_cve_cwe_lora_2026,
  title  = {Qwen3.5-9B CVE-to-CWE Classifier LoRA},
  author = {Banerjee, Aaron and Anbuselvan, Pooja and Jodhpurkar, Om},
  year   = {2026},
  url    = {https://huggingface.co/SASVAAI/Qwen-3.5-9B-CVE-to-CWE}
}