KinShield-20b: a QLoRA fine-tune of gpt-oss-20b

A QLoRA adapter for OpenAI's gpt-oss-20b that turns it into KinShield's phone-call evidence detector. KinShield is an entry in the AWS "Zero to Shipped" hackathon. The model reads a call transcript and returns strict JSON with:

• a risk level (LOW / MEDIUM / HIGH) and a score
• every warning sign, with the exact quoted line, a timestamp, one of 7 fixed signals and a weight

It answers directly on the harmony final channel, without writing out reasoning first.

Status: this is a research result. The live KinShield app still uses the stock gpt-oss-20b on Amazon Bedrock. This adapter is not hosted anywhere public.

Results: 63 hand-written benchmark calls (31 scams, 32 safe), none used in training

Model	Correct	Scams rated HIGH	Scams flagged (MEDIUM or HIGH)	Safe calls flagged
gpt-oss-20b, stock (Bedrock, same prompt)	61/63	29/31	31/31	0/32
gpt-oss-120b, stock (Bedrock, reasoning medium)	62/63	30/31	31/31	0/32
gpt-oss-20b + this LoRA	62/63	30/31	31/31	0/32

The adapter matches gpt-oss-120b, a model about 6× larger, and edges out the stock 20b by one call (scam_031). On the original 47-call set, all three score 46/47. The gain is small and the test set is small, so read it as "matches 120b", not "beats it".

This model then taught KinShield-Tiny v3, a 22M int8 classifier. v3 improved from 57/63 to 60/63 over v2, which learned from the stock 20b.

How it was made

1. Data. gpt-oss-20b on Bedrock generated 1,565 synthetic calls: scams, hard negatives and ordinary calls, in English and Hinglish.

2. Stronger labels. gpt-oss-120b (reasoning medium) labelled every call with KinShield's detector prompt and the same server-side normalisation.

3. Intent filter. A label was kept only if it agreed with what the call was written to be: scams must score MEDIUM or HIGH, and safe calls must score LOW. The 120b label was used first, with the 20b label as a fallback. 151 calls that neither model labelled correctly were dropped, leaving 1,201 calls for training and 213 for validation.

4. QLoRA. Trained with Unsloth on an NVIDIA DGX Spark (GB10):

   • 4-bit base model, LoRA rank 16 (alpha 32) on q/k/v/o and the MoE gate_up_proj/down_proj
   • 2 epochs, learning rate 2e-4 with a cosine schedule, effective batch 8, maximum length 3,072
   • loss on the assistant answer only

   Training took 90 minutes and used about 19 GB. Validation loss fell 0.049 → 0.039 → 0.037 → 0.035.

Use

from unsloth import FastLanguageModel
model, tok = FastLanguageModel.from_pretrained("Solomonwilsonr/kinshield-20b",
                                               max_seq_length=3072, load_in_4bit=True)
FastLanguageModel.for_inference(model)
SYSTEM = open("system_prompt.txt").read()  # KinShield detector prompt, included in this repo
prompt = tok.apply_chat_template(
    [{"role": "system", "content": SYSTEM},
     {"role": "user", "content": "[00:00] caller: Grandma, it's me...\n[00:05] victim: Leo?"}],
    tokenize=False, add_generation_prompt=True, reasoning_effort="low") + "<|channel|>final<|message|>"
out = model.generate(**tok(prompt, return_tensors="pt", add_special_tokens=False).to("cuda"),
                     max_new_tokens=800, do_sample=False)

Important: append <|channel|>final<|message|> to the generation prompt, exactly as in training. Without it, the model may start a short analysis channel first.

Limits

• Synthetic data only. All training data is synthetic and LLM-labelled; there are no real calls. The test set is 63 calls written by the same team, so a single call moves accuracy by about 1.6 points.
• Slower than the hosted model. On the DGX Spark (HF generate, 4-bit, batch 8) it takes about 5 s per call. The stock model on Bedrock answers in under 1 s.
• Text only. It handles English and Hinglish transcripts. It doesn't detect voice cloning; it reasons about what the caller asks for.
• Not a safety guarantee. It is a hackathon research prototype. Don't use it to make decisions about real people.