jgalego's picture
Upload folder using huggingface_hub
ba2f43e verified
|
Raw History Blame Contribute Delete
3.83 kB
---
base_model: Qwen/Qwen3.5-0.8B
datasets:
- DEEL-AI/NOTAM
- AirsideLabs/notam-typed-decisions
language: en
library_name: transformers
license: apache-2.0
pipeline_tag: text-generation
tags:
- aviation
- notam
- text-classification
model-index:
- name: notam-subject-qwen3.5-0.8b
results:
- task:
type: text-classification
dataset:
name: NOTAM subject, template-disjoint
type: AirsideLabs/notam-typed-decisions
metrics:
- type: accuracy
value: 0.948
name: Accuracy
- type: macro_f1
value: 0.942
name: Macro-F1
---
# notam-subject-qwen3.5-0.8b
[Qwen/Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) fine-tuned with LoRA to name the subject of a NOTAM: one of the 13 classes in [DEEL-AI/NOTAM](https://huggingface.co/datasets/DEEL-AI/NOTAM).
> [!WARNING]
> **Research aid only. Not for flight operations.** Not certified or validated for flight planning, dispatch or pilot briefing. Always read the original NOTAM.
> [!NOTE]
> **What is a NOTAM?** A NOTAM is a short, standardized notice that alerts flight crews and dispatchers to a condition that may affect a flight, such as a closed runway, an unavailable navigation aid or a temporary airspace restriction. The compressed wording can be cryptic: `RWY 09/27 CLSD DUE TO WIP` means runway 09/27 is closed because work is in progress.
## 🚀 Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "jgalego/notam-subject-qwen3.5-0.8b"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto")
messages = [
{"role": "system", "content": "Name the main subject of the NOTAM. Answer with one of: Airspaces, Communication and Radar, GPS, Landing Navaids, Obstacles, Parking, Procedure, Runway, Services and Facilities, Signs and Lights, Taxiway, Terminal or Enroute Navaids, Wildlife."},
{"role": "user", "content": "RWY 09/27 CLSD DUE TO WIP"},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
enable_thinking=False,
return_tensors="pt",
return_dict=True,
)
out = model.generate(**inputs, max_new_tokens=12, do_sample=False)
print(tokenizer.decode(out[0, inputs["input_ids"].shape[1] :], skip_special_tokens=True))
```
Labels: Airspaces, Communication and Radar, GPS, Landing Navaids, Obstacles, Parking, Procedure, Runway, Services and Facilities, Signs and Lights, Taxiway, Terminal or Enroute Navaids, Wildlife.
## 🏋️ Training
| | |
|---|---|
| Base | [Qwen/Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) |
| Data | [DEEL-AI/NOTAM](https://huggingface.co/datasets/DEEL-AI/NOTAM), train split |
| Method | LoRA, r=16, alpha=32, all linear layers, merged |
| Steps | 522 (2.0 epochs), batch size 16 |
| Learning rate | 0.0002, cosine |
| Mean train loss | 0.0672 |
| Hardware | NVIDIA A10G, 16 min |
## 📊 Results
Template-disjoint group of [AirsideLabs/notam-typed-decisions](https://huggingface.co/datasets/AirsideLabs/notam-typed-decisions): NOTAMs whose templates do not appear in training. Greedy decoding; answers outside the label set count as wrong.
| Model | n | Accuracy | Macro-F1 | Invalid |
|---|---|---|---|---|
| jgalego/notam-subject-qwen3.5-0.8b | 3252 | 0.948 | 0.942 | 1 |
| Qwen/Qwen3.5-0.8B | 3252 | 0.195 | 0.117 | 13 |
Reference baselines reported by [Airside Labs](https://huggingface.co/datasets/AirsideLabs/notam-typed-decisions/blob/main/baselines.md) on the same group (n=3,252):
| Model | Accuracy | Macro-F1 |
|---|---|---|
| TF-IDF + logistic regression | 0.936 | 0.930 |
| Qwen3.5-4B, self-distilled | 0.904 | 0.894 |
| Qwen3.5-4B, zero-shot | 0.787 | 0.750 |
## ⚠️ Limitations
- Trained on 13 fixed classes; NOTAMs about anything else get the nearest one.
- English ICAO-style NOTAM text only.