style: update readme

README.md

@@ -8,4 +8,168 @@ pipeline_tag: text-classification
 tags:
 - Speech Recognition
 - ATC
----
+---
+
+# ATC Communication Expert Model
+
+A fine-tuned model specialized in improving and analyzing Air Traffic Control (ATC) communications, extracting relevant information from raw transcripts.
+
+## Model Details
+
+### Model Description
+
+This model is a fine-tuned version of Llama-3.2-3B-Instruct optimized for processing Air Traffic Control communications. It can:
+
+- Improve raw ATC transcripts with proper punctuation and formatting
+- Identify communication intentions (pilot requests, ATC instructions, etc.)
+- Extract key information such as flight numbers, altitudes, headings, and other numerical data
+- Analyze speaker roles and communication patterns
+
+The model was fine-tuned using LoRA (Low-Rank Adaptation) with PEFT (Parameter-Efficient Fine-Tuning) techniques to efficiently adapt the Llama 3.2 model to this specialized domain.
+
+- **Developed by:** ATC NLP Team
+- **Model type:** Fine-tuned Llama 3.2 with LoRA adapters
+- **Language(s):** English, specialized for ATC terminology
+- **License:** Same as the base model (Llama 3.2)
+- **Finetuned from model:** meta-llama/Llama-3.2-3B-Instruct
+
+## Uses
+
+### Direct Use
+
+This model is intended for:
+- Transcribing and formatting raw ATC communications
+- Training ATC communication skills
+- Analyzing ATC communication patterns
+- Extracting structured data from ATC communications
+- Educational purposes for those learning ATC communication protocols
+
+### Downstream Use
+
+The model can be integrated into:
+- Air traffic management training systems
+- Communication analysis tools
+- ATC transcript post-processing pipelines
+- Aviation safety monitoring systems
+- Radio communication enhancement systems
+
+### Out-of-Scope Use
+
+This model is not suitable for:
+- Real-time ATC operations or safety-critical decision-making
+- Full language translation (it's specialized for ATC terminology only)
+- General language processing outside the ATC domain
+- Any application where model errors could impact flight safety
+
+## Bias, Risks, and Limitations
+
+- The model is specialized for ATC communications and may not perform well on general text
+- It may have limitations with accents or non-standard ATC phraseology
+- Performance depends on audio transcription quality for real-world applications
+- Not intended for safety-critical applications without human verification
+- May have biases based on the training data distribution
+
+### Recommendations
+
+- Always have human verification for safety-critical applications
+- Use in conjunction with standard ATC protocols, not as a replacement
+- Provide clear domain context for optimal performance
+- Test thoroughly with diverse ATC communications before deployment
+- Consider fine-tuning further on your specific ATC subdomain if needed
+
+## How to Get Started with the Model
+
+```python
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+# Load the model with LoRA adapters
+base_model = AutoModelForCausalLM.from_pretrained(
+    "meta-llama/Llama-3.2-3B-Instruct",
+    torch_dtype="auto",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")
+model = PeftModel.from_pretrained(base_model, "path_to_adapters")
+
+# Alternatively, use the merged model if available
+# model = AutoModelForCausalLM.from_pretrained("path_to_merged_model")
+# tokenizer = AutoTokenizer.from_pretrained("path_to_merged_model")
+
+# Process an ATC message
+instruction = "As an ATC communication expert, improve this transcript and analyze its intentions and data."
+message = "southwest five niner two turn left heading three four zero descend and maintain flight level two five zero"
+
+prompt = f"<|begin_of_text|><|header_start|>user<|header_end|>\n\n{instruction}\n\nOriginal: {message}<|eot|><|header_start|>assistant<|header_end|>\n\n"
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+# Generate improved transcript and analysis
+outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
+response = tokenizer.decode(outputs[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
+print(response)
+```
+
+## Training Details
+
+### Training Data
+
+The model was trained on a dataset of ATC communications with:
+- Original raw transcripts
+- Properly punctuated and formatted versions
+- Annotated intentions (PSC, PSR, PRP, PRQ, PRB, PAC, ASC, AGI, ACR, END)
+- Extracted numerical data (altitudes, headings, flight numbers, etc.)
+- Speaker and listener information
+
+### Training Procedure
+
+The model was fine-tuned using LoRA with the following approach:
+- Parameter-efficient fine-tuning using PEFT
+- LoRA applied to key attention layers (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)
+- Optimized with Unsloth for efficiency
+
+#### Training Hyperparameters
+
+- **Base model:** meta-llama/Llama-3.2-3B-Instruct
+- **LoRA rank:** 16
+- **LoRA alpha:** 16
+- **Learning rate:** 2e-4
+- **Batch size:** 4
+- **Gradient accumulation steps:** 4
+- **Epochs:** 3
+- **Warmup ratio:** 0.03
+- **Max sequence length:** 2048
+- **Training regime:** BF16 mixed precision where available, FP16 otherwise
+- **Optimizer:** AdamW 8-bit
+
+## Evaluation
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+The model was tested on a diverse set of ATC communications, including:
+- Clearances and instructions
+- Pilot requests and reports
+- Emergency communications
+- Different accents and speaking patterns
+
+## Technical Specifications
+
+### Model Architecture and Objective
+
+- **Base architecture:** Llama-3.2-3B-Instruct
+- **Fine-tuning method:** LoRA with PEFT
+- **Optimization library:** Unsloth
+- **Training objective:** Improving and analyzing ATC communications
+
+### Compute Infrastructure
+
+- **Framework versions:**
+  - PEFT 0.15.2
+  - Unsloth (latest version used during training)
+  - Transformers (compatible with the base model)
+  - PyTorch (with BF16 support where available)
+
+## Model Card Contact
+
+For issues or questions about this model, please open an discussion in the repository.