--- license: apache-2.0 language: - en base_model: - Qwen/Qwen3.6-27B pipeline_tag: text-generation library_name: transformers tags: - climate - climate-change - wind-energy - climate-discourse - opposition-detection - classification - qwen3 - fine-tuned - cards - joint - quantized - fp8 - image-text-to-text - multimodal - vision-language datasets: - C3DS/cards_sft_dataset --- # CARDS-Wind-Qwen3.6-27B-FP8 **Joint** climate-discourse classifier — a single Qwen3.6-27B backbone fine-tuned on **CARDS + Wind concatenated**, then FP8-dynamic quantized. One model handles both tasks: - **CARDS** — classification of climate-contrarian claims under the [Coan et al. (2025)](https://doi.org/10.1038/s44458-025-00029-z) hierarchical taxonomy. - **Wind** — three-level wind-energy opposition classification (detection / frames / claims). The model picks the right task from the system prompt: pass the CARDS system prompt for CARDS-style output, the Wind system prompt for Wind-style output. Same weights, same chat template. This is the FP8 deployment variant — ~27 GB on disk, fits on a single A100/H100/H200. ## Why a joint model - **One checkpoint, two tasks.** Saves disk + VRAM + ops complexity if you serve both classifiers in production. - **Same recipe, additive cost.** Trained with `cards/ft/train.py --joint` — Unsloth + LoRA r=16 α=16, 3 epochs, lr=2e-4 — identical to the cards FT models, with the wind dataset added to the training mix. - **Wind paper is forthcoming.** This model accompanies a forthcoming C3DS paper on wind-energy opposition discourse — research-preview status until the paper is out. ## Results (Wind test set) Evaluated on the wind-opposition test set (773 rows, 436 opposition-positive). Compared with the wind-only sibling, the BF16 joint model, and frontier APIs: ### Detection (binary) | Metric | Windy-27B FP8 | CARDS-Wind-27B (BF16) | **CARDS-Wind-27B (FP8 — this model)** | Claude Opus 4.7 | GPT-5.5 | |---|---|---|---|---|---| | Precision | 0.877 | 0.863 | **0.866** | 0.896 | **0.927** | | Recall | 0.920 | 0.911 | **0.917** | 0.890 | 0.846 | | F1 | **0.898** | 0.886 | **0.891** | 0.893 | 0.885 | ### Samples F1 (multi-label frame / claim accuracy) | View | Windy-27B FP8 | CARDS-Wind-27B (BF16) | **CARDS-Wind-27B (FP8)** | Claude Opus 4.7 | GPT-5.5 | |---|---|---|---|---|---| | Frames — all rows | 0.787 | 0.770 | **0.772** | 0.791 | 0.792 | | Frames — opposition only | 0.751 | 0.736 | **0.739** | 0.734 | 0.697 | | Claims — all rows | 0.755 | 0.733 | **0.738** | 0.754 | 0.745 | | Claims — opposition only | 0.694 | 0.668 | **0.677** | 0.667 | 0.614 | - **Joint training costs ~0.007 detection F1** vs the wind-only `Windy-27B-FP8` (0.891 vs 0.898) — small but real. - Still **ties or beats Claude Opus 4.7 on detection F1** (0.891 vs 0.893) and beats it on frames-opposition-only and claims-opposition-only samples F1. - **FP8 ≈ BF16** on every metric — quantization is effectively free here. - Zero parse failures on 773 test items. ### CARDS test set CARDS-side metrics for the joint model are **not separately reported** in this release — the cards-only sibling [`C3DS/CARDS-Qwen3.6-27B`](https://huggingface.co/C3DS/CARDS-Qwen3.6-27B) is the canonical reference for CARDS test-set numbers (samples F1 = 0.893 at L1, ties Opus 4.6). The joint model uses an identical CARDS training setup; expect comparable performance, but treat as approximate until reproduced. ## Usage ### Routing between the two tasks The model picks its behaviour from the system prompt. Both prompts are bundled in this repo: - [`cards_prompts.json`](./cards_prompts.json) — `slim_system_instruction` (CARDS) + `cot_trigger` - [`wind_prompts.json`](./wind_prompts.json) — `slim_system_instruction` (Wind) ### With vLLM ```bash vllm serve C3DS/CARDS-Wind-Qwen3.6-27B-FP8 \ --port 8000 \ --max-model-len 4096 \ --enable-prefix-caching \ --kv-cache-dtype fp8 \ --served-model-name CARDS-Wind-Qwen3.6-27B ``` ```python import json from huggingface_hub import hf_hub_download from openai import OpenAI cards = json.load(open(hf_hub_download("C3DS/CARDS-Wind-Qwen3.6-27B-FP8", "cards_prompts.json"))) wind = json.load(open(hf_hub_download("C3DS/CARDS-Wind-Qwen3.6-27B-FP8", "wind_prompts.json"))) client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy") def classify_cards(text): resp = client.chat.completions.create( model="CARDS-Wind-Qwen3.6-27B", messages=[ {"role": "system", "content": cards["slim_system_instruction"]}, {"role": "user", "content": f"### Text:\n{text}\n\n{cards['cot_trigger']}"}, ], temperature=0, max_tokens=4000, ) return resp.choices[0].message.content def classify_wind(text): resp = client.chat.completions.create( model="CARDS-Wind-Qwen3.6-27B", messages=[ {"role": "system", "content": wind["slim_system_instruction"]}, {"role": "user", "content": text}, ], temperature=0, max_tokens=4000, ) return resp.choices[0].message.content ``` Both modes produce a `…` reasoning trace followed by a YAML block. CARDS output is `categories: [...]`; Wind output is `opposition_detected`, `frames`, `claims`. ### Multimodal — image + text The base Qwen3.6-27B supports image inputs via the OpenAI-compatible `image_url` content part, and this fine-tune preserves that capability for both tasks. Switch tasks by switching the system prompt — CARDS prompt for CARDS-style output, Wind prompt for Wind-style output — and pass an image (with or without caption text) alongside. Serve vLLM with multimodal flags enabled: ```bash vllm serve C3DS/CARDS-Wind-Qwen3.6-27B-FP8 \ --port 8000 \ --max-model-len 8192 \ --trust-remote-code \ --limit-mm-per-prompt image=4 \ --enable-prefix-caching \ --kv-cache-dtype fp8 \ --served-model-name CARDS-Wind-Qwen3.6-27B ``` ```python import base64, json, mimetypes from pathlib import Path from huggingface_hub import hf_hub_download from openai import OpenAI cards = json.load(open(hf_hub_download("C3DS/CARDS-Wind-Qwen3.6-27B-FP8", "cards_prompts.json"))) wind = json.load(open(hf_hub_download("C3DS/CARDS-Wind-Qwen3.6-27B-FP8", "wind_prompts.json"))) def image_part(path): p = Path(path) mime = mimetypes.guess_type(p)[0] or "image/png" b64 = base64.b64encode(p.read_bytes()).decode() return {"type": "image_url", "image_url": {"url": f"data:{mime};base64,{b64}"}} client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy") # Wind classification on an image: resp = client.chat.completions.create( model="CARDS-Wind-Qwen3.6-27B", messages=[ {"role": "system", "content": wind["slim_system_instruction"]}, {"role": "user", "content": [ {"type": "text", "text": "Read the image and any caption below; classify the wind-opposition framing depicted."}, image_part("screenshot.png"), {"type": "text", "text": "### Caption:\n"}, ]}, ], temperature=0, max_tokens=4000, ) print(resp.choices[0].message.content) ``` ## Training & Quantization ### Joint fine-tuning - **Base model:** `Qwen/Qwen3.6-27B` - **Method:** LoRA (rank 16, α 16, dropout 0) on `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`, then merged into base weights - **Datasets:** [`C3DS/cards_sft_dataset`](https://huggingface.co/datasets/C3DS/cards_sft_dataset) (CARDS RECoT messages) + the wind RECoT messages corpus (forthcoming release) - **Mix:** datasets concatenated row-wise — no balancing or task-token; the model learns to pick its task from the system prompt - **Framework:** Unsloth + TRL `SFTTrainer`, invoked via `cards/ft/train.py --joint` - **Hyperparameters:** 3 epochs, `per_device_train_batch_size=1`, `gradient_accumulation_steps=8`, `lr=2e-4`, cosine schedule, 10 warmup steps, `max_seq_length=8192`, `adamw_8bit`, `bf16` ### FP8 quantization - **Scheme:** `fp8_e4m3` dynamic per-channel quantization (weights only). Activations stay in BF16; no calibration data required. - **Targets:** linear layers in transformer blocks; `lm_head` left in BF16. - **Tool:** `llmcompressor` with `QuantizationModifier(targets="Linear", scheme="FP8_DYNAMIC")` applied to the merged BF16 joint checkpoint. ## Limitations - **Forthcoming Wind paper.** Wind-side methodology, codebook, and dataset details are pending publication. - **CARDS-side metrics not separately re-evaluated** for the joint model — refer to the cards-only sibling for the canonical CARDS test-set numbers. - **Joint training trade-off.** Detection F1 on Wind is marginally lower than the wind-only `Windy-Qwen3.5-27B-FP8` (0.891 vs 0.898). Use the dedicated wind model if absolute wind detection F1 is the priority; use this one if you need both tasks from a single backbone. - **Thinking tokens.** Training used `enable_thinking=True`. Parse output after `` or disable thinking at inference. ## Related models - [`C3DS/CARDS-Qwen3.6-27B`](https://huggingface.co/C3DS/CARDS-Qwen3.6-27B) — CARDS-only Qwen3.6-27B sibling (same backbone, CARDS-only training). - [`C3DS/Windy-Qwen3.5-27B-FP8`](https://huggingface.co/C3DS/Windy-Qwen3.5-27B-FP8) — Wind-only 27B FP8 sibling (Qwen3.5 backbone). ## Citation For the CARDS side of this model, please cite: ```bibtex @article{coan2025cards, title = {Large language model reveals an increase in climate contrarian speech in the United States Congress}, author = {Coan, Travis G. and Malla, Ranadheer and Nanko, Mirjam O. and Kattrup, William and Roberts, J. Timmons and Cook, John and Boussalis, Constantine}, journal = {Communications Sustainability}, volume = {1}, pages = {37}, year = {2025}, doi = {10.1038/s44458-025-00029-z} } ``` A wind-side citation will be added when the corresponding paper is published. ## License Apache 2.0, inherited from Qwen3.6-27B.