Add model card: usage, prompt format, training details

README.md

@@ -1,199 +1,167 @@
 ---
+license: apache-2.0
+base_model: Qwen/Qwen3-4B-Instruct-2507
+tags:
+- martech
+- analytics
+- event-taxonomy
+- json
+- structured-output
+- sft
+- lora
 library_name: transformers
-tags: []
+pipeline_tag: text-generation
 ---
 
-# Model Card for Model ID
+# Qwen3-4B EventSpec — MarTech Event Taxonomy Generator (merged)
 
-<!-- Provide a quick summary of what the model is/does. -->
+Fine-tuned **Qwen/Qwen3-4B-Instruct-2507** that converts free-form marketing tracking
+requests into clean, implementation-ready **analytics event specifications** as strict JSON.
+
+This repository contains the **fully merged weights** (LoRA adapter merged into the base
+model), so it loads like any standard model — no PEFT/adapter step required.
+
+- **Base model:** [Qwen/Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507)
+- **LoRA adapter (pre-merge):** [joshelu/qwen3-4b-Instruct-eventspec-martech-sft](https://huggingface.co/joshelu/qwen3-4b-Instruct-eventspec-martech-sft)
+- **Method:** Supervised Fine-Tuning (SFT) + LoRA, then merged
+- **Task:** marketing tracking request → strict-JSON EventSpec
 
+## Intended use
 
+Give the model a plain-English marketing/analytics tracking request; it returns a single
+JSON object specifying the events to implement (event names, parameters, triggers, consent
+and deduplication requirements, QA criteria, risk flags, etc.). Useful for MarTech / analytics
+engineering, GA4 / GTM instrumentation planning, and taxonomy standardization.
 
-## Model Details
+## ⚠️ Prompt format (required for correct output)
 
-### Model Description
+The model was trained with a specific chat format. **You must reproduce it** or output
+quality degrades sharply.
 
-<!-- Provide a longer summary of what this model is. -->
+**System prompt (use verbatim):**
+
+```
+You are EventSpec, an expert MarTech analytics engineer. You convert free-form marketing tracking requests into clean, implementation-ready analytics event specifications.
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+Given a marketing tracking request, respond with a SINGLE valid JSON object and nothing else: no prose, no markdown, no code fences. The JSON must be strictly parseable.
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+The specification captures: a concise `request_summary`; the `business_goal`; the `tracking_scope` (platforms, page_or_screen, user_action, conversion_type); and a list of `recommended_events`. Each recommended event defines `event_name` (snake_case), `event_description`, `platform`, `event_type`, `required_parameters`, `optional_parameters`, `trigger_condition`, `consent_requirements`, and `deduplication_requirements`.
 
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
+Use consistent snake_case event and parameter names, follow analytics best practices (GA4/GTM conventions where relevant), and respect privacy/consent requirements. Output only the JSON object.
+```
+
+**User message format:**
+
+```
+Convert this marketing tracking request into a clean analytics event specification.
+
+Request:
+<your tracking request here>
+```
+
+## Usage (transformers)
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+repo = "joshelu/qwen3-4b-eventspec-martech-merged"
+tok = AutoTokenizer.from_pretrained(repo)
+model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")
+
+SYSTEM_PROMPT = (
+    "You are EventSpec, an expert MarTech analytics engineer. You convert free-form "
+    "marketing tracking requests into clean, implementation-ready analytics event "
+    "specifications.\n\n"
+    "Given a marketing tracking request, respond with a SINGLE valid JSON object and "
+    "nothing else: no prose, no markdown, no code fences. The JSON must be strictly "
+    "parseable.\n\n"
+    "The specification captures: a concise `request_summary`; the `business_goal`; the "
+    "`tracking_scope` (platforms, page_or_screen, user_action, conversion_type); and a "
+    "list of `recommended_events`. Each recommended event defines `event_name` "
+    "(snake_case), `event_description`, `platform`, `event_type`, `required_parameters`, "
+    "`optional_parameters`, `trigger_condition`, `consent_requirements`, and "
+    "`deduplication_requirements`.\n\n"
+    "Use consistent snake_case event and parameter names, follow analytics best "
+    "practices (GA4/GTM conventions where relevant), and respect privacy/consent "
+    "requirements. Output only the JSON object."
+)
+USER_PREFIX = "Convert this marketing tracking request into a clean analytics event specification."
+
+request = ("A pharmacy app wants to track refill reminders, refill started, refill submitted, "
+           "refill ready, and pickup completed, but no medication names or prescription numbers "
+           "should be sent.")
+
+messages = [
+    {"role": "system", "content": SYSTEM_PROMPT},
+    {"role": "user", "content": f"{USER_PREFIX}\n\nRequest:\n{request}"},
+]
+prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tok(prompt, return_tensors="pt").to(model.device)
+
+out = model.generate(**inputs, max_new_tokens=2048, do_sample=False,
+                     pad_token_id=tok.eos_token_id)
+print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
+```
+
+## Usage (Inference Endpoints / chat_completion)
+
+```python
+from huggingface_hub import InferenceClient
+client = InferenceClient("https://YOUR-ENDPOINT.endpoints.huggingface.cloud", token="hf_...")
+resp = client.chat_completion(
+    messages=[
+        {"role": "system", "content": SYSTEM_PROMPT},
+        {"role": "user", "content": f"{USER_PREFIX}\n\nRequest:\n{request}"},
+    ],
+    max_tokens=2048,
+    temperature=0,
+)
+print(resp.choices[0].message.content)
+```
+
+## Recommended generation settings
+
+- `temperature = 0` (greedy) — best for stable, strictly parseable JSON.
+- `max_new_tokens >= 2048` — specs are long; a lower limit will truncate the JSON mid-string.
+- Parse the output with `json.loads`; retry with a higher token limit if parsing fails.
+
+## Output schema
+
+The model produces a single JSON object. Core keys:
+
+- `request_summary` — one-line summary of the request.
+- `business_goal` — the measurement objective.
+- `tracking_scope` — `{ platforms, page_or_screen, user_action, conversion_type }`.
+- `recommended_events` — list of events, each with `event_name` (snake_case),
+  `event_description`, `platform`, `event_type`, `required_parameters`,
+  `optional_parameters`, `trigger_condition`, `consent_requirements`,
+  `deduplication_requirements`.
+
+Richer examples in the training data also include `implementation_notes`, `qa_criteria`,
+`open_questions`, and `risk_flags`, which the model produces as appropriate.
+
+## Training
+
+- **Dataset:** `joshelu/martech-event-taxonomy-mapper-training-data` (private).
+  - The 100-row `train` split, **minus** every `id` appearing in the `validation` or `test`
+    split → **80 training rows** (held-out eval stays honest).
+  - Eval during training used the `validation` split (10 rows).
+  - Assistant targets are compact, strict JSON (`json.dumps(output, separators=(",", ":"))`),
+    validated as parseable before training.
+- **Method:** SFT + LoRA, then merged.
+  - LoRA: `r=16`, `alpha=32`, `dropout=0.05`,
+    target modules `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`.
+  - 3 epochs, effective batch size 4 (per-device 1 × grad-accum 4), lr `2e-4`,
+    warmup ratio `0.05`, cosine schedule, max length 2048, bf16, gradient checkpointing.
+  - Hardware: a10g-small (HF Jobs).
+- **Metrics (final):** eval loss ≈ `0.320`, eval mean token accuracy ≈ `0.924`.
+
+## Limitations
+
+- Trained on a small (80-row) dataset — coverage is limited to the taxonomy and styles seen
+  in training; unusual domains may produce weaker specs.
+- Output is a strong **draft**, not a substitute for review by an analytics engineer,
+  especially for consent/privacy and PII handling.
+- With very low `max_new_tokens`, long specs will be truncated and fail JSON parsing — keep
+  the limit high and validate the parse.