| --- |
| license: mit |
| library_name: custom |
| pipeline_tag: text-classification |
| datasets: |
| - RKB109/rag-evaluation-lab-20260720-dataset |
| tags: |
| - synthetic-data |
| - transparent-baseline |
| - ai-evaluation |
| - text-classification |
| - question-answering |
| - text-ranking |
| - summarization |
| metrics: |
| - accuracy |
| --- |
| |
| # RAG Evaluation Lab Baseline Model |
|
|
| ## Model Description |
|
|
| This repository contains a small, transparent prototype model for |
| **RAG systems often ship without a stable regression set or failure taxonomy.** |
|
|
| The model combines per-label token weights with IDF-weighted evidence |
| retrieval. It was generated for reproducible architecture demonstrations and |
| does not call a hosted LLM. |
|
|
| ## Evaluation |
|
|
| - Held-out synthetic examples: 4 |
| - Accuracy: 0.75 |
| - Intended metrics: failure_class_accuracy, citation_coverage, release_gate_pass_rate |
|
|
| ## Intended Use |
|
|
| - Architecture prototyping |
| - CI and evaluation examples |
| - Local baseline comparisons |
| - Educational experimentation |
|
|
| ## Hugging Face Task Coverage |
|
|
| - `text-classification` |
| - `question-answering` |
| - `text-ranking` |
| - `summarization` |
|
|
| ## Limitations and Risks |
|
|
| Synthetic cases validate the harness, not a production RAG system. Teams must add representative domain examples. |
|
|
| The dataset is synthetic and small. Do not use this model for consequential |
| decisions without representative data, expert review, and production-grade |
| evaluation. |
|
|
| ## Reproducibility |
|
|
| The linked GitHub repository includes `train.py`, the exact dataset split, |
| evaluation code, and the model JSON format. |
|
|
