alirezaaminzadeh/petrosafe-rag-retriever

PetroSafe RAG Retriever

Hybrid BM25 + word TF-IDF + character TF-IDF lexical retrieval index over the bilingual (Persian/English) PetroSafe process-safety corpus. CPU-only — not a fine-tuned transformer, no GPU required to build or to serve.

Architecture is unchanged from hse-multimodal-rag-retriever: a word-level TF-IDF (1-2 grams), a character TF-IDF (char_wb 3-5 grams, robust to Persian morphology and typos), and an Okapi BM25 index, combined as 0.42*word + 0.28*char + 0.30*bm25. Query-side keyword overlap adds a small additional boost. This project only supplies new fitted weights over new (oil/gas HSE) bilingual content — see the companion dataset repo for the corpus.

Files

File	Description
hybrid_index.joblib	Fitted TfidfVectorizer (word + char) + BM25 index + chunk-id order
config.json	Index stats (backend, chunk count, vocab sizes)
eval_results.json	Retrieval/abstention metrics for this exact index (see dataset card for detail)

Index stats

{
  "backend": "hybrid_tfidf_bm25",
  "n_chunks": 56,
  "word_vocab": 4341,
  "char_vocab": 14403
}

Evaluation (reproducible via scripts/evaluate.py)

Metric	Value
Retrieval doc-hit rate	96.15%
Keyword hit rate	96.15%
Correct-abstention rate	100% (out-of-scope AND unknown-equipment-tag questions)

Grounding behavior (PLAN.md item 3)

If a question names an equipment-tag-shaped token (e.g. V-101) that has zero matching chunks in the corpus, the retriever's metadata filter excludes every candidate and returns no hits — the generator then abstains automatically rather than answering from an unrelated chunk. This is exercised explicitly in the evaluation set (4 of the 8 negative QA pairs use a plausible-looking but non-existent tag such as V-999, X-888, K-777, R-999).

Companion dataset: alirezaaminzadeh/petrosafe-rag-corpus-fa Space: alirezaaminzadeh/petrosafe-rag-fa