LLM text evaluation complement for Arabic ASR pipeline post-processing
Hi ELM Research Center team 👋
Universal Arabic ASR evaluation is important work. For teams building Arabic ASR pipelines with LLM post-processing (diacritization, summarization, QA), I built an evaluation framework for the LLM layer.
LLM Evaluation Framework covers the text generation component:
→ 🔍 Hallucination Rate — LLMs processing Arabic ASR output can hallucinate when transcription is unclear
→ 🎯 Accuracy — text task accuracy on Arabic content
→ 💰 Cost per 1K tokens — Arabic text processing at scale, cost planning
→ ⚡ Latency p95 — real-time Arabic speech + LLM pipeline latency budget
→ 🧠 Reasoning Quality — for summarization and explanation tasks
Live demo: https://huggingface.co/spaces/vigneshwar234/llm-eval-demo
GitHub: https://github.com/vignesh2027/LLM-Evaluation-Framework
Would love to discuss Arabic ASR + LLM pipeline evaluation!