README.md

metadata

title: Aqeedah-ao
emoji: 🕌
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 5.49.1
app_file: app.py
pinned: false

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

---

license: mit task_categories: - question-answering - text-retrieval language: - ar tags: - aqeedah - islamic-theology - arabic - rag - faiss - hybrid-search size_categories: - n<1K pretty_name: Aqeedah RAG Dataset configs: - config_name: default data_files: - split: train path: data/train-* dataset_info: features: - name: content dtype: string - name: meta struct: - name: author_name dtype: string - name: doc_name dtype: string - name: paragraph_number dtype: int64 - name: embeddings list: float64 splits: - name: train num_bytes: 43231133 num_examples: 5419 download_size: 30017136 dataset_size: 43231133

Aqeedah AI Assistant �

A Research Initiative by Najran University, Kingdom of Saudi Arabia

---

🚀 Quick Start

Try the Live Demo

🌐 Live Chatbot: https://huggingface.co/spaces/abdullah-alamodi/aqeedah-ai

Run Locally

1. Clone the repository:

git clone https://github.com/Abdullah-Alamodi/aqeedah-ai.git
cd aqeedah-ai

2. Install dependencies:

pip install -r requirements.txt

3. Set up environment variables: Create a .env file in the project root:

GEMINI_API_KEY=your_gemini_api_key_here

4. Run the chatbot:

python app.py

The app will launch at http://localhost:7860

Project Structure

aqeedah-ai/
├── app.py                 # Gradio chatbot interface
├── retrieval.py          # Hybrid RAG retrieval system
├── requirements.txt      # Python dependencies
├── rag_playground.ipynb  # Dataset preparation notebook
├── aqeedah_kb.json       # Source data (5419 paragraphs)
├── aqeedah_kb/           # Original source documents (DOCX/PDF)
├── .env                  # API keys (not in git)
├── .gitignore           
├── pyproject.toml        # Project configuration
└── README.md             # This file

Key Features:

• ✅ Simple Structure: Only 2 main Python files (app.py + retrieval.py)
• ✅ Cloud-First: Loads dataset from HuggingFace (no local .bin files)
• ✅ Production-Ready: Deployed on HF Spaces with Gradio
• ✅ Hybrid Retrieval: BM25 + Dense embeddings (AraBERT)

---

Aqeedah RAG Dataset 📚

A curated Arabic Islamic theology (Aqeedah) dataset with pre-computed FAISS embeddings, designed for advanced Retrieval-Augmented Generation (RAG) applications in Islamic scholarly research.

📋 Dataset Description

This dataset represents a specialized collection of 5419 paragraphs from authoritative Islamic theology texts, meticulously compiled and structured for computational analysis. The corpus focuses specifically on Aqeedah (Islamic creed), covering foundational topics in Islamic belief and theology.

Key Features:

• Authentic Arabic content with complete diacritics (Tashkeel) preserved for linguistic accuracy
• Pre-computed semantic embeddings (768-dimensional dense vectors) using state-of-the-art Arabic language models
• Rich scholarly metadata including source document names, author attributions, and precise paragraph references
• Optimized FAISS index for millisecond-scale semantic similarity search
• Hybrid retrieval support combining traditional keyword-based (BM25) and modern neural approaches

🎓 Research Context

This dataset was developed as part of an academic research initiative at Najran University, Kingdom of Saudi Arabia, under the supervision of Dr. Alya Alamodi, a distinguished scholar holding a Ph.D. in Islamic Theology (Aqeedah). Dr. Alamodi's expertise in classical Islamic sciences combined with modern computational approaches has shaped the careful curation and theological accuracy of this corpus.

The technical implementation and AI infrastructure were designed and developed by Abdullah Alamodi, M.Sc. candidate in Artificial Intelligence at IU International University of Applied Sciences, Germany. This collaboration represents an interdisciplinary effort bridging traditional Islamic scholarship with cutting-edge natural language processing and information retrieval technologies.

Research Objectives

1. Democratizing Access: Making authoritative Aqeedah knowledge computationally accessible for researchers and students
2. Semantic Search: Enabling meaning-based retrieval beyond keyword matching in classical Arabic texts
3. AI-Assisted Learning: Supporting intelligent question-answering systems for Islamic education
4. Scholarly Validation: Establishing benchmarks for Arabic NLP in religious domain-specific applications

📚 Source Texts

This dataset comprises carefully selected paragraphs from the following authoritative Islamic theology works:

1. شرح الطحاوية (Sharh al-Tahawiyyah) - صدر الدين محمد بن علاء الدين علي بن محمد ابن أبي العز الحنفي (Volumes 1-2)
2. كتاب التوحيد (Kitab al-Tawhid) - محمد بن عبد الوهاب
3. شرح العقيدة الواسطية (Sharh al-Aqidah al-Wasitiyyah) - محمد بن صالح بن محمد العثيمين (Volumes 1-2)
4. القول المفيد على كتاب التوحيد (Al-Qawl al-Mufid 'ala Kitab al-Tawhid) - محمد بن صالح بن محمد العثيمين (Volumes 1-4)
5. القول السديد شرح كتاب التوحيد (Al-Qawl al-Sadid Sharh Kitab al-Tawhid) - عبد الرحمن بن ناصر السعدي
6. أصول الإيمان (Usul al-Iman) - عبد العزيز بن عبد الله بن باز
7. الوجيز في عقيدة السلف الصالح أهل السنة والجماعة (Al-Wajiz fi Aqidah al-Salaf al-Salih) - عبد الله بن عبد الحميد الأثري
8. الإسلام أصوله ومبادئه (Al-Islam: Usuluhu wa Mabadi'uhu) - محمد بن عبد الله بن صالح السحيم
9. فتاوى نور على الدرب (Fatawa Nur 'ala al-Darb) - محمد بن صالح بن محمد العثيمين (Volumes 1-4)

🗂️ Dataset Structure

Data Fields

• paragraph_text (string): The Arabic text content with complete diacritical marks
• doc_name (string): Title of the source Islamic text
• author_name (string): Name of the classical or contemporary scholar
• paragraph_number (int): Sequential paragraph identifier within the source document
• embeddings (list of float): Pre-computed 768-dimensional embedding vector (L2-normalized)

Data Splits

This dataset contains a single split with 5419 carefully selected paragraphs from verified Islamic theology sources.

🤖 Embedding Model

Model: aubmindlab/bert-base-arabertv02

Technical Specifications:

• Architecture: BERT-Base (12 layers, 768 hidden dimensions)
• Pre-training: Arabic Wikipedia + other Arabic corpora
• Embedding Dimension: 768
• Text Normalization: Light preprocessing (preserves diacritics for theological accuracy)
• Pooling Strategy: Attention-masked average pooling
• Vector Normalization: L2 normalization for cosine similarity compatibility

🚀 Usage

Installation

pip install datasets faiss-cpu torch transformers pyarabic rank-bm25

Quick Start

from datasets import load_dataset
import torch
from transformers import AutoTokenizer, AutoModel
import pyarabic.araby as araby

# Load dataset with FAISS index
dataset = load_dataset("abdullah-alamodi/aqeedah-rag-dataset")

# Load the embedding model
model_name = "aubmindlab/bert-base-arabertv02"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

# Add FAISS index for fast retrieval
dataset['train'].add_faiss_index(column="embeddings")

# Helper function for embedding
def get_embedding(text):
    normalized = araby.normalize_hamza(text)
    text_input = f"query: {normalized}"
    
    inputs = tokenizer([text_input], padding=True, truncation=True, 
                      max_length=512, return_tensors='pt')
    
    with torch.no_grad():
        outputs = model(**inputs)
    
    # Average pooling
    embeddings = outputs.last_hidden_state.mean(dim=1)
    embeddings = torch.nn.functional.normalize(embeddings, p=2, dim=1)
    
    return embeddings[0].numpy()

# Search example
query = "ما معنى شهادة أن لا إله إلا الله؟"
query_embedding = get_embedding(query)

# Find top 5 similar documents
scores, retrieved = dataset['train'].get_nearest_examples(
    "embeddings", 
    query_embedding, 
    k=5
)

# Display results
for i, (score, text, meta) in enumerate(zip(
    scores, 
    retrieved['content'],
    retrieved['meta']
)):
    print(f"النص المسترجع للسؤال {i+1}".center(80, '-'))
    print(f"Score: {score:.4f}")
    print(f"Document: {meta['doc_name']} by {meta['author_name']}")
    print(f"Paragraph: {meta['paragraph_number']}")
    print(f"Text: {text[:200]}...")
    print("\n")

Hybrid Search (BM25 + Dense)

from rank_bm25 import BM25Okapi
import numpy as np
import pyarabic.araby as araby
from datasets import load_dataset
from transformers import AutoTokenizer, AutoModel
import torch

# Load dataset with FAISS index
dataset = load_dataset("abdullah-alamodi/aqeedah-rag-dataset")

# Load the embedding model
model_name = "aubmindlab/bert-base-arabertv02"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

# Add FAISS index for fast retrieval
dataset['train'].add_faiss_index(column="embeddings")

# Helper function for embedding
def get_embedding(text):
    normalized = araby.normalize_hamza(text)
    text_input = f"query: {normalized}"
    
    inputs = tokenizer([text_input], padding=True, truncation=True, 
                      max_length=512, return_tensors='pt')
    
    with torch.no_grad():
        outputs = model(**inputs)
    
    # Average pooling
    embeddings = outputs.last_hidden_state.mean(dim=1)
    embeddings = torch.nn.functional.normalize(embeddings, p=2, dim=1)
    
    return embeddings[0].numpy()

# Prepare BM25 index
def normalize_for_bm25(text):
    text = araby.normalize_hamza(text)
    text = araby.strip_diacritics(text)
    text = araby.strip_tatweel(text)
    return text

corpus = [normalize_for_bm25(doc['content']) for doc in dataset['train']]
tokenized = [doc.split() for doc in corpus]
bm25 = BM25Okapi(tokenized)

# Search function
def hybrid_search(query, top_k=5):
    # BM25 search
    norm_query = normalize_for_bm25(query)
    bm25_scores = bm25.get_scores(norm_query.split())
    bm25_top = np.argsort(bm25_scores)[::-1][:top_k]
    
    # Dense search
    query_emb = get_embedding(query)
    scores, faiss_results = dataset['train'].get_nearest_examples(
        "embeddings", query_emb, k=top_k
    )
    
    # Extract FAISS indices (they're already sorted by score)
    # Since get_nearest_examples returns actual data, we need to track indices differently
    # Simple approach: just combine the unique results
    
    # Get unique indices from both methods
    bm25_indices = set(bm25_top.tolist())
    
    # For FAISS, we'll use the returned results directly
    # Combine: prioritize FAISS results, then add BM25-only results
    combined_results = []
    seen_content = set()
    
    # Add FAISS results first
    for content, meta in zip(faiss_results['content'], faiss_results['meta']):
        if content not in seen_content:
            combined_results.append({'content': content, 'meta': meta})
            seen_content.add(content)
    
    # Add unique BM25 results
    for idx in bm25_top:
        doc = dataset['train'][int(idx)]
        if doc['content'] not in seen_content:
            combined_results.append(doc)
            seen_content.add(doc['content'])
            if len(combined_results) >= top_k * 2:  # Get up to 2x results
                break
    
    return combined_results[:top_k * 2]  # Return more results for better coverage

# Example usage
results = hybrid_search("ما هي أركان الإيمان؟", top_k=5)
for i, res in enumerate(results):
    print(f"Result {i+1}: {res['content']}\n")

📊 Dataset Statistics

• Total paragraphs: 5419
• Language: Classical and Modern Standard Arabic (ar)
• Domain: Islamic Theology (Aqeedah)
• Source texts: 17 volumes from 9 distinct scholarly works
• Average text length: ~951 characters per paragraph
• Embedding coverage: 100% of corpus

🎯 Intended Use

Primary Applications

• ✅ Scholarly RAG Systems: Building question-answering systems for Islamic theology education
• ✅ Semantic Search: Enabling meaning-based retrieval in classical Arabic religious texts
• ✅ Educational Technology: Supporting AI-powered learning platforms for Aqeedah studies
• ✅ Research Tools: Facilitating computational analysis of Islamic theological discourse

Research Domains

• Arabic Natural Language Processing (NLP)
• Information Retrieval in Religious Texts
• Cross-lingual Semantic Search
• Domain-Specific Language Models

---

🚀 Deployment to Hugging Face Spaces

Prerequisites

1. Create a Hugging Face account at huggingface.co
2. Get a Gemini API key from Google AI Studio

Deployment Steps

1. Create a new Space:

   • Go to huggingface.co/spaces
   • Click "Create new Space"
   • Name: aqeedah-ai
   • SDK: Select "Gradio"
   • License: MIT

2. Upload files:

   # Clone your HF Space
   git clone https://huggingface.co/spaces/abdullah-alamodi/aqeedah-ai
   cd aqeedah-ai
   
   # Copy necessary files
   cp /path/to/aqeedah-ai/app.py .
   cp /path/to/aqeedah-ai/retrieval.py .
   cp /path/to/aqeedah-ai/requirements.txt .
   
   # Commit and push
   git add .
   git commit -m "Initial deployment"
   git push
   

3. Set up secrets:

   • Go to your Space settings
   • Navigate to "Repository secrets"
   • Add secret: GEMINI_API_KEY = your_api_key

4. Your Space will automatically build and deploy! 🎉

Required Files for HF Spaces

• app.py - Main Gradio application
• retrieval.py - RAG retrieval logic
• requirements.txt - Python dependencies

Note: The dataset is automatically loaded from HuggingFace, so no need to upload the data files!

---

⚠️ Limitations & Considerations

Scope Limitations

• Domain Specificity: Exclusively focused on Islamic theology (Aqeedah); not suitable for general Arabic NLP tasks
• Language: Limited to Arabic; no multilingual support
• Corpus Size: 5419 paragraphs represent a focused collection, not exhaustive coverage of all Aqeedah literature
• Temporal Coverage: Focuses on established scholarly works; may not include the most recent publications

Theological Considerations

• This dataset is curated for academic and educational purposes
• Users should consult qualified Islamic scholars for authoritative religious guidance
• The dataset represents specific theological perspectives within Sunni Islamic tradition (Ahl al-Sunnah wa al-Jama'ah)

Technical Limitations

• Embeddings are model-specific (AraBERT v2); transfer to other models may require re-encoding
• FAISS index optimized for CPU inference; GPU acceleration requires additional configuration
• Diacritic preservation may affect compatibility with some NLP tools trained on non-diacritized text

📜 License

MIT License - This dataset is freely available for academic research, educational purposes, and commercial applications with proper attribution.

🙏 Citation

If you use this dataset in your research or applications, please cite:

@dataset{aqeedah_rag_dataset_2025,
  title={Aqeedah RAG Dataset: Arabic Islamic Theology Corpus with Pre-computed Embeddings},
  author={Alamodi, Alya and Alamodi, Abdullah},
  year={2025},
  institution={Najran University, Saudi Arabia},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/datasets/abdullah-alamodi/aqeedah-rag-dataset}},
  note={Curated by Dr. Alya Alamodi (Najran University), Technical Implementation by Abdullah Alamodi (IU International University of Applied Sciences)}
}

👥 Contributors

Principal Investigator & Theological Curation:
Dr. Alya Alamodi
Ph.D. in Islamic Theology (Aqeedah)
Najran University, Kingdom of Saudi Arabia

Technical Development & AI Implementation:
Abdullah Alamodi
M.Sc. Candidate in Artificial Intelligence
IU International University of Applied Sciences, Germany

📧 Contact

For questions regarding:

• Theological content and scholarly interpretation: Contact Dr. Alya Alamodi via Najran University
• Technical implementation and AI methodology: Contact Abdullah Alamodi
• General inquiries: Open an issue on the dataset repository

---

Acknowledgments: This work was supported by the academic resources of Najran University and developed with computational infrastructure provided by IU International University of Applied Sciences.