---
title: Hebrew Unified NLP
emoji: 📜
colorFrom: blue
colorTo: indigo
sdk: docker
pinned: false
license: mit
models:
  - dicta-il/dictabert-joint
  - thewh1teagle/phonikud-onnx
  - thewh1teagle/phonikud-tts-checkpoints
  - rhasspy/piper-voices
---

# 🇮🇱 Hebrew Unified NLP v2025.12.5

A comprehensive Hebrew NLP pipeline with multilingual TTS, bilingual UI (English/Hebrew), and per-sentence parallel processing.

## Features

- **Morphological Analysis**: DictaBERT-Joint for POS tagging, dependency parsing, NER, segmentation
- **Phonetic Processing**: Phonikud for Hebrew diacritization (nikud) and phoneme generation
- **Multilingual TTS**: 37 Piper voices (36 English + 1 Hebrew) with per-sentence language detection
- **Bilingual UI**: Full English/Hebrew interface with RTL support and translated labels
- **Parallel Processing**: Async pipeline with configurable workers for batch processing
- **Rich Output**: SVG dependency trees, aligned audio playlist, structured JSON

## Project Structure

```
hebrew-unified-nlp/
├── app.py                    # Main Streamlit application
├── schema.json               # JSON output schema (v2025.12.5)
├── requirements.txt          # Python dependencies
├── Dockerfile                # Container deployment
├── download_models.py        # Cross-platform model download script
├── .streamlit/
│   └── config.toml           # Streamlit configuration
├── static/
│   └── schema.json           # Schema served at /static/schema.json
└── onnx/
    ├── phonikud-1.0.int8.onnx    # Hebrew diacritization model
    └── piper-voices/
        ├── he_IL-phonikud.onnx       # Hebrew TTS voice
        ├── he_IL-phonikud.onnx.json
        ├── en_US-ryan-high.onnx      # Default English voice
        ├── en_US-ryan-high.onnx.json
        └── ...                        # Additional English voices
```

## ⚠️ Python Version Requirement

**Requires Python 3.10 - 3.12** (phonikud doesn't support Python 3.13 yet)

## Quick Start

### Standard Setup

```bash
# Create virtual environment
python -m venv .venv
.venv\Scripts\activate  # Windows
source .venv/bin/activate  # Linux/Mac

# Install system dependencies (Linux only)
sudo apt install graphviz libsndfile1 espeak-ng  # Ubuntu/Debian

# Install Python dependencies
pip install -r requirements.txt

# Download essential models (phonikud + Hebrew + English voice)
python download_models.py --essential

# Or download all 36 English voices
python download_models.py

# Run
streamlit run app.py
```

### With UV Package Manager (Faster)

```bash
# Install UV
curl -LsSf https://astral.sh/uv/install.sh | sh  # Linux/Mac
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"  # Windows

# Create venv with Python 3.11
uv venv --python 3.11
source .venv/bin/activate  # or .venv\Scripts\activate on Windows

# Install dependencies (10x faster than pip)
uv pip install -r requirements.txt

# Download models
python download_models.py --essential

# Run
streamlit run app.py
```

## Download Models

```bash
python download_models.py              # All 36 English voices + Hebrew
python download_models.py --essential  # Minimal: phonikud + Hebrew + en_US-ryan-high
```

The script uses HuggingFace Hub, skips existing files, and works on all platforms.

## Available Voices

### Hebrew
- `he_IL-phonikud` - Hebrew voice with proper nikud support

### English (36 voices)
**British (en_GB):**
alan, alba, aru, cori, jenny_dioco, northern_english_male, semaine, southern_english_female, vctk

**American (en_US):**
amy, arctic, bryce, danny, hfc_female, hfc_male, joe, john, kathleen, kusal, kristin, l2arctic, lessac, libritts, libritts_r, ljspeech, norman, ryan

Each voice has quality variants: low, medium, high (not all variants available for all voices).

## JSON Output Format

```json
{
  "meta": {
    "version": "2025.12.5",
    "timestamp": "2025-12-05T14:30:00Z",
    "models": {
      "phonikud": "phonikud-1.0.int8",
      "piper": "he_IL-phonikud / en_US-ryan-high",
      "dictabert": "dicta-il/dictabert-joint"
    },
    "processing_time_ms": 245.5,
    "sentence_count": 3,
    "parallel_workers": 4,
    "sentence_breaker": "pysbd"
  },
  "input": {
    "text": "הילד הלך לבית הספר. He walked home.",
    "language": "mixed"
  },
  "translations": {
    "pos": { "NOUN": {"en": "Noun", "he": "שם עצם"}, "..." : "..." },
    "dep": { "nsubj": {"en": "Subject", "he": "נושא"}, "..." : "..." },
    "ner": { "PER": {"en": "Person", "he": "אדם"}, "..." : "..." },
    "morph": { "Masc": {"en": "Masculine", "he": "זכר"}, "..." : "..." }
  },
  "sentences": [
    {
      "index": 0,
      "text": "הילד הלך לבית הספר.",
      "language": "he",
      "phonetics": {
        "diacritized": "הַיֶּלֶד הָלַךְ לְבֵית הַסֵּפֶר.",
        "phonemes": "h a j e l e d | h a l a x | l e v e j t | h a s e f e r"
      },
      "tokens": ["..."],
      "ner_entities": [],
      "speech": {
        "format": "wav",
        "sample_rate": 22050,
        "duration_ms": 1847,
        "voice": "he_IL-phonikud",
        "data_uri": "data:audio/wav;base64,..."
      }
    },
    {
      "index": 1,
      "text": "He walked home.",
      "language": "en",
      "speech": {
        "format": "wav",
        "sample_rate": 22050,
        "duration_ms": 1200,
        "voice": "en_US-ryan-high",
        "data_uri": "data:audio/wav;base64,..."
      }
    }
  ]
}
```

## API Usage

```python
from app import AsyncHebrewNLP
import asyncio

# Initialize
nlp = AsyncHebrewNLP(hf_token="YOUR_TOKEN", max_workers=4)

# Async processing with multilingual TTS
async def analyze():
    result = await nlp.process(
        "הילד הלך לבית הספר. He walked home.",
        include_audio=True,
        en_voice="en_US-ryan-high",  # English voice selection
        compute_mst=True
    )
    return result

result = asyncio.run(analyze())

# Or sync wrapper
result = nlp.process_sync("הילד הלך לבית הספר.")

# Access results
print(f"Sentences: {result['meta']['sentence_count']}")
for sent in result['sentences']:
    print(f"[{sent['index']}] ({sent['language']}) {sent['text']}")
    if sent['language'] == 'he':
        print(f"    Nikud: {sent['phonetics']['diacritized']}")
```

## Performance

| Sentences | Sequential | Parallel (4 workers) | Speedup |
|-----------|------------|---------------------|---------|
| 1 | ~120ms | ~120ms | 1x |
| 4 | ~480ms | ~150ms | 3.2x |
| 10 | ~1200ms | ~350ms | 3.4x |

## Docker

```bash
docker build -t hebrew-nlp .
docker run -p 8501:8501 hebrew-nlp
```

The Dockerfile pre-downloads essential models (phonikud + Hebrew voice + en_US-ryan-high).

## Bilingual UI

The interface supports both English and Hebrew with a language toggle in the sidebar. All labels are translated including:
- Part-of-speech tags (NOUN → שם עצם)
- Dependency relations (nsubj → נושא)
- NER entity types (PER → אדם)
- Morphological features (Masc → זכר, Sing → יחיד)
- Prefix types (DEF → ה׳ הידיעה)

## Credits

- [Dicta](https://dicta.org.il/) - DictaBERT-Joint model
- [thewh1teagle](https://github.com/thewh1teagle) - Phonikud diacritization & Hebrew TTS voice
- [Piper](https://github.com/rhasspy/piper) - Neural TTS engine
- [rhasspy/piper-voices](https://huggingface.co/rhasspy/piper-voices) - English TTS voices
- [pysbd](https://github.com/nipunsadvilkar/pySBD) - Sentence boundary detection

## License

MIT