---
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
- echo-dsrn
- linear-complexity
- recurrent-hybrid
datasets:
- sentence-transformers/all-nli
- mteb/sts-b
metrics:
- spearman
- pearson
language:
- en
---

# Echo-DSRN Embedding Model (Echo-DSRN-v0.1.3-Embed-Exp)

[![GitHub](https://img.shields.io/badge/GitHub-ethicalabs.ai-black.svg)](https://github.com/ethicalabs-ai/Echo-DSRN/)
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![Python](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![Model Collection](https://img.shields.io/badge/Echo--DSRN-HuggingFace-yellow.svg)](https://huggingface.co/collections/ethicalabs/echo-dsrn)
[![Hybrid Collection](https://img.shields.io/badge/Echo--Hybrid-HuggingFace-red.svg)](https://huggingface.co/collections/ethicalabs/echo-dsrn-hybrid)
[![Working Paper](https://img.shields.io/badge/Working--Paper-Echo_DSRN-green.svg)](https://github.com/ethicalabs-ai/Echo-DSRN/blob/main/PAPER.md)

> [!WARNING]
> This repository contains experimental models designed strictly for academic evaluation and research purposes.
> 
> Critical Constraints:
> * **No Production Deployment:** Experimental models must not be deployed in commercial, enterprise, or mission-critical environments under any circumstances. 
> * **No Liability:** Experimental models are provided "as-is" without warranties of any kind. The developers assume zero liability for downstream consequences, system integration failures, or regulatory non-compliance resulting from unauthorized deployment.

This is a high-performance **experimental** sentence embedding model based on the recurrent-hybrid **Echo-DSRN** architecture.

It scales **linearly** ($O(N)$) with sequence length, offering extreme efficiency and sub-millisecond latency on both CPU and GPU.

## 🚀 Model Details

- **Developer:** ethicalabs
- **Base Model:** [ethicalabs/Echo-DSRN-114M-v0.1.2](https://huggingface.co/ethicalabs/Echo-DSRN-114M-v0.1.2)
- **Architecture:** [Echo-DSRN](https://huggingface.co/ethicalabs/Echo-DSRN-114M-v0.1.2) Recurrent-Hybrid
- **Parameters:** 98.26M (98,264,064)
- **Embedding Dimension:** 2048

## 📊 Evaluation Results (MTEB STS)

Spearman Rank Correlation scores on Semantic Textual Similarity (STS) benchmark tasks:

| Benchmark Task | Echo-DSRN (Ours) |
| :--- | :---: |
| **STS12** | `0.6667` |
| **STS13** | `0.7692` |
| **STS14** | `0.7683` |
| **STS15** | `0.8227` |
| **STS16** | `0.7460` |
| **STSBenchmark** | `0.7293` |
| **SICK-R** | `0.7876` |
| **Average STS** | **`0.7557`** |

## ⚡ Efficiency and Systems Scaling Profile

Inference performance (latency and peak VRAM allocation) on GPU and CPU configurations across different sequence lengths:

### GPU Latency & VRAM Benchmark

| Sequence Length | Echo-DSRN Latency (GPU) | Echo-DSRN VRAM (GPU) |
| :---: | :---: | :---: |
| 128 | `15.93 ms` | `516.69 MB` |
| 256 | `17.56 ms` | `548.44 MB` |
| 512 | `32.14 ms` | `604.95 MB` |
| 1024 | `71.30 ms` | `710.96 MB` |
| 2048 | `155.26 ms` | `932.99 MB` |
| 4096 | N/A (OOR) | N/A (OOR) |

### CPU Latency Benchmark

| Sequence Length | Echo-DSRN Latency (CPU) |
| :---: | :---: |
| 128 | `48.50 ms` |
| 256 | `84.94 ms` |
| 512 | `160.93 ms` |
| 1024 | `328.93 ms` |
| 2048 | `727.57 ms` |
| 4096 | N/A (OOR) |

*Note: 'N/A (OOR)' indicates sequence length exceeds model's maximum position embedding range.*

## 🏗️ Architecture Details

| Property | Value |
| :--- | :--- |
| Layers | 8 |
| Hidden Dim | 512 |
| Vocab Size | 32017 |
| Attention Heads | 4 |

## 📊 Parameter Breakdown

| Component | Parameters | % of Total |
| :--- | :--- | :--- |
| **Total** | **98.26M (98,264,064)** | **100%** |
| Embeddings | 16.39M | 16.68% |
| DSRN Recurrent Blocks | 81.87M | 83.32% |
| Norms & Biases | 512 | 0.00% |

## 💻 Usage

You can load and use this model directly via `sentence-transformers`:

```python
from sentence_transformers import SentenceTransformer

# Load model with auto-mapping enabled
model = SentenceTransformer("ethicalabs/Echo-DSRN-v0.1.3-Embed-Exp", trust_remote_code=True)

# Encode text to get 2048-dimensional embeddings
sentences = ["The recurrent slow state contains the aligned sequence representations.", "Echo-DSRN has linear complexity."]
embeddings = model.encode(sentences)
print(embeddings.shape) # (2, 2048)
```

## 🛠️ Training Procedure

The model was trained in three sequential phases:

1. **Contrastive Pre-training**: Representation space alignment using natural language inference datasets.
2. **Fine-grained Similarity Tuning**: Fine-tuning using semantic textual similarity benchmarks to calibrate similarity scores.
3. **Multi-Task Generalization Tuning**: Training on NLI retrieval and STS semantic similarity.

---
*This model card was automatically generated by `scripts/generate_model_card.py`.*

---

## Note on tokenizer padding

The benchmark results on this card (STS table) were measured with left
padding (`padding_side: left`), and this model version reproduces them under
that convention. A right-padded training version is planned: right padding
keeps padded-batch embeddings consistent with single-request embeddings
(leading pad tokens do not pollute the recurrent state), so future
checkpoints will be batch-composition independent.