Upload ONNX FP32 + INT8 quantized sparse encoder for financial documents

README.md

@@ -0,0 +1,100 @@
+---
+language:
+- en
+license: apache-2.0
+tags:
+- sparse-encoder
+- onnx
+- quantized
+- int8
+- splade
+- financial
+- inference-optimized
+base_model: oneryalcin/fin-sparse-encoder-doc-v1
+pipeline_tag: feature-extraction
+library_name: onnxruntime
+---
+
+# fin-sparse-encoder-doc-v1-onnx
+
+ONNX + INT8 quantized version of [oneryalcin/fin-sparse-encoder-doc-v1](https://huggingface.co/oneryalcin/fin-sparse-encoder-doc-v1) for CPU-efficient document encoding.
+
+This is the **document encoder path only** — it produces sparse SPLADE vectors for indexing financial documents (SEC filings, earnings call transcripts). Query encoding uses a separate IDF lookup table (sub-ms, no neural model needed).
+
+## Model Variants
+
+| File | Format | Size | Use Case |
+|---|---|---|---|
+| `model.onnx` | FP32 | 647.9 MB | Maximum accuracy, GPU or high-memory CPU |
+| `model_quantized.onnx` | INT8 | 166.7 MB | **Recommended for CPU deployment** |
+
+## Performance
+
+### Domain Evaluation (Financial Documents)
+
+The parent model ([fin-sparse-encoder-doc-v1](https://huggingface.co/oneryalcin/fin-sparse-encoder-doc-v1)) was evaluated on 2,028 held-out financial test examples:
+
+| Metric | Base Model | Fine-tuned | Delta |
+|:---|:---|:---|:---|
+| **acc@1** | 39.9% | **55.2%** | +15.2% |
+| **acc@3** | 69.2% | **84.0%** | +14.8% |
+| **ndcg@10** | 0.681 | **0.781** | +10.0% |
+| median_rank | 2.0 | **1.0** | -1.0 |
+
+### Inference Latency (seq_len=512, 1 thread)
+
+Benchmarked on Apple M-series CPU. Server CPUs with AVX512-VNNI will see larger INT8 speedups (~2-3x).
+
+| Backend | p50 (ms) | p95 (ms) | Model Size |
+|---|---|---|---|
+| PyTorch FP32 | 186.3 | 192.8 | ~620 MB |
+| ONNX FP32 | 211.7 | 218.9 | 647.9 MB |
+| **ONNX INT8** | **164.4** | **166.9** | **166.7 MB** |
+
+## Usage
+
+```python
+import numpy as np
+import onnxruntime as ort
+from transformers import AutoTokenizer
+
+# Load
+tokenizer = AutoTokenizer.from_pretrained("oneryalcin/fin-sparse-encoder-doc-v1-onnx")
+sess = ort.InferenceSession("model_quantized.onnx", providers=["CPUExecutionProvider"])
+
+# Encode a document
+text = "Revenue increased 12% year over year to $4.2 billion in Q4 2023."
+inputs = tokenizer(text, return_tensors="np", padding="max_length", max_length=512, truncation=True)
+logits = sess.run(None, {"input_ids": inputs["input_ids"], "attention_mask": inputs["attention_mask"]})[0]
+
+# SpladePooling: log1p_relu activation (matches OpenSearch v3 models)
+masked = logits * inputs["attention_mask"][..., None]
+pooled = masked.max(axis=1)
+sparse_vector = np.log1p(np.log1p(np.maximum(pooled, 0.0)))  # [1, 30522]
+
+# Convert to token->weight dict (for inverted index)
+nonzero = np.nonzero(sparse_vector[0])[0]
+token_weights = {tokenizer.decode([tid]): float(sparse_vector[0, tid]) for tid in nonzero}
+print(f"Active dimensions: {len(token_weights)}")
+print(f"Top tokens: {sorted(token_weights.items(), key=lambda x: -x[1])[:10]}")
+```
+
+## Architecture
+
+```
+Input text
+  → Tokenizer (max_length=512)
+  → ONNX model (MLM logits) [batch, seq, 30522]
+  → SpladePooling: log(1 + log(1 + ReLU(max_over_seq(logits * mask))))
+  → Sparse vector [batch, 30522]
+```
+
+Base model: `opensearch-project/opensearch-neural-sparse-encoding-doc-v3-gte` (Alibaba-NLP/new-impl architecture).
+
+Fine-tuned on [financial-filings-sparse-retrieval-training](https://huggingface.co/datasets/oneryalcin/financial-filings-sparse-retrieval-training) (18K examples, 2 epochs).
+
+## Export Details
+
+- Exported via `torch.onnx.export` (legacy tracer, opset 17)
+- INT8: dynamic quantization via `onnxruntime.quantization.quantize_dynamic` (per-channel, QInt8)
+- Numerical verification: FP32 ONNX max diff vs PyTorch = 0.000057

benchmark.json

@@ -0,0 +1,20 @@
+{
+  "seq_len": 512,
+  "threads": 1,
+  "iters": 30,
+  "pytorch_fp32": {
+    "p50_ms": 186.2685834785225,
+    "p95_ms": 192.7833059511613,
+    "mean_ms": 187.10802356363274
+  },
+  "onnx_fp32": {
+    "p50_ms": 211.71175049676094,
+    "p95_ms": 218.8744774510269,
+    "mean_ms": 211.95321109941383
+  },
+  "onnx_int8": {
+    "p50_ms": 164.41352099354845,
+    "p95_ms": 166.8924230907578,
+    "mean_ms": 163.78664293248826
+  }
+}

config.json

@@ -0,0 +1,13 @@
+{
+  "base_model": "opensearch-project/opensearch-neural-sparse-encoding-doc-v3-gte",
+  "finetuned_from": "oneryalcin/fin-sparse-encoder-doc-v1",
+  "max_seq_length": 512,
+  "vocab_size": 30522,
+  "activation": "log1p_relu",
+  "pooling": "max",
+  "splade_postprocess": "log(1 + log(1 + ReLU(max_over_seq(logits * attention_mask))))",
+  "files": {
+    "model.onnx": "Full precision FP32 ONNX model",
+    "model_quantized.onnx": "Dynamic INT8 quantized ONNX model (recommended for CPU)"
+  }
+}

model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:887339947d416cd21db98dbe52ea33c4dba4a214b9e8a138e2ed7048d30665c1
+size 647930964

model_quantized.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27fd05ec5b30a4ddb3f989caa141ff4eda71a5afda913d2fc61f708f8b281124
+size 166738071

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,63 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "max_length": 8192,
+  "model_max_length": 8192,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "DistilBertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}