Add canonical HBS and Slovenian checkpoints

README.md

@@ -3,20 +3,148 @@ library_name: pytorch
 tags:
 - aspect-based-sentiment-analysis
 - south-slavic
+- text-classification
 license: other
 ---
 
 # AspectBench BERTić / SloBERTa
 
-Private model-only checkpoints for HBS and Slovenian document-level
-aspect-based sentiment analysis. This repository contains 4/4
-language-mode checkpoint slots. Missing slots are documented in
-`availability.json`; they are not substituted with another model.
+Model-only checkpoints for HBS and Slovenian document-level aspect-based
+sentiment analysis. This repository contains 4/4 language-mode
+checkpoint slots. It is used with the shared inference toolkit in
+[`nishan-chatterjee/aspect-based-sentiment-analysis`](https://huggingface.co/nishan-chatterjee/aspect-based-sentiment-analysis).
 
-Use the shared scripts from
-`nishan-chatterjee/aspect-based-sentiment-analysis`. Input articles must tag
-the target as `<aspect>...</aspect>`. Checkpoints contain model tensors only;
-optimizer, scheduler, and gradient-scaler state has been removed.
+## Input format
 
-Keep this repository private until all source-data and data-derived-weight
-permissions are confirmed.
+Every article must mark the target span with literal tags, even when using an
+unmasked checkpoint:
+
+```text
+Kupci su pohvalili <aspect>Primer Grupa</aspect> zbog brze usluge.
+```
+
+- `masked`: the tagged text is replaced with `[ASPECT]`; the model does not see
+  the target name.
+- `unmasked`: the tags are removed and the model sees the target name.
+- Gold `sentiment` is optional: `-1` = negative, `0` = neutral, `1` = positive.
+  It is reported in the result but never used to produce the prediction.
+
+## Available checkpoints
+
+| Language | Mode | Status | Best validation Macro-F1 |
+|---|---|---|---:|
+| hbs | masked | Available | 0.9151 (run 1) |
+| hbs | unmasked | Available | 0.9251 (run 0) |
+| slovenian | masked | Available | 0.8545 (run 2) |
+| slovenian | unmasked | Available | 0.8639 (run 2) |
+
+`availability.json` contains the machine-readable selection record. A missing
+checkpoint is never replaced with a checkpoint from another mode or run.
+
+## Getting started
+
+Create the portable environment from the toolkit repository:
+
+```bash
+conda env create -f environment.yml
+conda activate aspectbench
+```
+
+Or install the runtime packages in an existing environment:
+
+```bash
+python -m pip install -U torch transformers accelerate huggingface-hub sentencepiece numpy spacy sentence-transformers
+```
+
+Download the toolkit and this model repository into the expected directory
+layout:
+
+```python
+from pathlib import Path
+from huggingface_hub import snapshot_download
+
+ROOT = Path("huggingface")
+snapshot_download(
+    repo_id="nishan-chatterjee/aspect-based-sentiment-analysis",
+    local_dir=ROOT,
+)
+snapshot_download(
+    repo_id="nishan-chatterjee/aspectbench-slavic-specific",
+    local_dir=ROOT / "models" / "slavic-specific",
+)
+```
+
+The model repository includes the tokenizer and configuration assets required
+to reconstruct the architecture. No separate base-model cache is needed.
+
+## Python / Jupyter prediction
+
+```python
+from pathlib import Path
+import sys
+
+ROOT = Path("huggingface").resolve()
+sys.path.insert(0, str(ROOT / "scripts"))
+
+from inference import InferenceEngine
+
+engine = InferenceEngine(
+    model_name="slavic-specific",
+    language="hbs",
+    mode="masked",
+    model_root=ROOT / "models",
+    device="cuda",  # use "cpu" when no GPU is available
+)
+
+prediction = engine.predict(
+    {
+        "article": "Kupci su pohvalili <aspect>Primer Grupa</aspect> zbog brze usluge.",
+        "sentiment": 1,
+    },
+    mc_passes=10,
+)
+prediction
+```
+
+For a real batch, reuse the loaded engine:
+
+```python
+records = [
+    {"article": "Kupci su pohvalili <aspect>Primer Grupa</aspect>.", "sentiment": 1},
+    {"article": "Pritužbe na <aspect>Drugi Sistem</aspect> nisu riješene.", "sentiment": -1},
+]
+predictions = engine.predict_batch(records, batch_size=2, mc_passes=10)
+```
+
+## Command-line prediction
+
+Run from the toolkit directory:
+
+```bash
+python scripts/predict.py \
+  --model-name slavic-specific \
+  --language hbs \
+  --mode masked \
+  --model-root models \
+  --device cuda \
+  --mc-passes 10 \
+  --article 'Kupci su pohvalili <aspect>Primer Grupa</aspect> zbog brze usluge.' \
+  --sentiment 1
+```
+
+## Output fields
+
+| Field | Meaning |
+|---|---|
+| `input_article` | Original article, including `<aspect>` tags. |
+| `tagged_aspects` | Target strings extracted from the tags. |
+| `aspect_used` | Target representation actually supplied to the model. |
+| `gold_sentiment` | Optional user-supplied reference label. |
+| `predicted_sentiment` | Predicted integer label: `-1`, `0`, or `1`. |
+| `predicted_sentiment_name` | Human-readable class name. |
+| `class_probabilities` | Probability assigned to every sentiment class. |
+| `uncertainty_across_classes` | Entropy, confidence, probability margin, and—when MC dropout is enabled—mutual information and vote statistics. |
+| `inference` | Device, MC-dropout flag, and checkpoint path. |
+
+The `.pt` files contain model tensors only. Optimizer, scheduler, and
+gradient-scaler state is excluded.

availability.json

@@ -9,9 +9,10 @@
       "validation_macro_f1": 0.9151353276491324,
       "available": true,
       "unavailable_reason": null,
+      "expected_source_checkpoint": null,
       "base_model": "classla/bcms-bertic",
       "weight_path": "slavic-specific/hbs/masked.pt",
-      "status": "exported",
+      "status": "existing",
       "size": 442555219,
       "sha256": "09a5ef9aa704535145e35b52db88aa9fa4108ec71c8a669ddbb5575f27417b70",
       "tensor_count": 201,
@@ -25,9 +26,10 @@
       "validation_macro_f1": 0.9250658853407349,
       "available": true,
       "unavailable_reason": null,
+      "expected_source_checkpoint": null,
       "base_model": "classla/bcms-bertic",
       "weight_path": "slavic-specific/hbs/unmasked.pt",
-      "status": "exported",
+      "status": "existing",
       "size": 442558893,
       "sha256": "918c1fdad79ec5911ebde16b90a00651c9d7b0ab02dbff36005490c95586f3da",
       "tensor_count": 201,
@@ -41,9 +43,10 @@
       "validation_macro_f1": 0.8545257213567837,
       "available": true,
       "unavailable_reason": null,
+      "expected_source_checkpoint": null,
       "base_model": "EMBEDDIA/sloberta",
       "weight_path": "slavic-specific/slovenian/masked.pt",
-      "status": "exported",
+      "status": "existing",
       "size": 442573651,
       "sha256": "1f0c8b88ac36f4d4dfe34322e37f8c12bbc87ef842ca8142265ba60b6066668b",
       "tensor_count": 201,
@@ -57,9 +60,10 @@
       "validation_macro_f1": 0.8638678818708625,
       "available": true,
       "unavailable_reason": null,
+      "expected_source_checkpoint": null,
       "base_model": "EMBEDDIA/sloberta",
       "weight_path": "slavic-specific/slovenian/unmasked.pt",
-      "status": "exported",
+      "status": "existing",
       "size": 442577325,
       "sha256": "489572e0225aa8d9c58eb90dfb344c97cd5902b4342cfe1e99a893785651a8d6",
       "tensor_count": 201,

hbs/base_model/config.json

@@ -0,0 +1,32 @@
+{
+  "add_cross_attention": false,
+  "architectures": [
+    "ElectraForPreTraining"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": null,
+  "classifier_dropout": null,
+  "embedding_size": 768,
+  "eos_token_id": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "is_decoder": false,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "electra",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "summary_activation": "gelu",
+  "summary_last_dropout": 0.1,
+  "summary_type": "first",
+  "summary_use_proj": true,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.8.1",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 32000
+}

hbs/base_model/source.json

@@ -0,0 +1,7 @@
+{
+  "model": "slavic-specific",
+  "language": "hbs",
+  "upstream_base_model": "classla/bcms-bertic",
+  "purpose": "Tokenizer and architecture configuration for offline checkpoint reconstruction",
+  "contains_upstream_model_weights": false
+}

hbs/base_model/tokenizer_config.json

@@ -0,0 +1 @@
+{"do_lower_case": false, "max_len": 512, "init_inputs": [], "strip_accents":false}

slovenian/base_model/config.json

@@ -0,0 +1,28 @@
+{
+  "add_cross_attention": false,
+  "architectures": [
+    "CamembertForMaskedLM"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "is_decoder": false,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "camembert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.8.1",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 32005
+}

slovenian/base_model/source.json

@@ -0,0 +1,7 @@
+{
+  "model": "slavic-specific",
+  "language": "slovenian",
+  "upstream_base_model": "EMBEDDIA/sloberta",
+  "purpose": "Tokenizer and architecture configuration for offline checkpoint reconstruction",
+  "contains_upstream_model_weights": false
+}

slovenian/base_model/special_tokens_map.json

@@ -0,0 +1 @@
+{"bos_token": "<s>", "eos_token": "</s>", "unk_token": "<unk>", "sep_token": "</s>", "pad_token": "<pad>", "cls_token": "<s>", "mask_token": {"content": "<mask>", "single_word": false, "lstrip": true, "rstrip": false, "normalized": true}, "additional_special_tokens": ["<s>NOTUSED", "</s>NOTUSED"]}

slovenian/base_model/tokenizer_config.json

@@ -0,0 +1 @@
+{"bos_token": "<s>", "eos_token": "</s>", "sep_token": "</s>", "cls_token": "<s>", "unk_token": "<unk>", "pad_token": "<pad>", "mask_token": {"content": "<mask>", "single_word": false, "lstrip": true, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "additional_special_tokens": ["<s>NOTUSED", "</s>NOTUSED"], "special_tokens_map_file": null, "name_or_path": "EMBEDDIA/sloberta", "sp_model_kwargs": {}, "tokenizer_class": "CamembertTokenizer", "model_max_length": 512, "do_lower_case": false}