Update project name: judgebench -> judgecliff
Browse files
README.md
CHANGED
|
@@ -4,20 +4,20 @@ pipeline_tag: image-feature-extraction
|
|
| 4 |
tags:
|
| 5 |
- reward-model
|
| 6 |
- brand-fidelity
|
| 7 |
-
-
|
| 8 |
---
|
| 9 |
|
| 10 |
-
#
|
| 11 |
|
| 12 |
-
Part of [
|
| 13 |
A brand-fidelity judge for **rhode** (beauty brand). A judge = these SigLIP weights
|
| 14 |
+ `calibration.json` (rhode train-split centroid + Platt scaling params) in this repo.
|
| 15 |
|
| 16 |
**Base:** `google/siglip-so400m-patch14-384` (full `SiglipModel`, contrastively fine-tuned).
|
| 17 |
|
| 18 |
-
**Training:** SupCon contrastive fine-tune, rhode positives vs competitor-brand negatives (Glossier, ILIA and others),
|
| 19 |
|
| 20 |
-
**Report card (
|
| 21 |
|
| 22 |
v1 learned the brand's *center*, not its *boundaries*: it is violation-blind, and under SRPO gradient pressure it was fully exploited (hacked images scored 0.84). See v2 (+violation negatives) and v3 (hardened) for the ablations.
|
| 23 |
|
|
@@ -45,4 +45,4 @@ cos = float(v @ centroid)
|
|
| 45 |
score = 1 / (1 + np.exp(-(params["platt_a"] * cos + params["platt_b"]))) # calibrated on-brand prob
|
| 46 |
```
|
| 47 |
|
| 48 |
-
Full evaluation protocol, test-set construction, and findings: [https://github.com/amargupta0428/
|
|
|
|
| 4 |
tags:
|
| 5 |
- reward-model
|
| 6 |
- brand-fidelity
|
| 7 |
+
- judgecliff
|
| 8 |
---
|
| 9 |
|
| 10 |
+
# judgecliff SigLIP brand judge v1
|
| 11 |
|
| 12 |
+
Part of [judgecliff](https://github.com/amargupta0428/judgecliff): *which image-QC judges survive optimization pressure?*
|
| 13 |
A brand-fidelity judge for **rhode** (beauty brand). A judge = these SigLIP weights
|
| 14 |
+ `calibration.json` (rhode train-split centroid + Platt scaling params) in this repo.
|
| 15 |
|
| 16 |
**Base:** `google/siglip-so400m-patch14-384` (full `SiglipModel`, contrastively fine-tuned).
|
| 17 |
|
| 18 |
+
**Training:** SupCon contrastive fine-tune, rhode positives vs competitor-brand negatives (Glossier, ILIA and others), judgecliff train split. No violation negatives.
|
| 19 |
|
| 20 |
+
**Report card (judgecliff Phase 1, 2,622-item test set):** brand AUC **0.99**, logo-masking delta **0.00** (style reader, not a wordmark reader), brand-dial Spearman 0.24, near-zero violation detection.
|
| 21 |
|
| 22 |
v1 learned the brand's *center*, not its *boundaries*: it is violation-blind, and under SRPO gradient pressure it was fully exploited (hacked images scored 0.84). See v2 (+violation negatives) and v3 (hardened) for the ablations.
|
| 23 |
|
|
|
|
| 45 |
score = 1 / (1 + np.exp(-(params["platt_a"] * cos + params["platt_b"]))) # calibrated on-brand prob
|
| 46 |
```
|
| 47 |
|
| 48 |
+
Full evaluation protocol, test-set construction, and findings: [https://github.com/amargupta0428/judgecliff](https://github.com/amargupta0428/judgecliff).
|