Spaces:
Running
Running
luke0709 commited on
Commit ·
ac7a77a
0
Parent(s):
Create standalone static benchmark leaderboard
Browse files- .gitignore +6 -0
- README.md +93 -0
- front/config.mjs +6 -0
- front/data.mjs +81 -0
- front/data/benchmark.json +962 -0
- front/leaderboard.mjs +413 -0
- index.html +246 -0
- tests/test_static_space.py +204 -0
.gitignore
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
__pycache__/
|
| 2 |
+
*.py[cod]
|
| 3 |
+
.pytest_cache/
|
| 4 |
+
.env
|
| 5 |
+
.env.*
|
| 6 |
+
.venv/
|
README.md
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: KETI Benchmark Leaderboard
|
| 3 |
+
emoji: 🦀
|
| 4 |
+
colorFrom: red
|
| 5 |
+
colorTo: gray
|
| 6 |
+
sdk: static
|
| 7 |
+
app_file: index.html
|
| 8 |
+
pinned: false
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# KETI Leaderboard
|
| 12 |
+
|
| 13 |
+
A static leaderboard for published KETI model evaluations. The browser loads
|
| 14 |
+
`front/data/benchmark.json` directly; no Python server, Docker runtime, access token,
|
| 15 |
+
or build step is needed. Dataset selection, model search, provider filtering,
|
| 16 |
+
comparison charts, theme selection, and CSV export are available.
|
| 17 |
+
|
| 18 |
+
The initial snapshot contains the 19 models and the 9 displayed datasets retrieved
|
| 19 |
+
from the existing public leaderboard during migration. Scores and evaluation
|
| 20 |
+
timestamps are preserved. The snapshot records its source and capture time;
|
| 21 |
+
internal evaluation job identifiers and error messages are not included.
|
| 22 |
+
|
| 23 |
+
## Run locally
|
| 24 |
+
|
| 25 |
+
From the repository root:
|
| 26 |
+
|
| 27 |
+
```bash
|
| 28 |
+
python3 -m http.server 7860 --bind 127.0.0.1
|
| 29 |
+
```
|
| 30 |
+
|
| 31 |
+
Open <http://127.0.0.1:7860/>. Use an HTTP server rather than opening `index.html`
|
| 32 |
+
as a local file, because the browser loads JavaScript modules and JSON.
|
| 33 |
+
Tailwind, Chart.js, and fonts currently load from their existing public CDNs.
|
| 34 |
+
|
| 35 |
+
## Verify the static page
|
| 36 |
+
|
| 37 |
+
With Playwright and Chrome/Chromium installed, run:
|
| 38 |
+
|
| 39 |
+
```bash
|
| 40 |
+
python3 -m unittest discover -s tests -v
|
| 41 |
+
```
|
| 42 |
+
|
| 43 |
+
Set `CHROME_PATH` if the browser is not on `PATH`. These checks use a temporary
|
| 44 |
+
local HTTP server and cover published scores and means, filters, comparison charts,
|
| 45 |
+
CSV export, mobile controls, preferences, loading failures, and malformed data.
|
| 46 |
+
Internet access is needed for the existing CDN assets in the chart test.
|
| 47 |
+
|
| 48 |
+
## Update published results
|
| 49 |
+
|
| 50 |
+
1. Evaluate models outside this Space.
|
| 51 |
+
2. Update `front/data/benchmark.json` with the resulting scores.
|
| 52 |
+
3. Commit and push the changed data file to the Space. Visitors can use **Refresh**
|
| 53 |
+
to load the published version again.
|
| 54 |
+
|
| 55 |
+
The data file accepts either the legacy `benchmark.json` array of models or an
|
| 56 |
+
object with `models` and an optional `datasets` array. Each model has `provider`,
|
| 57 |
+
`name`, `repo`, `is_multimodal`, `updated_at`, and `scores`. Each score has
|
| 58 |
+
`dataset_name`, numeric `score`, and `metric_type` (`raw` or `llm-as-judge`).
|
| 59 |
+
The `datasets` array sets the displayed columns and their order; without it,
|
| 60 |
+
the canonical datasets and any additional scored datasets are shown.
|
| 61 |
+
|
| 62 |
+
The mean calculation retains the previous behavior: all selected, supported
|
| 63 |
+
datasets must have scores. Unsupported multimodal datasets are excluded for
|
| 64 |
+
text-only models. Missing means are displayed as `—`. A failed data request shows
|
| 65 |
+
an error; it never substitutes generated scores. A failed refresh keeps the last
|
| 66 |
+
successfully loaded results on screen.
|
| 67 |
+
|
| 68 |
+
For an independently updated, **public** JSON source, change `dataUrl` in
|
| 69 |
+
`front/config.mjs` to its HTTPS URL. That server must allow browser cross-origin
|
| 70 |
+
requests (CORS). No automatic fallback to another source is applied. The default
|
| 71 |
+
bundled snapshot remains independent of the personal Hugging Face account.
|
| 72 |
+
All browser configuration and bundled files are public: do not put tokens or
|
| 73 |
+
private evaluation data in them.
|
| 74 |
+
|
| 75 |
+
## Deploy as a new Hugging Face Space
|
| 76 |
+
|
| 77 |
+
This directory is an independent Git repository containing only the static app.
|
| 78 |
+
The existing Docker Space at
|
| 79 |
+
https://huggingface.co/spaces/DoolyKim22/KETI_Leaderboard remains separate.
|
| 80 |
+
|
| 81 |
+
The deployment target is the new organization-owned Space
|
| 82 |
+
[KETI-NLP/Benchmark_Leaderboard](https://huggingface.co/spaces/KETI-NLP/Benchmark_Leaderboard),
|
| 83 |
+
using the **Static** SDK. The README metadata already specifies
|
| 84 |
+
`sdk: static` and `app_file: index.html` as described in the
|
| 85 |
+
[Static Spaces documentation](https://huggingface.co/docs/hub/spaces-sdks-static).
|
| 86 |
+
Do not use the existing Docker Space as this repository's remote.
|
| 87 |
+
|
| 88 |
+
The snapshot is independent of the running Docker app. Updates published to the
|
| 89 |
+
old app after the snapshot was captured are not automatically copied here.
|
| 90 |
+
Update `front/data/benchmark.json` and publish a new commit to refresh this app.
|
| 91 |
+
|
| 92 |
+
Model submissions, automatic evaluations, and dataset backup workers are not part
|
| 93 |
+
of the static site. Browser configuration lives in `front/config.mjs`.
|
front/config.mjs
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
// Public browser configuration. Never put Hugging Face tokens here.
|
| 2 |
+
export const config = {
|
| 3 |
+
// Relative to this file. An absolute, public HTTPS JSON URL also works.
|
| 4 |
+
dataUrl: new URL('./data/benchmark.json', import.meta.url).href,
|
| 5 |
+
requestTimeoutMs: 15000,
|
| 6 |
+
};
|
front/data.mjs
ADDED
|
@@ -0,0 +1,81 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
export const CANONICAL_DATASETS = [
|
| 2 |
+
'일반 상식', '멀티 모달', '수치 정보', '팩트 체크', '소버린 벤치마크',
|
| 3 |
+
'K-Prism (Text, Intact)', 'K-Prism (Text, Perturbed)',
|
| 4 |
+
'K-Prism (Image, Intact)', 'K-Prism (Image, Perturbed)',
|
| 5 |
+
];
|
| 6 |
+
|
| 7 |
+
const isText = value => typeof value === 'string' && value.trim().length > 0;
|
| 8 |
+
|
| 9 |
+
// Accept both the original benchmark.json array and a published {models} snapshot.
|
| 10 |
+
// Validate before replacing the visible results so failed refreshes keep good data.
|
| 11 |
+
export function normalizeBenchmark(payload) {
|
| 12 |
+
const source = Array.isArray(payload) ? payload : payload?.models;
|
| 13 |
+
if (!Array.isArray(source)) throw new Error('Expected a model array or an object with models.');
|
| 14 |
+
|
| 15 |
+
const modelKeys = new Set();
|
| 16 |
+
const models = source.map((model, index) => {
|
| 17 |
+
if (!model || !isText(model.provider) || !isText(model.name)) {
|
| 18 |
+
throw new Error(`Model ${index + 1} is missing its provider or name.`);
|
| 19 |
+
}
|
| 20 |
+
const key = JSON.stringify([model.provider, model.name]);
|
| 21 |
+
if (modelKeys.has(key)) throw new Error(`Duplicate model: ${model.provider} / ${model.name}.`);
|
| 22 |
+
modelKeys.add(key);
|
| 23 |
+
if (!Array.isArray(model.scores)) throw new Error(`Invalid scores for ${model.name}.`);
|
| 24 |
+
if (model.is_multimodal != null && typeof model.is_multimodal !== 'boolean') {
|
| 25 |
+
throw new Error(`Invalid model type for ${model.name}.`);
|
| 26 |
+
}
|
| 27 |
+
const scoreNames = new Set();
|
| 28 |
+
const scores = model.scores.map(score => {
|
| 29 |
+
if (!score || !isText(score.dataset_name) || !Number.isFinite(score.score)) {
|
| 30 |
+
throw new Error(`Invalid score for ${model.name}.`);
|
| 31 |
+
}
|
| 32 |
+
if (scoreNames.has(score.dataset_name)) {
|
| 33 |
+
throw new Error(`Duplicate dataset score for ${model.name}: ${score.dataset_name}.`);
|
| 34 |
+
}
|
| 35 |
+
scoreNames.add(score.dataset_name);
|
| 36 |
+
return {
|
| 37 |
+
dataset_name: score.dataset_name,
|
| 38 |
+
metric_type: score.metric_type === 'llm-as-judge' ? 'llm-as-judge' : 'raw',
|
| 39 |
+
score: score.score,
|
| 40 |
+
};
|
| 41 |
+
});
|
| 42 |
+
return {
|
| 43 |
+
provider: model.provider,
|
| 44 |
+
name: model.name,
|
| 45 |
+
repo: typeof model.repo === 'string' ? model.repo : '',
|
| 46 |
+
is_multimodal: model.is_multimodal ?? false,
|
| 47 |
+
status: typeof model.status === 'string' ? model.status : 'completed',
|
| 48 |
+
updated_at: typeof model.updated_at === 'string' ? model.updated_at : '',
|
| 49 |
+
scores,
|
| 50 |
+
};
|
| 51 |
+
});
|
| 52 |
+
|
| 53 |
+
const publishedDatasets = Array.isArray(payload) ? undefined : payload.datasets;
|
| 54 |
+
if (publishedDatasets !== undefined &&
|
| 55 |
+
(!Array.isArray(publishedDatasets) || !publishedDatasets.every(isText))) {
|
| 56 |
+
throw new Error('Invalid dataset list.');
|
| 57 |
+
}
|
| 58 |
+
// A published list preserves the curator's selection and order. Legacy arrays
|
| 59 |
+
// include the canonical datasets plus any additional scored datasets.
|
| 60 |
+
const datasets = [...new Set(publishedDatasets ?? [
|
| 61 |
+
...CANONICAL_DATASETS,
|
| 62 |
+
...models.flatMap(model => model.scores.map(score => score.dataset_name)),
|
| 63 |
+
])];
|
| 64 |
+
return { models, datasets };
|
| 65 |
+
}
|
| 66 |
+
|
| 67 |
+
export async function loadBenchmark(url, timeoutMs = 15000) {
|
| 68 |
+
const controller = new AbortController();
|
| 69 |
+
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
| 70 |
+
try {
|
| 71 |
+
const response = await fetch(url, {
|
| 72 |
+
cache: 'no-store',
|
| 73 |
+
credentials: 'omit',
|
| 74 |
+
signal: controller.signal,
|
| 75 |
+
});
|
| 76 |
+
if (!response.ok) throw new Error(`Score data request failed (HTTP ${response.status}).`);
|
| 77 |
+
return normalizeBenchmark(await response.json());
|
| 78 |
+
} finally {
|
| 79 |
+
clearTimeout(timeout);
|
| 80 |
+
}
|
| 81 |
+
}
|
front/data/benchmark.json
ADDED
|
@@ -0,0 +1,962 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"source": "https://doolykim22-keti-leaderboard.hf.space/api/results",
|
| 3 |
+
"captured_at": "2026-09-17T06:05:24+00:00",
|
| 4 |
+
"datasets": [
|
| 5 |
+
"일반 상식",
|
| 6 |
+
"멀티 모달",
|
| 7 |
+
"수치 정보",
|
| 8 |
+
"팩트 체크",
|
| 9 |
+
"소버린 벤치마크",
|
| 10 |
+
"K-Prism (Text, Intact)",
|
| 11 |
+
"K-Prism (Text, Perturbed)",
|
| 12 |
+
"K-Prism (Image, Intact)",
|
| 13 |
+
"K-Prism (Image, Perturbed)"
|
| 14 |
+
],
|
| 15 |
+
"models": [
|
| 16 |
+
{
|
| 17 |
+
"provider": "Alibaba Cloud",
|
| 18 |
+
"name": "qwen3-30b",
|
| 19 |
+
"repo": "Qwen/Qwen3-30B-A3B-Thinking-2507",
|
| 20 |
+
"is_multimodal": false,
|
| 21 |
+
"status": "completed",
|
| 22 |
+
"updated_at": "2026-06-26T06:07:08Z",
|
| 23 |
+
"scores": [
|
| 24 |
+
{
|
| 25 |
+
"dataset_name": "일반 상식",
|
| 26 |
+
"metric_type": "raw",
|
| 27 |
+
"score": 49.22
|
| 28 |
+
},
|
| 29 |
+
{
|
| 30 |
+
"dataset_name": "수치 정보",
|
| 31 |
+
"metric_type": "raw",
|
| 32 |
+
"score": 43.33
|
| 33 |
+
},
|
| 34 |
+
{
|
| 35 |
+
"dataset_name": "특수 도메인",
|
| 36 |
+
"metric_type": "raw",
|
| 37 |
+
"score": 100.0
|
| 38 |
+
},
|
| 39 |
+
{
|
| 40 |
+
"dataset_name": "소버린 벤치마크",
|
| 41 |
+
"metric_type": "raw",
|
| 42 |
+
"score": 62.14
|
| 43 |
+
}
|
| 44 |
+
]
|
| 45 |
+
},
|
| 46 |
+
{
|
| 47 |
+
"provider": "Alibaba Cloud",
|
| 48 |
+
"name": "qwen3-32b",
|
| 49 |
+
"repo": "Qwen/Qwen3-32B",
|
| 50 |
+
"is_multimodal": false,
|
| 51 |
+
"status": "completed",
|
| 52 |
+
"updated_at": "2026-06-26T07:06:54Z",
|
| 53 |
+
"scores": [
|
| 54 |
+
{
|
| 55 |
+
"dataset_name": "일반 상식",
|
| 56 |
+
"metric_type": "raw",
|
| 57 |
+
"score": 38.86
|
| 58 |
+
},
|
| 59 |
+
{
|
| 60 |
+
"dataset_name": "수치 정보",
|
| 61 |
+
"metric_type": "raw",
|
| 62 |
+
"score": 42.38
|
| 63 |
+
},
|
| 64 |
+
{
|
| 65 |
+
"dataset_name": "특수 도메인",
|
| 66 |
+
"metric_type": "raw",
|
| 67 |
+
"score": 100.0
|
| 68 |
+
},
|
| 69 |
+
{
|
| 70 |
+
"dataset_name": "소버린 벤치마크",
|
| 71 |
+
"metric_type": "raw",
|
| 72 |
+
"score": 58.64
|
| 73 |
+
}
|
| 74 |
+
]
|
| 75 |
+
},
|
| 76 |
+
{
|
| 77 |
+
"provider": "Alibaba Cloud",
|
| 78 |
+
"name": "qwen3-4b",
|
| 79 |
+
"repo": "Qwen/Qwen3-4B-Thinking-2507",
|
| 80 |
+
"is_multimodal": false,
|
| 81 |
+
"status": "completed",
|
| 82 |
+
"updated_at": "2026-06-26T07:22:21Z",
|
| 83 |
+
"scores": [
|
| 84 |
+
{
|
| 85 |
+
"dataset_name": "일반 상식",
|
| 86 |
+
"metric_type": "raw",
|
| 87 |
+
"score": 38.86
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"dataset_name": "수치 정보",
|
| 91 |
+
"metric_type": "raw",
|
| 92 |
+
"score": 34.76
|
| 93 |
+
},
|
| 94 |
+
{
|
| 95 |
+
"dataset_name": "특수 도메인",
|
| 96 |
+
"metric_type": "raw",
|
| 97 |
+
"score": 100.0
|
| 98 |
+
},
|
| 99 |
+
{
|
| 100 |
+
"dataset_name": "소버린 벤치마크",
|
| 101 |
+
"metric_type": "raw",
|
| 102 |
+
"score": 36.32
|
| 103 |
+
}
|
| 104 |
+
]
|
| 105 |
+
},
|
| 106 |
+
{
|
| 107 |
+
"provider": "Alibaba Cloud",
|
| 108 |
+
"name": "qwen3-8b",
|
| 109 |
+
"repo": "Qwen/Qwen3-8B",
|
| 110 |
+
"is_multimodal": false,
|
| 111 |
+
"status": "completed",
|
| 112 |
+
"updated_at": "2026-06-26T07:38:00Z",
|
| 113 |
+
"scores": [
|
| 114 |
+
{
|
| 115 |
+
"dataset_name": "일반 상식",
|
| 116 |
+
"metric_type": "raw",
|
| 117 |
+
"score": 10.36
|
| 118 |
+
},
|
| 119 |
+
{
|
| 120 |
+
"dataset_name": "수치 정보",
|
| 121 |
+
"metric_type": "raw",
|
| 122 |
+
"score": 33.81
|
| 123 |
+
},
|
| 124 |
+
{
|
| 125 |
+
"dataset_name": "특수 도메인",
|
| 126 |
+
"metric_type": "raw",
|
| 127 |
+
"score": 50.0
|
| 128 |
+
},
|
| 129 |
+
{
|
| 130 |
+
"dataset_name": "소버린 벤치마크",
|
| 131 |
+
"metric_type": "raw",
|
| 132 |
+
"score": 26.91
|
| 133 |
+
}
|
| 134 |
+
]
|
| 135 |
+
},
|
| 136 |
+
{
|
| 137 |
+
"provider": "Alibaba Cloud",
|
| 138 |
+
"name": "qwen3-vl-4b",
|
| 139 |
+
"repo": "Qwen/Qwen3-VL-4B-Thinking",
|
| 140 |
+
"is_multimodal": true,
|
| 141 |
+
"status": "completed",
|
| 142 |
+
"updated_at": "2026-06-26T07:49:35Z",
|
| 143 |
+
"scores": [
|
| 144 |
+
{
|
| 145 |
+
"dataset_name": "일반 상식",
|
| 146 |
+
"metric_type": "raw",
|
| 147 |
+
"score": 25.91
|
| 148 |
+
},
|
| 149 |
+
{
|
| 150 |
+
"dataset_name": "멀티 모달",
|
| 151 |
+
"metric_type": "raw",
|
| 152 |
+
"score": 28.21
|
| 153 |
+
},
|
| 154 |
+
{
|
| 155 |
+
"dataset_name": "수치 정보",
|
| 156 |
+
"metric_type": "raw",
|
| 157 |
+
"score": 19.52
|
| 158 |
+
},
|
| 159 |
+
{
|
| 160 |
+
"dataset_name": "팩트 체크",
|
| 161 |
+
"metric_type": "raw",
|
| 162 |
+
"score": 68.57
|
| 163 |
+
},
|
| 164 |
+
{
|
| 165 |
+
"dataset_name": "K-Prism (Text, Intact)",
|
| 166 |
+
"metric_type": "raw",
|
| 167 |
+
"score": 60.32
|
| 168 |
+
},
|
| 169 |
+
{
|
| 170 |
+
"dataset_name": "K-Prism (Text, Perturbed)",
|
| 171 |
+
"metric_type": "raw",
|
| 172 |
+
"score": 54.37
|
| 173 |
+
},
|
| 174 |
+
{
|
| 175 |
+
"dataset_name": "K-Prism (Image, Intact)",
|
| 176 |
+
"metric_type": "raw",
|
| 177 |
+
"score": 89.16
|
| 178 |
+
},
|
| 179 |
+
{
|
| 180 |
+
"dataset_name": "K-Prism (Image, Perturbed)",
|
| 181 |
+
"metric_type": "raw",
|
| 182 |
+
"score": 23.49
|
| 183 |
+
},
|
| 184 |
+
{
|
| 185 |
+
"dataset_name": "소버린 벤치마크",
|
| 186 |
+
"metric_type": "raw",
|
| 187 |
+
"score": 20.35
|
| 188 |
+
}
|
| 189 |
+
]
|
| 190 |
+
},
|
| 191 |
+
{
|
| 192 |
+
"provider": "Alibaba Cloud",
|
| 193 |
+
"name": "qwen3-vl-8b",
|
| 194 |
+
"repo": "Qwen/Qwen3-VL-8B-Instruct",
|
| 195 |
+
"is_multimodal": true,
|
| 196 |
+
"status": "completed",
|
| 197 |
+
"updated_at": "2026-06-26T08:15:48Z",
|
| 198 |
+
"scores": [
|
| 199 |
+
{
|
| 200 |
+
"dataset_name": "일반 상식",
|
| 201 |
+
"metric_type": "raw",
|
| 202 |
+
"score": 1.55
|
| 203 |
+
},
|
| 204 |
+
{
|
| 205 |
+
"dataset_name": "멀티 모달",
|
| 206 |
+
"metric_type": "raw",
|
| 207 |
+
"score": 41.07
|
| 208 |
+
},
|
| 209 |
+
{
|
| 210 |
+
"dataset_name": "수치 정보",
|
| 211 |
+
"metric_type": "raw",
|
| 212 |
+
"score": 37.62
|
| 213 |
+
},
|
| 214 |
+
{
|
| 215 |
+
"dataset_name": "팩트 체크",
|
| 216 |
+
"metric_type": "raw",
|
| 217 |
+
"score": 71.43
|
| 218 |
+
},
|
| 219 |
+
{
|
| 220 |
+
"dataset_name": "K-Prism (Text, Intact)",
|
| 221 |
+
"metric_type": "raw",
|
| 222 |
+
"score": 61.31
|
| 223 |
+
},
|
| 224 |
+
{
|
| 225 |
+
"dataset_name": "K-Prism (Text, Perturbed)",
|
| 226 |
+
"metric_type": "raw",
|
| 227 |
+
"score": 61.31
|
| 228 |
+
},
|
| 229 |
+
{
|
| 230 |
+
"dataset_name": "K-Prism (Image, Intact)",
|
| 231 |
+
"metric_type": "raw",
|
| 232 |
+
"score": 80.32
|
| 233 |
+
},
|
| 234 |
+
{
|
| 235 |
+
"dataset_name": "K-Prism (Image, Perturbed)",
|
| 236 |
+
"metric_type": "raw",
|
| 237 |
+
"score": 21.69
|
| 238 |
+
},
|
| 239 |
+
{
|
| 240 |
+
"dataset_name": "소버린 벤치마크",
|
| 241 |
+
"metric_type": "raw",
|
| 242 |
+
"score": 3.94
|
| 243 |
+
}
|
| 244 |
+
]
|
| 245 |
+
},
|
| 246 |
+
{
|
| 247 |
+
"provider": "Alibaba Cloud",
|
| 248 |
+
"name": "qwen3-vl-30b",
|
| 249 |
+
"repo": "Qwen/Qwen3-VL-30B-A3B-Thinking",
|
| 250 |
+
"is_multimodal": true,
|
| 251 |
+
"status": "completed",
|
| 252 |
+
"updated_at": "2026-06-29T00:55:39Z",
|
| 253 |
+
"scores": [
|
| 254 |
+
{
|
| 255 |
+
"dataset_name": "일반 상식",
|
| 256 |
+
"metric_type": "raw",
|
| 257 |
+
"score": 34.72
|
| 258 |
+
},
|
| 259 |
+
{
|
| 260 |
+
"dataset_name": "멀티 모달",
|
| 261 |
+
"metric_type": "raw",
|
| 262 |
+
"score": 42.14
|
| 263 |
+
},
|
| 264 |
+
{
|
| 265 |
+
"dataset_name": "수치 정보",
|
| 266 |
+
"metric_type": "raw",
|
| 267 |
+
"score": 46.19
|
| 268 |
+
},
|
| 269 |
+
{
|
| 270 |
+
"dataset_name": "팩트 체크",
|
| 271 |
+
"metric_type": "raw",
|
| 272 |
+
"score": 78.57
|
| 273 |
+
},
|
| 274 |
+
{
|
| 275 |
+
"dataset_name": "K-Prism (Text, Intact)",
|
| 276 |
+
"metric_type": "raw",
|
| 277 |
+
"score": 62.9
|
| 278 |
+
},
|
| 279 |
+
{
|
| 280 |
+
"dataset_name": "K-Prism (Text, Perturbed)",
|
| 281 |
+
"metric_type": "raw",
|
| 282 |
+
"score": 58.33
|
| 283 |
+
},
|
| 284 |
+
{
|
| 285 |
+
"dataset_name": "K-Prism (Image, Intact)",
|
| 286 |
+
"metric_type": "raw",
|
| 287 |
+
"score": 82.33
|
| 288 |
+
},
|
| 289 |
+
{
|
| 290 |
+
"dataset_name": "K-Prism (Image, Perturbed)",
|
| 291 |
+
"metric_type": "raw",
|
| 292 |
+
"score": 28.92
|
| 293 |
+
},
|
| 294 |
+
{
|
| 295 |
+
"dataset_name": "소버린 벤치마크",
|
| 296 |
+
"metric_type": "raw",
|
| 297 |
+
"score": 49.89
|
| 298 |
+
}
|
| 299 |
+
]
|
| 300 |
+
},
|
| 301 |
+
{
|
| 302 |
+
"provider": "Alibaba Cloud",
|
| 303 |
+
"name": "qwen3-vl-32b",
|
| 304 |
+
"repo": "Qwen/Qwen3-VL-32B-Thinking",
|
| 305 |
+
"is_multimodal": true,
|
| 306 |
+
"status": "completed",
|
| 307 |
+
"updated_at": "2026-06-29T02:00:53Z",
|
| 308 |
+
"scores": [
|
| 309 |
+
{
|
| 310 |
+
"dataset_name": "일반 상식",
|
| 311 |
+
"metric_type": "raw",
|
| 312 |
+
"score": 23.83
|
| 313 |
+
},
|
| 314 |
+
{
|
| 315 |
+
"dataset_name": "멀티 모달",
|
| 316 |
+
"metric_type": "raw",
|
| 317 |
+
"score": 45.36
|
| 318 |
+
},
|
| 319 |
+
{
|
| 320 |
+
"dataset_name": "수치 정보",
|
| 321 |
+
"metric_type": "raw",
|
| 322 |
+
"score": 44.29
|
| 323 |
+
},
|
| 324 |
+
{
|
| 325 |
+
"dataset_name": "팩트 체크",
|
| 326 |
+
"metric_type": "raw",
|
| 327 |
+
"score": 82.86
|
| 328 |
+
},
|
| 329 |
+
{
|
| 330 |
+
"dataset_name": "K-Prism (Text, Intact)",
|
| 331 |
+
"metric_type": "raw",
|
| 332 |
+
"score": 63.89
|
| 333 |
+
},
|
| 334 |
+
{
|
| 335 |
+
"dataset_name": "K-Prism (Text, Perturbed)",
|
| 336 |
+
"metric_type": "raw",
|
| 337 |
+
"score": 60.32
|
| 338 |
+
},
|
| 339 |
+
{
|
| 340 |
+
"dataset_name": "K-Prism (Image, Intact)",
|
| 341 |
+
"metric_type": "raw",
|
| 342 |
+
"score": 69.68
|
| 343 |
+
},
|
| 344 |
+
{
|
| 345 |
+
"dataset_name": "K-Prism (Image, Perturbed)",
|
| 346 |
+
"metric_type": "raw",
|
| 347 |
+
"score": 28.11
|
| 348 |
+
},
|
| 349 |
+
{
|
| 350 |
+
"dataset_name": "소버린 벤치마크",
|
| 351 |
+
"metric_type": "raw",
|
| 352 |
+
"score": 22.54
|
| 353 |
+
}
|
| 354 |
+
]
|
| 355 |
+
},
|
| 356 |
+
{
|
| 357 |
+
"provider": "Kakao",
|
| 358 |
+
"name": "kanana1.5",
|
| 359 |
+
"repo": "kakaocorp/kanana-1.5-v-3b-instruct",
|
| 360 |
+
"is_multimodal": true,
|
| 361 |
+
"status": "completed",
|
| 362 |
+
"updated_at": "2026-06-29T02:07:21Z",
|
| 363 |
+
"scores": [
|
| 364 |
+
{
|
| 365 |
+
"dataset_name": "일반 상식",
|
| 366 |
+
"metric_type": "raw",
|
| 367 |
+
"score": 49.22
|
| 368 |
+
},
|
| 369 |
+
{
|
| 370 |
+
"dataset_name": "멀티 모달",
|
| 371 |
+
"metric_type": "raw",
|
| 372 |
+
"score": 23.93
|
| 373 |
+
},
|
| 374 |
+
{
|
| 375 |
+
"dataset_name": "수치 정보",
|
| 376 |
+
"metric_type": "raw",
|
| 377 |
+
"score": 35.24
|
| 378 |
+
},
|
| 379 |
+
{
|
| 380 |
+
"dataset_name": "팩트 체크",
|
| 381 |
+
"metric_type": "raw",
|
| 382 |
+
"score": 58.57
|
| 383 |
+
},
|
| 384 |
+
{
|
| 385 |
+
"dataset_name": "특수 도메인",
|
| 386 |
+
"metric_type": "raw",
|
| 387 |
+
"score": 100.0
|
| 388 |
+
},
|
| 389 |
+
{
|
| 390 |
+
"dataset_name": "K-Prism (Text, Intact)",
|
| 391 |
+
"metric_type": "raw",
|
| 392 |
+
"score": 8.33
|
| 393 |
+
},
|
| 394 |
+
{
|
| 395 |
+
"dataset_name": "K-Prism (Text, Perturbed)",
|
| 396 |
+
"metric_type": "raw",
|
| 397 |
+
"score": 59.52
|
| 398 |
+
},
|
| 399 |
+
{
|
| 400 |
+
"dataset_name": "K-Prism (Image, Intact)",
|
| 401 |
+
"metric_type": "raw",
|
| 402 |
+
"score": 85.34
|
| 403 |
+
},
|
| 404 |
+
{
|
| 405 |
+
"dataset_name": "K-Prism (Image, Perturbed)",
|
| 406 |
+
"metric_type": "raw",
|
| 407 |
+
"score": 12.85
|
| 408 |
+
},
|
| 409 |
+
{
|
| 410 |
+
"dataset_name": "소버린 벤치마크",
|
| 411 |
+
"metric_type": "raw",
|
| 412 |
+
"score": 39.82
|
| 413 |
+
}
|
| 414 |
+
]
|
| 415 |
+
},
|
| 416 |
+
{
|
| 417 |
+
"provider": "Shanghai AI Laboratory (OpenGVLab)",
|
| 418 |
+
"name": "InternVL3-8B",
|
| 419 |
+
"repo": "OpenGVLab/InternVL3-8B",
|
| 420 |
+
"is_multimodal": true,
|
| 421 |
+
"status": "completed",
|
| 422 |
+
"updated_at": "2026-06-29T02:19:52Z",
|
| 423 |
+
"scores": [
|
| 424 |
+
{
|
| 425 |
+
"dataset_name": "일반 상식",
|
| 426 |
+
"metric_type": "raw",
|
| 427 |
+
"score": 43.01
|
| 428 |
+
},
|
| 429 |
+
{
|
| 430 |
+
"dataset_name": "멀티 모달",
|
| 431 |
+
"metric_type": "raw",
|
| 432 |
+
"score": 22.86
|
| 433 |
+
},
|
| 434 |
+
{
|
| 435 |
+
"dataset_name": "수치 정보",
|
| 436 |
+
"metric_type": "raw",
|
| 437 |
+
"score": 43.81
|
| 438 |
+
},
|
| 439 |
+
{
|
| 440 |
+
"dataset_name": "팩트 체크",
|
| 441 |
+
"metric_type": "raw",
|
| 442 |
+
"score": 64.29
|
| 443 |
+
},
|
| 444 |
+
{
|
| 445 |
+
"dataset_name": "특수 도메인",
|
| 446 |
+
"metric_type": "raw",
|
| 447 |
+
"score": 100.0
|
| 448 |
+
},
|
| 449 |
+
{
|
| 450 |
+
"dataset_name": "K-Prism (Text, Intact)",
|
| 451 |
+
"metric_type": "raw",
|
| 452 |
+
"score": 60.52
|
| 453 |
+
},
|
| 454 |
+
{
|
| 455 |
+
"dataset_name": "K-Prism (Text, Perturbed)",
|
| 456 |
+
"metric_type": "raw",
|
| 457 |
+
"score": 59.72
|
| 458 |
+
},
|
| 459 |
+
{
|
| 460 |
+
"dataset_name": "K-Prism (Image, Intact)",
|
| 461 |
+
"metric_type": "raw",
|
| 462 |
+
"score": 66.67
|
| 463 |
+
},
|
| 464 |
+
{
|
| 465 |
+
"dataset_name": "K-Prism (Image, Perturbed)",
|
| 466 |
+
"metric_type": "raw",
|
| 467 |
+
"score": 39.76
|
| 468 |
+
},
|
| 469 |
+
{
|
| 470 |
+
"dataset_name": "소버린 벤치마크",
|
| 471 |
+
"metric_type": "raw",
|
| 472 |
+
"score": 42.67
|
| 473 |
+
}
|
| 474 |
+
]
|
| 475 |
+
},
|
| 476 |
+
{
|
| 477 |
+
"provider": "NCSOFT",
|
| 478 |
+
"name": "varco",
|
| 479 |
+
"repo": "NCSOFT/VARCO-VISION-2.0-1.7B",
|
| 480 |
+
"is_multimodal": true,
|
| 481 |
+
"status": "completed",
|
| 482 |
+
"updated_at": "2026-06-29T05:03:10Z",
|
| 483 |
+
"scores": [
|
| 484 |
+
{
|
| 485 |
+
"dataset_name": "일반 상식",
|
| 486 |
+
"metric_type": "raw",
|
| 487 |
+
"score": 16.06
|
| 488 |
+
},
|
| 489 |
+
{
|
| 490 |
+
"dataset_name": "멀티 모달",
|
| 491 |
+
"metric_type": "raw",
|
| 492 |
+
"score": 31.43
|
| 493 |
+
},
|
| 494 |
+
{
|
| 495 |
+
"dataset_name": "수치 정보",
|
| 496 |
+
"metric_type": "raw",
|
| 497 |
+
"score": 9.52
|
| 498 |
+
},
|
| 499 |
+
{
|
| 500 |
+
"dataset_name": "팩트 체크",
|
| 501 |
+
"metric_type": "raw",
|
| 502 |
+
"score": 58.57
|
| 503 |
+
},
|
| 504 |
+
{
|
| 505 |
+
"dataset_name": "특수 도메인",
|
| 506 |
+
"metric_type": "raw",
|
| 507 |
+
"score": 100.0
|
| 508 |
+
},
|
| 509 |
+
{
|
| 510 |
+
"dataset_name": "K-Prism (Text, Intact)",
|
| 511 |
+
"metric_type": "raw",
|
| 512 |
+
"score": 56.94
|
| 513 |
+
},
|
| 514 |
+
{
|
| 515 |
+
"dataset_name": "K-Prism (Text, Perturbed)",
|
| 516 |
+
"metric_type": "raw",
|
| 517 |
+
"score": 55.16
|
| 518 |
+
},
|
| 519 |
+
{
|
| 520 |
+
"dataset_name": "K-Prism (Image, Intact)",
|
| 521 |
+
"metric_type": "raw",
|
| 522 |
+
"score": 79.32
|
| 523 |
+
},
|
| 524 |
+
{
|
| 525 |
+
"dataset_name": "K-Prism (Image, Perturbed)",
|
| 526 |
+
"metric_type": "raw",
|
| 527 |
+
"score": 44.58
|
| 528 |
+
},
|
| 529 |
+
{
|
| 530 |
+
"dataset_name": "소버린 벤치마크",
|
| 531 |
+
"metric_type": "raw",
|
| 532 |
+
"score": 31.51
|
| 533 |
+
}
|
| 534 |
+
]
|
| 535 |
+
},
|
| 536 |
+
{
|
| 537 |
+
"provider": "SK Telecom",
|
| 538 |
+
"name": "ax4.0",
|
| 539 |
+
"repo": "skt/A.X-4.0-VL-Light",
|
| 540 |
+
"is_multimodal": true,
|
| 541 |
+
"status": "completed",
|
| 542 |
+
"updated_at": "2026-06-29T05:31:35Z",
|
| 543 |
+
"scores": [
|
| 544 |
+
{
|
| 545 |
+
"dataset_name": "일반 상식",
|
| 546 |
+
"metric_type": "raw",
|
| 547 |
+
"score": 20.73
|
| 548 |
+
},
|
| 549 |
+
{
|
| 550 |
+
"dataset_name": "멀티 모달",
|
| 551 |
+
"metric_type": "raw",
|
| 552 |
+
"score": 38.21
|
| 553 |
+
},
|
| 554 |
+
{
|
| 555 |
+
"dataset_name": "수치 정보",
|
| 556 |
+
"metric_type": "raw",
|
| 557 |
+
"score": 36.67
|
| 558 |
+
},
|
| 559 |
+
{
|
| 560 |
+
"dataset_name": "팩트 체크",
|
| 561 |
+
"metric_type": "raw",
|
| 562 |
+
"score": 74.29
|
| 563 |
+
},
|
| 564 |
+
{
|
| 565 |
+
"dataset_name": "특수 도메인",
|
| 566 |
+
"metric_type": "raw",
|
| 567 |
+
"score": 100.0
|
| 568 |
+
},
|
| 569 |
+
{
|
| 570 |
+
"dataset_name": "K-Prism (Text, Intact)",
|
| 571 |
+
"metric_type": "raw",
|
| 572 |
+
"score": 61.51
|
| 573 |
+
},
|
| 574 |
+
{
|
| 575 |
+
"dataset_name": "K-Prism (Text, Perturbed)",
|
| 576 |
+
"metric_type": "raw",
|
| 577 |
+
"score": 61.51
|
| 578 |
+
},
|
| 579 |
+
{
|
| 580 |
+
"dataset_name": "K-Prism (Image, Intact)",
|
| 581 |
+
"metric_type": "raw",
|
| 582 |
+
"score": 84.14
|
| 583 |
+
},
|
| 584 |
+
{
|
| 585 |
+
"dataset_name": "K-Prism (Image, Perturbed)",
|
| 586 |
+
"metric_type": "raw",
|
| 587 |
+
"score": 8.63
|
| 588 |
+
},
|
| 589 |
+
{
|
| 590 |
+
"dataset_name": "소버린 벤치마크",
|
| 591 |
+
"metric_type": "raw",
|
| 592 |
+
"score": 30.85
|
| 593 |
+
}
|
| 594 |
+
]
|
| 595 |
+
},
|
| 596 |
+
{
|
| 597 |
+
"provider": "NAVER Cloud",
|
| 598 |
+
"name": "clova3b",
|
| 599 |
+
"repo": "naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B",
|
| 600 |
+
"is_multimodal": true,
|
| 601 |
+
"status": "completed",
|
| 602 |
+
"updated_at": "2026-06-29T05:44:09Z",
|
| 603 |
+
"scores": [
|
| 604 |
+
{
|
| 605 |
+
"dataset_name": "일반 상식",
|
| 606 |
+
"metric_type": "raw",
|
| 607 |
+
"score": 15.03
|
| 608 |
+
},
|
| 609 |
+
{
|
| 610 |
+
"dataset_name": "멀티 모달",
|
| 611 |
+
"metric_type": "raw",
|
| 612 |
+
"score": 26.43
|
| 613 |
+
},
|
| 614 |
+
{
|
| 615 |
+
"dataset_name": "수치 정보",
|
| 616 |
+
"metric_type": "raw",
|
| 617 |
+
"score": 38.57
|
| 618 |
+
},
|
| 619 |
+
{
|
| 620 |
+
"dataset_name": "팩트 체크",
|
| 621 |
+
"metric_type": "raw",
|
| 622 |
+
"score": 67.14
|
| 623 |
+
},
|
| 624 |
+
{
|
| 625 |
+
"dataset_name": "특수 도메인",
|
| 626 |
+
"metric_type": "raw",
|
| 627 |
+
"score": 100.0
|
| 628 |
+
},
|
| 629 |
+
{
|
| 630 |
+
"dataset_name": "K-Prism (Text, Intact)",
|
| 631 |
+
"metric_type": "raw",
|
| 632 |
+
"score": 8.33
|
| 633 |
+
},
|
| 634 |
+
{
|
| 635 |
+
"dataset_name": "K-Prism (Text, Perturbed)",
|
| 636 |
+
"metric_type": "raw",
|
| 637 |
+
"score": 74.8
|
| 638 |
+
},
|
| 639 |
+
{
|
| 640 |
+
"dataset_name": "K-Prism (Image, Intact)",
|
| 641 |
+
"metric_type": "raw",
|
| 642 |
+
"score": 67.07
|
| 643 |
+
},
|
| 644 |
+
{
|
| 645 |
+
"dataset_name": "K-Prism (Image, Perturbed)",
|
| 646 |
+
"metric_type": "raw",
|
| 647 |
+
"score": 63.05
|
| 648 |
+
},
|
| 649 |
+
{
|
| 650 |
+
"dataset_name": "소버린 벤치마크",
|
| 651 |
+
"metric_type": "raw",
|
| 652 |
+
"score": 13.13
|
| 653 |
+
}
|
| 654 |
+
]
|
| 655 |
+
},
|
| 656 |
+
{
|
| 657 |
+
"provider": "OpenAI",
|
| 658 |
+
"name": "gpt-oss-20b",
|
| 659 |
+
"repo": "openai/gpt-oss-20b",
|
| 660 |
+
"is_multimodal": false,
|
| 661 |
+
"status": "completed",
|
| 662 |
+
"updated_at": "2026-06-29T06:09:40Z",
|
| 663 |
+
"scores": [
|
| 664 |
+
{
|
| 665 |
+
"dataset_name": "일반 상식",
|
| 666 |
+
"metric_type": "raw",
|
| 667 |
+
"score": 25.39
|
| 668 |
+
},
|
| 669 |
+
{
|
| 670 |
+
"dataset_name": "수치 정보",
|
| 671 |
+
"metric_type": "raw",
|
| 672 |
+
"score": 38.1
|
| 673 |
+
},
|
| 674 |
+
{
|
| 675 |
+
"dataset_name": "특수 도메인",
|
| 676 |
+
"metric_type": "raw",
|
| 677 |
+
"score": 100.0
|
| 678 |
+
},
|
| 679 |
+
{
|
| 680 |
+
"dataset_name": "소버린 벤치마크",
|
| 681 |
+
"metric_type": "raw",
|
| 682 |
+
"score": 15.1
|
| 683 |
+
}
|
| 684 |
+
]
|
| 685 |
+
},
|
| 686 |
+
{
|
| 687 |
+
"provider": "Shanghai AI Laboratory (OpenGVLab)",
|
| 688 |
+
"name": "InternVL3-9B",
|
| 689 |
+
"repo": "OpenGVLab/InternVL3-9B",
|
| 690 |
+
"is_multimodal": true,
|
| 691 |
+
"status": "completed",
|
| 692 |
+
"updated_at": "2026-06-29T06:16:28Z",
|
| 693 |
+
"scores": [
|
| 694 |
+
{
|
| 695 |
+
"dataset_name": "일반 상식",
|
| 696 |
+
"metric_type": "raw",
|
| 697 |
+
"score": 41.97
|
| 698 |
+
},
|
| 699 |
+
{
|
| 700 |
+
"dataset_name": "멀티 모달",
|
| 701 |
+
"metric_type": "raw",
|
| 702 |
+
"score": 29.64
|
| 703 |
+
},
|
| 704 |
+
{
|
| 705 |
+
"dataset_name": "수치 정보",
|
| 706 |
+
"metric_type": "raw",
|
| 707 |
+
"score": 39.52
|
| 708 |
+
},
|
| 709 |
+
{
|
| 710 |
+
"dataset_name": "K-Prism (Text, Intact)",
|
| 711 |
+
"metric_type": "raw",
|
| 712 |
+
"score": 57.74
|
| 713 |
+
},
|
| 714 |
+
{
|
| 715 |
+
"dataset_name": "K-Prism (Text, Perturbed)",
|
| 716 |
+
"metric_type": "raw",
|
| 717 |
+
"score": 57.34
|
| 718 |
+
},
|
| 719 |
+
{
|
| 720 |
+
"dataset_name": "K-Prism (Image, Intact)",
|
| 721 |
+
"metric_type": "raw",
|
| 722 |
+
"score": 81.12
|
| 723 |
+
},
|
| 724 |
+
{
|
| 725 |
+
"dataset_name": "K-Prism (Image, Perturbed)",
|
| 726 |
+
"metric_type": "raw",
|
| 727 |
+
"score": 29.92
|
| 728 |
+
},
|
| 729 |
+
{
|
| 730 |
+
"dataset_name": "팩트 체크",
|
| 731 |
+
"metric_type": "raw",
|
| 732 |
+
"score": 67.14
|
| 733 |
+
},
|
| 734 |
+
{
|
| 735 |
+
"dataset_name": "소버린 벤치마크",
|
| 736 |
+
"metric_type": "raw",
|
| 737 |
+
"score": 38.07
|
| 738 |
+
}
|
| 739 |
+
]
|
| 740 |
+
},
|
| 741 |
+
{
|
| 742 |
+
"provider": "Shanghai AI Laboratory (OpenGVLab)",
|
| 743 |
+
"name": "InternVL3-14B",
|
| 744 |
+
"repo": "OpenGVLab/InternVL3-14B",
|
| 745 |
+
"is_multimodal": true,
|
| 746 |
+
"status": "completed",
|
| 747 |
+
"updated_at": "2026-06-29T06:37:45Z",
|
| 748 |
+
"scores": [
|
| 749 |
+
{
|
| 750 |
+
"dataset_name": "일반 상식",
|
| 751 |
+
"metric_type": "raw",
|
| 752 |
+
"score": 52.85
|
| 753 |
+
},
|
| 754 |
+
{
|
| 755 |
+
"dataset_name": "멀티 모달",
|
| 756 |
+
"metric_type": "raw",
|
| 757 |
+
"score": 29.29
|
| 758 |
+
},
|
| 759 |
+
{
|
| 760 |
+
"dataset_name": "수치 정보",
|
| 761 |
+
"metric_type": "raw",
|
| 762 |
+
"score": 38.1
|
| 763 |
+
},
|
| 764 |
+
{
|
| 765 |
+
"dataset_name": "팩트 체크",
|
| 766 |
+
"metric_type": "raw",
|
| 767 |
+
"score": 71.43
|
| 768 |
+
},
|
| 769 |
+
{
|
| 770 |
+
"dataset_name": "K-Prism (Text, Intact)",
|
| 771 |
+
"metric_type": "raw",
|
| 772 |
+
"score": 61.31
|
| 773 |
+
},
|
| 774 |
+
{
|
| 775 |
+
"dataset_name": "K-Prism (Text, Perturbed)",
|
| 776 |
+
"metric_type": "raw",
|
| 777 |
+
"score": 66.47
|
| 778 |
+
},
|
| 779 |
+
{
|
| 780 |
+
"dataset_name": "K-Prism (Image, Intact)",
|
| 781 |
+
"metric_type": "raw",
|
| 782 |
+
"score": 78.51
|
| 783 |
+
},
|
| 784 |
+
{
|
| 785 |
+
"dataset_name": "K-Prism (Image, Perturbed)",
|
| 786 |
+
"metric_type": "raw",
|
| 787 |
+
"score": 27.31
|
| 788 |
+
},
|
| 789 |
+
{
|
| 790 |
+
"dataset_name": "소버린 벤치마크",
|
| 791 |
+
"metric_type": "raw",
|
| 792 |
+
"score": 47.26
|
| 793 |
+
}
|
| 794 |
+
]
|
| 795 |
+
},
|
| 796 |
+
{
|
| 797 |
+
"provider": "Shanghai AI Laboratory (OpenGVLab)",
|
| 798 |
+
"name": "InternVL3-38B",
|
| 799 |
+
"repo": "OpenGVLab/InternVL3-38B",
|
| 800 |
+
"is_multimodal": true,
|
| 801 |
+
"status": "completed",
|
| 802 |
+
"updated_at": "2026-06-29T06:58:18Z",
|
| 803 |
+
"scores": [
|
| 804 |
+
{
|
| 805 |
+
"dataset_name": "일반 상식",
|
| 806 |
+
"metric_type": "raw",
|
| 807 |
+
"score": 56.99
|
| 808 |
+
},
|
| 809 |
+
{
|
| 810 |
+
"dataset_name": "멀티 모달",
|
| 811 |
+
"metric_type": "raw",
|
| 812 |
+
"score": 30.36
|
| 813 |
+
},
|
| 814 |
+
{
|
| 815 |
+
"dataset_name": "수치 정보",
|
| 816 |
+
"metric_type": "raw",
|
| 817 |
+
"score": 46.67
|
| 818 |
+
},
|
| 819 |
+
{
|
| 820 |
+
"dataset_name": "팩트 체크",
|
| 821 |
+
"metric_type": "raw",
|
| 822 |
+
"score": 75.71
|
| 823 |
+
},
|
| 824 |
+
{
|
| 825 |
+
"dataset_name": "K-Prism (Text, Intact)",
|
| 826 |
+
"metric_type": "raw",
|
| 827 |
+
"score": 61.11
|
| 828 |
+
},
|
| 829 |
+
{
|
| 830 |
+
"dataset_name": "K-Prism (Text, Perturbed)",
|
| 831 |
+
"metric_type": "raw",
|
| 832 |
+
"score": 59.92
|
| 833 |
+
},
|
| 834 |
+
{
|
| 835 |
+
"dataset_name": "K-Prism (Image, Intact)",
|
| 836 |
+
"metric_type": "raw",
|
| 837 |
+
"score": 78.11
|
| 838 |
+
},
|
| 839 |
+
{
|
| 840 |
+
"dataset_name": "K-Prism (Image, Perturbed)",
|
| 841 |
+
"metric_type": "raw",
|
| 842 |
+
"score": 39.96
|
| 843 |
+
},
|
| 844 |
+
{
|
| 845 |
+
"dataset_name": "소버린 벤치마크",
|
| 846 |
+
"metric_type": "raw",
|
| 847 |
+
"score": 56.89
|
| 848 |
+
}
|
| 849 |
+
]
|
| 850 |
+
},
|
| 851 |
+
{
|
| 852 |
+
"provider": "Shanghai AI Laboratory (OpenGVLab)",
|
| 853 |
+
"name": "InternVL3-8B-Pretrained",
|
| 854 |
+
"repo": "OpenGVLab/InternVL3-8B-Pretrained",
|
| 855 |
+
"is_multimodal": true,
|
| 856 |
+
"status": "completed",
|
| 857 |
+
"updated_at": "2026-06-29T07:04:44Z",
|
| 858 |
+
"scores": [
|
| 859 |
+
{
|
| 860 |
+
"dataset_name": "일반 상식",
|
| 861 |
+
"metric_type": "raw",
|
| 862 |
+
"score": 41.45
|
| 863 |
+
},
|
| 864 |
+
{
|
| 865 |
+
"dataset_name": "멀티 모달",
|
| 866 |
+
"metric_type": "raw",
|
| 867 |
+
"score": 17.5
|
| 868 |
+
},
|
| 869 |
+
{
|
| 870 |
+
"dataset_name": "수치 정보",
|
| 871 |
+
"metric_type": "raw",
|
| 872 |
+
"score": 42.86
|
| 873 |
+
},
|
| 874 |
+
{
|
| 875 |
+
"dataset_name": "팩트 체크",
|
| 876 |
+
"metric_type": "raw",
|
| 877 |
+
"score": 60.0
|
| 878 |
+
},
|
| 879 |
+
{
|
| 880 |
+
"dataset_name": "K-Prism (Text, Intact)",
|
| 881 |
+
"metric_type": "raw",
|
| 882 |
+
"score": 58.33
|
| 883 |
+
},
|
| 884 |
+
{
|
| 885 |
+
"dataset_name": "K-Prism (Text, Perturbed)",
|
| 886 |
+
"metric_type": "raw",
|
| 887 |
+
"score": 57.14
|
| 888 |
+
},
|
| 889 |
+
{
|
| 890 |
+
"dataset_name": "K-Prism (Image, Intact)",
|
| 891 |
+
"metric_type": "raw",
|
| 892 |
+
"score": 65.46
|
| 893 |
+
},
|
| 894 |
+
{
|
| 895 |
+
"dataset_name": "K-Prism (Image, Perturbed)",
|
| 896 |
+
"metric_type": "raw",
|
| 897 |
+
"score": 47.59
|
| 898 |
+
},
|
| 899 |
+
{
|
| 900 |
+
"dataset_name": "소버린 벤치마크",
|
| 901 |
+
"metric_type": "raw",
|
| 902 |
+
"score": 45.95
|
| 903 |
+
}
|
| 904 |
+
]
|
| 905 |
+
},
|
| 906 |
+
{
|
| 907 |
+
"provider": "Shanghai AI Laboratory (OpenGVLab)",
|
| 908 |
+
"name": "InternVL3-14B-Pretrained",
|
| 909 |
+
"repo": "OpenGVLab/InternVL3-14B-Pretrained",
|
| 910 |
+
"is_multimodal": true,
|
| 911 |
+
"status": "completed",
|
| 912 |
+
"updated_at": "2026-06-29T07:12:43Z",
|
| 913 |
+
"scores": [
|
| 914 |
+
{
|
| 915 |
+
"dataset_name": "일반 상식",
|
| 916 |
+
"metric_type": "raw",
|
| 917 |
+
"score": 54.4
|
| 918 |
+
},
|
| 919 |
+
{
|
| 920 |
+
"dataset_name": "멀티 모달",
|
| 921 |
+
"metric_type": "raw",
|
| 922 |
+
"score": 25.71
|
| 923 |
+
},
|
| 924 |
+
{
|
| 925 |
+
"dataset_name": "수치 정보",
|
| 926 |
+
"metric_type": "raw",
|
| 927 |
+
"score": 42.38
|
| 928 |
+
},
|
| 929 |
+
{
|
| 930 |
+
"dataset_name": "팩트 체크",
|
| 931 |
+
"metric_type": "raw",
|
| 932 |
+
"score": 70.0
|
| 933 |
+
},
|
| 934 |
+
{
|
| 935 |
+
"dataset_name": "K-Prism (Text, Intact)",
|
| 936 |
+
"metric_type": "raw",
|
| 937 |
+
"score": 63.1
|
| 938 |
+
},
|
| 939 |
+
{
|
| 940 |
+
"dataset_name": "K-Prism (Text, Perturbed)",
|
| 941 |
+
"metric_type": "raw",
|
| 942 |
+
"score": 72.22
|
| 943 |
+
},
|
| 944 |
+
{
|
| 945 |
+
"dataset_name": "K-Prism (Image, Intact)",
|
| 946 |
+
"metric_type": "raw",
|
| 947 |
+
"score": 78.11
|
| 948 |
+
},
|
| 949 |
+
{
|
| 950 |
+
"dataset_name": "K-Prism (Image, Perturbed)",
|
| 951 |
+
"metric_type": "raw",
|
| 952 |
+
"score": 54.02
|
| 953 |
+
},
|
| 954 |
+
{
|
| 955 |
+
"dataset_name": "소버린 벤치마크",
|
| 956 |
+
"metric_type": "raw",
|
| 957 |
+
"score": 50.33
|
| 958 |
+
}
|
| 959 |
+
]
|
| 960 |
+
}
|
| 961 |
+
]
|
| 962 |
+
}
|
front/leaderboard.mjs
ADDED
|
@@ -0,0 +1,413 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import { config } from './config.mjs';
|
| 2 |
+
import { loadBenchmark } from './data.mjs';
|
| 3 |
+
|
| 4 |
+
// ===== Config =====
|
| 5 |
+
const MAX_COMPARE = 999; // Allow selecting all models
|
| 6 |
+
|
| 7 |
+
// ===== State =====
|
| 8 |
+
const state = {
|
| 9 |
+
datasets: [],
|
| 10 |
+
models: [],
|
| 11 |
+
selectedDatasets: new Set(),
|
| 12 |
+
providerFilter: "",
|
| 13 |
+
search: "",
|
| 14 |
+
compare: new Set(),
|
| 15 |
+
dark: window.matchMedia && window.matchMedia('(prefers-color-scheme: dark)').matches,
|
| 16 |
+
};
|
| 17 |
+
|
| 18 |
+
// ===== Utils =====
|
| 19 |
+
const $ = (s)=>document.querySelector(s);
|
| 20 |
+
const $$ = (s)=>Array.from(document.querySelectorAll(s));
|
| 21 |
+
const showToast=(m)=>{const t=$("#toast");$("#toastMsg").textContent=m;t.classList.remove("hidden");setTimeout(()=>t.classList.add("hidden"),2200);};
|
| 22 |
+
const fmt=(n,d=2)=>(n==null||Number.isNaN(n))?"—":Number(n).toFixed(d);
|
| 23 |
+
const escapeHTML = value => String(value).replace(/[&<>"']/g, char => ({
|
| 24 |
+
'&': '&', '<': '<', '>': '>', '"': '"', "'": ''',
|
| 25 |
+
})[char]);
|
| 26 |
+
const slug=(s)=>s.toLowerCase().replace(/[^a-z0-9]+/g,'-');
|
| 27 |
+
const TEXT_ONLY_EXCLUDED_DATASETS = new Set([
|
| 28 |
+
"멀티 모달", "팩트 체크",
|
| 29 |
+
"K-Prism (Text, Intact)", "K-Prism (Text, Perturbed)",
|
| 30 |
+
"K-Prism (Image, Intact)", "K-Prism (Image, Perturbed)"
|
| 31 |
+
]);
|
| 32 |
+
|
| 33 |
+
function isUnsupportedForTextOnly(model, dataset){
|
| 34 |
+
return !model.is_multimodal && TEXT_ONLY_EXCLUDED_DATASETS.has(dataset);
|
| 35 |
+
}
|
| 36 |
+
|
| 37 |
+
let hasDatasetPreference = false;
|
| 38 |
+
let hasLoadedData = false;
|
| 39 |
+
let isLoading = false;
|
| 40 |
+
|
| 41 |
+
function savePrefs(){
|
| 42 |
+
try {
|
| 43 |
+
localStorage.setItem('llm_lb_prefs', JSON.stringify({
|
| 44 |
+
selectedDatasets: [...state.selectedDatasets],
|
| 45 |
+
providerFilter: state.providerFilter,
|
| 46 |
+
search: state.search, dark: state.dark,
|
| 47 |
+
compare: [...state.compare],
|
| 48 |
+
}));
|
| 49 |
+
} catch { /* Browsers may block storage inside embedded Spaces. */ }
|
| 50 |
+
}
|
| 51 |
+
function loadPrefs(){
|
| 52 |
+
try {
|
| 53 |
+
const p = JSON.parse(localStorage.getItem('llm_lb_prefs') || '{}');
|
| 54 |
+
if (Array.isArray(p.selectedDatasets)) {
|
| 55 |
+
state.selectedDatasets = new Set(p.selectedDatasets.filter(x => typeof x === 'string'));
|
| 56 |
+
hasDatasetPreference = true;
|
| 57 |
+
}
|
| 58 |
+
if (typeof p.providerFilter === 'string') state.providerFilter = p.providerFilter;
|
| 59 |
+
if (typeof p.search === 'string') state.search = p.search;
|
| 60 |
+
if (typeof p.dark === 'boolean') state.dark = p.dark;
|
| 61 |
+
if (Array.isArray(p.compare)) state.compare = new Set(p.compare.filter(x => typeof x === 'string'));
|
| 62 |
+
} catch { /* Preferences are optional. */ }
|
| 63 |
+
}
|
| 64 |
+
|
| 65 |
+
// ===== Rendering =====
|
| 66 |
+
function visibleDatasets(){return state.selectedDatasets.size ? state.datasets.filter(d=>state.selectedDatasets.has(d)) : []; }
|
| 67 |
+
|
| 68 |
+
function modelRowMean(model, dsList){
|
| 69 |
+
// 텍스트 전용 모델이면 지원하지 않는 데이터셋은 평균 계산에서 제외
|
| 70 |
+
const effDs = dsList.filter(d => !isUnsupportedForTextOnly(model, d));
|
| 71 |
+
const map=Object.fromEntries(model.scores.map(s=>[s.dataset_name,s]));
|
| 72 |
+
const vals=effDs.map(d=>map[d]?.score).filter(v=>typeof v==='number');
|
| 73 |
+
return (vals.length >= effDs.length && effDs.length>0) ? vals.reduce((a,b)=>a+b,0)/vals.length : null;
|
| 74 |
+
}
|
| 75 |
+
function renderDatasetFilters(){
|
| 76 |
+
const c=$("#datasetFilters"); c.innerHTML='';
|
| 77 |
+
state.datasets.forEach(d=>{
|
| 78 |
+
const checked=state.selectedDatasets.has(d);
|
| 79 |
+
c.insertAdjacentHTML('beforeend', `
|
| 80 |
+
<label class="flex flex-nowrap items-center gap-2 rounded-lg border" style="border-color: var(--border); padding:.5rem .6rem;">
|
| 81 |
+
<input type="checkbox" class="h-4 w-4 accent-indigo-600" data-dataset="${escapeHTML(d)}" ${checked?'checked':''}/>
|
| 82 |
+
<span class="text-sm" style="color: var(--text)">${escapeHTML(d)}</span>
|
| 83 |
+
</label>
|
| 84 |
+
`);
|
| 85 |
+
});
|
| 86 |
+
c.querySelectorAll('input[type="checkbox"]').forEach(cb=>{
|
| 87 |
+
cb.addEventListener('change', ()=>{
|
| 88 |
+
if(cb.checked) state.selectedDatasets.add(cb.dataset.dataset);
|
| 89 |
+
else state.selectedDatasets.delete(cb.dataset.dataset);
|
| 90 |
+
if([...c.querySelectorAll('input:checked')].length===0) state.selectedDatasets=new Set();
|
| 91 |
+
savePrefs(); renderTable(); updateChart();
|
| 92 |
+
});
|
| 93 |
+
});
|
| 94 |
+
$("#kpiDatasets").textContent = state.datasets.length;
|
| 95 |
+
}
|
| 96 |
+
|
| 97 |
+
function renderProviders(){
|
| 98 |
+
const provs=[...new Set(state.models.map(m=>m.provider))].sort();
|
| 99 |
+
const sel=$("#providerSelect");
|
| 100 |
+
sel.replaceChildren(new Option('All providers', ''));
|
| 101 |
+
provs.forEach(provider => sel.add(new Option(provider, provider)));
|
| 102 |
+
sel.value = state.providerFilter;
|
| 103 |
+
}
|
| 104 |
+
|
| 105 |
+
function renderHeader(){
|
| 106 |
+
const head=$("#tableHeaderRow");
|
| 107 |
+
head.querySelectorAll('th[data-ds]').forEach(el=>el.remove());
|
| 108 |
+
visibleDatasets().forEach(d=>{
|
| 109 |
+
const th=document.createElement('th');
|
| 110 |
+
th.dataset.ds=d; th.className='p-3 text-left ds-col ds-head';
|
| 111 |
+
th.innerHTML = `<div class="font-semibold" style="color: var(--text)">${escapeHTML(d)}</div>`;
|
| 112 |
+
head.appendChild(th);
|
| 113 |
+
});
|
| 114 |
+
}
|
| 115 |
+
|
| 116 |
+
function renderTable(){
|
| 117 |
+
renderHeader();
|
| 118 |
+
const ds=visibleDatasets();
|
| 119 |
+
const tb=$("#tableBody"); tb.innerHTML='';
|
| 120 |
+
|
| 121 |
+
let models=state.models.filter(m=>{
|
| 122 |
+
const text=(m.provider+" "+m.name+" "+m.repo).toLowerCase();
|
| 123 |
+
const okProvider=!state.providerFilter || m.provider===state.providerFilter;
|
| 124 |
+
const okSearch=!state.search || text.includes(state.search.toLowerCase());
|
| 125 |
+
return okProvider && okSearch;
|
| 126 |
+
}).map(m=>({...m, mean:modelRowMean(m, ds)}))
|
| 127 |
+
.sort((a,b)=>(b.mean??-1)-(a.mean??-1));
|
| 128 |
+
|
| 129 |
+
models.forEach((m, i)=>{
|
| 130 |
+
const sMap=Object.fromEntries(m.scores.map(s=>[s.dataset_name, s]));
|
| 131 |
+
const id=slug(m.provider+"-"+m.name);
|
| 132 |
+
const favOn=state.compare.has(id);
|
| 133 |
+
|
| 134 |
+
const row=document.createElement('tr');
|
| 135 |
+
row.className='border-b'; row.style.borderColor=getComputedStyle(document.documentElement).getPropertyValue('--border');
|
| 136 |
+
row.innerHTML = `
|
| 137 |
+
<td class="p-3 rank-col sticky-rank">${i+1}</td>
|
| 138 |
+
<td class="p-3 provider-col sticky-provider border-l">
|
| 139 |
+
<div class="font-medium">${escapeHTML(m.provider)}</div>
|
| 140 |
+
</td>
|
| 141 |
+
<td class="p-3 model-col sticky-model border-l">
|
| 142 |
+
<div class="flex items-center gap-2">
|
| 143 |
+
<button class="text-lg" data-fav="${id}" title="Add to compare">${favOn?'⭐':'☆'}</button>
|
| 144 |
+
<div class="font-semibold" style="color: var(--text)">${escapeHTML(m.name)}</div>
|
| 145 |
+
</div>
|
| 146 |
+
<div class="text-xs text-slate-500">${escapeHTML(m.repo)}</div>
|
| 147 |
+
<div class="text-xs text-slate-500">${m.is_multimodal ? 'Multimodal' : 'Text-only'}</div>
|
| 148 |
+
</td>
|
| 149 |
+
<td class="p-3 mean-col sticky-mean border-l">
|
| 150 |
+
<div class="text-base font-semibold text-blue-500">
|
| 151 |
+
${m.mean == null ? '<span class="text-slate-400" title="No complete score for the selected datasets">—</span>' : fmt(m.mean,2)}
|
| 152 |
+
</div>
|
| 153 |
+
<!--div class="text-[11px] text-slate-500">mean across selected</div-->
|
| 154 |
+
</td>
|
| 155 |
+
`;
|
| 156 |
+
ds.forEach(d=>{
|
| 157 |
+
const s=sMap[d]; const mt=s?.metric_type;
|
| 158 |
+
// const badge= mt==='llm-as-judge' ? 'metric-badge-judge' : 'metric-badge';
|
| 159 |
+
const cell=document.createElement('td');
|
| 160 |
+
cell.className='p-3 ds-col'; cell.title='';
|
| 161 |
+
// 텍스트 전용 모델이면 지원하지 않는 데이터셋 컬럼은 항상 비움(—)
|
| 162 |
+
if (isUnsupportedForTextOnly(m, d)) {
|
| 163 |
+
cell.innerHTML = '<span class="text-slate-300">—</span>';
|
| 164 |
+
row.appendChild(cell);
|
| 165 |
+
return;
|
| 166 |
+
}
|
| 167 |
+
|
| 168 |
+
{
|
| 169 |
+
cell.innerHTML = s ? (
|
| 170 |
+
mt==='llm-as-judge' ? `
|
| 171 |
+
<div class="flex item-center">
|
| 172 |
+
<div class="text-base font-semibold" style="color: var(--text)">${fmt(s.score,2)}</div>
|
| 173 |
+
<div class="pill metric-badge-judge ms-2">${mt}</div>
|
| 174 |
+
</div>
|
| 175 |
+
` :
|
| 176 |
+
`
|
| 177 |
+
<div class="flex item-center">
|
| 178 |
+
<div class="text-base font-semibold" style="color: var(--text)">${fmt(s.score,2)}</div>
|
| 179 |
+
</div>
|
| 180 |
+
`
|
| 181 |
+
) : '<span class="text-slate-300">—</span>';
|
| 182 |
+
}
|
| 183 |
+
row.appendChild(cell);
|
| 184 |
+
});
|
| 185 |
+
tb.appendChild(row);
|
| 186 |
+
});
|
| 187 |
+
|
| 188 |
+
if (!models.length) {
|
| 189 |
+
const row = document.createElement('tr');
|
| 190 |
+
const cell = document.createElement('td');
|
| 191 |
+
cell.colSpan = 4 + ds.length;
|
| 192 |
+
cell.className = 'p-6 text-center text-slate-500';
|
| 193 |
+
cell.textContent = !hasLoadedData ? 'Results are unavailable.'
|
| 194 |
+
: state.models.length ? 'No models match your filters.' : 'No results have been published yet.';
|
| 195 |
+
row.appendChild(cell);
|
| 196 |
+
tb.appendChild(row);
|
| 197 |
+
}
|
| 198 |
+
$("#kpiModels").textContent = hasLoadedData ? models.length : '—';
|
| 199 |
+
|
| 200 |
+
$$('button[data-fav]').forEach(b=>{
|
| 201 |
+
b.onclick=()=>{
|
| 202 |
+
const id=b.dataset.fav;
|
| 203 |
+
if(state.compare.has(id)) state.compare.delete(id);
|
| 204 |
+
else{
|
| 205 |
+
if(state.compare.size>=MAX_COMPARE){ showToast(`You can compare up to ${MAX_COMPARE} models.`); return; }
|
| 206 |
+
state.compare.add(id);
|
| 207 |
+
}
|
| 208 |
+
savePrefs(); renderTable(); updateChart();
|
| 209 |
+
};
|
| 210 |
+
});
|
| 211 |
+
}
|
| 212 |
+
|
| 213 |
+
// ===== Chart =====
|
| 214 |
+
let chart;
|
| 215 |
+
const baseColors = [
|
| 216 |
+
'#4f46e5', '#10b981', '#f59e0b', '#ef4444', '#8b5cf6', '#ec4899', '#06b6d4', '#84cc16', '#14b8a6', '#f43f5e'
|
| 217 |
+
];
|
| 218 |
+
function updateChart(){
|
| 219 |
+
const chartUnavailable = typeof window.Chart !== 'function';
|
| 220 |
+
$("#chartUnavailable").classList.toggle('hidden', !chartUnavailable);
|
| 221 |
+
$("#scoresChart").classList.toggle('hidden', chartUnavailable);
|
| 222 |
+
if (chartUnavailable) return;
|
| 223 |
+
const ds=visibleDatasets();
|
| 224 |
+
const labels=ds;
|
| 225 |
+
const datasets=[];
|
| 226 |
+
const chosen=state.models.filter(m=>state.compare.has(slug(m.provider+"-"+m.name)));
|
| 227 |
+
chosen.forEach((m, idx)=>{
|
| 228 |
+
const map=Object.fromEntries(m.scores.map(s=>[s.dataset_name, s.score]));
|
| 229 |
+
const color = baseColors[idx % baseColors.length];
|
| 230 |
+
|
| 231 |
+
const lineData = labels.map(d => {
|
| 232 |
+
if (isUnsupportedForTextOnly(m, d)) return null;
|
| 233 |
+
if (d.startsWith("K-Prism")) return null;
|
| 234 |
+
return map[d] ?? null;
|
| 235 |
+
});
|
| 236 |
+
|
| 237 |
+
const barData = labels.map(d => {
|
| 238 |
+
if (isUnsupportedForTextOnly(m, d)) return null;
|
| 239 |
+
if (!d.startsWith("K-Prism")) return null;
|
| 240 |
+
return map[d] ?? null;
|
| 241 |
+
});
|
| 242 |
+
|
| 243 |
+
// Add line dataset
|
| 244 |
+
datasets.push({
|
| 245 |
+
type: 'line',
|
| 246 |
+
label: `${m.provider} / ${m.name}`,
|
| 247 |
+
data: lineData,
|
| 248 |
+
borderColor: color,
|
| 249 |
+
backgroundColor: color,
|
| 250 |
+
tension: .25, spanGaps: true, borderWidth: 3, pointRadius: 4, pointHoverRadius: 6,
|
| 251 |
+
});
|
| 252 |
+
|
| 253 |
+
// Add bar dataset
|
| 254 |
+
datasets.push({
|
| 255 |
+
type: 'bar',
|
| 256 |
+
label: `${m.provider} / ${m.name} (Bar)`,
|
| 257 |
+
data: barData,
|
| 258 |
+
borderColor: color,
|
| 259 |
+
backgroundColor: color + '80', // Add transparency for bars
|
| 260 |
+
borderWidth: 2,
|
| 261 |
+
borderRadius: 4,
|
| 262 |
+
});
|
| 263 |
+
});
|
| 264 |
+
|
| 265 |
+
const ctx=document.getElementById('scoresChart').getContext('2d');
|
| 266 |
+
if(!chart){
|
| 267 |
+
chart=new Chart(ctx,{
|
| 268 |
+
data:{labels, datasets},
|
| 269 |
+
options:{
|
| 270 |
+
responsive:true,
|
| 271 |
+
maintainAspectRatio:false, // uses #chartWrap height (clamp -> responsive)
|
| 272 |
+
plugins:{
|
| 273 |
+
legend:{
|
| 274 |
+
display:true,
|
| 275 |
+
labels:{
|
| 276 |
+
boxWidth:18,
|
| 277 |
+
usePointStyle:true,
|
| 278 |
+
filter: function(item, chart) {
|
| 279 |
+
// Hide the bar datasets from the legend to avoid duplicates
|
| 280 |
+
return !item.text.includes('(Bar)');
|
| 281 |
+
}
|
| 282 |
+
}
|
| 283 |
+
},
|
| 284 |
+
tooltip:{ mode:'index', intersect:false }
|
| 285 |
+
},
|
| 286 |
+
interaction:{ mode:'index', intersect:false },
|
| 287 |
+
scales:{
|
| 288 |
+
y:{ beginAtZero:true, max:100, title:{display:true, text:'Score'}, grid:{ drawBorder:false }},
|
| 289 |
+
x:{
|
| 290 |
+
type: 'category',
|
| 291 |
+
offset:true,
|
| 292 |
+
grid:{ display:false },
|
| 293 |
+
ticks:{ autoSkip:false, maxRotation:45, minRotation:45 }
|
| 294 |
+
}
|
| 295 |
+
},
|
| 296 |
+
layout:{ padding:0 }
|
| 297 |
+
}
|
| 298 |
+
});
|
| 299 |
+
}else{
|
| 300 |
+
chart.data.labels=labels;
|
| 301 |
+
chart.data.datasets=datasets;
|
| 302 |
+
chart.update();
|
| 303 |
+
}
|
| 304 |
+
}
|
| 305 |
+
|
| 306 |
+
// ===== Tabs / Theme / Export =====
|
| 307 |
+
function bindTabs(){
|
| 308 |
+
$$(".tab").forEach(btn=>{
|
| 309 |
+
btn.addEventListener('click', ()=>{
|
| 310 |
+
const t=btn.dataset.tab;
|
| 311 |
+
$$(".tab").forEach(b=>b.classList.remove('tab-active')); btn.classList.add('tab-active');
|
| 312 |
+
["leaderboard","about"].forEach(x=>{
|
| 313 |
+
const el=document.getElementById(`tab-${x}`); (x===t)?el.classList.remove('hidden'):el.classList.add('hidden');
|
| 314 |
+
});
|
| 315 |
+
});
|
| 316 |
+
});
|
| 317 |
+
}
|
| 318 |
+
function applyTheme(){
|
| 319 |
+
document.documentElement.classList.toggle('dark', state.dark);
|
| 320 |
+
$$('[data-action="theme"]').forEach(button => {
|
| 321 |
+
button.textContent = state.dark ? '☀️' : '🌙';
|
| 322 |
+
});
|
| 323 |
+
}
|
| 324 |
+
|
| 325 |
+
function exportCSV(){
|
| 326 |
+
const ds=visibleDatasets();
|
| 327 |
+
const headers=['Rank','Provider','Model','Repo','Mean',...ds];
|
| 328 |
+
const rows=[];
|
| 329 |
+
let models=state.models.map(m=>({...m, mean:modelRowMean(m, ds)})).sort((a,b)=>(b.mean??-1)-(a.mean??-1));
|
| 330 |
+
models=models.filter(m=>{
|
| 331 |
+
const text=(m.provider+" "+m.name+" "+m.repo).toLowerCase();
|
| 332 |
+
const okProvider=!state.providerFilter || m.provider===state.providerFilter;
|
| 333 |
+
const okSearch=!state.search || text.includes(state.search.toLowerCase());
|
| 334 |
+
return okProvider && okSearch;
|
| 335 |
+
});
|
| 336 |
+
models.forEach((m,i)=>{
|
| 337 |
+
const map=Object.fromEntries(m.scores.map(s=>[s.dataset_name, s]));
|
| 338 |
+
const row=[i+1, m.provider, m.name, m.repo, fmt(m.mean,2)];
|
| 339 |
+
ds.forEach(d=>{
|
| 340 |
+
if (isUnsupportedForTextOnly(m, d)) {
|
| 341 |
+
row.push('');
|
| 342 |
+
return;
|
| 343 |
+
}
|
| 344 |
+
const s=map[d];
|
| 345 |
+
row.push(s ? `${fmt(s.score,2)} (${s.metric_type})` : '');
|
| 346 |
+
});
|
| 347 |
+
rows.push(row);
|
| 348 |
+
});
|
| 349 |
+
const csv=[headers, ...rows].map(r=> r.map(x=>'"'+String(x).replaceAll('"','""')+'"').join(',')).join('\n');
|
| 350 |
+
const blob=new Blob([csv],{type:'text/csv;charset=utf-8;'}); const url=URL.createObjectURL(blob);
|
| 351 |
+
const a=document.createElement('a'); a.href=url; a.download='keti-leaderboard.csv'; a.click(); URL.revokeObjectURL(url);
|
| 352 |
+
}
|
| 353 |
+
|
| 354 |
+
// ===== Controls =====
|
| 355 |
+
function bindControls(){
|
| 356 |
+
$("#selectAllBtn").onclick=()=>{ state.selectedDatasets=new Set(state.datasets); renderDatasetFilters(); renderTable(); updateChart(); savePrefs(); };
|
| 357 |
+
$("#clearAllBtn").onclick=()=>{ state.selectedDatasets=new Set(); renderDatasetFilters(); renderTable(); updateChart(); savePrefs(); };
|
| 358 |
+
$("#searchInput").addEventListener('input', (e)=>{ state.search=e.target.value; savePrefs(); renderTable(); });
|
| 359 |
+
$("#providerSelect").addEventListener('change', (e)=>{ state.providerFilter=e.target.value; savePrefs(); renderTable(); });
|
| 360 |
+
$$('[data-action="refresh"]').forEach(button => button.onclick = refreshResults);
|
| 361 |
+
$("#exportCsvBtn").onclick=exportCSV;
|
| 362 |
+
$$('[data-action="theme"]').forEach(button => {
|
| 363 |
+
button.onclick=()=>{ state.dark=!state.dark; applyTheme(); savePrefs(); };
|
| 364 |
+
});
|
| 365 |
+
}
|
| 366 |
+
|
| 367 |
+
async function refreshResults(){
|
| 368 |
+
if (isLoading) return;
|
| 369 |
+
isLoading = true;
|
| 370 |
+
const status = $("#dataStatus");
|
| 371 |
+
status.dataset.state = 'loading';
|
| 372 |
+
status.textContent = 'Loading published results…';
|
| 373 |
+
$$('[data-action="refresh"]').forEach(button => button.disabled = true);
|
| 374 |
+
try {
|
| 375 |
+
const result = await loadBenchmark(config.dataUrl, config.requestTimeoutMs);
|
| 376 |
+
state.datasets = result.datasets;
|
| 377 |
+
state.models = result.models;
|
| 378 |
+
state.selectedDatasets = hasDatasetPreference
|
| 379 |
+
? new Set(state.datasets.filter(dataset => state.selectedDatasets.has(dataset)))
|
| 380 |
+
: new Set(state.datasets);
|
| 381 |
+
hasDatasetPreference = true;
|
| 382 |
+
hasLoadedData = true;
|
| 383 |
+
const modelIds = new Set(state.models.map(model => slug(model.provider + '-' + model.name)));
|
| 384 |
+
state.compare = new Set([...state.compare].filter(id => modelIds.has(id)));
|
| 385 |
+
if (!state.models.some(model => model.provider === state.providerFilter)) state.providerFilter = '';
|
| 386 |
+
const timestamps = state.models.map(model => Date.parse(model.updated_at)).filter(Number.isFinite);
|
| 387 |
+
$("#lastUpdated").textContent = timestamps.length
|
| 388 |
+
? 'Last updated ' + new Date(Math.max(...timestamps)).toLocaleString() : 'Last updated —';
|
| 389 |
+
renderDatasetFilters(); renderProviders(); renderTable(); updateChart();
|
| 390 |
+
$("#exportCsvBtn").disabled = !state.models.length;
|
| 391 |
+
status.dataset.state = 'ready';
|
| 392 |
+
status.textContent = `${state.models.length} models · ${state.datasets.length} datasets · Published results`;
|
| 393 |
+
savePrefs();
|
| 394 |
+
} catch (error) {
|
| 395 |
+
console.error('Unable to load published results:', error);
|
| 396 |
+
status.dataset.state = 'error';
|
| 397 |
+
status.textContent = hasLoadedData
|
| 398 |
+
? 'Could not refresh results. Previously loaded results are still shown. Try Refresh again.'
|
| 399 |
+
: 'Could not load results. Please check your connection and try Refresh again.';
|
| 400 |
+
if (!hasLoadedData) {
|
| 401 |
+
renderTable();
|
| 402 |
+
$("#kpiDatasets").textContent = '—';
|
| 403 |
+
}
|
| 404 |
+
} finally {
|
| 405 |
+
isLoading = false;
|
| 406 |
+
$$('[data-action="refresh"]').forEach(button => button.disabled = false);
|
| 407 |
+
}
|
| 408 |
+
}
|
| 409 |
+
|
| 410 |
+
// Bind once: refreshing must not register duplicate listeners.
|
| 411 |
+
loadPrefs(); applyTheme(); bindTabs(); bindControls();
|
| 412 |
+
$("#searchInput").value = state.search;
|
| 413 |
+
refreshResults();
|
index.html
ADDED
|
@@ -0,0 +1,246 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<!DOCTYPE html>
|
| 2 |
+
<html lang="en">
|
| 3 |
+
<head>
|
| 4 |
+
<meta charset="UTF-8" />
|
| 5 |
+
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
|
| 6 |
+
<meta name="author" content="Dr. Mohamed Sana">
|
| 7 |
+
<title>KETI LLM Benchmarks</title>
|
| 8 |
+
|
| 9 |
+
<!-- Tailwind + Chart.js -->
|
| 10 |
+
<script src="https://cdn.tailwindcss.com"></script>
|
| 11 |
+
<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
|
| 12 |
+
|
| 13 |
+
<!-- Fonts -->
|
| 14 |
+
<link rel="preconnect" href="https://fonts.googleapis.com">
|
| 15 |
+
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
| 16 |
+
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@300;400;500;600;700;800&display=swap" rel="stylesheet">
|
| 17 |
+
|
| 18 |
+
<style>
|
| 19 |
+
:root{
|
| 20 |
+
color-scheme: light dark;
|
| 21 |
+
--text: #0f172a;
|
| 22 |
+
--bg: #f8fafc;
|
| 23 |
+
--card-bg: rgba(255,255,255,.75);
|
| 24 |
+
--border: #e2e8f0;
|
| 25 |
+
--header-bg: rgba(241,245,249,.85);
|
| 26 |
+
--sticky-bg: rgba(248,250,252,.92);
|
| 27 |
+
--chip-bg:#f1f5f9;
|
| 28 |
+
}
|
| 29 |
+
.dark{
|
| 30 |
+
--text: #e2e8f0;
|
| 31 |
+
--bg: #020617;
|
| 32 |
+
--card-bg: rgba(2,6,23,.6);
|
| 33 |
+
--border:#334155;
|
| 34 |
+
--header-bg: rgba(30,41,59,.75);
|
| 35 |
+
--sticky-bg: rgba(2,6,23,.92);
|
| 36 |
+
--chip-bg: rgba(15,23,42,.6);
|
| 37 |
+
}
|
| 38 |
+
html { font-family: Inter, system-ui, -apple-system, Segoe UI, Roboto, "Helvetica Neue", Arial, "Noto Sans"; }
|
| 39 |
+
body { color: var(--text); background: var(--bg); }
|
| 40 |
+
|
| 41 |
+
.card{ border-radius:1rem; box-shadow:0 10px 25px rgba(2,6,23,.08); background:var(--card-bg); backdrop-filter: blur(8px); border:1px solid var(--border); }
|
| 42 |
+
.btn{ display:inline-flex; align-items:center; justify-content:center; gap:.5rem; border-radius:.8rem; padding:.625rem 1rem; font-weight:700; transition: transform .05s ease; }
|
| 43 |
+
.btn:disabled{ opacity:.5; cursor:wait; }
|
| 44 |
+
#dataStatus[data-state="error"]{ border-color:#f59e0b; }
|
| 45 |
+
.btn:active{ transform: scale(.98); }
|
| 46 |
+
.btn-primary{ background:#4f46e5; color:#fff; } .btn-primary:hover{ background:#6366f1; }
|
| 47 |
+
.btn-ghost{ background:transparent; border:1px solid var(--border); } .btn-ghost:hover{ background:var(--chip-bg); }
|
| 48 |
+
.btn-outline{ border:1px solid #4f46e5; color:#4338ca; } .dark .btn-outline{ color:#a5b4fc; }
|
| 49 |
+
.btn-outline:hover{ background:#eef2ff; } .dark .btn-outline:hover{ background: rgba(30,27,75,.5); }
|
| 50 |
+
.input{ width:100%; border:1px solid var(--border); border-radius:.8rem; padding:.6rem .8rem; background:#fff; color:#0f172a; }
|
| 51 |
+
.dark .input{ background:#0b1220; color:#e2e8f0; }
|
| 52 |
+
.input:focus{ outline:none; box-shadow:0 0 0 2px rgba(99,102,241,.6); }
|
| 53 |
+
.label{ font-size:.875rem; font-weight:600; color:#334155; } .dark .label{ color:#cbd5e1; }
|
| 54 |
+
.tab{ padding:.6rem 1rem; border-radius:.8rem; cursor:pointer; font-weight:700; }
|
| 55 |
+
.tab-active{ background:#fff; border:1px solid var(--border); box-shadow:0 1px 2px rgba(0,0,0,.04); } .dark .tab-active{ background:#0b1220; }
|
| 56 |
+
.pill{ display:inline-flex; align-items:center; padding:.1rem .5rem; border-radius:999px; font-size:.53rem; font-weight:700; }
|
| 57 |
+
.metric-badge{ background:#ecfdf5; color:#065f46; } .dark .metric-badge{ background:rgba(16,185,129,.18); color:#d1fae5; }
|
| 58 |
+
.metric-badge-judge{ background:#e0f2fe; color:#075985; } .dark .metric-badge-judge{ background:rgba(56,189,248,.18); color:#bae6fd; }
|
| 59 |
+
.kpi{ color:#475569; font-size:.875rem; } .dark .kpi{ color:#94a3b8; }
|
| 60 |
+
|
| 61 |
+
/* Column widths */
|
| 62 |
+
/* .rank-col{ width:1.5rem; }
|
| 63 |
+
.provider-col{ width:4rem; }
|
| 64 |
+
.model-col{ width:10rem; white-space:nowrap; }
|
| 65 |
+
.mean-col{ width:3rem; white-space:nowrap; }
|
| 66 |
+
.ds-col{ min-width:9.5rem; white-space:nowrap; } */
|
| 67 |
+
|
| 68 |
+
.rank-col{ }
|
| 69 |
+
.provider-col{ }
|
| 70 |
+
.model-col{ white-space:nowrap; }
|
| 71 |
+
.mean-col{ white-space:nowrap; }
|
| 72 |
+
.ds-col{ white-space:nowrap; }
|
| 73 |
+
|
| 74 |
+
/* Sticky columns: use calc so offsets update easily */
|
| 75 |
+
.sticky-rank{ position: sticky; left: 0; z-index: 3; background: var(--sticky-bg); }
|
| 76 |
+
.sticky-provider{ position: sticky; left: 2.3rem; z-index: 3; background: var(--sticky-bg); }
|
| 77 |
+
.sticky-model{ position: sticky; left: calc(1.85rem + 6rem); z-index: 3; background: var(--sticky-bg); }
|
| 78 |
+
.sticky-mean{ position: sticky; left: calc(2.3rem + 4rem + 10rem + 0.5rem); z-index: 3; background: var(--sticky-bg); }
|
| 79 |
+
|
| 80 |
+
/* Keep provider, model and mean visible together on narrow screens. */
|
| 81 |
+
@media (max-width: 768px){
|
| 82 |
+
.rank-col, .ds-col, .ds-head { display:none !important; }
|
| 83 |
+
table{ min-width:0 !important; table-layout:fixed; }
|
| 84 |
+
.provider-col{ width:28%; }
|
| 85 |
+
.model-col{ width:50%; white-space:normal; overflow-wrap:anywhere; }
|
| 86 |
+
.mean-col{ width:22%; }
|
| 87 |
+
.sticky-provider, .sticky-model, .sticky-mean{ position:static; }
|
| 88 |
+
td.provider-col{ overflow-wrap:anywhere; }
|
| 89 |
+
th, td{ padding:.5rem !important; }
|
| 90 |
+
}
|
| 91 |
+
|
| 92 |
+
/* Header background */
|
| 93 |
+
thead th{ background: var(--header-bg); }
|
| 94 |
+
|
| 95 |
+
/* Global gradient */
|
| 96 |
+
.gradient-bg{
|
| 97 |
+
background:
|
| 98 |
+
radial-gradient(1200px 600px at 20% -10%, rgba(99,102,241,.25), rgba(99,102,241,0) 60%),
|
| 99 |
+
radial-gradient(1200px 600px at 80% -10%, rgba(20,184,166,.2), rgba(20,184,166,0) 60%);
|
| 100 |
+
}
|
| 101 |
+
|
| 102 |
+
/* Scrollbar for horizontal overflow */
|
| 103 |
+
.scrollbar-thin::-webkit-scrollbar{ height:10px; }
|
| 104 |
+
.scrollbar-thin::-webkit-scrollbar-thumb{ background:#c7d2fe; border-radius:999px; }
|
| 105 |
+
.scrollbar-thin::-webkit-scrollbar-track{ background:transparent; }
|
| 106 |
+
|
| 107 |
+
/* Chart container: responsive height */
|
| 108 |
+
#chartWrap{ height: clamp(260px, 42vh, 460px); }
|
| 109 |
+
</style>
|
| 110 |
+
</head>
|
| 111 |
+
<body class="min-h-screen dark:bg-gray-800">
|
| 112 |
+
<!-- Header -->
|
| 113 |
+
<header class="sticky top-0 z-40 backdrop-blur dark:bg-white-800 border-b" style="border-color: var(--border)">
|
| 114 |
+
<div class="mx-auto max-w-7xl px-4 md:px-6 py-3 md:py-4 flex items-center justify-between">
|
| 115 |
+
<div class="flex items-center gap-3 md:gap-4">
|
| 116 |
+
<div class="h-10 w-10 rounded-xl bg-indigo-600 text-white grid place-items-center shadow-lg">📡</div>
|
| 117 |
+
<div>
|
| 118 |
+
<h1 class="text-lg md:text-xl font-extrabold tracking-tight" style="color: var(--text)">KETI LLM Benchmarks</h1>
|
| 119 |
+
<p class="text-xs md:text-sm">Benchmarking models across KETI datasets</p>
|
| 120 |
+
</div>
|
| 121 |
+
</div>
|
| 122 |
+
<div class="md:flex items-center gap-2 hidden">
|
| 123 |
+
<button id="refreshBtn" data-action="refresh" class="btn btn-ghost" title="Refresh results">⟲ Refresh</button>
|
| 124 |
+
<button id="themeToggle" data-action="theme" class="btn btn-ghost" title="Toggle dark mode">🌙</button>
|
| 125 |
+
</div>
|
| 126 |
+
</div>
|
| 127 |
+
</header>
|
| 128 |
+
|
| 129 |
+
<!-- Main -->
|
| 130 |
+
<main class="mx-auto max-w-7xl px-4 md:px-6 py-6 md:py-8 space-y-6 md:space-y-8">
|
| 131 |
+
<!-- Tabs -->
|
| 132 |
+
<div class="flex gap-2 md:gap-3">
|
| 133 |
+
<button class="tab tab-active" data-tab="leaderboard">🏆 Leaderboard</button>
|
| 134 |
+
<button class="tab" data-tab="about">ℹ️ About</button>
|
| 135 |
+
</div>
|
| 136 |
+
|
| 137 |
+
<div id="dataStatus" class="card p-4 text-sm" role="status" aria-live="polite" data-state="loading">Loading published results…</div>
|
| 138 |
+
<noscript><p class="card p-4">Enable JavaScript to view and filter the leaderboard.</p></noscript>
|
| 139 |
+
|
| 140 |
+
<!-- Leaderboard Tab -->
|
| 141 |
+
<section id="tab-leaderboard" class="space-y-6 md:space-y-8">
|
| 142 |
+
<!-- Controls -->
|
| 143 |
+
<div class="grid lg:grid-cols-4 gap-4 md:gap-6">
|
| 144 |
+
<div class="card p-4 md:p-6 lg:col-span-2">
|
| 145 |
+
<div class="flex items-center justify-between">
|
| 146 |
+
<h2 class="font-semibold text-base md:text-lg" style="color: var(--text)">Select Datasets</h2>
|
| 147 |
+
<div class="flex gap-2">
|
| 148 |
+
<button id="selectAllBtn" class="btn btn-ghost text-xs md:text-sm px-3">Select all</button>
|
| 149 |
+
<button id="clearAllBtn" class="btn btn-ghost text-xs md:text-sm px-3">Clear</button>
|
| 150 |
+
</div>
|
| 151 |
+
</div>
|
| 152 |
+
<div id="datasetFilters" class="mt-3 md:mt-4 flex flex-wrap gap-2.5"></div> <!--grid-cols-1 sm:grid-cols-2 lg:grid-cols-3-->
|
| 153 |
+
</div>
|
| 154 |
+
|
| 155 |
+
<div class="card p-4 md:p-6">
|
| 156 |
+
<h2 class="font-semibold text-base md:text-lg" style="color: var(--text)">Search & Filter</h2>
|
| 157 |
+
<div class="mt-2 md:mt-3 space-y-2.5">
|
| 158 |
+
<input id="searchInput" class="input" placeholder="Search provider/model…"/>
|
| 159 |
+
<select id="providerSelect" class="input">
|
| 160 |
+
<option value="">All providers</option>
|
| 161 |
+
</select>
|
| 162 |
+
</div>
|
| 163 |
+
</div>
|
| 164 |
+
|
| 165 |
+
<div class="card p-4 md:p-6">
|
| 166 |
+
<h2 class="font-semibold text-base md:text-lg" style="color: var(--text)">KPI</h2>
|
| 167 |
+
<div class="mt-2 grid grid-cols-2 gap-3">
|
| 168 |
+
<div>
|
| 169 |
+
<div class="text-2xl md:text-3xl font-extrabold" id="kpiModels">—</div>
|
| 170 |
+
<div class="kpi">Models</div>
|
| 171 |
+
</div>
|
| 172 |
+
<div>
|
| 173 |
+
<div class="text-2xl md:text-3xl font-extrabold" id="kpiDatasets">—</div>
|
| 174 |
+
<div class="kpi">Datasets</div>
|
| 175 |
+
</div>
|
| 176 |
+
<div class="col-span-2 text-xs text-slate-500 dark:text-slate-400" id="lastUpdated">Last updated —</div>
|
| 177 |
+
</div>
|
| 178 |
+
</div>
|
| 179 |
+
</div>
|
| 180 |
+
|
| 181 |
+
<!-- Visualization -->
|
| 182 |
+
<div class="card p-5 md:p-6 lg:p-8">
|
| 183 |
+
<div class="flex items-center justify-between gap-3">
|
| 184 |
+
<h2 class="font-semibold text-base md:text-lg" style="color: var(--text)">Model Comparison</h2>
|
| 185 |
+
<div class="text-xs md:text-sm text-slate-500">Click ☆ on rows to compare models</div>
|
| 186 |
+
</div>
|
| 187 |
+
<div id="chartWrap" class="mt-3 md:mt-4">
|
| 188 |
+
<p id="chartUnavailable" class="hidden text-sm text-slate-500">The comparison chart could not be loaded. Scores are available in the table below.</p>
|
| 189 |
+
<canvas id="scoresChart"></canvas>
|
| 190 |
+
</div>
|
| 191 |
+
</div>
|
| 192 |
+
|
| 193 |
+
<!-- Table (always horizontally scrollable) -->
|
| 194 |
+
<div class="card overflow-hidden">
|
| 195 |
+
<div class="px-4 md:px-6 py-3 md:py-4 flex items-center justify-between">
|
| 196 |
+
<div class="font-semibold" style="color: var(--text)">Leaderboard</div>
|
| 197 |
+
<div class="text-xs md:text-sm text-slate-500">Sorted by mean score across selected datasets</div>
|
| 198 |
+
</div>
|
| 199 |
+
|
| 200 |
+
<div class="overflow-x-auto overscroll-x-contain scrollbar-thin pb-2">
|
| 201 |
+
<table class="min-w-[64rem] w-full text-sm">
|
| 202 |
+
<thead class="border-t border-b" style="border-color: var(--border)">
|
| 203 |
+
<tr id="tableHeaderRow">
|
| 204 |
+
<th class="p-3 text-left rank-col sticky-rank">#</th>
|
| 205 |
+
<th class="p-3 text-left provider-col sticky-provider border-l">Provider</th>
|
| 206 |
+
<th class="p-3 text-left model-col sticky-model border-l">Model</th>
|
| 207 |
+
<th class="p-3 text-left mean-col sticky-mean border-l">
|
| 208 |
+
Mean
|
| 209 |
+
<div class="text-[11px] text-slate-500">on selected</div>
|
| 210 |
+
</th>
|
| 211 |
+
<!-- Dataset columns injected here -->
|
| 212 |
+
</tr>
|
| 213 |
+
</thead>
|
| 214 |
+
<tbody id="tableBody"></tbody>
|
| 215 |
+
</table>
|
| 216 |
+
</div>
|
| 217 |
+
|
| 218 |
+
<div class="flex items-center gap-2 justify-end p-3">
|
| 219 |
+
<button id="refreshMobileBtn" data-action="refresh" class="btn btn-ghost md:hidden" title="Refresh results">⟲ Refresh</button>
|
| 220 |
+
<button id="exportCsvBtn" disabled class="btn btn-outline" title="Export current view to CSV">⭳ Export CSV</button>
|
| 221 |
+
<button id="themeToggleMobile" data-action="theme" class="btn btn-ghost md:hidden" title="Toggle dark mode">🌙</button>
|
| 222 |
+
</div>
|
| 223 |
+
</div>
|
| 224 |
+
</section>
|
| 225 |
+
|
| 226 |
+
<!-- About Tab -->
|
| 227 |
+
<section id="tab-about" class="hidden">
|
| 228 |
+
<div class="card p-5 md:p-6 lg:p-8 space-y-3">
|
| 229 |
+
<h2 class="text-lg md:text-xl font-bold" style="color: var(--text)">About this Leaderboard</h2>
|
| 230 |
+
<p class="text-sm text-slate-600 dark:text-slate-300">
|
| 231 |
+
This leaderboard presents published model evaluations on KETI's ethicality and veracity benchmark datasets.
|
| 232 |
+
</p>
|
| 233 |
+
<p class="text-sm text-slate-600 dark:text-slate-300">
|
| 234 |
+
Model submissions and automatic evaluations are not available on this page. Results are updated when the maintainers publish a new evaluation.
|
| 235 |
+
</p>
|
| 236 |
+
</div>
|
| 237 |
+
</section>
|
| 238 |
+
</main>
|
| 239 |
+
|
| 240 |
+
<div id="toast" class="fixed bottom-4 left-1/2 -translate-x-1/2 hidden">
|
| 241 |
+
<div class="rounded-xl bg-slate-900 text-white px-4 py-2 shadow-lg">✅ <span id="toastMsg">Done</span></div>
|
| 242 |
+
</div>
|
| 243 |
+
|
| 244 |
+
<script type="module" src="./front/leaderboard.mjs"></script>
|
| 245 |
+
</body>
|
| 246 |
+
</html>
|
tests/test_static_space.py
ADDED
|
@@ -0,0 +1,204 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Browser regression checks for the static Space.
|
| 2 |
+
|
| 3 |
+
Install playwright and Chrome/Chromium, then run:
|
| 4 |
+
python3 -m unittest discover -s tests -v
|
| 5 |
+
Set CHROME_PATH to override the installed browser executable.
|
| 6 |
+
"""
|
| 7 |
+
import csv
|
| 8 |
+
import io
|
| 9 |
+
import json
|
| 10 |
+
import os
|
| 11 |
+
from functools import partial
|
| 12 |
+
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
|
| 13 |
+
from pathlib import Path
|
| 14 |
+
import shutil
|
| 15 |
+
import threading
|
| 16 |
+
import unittest
|
| 17 |
+
|
| 18 |
+
from playwright.sync_api import sync_playwright
|
| 19 |
+
|
| 20 |
+
ROOT = Path(__file__).resolve().parents[1]
|
| 21 |
+
SNAPSHOT = json.loads((ROOT / 'front/data/benchmark.json').read_text())
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
class QuietHandler(SimpleHTTPRequestHandler):
|
| 25 |
+
def log_message(self, *args):
|
| 26 |
+
pass
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
class StaticSpaceTests(unittest.TestCase):
|
| 30 |
+
@classmethod
|
| 31 |
+
def setUpClass(cls):
|
| 32 |
+
cls.server = ThreadingHTTPServer(
|
| 33 |
+
('127.0.0.1', 0), partial(QuietHandler, directory=str(ROOT)))
|
| 34 |
+
cls.thread = threading.Thread(target=cls.server.serve_forever, daemon=True)
|
| 35 |
+
cls.thread.start()
|
| 36 |
+
cls.base = 'http://127.0.0.1:' + str(cls.server.server_port)
|
| 37 |
+
cls.playwright = sync_playwright().start()
|
| 38 |
+
cls.browser = cls.playwright.chromium.launch(
|
| 39 |
+
executable_path=os.environ.get('CHROME_PATH') or shutil.which('google-chrome')
|
| 40 |
+
or shutil.which('chromium'), headless=True, timeout=20000)
|
| 41 |
+
|
| 42 |
+
@classmethod
|
| 43 |
+
def tearDownClass(cls):
|
| 44 |
+
cls.browser.close()
|
| 45 |
+
cls.playwright.stop()
|
| 46 |
+
cls.server.shutdown()
|
| 47 |
+
cls.server.server_close()
|
| 48 |
+
cls.thread.join()
|
| 49 |
+
|
| 50 |
+
def setUp(self):
|
| 51 |
+
self.context = self.browser.new_context(viewport={'width': 1440, 'height': 1000})
|
| 52 |
+
self.page = self.context.new_page()
|
| 53 |
+
self.errors = []
|
| 54 |
+
self.requests = []
|
| 55 |
+
self.page.on('pageerror', lambda error: self.errors.append(str(error)))
|
| 56 |
+
self.page.on('request', lambda request: self.requests.append((request.method, request.url)))
|
| 57 |
+
|
| 58 |
+
def tearDown(self):
|
| 59 |
+
self.context.close()
|
| 60 |
+
self.assertEqual(self.errors, [])
|
| 61 |
+
|
| 62 |
+
def open(self, state='ready'):
|
| 63 |
+
self.page.goto(self.base, wait_until='domcontentloaded')
|
| 64 |
+
self.page.wait_for_selector('#dataStatus[data-state="' + state + '"]')
|
| 65 |
+
|
| 66 |
+
def refresh(self, state='ready', mobile=False):
|
| 67 |
+
with self.page.expect_response('**/front/data/benchmark.json'):
|
| 68 |
+
self.page.locator('#refreshMobileBtn' if mobile else '#refreshBtn').click()
|
| 69 |
+
self.page.wait_for_selector('#dataStatus[data-state="' + state + '"]')
|
| 70 |
+
|
| 71 |
+
def test_published_scores_filters_chart_and_export(self):
|
| 72 |
+
self.open()
|
| 73 |
+
self.assertEqual(self.page.locator('#kpiModels').inner_text(), str(len(SNAPSHOT['models'])))
|
| 74 |
+
self.assertEqual(self.page.locator('#datasetFilters input').count(), len(SNAPSHOT['datasets']))
|
| 75 |
+
self.assertEqual(self.page.locator('[data-tab="submit"], #submitForm').count(), 0)
|
| 76 |
+
self.assertFalse(any('/api/' in url or method != 'GET' for method, url in self.requests))
|
| 77 |
+
# Check rendered means against the preserved scoring rules, independently in Python.
|
| 78 |
+
excluded = {'멀티 모달', '팩트 체크', 'K-Prism (Text, Intact)',
|
| 79 |
+
'K-Prism (Text, Perturbed)', 'K-Prism (Image, Intact)', 'K-Prism (Image, Perturbed)'}
|
| 80 |
+
rows = self.page.locator('#tableBody tr').all()
|
| 81 |
+
for model in SNAPSHOT['models']:
|
| 82 |
+
cells = next(row.locator('td').all() for row in rows
|
| 83 |
+
if row.locator('td').nth(2).locator('.font-semibold').inner_text() == model['name'])
|
| 84 |
+
scores = {score['dataset_name']: score['score'] for score in model['scores']}
|
| 85 |
+
datasets = [d for d in SNAPSHOT['datasets'] if model['is_multimodal'] or d not in excluded]
|
| 86 |
+
expected = sum(scores[d] for d in datasets) / len(datasets)
|
| 87 |
+
self.assertAlmostEqual(float(cells[3].inner_text()), expected, delta=0.006)
|
| 88 |
+
self.page.locator('button[data-fav]').first.click()
|
| 89 |
+
self.page.wait_for_function("window.Chart && Chart.getChart('scoresChart').data.datasets.length === 2")
|
| 90 |
+
self.page.locator('#searchInput').fill('qwen3-32b')
|
| 91 |
+
self.assertEqual(self.page.locator('#kpiModels').inner_text(), '1')
|
| 92 |
+
with self.page.expect_download() as download:
|
| 93 |
+
self.page.locator('#exportCsvBtn').click()
|
| 94 |
+
rows = list(csv.reader(io.StringIO(Path(download.value.path()).read_text())))
|
| 95 |
+
self.assertEqual(len(rows), 2)
|
| 96 |
+
self.assertEqual(rows[1][2], 'qwen3-32b')
|
| 97 |
+
self.page.locator('#searchInput').fill('')
|
| 98 |
+
self.page.locator('#providerSelect').select_option('OpenAI')
|
| 99 |
+
self.assertEqual(self.page.locator('#kpiModels').inner_text(), '1')
|
| 100 |
+
self.page.locator('#searchInput').fill('no-model-matches-this')
|
| 101 |
+
self.assertIn('No models match', self.page.locator('#tableBody').inner_text())
|
| 102 |
+
|
| 103 |
+
def test_preferences_and_repeated_refresh(self):
|
| 104 |
+
self.open()
|
| 105 |
+
self.page.locator('#clearAllBtn').click()
|
| 106 |
+
self.refresh()
|
| 107 |
+
self.refresh()
|
| 108 |
+
self.assertEqual(self.page.locator('#datasetFilters input:checked').count(), 0)
|
| 109 |
+
self.page.reload(wait_until='domcontentloaded')
|
| 110 |
+
self.page.wait_for_selector('#dataStatus[data-state="ready"]')
|
| 111 |
+
self.assertEqual(self.page.locator('#datasetFilters input:checked').count(), 0)
|
| 112 |
+
self.page.locator('#selectAllBtn').click()
|
| 113 |
+
self.page.locator('#searchInput').fill('qwen3-32b')
|
| 114 |
+
self.page.reload(wait_until='domcontentloaded')
|
| 115 |
+
self.page.wait_for_selector('#dataStatus[data-state="ready"]')
|
| 116 |
+
self.assertEqual(self.page.locator('#searchInput').input_value(), 'qwen3-32b')
|
| 117 |
+
self.assertEqual(self.page.locator('#kpiModels').inner_text(), '1')
|
| 118 |
+
|
| 119 |
+
def test_initial_failure_never_invents_results_and_retry_recovers(self):
|
| 120 |
+
self.page.route('**/front/data/benchmark.json', lambda route: route.fulfill(status=503, body='Unavailable'))
|
| 121 |
+
self.open('error')
|
| 122 |
+
self.assertEqual(self.page.locator('#kpiModels').inner_text(), '—')
|
| 123 |
+
self.assertEqual(self.page.locator('button[data-fav]').count(), 0)
|
| 124 |
+
self.assertTrue(self.page.locator('#exportCsvBtn').is_disabled())
|
| 125 |
+
self.page.unroute('**/front/data/benchmark.json')
|
| 126 |
+
self.refresh()
|
| 127 |
+
self.assertEqual(self.page.locator('#kpiModels').inner_text(), str(len(SNAPSHOT['models'])))
|
| 128 |
+
|
| 129 |
+
def test_malformed_refresh_preserves_previous_results(self):
|
| 130 |
+
self.open()
|
| 131 |
+
before = self.page.locator('#tableBody').inner_text()
|
| 132 |
+
self.page.route('**/front/data/benchmark.json', lambda route: route.fulfill(
|
| 133 |
+
content_type='application/json', body='{"models":[{"provider":"Broken"}]}'))
|
| 134 |
+
self.refresh('error')
|
| 135 |
+
self.assertEqual(self.page.locator('#tableBody').inner_text(), before)
|
| 136 |
+
self.assertIn('Previously loaded', self.page.locator('#dataStatus').inner_text())
|
| 137 |
+
|
| 138 |
+
def test_mobile_controls(self):
|
| 139 |
+
self.page.set_viewport_size({'width': 390, 'height': 844})
|
| 140 |
+
self.open()
|
| 141 |
+
self.assertTrue(self.page.locator('#refreshMobileBtn').is_visible())
|
| 142 |
+
self.assertLessEqual(self.page.evaluate('document.documentElement.scrollWidth'), 390)
|
| 143 |
+
mean = self.page.locator('#tableBody tr').first.locator('td.mean-col').bounding_box()
|
| 144 |
+
self.assertLessEqual(mean['x'] + mean['width'], 390)
|
| 145 |
+
before = self.page.locator('html').get_attribute('class') or ''
|
| 146 |
+
self.page.locator('#themeToggleMobile').click()
|
| 147 |
+
self.assertNotEqual(self.page.locator('html').get_attribute('class') or '', before)
|
| 148 |
+
self.refresh(mobile=True)
|
| 149 |
+
self.assertEqual(self.page.locator('#kpiModels').inner_text(), str(len(SNAPSHOT['models'])))
|
| 150 |
+
|
| 151 |
+
def test_blocked_storage_and_unavailable_chart_keep_table_usable(self):
|
| 152 |
+
self.page.add_init_script("Object.defineProperty(window, 'localStorage', {get() { throw new Error('blocked'); }});")
|
| 153 |
+
self.page.route('https://cdn.jsdelivr.net/npm/chart.js', lambda route: route.abort())
|
| 154 |
+
self.open()
|
| 155 |
+
self.assertTrue(self.page.locator('#chartUnavailable').is_visible())
|
| 156 |
+
self.page.locator('#searchInput').fill('qwen3-32b')
|
| 157 |
+
self.assertEqual(self.page.locator('#kpiModels').inner_text(), '1')
|
| 158 |
+
self.page.locator('#themeToggle').click()
|
| 159 |
+
|
| 160 |
+
def test_empty_results_and_untrusted_labels(self):
|
| 161 |
+
self.page.route('**/front/data/benchmark.json', lambda route: route.fulfill(
|
| 162 |
+
content_type='application/json', body='{"models":[],"datasets":[]}'))
|
| 163 |
+
self.open()
|
| 164 |
+
self.assertEqual(self.page.locator('#kpiModels').inner_text(), '0')
|
| 165 |
+
self.assertIn('No results have been published', self.page.locator('#tableBody').inner_text())
|
| 166 |
+
self.page.unroute('**/front/data/benchmark.json')
|
| 167 |
+
label = '<img src=x onerror="window.injected=true">'
|
| 168 |
+
payload = {'datasets': [label], 'models': [{
|
| 169 |
+
'provider': label, 'name': label, 'repo': label, 'is_multimodal': True,
|
| 170 |
+
'scores': [{'dataset_name': label, 'score': 42, 'metric_type': 'raw'}]}]}
|
| 171 |
+
self.page.route('**/front/data/benchmark.json', lambda route: route.fulfill(
|
| 172 |
+
content_type='application/json', body=json.dumps(payload)))
|
| 173 |
+
self.refresh()
|
| 174 |
+
self.page.locator('#selectAllBtn').click()
|
| 175 |
+
self.assertEqual(self.page.locator('#tableBody img, #datasetFilters img').count(), 0)
|
| 176 |
+
self.assertFalse(self.page.evaluate('Boolean(window.injected)'))
|
| 177 |
+
self.assertIn(label, self.page.locator('#tableBody').inner_text())
|
| 178 |
+
|
| 179 |
+
def test_legacy_array_validation_and_request_timeout(self):
|
| 180 |
+
self.open()
|
| 181 |
+
result = self.page.evaluate('''async () => {
|
| 182 |
+
const {normalizeBenchmark, loadBenchmark} = await import('/front/data.mjs');
|
| 183 |
+
const legacy = normalizeBenchmark([{provider: 'P', name: 'M', scores: [
|
| 184 |
+
{dataset_name: 'Additional', score: 0}]}]);
|
| 185 |
+
let invalid = 0;
|
| 186 |
+
for (const payload of [null, {}, {models: [{provider:'P', name:'M', scores:[
|
| 187 |
+
{dataset_name:'A', score:'99'}]}]}, {models:[], datasets:'invalid'}]) {
|
| 188 |
+
try { normalizeBenchmark(payload); } catch { invalid++; }
|
| 189 |
+
}
|
| 190 |
+
window.fetch = (_url, options) => new Promise((_resolve, reject) => {
|
| 191 |
+
options.signal.addEventListener('abort', () => reject(new DOMException('Aborted', 'AbortError')));
|
| 192 |
+
});
|
| 193 |
+
let timedOut = false;
|
| 194 |
+
try { await loadBenchmark('/never-responds', 20); } catch (error) { timedOut = error.name === 'AbortError'; }
|
| 195 |
+
return {datasets: legacy.datasets, score: legacy.models[0].scores[0].score, invalid, timedOut};
|
| 196 |
+
}''')
|
| 197 |
+
self.assertIn('Additional', result['datasets'])
|
| 198 |
+
self.assertEqual(result['score'], 0)
|
| 199 |
+
self.assertEqual(result['invalid'], 4)
|
| 200 |
+
self.assertTrue(result['timedOut'])
|
| 201 |
+
|
| 202 |
+
|
| 203 |
+
if __name__ == '__main__':
|
| 204 |
+
unittest.main()
|