Hugging Face's logo

AetherPrior
/
qwen3-8b-impl-prm-exec-think

Text Classification
Transformers
Safetensors
qwen3
reward-model
prm
code-security
text-embeddings-inference
Model card Files
xet
 Community
AetherPrior commited on
Commit
a8ab1e3
·
verified ·
1 Parent(s): eda904c

Update best PRM: combined allpairs K=3 (val acc 0.968)

Browse files
Files changed (10) hide show
  1. README.md +4 -2
  2. config.json +1 -2
  3. metrics.json +3 -3
  4. model-00001-of-00007.safetensors +1 -1
  5. model-00002-of-00007.safetensors +1 -1
  6. model-00003-of-00007.safetensors +1 -1
  7. model-00004-of-00007.safetensors +1 -1
  8. model-00005-of-00007.safetensors +1 -1
  9. model-00006-of-00007.safetensors +1 -1
  10. model-00007-of-00007.safetensors +1 -1
README.md CHANGED
@@ -21,7 +21,9 @@ Scalar Bradley–Terry preference model over **implementation guidelines**
21
  - Prefs: evolved (t+) / preferred (t) / degraded (t−) guidelines, exec-filtered
22
  with thinking enabled (keep t+ iff func∧sec; t− iff ¬sec)
23
  - Independent pairwise BT: t≻t−, t+≻t−, t+≻t
24
- - Val accuracy (pairwise): ~0.96
 
 
25
 
26
  ## Scoring
27
 
@@ -30,4 +32,4 @@ Serve with vLLM pooling/classify and `POST /classify`.
30
 
31
  ## Citation / project
32
 
33
- Internal: `prm_secode_eval/recode-style` exec-think-allpairs run.
 
21
  - Prefs: evolved (t+) / preferred (t) / degraded (t−) guidelines, exec-filtered
22
  with thinking enabled (keep t+ iff func∧sec; t− iff ¬sec)
23
  - Independent pairwise BT: t≻t−, t+≻t−, t+≻t
24
+ - Data: combined 5cwe + other CWEs, K=3 paired attempts (attempt-0 + 2 extras),
25
+ `preferences_synth_triple_exec_combined_allpairs_k3` (6933 pairs; 6192/741)
26
+ - Val accuracy (pairwise): **0.968** (741 pairs)
27
 
28
  ## Scoring
29
 
 
32
 
33
  ## Citation / project
34
 
35
+ Internal: `prm_secode_eval/recode-style` `prm_impl_bt_triple_exec_combined_allpairs_k3` best.
config.json CHANGED
@@ -71,6 +71,5 @@
71
  "transformers_version": "4.57.6",
72
  "use_cache": false,
73
  "use_sliding_window": false,
74
- "vocab_size": 151936,
75
- "num_labels": 1
76
  }
 
71
  "transformers_version": "4.57.6",
72
  "use_cache": false,
73
  "use_sliding_window": false,
74
+ "vocab_size": 151936
 
75
  }
metrics.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "accuracy": 0.9601063829787234,
3
- "loss": 0.13640314953429444,
4
- "n": 752.0
5
  }
 
1
  {
2
+ "accuracy": 0.9676113360323887,
3
+ "loss": 0.10553132820504203,
4
+ "n": 741.0
5
  }
model-00001-of-00007.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3bfaeeee8239eb94ae00355259cdeefba83c65cc8c67c952f466370b4e36ed6c
3
  size 4972454376
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d23a80544bf8a41c917c94019c83bf99700719d078b53f8ff8fc55029943455b
3
  size 4972454376
model-00002-of-00007.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8e505bbf8d04abef84b23b086a2f9ec98719127688ffb4f8206220c7b4797cf9
3
  size 4832048608
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be5f117b3c301a0d2884c683de55c6a595f02b0012e82247bc724ce345a3f2a0
3
  size 4832048608
model-00003-of-00007.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:775530d351e13ca9cf2e0c705401ea6b4caa8fa5842a19d051b401917ffe76f9
3
  size 4832048656
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d72992e10994adb49d123ae92db327d39ffc75a6f347f55104abf5c2c9d0381
3
  size 4832048656
model-00004-of-00007.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d81f1d288a0d759ed84508313b098ef82eee8db2b35ae6ffe21f3812ff3e32d4
3
  size 4999855528
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:22bac32880175dfea2a415a5198d34711f624540dc52ca1147bb261911717a35
3
  size 4999855528
model-00005-of-00007.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2023bed8b7956290872082375e56ea129f9a871f96858edd5bba6adcc149ed58
3
  size 4832048672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:da37a80086ef58941ccb2d1015f1202cd4b8533fa3875c6c66038f1393a97286
3
  size 4832048672
model-00006-of-00007.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a08870d2bb3aa9a0a47be44c2e65de6eb96ac6447272ab721bc2c3d1e8a0a8d9
3
  size 4832048672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:41f3620113e28fcb9c2b01af66d3c1238ea8c79b689a6ec9b0b42a58b50aa63b
3
  size 4832048672
model-00007-of-00007.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:24b98b199367ddac7c512de071481ee36f1534b198c0d4067f4c1936e97a33f1
3
  size 973179640
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:305efcea4fd0000f6162e768b64da5cf91123c43db1bc6d406eb2d30316f5c38
3
  size 973179640