Hugging Face's logo

benthecarman
/
MiMo-V2.6-Flash-RL-exl3

Text Generation
Safetensors
English
Chinese
mimo_v2
exl3
exllamav3
mimo
dgx-spark
speculative-decoding
conversational
custom_code
2-bit
Model card Files
xet
 Community
1
benthecarman commited on
Commit
c1e3fc1
·
verified ·
1 Parent(s): d42702c

Layer 47: interm_div instead of 6 bpw + bad-row override (2.27 bpw)

Browse files
Files changed (8) hide show
  1. README.md +58 -69
  2. config.json +378 -381
  3. eval/eval-overflow-idiv47-cal250.json +2535 -0
  4. eval/eval-overflow-idiv47-wiki64.json +861 -0
  5. model-00011-of-00012.safetensors +2 -2
  6. model-00012-of-00012.safetensors +2 -2
  7. model.safetensors.index.json +1 -1
  8. quantization_config.json +2 -2
README.md CHANGED
@@ -14,10 +14,10 @@ tags:
14
  - speculative-decoding
15
  ---
16
 
17
- # MiMo-V2.6-Flash-RL — EXL3 2.34 bpw (with the DFlash drafter)
18
 
19
  [XiaomiMiMo/MiMo-V2.6-Flash-RL](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL)
20
- quantized to **EXL3 ~2.34 bpw** so that a 309B-parameter MoE runs on **one 128 GB
21
  machine**. Built and measured end to end on an **NVIDIA DGX Spark (GB10, 121.6 GiB
22
  unified memory, aarch64)**.
23
 
@@ -25,13 +25,22 @@ The repo also ships Xiaomi's **DFlash** drafter in `dflash/`, repaired so it act
25
  loads, so speculative decoding works out of the box (**1.58x on coding, 1.70x on
26
  reasoning — and up to 2.08x at 250K context** — see [Speed](#speed-on-a-dgx-spark)).
27
 
 
 
 
 
 
 
 
 
 
28
  | | |
29
  |---|---|
30
- | Weights | 12 shards, **92,423,978,606 B = 92.42 GB = 86.08 GiB**; 92.50 GB / 86.15 GiB for the whole model directory (87 G on disk) |
31
- | Bitrate | **2.34 bpw** excluding head (converter's own figure), **2.36 bpw** as reported at load; head 6.00 |
32
  | Drafter | `dflash/`, BF16, 2.94 GB, +2.81 GiB at runtime |
33
- | Perplexity | **5.3615** wikitext-2 test, 64 x 2048 tokens (5.3174 over all 146 rows) |
34
- | Decode | **31.5 tok/s** @2K ctx, **28.5 tok/s** @32K, batch 1, no drafter; ~42 tok/s sustained with DFlash |
35
  | Peak host memory | **95.06 GiB** of 121.63 at 32K context |
36
  | Max context | **262,144 served with the drafter attached** (393,216 measured and working, with less headroom); ~980K tokens at 1 slot within a 25.4 GiB KV budget, model max 1,048,576 |
37
  | Modalities | **text only** — no vision tower, no audio tower, no MTP head |
@@ -61,10 +70,10 @@ faithful description of the base model), but no corresponding tensors exist here
61
  ### What is in this repo
62
 
63
  ```
64
- model-0000{1..12}-of-00012.safetensors EXL3 weights, 92.42 GB (86.08 GiB)
65
  model.safetensors.index.json 13.5 MB
66
  quantization_config.json 44.7 MB — per-tensor storage record, exllamav3 1.5.1
67
- config.json + "exl3_fp32_mlp_layers": "47" (see Known issues)
68
  tokenizer.json vocab.json merges.txt tokenizer_config.json chat_template.jinja
69
  generation_config.json
70
  configuration_mimo_v2.py modeling_mimo_v2.py upstream remote code, unmodified (auto_map)
@@ -80,9 +89,8 @@ eval/
80
  longctx.json the long-context sweep, every request (see Long context)
81
  ```
82
 
83
- The shards, index, `quantization_config.json` and the tokenizer files are byte-identical to
84
- the converter's output (sha256-verified on transfer). Only `config.json` differs from what
85
- the converter wrote: one key was added, described under [Known issues](#known-issues).
86
 
87
  ---
88
 
@@ -94,61 +102,51 @@ they matter and the per-module result is what actually landed:
94
  | module group | layers | bpw |
95
  |---|---|---|
96
  | routed experts (`mlp.experts.*.{gate,up,down}_proj`) | 24 layers (12–35) | **2.0** |
97
- | routed experts | 22 layers (1–11, 36–46) | **2.5** |
98
- | routed experts, **layer 47** | 1 layer | **6.0** (deliberate override — see below) |
99
  | attention (`q/k/v/o_proj`) | all 48 | **4.0** |
100
  | dense MLP, layer 0 (`gate/up/down_proj`) | 1 | **3.0** |
101
  | `lm_head` | — | **6.0** |
102
  | `embed_tokens`, all norms, router weights, `e_score_correction_bias`, attention sinks | — | **16-bit** (BF16, unquantized) |
103
 
104
- Converter's summary line: `Final bitrate (excluding head): 2.34 (--hq enabled)`; exllamav3
105
- reports **2.36 bpw / 6.00 head** when it loads the finished model.
106
 
107
  ### The command
108
 
109
  Converted with [exllamav3](https://github.com/turboderp-org/exllamav3) 1.5.1 plus the
110
- MiMo-V2.6 port, on a single RTX PRO 6000 (7.3 h, ~$29 of rented GPU):
 
111
 
112
  ```sh
113
  python convert.py \
114
  -i MiMo-V2.6-Flash-RL -o mimo-2.25bpw-hq -w work \
115
  -b 2.25 -hq \
116
- -cr 250 -cc 2048 \
117
- --module_bits 'model.layers.47.mlp.experts.*=6' \
118
- --max_bad_rows 0.4
119
  ```
120
 
121
  `-cr 250 -cc 2048` is the converter default (250 calibration rows x 2048 tokens) and was
122
  kept deliberately: measuring `-cr 120` saved only 6.1% of wall time, because calibration is
123
  ~9% of an MoE layer and LDLQ + trellis search is the other ~90%.
124
 
125
- ### The layer-47 note, and why 31 calibration rows were dropped
126
-
127
- Layer 47's routed-expert intermediates (`act_fn(gate(x)) * up(x)`, 2048 wide) **overflow
128
- fp16 on ~12% of calibration rows** under normal top-8 routing — and they do so in the
129
- *unquantized* arithmetic too. The count is essentially independent of bitrate: 32 of 250
130
- rows at 2.5 bpw, **31 of 250 at 6.0 bpw**. A 2.4x increase in bits moved one row, which
131
- rules out quantization error and points at the checkpoint: that layer simply sits near the
132
- fp16 ceiling for some real tokens. (Xiaomi's own reference stack runs bf16 there, which has
133
- the same 5-bit exponent as fp32 and does not overflow.)
134
 
135
- exllamav3 aborts a job when more than 10% of calibration rows go non-finite. 31/250 =
136
- 12.4%, just over the line. The fix was to **stop aborting**, via a new `--max_bad_rows`
137
- flag (upstream PR #397). The consequences are bounded:
 
138
 
139
- * The 31 rows are excluded **after** layer 47 has been quantized, so layer 47 itself saw
140
- all 250 rows.
141
- * The only module calibrated on the reduced set is **`lm_head`**, on 219 rows:
142
- `bpw 6.00 / rfn 0.006888 / sqnr 43.27` — in line with every other head in the project.
143
- * `model.norm` has no quant targets.
144
 
145
- Layer 47's experts were separately pinned to **6.0 bpw** (`--module_bits`) as insurance
146
- while that failure was being diagnosed. It was kept. Its reported `rfn 0.0312` looks worse
147
- than the 2.5 bpw layers' 0.0150–0.0175, but that is a measurement artefact: the reference
148
- state it is scored against is itself partly inf-contaminated in this layer.
 
149
 
150
- For inference the same issue is handled by `"exl3_fp32_mlp_layers": "47"` in `config.json`
151
- — see [Known issues](#known-issues).
 
152
 
153
  ---
154
 
@@ -160,15 +158,16 @@ wikitext-2 **test** split, non-overlapping rows, via `exllamav3/eval/ppl.py`:
160
 
161
  | rows x length | perplexity | scored tokens |
162
  |---|---|---|
163
- | 16 x 2048 | 5.2410 | 32,752 |
164
- | **64 x 2048** | **5.3615** | 131,008 |
165
- | 146 x 2048 (the entire test split) | 5.3174 | 298,862 |
166
 
167
- Row count moves the number by ~±0.04, so quote it with the number. **0 non-finite tokens**
168
- in all three runs.
 
169
 
170
  ### Benchmarks
171
 
 
 
172
  **Read this first.** These are absolute scores from one specific harness, so they are **not**
173
  comparable to anyone else's published figures. They *are* comparable to each other: the
174
  "reference FP8" column is the **unquantized model**, run through the **same script, the same
@@ -291,7 +290,8 @@ local side generated, so the two columns are always over the same items.
291
 
292
  ## Speed on a DGX Spark
293
 
294
- GB10, 121.6 GiB unified memory, batch 1, greedy, FP16 KV.
 
295
 
296
  ### Decode and prefill, no drafter
297
 
@@ -359,7 +359,7 @@ With 86.20 GiB of weights and a 10 GiB reserve, the KV budget is 25.43 GiB:
359
  | full (non-ring) KV, for comparison | 102,144 | 25,344 |
360
 
361
  64K context at 1 slot costs 1.86 GiB; 32K costs 1.02 GiB. Quantized paged KV (k=v=8) loads
362
- and generates coherently on top of these 2.36 bpw weights, but its quality is untested — the
363
  default stays FP16. The 39 sliding-window rings are always FP16.
364
 
365
  Those are the arithmetic limits. What was actually served, loaded and measured — up to
@@ -573,7 +573,7 @@ draft_model:
573
  previous request are inflated by page reuse (755 vs 473 tok/s at 130K here), so cold numbers
574
  are the ones quoted above.
575
  * Q8 paged KV halves the 27 KiB/token term and would make ~786K arithmetically fit, but
576
- quantized KV quality on top of 2.34 bpw weights is untested and the prefill time would be
577
  hours. The 39 sliding-window rings stay FP16 regardless.
578
  * Raw records: [`eval/longctx.json`](eval/longctx.json).
579
 
@@ -584,11 +584,10 @@ draft_model:
584
  ### Requirements
585
 
586
  This quant does **not** load on released exllamav3. `MiMoV2ForCausalLM` is not upstream yet;
587
- the port is in four open PRs (turboderp-org/exllamav3
588
  [#396](https://github.com/turboderp-org/exllamav3/pull/396),
589
- [#397](https://github.com/turboderp-org/exllamav3/pull/397),
590
  [#398](https://github.com/turboderp-org/exllamav3/pull/398),
591
- [#399](https://github.com/turboderp-org/exllamav3/pull/399)) and all four sit on one branch:
592
 
593
  * **exllamav3** — https://github.com/benthecarman/exllamav3/tree/mimo-v2.6-flash
594
  * **TabbyAPI** — https://github.com/benthecarman/tabbyAPI/tree/mimo-v2.6-flash
@@ -667,17 +666,14 @@ on the exllamav3 branch.
667
  | variable | default | what it does |
668
  |---|---|---|
669
  | `EXL3_LOAD_DEVICE` | unset | set to `cuda:0` on GB10 / any unified-memory box: single-device load, no autosplit, no MemFree arithmetic. **Recommended here.** |
670
- | `EXL3_MIMO_FP32_MLP_LAYERS` | from `config.json` → `"47"` | which layers run fp32 MoE intermediates. Syntax `47`, `40-47`, `0,47`, `all`, `none`, `-1`. |
671
- | `EXL3_MIMO_MLP_ACT_LIMIT` | `0` (off) | clamps `act_fn(g)` / `u` on the overridden layers; the only thing that bounds the ≤8-row fused *decode* kernel |
672
- | `EXL3_MIMO_FP32_MLP_FUSED` | `0` | restores the fused fp16 prefill tier on overridden layers — **A/B only**, it cancels the guard |
673
  | `EXL3_KEEP_PAGE_CACHE` | unset | keep shard page cache after load (don't, on unified memory) |
674
 
675
  You should see this at load:
676
 
677
  ```
678
- -- MiMoV2: fp32 MLP intermediates on layer(s) [47] (spec '47')
679
  -- EXL3_LOAD_DEVICE=cuda:0 (single-device load)
680
- -- Released 86.1 GiB of shard page cache
681
  ```
682
 
683
  ### The DFlash drafter
@@ -734,21 +730,14 @@ The drafter loads BF16 in 1.3 s / 2.81 GiB — no quantization needed. Its own K
734
  single divergence at a top1–top2 margin of 0.0156 = two fp16 ULP, reproduced identically
735
  with the ring disabled). Coding was token-identical across all four modes. Don't diff
736
  outputs across drafting modes and expect equality.
737
- * **Layer 47 runs fp32 MoE intermediates by default**, via `"exl3_fp32_mlp_layers": "47"` in
738
- `config.json`. This is the one key added to the converter's output. It costs **−2.5% of
739
- prefill and nothing measurable on decode** (ppl 5.3614 → 5.3615 at 64 rows, identical
740
- scored-token count; decode −0.3%, noise). wikitext-2 shows **0 bad rows in 146 rows with
741
- the guard off**, so the guard is not needed *for wikitext* — it is kept because the
742
- overflow is a property of *this checkpoint* (31/250 converter calibration rows,
743
- unquantized, bitrate-independent) and the failure it prevents is silent garbage, not an
744
- error. Override with `EXL3_MIMO_FP32_MLP_LAYERS=none`. Note the non-obvious half: the
745
- fused prefill tier (`exl3_moe`) hard-requires fp16 intermediates, so the override also has
746
- to drop that tier — which is where the 2.5% goes.
747
  * **Instruction-following on code is imperfect.** A sample `merge_intervals` completion was a
748
  correct, idiomatic, correctly-analysed algorithm with one real bug (it sorts the caller's
749
  list in place and assumes list elements, so it raises on the tuples the prompt specified)
750
  and it skipped the three asserts the prompt asked for. Consistent with a strong-but-not-
751
- perfect 2.34 bpw quant — and the benchmark deltas above say the FP8 model is only ~0.6 pp
752
  better at HumanEval+, so most of that is the base model, not the quantization.
753
  * **Tensor parallel is unsupported.** The port has only been built and run single-device.
754
  * **Long context is now measured, and it is TTFT-bound rather than memory-bound.** 8K → 350K
 
14
  - speculative-decoding
15
  ---
16
 
17
+ # MiMo-V2.6-Flash-RL — EXL3 2.27 bpw (with the DFlash drafter)
18
 
19
  [XiaomiMiMo/MiMo-V2.6-Flash-RL](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL)
20
+ quantized to **EXL3 ~2.27 bpw** so that a 309B-parameter MoE runs on **one 128 GB
21
  machine**. Built and measured end to end on an **NVIDIA DGX Spark (GB10, 121.6 GiB
22
  unified memory, aarch64)**.
23
 
 
25
  loads, so speculative decoding works out of the box (**1.58x on coding, 1.70x on
26
  reasoning — and up to 2.08x at 250K context** — see [Speed](#speed-on-a-dgx-spark)).
27
 
28
+ > **Updated 2026-09-23.** Layer 47 was re-quantized with a proper fix for its fp16 overflow
29
+ > (`interm_div`, see [below](#layer-47-and-interm_div)) instead of a 6 bpw override plus a raised
30
+ > bad-row limit. This build needs the current
31
+ > [`mimo-v2.6-flash` exllamav3 branch](https://github.com/benthecarman/exllamav3/tree/mimo-v2.6-flash);
32
+ > the previous weights do not load correctly on it, and these weights do not load correctly on the
33
+ > old branch. Update both together. Size, bitrate and perplexity below are for the new build. The
34
+ > benchmark, speed, memory and long-context sections were measured on the previous 2.34 bpw build
35
+ > (layers 0-46 and the drafter are byte-identical to it) and have not been re-run.
36
+
37
  | | |
38
  |---|---|
39
+ | Weights | 12 shards, **89,605,407,713 B = 89.61 GB = 83.45 GiB**; 89.68 GB / 83.52 GiB for the whole model directory |
40
+ | Bitrate | **2.27 bpw** excluding head (converter's own figure); head 6.00 |
41
  | Drafter | `dflash/`, BF16, 2.94 GB, +2.81 GiB at runtime |
42
+ | Perplexity | **5.4013** wikitext-2 test, 64 x 2048 tokens (previous build: 5.3615) |
43
+ | Decode | **31.5 tok/s** @2K ctx, **28.5 tok/s** @32K, batch 1, no drafter; ~42 tok/s sustained with DFlash (previous build; the new one measured 30.0 vs 30.3 tok/s for the old in the same short-prompt harness) |
44
  | Peak host memory | **95.06 GiB** of 121.63 at 32K context |
45
  | Max context | **262,144 served with the drafter attached** (393,216 measured and working, with less headroom); ~980K tokens at 1 slot within a 25.4 GiB KV budget, model max 1,048,576 |
46
  | Modalities | **text only** — no vision tower, no audio tower, no MTP head |
 
70
  ### What is in this repo
71
 
72
  ```
73
+ model-0000{1..12}-of-00012.safetensors EXL3 weights, 89.61 GB (83.45 GiB)
74
  model.safetensors.index.json 13.5 MB
75
  quantization_config.json 44.7 MB — per-tensor storage record, exllamav3 1.5.1
76
+ config.json
77
  tokenizer.json vocab.json merges.txt tokenizer_config.json chat_template.jinja
78
  generation_config.json
79
  configuration_mimo_v2.py modeling_mimo_v2.py upstream remote code, unmodified (auto_map)
 
89
  longctx.json the long-context sweep, every request (see Long context)
90
  ```
91
 
92
+ The shards, index, `quantization_config.json`, `config.json` and the tokenizer files are the
93
+ converter's output, unmodified. The files under `eval/` are from the previous build.
 
94
 
95
  ---
96
 
 
102
  | module group | layers | bpw |
103
  |---|---|---|
104
  | routed experts (`mlp.experts.*.{gate,up,down}_proj`) | 24 layers (12–35) | **2.0** |
105
+ | routed experts | 23 layers (1–11, 36–47) | **2.5** |
 
106
  | attention (`q/k/v/o_proj`) | all 48 | **4.0** |
107
  | dense MLP, layer 0 (`gate/up/down_proj`) | 1 | **3.0** |
108
  | `lm_head` | — | **6.0** |
109
  | `embed_tokens`, all norms, router weights, `e_score_correction_bias`, attention sinks | — | **16-bit** (BF16, unquantized) |
110
 
111
+ Converter's summary line: `Final bitrate (excluding head): 2.27 (--hq enabled)`.
 
112
 
113
  ### The command
114
 
115
  Converted with [exllamav3](https://github.com/turboderp-org/exllamav3) 1.5.1 plus the
116
+ MiMo-V2.6 port, on a single RTX PRO 6000 (7.3 h, ~$29 of rented GPU; layer 47 and the head
117
+ were later redone from the layer-47 checkpoint in 15 min):
118
 
119
  ```sh
120
  python convert.py \
121
  -i MiMo-V2.6-Flash-RL -o mimo-2.25bpw-hq -w work \
122
  -b 2.25 -hq \
123
+ -cr 250 -cc 2048
 
 
124
  ```
125
 
126
  `-cr 250 -cc 2048` is the converter default (250 calibration rows x 2048 tokens) and was
127
  kept deliberately: measuring `-cr 120` saved only 6.1% of wall time, because calibration is
128
  ~9% of an MoE layer and LDLQ + trellis search is the other ~90%.
129
 
130
+ ### Layer 47 and `interm_div`
 
 
 
 
 
 
 
 
131
 
132
+ On this checkpoint, layer 47's routed experts push `act(gate) * up` to about 84k on some
133
+ number tokens (mostly expert 208 channel 18 and expert 70 channel 1387), past the fp16 max of
134
+ 65504. `gate` and `up` on their own stay under 2.3k, and no other layer gets above 3k. Xiaomi's
135
+ reference runs bf16 there and stays finite; exllamav3 runs these intermediates in fp16.
136
 
137
+ The previous build worked around it at conversion time: layer 47's experts at 6 bpw, and a
138
+ raised limit on non-finite calibration rows (31 of 250 were dropped). It still produced
139
+ non-finite logits at inference on 33 of the converter's 250 calibration rows.
 
 
140
 
141
+ This build uses `interm_div = 128` on layer 47, the same mechanism exllamav3 uses for Laguna.
142
+ `up_proj` is scaled by 1/128 before quantization and `routed_scaling_factor` puts the 128 back
143
+ in fp32, so the peak becomes about 660. No calibration rows went non-finite during conversion,
144
+ and the finished model gives 0 non-finite rows on those same 250 rows. Layer 47's experts are
145
+ back at 2.5 bpw, which accounts for the smaller file and most of the small perplexity change.
146
 
147
+ Because the 1/128 lives in the quantized weights, these weights need a matching exllamav3
148
+ branch. fp32 intermediates are not a substitute: the fused prefill kernel stores fp16
149
+ regardless, and without it the activation kernel clamps the product to 65504.
150
 
151
  ---
152
 
 
158
 
159
  | rows x length | perplexity | scored tokens |
160
  |---|---|---|
161
+ | **64 x 2048** | **5.4013** | 131,008 |
 
 
162
 
163
+ Previous build, same method: 16 x 2048 5.2410, 64 x 2048 5.3615, 146 x 2048 (the entire test
164
+ split) 5.3174. Row count moves the number by ~±0.04, so quote it with the number.
165
+ **0 non-finite tokens** in every run.
166
 
167
  ### Benchmarks
168
 
169
+ **Measured on the previous 2.34 bpw build**, whose layer 47 differs from this one. Not re-run.
170
+
171
  **Read this first.** These are absolute scores from one specific harness, so they are **not**
172
  comparable to anyone else's published figures. They *are* comparable to each other: the
173
  "reference FP8" column is the **unquantized model**, run through the **same script, the same
 
290
 
291
  ## Speed on a DGX Spark
292
 
293
+ GB10, 121.6 GiB unified memory, batch 1, greedy, FP16 KV. Measured on the previous 2.34 bpw
294
+ build (86.1 GiB); the new one is 2.6 GiB smaller.
295
 
296
  ### Decode and prefill, no drafter
297
 
 
359
  | full (non-ring) KV, for comparison | 102,144 | 25,344 |
360
 
361
  64K context at 1 slot costs 1.86 GiB; 32K costs 1.02 GiB. Quantized paged KV (k=v=8) loads
362
+ and generates coherently on top of the previous 2.36 bpw weights, but its quality is untested — the
363
  default stays FP16. The 39 sliding-window rings are always FP16.
364
 
365
  Those are the arithmetic limits. What was actually served, loaded and measured — up to
 
573
  previous request are inflated by page reuse (755 vs 473 tok/s at 130K here), so cold numbers
574
  are the ones quoted above.
575
  * Q8 paged KV halves the 27 KiB/token term and would make ~786K arithmetically fit, but
576
+ quantized KV quality on top of ~2.3 bpw weights is untested and the prefill time would be
577
  hours. The 39 sliding-window rings stay FP16 regardless.
578
  * Raw records: [`eval/longctx.json`](eval/longctx.json).
579
 
 
584
  ### Requirements
585
 
586
  This quant does **not** load on released exllamav3. `MiMoV2ForCausalLM` is not upstream yet;
587
+ the port is in three open PRs (turboderp-org/exllamav3
588
  [#396](https://github.com/turboderp-org/exllamav3/pull/396),
 
589
  [#398](https://github.com/turboderp-org/exllamav3/pull/398),
590
+ [#399](https://github.com/turboderp-org/exllamav3/pull/399)) and all three sit on one branch:
591
 
592
  * **exllamav3** — https://github.com/benthecarman/exllamav3/tree/mimo-v2.6-flash
593
  * **TabbyAPI** — https://github.com/benthecarman/tabbyAPI/tree/mimo-v2.6-flash
 
666
  | variable | default | what it does |
667
  |---|---|---|
668
  | `EXL3_LOAD_DEVICE` | unset | set to `cuda:0` on GB10 / any unified-memory box: single-device load, no autosplit, no MemFree arithmetic. **Recommended here.** |
669
+ | `EXL3_MIMO_FP32_MLP_LAYERS` | `none` | debugging only: layers to run with fp32 MoE intermediates. Not needed with this build. |
 
 
670
  | `EXL3_KEEP_PAGE_CACHE` | unset | keep shard page cache after load (don't, on unified memory) |
671
 
672
  You should see this at load:
673
 
674
  ```
 
675
  -- EXL3_LOAD_DEVICE=cuda:0 (single-device load)
676
+ -- Released 83.5 GiB of shard page cache
677
  ```
678
 
679
  ### The DFlash drafter
 
730
  single divergence at a top1–top2 margin of 0.0156 = two fp16 ULP, reproduced identically
731
  with the ring disabled). Coding was token-identical across all four modes. Don't diff
732
  outputs across drafting modes and expect equality.
733
+ * **Weights and code must match.** Layer 47's `interm_div` is folded into the quantized weights.
734
+ This build on the old branch, or the old build on the current branch, runs without an error
735
+ and gives wrong output. See [Layer 47 and `interm_div`](#layer-47-and-interm_div).
 
 
 
 
 
 
 
736
  * **Instruction-following on code is imperfect.** A sample `merge_intervals` completion was a
737
  correct, idiomatic, correctly-analysed algorithm with one real bug (it sorts the caller's
738
  list in place and assumes list elements, so it raises on the tuples the prompt specified)
739
  and it skipped the three asserts the prompt asked for. Consistent with a strong-but-not-
740
+ perfect ~2.3 bpw quant — and the benchmark deltas above say the FP8 model is only ~0.6 pp
741
  better at HumanEval+, so most of that is the base model, not the quantization.
742
  * **Tensor parallel is unsupported.** The port has only been built and run single-device.
743
  * **Long context is now measured, and it is TTFT-bound rather than memory-bound.** 8K → 350K
config.json CHANGED
@@ -1,391 +1,388 @@
1
  {
2
- "add_full_attention_sink_bias": false,
3
- "add_swa_attention_sink_bias": true,
4
- "architectures": [
5
- "MiMoV2ForCausalLM"
6
- ],
7
- "attention_bias": false,
8
- "attention_chunk_size": 128,
9
- "attention_dropout": 0.0,
10
- "attention_projection_layout": "fused_qkv",
11
- "attention_value_scale": 0.707,
12
- "moe_router_dtype": "bfloat16",
13
- "audio_config": {
14
- "add_post_norm": true,
15
- "audio_channels": 20,
16
- "audio_segment_size": 6000,
17
- "group_size": 4,
18
- "input_full_attention": true,
19
- "input_local_attn_heads": 16,
20
- "input_local_dim": 1024,
21
- "input_local_head_dim": 64,
22
- "input_local_hidden_dropout": 0.0,
23
- "input_local_intermediate_size": 4096,
24
- "input_local_layers": 6,
25
- "out_hidden_size": 4096,
26
- "partial_rotary_factor": 1.0,
27
- "projection_layers": 2,
28
- "rope_theta": 640000,
29
- "speech_vocab_size": "1280",
30
- "speech_zeroemb_idx": "1024"
31
- },
32
- "audio_end_token_id": 151674,
33
- "audio_start_token_id": 151673,
34
- "audio_token_id": 151669,
35
- "auto_map": {
36
- "AutoConfig": "configuration_mimo_v2.MiMoV2Config",
37
- "AutoModel": "modeling_mimo_v2.MiMoV2Model",
38
- "AutoModelForCausalLM": "modeling_mimo_v2.MiMoV2ForCausalLM"
39
- },
40
- "bos_token_id": null,
41
- "dtype": "bfloat16",
42
- "eos_token_id": 151645,
43
- "head_dim": 192,
44
- "hidden_act": "silu",
45
- "hidden_size": 4096,
46
- "hybrid_block_size": null,
47
- "hybrid_layer_pattern": [
48
- 0,
49
- 1,
50
- 1,
51
- 1,
52
- 1,
53
- 0,
54
- 1,
55
- 1,
56
- 1,
57
- 1,
58
- 1,
59
- 0,
60
- 1,
61
- 1,
62
- 1,
63
- 1,
64
- 1,
65
- 0,
66
- 1,
67
- 1,
68
- 1,
69
- 1,
70
- 1,
71
- 0,
72
- 1,
73
- 1,
74
- 1,
75
- 1,
76
- 1,
77
- 0,
78
- 1,
79
- 1,
80
- 1,
81
- 1,
82
- 1,
83
- 0,
84
- 1,
85
- 1,
86
- 1,
87
- 1,
88
- 1,
89
- 0,
90
- 1,
91
- 1,
92
- 1,
93
- 1,
94
- 1,
95
- 0
96
- ],
97
- "image_token_id": 151655,
98
- "initializer_range": 0.02,
99
- "intermediate_size": 16384,
100
- "layernorm_epsilon": 1e-06,
101
- "max_position_embeddings": 1048576,
102
- "model_type": "mimo_v2",
103
- "moe_intermediate_size": 2048,
104
- "moe_layer_freq": [
105
- 0,
106
- 1,
107
- 1,
108
- 1,
109
- 1,
110
- 1,
111
- 1,
112
- 1,
113
- 1,
114
- 1,
115
- 1,
116
- 1,
117
- 1,
118
- 1,
119
- 1,
120
- 1,
121
- 1,
122
- 1,
123
- 1,
124
- 1,
125
- 1,
126
- 1,
127
- 1,
128
- 1,
129
- 1,
130
- 1,
131
- 1,
132
- 1,
133
- 1,
134
- 1,
135
- 1,
136
- 1,
137
- 1,
138
- 1,
139
- 1,
140
- 1,
141
- 1,
142
- 1,
143
- 1,
144
- 1,
145
- 1,
146
- 1,
147
- 1,
148
- 1,
149
- 1,
150
- 1,
151
- 1,
152
- 1
153
- ],
154
- "n_group": 1,
155
- "n_routed_experts": 256,
156
- "n_shared_experts": null,
157
- "norm_topk_prob": true,
158
- "num_attention_heads": 64,
159
- "num_experts_per_tok": 8,
160
- "num_hidden_layers": 48,
161
- "num_key_value_heads": 4,
162
- "num_nextn_predict_layers": 3,
163
- "pad_token_id": 151643,
164
- "partial_rotary_factor": 0.334,
165
- "processor_config": {
166
- "audio_avg_pooler": 2,
167
- "audio_channels": 20,
168
  "audio_end_token_id": 151674,
169
- "audio_fmax": null,
170
- "audio_fmin": 0,
171
- "audio_group_size": 4,
172
- "audio_hop_length": 240,
173
- "audio_input_id_per_second": 25.0,
174
- "audio_kernel_size": 3,
175
- "audio_n_mels": 128,
176
- "audio_nfft": 960,
177
- "audio_sampling_rate": 24000,
178
- "audio_segment_size": 6000,
179
  "audio_start_token_id": 151673,
180
- "audio_stride_size": 2,
181
  "audio_token_id": 151669,
182
- "audio_window_size": 960,
183
- "audio_zeroemb_idx": [
184
- 1024,
185
- 1024,
186
- 1024,
187
- 1024,
188
- 1024,
189
- 1024,
190
- 1024,
191
- 1024,
192
- 1024,
193
- 1024,
194
- 1024,
195
- 1024,
196
- 1024,
197
- 1024,
198
- 1024,
199
- 1024,
200
- 1024,
201
- 1024,
202
- 1024,
203
- 1024
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
204
  ],
205
- "fps": 1.0,
206
- "image_max_pixels": 8388608,
207
- "image_min_pixels": 8192,
208
  "image_token_id": 151655,
209
- "max_frames": 3600,
210
- "merge_size": 2,
211
- "min_frames": null,
212
- "num_frames": null,
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
213
  "pad_token_id": 151643,
214
- "patch_size": 16,
215
- "rope_type": "rope",
216
- "temporal_compression_ratio": 1,
217
- "temporal_patch_size": 2,
218
- "use_per_grid_t_timestamps": false,
219
- "use_video_timestamps": true,
220
- "video_audio_interleave_length": 0.0,
221
- "video_end_token_id": 151671,
222
- "video_max_pixels": 8388608,
223
- "video_min_pixels": 8192,
224
- "video_process_num_threads": 16,
225
- "video_start_token_id": 151670,
226
- "video_token_id": 151656,
227
- "video_tokens_per_second": 2,
228
- "video_total_max_pixels": 268435456,
229
- "vision_end_token_id": 151653,
230
- "vision_start_token_id": 151652
231
- },
232
- "quantization_config": {
233
- "quant_method": "exl3",
234
- "version": "1.5.1",
235
- "bits": 2,
236
- "bits_per_weight": 2.34,
237
- "bits_note": "bits is the integer floor for Hub config validation; bits_per_weight is the true average excluding the head (see README). Per-tensor widths are in quantization_config.json.",
238
- "head_bits": 6,
239
- "calibration": {
240
- "rows": 250,
241
- "cols": 2048
242
- },
243
- "out_scales": "always",
244
- "codebook": "mul1",
245
- "original_quantization_config": {
246
- "activation_scheme": "dynamic",
247
- "fmt": "e4m3",
248
- "ignored_layers": [
249
- "model.layers.0.self_attn.o_proj",
250
- "model.layers.1.self_attn.o_proj",
251
- "model.layers.10.self_attn.o_proj",
252
- "model.layers.11.self_attn.o_proj",
253
- "model.layers.12.self_attn.o_proj",
254
- "model.layers.13.self_attn.o_proj",
255
- "model.layers.14.self_attn.o_proj",
256
- "model.layers.15.self_attn.o_proj",
257
- "model.layers.16.self_attn.o_proj",
258
- "model.layers.17.self_attn.o_proj",
259
- "model.layers.18.self_attn.o_proj",
260
- "model.layers.19.self_attn.o_proj",
261
- "model.layers.2.self_attn.o_proj",
262
- "model.layers.20.self_attn.o_proj",
263
- "model.layers.21.self_attn.o_proj",
264
- "model.layers.22.self_attn.o_proj",
265
- "model.layers.23.self_attn.o_proj",
266
- "model.layers.24.self_attn.o_proj",
267
- "model.layers.25.self_attn.o_proj",
268
- "model.layers.26.self_attn.o_proj",
269
- "model.layers.27.self_attn.o_proj",
270
- "model.layers.28.self_attn.o_proj",
271
- "model.layers.29.self_attn.o_proj",
272
- "model.layers.3.self_attn.o_proj",
273
- "model.layers.30.self_attn.o_proj",
274
- "model.layers.31.self_attn.o_proj",
275
- "model.layers.32.self_attn.o_proj",
276
- "model.layers.33.self_attn.o_proj",
277
- "model.layers.34.self_attn.o_proj",
278
- "model.layers.35.self_attn.o_proj",
279
- "model.layers.36.self_attn.o_proj",
280
- "model.layers.37.self_attn.o_proj",
281
- "model.layers.38.self_attn.o_proj",
282
- "model.layers.39.self_attn.o_proj",
283
- "model.layers.4.self_attn.o_proj",
284
- "model.layers.40.self_attn.o_proj",
285
- "model.layers.41.self_attn.o_proj",
286
- "model.layers.42.self_attn.o_proj",
287
- "model.layers.43.self_attn.o_proj",
288
- "model.layers.44.self_attn.o_proj",
289
- "model.layers.45.self_attn.o_proj",
290
- "model.layers.46.self_attn.o_proj",
291
- "model.layers.47.self_attn.o_proj",
292
- "model.layers.5.self_attn.o_proj",
293
- "model.layers.6.self_attn.o_proj",
294
- "model.layers.7.self_attn.o_proj",
295
- "model.layers.8.self_attn.o_proj",
296
- "model.layers.9.self_attn.o_proj",
297
- "model.decoder.self_attn.o_proj"
298
- ],
299
- "mxfp4_block_size": 32,
300
- "quant_method": "fp8",
301
- "store_dtype": "mxfp4",
302
- "weight_block_size": [
303
- 128,
304
- 128
305
- ]
306
- }
307
- },
308
- "rope_parameters": {
309
  "partial_rotary_factor": 0.334,
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
310
  "rope_theta": 10000000.0,
311
- "rope_type": "default",
312
- "type": "default"
313
- },
314
- "rope_theta": 10000000.0,
315
- "routed_scaling_factor": null,
316
- "scoring_func": "sigmoid",
317
- "sliding_window": 128,
318
- "sliding_window_size": 128,
319
- "swa_head_dim": 192,
320
- "swa_num_attention_heads": 64,
321
- "swa_num_key_value_heads": 8,
322
- "swa_rope_theta": 10000.0,
323
- "swa_v_head_dim": 128,
324
- "tie_word_embeddings": false,
325
- "topk_group": 1,
326
- "topk_method": "noaux_tc",
327
- "transformers_version": "5.3.0",
328
- "use_cache": true,
329
- "v_head_dim": 128,
330
- "video_token_id": 151656,
331
- "vision_config": {
332
- "depth": 28,
333
- "fullatt_block_indexes": [
334
- 0,
335
- 9,
336
- 18,
337
- 27
338
- ],
339
- "hidden_act": "silu",
340
- "hidden_size": 1280,
341
- "in_chans": 3,
342
- "intermediate_size": 4608,
343
- "num_heads": 32,
344
- "num_key_value_heads": 8,
345
- "num_query_groups": 4,
346
- "out_hidden_size": 4096,
347
- "patch_size": 16,
348
- "spatial_merge_size": 2,
349
- "spatial_patch_size": 16,
350
- "temporal_patch_size": 2,
351
- "tokens_per_second": 2,
352
- "use_sink": true,
353
- "visual_token_window_size": 64,
354
- "vit_window_attn_types": [
355
- -1,
356
- 0,
357
- 0,
358
- 0,
359
- 0,
360
- 1,
361
- 1,
362
- 1,
363
- 1,
364
- -1,
365
- 0,
366
- 0,
367
- 0,
368
- 0,
369
- 1,
370
- 1,
371
- 1,
372
- 1,
373
- -1,
374
- 0,
375
- 0,
376
- 0,
377
- 0,
378
- 1,
379
- 1,
380
- 1,
381
- 1,
382
- -1
383
- ],
384
- "window_size": 128
385
- },
386
- "vision_end_token_id": 151653,
387
- "vision_model_type": "mimovl",
388
- "vision_start_token_id": 151652,
389
- "vocab_size": 152576,
390
- "exl3_fp32_mlp_layers": "47"
391
  }
 
1
  {
2
+ "add_full_attention_sink_bias": false,
3
+ "add_swa_attention_sink_bias": true,
4
+ "architectures": [
5
+ "MiMoV2ForCausalLM"
6
+ ],
7
+ "attention_bias": false,
8
+ "attention_chunk_size": 128,
9
+ "attention_dropout": 0.0,
10
+ "attention_projection_layout": "fused_qkv",
11
+ "attention_value_scale": 0.707,
12
+ "moe_router_dtype": "bfloat16",
13
+ "audio_config": {
14
+ "add_post_norm": true,
15
+ "audio_channels": 20,
16
+ "audio_segment_size": 6000,
17
+ "group_size": 4,
18
+ "input_full_attention": true,
19
+ "input_local_attn_heads": 16,
20
+ "input_local_dim": 1024,
21
+ "input_local_head_dim": 64,
22
+ "input_local_hidden_dropout": 0.0,
23
+ "input_local_intermediate_size": 4096,
24
+ "input_local_layers": 6,
25
+ "out_hidden_size": 4096,
26
+ "partial_rotary_factor": 1.0,
27
+ "projection_layers": 2,
28
+ "rope_theta": 640000,
29
+ "speech_vocab_size": "1280",
30
+ "speech_zeroemb_idx": "1024"
31
+ },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
  "audio_end_token_id": 151674,
 
 
 
 
 
 
 
 
 
 
33
  "audio_start_token_id": 151673,
 
34
  "audio_token_id": 151669,
35
+ "auto_map": {
36
+ "AutoConfig": "configuration_mimo_v2.MiMoV2Config",
37
+ "AutoModel": "modeling_mimo_v2.MiMoV2Model",
38
+ "AutoModelForCausalLM": "modeling_mimo_v2.MiMoV2ForCausalLM"
39
+ },
40
+ "bos_token_id": null,
41
+ "dtype": "bfloat16",
42
+ "eos_token_id": 151645,
43
+ "head_dim": 192,
44
+ "hidden_act": "silu",
45
+ "hidden_size": 4096,
46
+ "hybrid_block_size": null,
47
+ "hybrid_layer_pattern": [
48
+ 0,
49
+ 1,
50
+ 1,
51
+ 1,
52
+ 1,
53
+ 0,
54
+ 1,
55
+ 1,
56
+ 1,
57
+ 1,
58
+ 1,
59
+ 0,
60
+ 1,
61
+ 1,
62
+ 1,
63
+ 1,
64
+ 1,
65
+ 0,
66
+ 1,
67
+ 1,
68
+ 1,
69
+ 1,
70
+ 1,
71
+ 0,
72
+ 1,
73
+ 1,
74
+ 1,
75
+ 1,
76
+ 1,
77
+ 0,
78
+ 1,
79
+ 1,
80
+ 1,
81
+ 1,
82
+ 1,
83
+ 0,
84
+ 1,
85
+ 1,
86
+ 1,
87
+ 1,
88
+ 1,
89
+ 0,
90
+ 1,
91
+ 1,
92
+ 1,
93
+ 1,
94
+ 1,
95
+ 0
96
  ],
 
 
 
97
  "image_token_id": 151655,
98
+ "initializer_range": 0.02,
99
+ "intermediate_size": 16384,
100
+ "layernorm_epsilon": 1e-06,
101
+ "max_position_embeddings": 1048576,
102
+ "model_type": "mimo_v2",
103
+ "moe_intermediate_size": 2048,
104
+ "moe_layer_freq": [
105
+ 0,
106
+ 1,
107
+ 1,
108
+ 1,
109
+ 1,
110
+ 1,
111
+ 1,
112
+ 1,
113
+ 1,
114
+ 1,
115
+ 1,
116
+ 1,
117
+ 1,
118
+ 1,
119
+ 1,
120
+ 1,
121
+ 1,
122
+ 1,
123
+ 1,
124
+ 1,
125
+ 1,
126
+ 1,
127
+ 1,
128
+ 1,
129
+ 1,
130
+ 1,
131
+ 1,
132
+ 1,
133
+ 1,
134
+ 1,
135
+ 1,
136
+ 1,
137
+ 1,
138
+ 1,
139
+ 1,
140
+ 1,
141
+ 1,
142
+ 1,
143
+ 1,
144
+ 1,
145
+ 1,
146
+ 1,
147
+ 1,
148
+ 1,
149
+ 1,
150
+ 1,
151
+ 1,
152
+ 1
153
+ ],
154
+ "n_group": 1,
155
+ "n_routed_experts": 256,
156
+ "n_shared_experts": null,
157
+ "norm_topk_prob": true,
158
+ "num_attention_heads": 64,
159
+ "num_experts_per_tok": 8,
160
+ "num_hidden_layers": 48,
161
+ "num_key_value_heads": 4,
162
+ "num_nextn_predict_layers": 3,
163
  "pad_token_id": 151643,
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
164
  "partial_rotary_factor": 0.334,
165
+ "processor_config": {
166
+ "audio_avg_pooler": 2,
167
+ "audio_channels": 20,
168
+ "audio_end_token_id": 151674,
169
+ "audio_fmax": null,
170
+ "audio_fmin": 0,
171
+ "audio_group_size": 4,
172
+ "audio_hop_length": 240,
173
+ "audio_input_id_per_second": 25.0,
174
+ "audio_kernel_size": 3,
175
+ "audio_n_mels": 128,
176
+ "audio_nfft": 960,
177
+ "audio_sampling_rate": 24000,
178
+ "audio_segment_size": 6000,
179
+ "audio_start_token_id": 151673,
180
+ "audio_stride_size": 2,
181
+ "audio_token_id": 151669,
182
+ "audio_window_size": 960,
183
+ "audio_zeroemb_idx": [
184
+ 1024,
185
+ 1024,
186
+ 1024,
187
+ 1024,
188
+ 1024,
189
+ 1024,
190
+ 1024,
191
+ 1024,
192
+ 1024,
193
+ 1024,
194
+ 1024,
195
+ 1024,
196
+ 1024,
197
+ 1024,
198
+ 1024,
199
+ 1024,
200
+ 1024,
201
+ 1024,
202
+ 1024,
203
+ 1024
204
+ ],
205
+ "fps": 1.0,
206
+ "image_max_pixels": 8388608,
207
+ "image_min_pixels": 8192,
208
+ "image_token_id": 151655,
209
+ "max_frames": 3600,
210
+ "merge_size": 2,
211
+ "min_frames": null,
212
+ "num_frames": null,
213
+ "pad_token_id": 151643,
214
+ "patch_size": 16,
215
+ "rope_type": "rope",
216
+ "temporal_compression_ratio": 1,
217
+ "temporal_patch_size": 2,
218
+ "use_per_grid_t_timestamps": false,
219
+ "use_video_timestamps": true,
220
+ "video_audio_interleave_length": 0.0,
221
+ "video_end_token_id": 151671,
222
+ "video_max_pixels": 8388608,
223
+ "video_min_pixels": 8192,
224
+ "video_process_num_threads": 16,
225
+ "video_start_token_id": 151670,
226
+ "video_token_id": 151656,
227
+ "video_tokens_per_second": 2,
228
+ "video_total_max_pixels": 268435456,
229
+ "vision_end_token_id": 151653,
230
+ "vision_start_token_id": 151652
231
+ },
232
+ "quantization_config": {
233
+ "quant_method": "exl3",
234
+ "version": "1.5.1",
235
+ "bits": 2.27,
236
+ "head_bits": 6,
237
+ "calibration": {
238
+ "rows": 250,
239
+ "cols": 2048
240
+ },
241
+ "out_scales": "always",
242
+ "codebook": "mul1",
243
+ "original_quantization_config": {
244
+ "activation_scheme": "dynamic",
245
+ "fmt": "e4m3",
246
+ "ignored_layers": [
247
+ "model.layers.0.self_attn.o_proj",
248
+ "model.layers.1.self_attn.o_proj",
249
+ "model.layers.10.self_attn.o_proj",
250
+ "model.layers.11.self_attn.o_proj",
251
+ "model.layers.12.self_attn.o_proj",
252
+ "model.layers.13.self_attn.o_proj",
253
+ "model.layers.14.self_attn.o_proj",
254
+ "model.layers.15.self_attn.o_proj",
255
+ "model.layers.16.self_attn.o_proj",
256
+ "model.layers.17.self_attn.o_proj",
257
+ "model.layers.18.self_attn.o_proj",
258
+ "model.layers.19.self_attn.o_proj",
259
+ "model.layers.2.self_attn.o_proj",
260
+ "model.layers.20.self_attn.o_proj",
261
+ "model.layers.21.self_attn.o_proj",
262
+ "model.layers.22.self_attn.o_proj",
263
+ "model.layers.23.self_attn.o_proj",
264
+ "model.layers.24.self_attn.o_proj",
265
+ "model.layers.25.self_attn.o_proj",
266
+ "model.layers.26.self_attn.o_proj",
267
+ "model.layers.27.self_attn.o_proj",
268
+ "model.layers.28.self_attn.o_proj",
269
+ "model.layers.29.self_attn.o_proj",
270
+ "model.layers.3.self_attn.o_proj",
271
+ "model.layers.30.self_attn.o_proj",
272
+ "model.layers.31.self_attn.o_proj",
273
+ "model.layers.32.self_attn.o_proj",
274
+ "model.layers.33.self_attn.o_proj",
275
+ "model.layers.34.self_attn.o_proj",
276
+ "model.layers.35.self_attn.o_proj",
277
+ "model.layers.36.self_attn.o_proj",
278
+ "model.layers.37.self_attn.o_proj",
279
+ "model.layers.38.self_attn.o_proj",
280
+ "model.layers.39.self_attn.o_proj",
281
+ "model.layers.4.self_attn.o_proj",
282
+ "model.layers.40.self_attn.o_proj",
283
+ "model.layers.41.self_attn.o_proj",
284
+ "model.layers.42.self_attn.o_proj",
285
+ "model.layers.43.self_attn.o_proj",
286
+ "model.layers.44.self_attn.o_proj",
287
+ "model.layers.45.self_attn.o_proj",
288
+ "model.layers.46.self_attn.o_proj",
289
+ "model.layers.47.self_attn.o_proj",
290
+ "model.layers.5.self_attn.o_proj",
291
+ "model.layers.6.self_attn.o_proj",
292
+ "model.layers.7.self_attn.o_proj",
293
+ "model.layers.8.self_attn.o_proj",
294
+ "model.layers.9.self_attn.o_proj",
295
+ "model.decoder.self_attn.o_proj"
296
+ ],
297
+ "mxfp4_block_size": 32,
298
+ "quant_method": "fp8",
299
+ "store_dtype": "mxfp4",
300
+ "weight_block_size": [
301
+ 128,
302
+ 128
303
+ ]
304
+ }
305
+ },
306
+ "rope_parameters": {
307
+ "partial_rotary_factor": 0.334,
308
+ "rope_theta": 10000000.0,
309
+ "rope_type": "default",
310
+ "type": "default"
311
+ },
312
  "rope_theta": 10000000.0,
313
+ "routed_scaling_factor": null,
314
+ "scoring_func": "sigmoid",
315
+ "sliding_window": 128,
316
+ "sliding_window_size": 128,
317
+ "swa_head_dim": 192,
318
+ "swa_num_attention_heads": 64,
319
+ "swa_num_key_value_heads": 8,
320
+ "swa_rope_theta": 10000.0,
321
+ "swa_v_head_dim": 128,
322
+ "tie_word_embeddings": false,
323
+ "topk_group": 1,
324
+ "topk_method": "noaux_tc",
325
+ "transformers_version": "5.3.0",
326
+ "use_cache": true,
327
+ "v_head_dim": 128,
328
+ "video_token_id": 151656,
329
+ "vision_config": {
330
+ "depth": 28,
331
+ "fullatt_block_indexes": [
332
+ 0,
333
+ 9,
334
+ 18,
335
+ 27
336
+ ],
337
+ "hidden_act": "silu",
338
+ "hidden_size": 1280,
339
+ "in_chans": 3,
340
+ "intermediate_size": 4608,
341
+ "num_heads": 32,
342
+ "num_key_value_heads": 8,
343
+ "num_query_groups": 4,
344
+ "out_hidden_size": 4096,
345
+ "patch_size": 16,
346
+ "spatial_merge_size": 2,
347
+ "spatial_patch_size": 16,
348
+ "temporal_patch_size": 2,
349
+ "tokens_per_second": 2,
350
+ "use_sink": true,
351
+ "visual_token_window_size": 64,
352
+ "vit_window_attn_types": [
353
+ -1,
354
+ 0,
355
+ 0,
356
+ 0,
357
+ 0,
358
+ 1,
359
+ 1,
360
+ 1,
361
+ 1,
362
+ -1,
363
+ 0,
364
+ 0,
365
+ 0,
366
+ 0,
367
+ 1,
368
+ 1,
369
+ 1,
370
+ 1,
371
+ -1,
372
+ 0,
373
+ 0,
374
+ 0,
375
+ 0,
376
+ 1,
377
+ 1,
378
+ 1,
379
+ 1,
380
+ -1
381
+ ],
382
+ "window_size": 128
383
+ },
384
+ "vision_end_token_id": 151653,
385
+ "vision_model_type": "mimovl",
386
+ "vision_start_token_id": 151652,
387
+ "vocab_size": 152576
 
 
 
 
 
388
  }
eval/eval-overflow-idiv47-cal250.json ADDED
@@ -0,0 +1,2535 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "on": {
3
+ "mode": "on",
4
+ "t_load": 20.72385597229004,
5
+ "fp32_mlp_layers": [],
6
+ "fp32_mlp_spec": "None",
7
+ "env": {
8
+ "EXL3_MIMO_FP32_MLP_LAYERS": null,
9
+ "EXL3_MIMO_FP32_MLP_FUSED": null,
10
+ "EXL3_MIMO_MLP_ACT_LIMIT": null
11
+ },
12
+ "rows": 250,
13
+ "row_len": 2048,
14
+ "nonfinite_rows": 0,
15
+ "nonfinite_row_idx": [],
16
+ "mlp_nonfinite_rows": 0,
17
+ "mlp_nonfinite_row_idx": [],
18
+ "nonfinite_positions": 0,
19
+ "nonfinite_logit_values": 0,
20
+ "scored_tokens": 511750,
21
+ "excluded_tokens": 0,
22
+ "perplexity": 26.293274015792484,
23
+ "prefill_tok_s": 901.1689674764066,
24
+ "per_row": [
25
+ {
26
+ "row": 0,
27
+ "tokens": 2048,
28
+ "nonfinite_positions": 0,
29
+ "nonfinite_logits": 0,
30
+ "s": 3.0844359397888184,
31
+ "layers": {},
32
+ "avail_gib": 28.19278335571289
33
+ },
34
+ {
35
+ "row": 1,
36
+ "tokens": 2048,
37
+ "nonfinite_positions": 0,
38
+ "nonfinite_logits": 0,
39
+ "s": 2.233196496963501,
40
+ "layers": {},
41
+ "avail_gib": 28.192733764648438
42
+ },
43
+ {
44
+ "row": 2,
45
+ "tokens": 2048,
46
+ "nonfinite_positions": 0,
47
+ "nonfinite_logits": 0,
48
+ "s": 2.2321951389312744,
49
+ "layers": {},
50
+ "avail_gib": 28.19289779663086
51
+ },
52
+ {
53
+ "row": 3,
54
+ "tokens": 2048,
55
+ "nonfinite_positions": 0,
56
+ "nonfinite_logits": 0,
57
+ "s": 2.204251527786255,
58
+ "layers": {},
59
+ "avail_gib": 28.19296646118164
60
+ },
61
+ {
62
+ "row": 4,
63
+ "tokens": 2048,
64
+ "nonfinite_positions": 0,
65
+ "nonfinite_logits": 0,
66
+ "s": 2.234518527984619,
67
+ "layers": {},
68
+ "avail_gib": 28.19296646118164
69
+ },
70
+ {
71
+ "row": 5,
72
+ "tokens": 2048,
73
+ "nonfinite_positions": 0,
74
+ "nonfinite_logits": 0,
75
+ "s": 2.217644214630127,
76
+ "layers": {},
77
+ "avail_gib": 28.19278335571289
78
+ },
79
+ {
80
+ "row": 6,
81
+ "tokens": 2048,
82
+ "nonfinite_positions": 0,
83
+ "nonfinite_logits": 0,
84
+ "s": 2.2274632453918457,
85
+ "layers": {},
86
+ "avail_gib": 28.19296646118164
87
+ },
88
+ {
89
+ "row": 7,
90
+ "tokens": 2048,
91
+ "nonfinite_positions": 0,
92
+ "nonfinite_logits": 0,
93
+ "s": 2.236445188522339,
94
+ "layers": {},
95
+ "avail_gib": 28.19243621826172
96
+ },
97
+ {
98
+ "row": 8,
99
+ "tokens": 2048,
100
+ "nonfinite_positions": 0,
101
+ "nonfinite_logits": 0,
102
+ "s": 2.2299110889434814,
103
+ "layers": {},
104
+ "avail_gib": 28.19274139404297
105
+ },
106
+ {
107
+ "row": 9,
108
+ "tokens": 2048,
109
+ "nonfinite_positions": 0,
110
+ "nonfinite_logits": 0,
111
+ "s": 2.2392120361328125,
112
+ "layers": {},
113
+ "avail_gib": 28.194011688232422
114
+ },
115
+ {
116
+ "row": 10,
117
+ "tokens": 2048,
118
+ "nonfinite_positions": 0,
119
+ "nonfinite_logits": 0,
120
+ "s": 2.253910779953003,
121
+ "layers": {},
122
+ "avail_gib": 28.19558334350586
123
+ },
124
+ {
125
+ "row": 11,
126
+ "tokens": 2048,
127
+ "nonfinite_positions": 0,
128
+ "nonfinite_logits": 0,
129
+ "s": 2.2438693046569824,
130
+ "layers": {},
131
+ "avail_gib": 28.195110321044922
132
+ },
133
+ {
134
+ "row": 12,
135
+ "tokens": 2048,
136
+ "nonfinite_positions": 0,
137
+ "nonfinite_logits": 0,
138
+ "s": 2.2569949626922607,
139
+ "layers": {},
140
+ "avail_gib": 28.1968994140625
141
+ },
142
+ {
143
+ "row": 13,
144
+ "tokens": 2048,
145
+ "nonfinite_positions": 0,
146
+ "nonfinite_logits": 0,
147
+ "s": 2.2481133937835693,
148
+ "layers": {},
149
+ "avail_gib": 28.195465087890625
150
+ },
151
+ {
152
+ "row": 14,
153
+ "tokens": 2048,
154
+ "nonfinite_positions": 0,
155
+ "nonfinite_logits": 0,
156
+ "s": 2.243696928024292,
157
+ "layers": {},
158
+ "avail_gib": 28.19818115234375
159
+ },
160
+ {
161
+ "row": 15,
162
+ "tokens": 2048,
163
+ "nonfinite_positions": 0,
164
+ "nonfinite_logits": 0,
165
+ "s": 2.2543399333953857,
166
+ "layers": {},
167
+ "avail_gib": 28.19857406616211
168
+ },
169
+ {
170
+ "row": 16,
171
+ "tokens": 2048,
172
+ "nonfinite_positions": 0,
173
+ "nonfinite_logits": 0,
174
+ "s": 2.248088836669922,
175
+ "layers": {},
176
+ "avail_gib": 28.200374603271484
177
+ },
178
+ {
179
+ "row": 17,
180
+ "tokens": 2048,
181
+ "nonfinite_positions": 0,
182
+ "nonfinite_logits": 0,
183
+ "s": 2.2410244941711426,
184
+ "layers": {},
185
+ "avail_gib": 28.198211669921875
186
+ },
187
+ {
188
+ "row": 18,
189
+ "tokens": 2048,
190
+ "nonfinite_positions": 0,
191
+ "nonfinite_logits": 0,
192
+ "s": 2.244046926498413,
193
+ "layers": {},
194
+ "avail_gib": 28.204769134521484
195
+ },
196
+ {
197
+ "row": 19,
198
+ "tokens": 2048,
199
+ "nonfinite_positions": 0,
200
+ "nonfinite_logits": 0,
201
+ "s": 2.2509396076202393,
202
+ "layers": {},
203
+ "avail_gib": 28.20783233642578
204
+ },
205
+ {
206
+ "row": 20,
207
+ "tokens": 2048,
208
+ "nonfinite_positions": 0,
209
+ "nonfinite_logits": 0,
210
+ "s": 2.2497260570526123,
211
+ "layers": {},
212
+ "avail_gib": 28.207664489746094
213
+ },
214
+ {
215
+ "row": 21,
216
+ "tokens": 2048,
217
+ "nonfinite_positions": 0,
218
+ "nonfinite_logits": 0,
219
+ "s": 2.2314488887786865,
220
+ "layers": {},
221
+ "avail_gib": 28.209964752197266
222
+ },
223
+ {
224
+ "row": 22,
225
+ "tokens": 2048,
226
+ "nonfinite_positions": 0,
227
+ "nonfinite_logits": 0,
228
+ "s": 2.243046522140503,
229
+ "layers": {},
230
+ "avail_gib": 28.21146011352539
231
+ },
232
+ {
233
+ "row": 23,
234
+ "tokens": 2048,
235
+ "nonfinite_positions": 0,
236
+ "nonfinite_logits": 0,
237
+ "s": 2.240645408630371,
238
+ "layers": {},
239
+ "avail_gib": 28.213703155517578
240
+ },
241
+ {
242
+ "row": 24,
243
+ "tokens": 2048,
244
+ "nonfinite_positions": 0,
245
+ "nonfinite_logits": 0,
246
+ "s": 2.253235101699829,
247
+ "layers": {},
248
+ "avail_gib": 28.219684600830078
249
+ },
250
+ {
251
+ "row": 25,
252
+ "tokens": 2048,
253
+ "nonfinite_positions": 0,
254
+ "nonfinite_logits": 0,
255
+ "s": 2.2478795051574707,
256
+ "layers": {},
257
+ "avail_gib": 28.216556549072266
258
+ },
259
+ {
260
+ "row": 26,
261
+ "tokens": 2048,
262
+ "nonfinite_positions": 0,
263
+ "nonfinite_logits": 0,
264
+ "s": 2.287109136581421,
265
+ "layers": {},
266
+ "avail_gib": 28.221519470214844
267
+ },
268
+ {
269
+ "row": 27,
270
+ "tokens": 2048,
271
+ "nonfinite_positions": 0,
272
+ "nonfinite_logits": 0,
273
+ "s": 2.2605996131896973,
274
+ "layers": {},
275
+ "avail_gib": 28.22014617919922
276
+ },
277
+ {
278
+ "row": 28,
279
+ "tokens": 2048,
280
+ "nonfinite_positions": 0,
281
+ "nonfinite_logits": 0,
282
+ "s": 2.249145746231079,
283
+ "layers": {},
284
+ "avail_gib": 28.22210693359375
285
+ },
286
+ {
287
+ "row": 29,
288
+ "tokens": 2048,
289
+ "nonfinite_positions": 0,
290
+ "nonfinite_logits": 0,
291
+ "s": 2.245753526687622,
292
+ "layers": {},
293
+ "avail_gib": 28.221942901611328
294
+ },
295
+ {
296
+ "row": 30,
297
+ "tokens": 2048,
298
+ "nonfinite_positions": 0,
299
+ "nonfinite_logits": 0,
300
+ "s": 2.247159719467163,
301
+ "layers": {},
302
+ "avail_gib": 28.222126007080078
303
+ },
304
+ {
305
+ "row": 31,
306
+ "tokens": 2048,
307
+ "nonfinite_positions": 0,
308
+ "nonfinite_logits": 0,
309
+ "s": 2.268373489379883,
310
+ "layers": {},
311
+ "avail_gib": 28.222339630126953
312
+ },
313
+ {
314
+ "row": 32,
315
+ "tokens": 2048,
316
+ "nonfinite_positions": 0,
317
+ "nonfinite_logits": 0,
318
+ "s": 2.239926338195801,
319
+ "layers": {},
320
+ "avail_gib": 28.222640991210938
321
+ },
322
+ {
323
+ "row": 33,
324
+ "tokens": 2048,
325
+ "nonfinite_positions": 0,
326
+ "nonfinite_logits": 0,
327
+ "s": 2.256199836730957,
328
+ "layers": {},
329
+ "avail_gib": 28.22265625
330
+ },
331
+ {
332
+ "row": 34,
333
+ "tokens": 2048,
334
+ "nonfinite_positions": 0,
335
+ "nonfinite_logits": 0,
336
+ "s": 2.248387336730957,
337
+ "layers": {},
338
+ "avail_gib": 28.22247314453125
339
+ },
340
+ {
341
+ "row": 35,
342
+ "tokens": 2048,
343
+ "nonfinite_positions": 0,
344
+ "nonfinite_logits": 0,
345
+ "s": 2.249359607696533,
346
+ "layers": {},
347
+ "avail_gib": 28.221630096435547
348
+ },
349
+ {
350
+ "row": 36,
351
+ "tokens": 2048,
352
+ "nonfinite_positions": 0,
353
+ "nonfinite_logits": 0,
354
+ "s": 2.249767303466797,
355
+ "layers": {},
356
+ "avail_gib": 28.22386932373047
357
+ },
358
+ {
359
+ "row": 37,
360
+ "tokens": 2048,
361
+ "nonfinite_positions": 0,
362
+ "nonfinite_logits": 0,
363
+ "s": 2.2543318271636963,
364
+ "layers": {},
365
+ "avail_gib": 28.223682403564453
366
+ },
367
+ {
368
+ "row": 38,
369
+ "tokens": 2048,
370
+ "nonfinite_positions": 0,
371
+ "nonfinite_logits": 0,
372
+ "s": 2.261387586593628,
373
+ "layers": {},
374
+ "avail_gib": 28.22364044189453
375
+ },
376
+ {
377
+ "row": 39,
378
+ "tokens": 2048,
379
+ "nonfinite_positions": 0,
380
+ "nonfinite_logits": 0,
381
+ "s": 2.256837844848633,
382
+ "layers": {},
383
+ "avail_gib": 28.224308013916016
384
+ },
385
+ {
386
+ "row": 40,
387
+ "tokens": 2048,
388
+ "nonfinite_positions": 0,
389
+ "nonfinite_logits": 0,
390
+ "s": 2.2757441997528076,
391
+ "layers": {},
392
+ "avail_gib": 28.224246978759766
393
+ },
394
+ {
395
+ "row": 41,
396
+ "tokens": 2048,
397
+ "nonfinite_positions": 0,
398
+ "nonfinite_logits": 0,
399
+ "s": 2.2728872299194336,
400
+ "layers": {},
401
+ "avail_gib": 28.224124908447266
402
+ },
403
+ {
404
+ "row": 42,
405
+ "tokens": 2048,
406
+ "nonfinite_positions": 0,
407
+ "nonfinite_logits": 0,
408
+ "s": 2.2519993782043457,
409
+ "layers": {},
410
+ "avail_gib": 28.224132537841797
411
+ },
412
+ {
413
+ "row": 43,
414
+ "tokens": 2048,
415
+ "nonfinite_positions": 0,
416
+ "nonfinite_logits": 0,
417
+ "s": 2.271083116531372,
418
+ "layers": {},
419
+ "avail_gib": 28.229759216308594
420
+ },
421
+ {
422
+ "row": 44,
423
+ "tokens": 2048,
424
+ "nonfinite_positions": 0,
425
+ "nonfinite_logits": 0,
426
+ "s": 2.279876947402954,
427
+ "layers": {},
428
+ "avail_gib": 28.229965209960938
429
+ },
430
+ {
431
+ "row": 45,
432
+ "tokens": 2048,
433
+ "nonfinite_positions": 0,
434
+ "nonfinite_logits": 0,
435
+ "s": 2.2856831550598145,
436
+ "layers": {},
437
+ "avail_gib": 28.229846954345703
438
+ },
439
+ {
440
+ "row": 46,
441
+ "tokens": 2048,
442
+ "nonfinite_positions": 0,
443
+ "nonfinite_logits": 0,
444
+ "s": 2.276364803314209,
445
+ "layers": {},
446
+ "avail_gib": 28.230087280273438
447
+ },
448
+ {
449
+ "row": 47,
450
+ "tokens": 2048,
451
+ "nonfinite_positions": 0,
452
+ "nonfinite_logits": 0,
453
+ "s": 2.283644437789917,
454
+ "layers": {},
455
+ "avail_gib": 28.230281829833984
456
+ },
457
+ {
458
+ "row": 48,
459
+ "tokens": 2048,
460
+ "nonfinite_positions": 0,
461
+ "nonfinite_logits": 0,
462
+ "s": 2.289663076400757,
463
+ "layers": {},
464
+ "avail_gib": 28.226722717285156
465
+ },
466
+ {
467
+ "row": 49,
468
+ "tokens": 2048,
469
+ "nonfinite_positions": 0,
470
+ "nonfinite_logits": 0,
471
+ "s": 2.28182315826416,
472
+ "layers": {},
473
+ "avail_gib": 28.231056213378906
474
+ },
475
+ {
476
+ "row": 50,
477
+ "tokens": 2048,
478
+ "nonfinite_positions": 0,
479
+ "nonfinite_logits": 0,
480
+ "s": 2.2848188877105713,
481
+ "layers": {},
482
+ "avail_gib": 28.236217498779297
483
+ },
484
+ {
485
+ "row": 51,
486
+ "tokens": 2048,
487
+ "nonfinite_positions": 0,
488
+ "nonfinite_logits": 0,
489
+ "s": 2.267463445663452,
490
+ "layers": {},
491
+ "avail_gib": 28.235912322998047
492
+ },
493
+ {
494
+ "row": 52,
495
+ "tokens": 2048,
496
+ "nonfinite_positions": 0,
497
+ "nonfinite_logits": 0,
498
+ "s": 2.2938485145568848,
499
+ "layers": {},
500
+ "avail_gib": 28.235855102539062
501
+ },
502
+ {
503
+ "row": 53,
504
+ "tokens": 2048,
505
+ "nonfinite_positions": 0,
506
+ "nonfinite_logits": 0,
507
+ "s": 2.3179948329925537,
508
+ "layers": {},
509
+ "avail_gib": 28.235671997070312
510
+ },
511
+ {
512
+ "row": 54,
513
+ "tokens": 2048,
514
+ "nonfinite_positions": 0,
515
+ "nonfinite_logits": 0,
516
+ "s": 2.2737224102020264,
517
+ "layers": {},
518
+ "avail_gib": 28.23513412475586
519
+ },
520
+ {
521
+ "row": 55,
522
+ "tokens": 2048,
523
+ "nonfinite_positions": 0,
524
+ "nonfinite_logits": 0,
525
+ "s": 2.2663414478302,
526
+ "layers": {},
527
+ "avail_gib": 28.231521606445312
528
+ },
529
+ {
530
+ "row": 56,
531
+ "tokens": 2048,
532
+ "nonfinite_positions": 0,
533
+ "nonfinite_logits": 0,
534
+ "s": 2.2442331314086914,
535
+ "layers": {},
536
+ "avail_gib": 28.240036010742188
537
+ },
538
+ {
539
+ "row": 57,
540
+ "tokens": 2048,
541
+ "nonfinite_positions": 0,
542
+ "nonfinite_logits": 0,
543
+ "s": 2.260502576828003,
544
+ "layers": {},
545
+ "avail_gib": 28.23986053466797
546
+ },
547
+ {
548
+ "row": 58,
549
+ "tokens": 2048,
550
+ "nonfinite_positions": 0,
551
+ "nonfinite_logits": 0,
552
+ "s": 2.295023202896118,
553
+ "layers": {},
554
+ "avail_gib": 28.23967742919922
555
+ },
556
+ {
557
+ "row": 59,
558
+ "tokens": 2048,
559
+ "nonfinite_positions": 0,
560
+ "nonfinite_logits": 0,
561
+ "s": 2.2908742427825928,
562
+ "layers": {},
563
+ "avail_gib": 28.23957061767578
564
+ },
565
+ {
566
+ "row": 60,
567
+ "tokens": 2048,
568
+ "nonfinite_positions": 0,
569
+ "nonfinite_logits": 0,
570
+ "s": 2.2904586791992188,
571
+ "layers": {},
572
+ "avail_gib": 28.239608764648438
573
+ },
574
+ {
575
+ "row": 61,
576
+ "tokens": 2048,
577
+ "nonfinite_positions": 0,
578
+ "nonfinite_logits": 0,
579
+ "s": 2.2885043621063232,
580
+ "layers": {},
581
+ "avail_gib": 28.237449645996094
582
+ },
583
+ {
584
+ "row": 62,
585
+ "tokens": 2048,
586
+ "nonfinite_positions": 0,
587
+ "nonfinite_logits": 0,
588
+ "s": 2.2996487617492676,
589
+ "layers": {},
590
+ "avail_gib": 28.237449645996094
591
+ },
592
+ {
593
+ "row": 63,
594
+ "tokens": 2048,
595
+ "nonfinite_positions": 0,
596
+ "nonfinite_logits": 0,
597
+ "s": 2.2884092330932617,
598
+ "layers": {},
599
+ "avail_gib": 28.235179901123047
600
+ },
601
+ {
602
+ "row": 64,
603
+ "tokens": 2048,
604
+ "nonfinite_positions": 0,
605
+ "nonfinite_logits": 0,
606
+ "s": 2.297288656234741,
607
+ "layers": {},
608
+ "avail_gib": 28.235313415527344
609
+ },
610
+ {
611
+ "row": 65,
612
+ "tokens": 2048,
613
+ "nonfinite_positions": 0,
614
+ "nonfinite_logits": 0,
615
+ "s": 2.2047998905181885,
616
+ "layers": {},
617
+ "avail_gib": 28.23477554321289
618
+ },
619
+ {
620
+ "row": 66,
621
+ "tokens": 2048,
622
+ "nonfinite_positions": 0,
623
+ "nonfinite_logits": 0,
624
+ "s": 2.2022502422332764,
625
+ "layers": {},
626
+ "avail_gib": 28.23447036743164
627
+ },
628
+ {
629
+ "row": 67,
630
+ "tokens": 2048,
631
+ "nonfinite_positions": 0,
632
+ "nonfinite_logits": 0,
633
+ "s": 2.1980185508728027,
634
+ "layers": {},
635
+ "avail_gib": 28.232311248779297
636
+ },
637
+ {
638
+ "row": 68,
639
+ "tokens": 2048,
640
+ "nonfinite_positions": 0,
641
+ "nonfinite_logits": 0,
642
+ "s": 2.2770371437072754,
643
+ "layers": {},
644
+ "avail_gib": 28.232322692871094
645
+ },
646
+ {
647
+ "row": 69,
648
+ "tokens": 2048,
649
+ "nonfinite_positions": 0,
650
+ "nonfinite_logits": 0,
651
+ "s": 2.29390025138855,
652
+ "layers": {},
653
+ "avail_gib": 28.232322692871094
654
+ },
655
+ {
656
+ "row": 70,
657
+ "tokens": 2048,
658
+ "nonfinite_positions": 0,
659
+ "nonfinite_logits": 0,
660
+ "s": 2.296107292175293,
661
+ "layers": {},
662
+ "avail_gib": 28.232627868652344
663
+ },
664
+ {
665
+ "row": 71,
666
+ "tokens": 2048,
667
+ "nonfinite_positions": 0,
668
+ "nonfinite_logits": 0,
669
+ "s": 2.2780604362487793,
670
+ "layers": {},
671
+ "avail_gib": 28.232444763183594
672
+ },
673
+ {
674
+ "row": 72,
675
+ "tokens": 2048,
676
+ "nonfinite_positions": 0,
677
+ "nonfinite_logits": 0,
678
+ "s": 2.265984535217285,
679
+ "layers": {},
680
+ "avail_gib": 28.232322692871094
681
+ },
682
+ {
683
+ "row": 73,
684
+ "tokens": 2048,
685
+ "nonfinite_positions": 0,
686
+ "nonfinite_logits": 0,
687
+ "s": 2.2599167823791504,
688
+ "layers": {},
689
+ "avail_gib": 28.2313232421875
690
+ },
691
+ {
692
+ "row": 74,
693
+ "tokens": 2048,
694
+ "nonfinite_positions": 0,
695
+ "nonfinite_logits": 0,
696
+ "s": 2.2748374938964844,
697
+ "layers": {},
698
+ "avail_gib": 28.231433868408203
699
+ },
700
+ {
701
+ "row": 75,
702
+ "tokens": 2048,
703
+ "nonfinite_positions": 0,
704
+ "nonfinite_logits": 0,
705
+ "s": 2.258882761001587,
706
+ "layers": {},
707
+ "avail_gib": 28.231311798095703
708
+ },
709
+ {
710
+ "row": 76,
711
+ "tokens": 2048,
712
+ "nonfinite_positions": 0,
713
+ "nonfinite_logits": 0,
714
+ "s": 2.28346586227417,
715
+ "layers": {},
716
+ "avail_gib": 28.231552124023438
717
+ },
718
+ {
719
+ "row": 77,
720
+ "tokens": 2048,
721
+ "nonfinite_positions": 0,
722
+ "nonfinite_logits": 0,
723
+ "s": 2.2711660861968994,
724
+ "layers": {},
725
+ "avail_gib": 28.2315673828125
726
+ },
727
+ {
728
+ "row": 78,
729
+ "tokens": 2048,
730
+ "nonfinite_positions": 0,
731
+ "nonfinite_logits": 0,
732
+ "s": 2.275212526321411,
733
+ "layers": {},
734
+ "avail_gib": 28.230728149414062
735
+ },
736
+ {
737
+ "row": 79,
738
+ "tokens": 2048,
739
+ "nonfinite_positions": 0,
740
+ "nonfinite_logits": 0,
741
+ "s": 2.2859952449798584,
742
+ "layers": {},
743
+ "avail_gib": 28.22824478149414
744
+ },
745
+ {
746
+ "row": 80,
747
+ "tokens": 2048,
748
+ "nonfinite_positions": 0,
749
+ "nonfinite_logits": 0,
750
+ "s": 2.2918407917022705,
751
+ "layers": {},
752
+ "avail_gib": 28.228557586669922
753
+ },
754
+ {
755
+ "row": 81,
756
+ "tokens": 2048,
757
+ "nonfinite_positions": 0,
758
+ "nonfinite_logits": 0,
759
+ "s": 2.2759346961975098,
760
+ "layers": {},
761
+ "avail_gib": 28.228256225585938
762
+ },
763
+ {
764
+ "row": 82,
765
+ "tokens": 2048,
766
+ "nonfinite_positions": 0,
767
+ "nonfinite_logits": 0,
768
+ "s": 2.2963359355926514,
769
+ "layers": {},
770
+ "avail_gib": 28.228572845458984
771
+ },
772
+ {
773
+ "row": 83,
774
+ "tokens": 2048,
775
+ "nonfinite_positions": 0,
776
+ "nonfinite_logits": 0,
777
+ "s": 2.2598962783813477,
778
+ "layers": {},
779
+ "avail_gib": 28.22856903076172
780
+ },
781
+ {
782
+ "row": 84,
783
+ "tokens": 2048,
784
+ "nonfinite_positions": 0,
785
+ "nonfinite_logits": 0,
786
+ "s": 2.281230926513672,
787
+ "layers": {},
788
+ "avail_gib": 28.22826385498047
789
+ },
790
+ {
791
+ "row": 85,
792
+ "tokens": 2048,
793
+ "nonfinite_positions": 0,
794
+ "nonfinite_logits": 0,
795
+ "s": 2.2934606075286865,
796
+ "layers": {},
797
+ "avail_gib": 28.22856903076172
798
+ },
799
+ {
800
+ "row": 86,
801
+ "tokens": 2048,
802
+ "nonfinite_positions": 0,
803
+ "nonfinite_logits": 0,
804
+ "s": 2.2801008224487305,
805
+ "layers": {},
806
+ "avail_gib": 28.229084014892578
807
+ },
808
+ {
809
+ "row": 87,
810
+ "tokens": 2048,
811
+ "nonfinite_positions": 0,
812
+ "nonfinite_logits": 0,
813
+ "s": 2.275428056716919,
814
+ "layers": {},
815
+ "avail_gib": 28.229324340820312
816
+ },
817
+ {
818
+ "row": 88,
819
+ "tokens": 2048,
820
+ "nonfinite_positions": 0,
821
+ "nonfinite_logits": 0,
822
+ "s": 2.278841018676758,
823
+ "layers": {},
824
+ "avail_gib": 28.229568481445312
825
+ },
826
+ {
827
+ "row": 89,
828
+ "tokens": 2048,
829
+ "nonfinite_positions": 0,
830
+ "nonfinite_logits": 0,
831
+ "s": 2.285351276397705,
832
+ "layers": {},
833
+ "avail_gib": 28.229568481445312
834
+ },
835
+ {
836
+ "row": 90,
837
+ "tokens": 2048,
838
+ "nonfinite_positions": 0,
839
+ "nonfinite_logits": 0,
840
+ "s": 2.2924468517303467,
841
+ "layers": {},
842
+ "avail_gib": 28.229576110839844
843
+ },
844
+ {
845
+ "row": 91,
846
+ "tokens": 2048,
847
+ "nonfinite_positions": 0,
848
+ "nonfinite_logits": 0,
849
+ "s": 2.2807817459106445,
850
+ "layers": {},
851
+ "avail_gib": 28.229576110839844
852
+ },
853
+ {
854
+ "row": 92,
855
+ "tokens": 2048,
856
+ "nonfinite_positions": 0,
857
+ "nonfinite_logits": 0,
858
+ "s": 2.27436900138855,
859
+ "layers": {},
860
+ "avail_gib": 28.229576110839844
861
+ },
862
+ {
863
+ "row": 93,
864
+ "tokens": 2048,
865
+ "nonfinite_positions": 0,
866
+ "nonfinite_logits": 0,
867
+ "s": 2.2741737365722656,
868
+ "layers": {},
869
+ "avail_gib": 28.229705810546875
870
+ },
871
+ {
872
+ "row": 94,
873
+ "tokens": 2048,
874
+ "nonfinite_positions": 0,
875
+ "nonfinite_logits": 0,
876
+ "s": 2.2697737216949463,
877
+ "layers": {},
878
+ "avail_gib": 28.22995376586914
879
+ },
880
+ {
881
+ "row": 95,
882
+ "tokens": 2048,
883
+ "nonfinite_positions": 0,
884
+ "nonfinite_logits": 0,
885
+ "s": 2.272388458251953,
886
+ "layers": {},
887
+ "avail_gib": 28.228992462158203
888
+ },
889
+ {
890
+ "row": 96,
891
+ "tokens": 2048,
892
+ "nonfinite_positions": 0,
893
+ "nonfinite_logits": 0,
894
+ "s": 2.2541537284851074,
895
+ "layers": {},
896
+ "avail_gib": 28.228992462158203
897
+ },
898
+ {
899
+ "row": 97,
900
+ "tokens": 2048,
901
+ "nonfinite_positions": 0,
902
+ "nonfinite_logits": 0,
903
+ "s": 2.2744784355163574,
904
+ "layers": {},
905
+ "avail_gib": 28.229000091552734
906
+ },
907
+ {
908
+ "row": 98,
909
+ "tokens": 2048,
910
+ "nonfinite_positions": 0,
911
+ "nonfinite_logits": 0,
912
+ "s": 2.2808258533477783,
913
+ "layers": {},
914
+ "avail_gib": 28.2286376953125
915
+ },
916
+ {
917
+ "row": 99,
918
+ "tokens": 2048,
919
+ "nonfinite_positions": 0,
920
+ "nonfinite_logits": 0,
921
+ "s": 2.274012565612793,
922
+ "layers": {},
923
+ "avail_gib": 28.22827911376953
924
+ },
925
+ {
926
+ "row": 100,
927
+ "tokens": 2048,
928
+ "nonfinite_positions": 0,
929
+ "nonfinite_logits": 0,
930
+ "s": 2.2757771015167236,
931
+ "layers": {},
932
+ "avail_gib": 28.22827911376953
933
+ },
934
+ {
935
+ "row": 101,
936
+ "tokens": 2048,
937
+ "nonfinite_positions": 0,
938
+ "nonfinite_logits": 0,
939
+ "s": 2.265073299407959,
940
+ "layers": {},
941
+ "avail_gib": 28.227615356445312
942
+ },
943
+ {
944
+ "row": 102,
945
+ "tokens": 2048,
946
+ "nonfinite_positions": 0,
947
+ "nonfinite_logits": 0,
948
+ "s": 2.2816243171691895,
949
+ "layers": {},
950
+ "avail_gib": 28.22888946533203
951
+ },
952
+ {
953
+ "row": 103,
954
+ "tokens": 2048,
955
+ "nonfinite_positions": 0,
956
+ "nonfinite_logits": 0,
957
+ "s": 2.2655951976776123,
958
+ "layers": {},
959
+ "avail_gib": 28.23514175415039
960
+ },
961
+ {
962
+ "row": 104,
963
+ "tokens": 2048,
964
+ "nonfinite_positions": 0,
965
+ "nonfinite_logits": 0,
966
+ "s": 2.270556926727295,
967
+ "layers": {},
968
+ "avail_gib": 28.234691619873047
969
+ },
970
+ {
971
+ "row": 105,
972
+ "tokens": 2048,
973
+ "nonfinite_positions": 0,
974
+ "nonfinite_logits": 0,
975
+ "s": 2.275966167449951,
976
+ "layers": {},
977
+ "avail_gib": 28.233909606933594
978
+ },
979
+ {
980
+ "row": 106,
981
+ "tokens": 2048,
982
+ "nonfinite_positions": 0,
983
+ "nonfinite_logits": 0,
984
+ "s": 2.2740859985351562,
985
+ "layers": {},
986
+ "avail_gib": 28.234050750732422
987
+ },
988
+ {
989
+ "row": 107,
990
+ "tokens": 2048,
991
+ "nonfinite_positions": 0,
992
+ "nonfinite_logits": 0,
993
+ "s": 2.265089273452759,
994
+ "layers": {},
995
+ "avail_gib": 28.23448944091797
996
+ },
997
+ {
998
+ "row": 108,
999
+ "tokens": 2048,
1000
+ "nonfinite_positions": 0,
1001
+ "nonfinite_logits": 0,
1002
+ "s": 2.2689449787139893,
1003
+ "layers": {},
1004
+ "avail_gib": 28.234695434570312
1005
+ },
1006
+ {
1007
+ "row": 109,
1008
+ "tokens": 2048,
1009
+ "nonfinite_positions": 0,
1010
+ "nonfinite_logits": 0,
1011
+ "s": 2.2728350162506104,
1012
+ "layers": {},
1013
+ "avail_gib": 28.234573364257812
1014
+ },
1015
+ {
1016
+ "row": 110,
1017
+ "tokens": 2048,
1018
+ "nonfinite_positions": 0,
1019
+ "nonfinite_logits": 0,
1020
+ "s": 2.2876834869384766,
1021
+ "layers": {},
1022
+ "avail_gib": 28.234699249267578
1023
+ },
1024
+ {
1025
+ "row": 111,
1026
+ "tokens": 2048,
1027
+ "nonfinite_positions": 0,
1028
+ "nonfinite_logits": 0,
1029
+ "s": 2.269266128540039,
1030
+ "layers": {},
1031
+ "avail_gib": 28.234882354736328
1032
+ },
1033
+ {
1034
+ "row": 112,
1035
+ "tokens": 2048,
1036
+ "nonfinite_positions": 0,
1037
+ "nonfinite_logits": 0,
1038
+ "s": 2.2739131450653076,
1039
+ "layers": {},
1040
+ "avail_gib": 28.234073638916016
1041
+ },
1042
+ {
1043
+ "row": 113,
1044
+ "tokens": 2048,
1045
+ "nonfinite_positions": 0,
1046
+ "nonfinite_logits": 0,
1047
+ "s": 2.26838755607605,
1048
+ "layers": {},
1049
+ "avail_gib": 28.234390258789062
1050
+ },
1051
+ {
1052
+ "row": 114,
1053
+ "tokens": 2048,
1054
+ "nonfinite_positions": 0,
1055
+ "nonfinite_logits": 0,
1056
+ "s": 2.2643935680389404,
1057
+ "layers": {},
1058
+ "avail_gib": 28.234573364257812
1059
+ },
1060
+ {
1061
+ "row": 115,
1062
+ "tokens": 2048,
1063
+ "nonfinite_positions": 0,
1064
+ "nonfinite_logits": 0,
1065
+ "s": 2.271501302719116,
1066
+ "layers": {},
1067
+ "avail_gib": 28.234390258789062
1068
+ },
1069
+ {
1070
+ "row": 116,
1071
+ "tokens": 2048,
1072
+ "nonfinite_positions": 0,
1073
+ "nonfinite_logits": 0,
1074
+ "s": 2.276754379272461,
1075
+ "layers": {},
1076
+ "avail_gib": 28.232662200927734
1077
+ },
1078
+ {
1079
+ "row": 117,
1080
+ "tokens": 2048,
1081
+ "nonfinite_positions": 0,
1082
+ "nonfinite_logits": 0,
1083
+ "s": 2.277482271194458,
1084
+ "layers": {},
1085
+ "avail_gib": 28.232845306396484
1086
+ },
1087
+ {
1088
+ "row": 118,
1089
+ "tokens": 2048,
1090
+ "nonfinite_positions": 0,
1091
+ "nonfinite_logits": 0,
1092
+ "s": 2.267005205154419,
1093
+ "layers": {},
1094
+ "avail_gib": 28.233028411865234
1095
+ },
1096
+ {
1097
+ "row": 119,
1098
+ "tokens": 2048,
1099
+ "nonfinite_positions": 0,
1100
+ "nonfinite_logits": 0,
1101
+ "s": 2.2655699253082275,
1102
+ "layers": {},
1103
+ "avail_gib": 28.232723236083984
1104
+ },
1105
+ {
1106
+ "row": 120,
1107
+ "tokens": 2048,
1108
+ "nonfinite_positions": 0,
1109
+ "nonfinite_logits": 0,
1110
+ "s": 2.28139591217041,
1111
+ "layers": {},
1112
+ "avail_gib": 28.232852935791016
1113
+ },
1114
+ {
1115
+ "row": 121,
1116
+ "tokens": 2048,
1117
+ "nonfinite_positions": 0,
1118
+ "nonfinite_logits": 0,
1119
+ "s": 2.283163547515869,
1120
+ "layers": {},
1121
+ "avail_gib": 28.23284912109375
1122
+ },
1123
+ {
1124
+ "row": 122,
1125
+ "tokens": 2048,
1126
+ "nonfinite_positions": 0,
1127
+ "nonfinite_logits": 0,
1128
+ "s": 2.2710721492767334,
1129
+ "layers": {},
1130
+ "avail_gib": 28.232635498046875
1131
+ },
1132
+ {
1133
+ "row": 123,
1134
+ "tokens": 2048,
1135
+ "nonfinite_positions": 0,
1136
+ "nonfinite_logits": 0,
1137
+ "s": 2.285168409347534,
1138
+ "layers": {},
1139
+ "avail_gib": 28.229694366455078
1140
+ },
1141
+ {
1142
+ "row": 124,
1143
+ "tokens": 2048,
1144
+ "nonfinite_positions": 0,
1145
+ "nonfinite_logits": 0,
1146
+ "s": 2.276379346847534,
1147
+ "layers": {},
1148
+ "avail_gib": 28.226825714111328
1149
+ },
1150
+ {
1151
+ "row": 125,
1152
+ "tokens": 2048,
1153
+ "nonfinite_positions": 0,
1154
+ "nonfinite_logits": 0,
1155
+ "s": 2.266559362411499,
1156
+ "layers": {},
1157
+ "avail_gib": 28.230937957763672
1158
+ },
1159
+ {
1160
+ "row": 126,
1161
+ "tokens": 2048,
1162
+ "nonfinite_positions": 0,
1163
+ "nonfinite_logits": 0,
1164
+ "s": 2.2764394283294678,
1165
+ "layers": {},
1166
+ "avail_gib": 28.228374481201172
1167
+ },
1168
+ {
1169
+ "row": 127,
1170
+ "tokens": 2048,
1171
+ "nonfinite_positions": 0,
1172
+ "nonfinite_logits": 0,
1173
+ "s": 2.277794361114502,
1174
+ "layers": {},
1175
+ "avail_gib": 28.231189727783203
1176
+ },
1177
+ {
1178
+ "row": 128,
1179
+ "tokens": 2048,
1180
+ "nonfinite_positions": 0,
1181
+ "nonfinite_logits": 0,
1182
+ "s": 2.280662775039673,
1183
+ "layers": {},
1184
+ "avail_gib": 28.228626251220703
1185
+ },
1186
+ {
1187
+ "row": 129,
1188
+ "tokens": 2048,
1189
+ "nonfinite_positions": 0,
1190
+ "nonfinite_logits": 0,
1191
+ "s": 2.272444009780884,
1192
+ "layers": {},
1193
+ "avail_gib": 28.230960845947266
1194
+ },
1195
+ {
1196
+ "row": 130,
1197
+ "tokens": 2048,
1198
+ "nonfinite_positions": 0,
1199
+ "nonfinite_logits": 0,
1200
+ "s": 2.2595503330230713,
1201
+ "layers": {},
1202
+ "avail_gib": 28.229251861572266
1203
+ },
1204
+ {
1205
+ "row": 131,
1206
+ "tokens": 2048,
1207
+ "nonfinite_positions": 0,
1208
+ "nonfinite_logits": 0,
1209
+ "s": 2.2625155448913574,
1210
+ "layers": {},
1211
+ "avail_gib": 28.23178482055664
1212
+ },
1213
+ {
1214
+ "row": 132,
1215
+ "tokens": 2048,
1216
+ "nonfinite_positions": 0,
1217
+ "nonfinite_logits": 0,
1218
+ "s": 2.2744147777557373,
1219
+ "layers": {},
1220
+ "avail_gib": 28.235145568847656
1221
+ },
1222
+ {
1223
+ "row": 133,
1224
+ "tokens": 2048,
1225
+ "nonfinite_positions": 0,
1226
+ "nonfinite_logits": 0,
1227
+ "s": 2.2660746574401855,
1228
+ "layers": {},
1229
+ "avail_gib": 28.238624572753906
1230
+ },
1231
+ {
1232
+ "row": 134,
1233
+ "tokens": 2048,
1234
+ "nonfinite_positions": 0,
1235
+ "nonfinite_logits": 0,
1236
+ "s": 2.2734830379486084,
1237
+ "layers": {},
1238
+ "avail_gib": 28.235740661621094
1239
+ },
1240
+ {
1241
+ "row": 135,
1242
+ "tokens": 2048,
1243
+ "nonfinite_positions": 0,
1244
+ "nonfinite_logits": 0,
1245
+ "s": 2.265512704849243,
1246
+ "layers": {},
1247
+ "avail_gib": 28.23775863647461
1248
+ },
1249
+ {
1250
+ "row": 136,
1251
+ "tokens": 2048,
1252
+ "nonfinite_positions": 0,
1253
+ "nonfinite_logits": 0,
1254
+ "s": 2.2745816707611084,
1255
+ "layers": {},
1256
+ "avail_gib": 28.237972259521484
1257
+ },
1258
+ {
1259
+ "row": 137,
1260
+ "tokens": 2048,
1261
+ "nonfinite_positions": 0,
1262
+ "nonfinite_logits": 0,
1263
+ "s": 2.271151542663574,
1264
+ "layers": {},
1265
+ "avail_gib": 28.238285064697266
1266
+ },
1267
+ {
1268
+ "row": 138,
1269
+ "tokens": 2048,
1270
+ "nonfinite_positions": 0,
1271
+ "nonfinite_logits": 0,
1272
+ "s": 2.259950876235962,
1273
+ "layers": {},
1274
+ "avail_gib": 28.238582611083984
1275
+ },
1276
+ {
1277
+ "row": 139,
1278
+ "tokens": 2048,
1279
+ "nonfinite_positions": 0,
1280
+ "nonfinite_logits": 0,
1281
+ "s": 2.2638018131256104,
1282
+ "layers": {},
1283
+ "avail_gib": 28.2373046875
1284
+ },
1285
+ {
1286
+ "row": 140,
1287
+ "tokens": 2048,
1288
+ "nonfinite_positions": 0,
1289
+ "nonfinite_logits": 0,
1290
+ "s": 2.2719929218292236,
1291
+ "layers": {},
1292
+ "avail_gib": 28.237289428710938
1293
+ },
1294
+ {
1295
+ "row": 141,
1296
+ "tokens": 2048,
1297
+ "nonfinite_positions": 0,
1298
+ "nonfinite_logits": 0,
1299
+ "s": 2.280733108520508,
1300
+ "layers": {},
1301
+ "avail_gib": 28.237289428710938
1302
+ },
1303
+ {
1304
+ "row": 142,
1305
+ "tokens": 2048,
1306
+ "nonfinite_positions": 0,
1307
+ "nonfinite_logits": 0,
1308
+ "s": 2.255004405975342,
1309
+ "layers": {},
1310
+ "avail_gib": 28.23760223388672
1311
+ },
1312
+ {
1313
+ "row": 143,
1314
+ "tokens": 2048,
1315
+ "nonfinite_positions": 0,
1316
+ "nonfinite_logits": 0,
1317
+ "s": 2.267336130142212,
1318
+ "layers": {},
1319
+ "avail_gib": 28.237178802490234
1320
+ },
1321
+ {
1322
+ "row": 144,
1323
+ "tokens": 2048,
1324
+ "nonfinite_positions": 0,
1325
+ "nonfinite_logits": 0,
1326
+ "s": 2.2625856399536133,
1327
+ "layers": {},
1328
+ "avail_gib": 28.23681640625
1329
+ },
1330
+ {
1331
+ "row": 145,
1332
+ "tokens": 2048,
1333
+ "nonfinite_positions": 0,
1334
+ "nonfinite_logits": 0,
1335
+ "s": 2.274451494216919,
1336
+ "layers": {},
1337
+ "avail_gib": 28.23712158203125
1338
+ },
1339
+ {
1340
+ "row": 146,
1341
+ "tokens": 2048,
1342
+ "nonfinite_positions": 0,
1343
+ "nonfinite_logits": 0,
1344
+ "s": 2.2818996906280518,
1345
+ "layers": {},
1346
+ "avail_gib": 28.237136840820312
1347
+ },
1348
+ {
1349
+ "row": 147,
1350
+ "tokens": 2048,
1351
+ "nonfinite_positions": 0,
1352
+ "nonfinite_logits": 0,
1353
+ "s": 2.2767698764801025,
1354
+ "layers": {},
1355
+ "avail_gib": 28.236831665039062
1356
+ },
1357
+ {
1358
+ "row": 148,
1359
+ "tokens": 2048,
1360
+ "nonfinite_positions": 0,
1361
+ "nonfinite_logits": 0,
1362
+ "s": 2.274561643600464,
1363
+ "layers": {},
1364
+ "avail_gib": 28.236953735351562
1365
+ },
1366
+ {
1367
+ "row": 149,
1368
+ "tokens": 2048,
1369
+ "nonfinite_positions": 0,
1370
+ "nonfinite_logits": 0,
1371
+ "s": 2.2906551361083984,
1372
+ "layers": {},
1373
+ "avail_gib": 28.236953735351562
1374
+ },
1375
+ {
1376
+ "row": 150,
1377
+ "tokens": 2048,
1378
+ "nonfinite_positions": 0,
1379
+ "nonfinite_logits": 0,
1380
+ "s": 2.2731306552886963,
1381
+ "layers": {},
1382
+ "avail_gib": 28.236953735351562
1383
+ },
1384
+ {
1385
+ "row": 151,
1386
+ "tokens": 2048,
1387
+ "nonfinite_positions": 0,
1388
+ "nonfinite_logits": 0,
1389
+ "s": 2.27135968208313,
1390
+ "layers": {},
1391
+ "avail_gib": 28.236961364746094
1392
+ },
1393
+ {
1394
+ "row": 152,
1395
+ "tokens": 2048,
1396
+ "nonfinite_positions": 0,
1397
+ "nonfinite_logits": 0,
1398
+ "s": 2.268874168395996,
1399
+ "layers": {},
1400
+ "avail_gib": 28.23672866821289
1401
+ },
1402
+ {
1403
+ "row": 153,
1404
+ "tokens": 2048,
1405
+ "nonfinite_positions": 0,
1406
+ "nonfinite_logits": 0,
1407
+ "s": 2.271796941757202,
1408
+ "layers": {},
1409
+ "avail_gib": 28.23660659790039
1410
+ },
1411
+ {
1412
+ "row": 154,
1413
+ "tokens": 2048,
1414
+ "nonfinite_positions": 0,
1415
+ "nonfinite_logits": 0,
1416
+ "s": 2.2828872203826904,
1417
+ "layers": {},
1418
+ "avail_gib": 28.2462158203125
1419
+ },
1420
+ {
1421
+ "row": 155,
1422
+ "tokens": 2048,
1423
+ "nonfinite_positions": 0,
1424
+ "nonfinite_logits": 0,
1425
+ "s": 2.273867130279541,
1426
+ "layers": {},
1427
+ "avail_gib": 28.24462890625
1428
+ },
1429
+ {
1430
+ "row": 156,
1431
+ "tokens": 2048,
1432
+ "nonfinite_positions": 0,
1433
+ "nonfinite_logits": 0,
1434
+ "s": 2.274989366531372,
1435
+ "layers": {},
1436
+ "avail_gib": 28.244091033935547
1437
+ },
1438
+ {
1439
+ "row": 157,
1440
+ "tokens": 2048,
1441
+ "nonfinite_positions": 0,
1442
+ "nonfinite_logits": 0,
1443
+ "s": 2.2677552700042725,
1444
+ "layers": {},
1445
+ "avail_gib": 28.243907928466797
1446
+ },
1447
+ {
1448
+ "row": 158,
1449
+ "tokens": 2048,
1450
+ "nonfinite_positions": 0,
1451
+ "nonfinite_logits": 0,
1452
+ "s": 2.2672383785247803,
1453
+ "layers": {},
1454
+ "avail_gib": 28.244091033935547
1455
+ },
1456
+ {
1457
+ "row": 159,
1458
+ "tokens": 2048,
1459
+ "nonfinite_positions": 0,
1460
+ "nonfinite_logits": 0,
1461
+ "s": 2.27286696434021,
1462
+ "layers": {},
1463
+ "avail_gib": 28.24368667602539
1464
+ },
1465
+ {
1466
+ "row": 160,
1467
+ "tokens": 2048,
1468
+ "nonfinite_positions": 0,
1469
+ "nonfinite_logits": 0,
1470
+ "s": 2.2767386436462402,
1471
+ "layers": {},
1472
+ "avail_gib": 28.24297332763672
1473
+ },
1474
+ {
1475
+ "row": 161,
1476
+ "tokens": 2048,
1477
+ "nonfinite_positions": 0,
1478
+ "nonfinite_logits": 0,
1479
+ "s": 2.25927996635437,
1480
+ "layers": {},
1481
+ "avail_gib": 28.24315643310547
1482
+ },
1483
+ {
1484
+ "row": 162,
1485
+ "tokens": 2048,
1486
+ "nonfinite_positions": 0,
1487
+ "nonfinite_logits": 0,
1488
+ "s": 2.2692127227783203,
1489
+ "layers": {},
1490
+ "avail_gib": 28.24285125732422
1491
+ },
1492
+ {
1493
+ "row": 163,
1494
+ "tokens": 2048,
1495
+ "nonfinite_positions": 0,
1496
+ "nonfinite_logits": 0,
1497
+ "s": 2.259249448776245,
1498
+ "layers": {},
1499
+ "avail_gib": 28.242916107177734
1500
+ },
1501
+ {
1502
+ "row": 164,
1503
+ "tokens": 2048,
1504
+ "nonfinite_positions": 0,
1505
+ "nonfinite_logits": 0,
1506
+ "s": 2.2818603515625,
1507
+ "layers": {},
1508
+ "avail_gib": 28.242923736572266
1509
+ },
1510
+ {
1511
+ "row": 165,
1512
+ "tokens": 2048,
1513
+ "nonfinite_positions": 0,
1514
+ "nonfinite_logits": 0,
1515
+ "s": 2.270399808883667,
1516
+ "layers": {},
1517
+ "avail_gib": 28.242748260498047
1518
+ },
1519
+ {
1520
+ "row": 166,
1521
+ "tokens": 2048,
1522
+ "nonfinite_positions": 0,
1523
+ "nonfinite_logits": 0,
1524
+ "s": 2.2670066356658936,
1525
+ "layers": {},
1526
+ "avail_gib": 28.242931365966797
1527
+ },
1528
+ {
1529
+ "row": 167,
1530
+ "tokens": 2048,
1531
+ "nonfinite_positions": 0,
1532
+ "nonfinite_logits": 0,
1533
+ "s": 2.272343635559082,
1534
+ "layers": {},
1535
+ "avail_gib": 28.242267608642578
1536
+ },
1537
+ {
1538
+ "row": 168,
1539
+ "tokens": 2048,
1540
+ "nonfinite_positions": 0,
1541
+ "nonfinite_logits": 0,
1542
+ "s": 2.27726674079895,
1543
+ "layers": {},
1544
+ "avail_gib": 28.242576599121094
1545
+ },
1546
+ {
1547
+ "row": 169,
1548
+ "tokens": 2048,
1549
+ "nonfinite_positions": 0,
1550
+ "nonfinite_logits": 0,
1551
+ "s": 2.273228883743286,
1552
+ "layers": {},
1553
+ "avail_gib": 28.242271423339844
1554
+ },
1555
+ {
1556
+ "row": 170,
1557
+ "tokens": 2048,
1558
+ "nonfinite_positions": 0,
1559
+ "nonfinite_logits": 0,
1560
+ "s": 2.2726333141326904,
1561
+ "layers": {},
1562
+ "avail_gib": 28.242271423339844
1563
+ },
1564
+ {
1565
+ "row": 171,
1566
+ "tokens": 2048,
1567
+ "nonfinite_positions": 0,
1568
+ "nonfinite_logits": 0,
1569
+ "s": 2.2836949825286865,
1570
+ "layers": {},
1571
+ "avail_gib": 28.242576599121094
1572
+ },
1573
+ {
1574
+ "row": 172,
1575
+ "tokens": 2048,
1576
+ "nonfinite_positions": 0,
1577
+ "nonfinite_logits": 0,
1578
+ "s": 2.264988660812378,
1579
+ "layers": {},
1580
+ "avail_gib": 28.242286682128906
1581
+ },
1582
+ {
1583
+ "row": 173,
1584
+ "tokens": 2048,
1585
+ "nonfinite_positions": 0,
1586
+ "nonfinite_logits": 0,
1587
+ "s": 2.2700066566467285,
1588
+ "layers": {},
1589
+ "avail_gib": 28.241336822509766
1590
+ },
1591
+ {
1592
+ "row": 174,
1593
+ "tokens": 2048,
1594
+ "nonfinite_positions": 0,
1595
+ "nonfinite_logits": 0,
1596
+ "s": 2.2764651775360107,
1597
+ "layers": {},
1598
+ "avail_gib": 28.24273681640625
1599
+ },
1600
+ {
1601
+ "row": 175,
1602
+ "tokens": 2048,
1603
+ "nonfinite_positions": 0,
1604
+ "nonfinite_logits": 0,
1605
+ "s": 2.2701199054718018,
1606
+ "layers": {},
1607
+ "avail_gib": 28.244712829589844
1608
+ },
1609
+ {
1610
+ "row": 176,
1611
+ "tokens": 2048,
1612
+ "nonfinite_positions": 0,
1613
+ "nonfinite_logits": 0,
1614
+ "s": 2.2810449600219727,
1615
+ "layers": {},
1616
+ "avail_gib": 28.244590759277344
1617
+ },
1618
+ {
1619
+ "row": 177,
1620
+ "tokens": 2048,
1621
+ "nonfinite_positions": 0,
1622
+ "nonfinite_logits": 0,
1623
+ "s": 2.272693634033203,
1624
+ "layers": {},
1625
+ "avail_gib": 28.24460220336914
1626
+ },
1627
+ {
1628
+ "row": 178,
1629
+ "tokens": 2048,
1630
+ "nonfinite_positions": 0,
1631
+ "nonfinite_logits": 0,
1632
+ "s": 2.267113447189331,
1633
+ "layers": {},
1634
+ "avail_gib": 28.24460220336914
1635
+ },
1636
+ {
1637
+ "row": 179,
1638
+ "tokens": 2048,
1639
+ "nonfinite_positions": 0,
1640
+ "nonfinite_logits": 0,
1641
+ "s": 2.2739462852478027,
1642
+ "layers": {},
1643
+ "avail_gib": 28.24472427368164
1644
+ },
1645
+ {
1646
+ "row": 180,
1647
+ "tokens": 2048,
1648
+ "nonfinite_positions": 0,
1649
+ "nonfinite_logits": 0,
1650
+ "s": 2.270507574081421,
1651
+ "layers": {},
1652
+ "avail_gib": 28.244609832763672
1653
+ },
1654
+ {
1655
+ "row": 181,
1656
+ "tokens": 2048,
1657
+ "nonfinite_positions": 0,
1658
+ "nonfinite_logits": 0,
1659
+ "s": 2.2643849849700928,
1660
+ "layers": {},
1661
+ "avail_gib": 28.244609832763672
1662
+ },
1663
+ {
1664
+ "row": 182,
1665
+ "tokens": 2048,
1666
+ "nonfinite_positions": 0,
1667
+ "nonfinite_logits": 0,
1668
+ "s": 2.2640953063964844,
1669
+ "layers": {},
1670
+ "avail_gib": 28.244873046875
1671
+ },
1672
+ {
1673
+ "row": 183,
1674
+ "tokens": 2048,
1675
+ "nonfinite_positions": 0,
1676
+ "nonfinite_logits": 0,
1677
+ "s": 2.2670223712921143,
1678
+ "layers": {},
1679
+ "avail_gib": 28.2449951171875
1680
+ },
1681
+ {
1682
+ "row": 184,
1683
+ "tokens": 2048,
1684
+ "nonfinite_positions": 0,
1685
+ "nonfinite_logits": 0,
1686
+ "s": 2.2656946182250977,
1687
+ "layers": {},
1688
+ "avail_gib": 28.244873046875
1689
+ },
1690
+ {
1691
+ "row": 185,
1692
+ "tokens": 2048,
1693
+ "nonfinite_positions": 0,
1694
+ "nonfinite_logits": 0,
1695
+ "s": 2.2780978679656982,
1696
+ "layers": {},
1697
+ "avail_gib": 28.24517822265625
1698
+ },
1699
+ {
1700
+ "row": 186,
1701
+ "tokens": 2048,
1702
+ "nonfinite_positions": 0,
1703
+ "nonfinite_logits": 0,
1704
+ "s": 2.275624990463257,
1705
+ "layers": {},
1706
+ "avail_gib": 28.245193481445312
1707
+ },
1708
+ {
1709
+ "row": 187,
1710
+ "tokens": 2048,
1711
+ "nonfinite_positions": 0,
1712
+ "nonfinite_logits": 0,
1713
+ "s": 2.2652602195739746,
1714
+ "layers": {},
1715
+ "avail_gib": 28.244888305664062
1716
+ },
1717
+ {
1718
+ "row": 188,
1719
+ "tokens": 2048,
1720
+ "nonfinite_positions": 0,
1721
+ "nonfinite_logits": 0,
1722
+ "s": 2.2570362091064453,
1723
+ "layers": {},
1724
+ "avail_gib": 28.245197296142578
1725
+ },
1726
+ {
1727
+ "row": 189,
1728
+ "tokens": 2048,
1729
+ "nonfinite_positions": 0,
1730
+ "nonfinite_logits": 0,
1731
+ "s": 2.277578830718994,
1732
+ "layers": {},
1733
+ "avail_gib": 28.245616912841797
1734
+ },
1735
+ {
1736
+ "row": 190,
1737
+ "tokens": 2048,
1738
+ "nonfinite_positions": 0,
1739
+ "nonfinite_logits": 0,
1740
+ "s": 2.2676382064819336,
1741
+ "layers": {},
1742
+ "avail_gib": 28.244945526123047
1743
+ },
1744
+ {
1745
+ "row": 191,
1746
+ "tokens": 2048,
1747
+ "nonfinite_positions": 0,
1748
+ "nonfinite_logits": 0,
1749
+ "s": 2.2614905834198,
1750
+ "layers": {},
1751
+ "avail_gib": 28.245128631591797
1752
+ },
1753
+ {
1754
+ "row": 192,
1755
+ "tokens": 2048,
1756
+ "nonfinite_positions": 0,
1757
+ "nonfinite_logits": 0,
1758
+ "s": 2.2731006145477295,
1759
+ "layers": {},
1760
+ "avail_gib": 28.24422836303711
1761
+ },
1762
+ {
1763
+ "row": 193,
1764
+ "tokens": 2048,
1765
+ "nonfinite_positions": 0,
1766
+ "nonfinite_logits": 0,
1767
+ "s": 2.273102283477783,
1768
+ "layers": {},
1769
+ "avail_gib": 28.24435043334961
1770
+ },
1771
+ {
1772
+ "row": 194,
1773
+ "tokens": 2048,
1774
+ "nonfinite_positions": 0,
1775
+ "nonfinite_logits": 0,
1776
+ "s": 2.274975299835205,
1777
+ "layers": {},
1778
+ "avail_gib": 28.244064331054688
1779
+ },
1780
+ {
1781
+ "row": 195,
1782
+ "tokens": 2048,
1783
+ "nonfinite_positions": 0,
1784
+ "nonfinite_logits": 0,
1785
+ "s": 2.273124933242798,
1786
+ "layers": {},
1787
+ "avail_gib": 28.244064331054688
1788
+ },
1789
+ {
1790
+ "row": 196,
1791
+ "tokens": 2048,
1792
+ "nonfinite_positions": 0,
1793
+ "nonfinite_logits": 0,
1794
+ "s": 2.2689428329467773,
1795
+ "layers": {},
1796
+ "avail_gib": 28.243640899658203
1797
+ },
1798
+ {
1799
+ "row": 197,
1800
+ "tokens": 2048,
1801
+ "nonfinite_positions": 0,
1802
+ "nonfinite_logits": 0,
1803
+ "s": 2.2695798873901367,
1804
+ "layers": {},
1805
+ "avail_gib": 28.243824005126953
1806
+ },
1807
+ {
1808
+ "row": 198,
1809
+ "tokens": 2048,
1810
+ "nonfinite_positions": 0,
1811
+ "nonfinite_logits": 0,
1812
+ "s": 2.2811145782470703,
1813
+ "layers": {},
1814
+ "avail_gib": 28.24358367919922
1815
+ },
1816
+ {
1817
+ "row": 199,
1818
+ "tokens": 2048,
1819
+ "nonfinite_positions": 0,
1820
+ "nonfinite_logits": 0,
1821
+ "s": 2.2713797092437744,
1822
+ "layers": {},
1823
+ "avail_gib": 28.24359893798828
1824
+ },
1825
+ {
1826
+ "row": 200,
1827
+ "tokens": 2048,
1828
+ "nonfinite_positions": 0,
1829
+ "nonfinite_logits": 0,
1830
+ "s": 2.2627556324005127,
1831
+ "layers": {},
1832
+ "avail_gib": 28.24329376220703
1833
+ },
1834
+ {
1835
+ "row": 201,
1836
+ "tokens": 2048,
1837
+ "nonfinite_positions": 0,
1838
+ "nonfinite_logits": 0,
1839
+ "s": 2.2747464179992676,
1840
+ "layers": {},
1841
+ "avail_gib": 28.243118286132812
1842
+ },
1843
+ {
1844
+ "row": 202,
1845
+ "tokens": 2048,
1846
+ "nonfinite_positions": 0,
1847
+ "nonfinite_logits": 0,
1848
+ "s": 2.258565902709961,
1849
+ "layers": {},
1850
+ "avail_gib": 28.243118286132812
1851
+ },
1852
+ {
1853
+ "row": 203,
1854
+ "tokens": 2048,
1855
+ "nonfinite_positions": 0,
1856
+ "nonfinite_logits": 0,
1857
+ "s": 2.2429206371307373,
1858
+ "layers": {},
1859
+ "avail_gib": 28.243133544921875
1860
+ },
1861
+ {
1862
+ "row": 204,
1863
+ "tokens": 2048,
1864
+ "nonfinite_positions": 0,
1865
+ "nonfinite_logits": 0,
1866
+ "s": 2.2491681575775146,
1867
+ "layers": {},
1868
+ "avail_gib": 28.2412109375
1869
+ },
1870
+ {
1871
+ "row": 205,
1872
+ "tokens": 2048,
1873
+ "nonfinite_positions": 0,
1874
+ "nonfinite_logits": 0,
1875
+ "s": 2.248776435852051,
1876
+ "layers": {},
1877
+ "avail_gib": 28.24102783203125
1878
+ },
1879
+ {
1880
+ "row": 206,
1881
+ "tokens": 2048,
1882
+ "nonfinite_positions": 0,
1883
+ "nonfinite_logits": 0,
1884
+ "s": 2.263939380645752,
1885
+ "layers": {},
1886
+ "avail_gib": 28.24090576171875
1887
+ },
1888
+ {
1889
+ "row": 207,
1890
+ "tokens": 2048,
1891
+ "nonfinite_positions": 0,
1892
+ "nonfinite_logits": 0,
1893
+ "s": 2.252782106399536,
1894
+ "layers": {},
1895
+ "avail_gib": 28.243698120117188
1896
+ },
1897
+ {
1898
+ "row": 208,
1899
+ "tokens": 2048,
1900
+ "nonfinite_positions": 0,
1901
+ "nonfinite_logits": 0,
1902
+ "s": 2.257444143295288,
1903
+ "layers": {},
1904
+ "avail_gib": 28.236846923828125
1905
+ },
1906
+ {
1907
+ "row": 209,
1908
+ "tokens": 2048,
1909
+ "nonfinite_positions": 0,
1910
+ "nonfinite_logits": 0,
1911
+ "s": 2.2511751651763916,
1912
+ "layers": {},
1913
+ "avail_gib": 28.237747192382812
1914
+ },
1915
+ {
1916
+ "row": 210,
1917
+ "tokens": 2048,
1918
+ "nonfinite_positions": 0,
1919
+ "nonfinite_logits": 0,
1920
+ "s": 2.2460474967956543,
1921
+ "layers": {},
1922
+ "avail_gib": 28.238468170166016
1923
+ },
1924
+ {
1925
+ "row": 211,
1926
+ "tokens": 2048,
1927
+ "nonfinite_positions": 0,
1928
+ "nonfinite_logits": 0,
1929
+ "s": 2.2747092247009277,
1930
+ "layers": {},
1931
+ "avail_gib": 28.241111755371094
1932
+ },
1933
+ {
1934
+ "row": 212,
1935
+ "tokens": 2048,
1936
+ "nonfinite_positions": 0,
1937
+ "nonfinite_logits": 0,
1938
+ "s": 2.278061866760254,
1939
+ "layers": {},
1940
+ "avail_gib": 28.24154281616211
1941
+ },
1942
+ {
1943
+ "row": 213,
1944
+ "tokens": 2048,
1945
+ "nonfinite_positions": 0,
1946
+ "nonfinite_logits": 0,
1947
+ "s": 2.2866036891937256,
1948
+ "layers": {},
1949
+ "avail_gib": 28.24437713623047
1950
+ },
1951
+ {
1952
+ "row": 214,
1953
+ "tokens": 2048,
1954
+ "nonfinite_positions": 0,
1955
+ "nonfinite_logits": 0,
1956
+ "s": 2.282038688659668,
1957
+ "layers": {},
1958
+ "avail_gib": 28.24437713623047
1959
+ },
1960
+ {
1961
+ "row": 215,
1962
+ "tokens": 2048,
1963
+ "nonfinite_positions": 0,
1964
+ "nonfinite_logits": 0,
1965
+ "s": 2.2856950759887695,
1966
+ "layers": {},
1967
+ "avail_gib": 28.24437713623047
1968
+ },
1969
+ {
1970
+ "row": 216,
1971
+ "tokens": 2048,
1972
+ "nonfinite_positions": 0,
1973
+ "nonfinite_logits": 0,
1974
+ "s": 2.2836461067199707,
1975
+ "layers": {},
1976
+ "avail_gib": 28.24419403076172
1977
+ },
1978
+ {
1979
+ "row": 217,
1980
+ "tokens": 2048,
1981
+ "nonfinite_positions": 0,
1982
+ "nonfinite_logits": 0,
1983
+ "s": 2.279174566268921,
1984
+ "layers": {},
1985
+ "avail_gib": 28.244388580322266
1986
+ },
1987
+ {
1988
+ "row": 218,
1989
+ "tokens": 2048,
1990
+ "nonfinite_positions": 0,
1991
+ "nonfinite_logits": 0,
1992
+ "s": 2.288829803466797,
1993
+ "layers": {},
1994
+ "avail_gib": 28.243484497070312
1995
+ },
1996
+ {
1997
+ "row": 219,
1998
+ "tokens": 2048,
1999
+ "nonfinite_positions": 0,
2000
+ "nonfinite_logits": 0,
2001
+ "s": 2.2800896167755127,
2002
+ "layers": {},
2003
+ "avail_gib": 28.241802215576172
2004
+ },
2005
+ {
2006
+ "row": 220,
2007
+ "tokens": 2048,
2008
+ "nonfinite_positions": 0,
2009
+ "nonfinite_logits": 0,
2010
+ "s": 2.2828986644744873,
2011
+ "layers": {},
2012
+ "avail_gib": 28.241039276123047
2013
+ },
2014
+ {
2015
+ "row": 221,
2016
+ "tokens": 2048,
2017
+ "nonfinite_positions": 0,
2018
+ "nonfinite_logits": 0,
2019
+ "s": 2.2893941402435303,
2020
+ "layers": {},
2021
+ "avail_gib": 28.240734100341797
2022
+ },
2023
+ {
2024
+ "row": 222,
2025
+ "tokens": 2048,
2026
+ "nonfinite_positions": 0,
2027
+ "nonfinite_logits": 0,
2028
+ "s": 2.286463975906372,
2029
+ "layers": {},
2030
+ "avail_gib": 28.240798950195312
2031
+ },
2032
+ {
2033
+ "row": 223,
2034
+ "tokens": 2048,
2035
+ "nonfinite_positions": 0,
2036
+ "nonfinite_logits": 0,
2037
+ "s": 2.2787463665008545,
2038
+ "layers": {},
2039
+ "avail_gib": 28.240615844726562
2040
+ },
2041
+ {
2042
+ "row": 224,
2043
+ "tokens": 2048,
2044
+ "nonfinite_positions": 0,
2045
+ "nonfinite_logits": 0,
2046
+ "s": 2.275909662246704,
2047
+ "layers": {},
2048
+ "avail_gib": 28.240493774414062
2049
+ },
2050
+ {
2051
+ "row": 225,
2052
+ "tokens": 2048,
2053
+ "nonfinite_positions": 0,
2054
+ "nonfinite_logits": 0,
2055
+ "s": 2.2806859016418457,
2056
+ "layers": {},
2057
+ "avail_gib": 28.240631103515625
2058
+ },
2059
+ {
2060
+ "row": 226,
2061
+ "tokens": 2048,
2062
+ "nonfinite_positions": 0,
2063
+ "nonfinite_logits": 0,
2064
+ "s": 2.2763710021972656,
2065
+ "layers": {},
2066
+ "avail_gib": 28.24039077758789
2067
+ },
2068
+ {
2069
+ "row": 227,
2070
+ "tokens": 2048,
2071
+ "nonfinite_positions": 0,
2072
+ "nonfinite_logits": 0,
2073
+ "s": 2.28342866897583,
2074
+ "layers": {},
2075
+ "avail_gib": 28.24057388305664
2076
+ },
2077
+ {
2078
+ "row": 228,
2079
+ "tokens": 2048,
2080
+ "nonfinite_positions": 0,
2081
+ "nonfinite_logits": 0,
2082
+ "s": 2.2860209941864014,
2083
+ "layers": {},
2084
+ "avail_gib": 28.24057388305664
2085
+ },
2086
+ {
2087
+ "row": 229,
2088
+ "tokens": 2048,
2089
+ "nonfinite_positions": 0,
2090
+ "nonfinite_logits": 0,
2091
+ "s": 2.279675006866455,
2092
+ "layers": {},
2093
+ "avail_gib": 28.24016571044922
2094
+ },
2095
+ {
2096
+ "row": 230,
2097
+ "tokens": 2048,
2098
+ "nonfinite_positions": 0,
2099
+ "nonfinite_logits": 0,
2100
+ "s": 2.278358221054077,
2101
+ "layers": {},
2102
+ "avail_gib": 28.239627838134766
2103
+ },
2104
+ {
2105
+ "row": 231,
2106
+ "tokens": 2048,
2107
+ "nonfinite_positions": 0,
2108
+ "nonfinite_logits": 0,
2109
+ "s": 2.266968250274658,
2110
+ "layers": {},
2111
+ "avail_gib": 28.241138458251953
2112
+ },
2113
+ {
2114
+ "row": 232,
2115
+ "tokens": 2048,
2116
+ "nonfinite_positions": 0,
2117
+ "nonfinite_logits": 0,
2118
+ "s": 2.278071880340576,
2119
+ "layers": {},
2120
+ "avail_gib": 28.24077606201172
2121
+ },
2122
+ {
2123
+ "row": 233,
2124
+ "tokens": 2048,
2125
+ "nonfinite_positions": 0,
2126
+ "nonfinite_logits": 0,
2127
+ "s": 2.280172824859619,
2128
+ "layers": {},
2129
+ "avail_gib": 28.24042510986328
2130
+ },
2131
+ {
2132
+ "row": 234,
2133
+ "tokens": 2048,
2134
+ "nonfinite_positions": 0,
2135
+ "nonfinite_logits": 0,
2136
+ "s": 2.279249668121338,
2137
+ "layers": {},
2138
+ "avail_gib": 28.241703033447266
2139
+ },
2140
+ {
2141
+ "row": 235,
2142
+ "tokens": 2048,
2143
+ "nonfinite_positions": 0,
2144
+ "nonfinite_logits": 0,
2145
+ "s": 2.28027081489563,
2146
+ "layers": {},
2147
+ "avail_gib": 28.24158477783203
2148
+ },
2149
+ {
2150
+ "row": 236,
2151
+ "tokens": 2048,
2152
+ "nonfinite_positions": 0,
2153
+ "nonfinite_logits": 0,
2154
+ "s": 2.2786855697631836,
2155
+ "layers": {},
2156
+ "avail_gib": 28.241703033447266
2157
+ },
2158
+ {
2159
+ "row": 237,
2160
+ "tokens": 2048,
2161
+ "nonfinite_positions": 0,
2162
+ "nonfinite_logits": 0,
2163
+ "s": 2.2839698791503906,
2164
+ "layers": {},
2165
+ "avail_gib": 28.241580963134766
2166
+ },
2167
+ {
2168
+ "row": 238,
2169
+ "tokens": 2048,
2170
+ "nonfinite_positions": 0,
2171
+ "nonfinite_logits": 0,
2172
+ "s": 2.28499174118042,
2173
+ "layers": {},
2174
+ "avail_gib": 28.241718292236328
2175
+ },
2176
+ {
2177
+ "row": 239,
2178
+ "tokens": 2048,
2179
+ "nonfinite_positions": 0,
2180
+ "nonfinite_logits": 0,
2181
+ "s": 2.28157639503479,
2182
+ "layers": {},
2183
+ "avail_gib": 28.24251937866211
2184
+ },
2185
+ {
2186
+ "row": 240,
2187
+ "tokens": 2048,
2188
+ "nonfinite_positions": 0,
2189
+ "nonfinite_logits": 0,
2190
+ "s": 2.289623737335205,
2191
+ "layers": {},
2192
+ "avail_gib": 28.24282455444336
2193
+ },
2194
+ {
2195
+ "row": 241,
2196
+ "tokens": 2048,
2197
+ "nonfinite_positions": 0,
2198
+ "nonfinite_logits": 0,
2199
+ "s": 2.2939116954803467,
2200
+ "layers": {},
2201
+ "avail_gib": 28.24251937866211
2202
+ },
2203
+ {
2204
+ "row": 242,
2205
+ "tokens": 2048,
2206
+ "nonfinite_positions": 0,
2207
+ "nonfinite_logits": 0,
2208
+ "s": 2.272246837615967,
2209
+ "layers": {},
2210
+ "avail_gib": 28.242660522460938
2211
+ },
2212
+ {
2213
+ "row": 243,
2214
+ "tokens": 2048,
2215
+ "nonfinite_positions": 0,
2216
+ "nonfinite_logits": 0,
2217
+ "s": 2.2730720043182373,
2218
+ "layers": {},
2219
+ "avail_gib": 28.242843627929688
2220
+ },
2221
+ {
2222
+ "row": 244,
2223
+ "tokens": 2048,
2224
+ "nonfinite_positions": 0,
2225
+ "nonfinite_logits": 0,
2226
+ "s": 2.2905662059783936,
2227
+ "layers": {},
2228
+ "avail_gib": 28.242660522460938
2229
+ },
2230
+ {
2231
+ "row": 245,
2232
+ "tokens": 2048,
2233
+ "nonfinite_positions": 0,
2234
+ "nonfinite_logits": 0,
2235
+ "s": 2.2841897010803223,
2236
+ "layers": {},
2237
+ "avail_gib": 28.242721557617188
2238
+ },
2239
+ {
2240
+ "row": 246,
2241
+ "tokens": 2048,
2242
+ "nonfinite_positions": 0,
2243
+ "nonfinite_logits": 0,
2244
+ "s": 2.2826461791992188,
2245
+ "layers": {},
2246
+ "avail_gib": 28.242416381835938
2247
+ },
2248
+ {
2249
+ "row": 247,
2250
+ "tokens": 2048,
2251
+ "nonfinite_positions": 0,
2252
+ "nonfinite_logits": 0,
2253
+ "s": 2.286635160446167,
2254
+ "layers": {},
2255
+ "avail_gib": 28.24087142944336
2256
+ },
2257
+ {
2258
+ "row": 248,
2259
+ "tokens": 2048,
2260
+ "nonfinite_positions": 0,
2261
+ "nonfinite_logits": 0,
2262
+ "s": 2.2799458503723145,
2263
+ "layers": {},
2264
+ "avail_gib": 28.24074935913086
2265
+ },
2266
+ {
2267
+ "row": 249,
2268
+ "tokens": 2048,
2269
+ "nonfinite_positions": 0,
2270
+ "nonfinite_logits": 0,
2271
+ "s": 2.2756237983703613,
2272
+ "layers": {},
2273
+ "avail_gib": 28.24074935913086
2274
+ }
2275
+ ],
2276
+ "layer_probes": {
2277
+ "0": {
2278
+ "nonfinite": 0,
2279
+ "max_abs": 0.7462270259857178,
2280
+ "rows": 250
2281
+ },
2282
+ "1": {
2283
+ "nonfinite": 0,
2284
+ "max_abs": 0.30632928013801575,
2285
+ "rows": 250
2286
+ },
2287
+ "2": {
2288
+ "nonfinite": 0,
2289
+ "max_abs": 0.35474884510040283,
2290
+ "rows": 250
2291
+ },
2292
+ "3": {
2293
+ "nonfinite": 0,
2294
+ "max_abs": 0.3687923550605774,
2295
+ "rows": 250
2296
+ },
2297
+ "4": {
2298
+ "nonfinite": 0,
2299
+ "max_abs": 3.665431499481201,
2300
+ "rows": 250
2301
+ },
2302
+ "5": {
2303
+ "nonfinite": 0,
2304
+ "max_abs": 3.88875150680542,
2305
+ "rows": 250
2306
+ },
2307
+ "6": {
2308
+ "nonfinite": 0,
2309
+ "max_abs": 4.430764675140381,
2310
+ "rows": 250
2311
+ },
2312
+ "7": {
2313
+ "nonfinite": 0,
2314
+ "max_abs": 19.972728729248047,
2315
+ "rows": 250
2316
+ },
2317
+ "8": {
2318
+ "nonfinite": 0,
2319
+ "max_abs": 7.23068904876709,
2320
+ "rows": 250
2321
+ },
2322
+ "9": {
2323
+ "nonfinite": 0,
2324
+ "max_abs": 4.297088146209717,
2325
+ "rows": 250
2326
+ },
2327
+ "10": {
2328
+ "nonfinite": 0,
2329
+ "max_abs": 60.56414031982422,
2330
+ "rows": 250
2331
+ },
2332
+ "11": {
2333
+ "nonfinite": 0,
2334
+ "max_abs": 66.8353271484375,
2335
+ "rows": 250
2336
+ },
2337
+ "12": {
2338
+ "nonfinite": 0,
2339
+ "max_abs": 4.782232284545898,
2340
+ "rows": 250
2341
+ },
2342
+ "13": {
2343
+ "nonfinite": 0,
2344
+ "max_abs": 3.0799152851104736,
2345
+ "rows": 250
2346
+ },
2347
+ "14": {
2348
+ "nonfinite": 0,
2349
+ "max_abs": 3.2510175704956055,
2350
+ "rows": 250
2351
+ },
2352
+ "15": {
2353
+ "nonfinite": 0,
2354
+ "max_abs": 2.5488104820251465,
2355
+ "rows": 250
2356
+ },
2357
+ "16": {
2358
+ "nonfinite": 0,
2359
+ "max_abs": 1795.3656005859375,
2360
+ "rows": 250
2361
+ },
2362
+ "17": {
2363
+ "nonfinite": 0,
2364
+ "max_abs": 3.858363151550293,
2365
+ "rows": 250
2366
+ },
2367
+ "18": {
2368
+ "nonfinite": 0,
2369
+ "max_abs": 2.80590558052063,
2370
+ "rows": 250
2371
+ },
2372
+ "19": {
2373
+ "nonfinite": 0,
2374
+ "max_abs": 3.7625536918640137,
2375
+ "rows": 250
2376
+ },
2377
+ "20": {
2378
+ "nonfinite": 0,
2379
+ "max_abs": 3.0250000953674316,
2380
+ "rows": 250
2381
+ },
2382
+ "21": {
2383
+ "nonfinite": 0,
2384
+ "max_abs": 4.954375267028809,
2385
+ "rows": 250
2386
+ },
2387
+ "22": {
2388
+ "nonfinite": 0,
2389
+ "max_abs": 14.091765403747559,
2390
+ "rows": 250
2391
+ },
2392
+ "23": {
2393
+ "nonfinite": 0,
2394
+ "max_abs": 5.796649932861328,
2395
+ "rows": 250
2396
+ },
2397
+ "24": {
2398
+ "nonfinite": 0,
2399
+ "max_abs": 5.675963878631592,
2400
+ "rows": 250
2401
+ },
2402
+ "25": {
2403
+ "nonfinite": 0,
2404
+ "max_abs": 5.68497896194458,
2405
+ "rows": 250
2406
+ },
2407
+ "26": {
2408
+ "nonfinite": 0,
2409
+ "max_abs": 8.087140083312988,
2410
+ "rows": 250
2411
+ },
2412
+ "27": {
2413
+ "nonfinite": 0,
2414
+ "max_abs": 7.12436580657959,
2415
+ "rows": 250
2416
+ },
2417
+ "28": {
2418
+ "nonfinite": 0,
2419
+ "max_abs": 18.333961486816406,
2420
+ "rows": 250
2421
+ },
2422
+ "29": {
2423
+ "nonfinite": 0,
2424
+ "max_abs": 9.990346908569336,
2425
+ "rows": 250
2426
+ },
2427
+ "30": {
2428
+ "nonfinite": 0,
2429
+ "max_abs": 8.744108200073242,
2430
+ "rows": 250
2431
+ },
2432
+ "31": {
2433
+ "nonfinite": 0,
2434
+ "max_abs": 11.40385627746582,
2435
+ "rows": 250
2436
+ },
2437
+ "32": {
2438
+ "nonfinite": 0,
2439
+ "max_abs": 10.602338790893555,
2440
+ "rows": 250
2441
+ },
2442
+ "33": {
2443
+ "nonfinite": 0,
2444
+ "max_abs": 14.826345443725586,
2445
+ "rows": 250
2446
+ },
2447
+ "34": {
2448
+ "nonfinite": 0,
2449
+ "max_abs": 26.637428283691406,
2450
+ "rows": 250
2451
+ },
2452
+ "35": {
2453
+ "nonfinite": 0,
2454
+ "max_abs": 19.168657302856445,
2455
+ "rows": 250
2456
+ },
2457
+ "36": {
2458
+ "nonfinite": 0,
2459
+ "max_abs": 16.678123474121094,
2460
+ "rows": 250
2461
+ },
2462
+ "37": {
2463
+ "nonfinite": 0,
2464
+ "max_abs": 25.9617919921875,
2465
+ "rows": 250
2466
+ },
2467
+ "38": {
2468
+ "nonfinite": 0,
2469
+ "max_abs": 21.801483154296875,
2470
+ "rows": 250
2471
+ },
2472
+ "39": {
2473
+ "nonfinite": 0,
2474
+ "max_abs": 25.854284286499023,
2475
+ "rows": 250
2476
+ },
2477
+ "40": {
2478
+ "nonfinite": 0,
2479
+ "max_abs": 56.554222106933594,
2480
+ "rows": 250
2481
+ },
2482
+ "41": {
2483
+ "nonfinite": 0,
2484
+ "max_abs": 1509.88818359375,
2485
+ "rows": 250
2486
+ },
2487
+ "42": {
2488
+ "nonfinite": 0,
2489
+ "max_abs": 112.59537506103516,
2490
+ "rows": 250
2491
+ },
2492
+ "43": {
2493
+ "nonfinite": 0,
2494
+ "max_abs": 53.72162628173828,
2495
+ "rows": 250
2496
+ },
2497
+ "44": {
2498
+ "nonfinite": 0,
2499
+ "max_abs": 155.66590881347656,
2500
+ "rows": 250
2501
+ },
2502
+ "45": {
2503
+ "nonfinite": 0,
2504
+ "max_abs": 71.36011505126953,
2505
+ "rows": 250
2506
+ },
2507
+ "46": {
2508
+ "nonfinite": 0,
2509
+ "max_abs": 1866.5113525390625,
2510
+ "rows": 250
2511
+ },
2512
+ "47": {
2513
+ "nonfinite": 0,
2514
+ "max_abs": 3417.11279296875,
2515
+ "rows": 250
2516
+ }
2517
+ },
2518
+ "prefill_tok_s_clean": 947.6040044130663,
2519
+ "decode_tok_s": 25.51445672601131,
2520
+ "gen_text": " Use the following words: speculation, overproduction, tariffs, unemployment, poverty, banks, loans, shares, investors, confidence, spending, trade, economy,",
2521
+ "pagecache": {
2522
+ "files": 12,
2523
+ "bytes_fadvised": 89605407713,
2524
+ "cached_before": 21865779200,
2525
+ "cached_after": 5863587840,
2526
+ "avail_before": 31901065216,
2527
+ "avail_after": 31912345600,
2528
+ "resident_before": -1,
2529
+ "resident_after": -1
2530
+ },
2531
+ "torch_peak_gib": 88.70163536071777,
2532
+ "host_peak_gib": 86.00949478149414,
2533
+ "wall_s": 730.006758928299
2534
+ }
2535
+ }
eval/eval-overflow-idiv47-wiki64.json ADDED
@@ -0,0 +1,861 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "on": {
3
+ "mode": "on",
4
+ "t_load": 20.452963829040527,
5
+ "fp32_mlp_layers": [],
6
+ "fp32_mlp_spec": "None",
7
+ "env": {
8
+ "EXL3_MIMO_FP32_MLP_LAYERS": null,
9
+ "EXL3_MIMO_FP32_MLP_FUSED": null,
10
+ "EXL3_MIMO_MLP_ACT_LIMIT": null
11
+ },
12
+ "rows": 64,
13
+ "row_len": 2048,
14
+ "nonfinite_rows": 0,
15
+ "nonfinite_row_idx": [],
16
+ "mlp_nonfinite_rows": 0,
17
+ "mlp_nonfinite_row_idx": [],
18
+ "nonfinite_positions": 0,
19
+ "nonfinite_logit_values": 0,
20
+ "scored_tokens": 131008,
21
+ "excluded_tokens": 0,
22
+ "perplexity": 5.401255149023906,
23
+ "prefill_tok_s": 897.4421687860723,
24
+ "per_row": [
25
+ {
26
+ "row": 0,
27
+ "tokens": 2048,
28
+ "nonfinite_positions": 0,
29
+ "nonfinite_logits": 0,
30
+ "s": 3.3031835556030273,
31
+ "layers": {},
32
+ "avail_gib": 28.38677978515625
33
+ },
34
+ {
35
+ "row": 1,
36
+ "tokens": 2048,
37
+ "nonfinite_positions": 0,
38
+ "nonfinite_logits": 0,
39
+ "s": 2.2504191398620605,
40
+ "layers": {},
41
+ "avail_gib": 28.386825561523438
42
+ },
43
+ {
44
+ "row": 2,
45
+ "tokens": 2048,
46
+ "nonfinite_positions": 0,
47
+ "nonfinite_logits": 0,
48
+ "s": 2.261091947555542,
49
+ "layers": {},
50
+ "avail_gib": 28.386947631835938
51
+ },
52
+ {
53
+ "row": 3,
54
+ "tokens": 2048,
55
+ "nonfinite_positions": 0,
56
+ "nonfinite_logits": 0,
57
+ "s": 2.2396044731140137,
58
+ "layers": {},
59
+ "avail_gib": 28.386825561523438
60
+ },
61
+ {
62
+ "row": 4,
63
+ "tokens": 2048,
64
+ "nonfinite_positions": 0,
65
+ "nonfinite_logits": 0,
66
+ "s": 2.258612871170044,
67
+ "layers": {},
68
+ "avail_gib": 28.38714599609375
69
+ },
70
+ {
71
+ "row": 5,
72
+ "tokens": 2048,
73
+ "nonfinite_positions": 0,
74
+ "nonfinite_logits": 0,
75
+ "s": 2.2664546966552734,
76
+ "layers": {},
77
+ "avail_gib": 28.38714599609375
78
+ },
79
+ {
80
+ "row": 6,
81
+ "tokens": 2048,
82
+ "nonfinite_positions": 0,
83
+ "nonfinite_logits": 0,
84
+ "s": 2.271862268447876,
85
+ "layers": {},
86
+ "avail_gib": 28.3868408203125
87
+ },
88
+ {
89
+ "row": 7,
90
+ "tokens": 2048,
91
+ "nonfinite_positions": 0,
92
+ "nonfinite_logits": 0,
93
+ "s": 2.270273208618164,
94
+ "layers": {},
95
+ "avail_gib": 28.386611938476562
96
+ },
97
+ {
98
+ "row": 8,
99
+ "tokens": 2048,
100
+ "nonfinite_positions": 0,
101
+ "nonfinite_logits": 0,
102
+ "s": 2.2617266178131104,
103
+ "layers": {},
104
+ "avail_gib": 28.386734008789062
105
+ },
106
+ {
107
+ "row": 9,
108
+ "tokens": 2048,
109
+ "nonfinite_positions": 0,
110
+ "nonfinite_logits": 0,
111
+ "s": 2.2640647888183594,
112
+ "layers": {},
113
+ "avail_gib": 28.38663101196289
114
+ },
115
+ {
116
+ "row": 10,
117
+ "tokens": 2048,
118
+ "nonfinite_positions": 0,
119
+ "nonfinite_logits": 0,
120
+ "s": 2.2583391666412354,
121
+ "layers": {},
122
+ "avail_gib": 28.38663101196289
123
+ },
124
+ {
125
+ "row": 11,
126
+ "tokens": 2048,
127
+ "nonfinite_positions": 0,
128
+ "nonfinite_logits": 0,
129
+ "s": 2.276468515396118,
130
+ "layers": {},
131
+ "avail_gib": 28.386390686035156
132
+ },
133
+ {
134
+ "row": 12,
135
+ "tokens": 2048,
136
+ "nonfinite_positions": 0,
137
+ "nonfinite_logits": 0,
138
+ "s": 2.261735439300537,
139
+ "layers": {},
140
+ "avail_gib": 28.40896987915039
141
+ },
142
+ {
143
+ "row": 13,
144
+ "tokens": 2048,
145
+ "nonfinite_positions": 0,
146
+ "nonfinite_logits": 0,
147
+ "s": 2.2721424102783203,
148
+ "layers": {},
149
+ "avail_gib": 28.415027618408203
150
+ },
151
+ {
152
+ "row": 14,
153
+ "tokens": 2048,
154
+ "nonfinite_positions": 0,
155
+ "nonfinite_logits": 0,
156
+ "s": 2.272073268890381,
157
+ "layers": {},
158
+ "avail_gib": 28.42108917236328
159
+ },
160
+ {
161
+ "row": 15,
162
+ "tokens": 2048,
163
+ "nonfinite_positions": 0,
164
+ "nonfinite_logits": 0,
165
+ "s": 2.2641043663024902,
166
+ "layers": {},
167
+ "avail_gib": 28.426803588867188
168
+ },
169
+ {
170
+ "row": 16,
171
+ "tokens": 2048,
172
+ "nonfinite_positions": 0,
173
+ "nonfinite_logits": 0,
174
+ "s": 2.276360034942627,
175
+ "layers": {},
176
+ "avail_gib": 28.431537628173828
177
+ },
178
+ {
179
+ "row": 17,
180
+ "tokens": 2048,
181
+ "nonfinite_positions": 0,
182
+ "nonfinite_logits": 0,
183
+ "s": 2.266751527786255,
184
+ "layers": {},
185
+ "avail_gib": 28.43682098388672
186
+ },
187
+ {
188
+ "row": 18,
189
+ "tokens": 2048,
190
+ "nonfinite_positions": 0,
191
+ "nonfinite_logits": 0,
192
+ "s": 2.2413017749786377,
193
+ "layers": {},
194
+ "avail_gib": 28.443157196044922
195
+ },
196
+ {
197
+ "row": 19,
198
+ "tokens": 2048,
199
+ "nonfinite_positions": 0,
200
+ "nonfinite_logits": 0,
201
+ "s": 2.256202459335327,
202
+ "layers": {},
203
+ "avail_gib": 28.444713592529297
204
+ },
205
+ {
206
+ "row": 20,
207
+ "tokens": 2048,
208
+ "nonfinite_positions": 0,
209
+ "nonfinite_logits": 0,
210
+ "s": 2.2460057735443115,
211
+ "layers": {},
212
+ "avail_gib": 28.454132080078125
213
+ },
214
+ {
215
+ "row": 21,
216
+ "tokens": 2048,
217
+ "nonfinite_positions": 0,
218
+ "nonfinite_logits": 0,
219
+ "s": 2.2562952041625977,
220
+ "layers": {},
221
+ "avail_gib": 28.439990997314453
222
+ },
223
+ {
224
+ "row": 22,
225
+ "tokens": 2048,
226
+ "nonfinite_positions": 0,
227
+ "nonfinite_logits": 0,
228
+ "s": 2.2606594562530518,
229
+ "layers": {},
230
+ "avail_gib": 28.453662872314453
231
+ },
232
+ {
233
+ "row": 23,
234
+ "tokens": 2048,
235
+ "nonfinite_positions": 0,
236
+ "nonfinite_logits": 0,
237
+ "s": 2.260310173034668,
238
+ "layers": {},
239
+ "avail_gib": 28.467552185058594
240
+ },
241
+ {
242
+ "row": 24,
243
+ "tokens": 2048,
244
+ "nonfinite_positions": 0,
245
+ "nonfinite_logits": 0,
246
+ "s": 2.2555243968963623,
247
+ "layers": {},
248
+ "avail_gib": 28.47498321533203
249
+ },
250
+ {
251
+ "row": 25,
252
+ "tokens": 2048,
253
+ "nonfinite_positions": 0,
254
+ "nonfinite_logits": 0,
255
+ "s": 2.2642104625701904,
256
+ "layers": {},
257
+ "avail_gib": 28.478965759277344
258
+ },
259
+ {
260
+ "row": 26,
261
+ "tokens": 2048,
262
+ "nonfinite_positions": 0,
263
+ "nonfinite_logits": 0,
264
+ "s": 2.266909122467041,
265
+ "layers": {},
266
+ "avail_gib": 28.481117248535156
267
+ },
268
+ {
269
+ "row": 27,
270
+ "tokens": 2048,
271
+ "nonfinite_positions": 0,
272
+ "nonfinite_logits": 0,
273
+ "s": 2.271202325820923,
274
+ "layers": {},
275
+ "avail_gib": 28.47915267944336
276
+ },
277
+ {
278
+ "row": 28,
279
+ "tokens": 2048,
280
+ "nonfinite_positions": 0,
281
+ "nonfinite_logits": 0,
282
+ "s": 2.2683095932006836,
283
+ "layers": {},
284
+ "avail_gib": 28.479488372802734
285
+ },
286
+ {
287
+ "row": 29,
288
+ "tokens": 2048,
289
+ "nonfinite_positions": 0,
290
+ "nonfinite_logits": 0,
291
+ "s": 2.2775771617889404,
292
+ "layers": {},
293
+ "avail_gib": 28.482646942138672
294
+ },
295
+ {
296
+ "row": 30,
297
+ "tokens": 2048,
298
+ "nonfinite_positions": 0,
299
+ "nonfinite_logits": 0,
300
+ "s": 2.2759287357330322,
301
+ "layers": {},
302
+ "avail_gib": 28.482372283935547
303
+ },
304
+ {
305
+ "row": 31,
306
+ "tokens": 2048,
307
+ "nonfinite_positions": 0,
308
+ "nonfinite_logits": 0,
309
+ "s": 2.267073631286621,
310
+ "layers": {},
311
+ "avail_gib": 28.482906341552734
312
+ },
313
+ {
314
+ "row": 32,
315
+ "tokens": 2048,
316
+ "nonfinite_positions": 0,
317
+ "nonfinite_logits": 0,
318
+ "s": 2.270385503768921,
319
+ "layers": {},
320
+ "avail_gib": 28.48505401611328
321
+ },
322
+ {
323
+ "row": 33,
324
+ "tokens": 2048,
325
+ "nonfinite_positions": 0,
326
+ "nonfinite_logits": 0,
327
+ "s": 2.2820167541503906,
328
+ "layers": {},
329
+ "avail_gib": 28.48544692993164
330
+ },
331
+ {
332
+ "row": 34,
333
+ "tokens": 2048,
334
+ "nonfinite_positions": 0,
335
+ "nonfinite_logits": 0,
336
+ "s": 2.2677550315856934,
337
+ "layers": {},
338
+ "avail_gib": 28.48526382446289
339
+ },
340
+ {
341
+ "row": 35,
342
+ "tokens": 2048,
343
+ "nonfinite_positions": 0,
344
+ "nonfinite_logits": 0,
345
+ "s": 2.2561681270599365,
346
+ "layers": {},
347
+ "avail_gib": 28.48797607421875
348
+ },
349
+ {
350
+ "row": 36,
351
+ "tokens": 2048,
352
+ "nonfinite_positions": 0,
353
+ "nonfinite_logits": 0,
354
+ "s": 2.270458459854126,
355
+ "layers": {},
356
+ "avail_gib": 28.48809814453125
357
+ },
358
+ {
359
+ "row": 37,
360
+ "tokens": 2048,
361
+ "nonfinite_positions": 0,
362
+ "nonfinite_logits": 0,
363
+ "s": 2.25923490524292,
364
+ "layers": {},
365
+ "avail_gib": 28.489154815673828
366
+ },
367
+ {
368
+ "row": 38,
369
+ "tokens": 2048,
370
+ "nonfinite_positions": 0,
371
+ "nonfinite_logits": 0,
372
+ "s": 2.265436887741089,
373
+ "layers": {},
374
+ "avail_gib": 28.489154815673828
375
+ },
376
+ {
377
+ "row": 39,
378
+ "tokens": 2048,
379
+ "nonfinite_positions": 0,
380
+ "nonfinite_logits": 0,
381
+ "s": 2.264444589614868,
382
+ "layers": {},
383
+ "avail_gib": 28.489322662353516
384
+ },
385
+ {
386
+ "row": 40,
387
+ "tokens": 2048,
388
+ "nonfinite_positions": 0,
389
+ "nonfinite_logits": 0,
390
+ "s": 2.2622859477996826,
391
+ "layers": {},
392
+ "avail_gib": 28.48931884765625
393
+ },
394
+ {
395
+ "row": 41,
396
+ "tokens": 2048,
397
+ "nonfinite_positions": 0,
398
+ "nonfinite_logits": 0,
399
+ "s": 2.262871742248535,
400
+ "layers": {},
401
+ "avail_gib": 28.488906860351562
402
+ },
403
+ {
404
+ "row": 42,
405
+ "tokens": 2048,
406
+ "nonfinite_positions": 0,
407
+ "nonfinite_logits": 0,
408
+ "s": 2.268205165863037,
409
+ "layers": {},
410
+ "avail_gib": 28.489765167236328
411
+ },
412
+ {
413
+ "row": 43,
414
+ "tokens": 2048,
415
+ "nonfinite_positions": 0,
416
+ "nonfinite_logits": 0,
417
+ "s": 2.280674695968628,
418
+ "layers": {},
419
+ "avail_gib": 28.48891830444336
420
+ },
421
+ {
422
+ "row": 44,
423
+ "tokens": 2048,
424
+ "nonfinite_positions": 0,
425
+ "nonfinite_logits": 0,
426
+ "s": 2.2669315338134766,
427
+ "layers": {},
428
+ "avail_gib": 28.488819122314453
429
+ },
430
+ {
431
+ "row": 45,
432
+ "tokens": 2048,
433
+ "nonfinite_positions": 0,
434
+ "nonfinite_logits": 0,
435
+ "s": 2.2688441276550293,
436
+ "layers": {},
437
+ "avail_gib": 28.489154815673828
438
+ },
439
+ {
440
+ "row": 46,
441
+ "tokens": 2048,
442
+ "nonfinite_positions": 0,
443
+ "nonfinite_logits": 0,
444
+ "s": 2.2587730884552,
445
+ "layers": {},
446
+ "avail_gib": 28.488914489746094
447
+ },
448
+ {
449
+ "row": 47,
450
+ "tokens": 2048,
451
+ "nonfinite_positions": 0,
452
+ "nonfinite_logits": 0,
453
+ "s": 2.2675859928131104,
454
+ "layers": {},
455
+ "avail_gib": 28.488250732421875
456
+ },
457
+ {
458
+ "row": 48,
459
+ "tokens": 2048,
460
+ "nonfinite_positions": 0,
461
+ "nonfinite_logits": 0,
462
+ "s": 2.2670328617095947,
463
+ "layers": {},
464
+ "avail_gib": 28.494060516357422
465
+ },
466
+ {
467
+ "row": 49,
468
+ "tokens": 2048,
469
+ "nonfinite_positions": 0,
470
+ "nonfinite_logits": 0,
471
+ "s": 2.261099338531494,
472
+ "layers": {},
473
+ "avail_gib": 28.494060516357422
474
+ },
475
+ {
476
+ "row": 50,
477
+ "tokens": 2048,
478
+ "nonfinite_positions": 0,
479
+ "nonfinite_logits": 0,
480
+ "s": 2.2548439502716064,
481
+ "layers": {},
482
+ "avail_gib": 28.494060516357422
483
+ },
484
+ {
485
+ "row": 51,
486
+ "tokens": 2048,
487
+ "nonfinite_positions": 0,
488
+ "nonfinite_logits": 0,
489
+ "s": 2.281060218811035,
490
+ "layers": {},
491
+ "avail_gib": 28.497692108154297
492
+ },
493
+ {
494
+ "row": 52,
495
+ "tokens": 2048,
496
+ "nonfinite_positions": 0,
497
+ "nonfinite_logits": 0,
498
+ "s": 2.259843349456787,
499
+ "layers": {},
500
+ "avail_gib": 28.498416900634766
501
+ },
502
+ {
503
+ "row": 53,
504
+ "tokens": 2048,
505
+ "nonfinite_positions": 0,
506
+ "nonfinite_logits": 0,
507
+ "s": 2.257277488708496,
508
+ "layers": {},
509
+ "avail_gib": 28.500167846679688
510
+ },
511
+ {
512
+ "row": 54,
513
+ "tokens": 2048,
514
+ "nonfinite_positions": 0,
515
+ "nonfinite_logits": 0,
516
+ "s": 2.273444890975952,
517
+ "layers": {},
518
+ "avail_gib": 28.499771118164062
519
+ },
520
+ {
521
+ "row": 55,
522
+ "tokens": 2048,
523
+ "nonfinite_positions": 0,
524
+ "nonfinite_logits": 0,
525
+ "s": 2.265801429748535,
526
+ "layers": {},
527
+ "avail_gib": 28.503440856933594
528
+ },
529
+ {
530
+ "row": 56,
531
+ "tokens": 2048,
532
+ "nonfinite_positions": 0,
533
+ "nonfinite_logits": 0,
534
+ "s": 2.2734181880950928,
535
+ "layers": {},
536
+ "avail_gib": 28.498462677001953
537
+ },
538
+ {
539
+ "row": 57,
540
+ "tokens": 2048,
541
+ "nonfinite_positions": 0,
542
+ "nonfinite_logits": 0,
543
+ "s": 2.2797374725341797,
544
+ "layers": {},
545
+ "avail_gib": 28.501014709472656
546
+ },
547
+ {
548
+ "row": 58,
549
+ "tokens": 2048,
550
+ "nonfinite_positions": 0,
551
+ "nonfinite_logits": 0,
552
+ "s": 2.264260768890381,
553
+ "layers": {},
554
+ "avail_gib": 28.498573303222656
555
+ },
556
+ {
557
+ "row": 59,
558
+ "tokens": 2048,
559
+ "nonfinite_positions": 0,
560
+ "nonfinite_logits": 0,
561
+ "s": 2.2819690704345703,
562
+ "layers": {},
563
+ "avail_gib": 28.501136779785156
564
+ },
565
+ {
566
+ "row": 60,
567
+ "tokens": 2048,
568
+ "nonfinite_positions": 0,
569
+ "nonfinite_logits": 0,
570
+ "s": 2.260755777359009,
571
+ "layers": {},
572
+ "avail_gib": 28.49835968017578
573
+ },
574
+ {
575
+ "row": 61,
576
+ "tokens": 2048,
577
+ "nonfinite_positions": 0,
578
+ "nonfinite_logits": 0,
579
+ "s": 2.284695863723755,
580
+ "layers": {},
581
+ "avail_gib": 28.49734115600586
582
+ },
583
+ {
584
+ "row": 62,
585
+ "tokens": 2048,
586
+ "nonfinite_positions": 0,
587
+ "nonfinite_logits": 0,
588
+ "s": 2.270501136779785,
589
+ "layers": {},
590
+ "avail_gib": 28.49892807006836
591
+ },
592
+ {
593
+ "row": 63,
594
+ "tokens": 2048,
595
+ "nonfinite_positions": 0,
596
+ "nonfinite_logits": 0,
597
+ "s": 2.280043601989746,
598
+ "layers": {},
599
+ "avail_gib": 28.497615814208984
600
+ }
601
+ ],
602
+ "layer_probes": {
603
+ "0": {
604
+ "nonfinite": 0,
605
+ "max_abs": 0.7595411539077759,
606
+ "rows": 64
607
+ },
608
+ "1": {
609
+ "nonfinite": 0,
610
+ "max_abs": 0.13004069030284882,
611
+ "rows": 64
612
+ },
613
+ "2": {
614
+ "nonfinite": 0,
615
+ "max_abs": 0.20078472793102264,
616
+ "rows": 64
617
+ },
618
+ "3": {
619
+ "nonfinite": 0,
620
+ "max_abs": 0.28598278760910034,
621
+ "rows": 64
622
+ },
623
+ "4": {
624
+ "nonfinite": 0,
625
+ "max_abs": 0.6614945530891418,
626
+ "rows": 64
627
+ },
628
+ "5": {
629
+ "nonfinite": 0,
630
+ "max_abs": 0.5550518035888672,
631
+ "rows": 64
632
+ },
633
+ "6": {
634
+ "nonfinite": 0,
635
+ "max_abs": 0.9114603400230408,
636
+ "rows": 64
637
+ },
638
+ "7": {
639
+ "nonfinite": 0,
640
+ "max_abs": 9.167235374450684,
641
+ "rows": 64
642
+ },
643
+ "8": {
644
+ "nonfinite": 0,
645
+ "max_abs": 4.132124900817871,
646
+ "rows": 64
647
+ },
648
+ "9": {
649
+ "nonfinite": 0,
650
+ "max_abs": 3.3539884090423584,
651
+ "rows": 64
652
+ },
653
+ "10": {
654
+ "nonfinite": 0,
655
+ "max_abs": 57.97081756591797,
656
+ "rows": 64
657
+ },
658
+ "11": {
659
+ "nonfinite": 0,
660
+ "max_abs": 63.06317901611328,
661
+ "rows": 64
662
+ },
663
+ "12": {
664
+ "nonfinite": 0,
665
+ "max_abs": 3.366964340209961,
666
+ "rows": 64
667
+ },
668
+ "13": {
669
+ "nonfinite": 0,
670
+ "max_abs": 2.348827600479126,
671
+ "rows": 64
672
+ },
673
+ "14": {
674
+ "nonfinite": 0,
675
+ "max_abs": 2.6969079971313477,
676
+ "rows": 64
677
+ },
678
+ "15": {
679
+ "nonfinite": 0,
680
+ "max_abs": 2.7022764682769775,
681
+ "rows": 64
682
+ },
683
+ "16": {
684
+ "nonfinite": 0,
685
+ "max_abs": 1293.156494140625,
686
+ "rows": 64
687
+ },
688
+ "17": {
689
+ "nonfinite": 0,
690
+ "max_abs": 2.4514408111572266,
691
+ "rows": 64
692
+ },
693
+ "18": {
694
+ "nonfinite": 0,
695
+ "max_abs": 2.0526301860809326,
696
+ "rows": 64
697
+ },
698
+ "19": {
699
+ "nonfinite": 0,
700
+ "max_abs": 3.2425639629364014,
701
+ "rows": 64
702
+ },
703
+ "20": {
704
+ "nonfinite": 0,
705
+ "max_abs": 3.000279664993286,
706
+ "rows": 64
707
+ },
708
+ "21": {
709
+ "nonfinite": 0,
710
+ "max_abs": 3.818528890609741,
711
+ "rows": 64
712
+ },
713
+ "22": {
714
+ "nonfinite": 0,
715
+ "max_abs": 8.523226737976074,
716
+ "rows": 64
717
+ },
718
+ "23": {
719
+ "nonfinite": 0,
720
+ "max_abs": 6.562700271606445,
721
+ "rows": 64
722
+ },
723
+ "24": {
724
+ "nonfinite": 0,
725
+ "max_abs": 4.881248474121094,
726
+ "rows": 64
727
+ },
728
+ "25": {
729
+ "nonfinite": 0,
730
+ "max_abs": 5.375034809112549,
731
+ "rows": 64
732
+ },
733
+ "26": {
734
+ "nonfinite": 0,
735
+ "max_abs": 7.3661789894104,
736
+ "rows": 64
737
+ },
738
+ "27": {
739
+ "nonfinite": 0,
740
+ "max_abs": 7.176695823669434,
741
+ "rows": 64
742
+ },
743
+ "28": {
744
+ "nonfinite": 0,
745
+ "max_abs": 15.98316478729248,
746
+ "rows": 64
747
+ },
748
+ "29": {
749
+ "nonfinite": 0,
750
+ "max_abs": 9.4517240524292,
751
+ "rows": 64
752
+ },
753
+ "30": {
754
+ "nonfinite": 0,
755
+ "max_abs": 9.653580665588379,
756
+ "rows": 64
757
+ },
758
+ "31": {
759
+ "nonfinite": 0,
760
+ "max_abs": 9.589120864868164,
761
+ "rows": 64
762
+ },
763
+ "32": {
764
+ "nonfinite": 0,
765
+ "max_abs": 9.34403133392334,
766
+ "rows": 64
767
+ },
768
+ "33": {
769
+ "nonfinite": 0,
770
+ "max_abs": 10.151235580444336,
771
+ "rows": 64
772
+ },
773
+ "34": {
774
+ "nonfinite": 0,
775
+ "max_abs": 22.586336135864258,
776
+ "rows": 64
777
+ },
778
+ "35": {
779
+ "nonfinite": 0,
780
+ "max_abs": 15.88249683380127,
781
+ "rows": 64
782
+ },
783
+ "36": {
784
+ "nonfinite": 0,
785
+ "max_abs": 15.582351684570312,
786
+ "rows": 64
787
+ },
788
+ "37": {
789
+ "nonfinite": 0,
790
+ "max_abs": 23.97977066040039,
791
+ "rows": 64
792
+ },
793
+ "38": {
794
+ "nonfinite": 0,
795
+ "max_abs": 23.660140991210938,
796
+ "rows": 64
797
+ },
798
+ "39": {
799
+ "nonfinite": 0,
800
+ "max_abs": 26.511369705200195,
801
+ "rows": 64
802
+ },
803
+ "40": {
804
+ "nonfinite": 0,
805
+ "max_abs": 57.97829818725586,
806
+ "rows": 64
807
+ },
808
+ "41": {
809
+ "nonfinite": 0,
810
+ "max_abs": 1302.3048095703125,
811
+ "rows": 64
812
+ },
813
+ "42": {
814
+ "nonfinite": 0,
815
+ "max_abs": 108.6211166381836,
816
+ "rows": 64
817
+ },
818
+ "43": {
819
+ "nonfinite": 0,
820
+ "max_abs": 57.749969482421875,
821
+ "rows": 64
822
+ },
823
+ "44": {
824
+ "nonfinite": 0,
825
+ "max_abs": 43.385215759277344,
826
+ "rows": 64
827
+ },
828
+ "45": {
829
+ "nonfinite": 0,
830
+ "max_abs": 67.61186218261719,
831
+ "rows": 64
832
+ },
833
+ "46": {
834
+ "nonfinite": 0,
835
+ "max_abs": 1983.665771484375,
836
+ "rows": 64
837
+ },
838
+ "47": {
839
+ "nonfinite": 0,
840
+ "max_abs": 1731.05810546875,
841
+ "rows": 64
842
+ }
843
+ },
844
+ "prefill_tok_s_clean": 942.968479913356,
845
+ "decode_tok_s": 29.976999377087868,
846
+ "gen_text": " Use the following words: speculation, overproduction, tariffs, unemployment, poverty, banks, loans, shares, investors, confidence, spending, trade, economy, recession, depression, crash, wealth, inequality, boom, bust, bust, collapse, decline, crisis, panic, unemployment, poverty, banks, loans, shares, investors, confidence, spending, trade, economy, recession, depression, crash, wealth, inequali",
847
+ "pagecache": {
848
+ "files": 12,
849
+ "bytes_fadvised": 89605407713,
850
+ "cached_before": 5458665472,
851
+ "cached_after": 5458665472,
852
+ "avail_before": 32145887232,
853
+ "avail_after": 32145899520,
854
+ "resident_before": -1,
855
+ "resident_after": -1
856
+ },
857
+ "torch_peak_gib": 88.70163536071777,
858
+ "host_peak_gib": 90.02111434936523,
859
+ "wall_s": 188.25873827934265
860
+ }
861
+ }
model-00011-of-00012.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:03feb0c410d6b50abe1f704681a360368ee640960a6e1ddd0b30b6a0d1f3d6e4
3
- size 6220502916
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:87e5e49c1d404ec711dc96b1f582d1b12da9220d829a6f6f84c9cac92392607e
3
+ size 8290864253
model-00012-of-00012.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:465e48eb404c85206a61c9a2400fdac907098c5f0354745eb8c375b4858c3cc9
3
- size 5357959367
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1285fdc2a63f35218f360261e6a97ef37e965c70984e16c641e28e6ec9142d4d
3
+ size 469027137
model.safetensors.index.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:95024ecbabbbb52a49b05ea902fa5922682eebc00b70a10940c51535422c954e
3
  size 13509487
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:763647a8ee428a08c428e37f029ad17e9dad2b42a966e4246b7e08f852f34e78
3
  size 13509487
quantization_config.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a86480a13eda9cb485cfb84286151ffd899902e690979992e388e498d5da34c7
3
- size 44679438
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0903a766d689e688ba55e18c9746df3412eac6b565e973c95ad5cdad8c40675d
3
+ size 44680974