| 0.00.039.723 I common_init_result: fitting params to device memory ... |
| 0.00.039.726 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on) |
| 0.00.396.902 W llama_model_loader: direct I/O is enabled, disabling mmap |
| 0.20.542.121 W llama_context: n_ctx_seq (2048) < n_ctx_train (262144) -- the full capacity of the model will not be utilized |
| 0.20.588.569 W common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable) |
| 0.20.783.826 I |
| 0.20.783.939 I system_info: n_threads = 16 (n_threads_batch = 16) / 32 | ROCm : NO_VMM = 1 | PEER_MAX_BATCH_SIZE = 128 | FA_ALL_QUANTS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | |
| 0.20.898.218 I kl_divergence: computing over 40 chunks, n_ctx=2048, batch_size=2048, n_seq=1 |
| 0.22.619.669 I kl_divergence: 1.72 seconds per pass - ETA 1.13 minutes |
|
|
| chunk PPL ln(PPL(Q)/PPL(base)) KL Divergence Δp RMS Same top p |
| 1 5.7545 ± 0.4426 0.01066 ± 0.01514 0.10310 ± 0.00565 9.252 ± 0.547 % 86.901 ± 1.055 % |
| 2 6.7692 ± 0.3702 0.01553 ± 0.01057 0.08979 ± 0.00354 8.313 ± 0.392 % 87.097 ± 0.741 % |
| 3 7.2118 ± 0.3286 0.02530 ± 0.00859 0.09142 ± 0.00371 7.924 ± 0.306 % 86.901 ± 0.609 % |
| 4 7.4361 ± 0.3008 0.02022 ± 0.00807 0.09891 ± 0.00569 8.314 ± 0.317 % 87.414 ± 0.519 % |
| 5 7.2981 ± 0.2661 0.02160 ± 0.00714 0.09587 ± 0.00463 8.276 ± 0.280 % 87.527 ± 0.462 % |
| 6 6.3222 ± 0.2036 0.02801 ± 0.00679 0.09876 ± 0.00434 9.164 ± 0.306 % 87.814 ± 0.418 % |
| 7 5.8837 ± 0.1729 0.02475 ± 0.00651 0.10562 ± 0.00468 9.504 ± 0.293 % 87.837 ± 0.386 % |
| 8 5.8219 ± 0.1591 0.02653 ± 0.00598 0.10395 ± 0.00416 9.429 ± 0.268 % 87.732 ± 0.363 % |
| 9 6.1195 ± 0.1584 0.02383 ± 0.00560 0.10437 ± 0.00375 9.307 ± 0.250 % 87.281 ± 0.347 % |
| 10 6.2384 ± 0.1544 0.02466 ± 0.00519 0.10032 ± 0.00340 9.080 ± 0.233 % 87.331 ± 0.329 % |
| 11 6.3092 ± 0.1486 0.02715 ± 0.00488 0.09769 ± 0.00312 8.996 ± 0.220 % 87.292 ± 0.314 % |
| 12 6.5586 ± 0.1493 0.02759 ± 0.00459 0.09446 ± 0.00286 8.773 ± 0.208 % 87.235 ± 0.301 % |
| 13 6.6005 ± 0.1440 0.02706 ± 0.00438 0.09304 ± 0.00266 8.721 ± 0.198 % 87.292 ± 0.289 % |
| 14 6.6554 ± 0.1398 0.02657 ± 0.00417 0.09137 ± 0.00249 8.562 ± 0.187 % 87.264 ± 0.279 % |
| 15 6.6958 ± 0.1359 0.02640 ± 0.00402 0.09068 ± 0.00234 8.515 ± 0.178 % 87.175 ± 0.270 % |
| 16 6.8581 ± 0.1350 0.02363 ± 0.00387 0.08962 ± 0.00221 8.427 ± 0.172 % 87.060 ± 0.262 % |
| 17 6.9067 ± 0.1315 0.02373 ± 0.00373 0.08844 ± 0.00209 8.335 ± 0.164 % 87.120 ± 0.254 % |
| 18 6.9978 ± 0.1297 0.02365 ± 0.00361 0.08829 ± 0.00199 8.313 ± 0.158 % 87.075 ± 0.247 % |
| 19 6.9441 ± 0.1257 0.02285 ± 0.00349 0.08712 ± 0.00190 8.208 ± 0.152 % 87.143 ± 0.240 % |
| 20 6.6745 ± 0.1169 0.02258 ± 0.00348 0.09166 ± 0.00187 8.508 ± 0.146 % 87.082 ± 0.234 % |
| 21 6.6839 ± 0.1140 0.02149 ± 0.00341 0.09200 ± 0.00180 8.487 ± 0.142 % 87.041 ± 0.229 % |
| 22 6.6958 ± 0.1115 0.02127 ± 0.00336 0.09301 ± 0.00182 8.516 ± 0.141 % 87.079 ± 0.224 % |
| 23 6.7561 ± 0.1101 0.02283 ± 0.00329 0.09310 ± 0.00178 8.515 ± 0.138 % 86.974 ± 0.219 % |
| 24 6.7528 ± 0.1075 0.02230 ± 0.00321 0.09247 ± 0.00172 8.468 ± 0.134 % 87.036 ± 0.214 % |
| 25 6.7907 ± 0.1060 0.02318 ± 0.00313 0.09180 ± 0.00165 8.416 ± 0.131 % 87.003 ± 0.210 % |
| 26 6.7642 ± 0.1034 0.02375 ± 0.00307 0.09231 ± 0.00162 8.481 ± 0.129 % 87.067 ± 0.206 % |
| 27 6.9325 ± 0.1047 0.02399 ± 0.00301 0.09164 ± 0.00159 8.400 ± 0.127 % 87.068 ± 0.202 % |
| 28 7.0252 ± 0.1045 0.02441 ± 0.00295 0.09094 ± 0.00155 8.369 ± 0.125 % 87.125 ± 0.198 % |
| 29 7.0215 ± 0.1026 0.02475 ± 0.00290 0.09104 ± 0.00151 8.421 ± 0.124 % 87.144 ± 0.194 % |
| 30 6.9731 ± 0.1000 0.02583 ± 0.00286 0.09121 ± 0.00147 8.431 ± 0.121 % 87.120 ± 0.191 % |
| 31 6.8743 ± 0.0967 0.02607 ± 0.00280 0.09057 ± 0.00143 8.408 ± 0.119 % 87.207 ± 0.188 % |
| 32 6.7690 ± 0.0936 0.02490 ± 0.00278 0.09337 ± 0.00160 8.561 ± 0.119 % 87.201 ± 0.185 % |
| 33 6.7036 ± 0.0910 0.02507 ± 0.00273 0.09280 ± 0.00156 8.553 ± 0.117 % 87.263 ± 0.181 % |
| 34 6.6897 ± 0.0893 0.02516 ± 0.00267 0.09179 ± 0.00151 8.490 ± 0.115 % 87.315 ± 0.178 % |
| 35 6.6976 ± 0.0881 0.02453 ± 0.00261 0.09113 ± 0.00148 8.437 ± 0.113 % 87.393 ± 0.175 % |
| 36 6.7103 ± 0.0872 0.02395 ± 0.00257 0.09082 ± 0.00144 8.395 ± 0.110 % 87.404 ± 0.173 % |
| 37 6.6143 ± 0.0845 0.02333 ± 0.00253 0.08990 ± 0.00141 8.356 ± 0.108 % 87.451 ± 0.170 % |
| 38 6.5394 ± 0.0821 0.02247 ± 0.00250 0.08996 ± 0.00138 8.378 ± 0.107 % 87.452 ± 0.168 % |
| 39 6.4598 ± 0.0798 0.02255 ± 0.00247 0.08964 ± 0.00135 8.375 ± 0.105 % 87.430 ± 0.166 % |
| 40 6.3681 ± 0.0773 0.02208 ± 0.00243 0.08897 ± 0.00132 8.370 ± 0.103 % 87.454 ± 0.164 % |
|
|
| ====== Perplexity statistics ====== |
| Mean PPL(Q) : 6.368075 ± 0.077317 |
| Mean PPL(base) : 6.228979 ± 0.075322 |
| Cor(ln(PPL(Q)), ln(PPL(base))): 97.99% |
| Mean ln(PPL(Q)/PPL(base)) : 0.022085 ± 0.002430 |
| Mean PPL(Q)/PPL(base) : 1.022330 ± 0.002484 |
| Mean PPL(Q)-PPL(base) : 0.139096 ± 0.015431 |
|
|
| ====== KL divergence statistics ====== |
| Mean KLD: 0.088974 ± 0.001318 |
| Maximum KLD: 14.587295 |
| 99.9% KLD: 2.825675 |
| 99.0% KLD: 0.784672 |
| 95.0% KLD: 0.301152 |
| 90.0% KLD: 0.188552 |
| Median KLD: 0.039738 |
| 10.0% KLD: 0.000526 |
| 5.0% KLD: 0.000124 |
| 1.0% KLD: -0.000056 |
| 0.1% KLD: -0.000307 |
| Minimum KLD: -0.000906 |
|
|
| ====== Token probability statistics ====== |
| Mean Δp: -0.439 ± 0.041 % |
| Maximum Δp: 97.378% |
| 99.9% Δp: 51.015% |
| 99.0% Δp: 22.861% |
| 95.0% Δp: 10.138% |
| 90.0% Δp: 5.884% |
| 75.0% Δp: 1.147% |
| Median Δp: -0.001% |
| 25.0% Δp: -1.525% |
| 10.0% Δp: -6.868% |
| 5.0% Δp: -12.274% |
| 1.0% Δp: -29.206% |
| 0.1% Δp: -67.613% |
| Minimum Δp: -98.545% |
| RMS Δp : 8.370 ± 0.103 % |
| Same top p: 87.454 ± 0.164 % |
|
|
|
|
