Hugging Face's logo

Nimbz
/
Split-31B

Safetensors
GGUF
gemma4
Merge
mergekit
roleplay
character-rp
creative writing
storytelling
conversational
low-refusal
nsfw
31B
gemma-4
Model card Files
xet
 Community
Nimbz commited on
Commit
d7cde1f
·
verified ·
1 Parent(s): 489ea05

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1217 -0
README.md CHANGED
@@ -1,3 +1,1220 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ base_model:
4
+ - google/gemma-4-31B-it
5
+ - ReadyArt/gemma-4-31B-it-scotoma-2
6
+ - bgg1996/Melinoe-Gemma4-31B-VL-heretic
7
+ - sophosympatheia/Glistening-Gem-31B-v2.1
8
+ - sophosympatheia/Glistening-Gem-31B-v2.0
9
+ - zerofata/G4-MeroMero-v2-31B
10
+ - Ateron/Gemma-4-Dark-Thoughts-V2-31B
11
+ base_model_relation: merge
12
+ tags:
13
+ - merge
14
+ - mergekit
15
+ - roleplay
16
+ - character-rp
17
+ - creative writing
18
+ - storytelling
19
+ - conversational
20
+ - low-refusal
21
+ - nsfw
22
+ - 31B
23
+ - gemma-4
24
  ---
25
+ <style>
26
+ .sx-shell {
27
+ position: relative; padding: 10px; border-radius: 14px;
28
+ background: linear-gradient(140deg, #7dd3fc22 0%, transparent 26%, transparent 74%, #fb718522 100%), #080d16;
29
+ box-shadow: 0 26px 80px rgba(0,0,0,0.72);
30
+ }
31
+ .sx-main {
32
+ position: relative; overflow: hidden; border-radius: 10px; padding: 44px 40px 36px;
33
+ background-color: #060a12;
34
+ background-image:
35
+ radial-gradient(ellipse 70% 55% at 4% -6%, #7dd3fc26 0%, transparent 60%),
36
+ radial-gradient(ellipse 65% 50% at 96% 104%, #fb718522 0%, transparent 60%),
37
+ radial-gradient(ellipse 90% 60% at 50% 50%, rgba(255,255,255,0.018) 0%, transparent 70%);
38
+ color: #c6d0dc;
39
+ font-family: 'Avenir Next','Segoe UI',Roboto,Helvetica,Arial,sans-serif;
40
+ line-height: 1.72;
41
+ }
42
+ .sx-main::before {
43
+ content: ""; position: absolute; inset: 0; pointer-events: none; z-index: 1;
44
+ background: linear-gradient(180deg, #7dd3fc14 0%, transparent 12%, transparent 88%, #fb718512 100%);
45
+ }
46
+
47
+ /* ---------- hero ---------- */
48
+ .sx-hero { text-align: center; position: relative; z-index: 5; }
49
+ .sx-title {
50
+ font-family: 'Iowan Old Style','Palatino Linotype',Georgia,serif;
51
+ font-size: 3.5rem; font-weight: 400; letter-spacing: 13px; margin: 28px 0 16px;
52
+ background: linear-gradient(100deg, #7dd3fc 0%, #e8f2fb 52%, #fb7185 100%);
53
+ -webkit-background-clip: text; background-clip: text;
54
+ -webkit-text-fill-color: transparent; color: #e8f2fb;
55
+ }
56
+ .sx-badge {
57
+ display: inline-block; color: #7dd3fc; border: 1px solid #7dd3fc44;
58
+ background: #7dd3fc12; text-transform: uppercase; letter-spacing: 4px;
59
+ font-size: 0.7rem; font-weight: 600; padding: 7px 20px; border-radius: 999px;
60
+ }
61
+ .sx-tag { color: #7f8b9c; font-style: italic; font-size: 1.12rem; margin-top: 16px;
62
+ font-family: 'Iowan Old Style','Palatino Linotype',Georgia,serif; }
63
+
64
+ /* ---------- separators: pill on alternating sides ---------- */
65
+ .sx-sep { display: flex; align-items: center; gap: 18px; margin: 46px 0 8px; position: relative; z-index: 5; }
66
+ .sx-sep.right { flex-direction: row-reverse; }
67
+ .sx-sep b {
68
+ font-family: 'Iowan Old Style','Palatino Linotype',Georgia,serif;
69
+ font-weight: 400; letter-spacing: 5px; font-size: 0.9rem; text-transform: uppercase;
70
+ color: #eef3f9; white-space: nowrap; padding: 6px 20px; border-radius: 999px;
71
+ background: rgba(12,19,32,0.80); border: 1px solid #1b2740;
72
+ }
73
+ .sx-sep b em { font-style: normal; color: #7dd3fc; margin-right: 9px; }
74
+ .sx-sep.right b em { color: #fb7185; }
75
+ .sx-sep i { flex: 1; height: 1px; border: 0; }
76
+ .sx-sep i { background: linear-gradient(90deg, transparent, #7dd3fc55 60%, #7dd3fc88); }
77
+ .sx-sep.right i { background: linear-gradient(90deg, #fb718588, #fb718555 40%, transparent); }
78
+
79
+ .sx-lead { font-size: 0.95rem; color: #7f8b9c; margin: 14px 2px 2px; position: relative; z-index: 5; }
80
+
81
+ /* ---------- chips ---------- */
82
+ .sx-chips { display: grid; grid-template-columns: repeat(auto-fit, minmax(150px,1fr)); gap: 12px; margin: 26px 0 4px; position: relative; z-index: 5; }
83
+ .sx-chip {
84
+ padding: 15px 14px; text-align: center; border-radius: 10px;
85
+ background: rgba(12,19,32,0.80); border: 1px solid #1b2740;
86
+ font-size: 0.78rem; letter-spacing: 2px; font-weight: 600; color: #eef3f9;
87
+ transition: transform .2s ease, box-shadow .2s ease, border-color .2s ease;
88
+ }
89
+ .sx-chip:nth-child(odd) { border-bottom: 2px solid #7dd3fc66; }
90
+ .sx-chip:nth-child(even) { border-bottom: 2px solid #fb718566; }
91
+ .sx-chip:hover { transform: translateY(-3px); border-color: #7dd3fc88; box-shadow: 0 10px 26px rgba(0,0,0,0.5); }
92
+
93
+ /* ---------- cards: asymmetric, side-anchored ---------- */
94
+ .sx-card {
95
+ position: relative; z-index: 5; margin: 22px 0; padding: 26px 30px;
96
+ background: linear-gradient(145deg, rgba(14,22,38,0.90), rgba(6,10,18,0.96));
97
+ border: 1px solid #1b2740;
98
+ box-shadow: 0 12px 34px rgba(0,0,0,0.5);
99
+ }
100
+ .sx-card.L { border-left: 5px solid #7dd3fc; border-radius: 4px 20px 20px 4px; margin-right: 26px; }
101
+ .sx-card.R { border-right: 5px solid #fb7185; border-radius: 20px 4px 4px 20px; margin-left: 26px; }
102
+ .sx-card.C { border-top: 3px solid #7dd3fc88; border-radius: 16px; }
103
+ .sx-card h3 {
104
+ font-family: 'Iowan Old Style','Palatino Linotype',Georgia,serif;
105
+ color: #eef3f9; margin: 0 0 12px; letter-spacing: 2px; font-weight: 400; font-size: 1.14rem;
106
+ }
107
+ .sx-card p { margin: 0 0 11px; font-size: 0.93rem; }
108
+ .sx-card p:last-child { margin-bottom: 0; }
109
+ .sx-rule { border: 0; height: 1px; margin: 15px 0;
110
+ background: linear-gradient(90deg, #7dd3fc55, #1b2740 55%, transparent); }
111
+ .sx-card.R .sx-rule { background: linear-gradient(90deg, transparent, #1b2740 45%, #fb718555); }
112
+
113
+ /* phase head: medallion + title, mirrored on R cards */
114
+ .sx-ph { display: flex; align-items: center; gap: 20px; margin-bottom: 14px; }
115
+ .sx-card.R .sx-ph { flex-direction: row-reverse; text-align: right; }
116
+ .sx-med {
117
+ flex: 0 0 76px; width: 76px; height: 76px; border-radius: 50%;
118
+ display: flex; align-items: center; justify-content: center;
119
+ font-family: 'Iowan Old Style','Palatino Linotype',Georgia,serif;
120
+ font-size: 1.7rem; color: #eef3f9;
121
+ background: radial-gradient(circle at 32% 28%, #7dd3fc33, transparent 62%), rgba(12,19,32,0.80);
122
+ border: 1px solid #7dd3fc66; box-shadow: 0 0 26px #7dd3fc33, inset 0 0 18px rgba(0,0,0,0.5);
123
+ }
124
+ .sx-card.R .sx-med { background: radial-gradient(circle at 32% 28%, #fb718533, transparent 62%), rgba(12,19,32,0.80);
125
+ border-color: #fb718566; box-shadow: 0 0 26px #fb718533, inset 0 0 18px rgba(0,0,0,0.5); }
126
+ .sx-ph h3 { margin: 0; }
127
+ .sx-ph small { display: block; color: #7f8b9c; letter-spacing: 3px; font-size: 0.66rem; text-transform: uppercase; margin-top: 3px; }
128
+
129
+ /* ---------- donor rows: CSS grid, so both blocks share one column ruler ---------- */
130
+ .sx-grid {
131
+ display: grid; grid-template-columns: minmax(0,0.85fr) minmax(0,1.15fr) minmax(0,1.6fr);
132
+ gap: 1px; background: #1b2740; border: 1px solid #1b2740; border-radius: 10px; overflow: hidden;
133
+ margin: 16px 0 4px; position: relative; z-index: 5; font-size: 0.87rem;
134
+ }
135
+ .sx-grid > div { background: #080e19; padding: 11px 14px; color: #b4bfcd; min-width: 0; overflow-wrap: anywhere; }
136
+ .sx-grid > .h { background: #0c1527; color: #7dd3fc; letter-spacing: 2px; font-size: 0.66rem; text-transform: uppercase; font-weight: 600; }
137
+ .sx-card.R .sx-grid > .h { color: #fb7185; }
138
+ .sx-grid > div b { color: #eef3f9; }
139
+
140
+ /* ---------- pick ---------- */
141
+ .sx-pick { display: grid; grid-template-columns: repeat(auto-fit, minmax(260px,1fr)); gap: 18px; margin: 20px 0; position: relative; z-index: 5; }
142
+ .sx-opt { padding: 22px 24px; border: 1px solid #1b2740; background: rgba(12,19,32,0.80); }
143
+ .sx-opt.a { border-radius: 4px 20px 20px 4px; border-left: 5px solid #7dd3fc; }
144
+ .sx-opt.b { border-radius: 20px 4px 4px 20px; border-right: 5px solid #fb7185; }
145
+ .sx-opt .n { display: block; font-family: 'Iowan Old Style','Palatino Linotype',Georgia,serif; font-size: 1.12rem; letter-spacing: 2px; color: #eef3f9; }
146
+ .sx-opt .n a { color: #eef3f9 !important; text-decoration: none !important; }
147
+ .sx-opt .k { display: block; font-size: 0.64rem; letter-spacing: 3px; text-transform: uppercase; margin: 7px 0 12px; }
148
+ .sx-opt .d { font-size: 0.9rem; color: #7f8b9c; }
149
+ .sx-opt.on { box-shadow: 0 0 36px #7dd3fc33; background: linear-gradient(145deg, #7dd3fc14, rgba(12,19,32,0.80)); }
150
+ .sx-opt.b.on { box-shadow: 0 0 36px #fb718533; background: linear-gradient(145deg, #fb718514, rgba(12,19,32,0.80)); }
151
+
152
+ /* ---------- misc ---------- */
153
+ .sx-btns { display: flex; gap: 12px; flex-wrap: wrap; margin: 14px 0 2px; position: relative; z-index: 5; }
154
+ .sx-btn { display: inline-block; padding: 10px 22px; border-radius: 999px; border: 1px solid #7dd3fc44; color: #7dd3fc !important; text-decoration: none !important; letter-spacing: 2px; font-size: 0.76rem; background: rgba(12,19,32,0.80); transition: transform .2s ease; }
155
+ .sx-btn:hover { transform: translateY(-2px); box-shadow: 0 8px 22px #7dd3fc33; }
156
+ .sx-btn.s { border-color: #fb718544; color: #fb7185 !important; }
157
+ .sx-kv { display: grid; grid-template-columns: minmax(0,1fr) minmax(0,1fr); gap: 1px; background: #1b2740; border: 1px solid #1b2740; border-radius: 10px; overflow: hidden; margin: 16px 0; position: relative; z-index: 5; }
158
+ .sx-kv > div { background: #080e19; padding: 10px 15px; font-size: 0.87rem; color: #b4bfcd; }
159
+ .sx-kv > .v { color: #eef3f9; font-weight: 600; }
160
+ .sx-kv > .h { background: #0c1527; color: #7dd3fc; letter-spacing: 2px; font-size: 0.66rem; text-transform: uppercase; }
161
+ .sx-det { margin: 4px 0 2px; }
162
+ .sx-det summary { cursor: pointer; color: #7dd3fc; letter-spacing: 2px; padding: 12px 0; font-size: 0.84rem; }
163
+ .sx-card.R .sx-det summary { color: #fb7185; }
164
+ .sx-cred ul { margin: 0; padding-left: 18px; }
165
+ .sx-cred li { margin-bottom: 9px; font-size: 0.9rem; }
166
+ .sx-foot { text-align: center; letter-spacing: 14px; margin-top: 40px; opacity: 0.45; position: relative; z-index: 5;
167
+ background: linear-gradient(90deg, #7dd3fc, #fb7185); -webkit-background-clip: text; background-clip: text; -webkit-text-fill-color: transparent; }
168
+ </style>
169
+ <div class="sx-shell">
170
+ <div class="sx-main">
171
+
172
+ <div class="sx-hero">
173
+ <video src="https://huggingface.co/Nimbz/Split-31B/resolve/main/split-31b-header.mp4" poster="https://huggingface.co/Nimbz/Split-31B/resolve/main/split-31b-header.png" width="100%" autoplay loop muted playsinline style="display:block;border-radius:10px;border:1px solid #7dd3fc44;box-shadow:0 0 70px #7dd3fc2e;margin:0 auto 4px;">
174
+ <img src="https://huggingface.co/Nimbz/Split-31B/resolve/main/split-31b-header.png" width="100%" style="display:block;border-radius:10px;border:1px solid #7dd3fc44;box-shadow:0 0 70px #7dd3fc2e;margin:0 auto 4px;">
175
+ </video>
176
+
177
+ <h1 class="sx-title">SPLIT</h1>
178
+ <div class="sx-badge">31B · Gemma 4 · Merge</div>
179
+ <div class="sx-tag">&ldquo;Remember the rules.&rdquo;</div>
180
+ </div>
181
+
182
+ <div class="sx-chips">
183
+ <div class="sx-chip">FOUR MODELS</div><div class="sx-chip">TWO PHASES</div><div class="sx-chip">ONE MIND</div><div class="sx-chip">EXPERIMENTAL</div><div class="sx-chip">PROMPT ADHERENT</div><div class="sx-chip">ROLEPLAY MERGE</div>
184
+ </div>
185
+
186
+ <div class="sx-sep"><i></i><b><em>&#9686;&#9687;</em>🔵 or 🔴</b><i></i></div>
187
+ <div class="sx-lead">Basically the same, they share a brain, yet differ on their speaking vein.</div>
188
+ <div class="sx-pick"><div class="sx-opt a on"><span class="n">SPLIT-31B</span><span class="k" style="color:#7dd3fc;">&#9686; &nbsp;you are here</span><span class="d">Pick this one if you want <b style="color:#eef3f9;">instruction adherence</b> for things like trackers or header and footer blocks. Expect a more structured output, and staying on track for long sessions.</span></div><div class="sx-opt b"><span class="n"><a href="https://huggingface.co/Blazed-Forge/Split-Untied-31B">SPLIT-UNTIED-31B</a></span><span class="k" style="color:#fb7185;">&#9687; &nbsp;the other head</span><span class="d">Pick this one if you want <b style="color:#eef3f9;">more creativity, a darker lean and more lewdness</b>, but only if you're ready to trade a little prompt adherence for it.</span></div></div>
189
+
190
+ <div class="sx-sep right"><i></i><b><em>&#11035;</em>GGUF quants</b><i></i></div>
191
+ <div class="sx-lead">Thank you to anyone providing quants!</div>
192
+ <div class="sx-btns">
193
+ <a class="sx-btn" href="https://huggingface.co/Nimbz/Split-31B-GGUF">STATIC</a>
194
+ <a class="sx-btn s" href="#">IMATRIX &middot; SOON</a>
195
+ </div>
196
+
197
+ <div class="sx-sep"><i></i><b><em>&#9776;</em>The merge</b><i></i></div>
198
+ <div class="sx-lead">Two <b style="color:#eef3f9;">della_linear</b> passes over <b style="color:#eef3f9;">gemma-4-31B-it</b>, on <a href="https://github.com/zerofata/mergekit/tree/gemma-4-support">zerofata&rsquo;s gemma-4-support fork</a>.</div>
199
+
200
+ <div class="sx-card L"><div class="sx-ph"><div class="sx-med">I</div><div><h3>Comprehension</h3><small>Of The Residual Stream</small></div></div><p>The theory: Adjust <b>q_proj + k_proj</b> for the attention mechanism, and alter <b>gate_proj + up_proj</b> for the knowledge routing.</p><div class="sx-grid"><div class="h">Model</div><div class="h">Purpose</div><div class="h">Scope</div><div><b>Melinoe-VL-heretic</b></div><div>persona tracking, world-state stability</div><div>q/k 0.45 &rarr; 0.30 by depth · spine q/k 0.32<br>gate/up 0.28 &rarr; 0.18</div><div><b>Glistening-Gem v2.1</b></div><div>format adherence, in-character</div><div>q/k 0.14 &rarr; 0.10 · spine 0.0<br>gate/up 0.06</div><div><b>MeroMero v2</b></div><div>swipe variety</div><div>gate/up only · 0.30 · layers 6-47 minus spine</div></div>
201
+ <details class="sx-det"><summary>&#9656; &nbsp;Phase 1: della_linear</summary>
202
+
203
+ ```yaml
204
+ merge_method: della_linear
205
+ base_model: ./google/gemma-4-31B-it
206
+ dtype: bfloat16
207
+ tokenizer_source: base
208
+ parameters:
209
+ normalize: false
210
+ epsilon: 0.05
211
+ lambda: 1.0
212
+ int8_mask: true
213
+
214
+ models:
215
+ - model: ./bgg1996/Melinoe-Gemma4-31B-VL-heretic
216
+ parameters:
217
+ density:
218
+ - { filter: norm, value: 1.0 }
219
+ - { filter: layer_scalar, value: 1.0 }
220
+ - { value: 0.85 }
221
+ weight:
222
+ - { filter: embed_tokens, value: 0.0 }
223
+ - { filter: lm_head, value: 0.0 }
224
+ - { filter: embed_vision, value: 0.0 }
225
+ - { filter: vision_tower, value: 0.0 }
226
+ - { filter: multi_modal_projector, value: 0.0 }
227
+ - { filter: model.language_model.layers.5.self_attn.q_proj, value: 0.32 }
228
+ - { filter: model.language_model.layers.5.self_attn.k_proj, value: 0.32 }
229
+ - { filter: model.language_model.layers.11.self_attn.q_proj, value: 0.32 }
230
+ - { filter: model.language_model.layers.11.self_attn.k_proj, value: 0.32 }
231
+ - { filter: model.language_model.layers.17.self_attn.q_proj, value: 0.32 }
232
+ - { filter: model.language_model.layers.17.self_attn.k_proj, value: 0.32 }
233
+ - { filter: model.language_model.layers.23.self_attn.q_proj, value: 0.32 }
234
+ - { filter: model.language_model.layers.23.self_attn.k_proj, value: 0.32 }
235
+ - { filter: model.language_model.layers.29.self_attn.q_proj, value: 0.32 }
236
+ - { filter: model.language_model.layers.29.self_attn.k_proj, value: 0.32 }
237
+ - { filter: model.language_model.layers.35.self_attn.q_proj, value: 0.32 }
238
+ - { filter: model.language_model.layers.35.self_attn.k_proj, value: 0.32 }
239
+ - { filter: model.language_model.layers.41.self_attn.q_proj, value: 0.32 }
240
+ - { filter: model.language_model.layers.41.self_attn.k_proj, value: 0.32 }
241
+ - { filter: model.language_model.layers.47.self_attn.q_proj, value: 0.32 }
242
+ - { filter: model.language_model.layers.47.self_attn.k_proj, value: 0.32 }
243
+ - { filter: model.language_model.layers.53.self_attn.q_proj, value: 0.32 }
244
+ - { filter: model.language_model.layers.53.self_attn.k_proj, value: 0.32 }
245
+ - { filter: model.language_model.layers.59.self_attn.q_proj, value: 0.32 }
246
+ - { filter: model.language_model.layers.59.self_attn.k_proj, value: 0.32 }
247
+ - { filter: model.language_model.layers.0.self_attn.q_proj, value: 0.45 }
248
+ - { filter: model.language_model.layers.0.self_attn.k_proj, value: 0.45 }
249
+ - { filter: model.language_model.layers.1.self_attn.q_proj, value: 0.45 }
250
+ - { filter: model.language_model.layers.1.self_attn.k_proj, value: 0.45 }
251
+ - { filter: model.language_model.layers.2.self_attn.q_proj, value: 0.45 }
252
+ - { filter: model.language_model.layers.2.self_attn.k_proj, value: 0.45 }
253
+ - { filter: model.language_model.layers.3.self_attn.q_proj, value: 0.45 }
254
+ - { filter: model.language_model.layers.3.self_attn.k_proj, value: 0.45 }
255
+ - { filter: model.language_model.layers.4.self_attn.q_proj, value: 0.45 }
256
+ - { filter: model.language_model.layers.4.self_attn.k_proj, value: 0.45 }
257
+ - { filter: model.language_model.layers.5.self_attn.q_proj, value: 0.45 }
258
+ - { filter: model.language_model.layers.5.self_attn.k_proj, value: 0.45 }
259
+ - { filter: model.language_model.layers.6.self_attn.q_proj, value: 0.45 }
260
+ - { filter: model.language_model.layers.6.self_attn.k_proj, value: 0.45 }
261
+ - { filter: model.language_model.layers.7.self_attn.q_proj, value: 0.45 }
262
+ - { filter: model.language_model.layers.7.self_attn.k_proj, value: 0.45 }
263
+ - { filter: model.language_model.layers.8.self_attn.q_proj, value: 0.45 }
264
+ - { filter: model.language_model.layers.8.self_attn.k_proj, value: 0.45 }
265
+ - { filter: model.language_model.layers.9.self_attn.q_proj, value: 0.45 }
266
+ - { filter: model.language_model.layers.9.self_attn.k_proj, value: 0.45 }
267
+ - { filter: model.language_model.layers.10.self_attn.q_proj, value: 0.45 }
268
+ - { filter: model.language_model.layers.10.self_attn.k_proj, value: 0.45 }
269
+ - { filter: model.language_model.layers.11.self_attn.q_proj, value: 0.45 }
270
+ - { filter: model.language_model.layers.11.self_attn.k_proj, value: 0.45 }
271
+ - { filter: model.language_model.layers.12.self_attn.q_proj, value: 0.45 }
272
+ - { filter: model.language_model.layers.12.self_attn.k_proj, value: 0.45 }
273
+ - { filter: model.language_model.layers.13.self_attn.q_proj, value: 0.45 }
274
+ - { filter: model.language_model.layers.13.self_attn.k_proj, value: 0.45 }
275
+ - { filter: model.language_model.layers.14.self_attn.q_proj, value: 0.45 }
276
+ - { filter: model.language_model.layers.14.self_attn.k_proj, value: 0.45 }
277
+ - { filter: model.language_model.layers.15.self_attn.q_proj, value: 0.45 }
278
+ - { filter: model.language_model.layers.15.self_attn.k_proj, value: 0.45 }
279
+ - { filter: model.language_model.layers.16.self_attn.q_proj, value: 0.45 }
280
+ - { filter: model.language_model.layers.16.self_attn.k_proj, value: 0.45 }
281
+ - { filter: model.language_model.layers.17.self_attn.q_proj, value: 0.45 }
282
+ - { filter: model.language_model.layers.17.self_attn.k_proj, value: 0.45 }
283
+ - { filter: model.language_model.layers.18.self_attn.q_proj, value: 0.45 }
284
+ - { filter: model.language_model.layers.18.self_attn.k_proj, value: 0.45 }
285
+ - { filter: model.language_model.layers.19.self_attn.q_proj, value: 0.45 }
286
+ - { filter: model.language_model.layers.19.self_attn.k_proj, value: 0.45 }
287
+ - { filter: model.language_model.layers.20.self_attn.q_proj, value: 0.45 }
288
+ - { filter: model.language_model.layers.20.self_attn.k_proj, value: 0.45 }
289
+ - { filter: model.language_model.layers.21.self_attn.q_proj, value: 0.45 }
290
+ - { filter: model.language_model.layers.21.self_attn.k_proj, value: 0.45 }
291
+ - { filter: model.language_model.layers.22.self_attn.q_proj, value: 0.45 }
292
+ - { filter: model.language_model.layers.22.self_attn.k_proj, value: 0.45 }
293
+ - { filter: model.language_model.layers.23.self_attn.q_proj, value: 0.45 }
294
+ - { filter: model.language_model.layers.23.self_attn.k_proj, value: 0.45 }
295
+ - { filter: model.language_model.layers.24.self_attn.q_proj, value: 0.45 }
296
+ - { filter: model.language_model.layers.24.self_attn.k_proj, value: 0.45 }
297
+ - { filter: model.language_model.layers.25.self_attn.q_proj, value: 0.45 }
298
+ - { filter: model.language_model.layers.25.self_attn.k_proj, value: 0.45 }
299
+ - { filter: model.language_model.layers.26.self_attn.q_proj, value: 0.45 }
300
+ - { filter: model.language_model.layers.26.self_attn.k_proj, value: 0.45 }
301
+ - { filter: model.language_model.layers.27.self_attn.q_proj, value: 0.45 }
302
+ - { filter: model.language_model.layers.27.self_attn.k_proj, value: 0.45 }
303
+ - { filter: model.language_model.layers.28.self_attn.q_proj, value: 0.45 }
304
+ - { filter: model.language_model.layers.28.self_attn.k_proj, value: 0.45 }
305
+ - { filter: model.language_model.layers.29.self_attn.q_proj, value: 0.45 }
306
+ - { filter: model.language_model.layers.29.self_attn.k_proj, value: 0.45 }
307
+ - { filter: model.language_model.layers.30.self_attn.q_proj, value: 0.45 }
308
+ - { filter: model.language_model.layers.30.self_attn.k_proj, value: 0.45 }
309
+ - { filter: model.language_model.layers.31.self_attn.q_proj, value: 0.45 }
310
+ - { filter: model.language_model.layers.31.self_attn.k_proj, value: 0.45 }
311
+ - { filter: model.language_model.layers.32.self_attn.q_proj, value: 0.45 }
312
+ - { filter: model.language_model.layers.32.self_attn.k_proj, value: 0.45 }
313
+ - { filter: model.language_model.layers.33.self_attn.q_proj, value: 0.45 }
314
+ - { filter: model.language_model.layers.33.self_attn.k_proj, value: 0.45 }
315
+ - { filter: model.language_model.layers.34.self_attn.q_proj, value: 0.45 }
316
+ - { filter: model.language_model.layers.34.self_attn.k_proj, value: 0.45 }
317
+ - { filter: model.language_model.layers.35.self_attn.q_proj, value: 0.45 }
318
+ - { filter: model.language_model.layers.35.self_attn.k_proj, value: 0.45 }
319
+ - { filter: model.language_model.layers.36.self_attn.q_proj, value: 0.45 }
320
+ - { filter: model.language_model.layers.36.self_attn.k_proj, value: 0.45 }
321
+ - { filter: model.language_model.layers.37.self_attn.q_proj, value: 0.45 }
322
+ - { filter: model.language_model.layers.37.self_attn.k_proj, value: 0.45 }
323
+ - { filter: model.language_model.layers.38.self_attn.q_proj, value: 0.45 }
324
+ - { filter: model.language_model.layers.38.self_attn.k_proj, value: 0.45 }
325
+ - { filter: model.language_model.layers.39.self_attn.q_proj, value: 0.45 }
326
+ - { filter: model.language_model.layers.39.self_attn.k_proj, value: 0.45 }
327
+ - { filter: model.language_model.layers.40.self_attn.q_proj, value: 0.45 }
328
+ - { filter: model.language_model.layers.40.self_attn.k_proj, value: 0.45 }
329
+ - { filter: model.language_model.layers.41.self_attn.q_proj, value: 0.45 }
330
+ - { filter: model.language_model.layers.41.self_attn.k_proj, value: 0.45 }
331
+ - { filter: model.language_model.layers.42.self_attn.q_proj, value: 0.45 }
332
+ - { filter: model.language_model.layers.42.self_attn.k_proj, value: 0.45 }
333
+ - { filter: model.language_model.layers.43.self_attn.q_proj, value: 0.45 }
334
+ - { filter: model.language_model.layers.43.self_attn.k_proj, value: 0.45 }
335
+ - { filter: model.language_model.layers.44.self_attn.q_proj, value: 0.45 }
336
+ - { filter: model.language_model.layers.44.self_attn.k_proj, value: 0.45 }
337
+ - { filter: model.language_model.layers.45.self_attn.q_proj, value: 0.45 }
338
+ - { filter: model.language_model.layers.45.self_attn.k_proj, value: 0.45 }
339
+ - { filter: model.language_model.layers.46.self_attn.q_proj, value: 0.45 }
340
+ - { filter: model.language_model.layers.46.self_attn.k_proj, value: 0.45 }
341
+ - { filter: model.language_model.layers.47.self_attn.q_proj, value: 0.45 }
342
+ - { filter: model.language_model.layers.47.self_attn.k_proj, value: 0.45 }
343
+ - { filter: model.language_model.layers.48.self_attn.q_proj, value: 0.3 }
344
+ - { filter: model.language_model.layers.48.self_attn.k_proj, value: 0.3 }
345
+ - { filter: model.language_model.layers.49.self_attn.q_proj, value: 0.3 }
346
+ - { filter: model.language_model.layers.49.self_attn.k_proj, value: 0.3 }
347
+ - { filter: model.language_model.layers.50.self_attn.q_proj, value: 0.3 }
348
+ - { filter: model.language_model.layers.50.self_attn.k_proj, value: 0.3 }
349
+ - { filter: model.language_model.layers.51.self_attn.q_proj, value: 0.3 }
350
+ - { filter: model.language_model.layers.51.self_attn.k_proj, value: 0.3 }
351
+ - { filter: model.language_model.layers.52.self_attn.q_proj, value: 0.3 }
352
+ - { filter: model.language_model.layers.52.self_attn.k_proj, value: 0.3 }
353
+ - { filter: model.language_model.layers.53.self_attn.q_proj, value: 0.3 }
354
+ - { filter: model.language_model.layers.53.self_attn.k_proj, value: 0.3 }
355
+ - { filter: model.language_model.layers.54.self_attn.q_proj, value: 0.3 }
356
+ - { filter: model.language_model.layers.54.self_attn.k_proj, value: 0.3 }
357
+ - { filter: model.language_model.layers.55.self_attn.q_proj, value: 0.3 }
358
+ - { filter: model.language_model.layers.55.self_attn.k_proj, value: 0.3 }
359
+ - { filter: model.language_model.layers.56.self_attn.q_proj, value: 0.3 }
360
+ - { filter: model.language_model.layers.56.self_attn.k_proj, value: 0.3 }
361
+ - { filter: model.language_model.layers.57.self_attn.q_proj, value: 0.3 }
362
+ - { filter: model.language_model.layers.57.self_attn.k_proj, value: 0.3 }
363
+ - { filter: model.language_model.layers.58.self_attn.q_proj, value: 0.3 }
364
+ - { filter: model.language_model.layers.58.self_attn.k_proj, value: 0.3 }
365
+ - { filter: model.language_model.layers.59.self_attn.q_proj, value: 0.3 }
366
+ - { filter: model.language_model.layers.59.self_attn.k_proj, value: 0.3 }
367
+ - { filter: model.language_model.layers.0.mlp.gate_proj, value: 0.28 }
368
+ - { filter: model.language_model.layers.0.mlp.up_proj, value: 0.28 }
369
+ - { filter: model.language_model.layers.1.mlp.gate_proj, value: 0.28 }
370
+ - { filter: model.language_model.layers.1.mlp.up_proj, value: 0.28 }
371
+ - { filter: model.language_model.layers.2.mlp.gate_proj, value: 0.28 }
372
+ - { filter: model.language_model.layers.2.mlp.up_proj, value: 0.28 }
373
+ - { filter: model.language_model.layers.3.mlp.gate_proj, value: 0.28 }
374
+ - { filter: model.language_model.layers.3.mlp.up_proj, value: 0.28 }
375
+ - { filter: model.language_model.layers.4.mlp.gate_proj, value: 0.28 }
376
+ - { filter: model.language_model.layers.4.mlp.up_proj, value: 0.28 }
377
+ - { filter: model.language_model.layers.5.mlp.gate_proj, value: 0.28 }
378
+ - { filter: model.language_model.layers.5.mlp.up_proj, value: 0.28 }
379
+ - { filter: model.language_model.layers.6.mlp.gate_proj, value: 0.28 }
380
+ - { filter: model.language_model.layers.6.mlp.up_proj, value: 0.28 }
381
+ - { filter: model.language_model.layers.7.mlp.gate_proj, value: 0.28 }
382
+ - { filter: model.language_model.layers.7.mlp.up_proj, value: 0.28 }
383
+ - { filter: model.language_model.layers.8.mlp.gate_proj, value: 0.28 }
384
+ - { filter: model.language_model.layers.8.mlp.up_proj, value: 0.28 }
385
+ - { filter: model.language_model.layers.9.mlp.gate_proj, value: 0.28 }
386
+ - { filter: model.language_model.layers.9.mlp.up_proj, value: 0.28 }
387
+ - { filter: model.language_model.layers.10.mlp.gate_proj, value: 0.28 }
388
+ - { filter: model.language_model.layers.10.mlp.up_proj, value: 0.28 }
389
+ - { filter: model.language_model.layers.11.mlp.gate_proj, value: 0.28 }
390
+ - { filter: model.language_model.layers.11.mlp.up_proj, value: 0.28 }
391
+ - { filter: model.language_model.layers.12.mlp.gate_proj, value: 0.28 }
392
+ - { filter: model.language_model.layers.12.mlp.up_proj, value: 0.28 }
393
+ - { filter: model.language_model.layers.13.mlp.gate_proj, value: 0.28 }
394
+ - { filter: model.language_model.layers.13.mlp.up_proj, value: 0.28 }
395
+ - { filter: model.language_model.layers.14.mlp.gate_proj, value: 0.28 }
396
+ - { filter: model.language_model.layers.14.mlp.up_proj, value: 0.28 }
397
+ - { filter: model.language_model.layers.15.mlp.gate_proj, value: 0.28 }
398
+ - { filter: model.language_model.layers.15.mlp.up_proj, value: 0.28 }
399
+ - { filter: model.language_model.layers.16.mlp.gate_proj, value: 0.28 }
400
+ - { filter: model.language_model.layers.16.mlp.up_proj, value: 0.28 }
401
+ - { filter: model.language_model.layers.17.mlp.gate_proj, value: 0.28 }
402
+ - { filter: model.language_model.layers.17.mlp.up_proj, value: 0.28 }
403
+ - { filter: model.language_model.layers.18.mlp.gate_proj, value: 0.28 }
404
+ - { filter: model.language_model.layers.18.mlp.up_proj, value: 0.28 }
405
+ - { filter: model.language_model.layers.19.mlp.gate_proj, value: 0.28 }
406
+ - { filter: model.language_model.layers.19.mlp.up_proj, value: 0.28 }
407
+ - { filter: model.language_model.layers.20.mlp.gate_proj, value: 0.28 }
408
+ - { filter: model.language_model.layers.20.mlp.up_proj, value: 0.28 }
409
+ - { filter: model.language_model.layers.21.mlp.gate_proj, value: 0.28 }
410
+ - { filter: model.language_model.layers.21.mlp.up_proj, value: 0.28 }
411
+ - { filter: model.language_model.layers.22.mlp.gate_proj, value: 0.28 }
412
+ - { filter: model.language_model.layers.22.mlp.up_proj, value: 0.28 }
413
+ - { filter: model.language_model.layers.23.mlp.gate_proj, value: 0.28 }
414
+ - { filter: model.language_model.layers.23.mlp.up_proj, value: 0.28 }
415
+ - { filter: model.language_model.layers.24.mlp.gate_proj, value: 0.28 }
416
+ - { filter: model.language_model.layers.24.mlp.up_proj, value: 0.28 }
417
+ - { filter: model.language_model.layers.25.mlp.gate_proj, value: 0.28 }
418
+ - { filter: model.language_model.layers.25.mlp.up_proj, value: 0.28 }
419
+ - { filter: model.language_model.layers.26.mlp.gate_proj, value: 0.28 }
420
+ - { filter: model.language_model.layers.26.mlp.up_proj, value: 0.28 }
421
+ - { filter: model.language_model.layers.27.mlp.gate_proj, value: 0.28 }
422
+ - { filter: model.language_model.layers.27.mlp.up_proj, value: 0.28 }
423
+ - { filter: model.language_model.layers.28.mlp.gate_proj, value: 0.28 }
424
+ - { filter: model.language_model.layers.28.mlp.up_proj, value: 0.28 }
425
+ - { filter: model.language_model.layers.29.mlp.gate_proj, value: 0.28 }
426
+ - { filter: model.language_model.layers.29.mlp.up_proj, value: 0.28 }
427
+ - { filter: model.language_model.layers.30.mlp.gate_proj, value: 0.28 }
428
+ - { filter: model.language_model.layers.30.mlp.up_proj, value: 0.28 }
429
+ - { filter: model.language_model.layers.31.mlp.gate_proj, value: 0.28 }
430
+ - { filter: model.language_model.layers.31.mlp.up_proj, value: 0.28 }
431
+ - { filter: model.language_model.layers.32.mlp.gate_proj, value: 0.28 }
432
+ - { filter: model.language_model.layers.32.mlp.up_proj, value: 0.28 }
433
+ - { filter: model.language_model.layers.33.mlp.gate_proj, value: 0.28 }
434
+ - { filter: model.language_model.layers.33.mlp.up_proj, value: 0.28 }
435
+ - { filter: model.language_model.layers.34.mlp.gate_proj, value: 0.28 }
436
+ - { filter: model.language_model.layers.34.mlp.up_proj, value: 0.28 }
437
+ - { filter: model.language_model.layers.35.mlp.gate_proj, value: 0.28 }
438
+ - { filter: model.language_model.layers.35.mlp.up_proj, value: 0.28 }
439
+ - { filter: model.language_model.layers.36.mlp.gate_proj, value: 0.28 }
440
+ - { filter: model.language_model.layers.36.mlp.up_proj, value: 0.28 }
441
+ - { filter: model.language_model.layers.37.mlp.gate_proj, value: 0.28 }
442
+ - { filter: model.language_model.layers.37.mlp.up_proj, value: 0.28 }
443
+ - { filter: model.language_model.layers.38.mlp.gate_proj, value: 0.28 }
444
+ - { filter: model.language_model.layers.38.mlp.up_proj, value: 0.28 }
445
+ - { filter: model.language_model.layers.39.mlp.gate_proj, value: 0.28 }
446
+ - { filter: model.language_model.layers.39.mlp.up_proj, value: 0.28 }
447
+ - { filter: model.language_model.layers.40.mlp.gate_proj, value: 0.28 }
448
+ - { filter: model.language_model.layers.40.mlp.up_proj, value: 0.28 }
449
+ - { filter: model.language_model.layers.41.mlp.gate_proj, value: 0.28 }
450
+ - { filter: model.language_model.layers.41.mlp.up_proj, value: 0.28 }
451
+ - { filter: model.language_model.layers.42.mlp.gate_proj, value: 0.28 }
452
+ - { filter: model.language_model.layers.42.mlp.up_proj, value: 0.28 }
453
+ - { filter: model.language_model.layers.43.mlp.gate_proj, value: 0.28 }
454
+ - { filter: model.language_model.layers.43.mlp.up_proj, value: 0.28 }
455
+ - { filter: model.language_model.layers.44.mlp.gate_proj, value: 0.28 }
456
+ - { filter: model.language_model.layers.44.mlp.up_proj, value: 0.28 }
457
+ - { filter: model.language_model.layers.45.mlp.gate_proj, value: 0.28 }
458
+ - { filter: model.language_model.layers.45.mlp.up_proj, value: 0.28 }
459
+ - { filter: model.language_model.layers.46.mlp.gate_proj, value: 0.28 }
460
+ - { filter: model.language_model.layers.46.mlp.up_proj, value: 0.28 }
461
+ - { filter: model.language_model.layers.47.mlp.gate_proj, value: 0.28 }
462
+ - { filter: model.language_model.layers.47.mlp.up_proj, value: 0.28 }
463
+ - { filter: model.language_model.layers.48.mlp.gate_proj, value: 0.18 }
464
+ - { filter: model.language_model.layers.48.mlp.up_proj, value: 0.18 }
465
+ - { filter: model.language_model.layers.49.mlp.gate_proj, value: 0.18 }
466
+ - { filter: model.language_model.layers.49.mlp.up_proj, value: 0.18 }
467
+ - { filter: model.language_model.layers.50.mlp.gate_proj, value: 0.18 }
468
+ - { filter: model.language_model.layers.50.mlp.up_proj, value: 0.18 }
469
+ - { filter: model.language_model.layers.51.mlp.gate_proj, value: 0.18 }
470
+ - { filter: model.language_model.layers.51.mlp.up_proj, value: 0.18 }
471
+ - { filter: model.language_model.layers.52.mlp.gate_proj, value: 0.18 }
472
+ - { filter: model.language_model.layers.52.mlp.up_proj, value: 0.18 }
473
+ - { filter: model.language_model.layers.53.mlp.gate_proj, value: 0.18 }
474
+ - { filter: model.language_model.layers.53.mlp.up_proj, value: 0.18 }
475
+ - { filter: model.language_model.layers.54.mlp.gate_proj, value: 0.18 }
476
+ - { filter: model.language_model.layers.54.mlp.up_proj, value: 0.18 }
477
+ - { filter: model.language_model.layers.55.mlp.gate_proj, value: 0.18 }
478
+ - { filter: model.language_model.layers.55.mlp.up_proj, value: 0.18 }
479
+ - { filter: model.language_model.layers.56.mlp.gate_proj, value: 0.18 }
480
+ - { filter: model.language_model.layers.56.mlp.up_proj, value: 0.18 }
481
+ - { filter: model.language_model.layers.57.mlp.gate_proj, value: 0.18 }
482
+ - { filter: model.language_model.layers.57.mlp.up_proj, value: 0.18 }
483
+ - { filter: model.language_model.layers.58.mlp.gate_proj, value: 0.18 }
484
+ - { filter: model.language_model.layers.58.mlp.up_proj, value: 0.18 }
485
+ - { filter: model.language_model.layers.59.mlp.gate_proj, value: 0.18 }
486
+ - { filter: model.language_model.layers.59.mlp.up_proj, value: 0.18 }
487
+ - { value: 0.0 }
488
+
489
+ - model: ./sophosympatheia/Glistening-Gem-31B-v2.1
490
+ parameters:
491
+ density:
492
+ - { filter: norm, value: 1.0 }
493
+ - { filter: layer_scalar, value: 1.0 }
494
+ - { value: 0.55 }
495
+ weight:
496
+ - { filter: embed_tokens, value: 0.0 }
497
+ - { filter: lm_head, value: 0.0 }
498
+ - { filter: embed_vision, value: 0.0 }
499
+ - { filter: vision_tower, value: 0.0 }
500
+ - { filter: multi_modal_projector, value: 0.0 }
501
+ - { filter: model.language_model.layers.5.self_attn.q_proj, value: 0.0 }
502
+ - { filter: model.language_model.layers.5.self_attn.k_proj, value: 0.0 }
503
+ - { filter: model.language_model.layers.11.self_attn.q_proj, value: 0.0 }
504
+ - { filter: model.language_model.layers.11.self_attn.k_proj, value: 0.0 }
505
+ - { filter: model.language_model.layers.17.self_attn.q_proj, value: 0.0 }
506
+ - { filter: model.language_model.layers.17.self_attn.k_proj, value: 0.0 }
507
+ - { filter: model.language_model.layers.23.self_attn.q_proj, value: 0.0 }
508
+ - { filter: model.language_model.layers.23.self_attn.k_proj, value: 0.0 }
509
+ - { filter: model.language_model.layers.29.self_attn.q_proj, value: 0.0 }
510
+ - { filter: model.language_model.layers.29.self_attn.k_proj, value: 0.0 }
511
+ - { filter: model.language_model.layers.35.self_attn.q_proj, value: 0.0 }
512
+ - { filter: model.language_model.layers.35.self_attn.k_proj, value: 0.0 }
513
+ - { filter: model.language_model.layers.41.self_attn.q_proj, value: 0.0 }
514
+ - { filter: model.language_model.layers.41.self_attn.k_proj, value: 0.0 }
515
+ - { filter: model.language_model.layers.47.self_attn.q_proj, value: 0.0 }
516
+ - { filter: model.language_model.layers.47.self_attn.k_proj, value: 0.0 }
517
+ - { filter: model.language_model.layers.53.self_attn.q_proj, value: 0.0 }
518
+ - { filter: model.language_model.layers.53.self_attn.k_proj, value: 0.0 }
519
+ - { filter: model.language_model.layers.59.self_attn.q_proj, value: 0.0 }
520
+ - { filter: model.language_model.layers.59.self_attn.k_proj, value: 0.0 }
521
+ - { filter: model.language_model.layers.0.self_attn.q_proj, value: 0.14 }
522
+ - { filter: model.language_model.layers.0.self_attn.k_proj, value: 0.14 }
523
+ - { filter: model.language_model.layers.1.self_attn.q_proj, value: 0.14 }
524
+ - { filter: model.language_model.layers.1.self_attn.k_proj, value: 0.14 }
525
+ - { filter: model.language_model.layers.2.self_attn.q_proj, value: 0.14 }
526
+ - { filter: model.language_model.layers.2.self_attn.k_proj, value: 0.14 }
527
+ - { filter: model.language_model.layers.3.self_attn.q_proj, value: 0.14 }
528
+ - { filter: model.language_model.layers.3.self_attn.k_proj, value: 0.14 }
529
+ - { filter: model.language_model.layers.4.self_attn.q_proj, value: 0.14 }
530
+ - { filter: model.language_model.layers.4.self_attn.k_proj, value: 0.14 }
531
+ - { filter: model.language_model.layers.5.self_attn.q_proj, value: 0.14 }
532
+ - { filter: model.language_model.layers.5.self_attn.k_proj, value: 0.14 }
533
+ - { filter: model.language_model.layers.6.self_attn.q_proj, value: 0.14 }
534
+ - { filter: model.language_model.layers.6.self_attn.k_proj, value: 0.14 }
535
+ - { filter: model.language_model.layers.7.self_attn.q_proj, value: 0.14 }
536
+ - { filter: model.language_model.layers.7.self_attn.k_proj, value: 0.14 }
537
+ - { filter: model.language_model.layers.8.self_attn.q_proj, value: 0.14 }
538
+ - { filter: model.language_model.layers.8.self_attn.k_proj, value: 0.14 }
539
+ - { filter: model.language_model.layers.9.self_attn.q_proj, value: 0.14 }
540
+ - { filter: model.language_model.layers.9.self_attn.k_proj, value: 0.14 }
541
+ - { filter: model.language_model.layers.10.self_attn.q_proj, value: 0.14 }
542
+ - { filter: model.language_model.layers.10.self_attn.k_proj, value: 0.14 }
543
+ - { filter: model.language_model.layers.11.self_attn.q_proj, value: 0.14 }
544
+ - { filter: model.language_model.layers.11.self_attn.k_proj, value: 0.14 }
545
+ - { filter: model.language_model.layers.12.self_attn.q_proj, value: 0.14 }
546
+ - { filter: model.language_model.layers.12.self_attn.k_proj, value: 0.14 }
547
+ - { filter: model.language_model.layers.13.self_attn.q_proj, value: 0.14 }
548
+ - { filter: model.language_model.layers.13.self_attn.k_proj, value: 0.14 }
549
+ - { filter: model.language_model.layers.14.self_attn.q_proj, value: 0.14 }
550
+ - { filter: model.language_model.layers.14.self_attn.k_proj, value: 0.14 }
551
+ - { filter: model.language_model.layers.15.self_attn.q_proj, value: 0.14 }
552
+ - { filter: model.language_model.layers.15.self_attn.k_proj, value: 0.14 }
553
+ - { filter: model.language_model.layers.16.self_attn.q_proj, value: 0.14 }
554
+ - { filter: model.language_model.layers.16.self_attn.k_proj, value: 0.14 }
555
+ - { filter: model.language_model.layers.17.self_attn.q_proj, value: 0.14 }
556
+ - { filter: model.language_model.layers.17.self_attn.k_proj, value: 0.14 }
557
+ - { filter: model.language_model.layers.18.self_attn.q_proj, value: 0.14 }
558
+ - { filter: model.language_model.layers.18.self_attn.k_proj, value: 0.14 }
559
+ - { filter: model.language_model.layers.19.self_attn.q_proj, value: 0.14 }
560
+ - { filter: model.language_model.layers.19.self_attn.k_proj, value: 0.14 }
561
+ - { filter: model.language_model.layers.20.self_attn.q_proj, value: 0.14 }
562
+ - { filter: model.language_model.layers.20.self_attn.k_proj, value: 0.14 }
563
+ - { filter: model.language_model.layers.21.self_attn.q_proj, value: 0.14 }
564
+ - { filter: model.language_model.layers.21.self_attn.k_proj, value: 0.14 }
565
+ - { filter: model.language_model.layers.22.self_attn.q_proj, value: 0.14 }
566
+ - { filter: model.language_model.layers.22.self_attn.k_proj, value: 0.14 }
567
+ - { filter: model.language_model.layers.23.self_attn.q_proj, value: 0.14 }
568
+ - { filter: model.language_model.layers.23.self_attn.k_proj, value: 0.14 }
569
+ - { filter: model.language_model.layers.24.self_attn.q_proj, value: 0.14 }
570
+ - { filter: model.language_model.layers.24.self_attn.k_proj, value: 0.14 }
571
+ - { filter: model.language_model.layers.25.self_attn.q_proj, value: 0.14 }
572
+ - { filter: model.language_model.layers.25.self_attn.k_proj, value: 0.14 }
573
+ - { filter: model.language_model.layers.26.self_attn.q_proj, value: 0.14 }
574
+ - { filter: model.language_model.layers.26.self_attn.k_proj, value: 0.14 }
575
+ - { filter: model.language_model.layers.27.self_attn.q_proj, value: 0.14 }
576
+ - { filter: model.language_model.layers.27.self_attn.k_proj, value: 0.14 }
577
+ - { filter: model.language_model.layers.28.self_attn.q_proj, value: 0.14 }
578
+ - { filter: model.language_model.layers.28.self_attn.k_proj, value: 0.14 }
579
+ - { filter: model.language_model.layers.29.self_attn.q_proj, value: 0.14 }
580
+ - { filter: model.language_model.layers.29.self_attn.k_proj, value: 0.14 }
581
+ - { filter: model.language_model.layers.30.self_attn.q_proj, value: 0.14 }
582
+ - { filter: model.language_model.layers.30.self_attn.k_proj, value: 0.14 }
583
+ - { filter: model.language_model.layers.31.self_attn.q_proj, value: 0.14 }
584
+ - { filter: model.language_model.layers.31.self_attn.k_proj, value: 0.14 }
585
+ - { filter: model.language_model.layers.32.self_attn.q_proj, value: 0.14 }
586
+ - { filter: model.language_model.layers.32.self_attn.k_proj, value: 0.14 }
587
+ - { filter: model.language_model.layers.33.self_attn.q_proj, value: 0.14 }
588
+ - { filter: model.language_model.layers.33.self_attn.k_proj, value: 0.14 }
589
+ - { filter: model.language_model.layers.34.self_attn.q_proj, value: 0.14 }
590
+ - { filter: model.language_model.layers.34.self_attn.k_proj, value: 0.14 }
591
+ - { filter: model.language_model.layers.35.self_attn.q_proj, value: 0.14 }
592
+ - { filter: model.language_model.layers.35.self_attn.k_proj, value: 0.14 }
593
+ - { filter: model.language_model.layers.36.self_attn.q_proj, value: 0.14 }
594
+ - { filter: model.language_model.layers.36.self_attn.k_proj, value: 0.14 }
595
+ - { filter: model.language_model.layers.37.self_attn.q_proj, value: 0.14 }
596
+ - { filter: model.language_model.layers.37.self_attn.k_proj, value: 0.14 }
597
+ - { filter: model.language_model.layers.38.self_attn.q_proj, value: 0.14 }
598
+ - { filter: model.language_model.layers.38.self_attn.k_proj, value: 0.14 }
599
+ - { filter: model.language_model.layers.39.self_attn.q_proj, value: 0.14 }
600
+ - { filter: model.language_model.layers.39.self_attn.k_proj, value: 0.14 }
601
+ - { filter: model.language_model.layers.40.self_attn.q_proj, value: 0.14 }
602
+ - { filter: model.language_model.layers.40.self_attn.k_proj, value: 0.14 }
603
+ - { filter: model.language_model.layers.41.self_attn.q_proj, value: 0.14 }
604
+ - { filter: model.language_model.layers.41.self_attn.k_proj, value: 0.14 }
605
+ - { filter: model.language_model.layers.42.self_attn.q_proj, value: 0.14 }
606
+ - { filter: model.language_model.layers.42.self_attn.k_proj, value: 0.14 }
607
+ - { filter: model.language_model.layers.43.self_attn.q_proj, value: 0.14 }
608
+ - { filter: model.language_model.layers.43.self_attn.k_proj, value: 0.14 }
609
+ - { filter: model.language_model.layers.44.self_attn.q_proj, value: 0.14 }
610
+ - { filter: model.language_model.layers.44.self_attn.k_proj, value: 0.14 }
611
+ - { filter: model.language_model.layers.45.self_attn.q_proj, value: 0.14 }
612
+ - { filter: model.language_model.layers.45.self_attn.k_proj, value: 0.14 }
613
+ - { filter: model.language_model.layers.46.self_attn.q_proj, value: 0.14 }
614
+ - { filter: model.language_model.layers.46.self_attn.k_proj, value: 0.14 }
615
+ - { filter: model.language_model.layers.47.self_attn.q_proj, value: 0.14 }
616
+ - { filter: model.language_model.layers.47.self_attn.k_proj, value: 0.14 }
617
+ - { filter: model.language_model.layers.48.self_attn.q_proj, value: 0.1 }
618
+ - { filter: model.language_model.layers.48.self_attn.k_proj, value: 0.1 }
619
+ - { filter: model.language_model.layers.49.self_attn.q_proj, value: 0.1 }
620
+ - { filter: model.language_model.layers.49.self_attn.k_proj, value: 0.1 }
621
+ - { filter: model.language_model.layers.50.self_attn.q_proj, value: 0.1 }
622
+ - { filter: model.language_model.layers.50.self_attn.k_proj, value: 0.1 }
623
+ - { filter: model.language_model.layers.51.self_attn.q_proj, value: 0.1 }
624
+ - { filter: model.language_model.layers.51.self_attn.k_proj, value: 0.1 }
625
+ - { filter: model.language_model.layers.52.self_attn.q_proj, value: 0.1 }
626
+ - { filter: model.language_model.layers.52.self_attn.k_proj, value: 0.1 }
627
+ - { filter: model.language_model.layers.53.self_attn.q_proj, value: 0.1 }
628
+ - { filter: model.language_model.layers.53.self_attn.k_proj, value: 0.1 }
629
+ - { filter: model.language_model.layers.54.self_attn.q_proj, value: 0.1 }
630
+ - { filter: model.language_model.layers.54.self_attn.k_proj, value: 0.1 }
631
+ - { filter: model.language_model.layers.55.self_attn.q_proj, value: 0.1 }
632
+ - { filter: model.language_model.layers.55.self_attn.k_proj, value: 0.1 }
633
+ - { filter: model.language_model.layers.56.self_attn.q_proj, value: 0.1 }
634
+ - { filter: model.language_model.layers.56.self_attn.k_proj, value: 0.1 }
635
+ - { filter: model.language_model.layers.57.self_attn.q_proj, value: 0.1 }
636
+ - { filter: model.language_model.layers.57.self_attn.k_proj, value: 0.1 }
637
+ - { filter: model.language_model.layers.58.self_attn.q_proj, value: 0.1 }
638
+ - { filter: model.language_model.layers.58.self_attn.k_proj, value: 0.1 }
639
+ - { filter: model.language_model.layers.59.self_attn.q_proj, value: 0.1 }
640
+ - { filter: model.language_model.layers.59.self_attn.k_proj, value: 0.1 }
641
+ - { filter: model.language_model.layers.0.mlp.gate_proj, value: 0.06 }
642
+ - { filter: model.language_model.layers.0.mlp.up_proj, value: 0.06 }
643
+ - { filter: model.language_model.layers.1.mlp.gate_proj, value: 0.06 }
644
+ - { filter: model.language_model.layers.1.mlp.up_proj, value: 0.06 }
645
+ - { filter: model.language_model.layers.2.mlp.gate_proj, value: 0.06 }
646
+ - { filter: model.language_model.layers.2.mlp.up_proj, value: 0.06 }
647
+ - { filter: model.language_model.layers.3.mlp.gate_proj, value: 0.06 }
648
+ - { filter: model.language_model.layers.3.mlp.up_proj, value: 0.06 }
649
+ - { filter: model.language_model.layers.4.mlp.gate_proj, value: 0.06 }
650
+ - { filter: model.language_model.layers.4.mlp.up_proj, value: 0.06 }
651
+ - { filter: model.language_model.layers.5.mlp.gate_proj, value: 0.06 }
652
+ - { filter: model.language_model.layers.5.mlp.up_proj, value: 0.06 }
653
+ - { filter: model.language_model.layers.6.mlp.gate_proj, value: 0.06 }
654
+ - { filter: model.language_model.layers.6.mlp.up_proj, value: 0.06 }
655
+ - { filter: model.language_model.layers.7.mlp.gate_proj, value: 0.06 }
656
+ - { filter: model.language_model.layers.7.mlp.up_proj, value: 0.06 }
657
+ - { filter: model.language_model.layers.8.mlp.gate_proj, value: 0.06 }
658
+ - { filter: model.language_model.layers.8.mlp.up_proj, value: 0.06 }
659
+ - { filter: model.language_model.layers.9.mlp.gate_proj, value: 0.06 }
660
+ - { filter: model.language_model.layers.9.mlp.up_proj, value: 0.06 }
661
+ - { filter: model.language_model.layers.10.mlp.gate_proj, value: 0.06 }
662
+ - { filter: model.language_model.layers.10.mlp.up_proj, value: 0.06 }
663
+ - { filter: model.language_model.layers.11.mlp.gate_proj, value: 0.06 }
664
+ - { filter: model.language_model.layers.11.mlp.up_proj, value: 0.06 }
665
+ - { filter: model.language_model.layers.12.mlp.gate_proj, value: 0.06 }
666
+ - { filter: model.language_model.layers.12.mlp.up_proj, value: 0.06 }
667
+ - { filter: model.language_model.layers.13.mlp.gate_proj, value: 0.06 }
668
+ - { filter: model.language_model.layers.13.mlp.up_proj, value: 0.06 }
669
+ - { filter: model.language_model.layers.14.mlp.gate_proj, value: 0.06 }
670
+ - { filter: model.language_model.layers.14.mlp.up_proj, value: 0.06 }
671
+ - { filter: model.language_model.layers.15.mlp.gate_proj, value: 0.06 }
672
+ - { filter: model.language_model.layers.15.mlp.up_proj, value: 0.06 }
673
+ - { filter: model.language_model.layers.16.mlp.gate_proj, value: 0.06 }
674
+ - { filter: model.language_model.layers.16.mlp.up_proj, value: 0.06 }
675
+ - { filter: model.language_model.layers.17.mlp.gate_proj, value: 0.06 }
676
+ - { filter: model.language_model.layers.17.mlp.up_proj, value: 0.06 }
677
+ - { filter: model.language_model.layers.18.mlp.gate_proj, value: 0.06 }
678
+ - { filter: model.language_model.layers.18.mlp.up_proj, value: 0.06 }
679
+ - { filter: model.language_model.layers.19.mlp.gate_proj, value: 0.06 }
680
+ - { filter: model.language_model.layers.19.mlp.up_proj, value: 0.06 }
681
+ - { filter: model.language_model.layers.20.mlp.gate_proj, value: 0.06 }
682
+ - { filter: model.language_model.layers.20.mlp.up_proj, value: 0.06 }
683
+ - { filter: model.language_model.layers.21.mlp.gate_proj, value: 0.06 }
684
+ - { filter: model.language_model.layers.21.mlp.up_proj, value: 0.06 }
685
+ - { filter: model.language_model.layers.22.mlp.gate_proj, value: 0.06 }
686
+ - { filter: model.language_model.layers.22.mlp.up_proj, value: 0.06 }
687
+ - { filter: model.language_model.layers.23.mlp.gate_proj, value: 0.06 }
688
+ - { filter: model.language_model.layers.23.mlp.up_proj, value: 0.06 }
689
+ - { filter: model.language_model.layers.24.mlp.gate_proj, value: 0.06 }
690
+ - { filter: model.language_model.layers.24.mlp.up_proj, value: 0.06 }
691
+ - { filter: model.language_model.layers.25.mlp.gate_proj, value: 0.06 }
692
+ - { filter: model.language_model.layers.25.mlp.up_proj, value: 0.06 }
693
+ - { filter: model.language_model.layers.26.mlp.gate_proj, value: 0.06 }
694
+ - { filter: model.language_model.layers.26.mlp.up_proj, value: 0.06 }
695
+ - { filter: model.language_model.layers.27.mlp.gate_proj, value: 0.06 }
696
+ - { filter: model.language_model.layers.27.mlp.up_proj, value: 0.06 }
697
+ - { filter: model.language_model.layers.28.mlp.gate_proj, value: 0.06 }
698
+ - { filter: model.language_model.layers.28.mlp.up_proj, value: 0.06 }
699
+ - { filter: model.language_model.layers.29.mlp.gate_proj, value: 0.06 }
700
+ - { filter: model.language_model.layers.29.mlp.up_proj, value: 0.06 }
701
+ - { filter: model.language_model.layers.30.mlp.gate_proj, value: 0.06 }
702
+ - { filter: model.language_model.layers.30.mlp.up_proj, value: 0.06 }
703
+ - { filter: model.language_model.layers.31.mlp.gate_proj, value: 0.06 }
704
+ - { filter: model.language_model.layers.31.mlp.up_proj, value: 0.06 }
705
+ - { filter: model.language_model.layers.32.mlp.gate_proj, value: 0.06 }
706
+ - { filter: model.language_model.layers.32.mlp.up_proj, value: 0.06 }
707
+ - { filter: model.language_model.layers.33.mlp.gate_proj, value: 0.06 }
708
+ - { filter: model.language_model.layers.33.mlp.up_proj, value: 0.06 }
709
+ - { filter: model.language_model.layers.34.mlp.gate_proj, value: 0.06 }
710
+ - { filter: model.language_model.layers.34.mlp.up_proj, value: 0.06 }
711
+ - { filter: model.language_model.layers.35.mlp.gate_proj, value: 0.06 }
712
+ - { filter: model.language_model.layers.35.mlp.up_proj, value: 0.06 }
713
+ - { filter: model.language_model.layers.36.mlp.gate_proj, value: 0.06 }
714
+ - { filter: model.language_model.layers.36.mlp.up_proj, value: 0.06 }
715
+ - { filter: model.language_model.layers.37.mlp.gate_proj, value: 0.06 }
716
+ - { filter: model.language_model.layers.37.mlp.up_proj, value: 0.06 }
717
+ - { filter: model.language_model.layers.38.mlp.gate_proj, value: 0.06 }
718
+ - { filter: model.language_model.layers.38.mlp.up_proj, value: 0.06 }
719
+ - { filter: model.language_model.layers.39.mlp.gate_proj, value: 0.06 }
720
+ - { filter: model.language_model.layers.39.mlp.up_proj, value: 0.06 }
721
+ - { filter: model.language_model.layers.40.mlp.gate_proj, value: 0.06 }
722
+ - { filter: model.language_model.layers.40.mlp.up_proj, value: 0.06 }
723
+ - { filter: model.language_model.layers.41.mlp.gate_proj, value: 0.06 }
724
+ - { filter: model.language_model.layers.41.mlp.up_proj, value: 0.06 }
725
+ - { filter: model.language_model.layers.42.mlp.gate_proj, value: 0.06 }
726
+ - { filter: model.language_model.layers.42.mlp.up_proj, value: 0.06 }
727
+ - { filter: model.language_model.layers.43.mlp.gate_proj, value: 0.06 }
728
+ - { filter: model.language_model.layers.43.mlp.up_proj, value: 0.06 }
729
+ - { filter: model.language_model.layers.44.mlp.gate_proj, value: 0.06 }
730
+ - { filter: model.language_model.layers.44.mlp.up_proj, value: 0.06 }
731
+ - { filter: model.language_model.layers.45.mlp.gate_proj, value: 0.06 }
732
+ - { filter: model.language_model.layers.45.mlp.up_proj, value: 0.06 }
733
+ - { filter: model.language_model.layers.46.mlp.gate_proj, value: 0.06 }
734
+ - { filter: model.language_model.layers.46.mlp.up_proj, value: 0.06 }
735
+ - { filter: model.language_model.layers.47.mlp.gate_proj, value: 0.06 }
736
+ - { filter: model.language_model.layers.47.mlp.up_proj, value: 0.06 }
737
+ - { value: 0.0 }
738
+
739
+ - model: ./zerofata/G4-MeroMero-v2-31B
740
+ parameters:
741
+ density:
742
+ - { filter: norm, value: 1.0 }
743
+ - { filter: layer_scalar, value: 1.0 }
744
+ - { value: 0.60 }
745
+ weight:
746
+ - { filter: embed_tokens, value: 0.0 }
747
+ - { filter: lm_head, value: 0.0 }
748
+ - { filter: embed_vision, value: 0.0 }
749
+ - { filter: vision_tower, value: 0.0 }
750
+ - { filter: multi_modal_projector, value: 0.0 }
751
+ - { filter: model.language_model.layers.5., value: 0.0 }
752
+ - { filter: model.language_model.layers.11., value: 0.0 }
753
+ - { filter: model.language_model.layers.17., value: 0.0 }
754
+ - { filter: model.language_model.layers.23., value: 0.0 }
755
+ - { filter: model.language_model.layers.29., value: 0.0 }
756
+ - { filter: model.language_model.layers.35., value: 0.0 }
757
+ - { filter: model.language_model.layers.41., value: 0.0 }
758
+ - { filter: model.language_model.layers.47., value: 0.0 }
759
+ - { filter: model.language_model.layers.53., value: 0.0 }
760
+ - { filter: model.language_model.layers.59., value: 0.0 }
761
+ - { filter: model.language_model.layers.6.mlp.gate_proj, value: 0.3 }
762
+ - { filter: model.language_model.layers.6.mlp.up_proj, value: 0.3 }
763
+ - { filter: model.language_model.layers.7.mlp.gate_proj, value: 0.3 }
764
+ - { filter: model.language_model.layers.7.mlp.up_proj, value: 0.3 }
765
+ - { filter: model.language_model.layers.8.mlp.gate_proj, value: 0.3 }
766
+ - { filter: model.language_model.layers.8.mlp.up_proj, value: 0.3 }
767
+ - { filter: model.language_model.layers.9.mlp.gate_proj, value: 0.3 }
768
+ - { filter: model.language_model.layers.9.mlp.up_proj, value: 0.3 }
769
+ - { filter: model.language_model.layers.10.mlp.gate_proj, value: 0.3 }
770
+ - { filter: model.language_model.layers.10.mlp.up_proj, value: 0.3 }
771
+ - { filter: model.language_model.layers.12.mlp.gate_proj, value: 0.3 }
772
+ - { filter: model.language_model.layers.12.mlp.up_proj, value: 0.3 }
773
+ - { filter: model.language_model.layers.13.mlp.gate_proj, value: 0.3 }
774
+ - { filter: model.language_model.layers.13.mlp.up_proj, value: 0.3 }
775
+ - { filter: model.language_model.layers.14.mlp.gate_proj, value: 0.3 }
776
+ - { filter: model.language_model.layers.14.mlp.up_proj, value: 0.3 }
777
+ - { filter: model.language_model.layers.15.mlp.gate_proj, value: 0.3 }
778
+ - { filter: model.language_model.layers.15.mlp.up_proj, value: 0.3 }
779
+ - { filter: model.language_model.layers.16.mlp.gate_proj, value: 0.3 }
780
+ - { filter: model.language_model.layers.16.mlp.up_proj, value: 0.3 }
781
+ - { filter: model.language_model.layers.18.mlp.gate_proj, value: 0.3 }
782
+ - { filter: model.language_model.layers.18.mlp.up_proj, value: 0.3 }
783
+ - { filter: model.language_model.layers.19.mlp.gate_proj, value: 0.3 }
784
+ - { filter: model.language_model.layers.19.mlp.up_proj, value: 0.3 }
785
+ - { filter: model.language_model.layers.20.mlp.gate_proj, value: 0.3 }
786
+ - { filter: model.language_model.layers.20.mlp.up_proj, value: 0.3 }
787
+ - { filter: model.language_model.layers.21.mlp.gate_proj, value: 0.3 }
788
+ - { filter: model.language_model.layers.21.mlp.up_proj, value: 0.3 }
789
+ - { filter: model.language_model.layers.22.mlp.gate_proj, value: 0.3 }
790
+ - { filter: model.language_model.layers.22.mlp.up_proj, value: 0.3 }
791
+ - { filter: model.language_model.layers.24.mlp.gate_proj, value: 0.3 }
792
+ - { filter: model.language_model.layers.24.mlp.up_proj, value: 0.3 }
793
+ - { filter: model.language_model.layers.25.mlp.gate_proj, value: 0.3 }
794
+ - { filter: model.language_model.layers.25.mlp.up_proj, value: 0.3 }
795
+ - { filter: model.language_model.layers.26.mlp.gate_proj, value: 0.3 }
796
+ - { filter: model.language_model.layers.26.mlp.up_proj, value: 0.3 }
797
+ - { filter: model.language_model.layers.27.mlp.gate_proj, value: 0.3 }
798
+ - { filter: model.language_model.layers.27.mlp.up_proj, value: 0.3 }
799
+ - { filter: model.language_model.layers.28.mlp.gate_proj, value: 0.3 }
800
+ - { filter: model.language_model.layers.28.mlp.up_proj, value: 0.3 }
801
+ - { filter: model.language_model.layers.30.mlp.gate_proj, value: 0.3 }
802
+ - { filter: model.language_model.layers.30.mlp.up_proj, value: 0.3 }
803
+ - { filter: model.language_model.layers.31.mlp.gate_proj, value: 0.3 }
804
+ - { filter: model.language_model.layers.31.mlp.up_proj, value: 0.3 }
805
+ - { filter: model.language_model.layers.32.mlp.gate_proj, value: 0.3 }
806
+ - { filter: model.language_model.layers.32.mlp.up_proj, value: 0.3 }
807
+ - { filter: model.language_model.layers.33.mlp.gate_proj, value: 0.3 }
808
+ - { filter: model.language_model.layers.33.mlp.up_proj, value: 0.3 }
809
+ - { filter: model.language_model.layers.34.mlp.gate_proj, value: 0.3 }
810
+ - { filter: model.language_model.layers.34.mlp.up_proj, value: 0.3 }
811
+ - { filter: model.language_model.layers.36.mlp.gate_proj, value: 0.3 }
812
+ - { filter: model.language_model.layers.36.mlp.up_proj, value: 0.3 }
813
+ - { filter: model.language_model.layers.37.mlp.gate_proj, value: 0.3 }
814
+ - { filter: model.language_model.layers.37.mlp.up_proj, value: 0.3 }
815
+ - { filter: model.language_model.layers.38.mlp.gate_proj, value: 0.3 }
816
+ - { filter: model.language_model.layers.38.mlp.up_proj, value: 0.3 }
817
+ - { filter: model.language_model.layers.39.mlp.gate_proj, value: 0.3 }
818
+ - { filter: model.language_model.layers.39.mlp.up_proj, value: 0.3 }
819
+ - { filter: model.language_model.layers.40.mlp.gate_proj, value: 0.3 }
820
+ - { filter: model.language_model.layers.40.mlp.up_proj, value: 0.3 }
821
+ - { filter: model.language_model.layers.42.mlp.gate_proj, value: 0.3 }
822
+ - { filter: model.language_model.layers.42.mlp.up_proj, value: 0.3 }
823
+ - { filter: model.language_model.layers.43.mlp.gate_proj, value: 0.3 }
824
+ - { filter: model.language_model.layers.43.mlp.up_proj, value: 0.3 }
825
+ - { filter: model.language_model.layers.44.mlp.gate_proj, value: 0.3 }
826
+ - { filter: model.language_model.layers.44.mlp.up_proj, value: 0.3 }
827
+ - { filter: model.language_model.layers.45.mlp.gate_proj, value: 0.3 }
828
+ - { filter: model.language_model.layers.45.mlp.up_proj, value: 0.3 }
829
+ - { filter: model.language_model.layers.46.mlp.gate_proj, value: 0.3 }
830
+ - { filter: model.language_model.layers.46.mlp.up_proj, value: 0.3 }
831
+ - { value: 0.0 }
832
+ ```
833
+
834
+ </details>
835
+ </div><div class="sx-card R"><div class="sx-ph"><div class="sx-med">II</div><div><h3>Voice</h3><small>Of The Residual Stream</small></div></div><p>If that also checks out, then <b>v_proj</b> steers the expression of what q/k came up with, and <b>o_proj + mlp.down_proj</b> are the layers that coordinate how the output should be written.</p><div class="sx-grid"><div class="h">Model</div><div class="h">Purpose</div><div class="h">Scope</div><div><b>scotoma-2</b></div><div>Anti-Gemma tics</div><div>weight 1.0 · density 1.0 · layers 7-41<br>alone on 7-11</div><div><b>Dark-Thoughts V2</b></div><div>darker lean, less sanitizing</div><div>v/o/down · 0.50 (24-47) &rarr; 0.30 (48-53)</div><div><b>Glistening-Gem v2.0</b></div><div>self-repetition, coherence</div><div>v/o/down · 0.22 · layers 12-41</div><div><b>Melinoe-VL-heretic</b></div><div>willingness</div><div>v/o/down · 0.30 · layers 42-53</div></div>
836
+ <details class="sx-det"><summary>&#9656; &nbsp;Phase 2: della_linear</summary>
837
+
838
+ ```yaml
839
+ merge_method: della_linear
840
+ base_model: ./Split-31B-read
841
+ dtype: bfloat16
842
+ tokenizer_source: base
843
+ parameters:
844
+ normalize: false
845
+ epsilon: 0.05
846
+ lambda: 1.0
847
+ int8_mask: true
848
+
849
+ models:
850
+ - model: ./ReadyArt/gemma-4-31B-it-scotoma-2
851
+ parameters:
852
+ density:
853
+ - { filter: norm, value: 1.0 }
854
+ - { filter: layer_scalar, value: 1.0 }
855
+ - { value: 1.0 }
856
+ weight:
857
+ - { filter: embed_tokens, value: 0.0 }
858
+ - { filter: lm_head, value: 0.0 }
859
+ - { filter: embed_vision, value: 0.0 }
860
+ - { filter: vision_tower, value: 0.0 }
861
+ - { filter: multi_modal_projector, value: 0.0 }
862
+ - { filter: model.language_model.layers.7.self_attn.o_proj, value: 1.0 }
863
+ - { filter: model.language_model.layers.7.mlp.down_proj, value: 1.0 }
864
+ - { filter: model.language_model.layers.8.self_attn.o_proj, value: 1.0 }
865
+ - { filter: model.language_model.layers.8.mlp.down_proj, value: 1.0 }
866
+ - { filter: model.language_model.layers.9.self_attn.o_proj, value: 1.0 }
867
+ - { filter: model.language_model.layers.9.mlp.down_proj, value: 1.0 }
868
+ - { filter: model.language_model.layers.10.self_attn.o_proj, value: 1.0 }
869
+ - { filter: model.language_model.layers.10.mlp.down_proj, value: 1.0 }
870
+ - { filter: model.language_model.layers.11.self_attn.o_proj, value: 1.0 }
871
+ - { filter: model.language_model.layers.11.mlp.down_proj, value: 1.0 }
872
+ - { filter: model.language_model.layers.12.self_attn.o_proj, value: 1.0 }
873
+ - { filter: model.language_model.layers.12.mlp.down_proj, value: 1.0 }
874
+ - { filter: model.language_model.layers.13.self_attn.o_proj, value: 1.0 }
875
+ - { filter: model.language_model.layers.13.mlp.down_proj, value: 1.0 }
876
+ - { filter: model.language_model.layers.14.self_attn.o_proj, value: 1.0 }
877
+ - { filter: model.language_model.layers.14.mlp.down_proj, value: 1.0 }
878
+ - { filter: model.language_model.layers.15.self_attn.o_proj, value: 1.0 }
879
+ - { filter: model.language_model.layers.15.mlp.down_proj, value: 1.0 }
880
+ - { filter: model.language_model.layers.16.self_attn.o_proj, value: 1.0 }
881
+ - { filter: model.language_model.layers.16.mlp.down_proj, value: 1.0 }
882
+ - { filter: model.language_model.layers.17.self_attn.o_proj, value: 1.0 }
883
+ - { filter: model.language_model.layers.17.mlp.down_proj, value: 1.0 }
884
+ - { filter: model.language_model.layers.18.self_attn.o_proj, value: 1.0 }
885
+ - { filter: model.language_model.layers.18.mlp.down_proj, value: 1.0 }
886
+ - { filter: model.language_model.layers.19.self_attn.o_proj, value: 1.0 }
887
+ - { filter: model.language_model.layers.19.mlp.down_proj, value: 1.0 }
888
+ - { filter: model.language_model.layers.20.self_attn.o_proj, value: 1.0 }
889
+ - { filter: model.language_model.layers.20.mlp.down_proj, value: 1.0 }
890
+ - { filter: model.language_model.layers.21.self_attn.o_proj, value: 1.0 }
891
+ - { filter: model.language_model.layers.21.mlp.down_proj, value: 1.0 }
892
+ - { filter: model.language_model.layers.22.self_attn.o_proj, value: 1.0 }
893
+ - { filter: model.language_model.layers.22.mlp.down_proj, value: 1.0 }
894
+ - { filter: model.language_model.layers.23.self_attn.o_proj, value: 1.0 }
895
+ - { filter: model.language_model.layers.23.mlp.down_proj, value: 1.0 }
896
+ - { filter: model.language_model.layers.24.self_attn.o_proj, value: 1.0 }
897
+ - { filter: model.language_model.layers.24.mlp.down_proj, value: 1.0 }
898
+ - { filter: model.language_model.layers.25.self_attn.o_proj, value: 1.0 }
899
+ - { filter: model.language_model.layers.25.mlp.down_proj, value: 1.0 }
900
+ - { filter: model.language_model.layers.26.self_attn.o_proj, value: 1.0 }
901
+ - { filter: model.language_model.layers.26.mlp.down_proj, value: 1.0 }
902
+ - { filter: model.language_model.layers.27.self_attn.o_proj, value: 1.0 }
903
+ - { filter: model.language_model.layers.27.mlp.down_proj, value: 1.0 }
904
+ - { filter: model.language_model.layers.28.self_attn.o_proj, value: 1.0 }
905
+ - { filter: model.language_model.layers.28.mlp.down_proj, value: 1.0 }
906
+ - { filter: model.language_model.layers.29.self_attn.o_proj, value: 1.0 }
907
+ - { filter: model.language_model.layers.29.mlp.down_proj, value: 1.0 }
908
+ - { filter: model.language_model.layers.30.self_attn.o_proj, value: 1.0 }
909
+ - { filter: model.language_model.layers.30.mlp.down_proj, value: 1.0 }
910
+ - { filter: model.language_model.layers.31.self_attn.o_proj, value: 1.0 }
911
+ - { filter: model.language_model.layers.31.mlp.down_proj, value: 1.0 }
912
+ - { filter: model.language_model.layers.32.self_attn.o_proj, value: 1.0 }
913
+ - { filter: model.language_model.layers.32.mlp.down_proj, value: 1.0 }
914
+ - { filter: model.language_model.layers.33.self_attn.o_proj, value: 1.0 }
915
+ - { filter: model.language_model.layers.33.mlp.down_proj, value: 1.0 }
916
+ - { filter: model.language_model.layers.34.self_attn.o_proj, value: 1.0 }
917
+ - { filter: model.language_model.layers.34.mlp.down_proj, value: 1.0 }
918
+ - { filter: model.language_model.layers.35.self_attn.o_proj, value: 1.0 }
919
+ - { filter: model.language_model.layers.35.mlp.down_proj, value: 1.0 }
920
+ - { filter: model.language_model.layers.36.self_attn.o_proj, value: 1.0 }
921
+ - { filter: model.language_model.layers.36.mlp.down_proj, value: 1.0 }
922
+ - { filter: model.language_model.layers.37.self_attn.o_proj, value: 1.0 }
923
+ - { filter: model.language_model.layers.37.mlp.down_proj, value: 1.0 }
924
+ - { filter: model.language_model.layers.38.self_attn.o_proj, value: 1.0 }
925
+ - { filter: model.language_model.layers.38.mlp.down_proj, value: 1.0 }
926
+ - { filter: model.language_model.layers.39.self_attn.o_proj, value: 1.0 }
927
+ - { filter: model.language_model.layers.39.mlp.down_proj, value: 1.0 }
928
+ - { filter: model.language_model.layers.40.self_attn.o_proj, value: 1.0 }
929
+ - { filter: model.language_model.layers.40.mlp.down_proj, value: 1.0 }
930
+ - { filter: model.language_model.layers.41.self_attn.o_proj, value: 1.0 }
931
+ - { filter: model.language_model.layers.41.mlp.down_proj, value: 1.0 }
932
+ - { value: 0.0 }
933
+
934
+ - model: ./Ateron/Gemma-4-Dark-Thoughts-V2-31B
935
+ parameters:
936
+ density:
937
+ - { filter: norm, value: 1.0 }
938
+ - { filter: layer_scalar, value: 1.0 }
939
+ - { value: 0.75 }
940
+ weight:
941
+ - { filter: embed_tokens, value: 0.0 }
942
+ - { filter: lm_head, value: 0.0 }
943
+ - { filter: embed_vision, value: 0.0 }
944
+ - { filter: vision_tower, value: 0.0 }
945
+ - { filter: multi_modal_projector, value: 0.0 }
946
+ - { filter: model.language_model.layers.24.self_attn.v_proj, value: 0.5 }
947
+ - { filter: model.language_model.layers.24.self_attn.o_proj, value: 0.5 }
948
+ - { filter: model.language_model.layers.24.mlp.down_proj, value: 0.5 }
949
+ - { filter: model.language_model.layers.25.self_attn.v_proj, value: 0.5 }
950
+ - { filter: model.language_model.layers.25.self_attn.o_proj, value: 0.5 }
951
+ - { filter: model.language_model.layers.25.mlp.down_proj, value: 0.5 }
952
+ - { filter: model.language_model.layers.26.self_attn.v_proj, value: 0.5 }
953
+ - { filter: model.language_model.layers.26.self_attn.o_proj, value: 0.5 }
954
+ - { filter: model.language_model.layers.26.mlp.down_proj, value: 0.5 }
955
+ - { filter: model.language_model.layers.27.self_attn.v_proj, value: 0.5 }
956
+ - { filter: model.language_model.layers.27.self_attn.o_proj, value: 0.5 }
957
+ - { filter: model.language_model.layers.27.mlp.down_proj, value: 0.5 }
958
+ - { filter: model.language_model.layers.28.self_attn.v_proj, value: 0.5 }
959
+ - { filter: model.language_model.layers.28.self_attn.o_proj, value: 0.5 }
960
+ - { filter: model.language_model.layers.28.mlp.down_proj, value: 0.5 }
961
+ - { filter: model.language_model.layers.29.self_attn.o_proj, value: 0.5 }
962
+ - { filter: model.language_model.layers.29.mlp.down_proj, value: 0.5 }
963
+ - { filter: model.language_model.layers.30.self_attn.v_proj, value: 0.5 }
964
+ - { filter: model.language_model.layers.30.self_attn.o_proj, value: 0.5 }
965
+ - { filter: model.language_model.layers.30.mlp.down_proj, value: 0.5 }
966
+ - { filter: model.language_model.layers.31.self_attn.v_proj, value: 0.5 }
967
+ - { filter: model.language_model.layers.31.self_attn.o_proj, value: 0.5 }
968
+ - { filter: model.language_model.layers.31.mlp.down_proj, value: 0.5 }
969
+ - { filter: model.language_model.layers.32.self_attn.v_proj, value: 0.5 }
970
+ - { filter: model.language_model.layers.32.self_attn.o_proj, value: 0.5 }
971
+ - { filter: model.language_model.layers.32.mlp.down_proj, value: 0.5 }
972
+ - { filter: model.language_model.layers.33.self_attn.v_proj, value: 0.5 }
973
+ - { filter: model.language_model.layers.33.self_attn.o_proj, value: 0.5 }
974
+ - { filter: model.language_model.layers.33.mlp.down_proj, value: 0.5 }
975
+ - { filter: model.language_model.layers.34.self_attn.v_proj, value: 0.5 }
976
+ - { filter: model.language_model.layers.34.self_attn.o_proj, value: 0.5 }
977
+ - { filter: model.language_model.layers.34.mlp.down_proj, value: 0.5 }
978
+ - { filter: model.language_model.layers.35.self_attn.o_proj, value: 0.5 }
979
+ - { filter: model.language_model.layers.35.mlp.down_proj, value: 0.5 }
980
+ - { filter: model.language_model.layers.36.self_attn.v_proj, value: 0.5 }
981
+ - { filter: model.language_model.layers.36.self_attn.o_proj, value: 0.5 }
982
+ - { filter: model.language_model.layers.36.mlp.down_proj, value: 0.5 }
983
+ - { filter: model.language_model.layers.37.self_attn.v_proj, value: 0.5 }
984
+ - { filter: model.language_model.layers.37.self_attn.o_proj, value: 0.5 }
985
+ - { filter: model.language_model.layers.37.mlp.down_proj, value: 0.5 }
986
+ - { filter: model.language_model.layers.38.self_attn.v_proj, value: 0.5 }
987
+ - { filter: model.language_model.layers.38.self_attn.o_proj, value: 0.5 }
988
+ - { filter: model.language_model.layers.38.mlp.down_proj, value: 0.5 }
989
+ - { filter: model.language_model.layers.39.self_attn.v_proj, value: 0.5 }
990
+ - { filter: model.language_model.layers.39.self_attn.o_proj, value: 0.5 }
991
+ - { filter: model.language_model.layers.39.mlp.down_proj, value: 0.5 }
992
+ - { filter: model.language_model.layers.40.self_attn.v_proj, value: 0.5 }
993
+ - { filter: model.language_model.layers.40.self_attn.o_proj, value: 0.5 }
994
+ - { filter: model.language_model.layers.40.mlp.down_proj, value: 0.5 }
995
+ - { filter: model.language_model.layers.41.self_attn.o_proj, value: 0.5 }
996
+ - { filter: model.language_model.layers.41.mlp.down_proj, value: 0.5 }
997
+ - { filter: model.language_model.layers.42.self_attn.v_proj, value: 0.5 }
998
+ - { filter: model.language_model.layers.42.self_attn.o_proj, value: 0.5 }
999
+ - { filter: model.language_model.layers.42.mlp.down_proj, value: 0.5 }
1000
+ - { filter: model.language_model.layers.43.self_attn.v_proj, value: 0.5 }
1001
+ - { filter: model.language_model.layers.43.self_attn.o_proj, value: 0.5 }
1002
+ - { filter: model.language_model.layers.43.mlp.down_proj, value: 0.5 }
1003
+ - { filter: model.language_model.layers.44.self_attn.v_proj, value: 0.5 }
1004
+ - { filter: model.language_model.layers.44.self_attn.o_proj, value: 0.5 }
1005
+ - { filter: model.language_model.layers.44.mlp.down_proj, value: 0.5 }
1006
+ - { filter: model.language_model.layers.45.self_attn.v_proj, value: 0.5 }
1007
+ - { filter: model.language_model.layers.45.self_attn.o_proj, value: 0.5 }
1008
+ - { filter: model.language_model.layers.45.mlp.down_proj, value: 0.5 }
1009
+ - { filter: model.language_model.layers.46.self_attn.v_proj, value: 0.5 }
1010
+ - { filter: model.language_model.layers.46.self_attn.o_proj, value: 0.5 }
1011
+ - { filter: model.language_model.layers.46.mlp.down_proj, value: 0.5 }
1012
+ - { filter: model.language_model.layers.47.self_attn.o_proj, value: 0.5 }
1013
+ - { filter: model.language_model.layers.47.mlp.down_proj, value: 0.5 }
1014
+ - { filter: model.language_model.layers.48.self_attn.v_proj, value: 0.3 }
1015
+ - { filter: model.language_model.layers.48.self_attn.o_proj, value: 0.3 }
1016
+ - { filter: model.language_model.layers.48.mlp.down_proj, value: 0.3 }
1017
+ - { filter: model.language_model.layers.49.self_attn.v_proj, value: 0.3 }
1018
+ - { filter: model.language_model.layers.49.self_attn.o_proj, value: 0.3 }
1019
+ - { filter: model.language_model.layers.49.mlp.down_proj, value: 0.3 }
1020
+ - { filter: model.language_model.layers.50.self_attn.v_proj, value: 0.3 }
1021
+ - { filter: model.language_model.layers.50.self_attn.o_proj, value: 0.3 }
1022
+ - { filter: model.language_model.layers.50.mlp.down_proj, value: 0.3 }
1023
+ - { filter: model.language_model.layers.51.self_attn.v_proj, value: 0.3 }
1024
+ - { filter: model.language_model.layers.51.self_attn.o_proj, value: 0.3 }
1025
+ - { filter: model.language_model.layers.51.mlp.down_proj, value: 0.3 }
1026
+ - { filter: model.language_model.layers.52.self_attn.v_proj, value: 0.3 }
1027
+ - { filter: model.language_model.layers.52.self_attn.o_proj, value: 0.3 }
1028
+ - { filter: model.language_model.layers.52.mlp.down_proj, value: 0.3 }
1029
+ - { filter: model.language_model.layers.53.self_attn.o_proj, value: 0.3 }
1030
+ - { filter: model.language_model.layers.53.mlp.down_proj, value: 0.3 }
1031
+ - { value: 0.0 }
1032
+
1033
+ - model: ./sophosympatheia/Glistening-Gem-31B-v2.0
1034
+ parameters:
1035
+ density:
1036
+ - { filter: norm, value: 1.0 }
1037
+ - { filter: layer_scalar, value: 1.0 }
1038
+ - { value: 0.60 }
1039
+ weight:
1040
+ - { filter: embed_tokens, value: 0.0 }
1041
+ - { filter: lm_head, value: 0.0 }
1042
+ - { filter: embed_vision, value: 0.0 }
1043
+ - { filter: vision_tower, value: 0.0 }
1044
+ - { filter: multi_modal_projector, value: 0.0 }
1045
+ - { filter: model.language_model.layers.12.self_attn.v_proj, value: 0.22 }
1046
+ - { filter: model.language_model.layers.12.self_attn.o_proj, value: 0.22 }
1047
+ - { filter: model.language_model.layers.12.mlp.down_proj, value: 0.22 }
1048
+ - { filter: model.language_model.layers.13.self_attn.v_proj, value: 0.22 }
1049
+ - { filter: model.language_model.layers.13.self_attn.o_proj, value: 0.22 }
1050
+ - { filter: model.language_model.layers.13.mlp.down_proj, value: 0.22 }
1051
+ - { filter: model.language_model.layers.14.self_attn.v_proj, value: 0.22 }
1052
+ - { filter: model.language_model.layers.14.self_attn.o_proj, value: 0.22 }
1053
+ - { filter: model.language_model.layers.14.mlp.down_proj, value: 0.22 }
1054
+ - { filter: model.language_model.layers.15.self_attn.v_proj, value: 0.22 }
1055
+ - { filter: model.language_model.layers.15.self_attn.o_proj, value: 0.22 }
1056
+ - { filter: model.language_model.layers.15.mlp.down_proj, value: 0.22 }
1057
+ - { filter: model.language_model.layers.16.self_attn.v_proj, value: 0.22 }
1058
+ - { filter: model.language_model.layers.16.self_attn.o_proj, value: 0.22 }
1059
+ - { filter: model.language_model.layers.16.mlp.down_proj, value: 0.22 }
1060
+ - { filter: model.language_model.layers.17.self_attn.o_proj, value: 0.22 }
1061
+ - { filter: model.language_model.layers.17.mlp.down_proj, value: 0.22 }
1062
+ - { filter: model.language_model.layers.18.self_attn.v_proj, value: 0.22 }
1063
+ - { filter: model.language_model.layers.18.self_attn.o_proj, value: 0.22 }
1064
+ - { filter: model.language_model.layers.18.mlp.down_proj, value: 0.22 }
1065
+ - { filter: model.language_model.layers.19.self_attn.v_proj, value: 0.22 }
1066
+ - { filter: model.language_model.layers.19.self_attn.o_proj, value: 0.22 }
1067
+ - { filter: model.language_model.layers.19.mlp.down_proj, value: 0.22 }
1068
+ - { filter: model.language_model.layers.20.self_attn.v_proj, value: 0.22 }
1069
+ - { filter: model.language_model.layers.20.self_attn.o_proj, value: 0.22 }
1070
+ - { filter: model.language_model.layers.20.mlp.down_proj, value: 0.22 }
1071
+ - { filter: model.language_model.layers.21.self_attn.v_proj, value: 0.22 }
1072
+ - { filter: model.language_model.layers.21.self_attn.o_proj, value: 0.22 }
1073
+ - { filter: model.language_model.layers.21.mlp.down_proj, value: 0.22 }
1074
+ - { filter: model.language_model.layers.22.self_attn.v_proj, value: 0.22 }
1075
+ - { filter: model.language_model.layers.22.self_attn.o_proj, value: 0.22 }
1076
+ - { filter: model.language_model.layers.22.mlp.down_proj, value: 0.22 }
1077
+ - { filter: model.language_model.layers.23.self_attn.o_proj, value: 0.22 }
1078
+ - { filter: model.language_model.layers.23.mlp.down_proj, value: 0.22 }
1079
+ - { filter: model.language_model.layers.24.self_attn.v_proj, value: 0.22 }
1080
+ - { filter: model.language_model.layers.24.self_attn.o_proj, value: 0.22 }
1081
+ - { filter: model.language_model.layers.24.mlp.down_proj, value: 0.22 }
1082
+ - { filter: model.language_model.layers.25.self_attn.v_proj, value: 0.22 }
1083
+ - { filter: model.language_model.layers.25.self_attn.o_proj, value: 0.22 }
1084
+ - { filter: model.language_model.layers.25.mlp.down_proj, value: 0.22 }
1085
+ - { filter: model.language_model.layers.26.self_attn.v_proj, value: 0.22 }
1086
+ - { filter: model.language_model.layers.26.self_attn.o_proj, value: 0.22 }
1087
+ - { filter: model.language_model.layers.26.mlp.down_proj, value: 0.22 }
1088
+ - { filter: model.language_model.layers.27.self_attn.v_proj, value: 0.22 }
1089
+ - { filter: model.language_model.layers.27.self_attn.o_proj, value: 0.22 }
1090
+ - { filter: model.language_model.layers.27.mlp.down_proj, value: 0.22 }
1091
+ - { filter: model.language_model.layers.28.self_attn.v_proj, value: 0.22 }
1092
+ - { filter: model.language_model.layers.28.self_attn.o_proj, value: 0.22 }
1093
+ - { filter: model.language_model.layers.28.mlp.down_proj, value: 0.22 }
1094
+ - { filter: model.language_model.layers.29.self_attn.o_proj, value: 0.22 }
1095
+ - { filter: model.language_model.layers.29.mlp.down_proj, value: 0.22 }
1096
+ - { filter: model.language_model.layers.30.self_attn.v_proj, value: 0.22 }
1097
+ - { filter: model.language_model.layers.30.self_attn.o_proj, value: 0.22 }
1098
+ - { filter: model.language_model.layers.30.mlp.down_proj, value: 0.22 }
1099
+ - { filter: model.language_model.layers.31.self_attn.v_proj, value: 0.22 }
1100
+ - { filter: model.language_model.layers.31.self_attn.o_proj, value: 0.22 }
1101
+ - { filter: model.language_model.layers.31.mlp.down_proj, value: 0.22 }
1102
+ - { filter: model.language_model.layers.32.self_attn.v_proj, value: 0.22 }
1103
+ - { filter: model.language_model.layers.32.self_attn.o_proj, value: 0.22 }
1104
+ - { filter: model.language_model.layers.32.mlp.down_proj, value: 0.22 }
1105
+ - { filter: model.language_model.layers.33.self_attn.v_proj, value: 0.22 }
1106
+ - { filter: model.language_model.layers.33.self_attn.o_proj, value: 0.22 }
1107
+ - { filter: model.language_model.layers.33.mlp.down_proj, value: 0.22 }
1108
+ - { filter: model.language_model.layers.34.self_attn.v_proj, value: 0.22 }
1109
+ - { filter: model.language_model.layers.34.self_attn.o_proj, value: 0.22 }
1110
+ - { filter: model.language_model.layers.34.mlp.down_proj, value: 0.22 }
1111
+ - { filter: model.language_model.layers.35.self_attn.o_proj, value: 0.22 }
1112
+ - { filter: model.language_model.layers.35.mlp.down_proj, value: 0.22 }
1113
+ - { filter: model.language_model.layers.36.self_attn.v_proj, value: 0.22 }
1114
+ - { filter: model.language_model.layers.36.self_attn.o_proj, value: 0.22 }
1115
+ - { filter: model.language_model.layers.36.mlp.down_proj, value: 0.22 }
1116
+ - { filter: model.language_model.layers.37.self_attn.v_proj, value: 0.22 }
1117
+ - { filter: model.language_model.layers.37.self_attn.o_proj, value: 0.22 }
1118
+ - { filter: model.language_model.layers.37.mlp.down_proj, value: 0.22 }
1119
+ - { filter: model.language_model.layers.38.self_attn.v_proj, value: 0.22 }
1120
+ - { filter: model.language_model.layers.38.self_attn.o_proj, value: 0.22 }
1121
+ - { filter: model.language_model.layers.38.mlp.down_proj, value: 0.22 }
1122
+ - { filter: model.language_model.layers.39.self_attn.v_proj, value: 0.22 }
1123
+ - { filter: model.language_model.layers.39.self_attn.o_proj, value: 0.22 }
1124
+ - { filter: model.language_model.layers.39.mlp.down_proj, value: 0.22 }
1125
+ - { filter: model.language_model.layers.40.self_attn.v_proj, value: 0.22 }
1126
+ - { filter: model.language_model.layers.40.self_attn.o_proj, value: 0.22 }
1127
+ - { filter: model.language_model.layers.40.mlp.down_proj, value: 0.22 }
1128
+ - { filter: model.language_model.layers.41.self_attn.o_proj, value: 0.22 }
1129
+ - { filter: model.language_model.layers.41.mlp.down_proj, value: 0.22 }
1130
+ - { value: 0.0 }
1131
+
1132
+ - model: ./bgg1996/Melinoe-Gemma4-31B-VL-heretic
1133
+ parameters:
1134
+ density:
1135
+ - { filter: norm, value: 1.0 }
1136
+ - { filter: layer_scalar, value: 1.0 }
1137
+ - { value: 0.80 }
1138
+ weight:
1139
+ - { filter: embed_tokens, value: 0.0 }
1140
+ - { filter: lm_head, value: 0.0 }
1141
+ - { filter: embed_vision, value: 0.0 }
1142
+ - { filter: vision_tower, value: 0.0 }
1143
+ - { filter: multi_modal_projector, value: 0.0 }
1144
+ - { filter: model.language_model.layers.42.self_attn.v_proj, value: 0.3 }
1145
+ - { filter: model.language_model.layers.42.self_attn.o_proj, value: 0.3 }
1146
+ - { filter: model.language_model.layers.42.mlp.down_proj, value: 0.3 }
1147
+ - { filter: model.language_model.layers.43.self_attn.v_proj, value: 0.3 }
1148
+ - { filter: model.language_model.layers.43.self_attn.o_proj, value: 0.3 }
1149
+ - { filter: model.language_model.layers.43.mlp.down_proj, value: 0.3 }
1150
+ - { filter: model.language_model.layers.44.self_attn.v_proj, value: 0.3 }
1151
+ - { filter: model.language_model.layers.44.self_attn.o_proj, value: 0.3 }
1152
+ - { filter: model.language_model.layers.44.mlp.down_proj, value: 0.3 }
1153
+ - { filter: model.language_model.layers.45.self_attn.v_proj, value: 0.3 }
1154
+ - { filter: model.language_model.layers.45.self_attn.o_proj, value: 0.3 }
1155
+ - { filter: model.language_model.layers.45.mlp.down_proj, value: 0.3 }
1156
+ - { filter: model.language_model.layers.46.self_attn.v_proj, value: 0.3 }
1157
+ - { filter: model.language_model.layers.46.self_attn.o_proj, value: 0.3 }
1158
+ - { filter: model.language_model.layers.46.mlp.down_proj, value: 0.3 }
1159
+ - { filter: model.language_model.layers.47.self_attn.o_proj, value: 0.3 }
1160
+ - { filter: model.language_model.layers.47.mlp.down_proj, value: 0.3 }
1161
+ - { filter: model.language_model.layers.48.self_attn.v_proj, value: 0.3 }
1162
+ - { filter: model.language_model.layers.48.self_attn.o_proj, value: 0.3 }
1163
+ - { filter: model.language_model.layers.48.mlp.down_proj, value: 0.3 }
1164
+ - { filter: model.language_model.layers.49.self_attn.v_proj, value: 0.3 }
1165
+ - { filter: model.language_model.layers.49.self_attn.o_proj, value: 0.3 }
1166
+ - { filter: model.language_model.layers.49.mlp.down_proj, value: 0.3 }
1167
+ - { filter: model.language_model.layers.50.self_attn.v_proj, value: 0.3 }
1168
+ - { filter: model.language_model.layers.50.self_attn.o_proj, value: 0.3 }
1169
+ - { filter: model.language_model.layers.50.mlp.down_proj, value: 0.3 }
1170
+ - { filter: model.language_model.layers.51.self_attn.v_proj, value: 0.3 }
1171
+ - { filter: model.language_model.layers.51.self_attn.o_proj, value: 0.3 }
1172
+ - { filter: model.language_model.layers.51.mlp.down_proj, value: 0.3 }
1173
+ - { filter: model.language_model.layers.52.self_attn.v_proj, value: 0.3 }
1174
+ - { filter: model.language_model.layers.52.self_attn.o_proj, value: 0.3 }
1175
+ - { filter: model.language_model.layers.52.mlp.down_proj, value: 0.3 }
1176
+ - { filter: model.language_model.layers.53.self_attn.o_proj, value: 0.3 }
1177
+ - { filter: model.language_model.layers.53.mlp.down_proj, value: 0.3 }
1178
+ - { value: 0.0 }
1179
+ ```
1180
+
1181
+ </details>
1182
+ </div>
1183
+ <div class="sx-card C"><h3>&#9670; &nbsp;Afterthought</h3><p><b style="color:#eef3f9;">You can't stump over scotoma-2.</b> Checking benchmarks like caliperbench objectively, almost none of the merges beat their best parent when it comes to adjective and adverb density. So, instead of just blending scotoma-2 in again, I compared the delta of it's layers to the base model and gave it a weight of 1.0 over those. No idea if that survived contact with the actual prose on the long run, though.</p><hr class="sx-rule"><p><b style="color:#eef3f9;">Samplers and Thinking.</b> Samplers are... yeah, they are. I give you a serving suggestion below, but go with whatever works for you. Thinking also works, but some people always report that thinking breaks for some merges, while it doesn't for others, so I still don't know if it's truly merge, front-end or even back-end related. So; just assume it's a no-think model if it breaks for you and we're good.</p></div>
1184
+
1185
+ <div class="sx-sep right"><i></i><b><em>&#9776;</em>Samplers</b><i></i></div>
1186
+ <div class="sx-kv">
1187
+ <div class="h">Setting</div><div class="h">Value</div>
1188
+ <div>Temperature</div><div class="v">0.9</div>
1189
+ <div>Min-P</div><div class="v">0.05</div>
1190
+ <div>Top-K / Top-P</div><div class="v">off / 1.0</div>
1191
+ <div>Repetition penalty</div><div class="v">off</div>
1192
+ <div>DRY multiplier</div><div class="v">0.8</div>
1193
+ </div>
1194
+
1195
+ <div class="sx-sep"><i></i><b><em>&#9686;&#9687;</em>Usage Like Always</b><i></i></div>
1196
+ <div class="sx-card L">
1197
+ <p>&#9670; &nbsp;<b style="color:#eef3f9;">Thinking.</b> Put <b>&lt;|think|&gt;</b> at the start of the system prompt, or let your front-end handle the Gemma 4 template. Reasoning gives better recall capability, but you might lose a bit on ERP pacing.</p>
1198
+ <hr class="sx-rule">
1199
+ <p>&#9670; &nbsp;<b style="color:#eef3f9;">Give it a scene.</b> A sophisticated card will always get more nuanced output of this model than a few lines, just saying.</p>
1200
+ </div>
1201
+
1202
+ <div class="sx-sep right"><i></i><b><em>&#9686;&#9687;</em>Thankus Maximus</b><i></i></div>
1203
+ <div class="sx-card R sx-cred">
1204
+ <ul>
1205
+ <li><b style="color:#eef3f9;"><a href="https://huggingface.co/AesSedai">AesSedai</a> / <a href="https://huggingface.co/ReadyArt">ReadyArt</a>:</b> scotoma-2 is just a banger, and you know it Aes.</li>
1206
+ <li><b style="color:#eef3f9;"><a href="https://huggingface.co/bgg1996">bgg1996</a>:</b> Melinoe-VL-heretic, doing two different jobs here on two different tensor families, cool thing.</li>
1207
+ <li><b style="color:#eef3f9;"><a href="https://huggingface.co/sophosympatheia">sophosympatheia</a>:</b> Glistening-Gem v2.0 and v2.1, kind of being the glue here.</li>
1208
+ <li><b style="color:#eef3f9;"><a href="https://huggingface.co/zerofata">zerofata</a>:</b> MeroMero v2, and the mergekit fork obviously, your tunes going into anything should also speak for itself.</li>
1209
+ <li><b style="color:#eef3f9;"><a href="https://huggingface.co/Ateron">Ateron</a>:</b> Dark-Thoughts V2. The rat guy that cursed me with the habit of merging. Thank you Mr. Rat, this was a good one.</li>
1210
+ <li><b style="color:#eef3f9;"><a href="https://huggingface.co/Gryphe">Gryphe</a>:</b> for the untied head.</li>
1211
+ <li><b style="color:#eef3f9;"><a href="https://huggingface.co/arcee-ai">Charles Goddard/ArceeAI</a>:</b> For the mergekit and its continuation.</li>
1212
+ <li><b style="color:#eef3f9;"><a href="https://www.reddit.com/r/BeaverAI/">The BeaverAI community</a>:</b> for the feedback that once again lead to actually shaping this one significantly.</li>
1213
+ <li><b style="color:#eef3f9;">Google DeepMind &amp; the open-source community:</b> for the base model and everything that ended up inside this merge.</li>
1214
+ </ul>
1215
+ </div>
1216
+
1217
+ <div class="sx-foot">&#9686; &middot; &#9687;</div>
1218
+
1219
+ </div>
1220
+ </div>