Hugging Face's logo

Undi95
/
Dawn-v2-70B-2.55bpw-h6-exl2

Text Generation
Transformers
Safetensors
llama
Not-For-All-Audiences
nsfw
text-generation-inference
Model card Files
xet
 Community
Undi95 commited on
Commit
05ea9ef
·
1 Parent(s): 1c40dff

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +417 -0
README.md ADDED
@@ -0,0 +1,417 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ tags:
4
+ - not-for-all-audiences
5
+ - nsfw
6
+ ---
7
+
8
+ <center>[<a href="https://huggingface.co/Undi95/Dawn-v2-70B">fp16</a> - <a href="https://huggingface.co/Undi95/Dawn-v2-70B-GGUF">gguf</a> - exl2 : <a href="https://huggingface.co/Undi95/Dawn-v2-70B-2.55bpw-h6-exl2">2.55bpw</a>]</center>
9
+ </br>
10
+ <div style="width: 100%;">
11
+ <img src="https://cdn-uploads.huggingface.co/production/uploads/63ab1241ad514ca8d1430003/Cxcfqi4WdtXCNLnaIqSRB.png" style="width: 75%; min-width: 200px; display: block; margin: auto;">
12
+ </div>
13
+
14
+ <!-- description start -->
15
+ ## Description
16
+
17
+ This repo contains exl2 (2.55bpw, h6) files of Dawn-70B, a merge I have done with the new [layer shuffle](https://github.com/cg123/mergekit/blob/main/mergekit/scripts/layershuffle.py) method from mergekit.
18
+
19
+ [UtopiaXL](https://huggingface.co/Undi95/UtopiaXL-13B) was a huge success for me, I really liked it, so I took the same path to do this 70B: A good base, some psychologic data, some medical data, a little bit of this, of that, and LimaRP at the end as always.
20
+
21
+ NOTE: This repo contain the file [measurement.json](https://huggingface.co/Undi95/Dawn-v2-70B-2.4bpw-h6-exl2/blob/main/measurement.json) needed to do your own exl2 quant (I use [wikitext](https://huggingface.co/datasets/wikitext/resolve/refs%2Fconvert%2Fparquet/wikitext-2-raw-v1/train/0000.parquet)).
22
+ <!-- description end -->
23
+ <!-- description start -->
24
+ ## Models and loras used
25
+
26
+ - [Sao10K/Euryale-1.3-L2-70B](https://huggingface.co/Sao10K/Euryale-1.3-L2-70B)
27
+ - [Xwin-LM/Xwin-LM-70B-V0.1](https://huggingface.co/Xwin-LM/Xwin-LM-70B-V0.1)
28
+ - [ehartford/Samantha-1.11-70b](https://huggingface.co/ehartford/Samantha-1.11-70b)
29
+ - [NousResearch/Nous-Hermes-Llama2-70b](https://huggingface.co/NousResearch/Nous-Hermes-Llama2-70b)
30
+ - [augtoma/qCammel-70-x](https://huggingface.co/augtoma/qCammel-70-x)
31
+ - [jondurbin/airoboros-l2-c70b-3.1.2](https://huggingface.co/jondurbin/airoboros-l2-c70b-3.1.2)
32
+ - [fangloveskari/ORCA_LLaMA_70B_QLoRA](https://huggingface.co/fangloveskari/ORCA_LLaMA_70B_QLoRA)
33
+ - [Doctor-Shotgun/limarpv3-llama2-70b-qlora](https://huggingface.co/Doctor-Shotgun/limarpv3-llama2-70b-qlora)
34
+
35
+ <!-- description end -->
36
+ ## The sauce
37
+ ```
38
+ !mergekit-layershuffle ./Dawn-v2-70B \
39
+ --model Sao10K/Euryale-1.3-L2-70B --weight 0.3 \
40
+ --model Xwin-LM/Xwin-LM-70B-V0.1 --weight 0.2 \
41
+ --model ehartford/Samantha-1.11-70b --weight 0.1 \
42
+ --model NousResearch/Nous-Hermes-Llama2-70b --weight 0.05 \
43
+ --model augtoma/qCammel-70-x --weight 0.05 \
44
+ --model jondurbin/airoboros-l2-c70b-3.1.2 --weight 0.2 \
45
+ --model fangloveskari/ORCA_LLaMA_70B_QLoRA --weight 0.1 \
46
+ --write-yaml Dawn-v2-70B.yaml
47
+
48
+ =========================
49
+
50
+ merge_method: passthrough
51
+ slices:
52
+ - sources:
53
+ - layer_range:
54
+ - 0
55
+ - 1
56
+ model: fangloveskari/ORCA_LLaMA_70B_QLoRA
57
+ - sources:
58
+ - layer_range:
59
+ - 1
60
+ - 2
61
+ model: jondurbin/airoboros-l2-c70b-3.1.2
62
+ - sources:
63
+ - layer_range:
64
+ - 2
65
+ - 3
66
+ model: Sao10K/Euryale-1.3-L2-70B
67
+ - sources:
68
+ - layer_range:
69
+ - 3
70
+ - 4
71
+ model: jondurbin/airoboros-l2-c70b-3.1.2
72
+ - sources:
73
+ - layer_range:
74
+ - 4
75
+ - 5
76
+ model: fangloveskari/ORCA_LLaMA_70B_QLoRA
77
+ - sources:
78
+ - layer_range:
79
+ - 5
80
+ - 6
81
+ model: ehartford/Samantha-1.11-70b
82
+ - sources:
83
+ - layer_range:
84
+ - 6
85
+ - 8
86
+ model: Xwin-LM/Xwin-LM-70B-V0.1
87
+ - sources:
88
+ - layer_range:
89
+ - 8
90
+ - 9
91
+ model: ehartford/Samantha-1.11-70b
92
+ - sources:
93
+ - layer_range:
94
+ - 9
95
+ - 10
96
+ model: Sao10K/Euryale-1.3-L2-70B
97
+ - sources:
98
+ - layer_range:
99
+ - 10
100
+ - 11
101
+ model: ehartford/Samantha-1.11-70b
102
+ - sources:
103
+ - layer_range:
104
+ - 11
105
+ - 12
106
+ model: jondurbin/airoboros-l2-c70b-3.1.2
107
+ - sources:
108
+ - layer_range:
109
+ - 12
110
+ - 13
111
+ model: fangloveskari/ORCA_LLaMA_70B_QLoRA
112
+ - sources:
113
+ - layer_range:
114
+ - 13
115
+ - 14
116
+ model: Sao10K/Euryale-1.3-L2-70B
117
+ - sources:
118
+ - layer_range:
119
+ - 14
120
+ - 15
121
+ model: fangloveskari/ORCA_LLaMA_70B_QLoRA
122
+ - sources:
123
+ - layer_range:
124
+ - 15
125
+ - 16
126
+ model: Sao10K/Euryale-1.3-L2-70B
127
+ - sources:
128
+ - layer_range:
129
+ - 16
130
+ - 17
131
+ model: fangloveskari/ORCA_LLaMA_70B_QLoRA
132
+ - sources:
133
+ - layer_range:
134
+ - 17
135
+ - 18
136
+ model: jondurbin/airoboros-l2-c70b-3.1.2
137
+ - sources:
138
+ - layer_range:
139
+ - 18
140
+ - 19
141
+ model: NousResearch/Nous-Hermes-Llama2-70b
142
+ - sources:
143
+ - layer_range:
144
+ - 19
145
+ - 20
146
+ model: Xwin-LM/Xwin-LM-70B-V0.1
147
+ - sources:
148
+ - layer_range:
149
+ - 20
150
+ - 21
151
+ model: Sao10K/Euryale-1.3-L2-70B
152
+ - sources:
153
+ - layer_range:
154
+ - 21
155
+ - 22
156
+ model: ehartford/Samantha-1.11-70b
157
+ - sources:
158
+ - layer_range:
159
+ - 22
160
+ - 23
161
+ model: jondurbin/airoboros-l2-c70b-3.1.2
162
+ - sources:
163
+ - layer_range:
164
+ - 23
165
+ - 24
166
+ model: augtoma/qCammel-70-x
167
+ - sources:
168
+ - layer_range:
169
+ - 24
170
+ - 25
171
+ model: Sao10K/Euryale-1.3-L2-70B
172
+ - sources:
173
+ - layer_range:
174
+ - 25
175
+ - 27
176
+ model: jondurbin/airoboros-l2-c70b-3.1.2
177
+ - sources:
178
+ - layer_range:
179
+ - 27
180
+ - 28
181
+ model: Xwin-LM/Xwin-LM-70B-V0.1
182
+ - sources:
183
+ - layer_range:
184
+ - 28
185
+ - 29
186
+ model: ehartford/Samantha-1.11-70b
187
+ - sources:
188
+ - layer_range:
189
+ - 29
190
+ - 30
191
+ model: Sao10K/Euryale-1.3-L2-70B
192
+ - sources:
193
+ - layer_range:
194
+ - 30
195
+ - 32
196
+ model: Xwin-LM/Xwin-LM-70B-V0.1
197
+ - sources:
198
+ - layer_range:
199
+ - 32
200
+ - 33
201
+ model: ehartford/Samantha-1.11-70b
202
+ - sources:
203
+ - layer_range:
204
+ - 33
205
+ - 34
206
+ model: augtoma/qCammel-70-x
207
+ - sources:
208
+ - layer_range:
209
+ - 34
210
+ - 35
211
+ model: Xwin-LM/Xwin-LM-70B-V0.1
212
+ - sources:
213
+ - layer_range:
214
+ - 35
215
+ - 37
216
+ model: Sao10K/Euryale-1.3-L2-70B
217
+ - sources:
218
+ - layer_range:
219
+ - 37
220
+ - 38
221
+ model: jondurbin/airoboros-l2-c70b-3.1.2
222
+ - sources:
223
+ - layer_range:
224
+ - 38
225
+ - 39
226
+ model: ehartford/Samantha-1.11-70b
227
+ - sources:
228
+ - layer_range:
229
+ - 39
230
+ - 40
231
+ model: augtoma/qCammel-70-x
232
+ - sources:
233
+ - layer_range:
234
+ - 40
235
+ - 41
236
+ model: Xwin-LM/Xwin-LM-70B-V0.1
237
+ - sources:
238
+ - layer_range:
239
+ - 41
240
+ - 42
241
+ model: ehartford/Samantha-1.11-70b
242
+ - sources:
243
+ - layer_range:
244
+ - 42
245
+ - 43
246
+ model: Sao10K/Euryale-1.3-L2-70B
247
+ - sources:
248
+ - layer_range:
249
+ - 43
250
+ - 44
251
+ model: Xwin-LM/Xwin-LM-70B-V0.1
252
+ - sources:
253
+ - layer_range:
254
+ - 44
255
+ - 45
256
+ model: NousResearch/Nous-Hermes-Llama2-70b
257
+ - sources:
258
+ - layer_range:
259
+ - 45
260
+ - 46
261
+ model: jondurbin/airoboros-l2-c70b-3.1.2
262
+ - sources:
263
+ - layer_range:
264
+ - 46
265
+ - 48
266
+ model: ehartford/Samantha-1.11-70b
267
+ - sources:
268
+ - layer_range:
269
+ - 48
270
+ - 49
271
+ model: Sao10K/Euryale-1.3-L2-70B
272
+ - sources:
273
+ - layer_range:
274
+ - 49
275
+ - 50
276
+ model: Xwin-LM/Xwin-LM-70B-V0.1
277
+ - sources:
278
+ - layer_range:
279
+ - 50
280
+ - 51
281
+ model: jondurbin/airoboros-l2-c70b-3.1.2
282
+ - sources:
283
+ - layer_range:
284
+ - 51
285
+ - 54
286
+ model: fangloveskari/ORCA_LLaMA_70B_QLoRA
287
+ - sources:
288
+ - layer_range:
289
+ - 54
290
+ - 55
291
+ model: jondurbin/airoboros-l2-c70b-3.1.2
292
+ - sources:
293
+ - layer_range:
294
+ - 55
295
+ - 56
296
+ model: fangloveskari/ORCA_LLaMA_70B_QLoRA
297
+ - sources:
298
+ - layer_range:
299
+ - 56
300
+ - 58
301
+ model: jondurbin/airoboros-l2-c70b-3.1.2
302
+ - sources:
303
+ - layer_range:
304
+ - 58
305
+ - 59
306
+ model: Sao10K/Euryale-1.3-L2-70B
307
+ - sources:
308
+ - layer_range:
309
+ - 59
310
+ - 60
311
+ model: Xwin-LM/Xwin-LM-70B-V0.1
312
+ - sources:
313
+ - layer_range:
314
+ - 60
315
+ - 62
316
+ model: jondurbin/airoboros-l2-c70b-3.1.2
317
+ - sources:
318
+ - layer_range:
319
+ - 62
320
+ - 63
321
+ model: Xwin-LM/Xwin-LM-70B-V0.1
322
+ - sources:
323
+ - layer_range:
324
+ - 63
325
+ - 64
326
+ model: fangloveskari/ORCA_LLaMA_70B_QLoRA
327
+ - sources:
328
+ - layer_range:
329
+ - 64
330
+ - 65
331
+ model: NousResearch/Nous-Hermes-Llama2-70b
332
+ - sources:
333
+ - layer_range:
334
+ - 65
335
+ - 66
336
+ model: Sao10K/Euryale-1.3-L2-70B
337
+ - sources:
338
+ - layer_range:
339
+ - 66
340
+ - 67
341
+ model: Xwin-LM/Xwin-LM-70B-V0.1
342
+ - sources:
343
+ - layer_range:
344
+ - 67
345
+ - 68
346
+ model: augtoma/qCammel-70-x
347
+ - sources:
348
+ - layer_range:
349
+ - 68
350
+ - 70
351
+ model: Xwin-LM/Xwin-LM-70B-V0.1
352
+ - sources:
353
+ - layer_range:
354
+ - 70
355
+ - 71
356
+ model: augtoma/qCammel-70-x
357
+ - sources:
358
+ - layer_range:
359
+ - 71
360
+ - 72
361
+ model: Xwin-LM/Xwin-LM-70B-V0.1
362
+ - sources:
363
+ - layer_range:
364
+ - 72
365
+ - 73
366
+ model: Sao10K/Euryale-1.3-L2-70B
367
+ - sources:
368
+ - layer_range:
369
+ - 73
370
+ - 75
371
+ model: jondurbin/airoboros-l2-c70b-3.1.2
372
+ - sources:
373
+ - layer_range:
374
+ - 75
375
+ - 76
376
+ model: Sao10K/Euryale-1.3-L2-70B
377
+ - sources:
378
+ - layer_range:
379
+ - 76
380
+ - 77
381
+ model: augtoma/qCammel-70-x
382
+ - sources:
383
+ - layer_range:
384
+ - 77
385
+ - 78
386
+ model: Xwin-LM/Xwin-LM-70B-V0.1
387
+ - sources:
388
+ - layer_range:
389
+ - 78
390
+ - 79
391
+ model: NousResearch/Nous-Hermes-Llama2-70b
392
+ - sources:
393
+ - layer_range:
394
+ - 79
395
+ - 80
396
+ model: Xwin-LM/Xwin-LM-70B-V0.1
397
+
398
+
399
+ =========================
400
+
401
+ => Applying Doctor-Shotgun/limarpv3-llama2-70b-qlora x 0.35
402
+ ```
403
+ <!-- prompt-template start -->
404
+ ## Prompt template: Alpaca
405
+
406
+ ```
407
+ Below is an instruction that describes a task. Write a response that appropriately completes the request.
408
+
409
+ ### Instruction:
410
+ {prompt}
411
+
412
+ ### Response:
413
+
414
+ ```
415
+ A big thanks to [Charles](https://huggingface.co/chargoddard) for adding the layer shuffle method to his tool [mergekit](https://github.com/cg123/mergekit/tree/main) and [Henky/KoboldAI](https://koboldai.org/) for the machine he let me use.
416
+
417
+ If you want to support me, you can [here](https://ko-fi.com/undiai).