license: cc-by-nc-4.0
tags:
- not-for-all-audiences
Quantized using 200 samples of 8192 tokens from an RP-oriented PIPPA dataset.
Branches:
main--measurement.json2.25b6h-- 2.25bpw, 6bit lm_head3.5b6h-- 3.5bpw, 6bit lm_head3.7b6h-- 3.7bpw, 6bit lm_head5b6h-- 5bpw, 6bit lm_head6b6h-- 6bpw, 6bit lm_head
Requires ExllamaV2 version 0.0.12 and up.
Original model link: Envoid/Fish-8x7B
This quant was broken. It is fixed now.
Original model README below.
Caution: This model may output adult content.
FISH - Fusion of Intuitive Semantic Heuristics
For this model I took the following LoRAs
SeanWu25/Mixtral_8x7b_Medicine
and
And merged them onto their base model.
I then did a simple linear merge between them, an experimental unreleased 8x7B model, and an unreleased model that was a intermediate step in creating Envoid/BondBurger-8x7B
The end results are a surprisingly good model for role palying style entertainment.
At first I was disappointed with the results but ended up settling on the following sampler parameters which really bring it to life.
It does eccentuate characters that have particularly aggressive personalities.
Because this is part of the whole SensualNousInstruct family of models it still suffers from the same tokenizer/special tokens weirdness.
This model has only been tested in Q8 GGUF form due to hardware limitations.
It responds well to [INST] do a thing [/INST] instruct style formatting (although uses the ChatML special tokens)