what's the exact base model used?
using the lora adapter requires using it with the exact base it was trained from, the readme states Base model obliterated DeepSeek-V4-Flash-0731 but points at normal deep seek v4 flash as the base model in the metadata and doesn't like the specific alliteration used, please add a link to the actual base this is trained from.
Idealy base model in the readme/metadata (as in the think hf displays) should point at the actual base model (in this case the abliterated DeepSeek-V4-Flash-0731 not plain DeepSeek-V4-Flash-0731) but at minimum there should be a link
I abliterated the official Deepseek model from scratch prior to training, so there's no other base model to point to.
What is the lora adapter for then if you kept the abliterated base weights haha.
Someone requested the lora adapter, so I provided it 😄, as simple as that. If you know me, I'm open to polite requests.
As for the lora itself, it can be merged with other Deepseek loras, same as any frankenmerges of loras (trained against various checkpoints) have done since forever.
cou
I abliterated the official Deepseek model from scratch prior to training, so there's no other base model to point to.
could you please upload either:
- the full lossless safetensors version of this model (unless you trained the lora with QAT this probably has to be BF16 all the way because the changes the lora training makes probably aren't lossless in MXFP4 + fp8
- a full sized lossless GGUF (again since the lora probably wasn't trained with QAT this probably should be BF16 to be perfect)
- the full saftensors version of your abliteration so i can merge the lora into it and get a lossless base that way.
I'd like to make my own ik_llama.cpp IQ_K quants for this model since they are higher quality then vanilla llama.cpp quants similar to what I just uploaded for base DeepSeek V4 Flash 0731 here https://huggingface.co/KeinNiemand/DeepSeek-V4-Flash-0731-IK_GGUF
HeatSeeker was trained directly against an IQ2-XS GGUF version of my abliteration, not against a BF16/native-MXFP4 base. So this was effectively quantized-base LoRA training, not QAT (the intended goal being a QAT-ish adaptation for <128 GB users).
I do have the highest-fidelity merged representation of the model as actually trained, but it’s a half-terabyte checkpoint and isn’t practical to upload or host.
I can attempt to reconstruct the higher-precision pre-IQ2 abliteration if you want to experiment with merging the LoRA into that. Since the adapter was optimized against the IQ2-XS base, though, I’d consider that an alternate merge rather than the exact HeatSeeker training lineage.
I do have the highest-fidelity merged representation of the model as actually trained, but it’s a half-terabyte checkpoint and isn’t practical to upload or host.
I'd really like full 500GB checkpoint, HF provides 8TB+ of free storage so uploading it should easily be possible (has to be chunked think 200GB max for 1 file) but I guess how practical depends heavily on your internet speed I guess I'm somewhat lucky with 300Mbit/s upload speed, allowing me to upload a 900GB quant set overnight.
If you can't provide perhaps consider uploading both a full or (like native mxfp4/fp8 safetensors which should be far smaller than a 500GB full bf16 or mxfp4/bf16 for a gguf cause fp8 not perfect translation) of your abliteration + the IQ2-XS this was trained against. From there I can merge the lora and reconstruct the full 500GB thing
With the IQ2_XS version of your alliteration + the lora it should be possible to reconstruct that ~500GB full fidelty checkpoint exactly which would be the most faithful base to make more quants from.
Like you said merging the lora into a higher precision base of your abliteration would be an alternate merge but given that it's the same abliteration but higher precision there a non zero chance it may end up better not worse. => I'd like both high and IQ2_XS versions of the alliteration so I can try both merge options.
No idea if training against an IQ2_XS GGUF actually provides pseudo QAT ish adaptation or just makes the final result worse did you do any testing or is there any research on whether that idea actually works? If it doesn't it may just give you a worse model with no gains.
Huihui-DeepSeek-V4-Flash-0731-abliterated this is shown as base model on inspecting gguf from grok
Huihui-DeepSeek-V4-Flash-0731-abliterated this is shown as base model on inspecting gguf from grok
As I noted here:
Yes and no. For lora training purposes in Mswift, I re-abliterated the official Deepseek model from scratch using the huihui recipe (because huihui didn't provide a gguf of the needed quant size that left sufficient vram headroom for the lora training) and then kept the refusal-direction weights frozen during the training.
So it has a complicated history (now having reviewed the 11 day process from idea to trained model):
The basic premise is that I wanted to create an rp finetune of DSv4 Flash 0731. Axolotl (or rather FSDP) did not cleanly cooperate with split model training. Fine, train a gguf lora using modelscope swift. Problem is that to do this we needed to use a single gpu. To leave enough vram headroom, a smaller gguf was needed. Which was actually great because it meant the model could be offered to more people with less hardware.
But, no such already abliterated gguf of sufficient size existed, so I had to abliterate and quantize my own so training could proceed. Training proceeded for 60 hours.
I published the model. People asked for other bit versions. I obliged. Hence, what you're seeing as the base model. 😃
Hope that helps
Huihui-DeepSeek-V4-Flash-0731-abliterated this is shown as base model on inspecting gguf from grok
As I noted here:
Yes and no. For lora training purposes in Mswift, I re-abliterated the official Deepseek model from scratch using the huihui recipe (because huihui didn't provide a gguf of the needed quant size that left sufficient vram headroom for the lora training) and then kept the refusal-direction weights frozen during the training.
So it has a complicated history (now having reviewed the 11 day process from idea to trained model):
The basic premise is that I wanted to create an rp finetune of DSv4 Flash 0731. Axolotl (or rather FSDP) did not cleanly cooperate with split model training. Fine, train a gguf lora using modelscope swift. Problem is that to do this we needed to use a single gpu. To leave enough vram headroom, a smaller gguf was needed. Which was actually great because it meant the model could be offered to more people with less hardware.
But, no such already abliterated gguf of sufficient size existed, so I had to abliterate and quantize my own so training could proceed. Training proceeded for 60 hours.
I published the model. People asked for other bit versions. I obliged. Hence, what you're seeing as the base model. 😃
Hope that helps
So can/will you share that base model or not, if it takes time that's fine but I'd like a yes/no answer sooner rather then later so I know if I should move on to other project maybe test if a plain alliterated DSV4 flash gguf is any good
I too would appreciate a temporary upload of this, the full-precision abliterated model:
I re-abliterated the official Deepseek model from scratch using the huihui recipe (because huihui didn't provide a gguf of the needed quant size that left sufficient vram headroom for the lora training)
As even if the lora was trained on an IQ2, it would be a better base to requantize from.