correct example?
#1
by vladmandic - opened
example in readme downloads weights, instantiates transformer model from original config, patches transformer - but never actually uses downloaded weights?
Yeah, this repo originally started as a testbed for data-free quantization, but I later overrode it with a calibrated approach, so things are a bit mixed up at the moment. I’ve updated the README to clarify both approaches. Just note that these are my attempts to get Nunchaku working with MiniMax-H3, so the quality may not be optimal yet.
understood - thanks.
btw, why not prune the transformer weights (e.g. https://huggingface.co/multimodalart/MiniMax-H3-Pruned)?
It's a good recommendation. I will give it a try