Configuration Parsing Warning:In config.json: "quantization_config.bits" must be an integer
mistralai / Mistral-Medium-3.5-128B-text
Information
Vision tower stripped to save space. Fits in 48gb VRAM with 16k context, kv quantized to 6,6
The KL is kind of high, so I verified it against another quant, and it measured in line.
2.50bpw exl3 quantization of Mistral-Medium-3.5-128B TEXT ONLY via exllamav3.
repo generated automatically with ezexl3.
The KL is kind of high, so I verified it against another quant, and it measured in line.
2.50bpw exl3 quantization of Mistral-Medium-3.5-128B TEXT ONLY via exllamav3.
repo generated automatically with ezexl3.
Repo Data
CLI Download
hf download UnstableLlama/Mistral-Medium-3.5-128B-exl3-2.50bpw --local-dir ./Mistral-Medium-3.5-128B-exl3-2.50bpw
- Downloads last month
- 9
Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support