Edit Models filters

Model Tree

Apps

Docker Model Runner

Unsloth Desktop

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

arxiv: 2407.11062

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

100

Base only

Active filters: 2407.11062

sdkyuan/qwen3.8-27B-qat-q2_0-gguf

Text Generation • 27B • Updated 10 days ago • 9.35k • 25

ChenMnZ/Llama-2-13b-EfficientQAT-w2g128

Text Generation • 1B • Updated Jul 22, 2024 • 8

ChenMnZ/Llama-2-13b-EfficientQAT-w2g64

Text Generation • 1B • Updated Jul 22, 2024 • 7

ChenMnZ/Llama-2-13b-EfficientQAT-w3g128

Text Generation • 2B • Updated Jul 22, 2024 • 12

ChenMnZ/Llama-2-13b-EfficientQAT-w4g128

Text Generation • 2B • Updated Jul 22, 2024 • 7

ChenMnZ/Llama-2-70b-EfficientQAT-w2g128

Text Generation • 5B • Updated Jul 22, 2024 • 8

ChenMnZ/Llama-2-70b-EfficientQAT-w2g64

Text Generation • 6B • Updated Jul 22, 2024 • 7

ChenMnZ/Llama-2-70b-EfficientQAT-w3g128

Text Generation • 8B • Updated Jul 22, 2024 • 10 • 1

ChenMnZ/Llama-2-70b-EfficientQAT-w4g128

Text Generation • 10B • Updated Jul 22, 2024 • 12 • 1

ChenMnZ/Llama-2-7b-EfficientQAT-w2g128

Text Generation • 0.7B • Updated Jul 22, 2024 • 8

ChenMnZ/Llama-2-7b-EfficientQAT-w3g128

Text Generation • 1.0B • Updated Jul 22, 2024 • 7

ChenMnZ/Llama-2-7b-EfficientQAT-w4g128

Text Generation • 1B • Updated Jul 22, 2024 • 9

ChenMnZ/Llama-3-70b-EfficientQAT-w2g64

Text Generation • 8B • Updated Jul 22, 2024 • 9 • 1

ChenMnZ/Llama-3-70b-EfficientQAT-w3g128

Text Generation • 10B • Updated Jul 22, 2024 • 7 • 1

ChenMnZ/Llama-3-70b-EfficientQAT-w4g128

Text Generation • 11B • Updated Jul 22, 2024 • 8 • 1

ChenMnZ/Llama-3-70b-instruct-EfficientQAT-w2g128

Text Generation • 7B • Updated Jul 22, 2024 • 7

ChenMnZ/Llama-3-70b-instruct-EfficientQAT-w2g64

Text Generation • 8B • Updated Jul 22, 2024 • 7

ChenMnZ/Llama-3-70b-instruct-EfficientQAT-w3g128

Text Generation • 10B • Updated Jul 22, 2024 • 10

ChenMnZ/Llama-3-70b-instruct-EfficientQAT-w4g128

Text Generation • 11B • Updated Jul 22, 2024 • 9

ChenMnZ/Llama-3-8b-EfficientQAT-w2g128

Text Generation • 2B • Updated Jul 22, 2024 • 7

ChenMnZ/Llama-3-8b-EfficientQAT-w2g64

Text Generation • 2B • Updated Jul 22, 2024 • 9

ChenMnZ/Llama-3-8b-EfficientQAT-w3g128

Text Generation • 2B • Updated Jul 22, 2024 • 82

ChenMnZ/Llama-3-8b-EfficientQAT-w4g128

Text Generation • 2B • Updated Jul 22, 2024 • 10

ChenMnZ/Llama-3-8b-instruct-EfficientQAT-w2g128

Text Generation • 2B • Updated Jul 22, 2024 • 13

ChenMnZ/Llama-3-8b-instruct-EfficientQAT-w2g64

Text Generation • 2B • Updated Jul 22, 2024 • 13

ChenMnZ/Llama-3-8b-instruct-EfficientQAT-w3g128

Text Generation • 2B • Updated Jul 22, 2024 • 10

ChenMnZ/Llama-3-8b-instruct-EfficientQAT-w4g128

Text Generation • 2B • Updated Jul 22, 2024 • 8

ChenMnZ/Llama-2-7b-EfficientQAT-w2g64

Text Generation • 0.8B • Updated Jul 22, 2024 • 11

ChenMnZ/Llama-3-70b-EfficientQAT-w2g128

Text Generation • 7B • Updated Jul 22, 2024 • 9 • 1

ChenMnZ/Llama-2-13b-BlockAP-w2g128

Text Generation • 1B • Updated Jul 21, 2024 • 9