Top 100 Quantization Repositories

Back to Home

Ranking

Ranking	Project Name	Stars	Forks	Language	Open Issues	Description	Last Commit
1	LlamaFactory	71,487	8,718	Python	966	Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)	2026-05-21
2	faster-whisper	23,059	1,880	Python	282	Faster Whisper transcription with CTranslate2	2025-11-19
3	Chinese-LLaMA-Alpaca	18,944	1,852	Python	1	中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)	2026-04-19
4	Qbot	17,392	2,455	Jupyter Notebook	67	[🔥updating ...] AI 自动量化交易机器人(完全本地部署) AI-powered Quantitative Investment Research Platform. 📃 online docs: https://ufund-me.github.io/Qbot ✨ :news: qbot-mini: https://github.com/Charmve/iQuant	2026-03-11
5	qlora	10,909	873	Jupyter Notebook	193	QLoRA: Efficient Finetuning of Quantized LLMs	2024-06-10
6	bitsandbytes	8,219	855	Python	32	Accessible large language models via k-bit quantization for PyTorch.	2026-05-21
7	lit-llama	6,078	521	Python	100	Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.	2025-07-01
8	pngquant	5,640	497	C	55	Lossy PNG compressor — pngquant command based on libimagequant library	2026-03-28
9	AutoGPTQ	5,062	543	Python	241	An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.	2025-04-11
10	CTranslate2	4,493	488	C++	219	Fast inference engine for Transformer models	2026-05-19
11	vector-quantize-pytorch	3,940	326	Python	49	Vector (and Scalar) Quantization, in Pytorch	2026-05-16
12	nunchaku	3,854	252	Python	4	[ICLR2025 Spotlight] SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models	2026-03-07
13	ComfyUI-GGUF	3,647	302	Python	188	GGUF Quantization support for native ComfyUI models	2026-01-12
14	llm-awq	3,539	315	Python	169	[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration	2025-07-17
15	optimum	3,397	644	Python	262	🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools	2026-05-18
16	SageAttention	3,378	421	Cuda	163	[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.	2026-01-17
17	llm-compressor	3,272	521	Python	66	Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM	2026-05-21
18	Pretrained-Language-Model	3,163	642	Python	99	Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.	2024-01-22
19	deepsparse	3,161	192	Python	1	Sparsity-aware deep learning inference runtime for CPUs	2025-06-02
20	GPTQ-for-LLaMa	3,072	452	Python	62	4 bits quantization of LLaMA using GPTQ	2024-07-13
21	nlp-architect	2,933	443	Python	14	A model library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing neural networks	2022-11-07
22	exllama	2,920	223	Python	61	A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.	2023-09-30
23	ComfyUI-nunchaku	2,877	160	Python	6	ComfyUI Plugin of Nunchaku	2026-02-19
24	ao	2,830	508	Python	413	PyTorch native quantization and sparsity for training and inference	2026-05-21
25	Model-Optimizer	2,737	404	Python	52	A unified library of SOTA model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks lik...	2026-05-22
26	pytorch-playground	2,718	623	Python	8	Base pretrained models and datasets in pytorch (MNIST, SVHN, CIFAR10, CIFAR100, STL10, AlexNet, VGG16, VGG19, ResNet, Inception, SqueezeNet)	2022-11-22
27	xTuring	2,666	212	Python	0	Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHX...	2026-03-04
28	neural-compressor	2,643	305	Python	1	SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime	2026-05-22
29	aimet	2,621	451	Python	21	AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.	2026-05-22
30	Awesome-Model-Quantization	2,379	239	-	0	A list of papers, docs, codes about model quantization. This repo is aimed to provide the info for model quantization research, we are continuously improving the project. Welcome to PR the works (pape...	2026-05-11
31	AutoAWQ	2,337	301	Python	201	AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:	2025-05-11
32	mixtral-offloading	2,331	228	Python	22	Run Mixtral-8x7B models in Colab or consumer desktops	2024-04-08
33	Olive	2,316	296	Python	22	Olive: Simplify ML Model Finetuning, Conversion, Quantization, and Optimization for CPUs, GPUs and NPUs.	2026-05-21
34	gptq	2,307	196	Python	26	Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".	2024-03-27
35	micronet	2,273	474	Python	92	micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Ari...	2025-05-06
36	AI-Engineering.academy	2,209	255	Jupyter Notebook	6	Mastering Applied AI, One Concept at a Time	2026-02-27
37	deepseek-ocr.rs	2,164	170	Rust	15	Rust multi‑backend OCR/VLM engine (DeepSeek‑OCR-1/2, PaddleOCR‑VL, DotsOCR) with DSQ quantization and an OpenAI‑compatible server & CLI – run locally without Python.	2026-02-21
38	turbovec	2,091	186	Python	5	A vector index built on TurboQuant, written in Rust with Python bindings	2026-05-21
39	intel-extension-for-pytorch	2,014	315	Python	198	A Python package for extending the official PyTorch that can easily obtain performance on Intel platform	2026-03-30
40	ppq	1,799	275	Python	81	PPL Quantization Tool (PPQ) is a powerful offline neural network quantization tool.	2024-03-28
41	mmrazor	1,671	245	Python	145	OpenMMLab Model Compression Toolbox and Benchmark.	2024-06-11
42	smoothquant	1,649	204	Python	72	[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models	2024-07-12
43	PaddleSlim	1,615	353	Python	22	PaddleSlim is an open-source library for deep model compression and architecture search.	2026-01-04
44	picolm	1,613	201	C	13	Run a 1-billion parameter LLM on a $10 board with 256MB RAM	2026-02-22
45	ai-engineering-interview-questions	1,602	302	Markdown	0	Your Cheat Sheet for AI Engineering Interview – Questions and Answers.	2026-05-21
46	model-optimization	1,571	347	Python	200	A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.	2026-05-19
47	rwkv.cpp	1,569	128	C++	26	INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model	2025-03-23
48	QNNPACK	1,549	221	C	7	Quantized Neural Network PACKage - mobile-optimized implementation of quantized neural network operators	2019-08-28
49	brevitas	1,533	243	Python	178	Brevitas: neural network quantization in PyTorch	2026-05-20
50	turboquant	1,414	175	Python	8	TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration	2026-03-27
51	auto-round	1,404	134	Python	86	A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.	2026-05-22
52	gpu_poor	1,399	89	JavaScript	8	Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization	2024-12-03
53	perfectPixel	1,394	87	Python	1	Refine and quantize messy AI pixel art into clean, perfect pixels.	2026-02-01
54	AQLM	1,320	194	Python	9	Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf and PV-Tuning: Beyond Straight-Through Estimation for Extrem...	2026-02-26
55	Efficient-Computing	1,302	219	Jupyter Notebook	26	Efficient computing methods developed by Huawei Noah's Ark Lab	2024-11-05
56	binance-quantization	1,295	330	Python	7	虚拟货币(BTC、ETH)炒币量化系统项目。币安交易所-量化交易-网格策略实践。火币、OKEX热门交易所未来都支持。最简单收益最靠谱的项目，包教包会。	2023-01-30
57	training_extensions	1,229	466	Python	128	Train, Evaluate, Optimize, Deploy Computer Vision Models via OpenVINO™	2026-05-22
58	AngelSlim	1,219	132	Python	53	Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.	2026-05-21
59	cache-dit	1,178	70	Python	86	A PyTorch-native inference engine with cache, parallelism, quantization for Diffusion Transformers.	2026-05-21
60	nncf	1,164	294	Python	21	Neural Network Compression Framework for enhanced OpenVINO™ inference	2026-05-21
61	clean-fid	1,156	82	Python	31	PyTorch - FID calculation with proper image resizing and quantization steps [CVPR 2022]	2025-08-02
62	GPTQModel	1,154	184	Python	43	LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.	2026-05-22
63	z80ai	1,088	47	Python	6	Z80-μLM is a 2-bit quantized language model small enough to run on an 8-bit Z80 processor. Train conversational models in Python, export them as CP/M .COM binaries, and chat with your vintage computer...	2026-04-29
64	marlin	1,078	88	Python	31	FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.	2024-09-04
65	optimum-quanto	1,041	86	Python	0	A pytorch quantization backend for optimum	2026-04-02
66	YOLOv5-Multibackbone-Compression	1,021	198	Python	55	YOLOv5 Series Multi-backbone(TPH-YOLOv5, Ghostnet, ShuffleNetv2, Mobilenetv3Small, EfficientNetLite, PP-LCNet, SwinTransformer YOLO), Module(CBAM, DCN), Pruning (EagleEye, Network Slimming), Quantizat...	2022-04-29
67	rq-vae-transformer	1,018	113	Jupyter Notebook	10	The official implementation of Autoregressive Image Generation using Residual Quantization (CVPR '22)	2024-01-03
68	MindPipe	1,006	24	Python	0	A powerful model compression framework for LLMs and LVLMs, adapted for NVIDIA GPUs and Huawei Ascend NPUs.	2026-05-20
69	finn	997	298	Python	68	Dataflow compiler for QNN inference on FPGAs	2026-05-21
70	SpargeAttn	995	89	Cuda	59	[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.	2026-02-25
71	YOLOs-CPP	975	142	C++	1	Cross-Platform Production-ready C++ inference engine for YOLO models (v5-v12, YOLO26). Unified API for detection, segmentation, pose estimation, OBB, and classification. Built on ONNX Runtime and Open...	2026-04-13
72	onnx2tf	958	100	Python	2	A tool for converting ONNX files to LiteRT/TFLite/TensorFlow, PyTorch native code (nn.Module), TorchScript (.pt), state_dict (.pt), Exported Program (.pt2), and Dynamo ONNX. It also supports direct co...	2026-04-01
73	pytorch-vqvae	954	140	Python	8	Vector Quantized VAEs - PyTorch Implementation	2023-07-12
74	TinyChatEngine	952	98	C++	35	TinyChatEngine: On-Device LLM Inference Library	2024-07-04
75	tinyengine	943	160	C	36	[NeurIPS 2020] MCUNet: Tiny Deep Learning on IoT Devices; [NeurIPS 2021] MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning; [NeurIPS 2022] MCUNetV3: On-Device Training Under 256K...	2024-11-27
76	hqq	940	90	Python	2	Official implementation of Half-Quadratic Quantization (HQQ)	2026-02-26
77	libimagequant	910	141	Rust	10	Palette quantization library that powers pngquant and other PNG optimizers	2026-02-11
78	OmniQuant	898	78	Python	30	[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.	2025-11-26
79	vqvae	895	102	Jupyter Notebook	7	A pytorch implementation of the vector quantized variational autoencoder (https://arxiv.org/abs/1711.00937)	2022-12-08
80	exllamav3	889	100	Python	71	An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs	2026-05-21
81	MQBench	869	143	Python	11	Model Quantization Benchmark	2025-04-20
82	Awesome-AutoML-and-Lightweight-Models	856	161	-	0	A list of high-quality (newest) AutoML works and lightweight models including 1.) Neural Architecture Search, 2.) Lightweight Structures, 3.) Model Compression, Quantization and Acceleration, 4.) Hype...	2021-06-19
83	omniserve	838	63	C++	47	[MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention	2025-03-06
84	Awesome-Quantization-Papers	822	65	-	2	List of papers related to neural network quantization in recent AI conferences and journals.	2025-03-27
85	Orion	810	59	Python	41	Orion-14B is a family of models includes a 14B foundation LLM, and a series of models: a chat model, a long context model, a quantized model, a RAG fine-tuned model, and an Agent fine-tuned model. Ori...	2024-06-03
86	bitsandbytes	779	47	-	9	Library for 8-bit optimizers and quantization routines.	2022-08-18
87	awesome-emdl	767	168	-	0	Embedded and mobile deep learning research resources	2023-03-14
88	BitBLAS	764	59	Python	33	BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.	2025-08-06
89	Word2Bits	753	34	C++	1	Quantized word vectors that take 8x-16x less space than regular word vectors	2020-03-31
90	SqueezeLLM	720	50	Python	16	[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization	2024-08-13
91	LightCompress	715	80	Python	42	[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.	2026-05-14
92	BNN-PYNQ	705	305	Jupyter Notebook	9	Quantized Neural Networks (QNNs) on PYNQ	2022-01-04
93	deepvac	681	102	Python	16	PyTorch Project Specification.	2021-08-05
94	VPTQ	679	52	Python	26	VPTQ, A Flexible and Extreme low-bit quantization algorithm	2025-04-25
95	Cornucopia-LLaMA-Fin-Chinese	657	67	Python	17	聚宝盆(Cornucopia): 中文金融系列开源可商用大模型，并提供一套高效轻量化的垂直领域LLM训练框架(Pretraining、SFT、RLHF、Quantize等)	2023-06-30
96	KD_Lib	650	61	Python	14	A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.	2023-03-01
97	NOVA	647	22	Python	0	[ICLR 2025] Autoregressive Video Generation without Vector Quantization	2025-10-29
98	deep-vector-quantization	646	57	Jupyter Notebook	3	VQVAEs, GumbelSoftmaxes and friends	2021-11-20
99	hailo_model_zoo	644	83	Python	0	The Hailo Model Zoo includes pre-trained models and a full building and evaluation environment	2026-04-30
100	kill-the-bits	631	122	Python	0	Code for: "And the bit goes down: Revisiting the quantization of neural networks"	2020-11-09