<![CDATA[VLLM with mixed quantization]]>Findings from Minachist's huggingface profile.

Before anything, there's a really good blog post detailing the internals in Minachist's blog, found here.

Turns out these quantizations are the best I've found (compared to their size, for vLLM) by a HUGE margin, (photos from the blog linked above):

image.jpeg

image.jpeg

Sources (and good quantizations/resources):
https://minamism.com/posts/vllm-int5-7-quantization/
https://huggingface.co/Minachist/Qwen3.8-27B-INT6-Mixed-AutoRound
https://huggingface.co/Minachist/Qwen3.8-Flash-Next-INT4-Mixed-AutoRound

No AI used
]]>https://airigs.net/topic/6/vllm-with-mixed-quantizationRSS for NodeMon, 02 Nov 2026 07:04:02 GMTMon, 28 Sep 2026 16:30:26 GMT60