<![CDATA[NUMA aware inference with lk_moe / lvllm / lsglang]]>

<![CDATA[NUMA aware inference with lk_moe / lvllm / lsglang]]>Links:

Open source implementation of lk-moe: xiaotu-moe

These inference engines seem to implement features that greatly improve CPU and CPU offload performance with the NUMA parallel architecture, still to be tested, would appreciate benchmarks! (There are some numbers in the GitHub repos.)

No AI used

]]>https://airigs.net/topic/5/numa-aware-inference-with-lk_moe-lvllm-lsglangRSS for NodeMon, 02 Nov 2026 07:03:07 GMTMon, 28 Sep 2026 16:25:26 GMT60