Local-Llm

Running Kimi K3 on an M1 Max

Kimi K3 is 2.8 trillion parameters and 1.56TB of weights on Hugging Face. Deltafin runs it on a 64GB M1 Max by keeping a roughly 60GB int8 spine resident and streaming the 82,432 routed experts off disk in MXFP4 as the router calls for them. About 215GB of disk in streaming mode, ~1.7TB for the full install, and roughly 16 seconds per token.

Sixteen seconds a token. Ouch.