Inference Optimization

The 2.8T MoE model Kimi K3 requires 1.45 TB of memory, exceeding any laptop's RAM. The Deltafin fork solves this by streaming expert weights from NVMe SSDs to RAM layer by layer. This article breaks down benchmarks on a 128GB M5 Max MacBook Pro, revealing true decode rates, SSD scaling, and practical limitations for local inference.

Three independent measurement waves put numbers on the System One class: 92–214 ms per decision, under 1 cent for 8 requests, JevBench 75.3 — and on re-measurement, the 200x factor shrinks to roughly 25x.