Mirai

Mirai is a developer tool. Mirai Labs builds uzu, a Rust inference engine, plus compact 4-bit and 8-bit Qwen, LFM and Muse-Glimmer models for realtime, offline AI on iPhone, iPad and Mac, benchmarked against MLX and llama.cpp.

Mirai is an on-device AI stack for Apple hardware, built by Mirai Labs around a single premise: interactive assistants fail when the model is slow. The company ships three connected pieces. uzu is a Rust inference engine written for batch size one, the pattern every local request actually follows. A family of compact 4-bit and 8-bit models (Qwen, LFM and Muse-Glimmer variants) is tuned for memory-bound decoding on iPhone, iPad and Mac. A conversion and optimization toolkit handles quantization, compressing weights without wrecking interaction latency. The site benchmarks this stack against MLX and llama.cpp. The intended experience is not a chatbox. The homepage demo shows a local model planning a short tool sequence, creating a calendar event, inviting attendees and posting to Slack, while reporting roughly 1,238 tokens per second and a 71ms time to first token on an M4 Pro in airplane mode. The pitch is that one instruction should resolve several services without launching an app for each. That makes Mirai relevant to developers building assistants, productivity tools or any interface where a sub-200ms response matters. The work is deliberately full stack: model architecture, inference engine, quantization and the application layer are co-designed for Apple Silicon rather than ported from cloud serving. Mirai positions itself as a frontier lab for local intelligence rather than a wrapper around existing runtimes. If your alternative is a cloud API with network latency, or llama.cpp on a laptop, Mirai is competing on the last few hundred milliseconds and on privacy by default. Caveats are structural. The stack targets Apple devices only, so Android and Windows are out of scope. Several performance figures come from vendor benchmarks and a single demo device. Public documentation, tooling maturity and pricing are thin compared with established runtimes, and the app-layer integrations shown are illustrative rather than a shipped catalog. Expect to evaluate it as early infrastructure for engineers, not as a finished consumer assistant.