Bonsai 27B: The First 27B AI Model That Runs on Your Phone

A 27-billion-parameter AI model that runs on your phone. That sentence would have sounded absurd six months ago. Today, a company called PrismML made it real with Bonsai 27B — the first model of its capability class to run on a smartphone, and a release that fundamentally changes where AI can operate.

Bonsai 27B is based on Qwen3.6 27B and comes in two variants: a ternary version at 5.9 GB that runs on everyday laptops, and a 1-bit version at just 3.9 GB that fits within the memory budget of an iPhone 17 Pro. Both are multimodal, support a 262K-token context window, and ship under the Apache 2.0 License. Oh, and they support speculative decoding for lossless speed gains on top of the compression.

The Compression Math That Makes It Possible

Here is why this matters. A standard 27B model in 16-bit precision occupies roughly 54 GB. Even an aggressive 4-bit quantization comes in at 18 GB — too large for a phone, too large for most laptops. PrismML's approach is fundamentally different. Instead of traditional quantization, they use ternary weights ({-1, 0, +1}) and binary weights ({-1, +1}) with FP16 group-wise scaling across the entire network — embeddings, attention, MLPs, and the LM head. No higher-precision escape hatches anywhere.

The results speak for themselves:

• Ternary Bonsai 27B: 5.9 GB, retaining 95% of the full-precision baseline across 15 benchmarks
• 1-bit Bonsai 27B: 3.9 GB, retaining 90% of the full-precision baseline
• 1-bit intelligence density: 0.53 per GB — over 10x the full-precision baseline

To put that in perspective, the most aggressive conventional low-bit build of the same model scores significantly lower than 1-bit Bonsai 27B while using 2.5x more memory. This is not incremental progress. It is a Pareto shift.

What Survives the Compression?

The benchmark breakdown reveals something fascinating about what degrades and what does not when you compress a model this aggressively:

• Math (GSM8K, MATH-500, AIME25, AIME26): Ternary retains 97.9% — nearly untouched
• Coding (HumanEval+, MBPP+, LiveCodeBench): Ternary retains 97.0%
• Agentic and tool-calling (BFCL v3, TauBench): Ternary retains 92.5%
• Vision (MMMU Pro, OCRBench): Ternary retains 89.8%

Math and coding — the capabilities that agentic workloads depend on most — are nearly untouched. Tool calling stays within a few points of full precision. The vision tower ships in a compact 4-bit form so the model can process screenshots, documents, and camera input on-device. This is not a stripped-down toy model. It is a model you can trust with real work.

Why Local Execution Changes Everything

The most valuable AI workloads are shifting from single responses to sustained agentic work. An agent does not make one model call — it makes hundreds, each carrying context, producing structured output, and feeding the next step. Running that loop entirely on a cloud API imposes structural constraints: every step is a remote request, per-token costs accumulate, and every plan, tool call, and intermediate result crosses the network — including the user's private files and screen data.

When a model capable of sustained agentic work fits on the device, the economics flip:

• The marginal cost of a hundred-step agent loop is zero
• User data never leaves the machine — privacy by construction
• Assistants work offline, with no latency from network round-trips
• Hybrid deployments route easy tasks locally and reserve cloud models for the hardest steps

This last point is particularly important. PrismML explicitly frames Bonsai 27B as enabling hybrid deployment architectures: a capable local model handles routine and privacy-sensitive tasks, while frontier cloud models are reserved only for the most challenging steps. This collapses the per-task cost of agentic systems dramatically.

Performance on Real Hardware

Speed matters when the model is doing agentic work that requires many sequential calls. Bonsai 27B delivers:

• Up to 163 tokens/second in 1-bit on an NVIDIA GeForce RTX 5090
• Up to 134 tokens/second in Ternary on the same GPU
• Up to 87 tokens/second in 1-bit on an Apple M5 Max
• Up to 58 tokens/second in Ternary on an M5 Max

For context, a typical reading speed is about 250 words per minute. Even the slowest of these configurations generates text far faster than you can read it. On a phone, the 1-bit variant at 3.9 GB fits within the approximately 6 GB of memory that iOS exposes to an app — with room left over for the KV cache and activations needed for a 262K-token context window.

The Bigger Picture: Intelligence Density as a Frontier

PrismML frames this release in terms of a concept they call intelligence density — the useful capability of a model per gigabyte of memory. It is a metric that deserves more attention. Raw capability determines what a model can do, but density determines where it can do it. Every leftward shift of the intelligence-per-gigabyte frontier expands the set of devices, products, and environments where advanced AI can operate.

Early computers filled rooms. Then they lived on desks. Then they moved to our laps, and finally to our pockets. Intelligence is making the same journey, and Bonsai 27B is its largest step yet. The model weights are available today under Apache 2.0, with native support for Apple devices via MLX and NVIDIA GPUs via CUDA.

What This Means for Developers

If you are building AI-powered applications, this release deserves your attention. The combination of 27B-class reasoning, tool-calling, and vision at a footprint that fits on consumer hardware means you can now build genuinely capable on-device agents — assistants that operate real tools, workflows that run unattended, and research pipelines that synthesize documents — all without a cloud API call.

PrismML is also offering a free, limited-time developer preview API for those who want to try the model before deploying locally. The full technical details of the compression methodology and benchmarking process are available in their whitepaper on GitHub.

The frontier of AI has been defined by increasingly large models running in increasingly large data centers. Bonsai 27B suggests that the next frontier might be defined by how much intelligence you can fit in your pocket.