PrismML just put a 27-billion-parameter AI model on a phone.
Not a tiny chatbot wearing a trench coat. Not a cloud model streaming answers to an app. The company says its new Bonsai 27B model runs locally on an iPhone 17 Pro Max at roughly 11 tokens per second while its language-model weights occupy just 3.9GB.
That headline sounds borderline ridiculous until you look at how PrismML pulled it off.
Bonsai 27B uses an end-to-end low-bit architecture that compresses the language weights to roughly 1.125 bits per weight, compared with the 16 bits commonly used by a full-precision model. PrismML says that cuts the model from about 54GB to 3.9GB while retaining 89.5% of the full-precision model’s average benchmark score.
There’s also a larger ternary version that uses three possible weight values instead of two. It takes up about 5.9GB and retains a reported 94.6% of the full-precision benchmark average.
That’s the trade in one sentence: the binary model prioritizes portability, while the ternary model spends a little more memory to preserve more capability.
What is Bonsai 27B?
Bonsai 27B is a multimodal model derived from Qwen3.6-27B. It accepts text and images and is designed for reasoning, coding, tool use, planning, and agent-style workflows.
PrismML released two main variants:
| Model | Weight format | Approx. footprint | Reported performance vs. FP16 | |—|—:|—:|—:| | Bonsai 27B | 1-bit binary | 3.9GB | 89.5% | | Ternary Bonsai 27B | 1.58-bit ternary | 5.9GB | 94.6% | | Qwen3.6-27B FP16 baseline | 16-bit | About 54GB | 100% |
The binary version constrains each language weight to either -1 or +1, with one shared FP16 scaling factor for every group of 128 weights. That scaling overhead is why the true average lands at 1.125 bits per weight instead of exactly one bit.
PrismML also says the low-bit treatment covers the embeddings, attention projections, MLP projections, and language-model head. In other words, this isn’t a normal model with a few aggressively compressed layers hiding behind a “1-bit” label.
The optional vision tower uses a separate 4-bit package of roughly 0.63GB and only needs to be loaded when the model is processing an image.
Yes, It Really Runs on an iPhone
According to PrismML’s testing, the MLX version of Bonsai 27B generates around 11 tokens per second on an iPhone 17 Pro Max using its custom MLX Swift runtime.
The company also reports the following generation speeds for the 1-bit model:
| Hardware | Reported generation speed | |—|—:| | iPhone 17 Pro Max | About 11 tokens/sec | | Apple M4 Pro | 26 tokens/sec | | Apple M5 Pro | 44.2 tokens/sec | | Apple M5 Max | 66.4 tokens/sec | | NVIDIA H100 | 104.8 tokens/sec |
Eleven tokens per second isn’t going to embarrass a giant cloud inference cluster. It doesn’t need to.
That speed is already fast enough for an interactive local assistant. More importantly, prompts, screenshots, documents, and camera input can stay on the device instead of being sent to a remote server.
That opens the door to a different category of mobile AI app: private assistants, offline coding tools, local document analysis, camera-based workflows, and personal agents that can keep running without paying an API bill for every token.
The 3.9GB Number Needs Context
The 3.9GB figure describes the language-model weights, not the total memory a device will use while the model is running.
Inference also needs memory for the KV cache, activations, context, and runtime buffers. PrismML’s own model card reports peak memory of about 5.2GB at a 4,000-token context window. At 100,000 tokens, it reports approximately 11.6GB without KV-cache compression or roughly 6.8GB with the 4-bit KV cache enabled.
The model supports a claimed context length of up to 262,000 tokens, but PrismML estimates that the full window requires around 9.4GB of peak memory with KV-cache compression.
So, yes, the weights fit in 3.9GB. No, that does not mean every 4GB phone can suddenly run a 27B model with a quarter-million-token context window.
The phone demo targets Apple’s highest-end iPhone hardware, and practical context length will still depend on the device’s available memory and the app’s memory allowance.
That doesn’t make the result less impressive. It just keeps the headline attached to reality.
How Much Capability Survived the Compression?
PrismML evaluated the binary model across 15 benchmarks covering reasoning, math, coding, instruction following, tool use, and vision.
The full-precision Qwen3.6-27B baseline averaged 85.07, while the 1-bit Bonsai model averaged 76.11. That works out to 89.5% of the baseline average.
The strongest retained areas were math and coding:
- Math category: 91.66 for Bonsai versus 95.33 for FP16
- Coding category: 81.88 versus 88.74
- Knowledge and reasoning: 73.39 versus 83.15
- Agentic and tool calling: 66.03 versus 80.00
- Vision: 59.57 versus 72.61
Those numbers reveal both sides of the release.
Bonsai 27B appears to preserve a surprising amount of reasoning and coding ability for a model this compressed. The losses become more visible in tool use, instruction following, and vision.
PrismML also explicitly warns that long-horizon agentic coding, including multi-file workflows that repeatedly run tests and repair code, is not yet a strong target for this release. A version tuned specifically for agentic coding is listed on the company’s roadmap.
That matters because “agentic model” can cover everything from calling a weather tool once to autonomously wrestling with a repository for six hours. Bonsai 27B may be useful for local agents, but it hasn’t earned the title of local coding workhorse yet.
Why This Release Matters
The AI industry has spent the last few years treating larger data centers as the default answer to every capability problem.
Bonsai 27B points in the opposite direction.
Instead of asking how much more infrastructure a model needs, PrismML is asking how much intelligence can fit inside the hardware people already own.
A useful 27B-class model running locally changes the economics and architecture of AI software:
- Privacy improves because sensitive inputs can remain on-device.
- Latency becomes predictable because the app doesn’t need a network round trip.
- Offline AI becomes practical for travel, field work, and unreliable connections.
- Developers can reduce inference costs by moving suitable workloads away from paid APIs.
- Personal agents can run continuously without transmitting every interaction to the cloud.
Cloud models will still dominate when maximum capability matters. The point isn’t that a 3.9GB local model replaces the frontier.
The point is that a surprisingly large slice of useful AI work may no longer require the frontier at all.
My Take
This is one of those releases where the marketing headline is aggressive, but the underlying engineering is interesting enough to survive a closer look.
A 27B multimodal model generating text at 11 tokens per second on an iPhone is a legitimate milestone, even with the quality trade-offs and memory caveats.
The biggest question now is independent testing.
The performance figures and benchmarks currently come from PrismML’s own release materials and model cards. I want to see developers reproduce the iPhone speeds, test realistic battery drain, measure thermal throttling, and compare output quality against conventional 4-bit and 2-bit builds in real workflows.
If those results hold up, Bonsai 27B isn’t just a clever compression demo. It’s evidence that serious local agents may arrive through better model architecture and inference kernels, not merely by waiting for phones to gain absurd amounts of RAM.
Bonsai 27B is available now under the Apache 2.0 license, with binary and ternary variants published through PrismML’s Hugging Face collection.
Sources
- PrismML’s Bonsai 27B announcement
- 1-bit Bonsai 27B model card
- Bonsai 27B MLX model for Apple Silicon
- Bonsai 27B model collection
