On-device AI: what it means when your phone runs AI locally

For years, "AI" on your phone mostly meant sending your data to a company's servers, letting a big model process it there, and getting a result back. On-device AI flips that: the model runs on the chip inside your phone or laptop.

Why companies are doing it

• Speed. No round trip to a server, so features like live transcription or photo editing respond instantly.
• Privacy. Your voice recordings, messages and photos can be processed without leaving the device.
• Works offline. Translation or summarisation can work on a plane or with a weak signal.
• Cost. Running AI on millions of servers is expensive; your phone's chip is already paid for.

What makes it possible

Modern phone and laptop chips include a dedicated NPU (neural processing unit) designed for the maths AI models use. Manufacturers also shrink models through techniques like quantisation, storing numbers with less precision so a model fits in a phone's memory.

The trade-offs

On-device models are much smaller than the largest cloud models, so they are better at focused tasks — summarising a notification, cleaning up a photo, suggesting a reply — than at open-ended reasoning. Many products therefore use a hybrid approach: simple requests stay on the device, harder ones go to the cloud, sometimes with a prompt asking your permission first.

Running AI locally also uses battery and memory. That is one reason newer AI features are often limited to phones with more RAM.

What to check when buying

If on-device AI matters to you, look at how much RAM the phone has, whether the maker explains which features run locally, and how long the device will receive software updates — AI features are increasingly delivered through updates.