How to Add AI Features to a Mobile App: Cloud vs On-Device
Practice4 minInnovialabs editorial team
In short
Cloud models are smarter and easier to launch but need a network and a backend proxy for keys; on-device models are fast, offline and private but limited in capability and supported devices.
The short answer
There are two ways to add AI to a mobile app:
- Cloud: the app sends a request to a server running a large model (via an LLM provider's API). Good for complex tasks: open-ended chat, analyzing long texts, content generation.
- On-device: the model runs on the phone itself. Good for fast, narrow tasks: text recognition in photos, barcode scanning, image classification, short suggestions and summaries.
The best option is often a hybrid: simple and latency-sensitive work happens on the device, complex work in the cloud.
Comparison on the key factors
| Factor | Cloud LLM APIs | On-device model |
|---|---|---|
| Quality on complex tasks | high | limited, models are smaller |
| Latency | depends on network and load | low, no network round trip |
| Offline | does not work | works |
| Privacy | data goes to a server | data stays on the phone |
| App size | barely changes | grows if the model is bundled |
| Cost | pay per request | no per-request fee, but battery use |
| Device support | anything with internet | depends on the model and phone hardware |
| Model updates | on the server, no release | with the app or a model download |
On-device tools
- Core ML (iOS) is Apple's framework for running models on the device. Models from popular ML frameworks can be converted to the Core ML format. For common tasks there are ready-made Vision and Natural Language frameworks.
- ML Kit (Android and iOS) offers ready-made Google APIs: text, barcode and face recognition, translation, language identification. Some models are bundled with the app, others are downloaded on first use.
- Gemini Nano is Google's compact on-device language model. It is available only on some Android devices, so you need a fallback for the rest.
- TensorFlow Lite / LiteRT and similar runtimes are for when you have your own trained model.
Before choosing, check which of your users' devices the model will actually run on. For an audience with many budget Android phones, this is critical.
How to choose for a specific feature
Ask four questions:
- Does it need to work offline? A document scanner in a warehouse or on the road leans on-device.
- How sensitive is the data? Medical photos, documents and private messages are better processed locally.
- How complex is the task? An open answer about a catalog or a long contract belongs in the cloud.
- How important is instant response? Typing suggestions or real-time camera processing belong on the device.
A hybrid example: OCR a receipt on the phone with ML Kit or Vision, then have a cloud model parse and categorize the items from the recognized text, without sending the photo.
Protecting API keys
The key rule: the LLM provider's key is never stored in the app. Apps can be decompiled and the key extracted, after which anyone can make requests at your expense.
The correct setup:
App -> your backend (user auth, limits) -> LLM API
- The key lives only on the server.
- The server authenticates the user and limits requests per person.
- Use app attestation (App Attest on iOS, Play Integrity on Android) to reject requests that do not come from your app.
- Log usage and set alerts for unusual spikes.
- Keep the prompt and business logic on the server so you can change them without a store release.
Common mistakes
- An API key in the app code or build config.
- Choosing an on-device model without testing on your audience's real devices.
- Bundling a heavy model into the build when it could be downloaded after install.
- No fallback when the cloud is unreachable or the device does not support the local model.
- Sending whole photos and documents to the cloud when recognized text is enough.