Build intelligent Android apps: On-device inference

Posted by Caren Chang, Developer Relations Engineer, Android Developer Relations Welcome back to the blog post series " Build intelligent Android apps " where we take a basic Android app and transform it into a personalized, intelligent, and agentic experience. In our previous post we introduced Jetpacker , the demo app we'll use throughout this series. In this blog post, we will share how you can use Gemini Nano through ML Kit’s Prompt API to build intelligent on-device features.

Building intelligent on-device features refers to the ability to process prompts and data directly on a device without sending data to a server. This offers a few advantages: User data can be processed locally on the device, preserving user privacy Functionality of the model is reliable even with spotty or no internet connection No additional cloud inference cost , since everything runs on the user’s hardware With the benefits of on-device in mind, we identified three features to add in Jetpacker that can improve the user experience: summarizing trip itineraries, managing expenses, and capturing voice notes. On-device features in Jetpacker: Summarizing trip itineraries, managing expenses, and voice notes High quality tailored summarization of short texts The itinerary screen gives users a quick overview of all activities for a given trip.

Since this screen contains a lot of information, it can quickly become overwhelming. To help users prepare without feeling overwhelmed, we can add a ‘ Get ready for your trip ’ section at the top. The romantic Paris trip is summarized as a classic Parisian adventure blending art, sights, and delicious food.

A tip and some useful phrases are also added. By inputting a trip itinerary and asking an LLM to summarize it, we can generate a quick summary of the trip along with packing tips and useful local phrases. This is a great use case for an on-device model for several reasons: Performance and quality : Both the input and output text are relatively short.

With that, we can expect the performance and quality of an on-device solution to be on par with more powerful cloud models. Scalability : Shifting inference on-device allows us to scale this feature from a few users to millions without worrying about managing increasing cloud inference costs. Low latency and reliability : On-device inference guarantees low latency, providing a reliable experience even when users are offline.

To build with on-device, we use Gemini Nano , Google’s most efficient model optimized for mobile devices. Gemini Nano was first introduced a few years ago, and is now running on over 140 million devices. The latest version of the model, Gemini Nano 4, is built on the architecture foundation of the recently released Gemma 4 model , and is further optimized for maximum battery and performance efficiency.