ML on Device in Mobile Development: What It Is, Frameworks, and How It Works

Q: How is Core ML different from TensorFlow Lite?

Core ML is Apple's framework for iOS and macOS with Neural Engine acceleration. TensorFlow Lite is Google's cross-platform solution for Android, iOS, and Linux. Core ML offers better performance on Apple devices, TFLite offers versatility.

Q: What tasks does ML Kit solve?

ML Kit solves typical computer vision and NLP tasks: text, face, barcode, object, and pose recognition. All APIs work on the device without internet and don't require creating your own model.

Q: Is internet required for on-device ML?

No, on-device ML works completely locally. Models are loaded onto the device and executed without an internet connection. ML Kit also offers cloud-based API versions with higher accuracy, but on-device mode is available offline.

Q: Which framework should I choose for ML in mobile development?

For iOS-only, choose Core ML — it uses the Neural Engine and is tightly integrated with the Apple ecosystem. For cross-platform projects, choose TensorFlow Lite. For typical tasks without a custom model, choose ML Kit with ready-made APIs.

Author: IT Sectr Published: 2026-07-29 Reading time: 11 min

ML on device (on-device ML) — running machine learning models directly on a mobile device without sending data to a server. According to a Google AI, 2025 report, over 70% of users prefer apps with on-device ML due to privacy and lack of network latency. Core ML from Apple, TensorFlow Lite from Google, and ML Kit allow solving Vision, NLP, face and object recognition tasks entirely on the device — without internet and with minimal power consumption.

Key Takeaways

ML on device — running machine learning models directly on a mobile device without server-side data processing.
Core ML — Apple's framework with hardware acceleration on Neural Engine for iOS and macOS.
TensorFlow Lite — Google's cross-platform solution with quantization and GPU, NNAPI, and XNNPACK delegates.
ML Kit — SDK with ready-made APIs for text, face, barcode, and object recognition without building a model.
Vision and NaturalLanguage — built-in ML tools on iOS for image and text analysis.

What Is ML on Device and Why Is It Needed in Mobile Apps

ML on device (on-device ML) is an approach where machine learning models run directly on a mobile device without sending data to a remote server. Privacy is the key advantage: user data never leaves the device. According to Google research (2024), 63% of users refuse ML features that require sending data to a server. On-device ML eliminates network latency, works offline, and reduces power consumption through hardware acceleration.

Advantages of On-Device ML Over Cloud Solutions

On-device ML processes data in milliseconds instead of seconds — critically important for real-time face and object recognition. No internet connection required is another advantage: ML features are available in airplane mode and regions with unstable connectivity. Core ML uses the Neural Engine in Apple A12+ chips, delivering 11 trillion operations per second at under 1W power consumption. For mobile apps, on-device ML has become the de facto standard in Vision, NLP, and object recognition tasks.

Key On-Device ML Use Cases in Mobile Apps

ML in mobile apps is used for face authentication (Face ID), AR filters in Snapchat and Instagram, fitness trackers with Pose Detection, OCR scanning in Adobe Scan, and predictive text input in keyboards. Custom models for specific tasks are created via Core ML (iOS) or TensorFlow Lite (Android). ML Kit is suitable for typical scenarios — text, face, and barcode recognition without needing to train a model.

Core ML: Apple's Framework for On-Device ML on iOS

Core ML is Apple's framework for running ML models on iPhone, iPad, Mac, and Apple Watch. It automatically selects the optimal hardware acceleration: Neural Engine for neural networks on devices with A12+ (11 TOPS), GPU for image processing tasks, and CPU for sequential computations. Core ML supports .mlmodel (original) and .mlmodelc (compiled) formats. Model conversion from PyTorch and TensorFlow is done via coremltools — a Python library that preserves topology and weights.

Create ML and coremltools

Create ML is Apple's app for training simple models without writing code. For production, use coremltools — a conversion utility from PyTorch, TensorFlow, and scikit-learn. Coremltools supports quantization from float32 to float16 and int8, color space palettization, and removing redundant operations to reduce model size.

python

import coremltools as ct

model = ct.convert(
    "resnet50.mlmodel",
    source="pytorch",
    convert_to="mlprogram"
)
model.save("Resnet50.mlpackage")

Neural Engine and Core ML Hardware Acceleration

The Neural Engine is a specialized neural processor in Apple A12 chips and later. 16 cores perform up to 11 trillion operations per second at under 1W power consumption. Core ML automatically routes neural network computations to the Neural Engine, and GPU-compatible ones to Metal GPU. The developer doesn't need to choose the device — Core ML does this through Performance Report, analyzing the model and available hardware.

TensorFlow Lite: Cross-Platform On-Device ML from Google

TensorFlow Lite (TFLite) is Google's cross-platform solution for running ML models on Android, iOS, and Linux. Models use the .tflite format and are created via TFLiteConverter from the TensorFlow ecosystem. TFLite supports quantization from float32 to int8, reducing model size by 4x while maintaining 97–99% accuracy on classification tasks. Google Play Services for TFLite automatically updates the runtime without requiring an app update.

TFLiteConverter and Model Quantization

TFLiteConverter is the primary tool for converting TensorFlow models to .tflite format. Post-training int8 quantization compresses a model from 300 MB to 75 MB without access to the full dataset. Quantization-aware training (QAT) yields minimal accuracy loss — less than 1% on ImageNet. TFLiteConverter also supports graph pruning and removing operations not supported by the TFLite runtime.

python

import tensorflow as tf

converter = tf.lite.TFLiteConverter.from_saved_model(
    "saved_model_dir"
)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()

TFLite Delegate: GPU, NNAPI, and XNNPACK

Without a delegate, TFLite runs the model on CPU 3–5 times slower than with hardware acceleration. GPU Delegate uses OpenCL and OpenGL ES 3.1 on Android, Core ML Delegate uses Neural Engine on iOS. NNAPI Delegate utilizes NPU and DSP on Android 8.1+. XNNPACK Delegate provides cross-platform acceleration on ARM CPU with optimizations for mobile processors. To select a delegate, use TfLiteGpuDelegate.create() with a null check on the result.

ML Kit: Ready-Made Solutions for ML in Mobile Development

ML Kit is Google's SDK with ready-made APIs for on-device ML. In mobile apps, ML Kit is used for text recognition (50+ languages, including handwriting), Face Detection (468 facial key points), Barcode Scanning (QR, EAN-13, Code 128, PDF417, Data Matrix), and Object Detection. All APIs work entirely on the device without internet. ML Kit is available on iOS and Android with a unified API.

Face Detection and Text Recognition in ML Kit

Face Detection returns the coordinates of the face contour, eyebrows, eyes, nose, and lips, as well as head tilt angle and eye state. AR masks and camera filters are the most popular use case. Text Recognition extracts text from photos with up to 99% accuracy on printed characters. The API supports real-time recognition via CameraX.

kotlin

val recognizer = TextRecognition.getClient()
val image = InputImage.fromBitmap(bitmap)

recognizer.process(image)
    .addOnSuccessListener { result ->
        result.textBlocks.forEach { block ->
            println(block.text)
        }
    }

Barcode Scanning and Object Detection in ML Kit

Barcode Scanning recognizes barcodes in real-time camera video stream. Object Detection identifies objects in an image with bounding boxes and class labels (person, car, animal). Pose Detection determines 33 key body points for fitness apps and motion analysis. Image Labeling returns up to 10 semantic image labels — "nature", "city", "food".

Vision and NaturalLanguage: Built-In ML Tools on iOS and Android

Apple provides built-in ML solutions through Vision and NaturalLanguage without installing third-party SDKs. Vision (VNRequest) performs face, text, barcode, and contour detection on the device. NaturalLanguage (NLTokenizer) provides tokenization, lemmatization, language identification, and sentiment analysis. On Android, equivalents are implemented via ML Kit (recognition) and SpeechRecognizer from android.speech (speech). Built-in tools don't require model downloads — they are pre-installed in the system.

Vision Framework: VNRequest and VNCoreMLRequest

Vision includes VNDetectFaceRectanglesRequest (face detection), VNRecognizeTextRequest (text recognition), VNDetectBarcodesRequest (barcodes), and VNDetectHumanBodyPoseRequest (skeleton). VNCoreMLRequest integrates a custom Core ML model into the Vision pipeline — for example, emotion classification on detected faces. All requests run on the Neural Engine with minimal latency.

swift

let request = VNRecognizeTextRequest { request, error in
    guard let observations = request.results
        as? [VNRecognizedTextObservation] else { return }
    for observation in observations {
        let topCandidate = observation
            .topCandidates(1).first
        print(topCandidate?.string as? String ?? "")
    }
}

let handler = VNImageRequestHandler(
    cgImage: image, options: [:]
)
try? handler.perform([request])

NaturalLanguage and SFSpeechRecognizer on iOS

NaturalLanguage provides NLTokenizer for splitting text into tokens, NLLanguageRecognizer for language identification (72 languages), and NLSentimentAnalyzer for text sentiment analysis. SFSpeechRecognizer is a speech recognition API supporting 50+ languages on device (iOS 13+). Requires SFSpeechRecognizerAuthorizationStatus permission before use. Results come asynchronously via SFSpeechRecognitionResultHandler.

Frequently Asked Questions

What is ML on device?

ML on device (on-device ML) is an approach where machine learning models run directly on a mobile device without sending data to a server. Core ML, TensorFlow Lite, and ML Kit are the main frameworks for on-device ML.

How is Core ML different from TensorFlow Lite?

Core ML is Apple's framework for iOS and macOS with Neural Engine acceleration. TensorFlow Lite is Google's cross-platform solution for Android, iOS, and Linux. Core ML offers better performance on Apple devices, TFLite offers versatility.

What tasks does ML Kit solve?

ML Kit solves typical computer vision and NLP tasks: text, face, barcode, object, and pose recognition. All APIs work on the device without internet and don't require creating your own model.

Is internet required for on-device ML?

No, on-device ML works completely locally. Models are loaded onto the device and executed without an internet connection. ML Kit also offers cloud-based API versions with higher accuracy, but on-device mode is available offline.

Which framework should I choose for ML in mobile development?

For iOS-only, choose Core ML — it uses the Neural Engine and is tightly integrated with the Apple ecosystem. For cross-platform projects, choose TensorFlow Lite. For typical tasks without a custom model, choose ML Kit with ready-made APIs.

Summary

ML on device — running models directly on a mobile device without sending data to a server, with zero network latency.
Core ML — Apple's framework with hardware acceleration on Neural Engine for iOS and macOS.
TensorFlow Lite — Google's cross-platform solution with int8 quantization and GPU, NNAPI, XNNPACK delegates.
ML Kit — SDK with ready-made APIs for text, face, and barcode recognition without building a model.
Vision and NaturalLanguage — built-in ML tools on iOS without installing third-party libraries.
Quantization reduces model size by 4x with minimal accuracy loss — the standard for mobile apps.
On-device ML is a mandatory standard for apps with sensitive data and offline use cases.

We will develop a mobile application turnkey

IT Sectr creates iOS and Android applications for startups and businesses since 2017. We will advise you and propose the best solution.

Discuss the project