TensorFlow Lite — What It Is, Key Features and Applications

What Is TensorFlow Lite

TensorFlow Lite is a specialized runtime for running machine learning models on devices with limited resources. Unlike the full TensorFlow, TFLite does not support training or backpropagation — only inference. This allows the binary size of the library to be kept minimal: about 300 KB for the base interpreter on Android.

The TFLite architecture is built around the .tflite format, based on FlatBuffers. FlatBuffers provides direct access to data without a parsing stage, which is critical for mobile devices with limited memory. A model in .tflite format contains the computation graph, weights, and metadata in a single binary file.

According to Google I/O 2024, TFLite is used in Google Photos, Gboard, YouTube, and other Google applications with a combined audience of over 4 billion devices. The framework supports all major architectures: CNN, RNN, LSTM, Transformer, and custom operations via delegates.

TensorFlow Lite Architecture

The TFLite runtime consists of four components. TFLite Converter takes a model from TensorFlow (SavedModel, Keras, ConcreteFunction) and converts it to .tflite format with optional optimization. TFLite Interpreter loads the .tflite file and performs inference, managing tensors and memory.

Delegates are components that offload operation execution to specialized hardware. GPU Delegate uses OpenGL ES and Metal, NNAPI Delegate uses the Android Neural Networks API, Core ML Delegate uses Apple Core ML. XNNPACK Delegate optimizes execution on ARM CPU. Each delegate supports a specific set of operators.

The fourth component is the Task Library, which provides high-level APIs for typical tasks: image classification, object detection, segmentation, NLU. The Task Library works on top of the Interpreter and hides the details of tensor management.

Model Optimization and Quantization

TFLite supports three levels of quantization. Full integer INT8 quantization converts weights and activations from FP32 to 8-bit integers. Model size is reduced by 4x, and inference speed on devices with INT8 support increases up to 3x. FP16 quantization reduces size by half with minimal accuracy loss.

Dynamic quantization keeps weights in INT8 but performs computations in FP32. This mode is suitable for models sensitive to accuracy and gives up to 4x size reduction while maintaining quality. According to Google ML Performance Benchmarks, dynamic quantization is recommended for NLP models.

Comparison of TFLite Quantization Modes

Hardware Acceleration on Android and iOS

On Android, the main acceleration mechanism is the NNAPI Delegate, which offloads operation execution to NPU, DSP, or GPU via the Android Neural Networks API. NNAPI is available on devices with Android 8.1+ and supports INT8 and FP16 computations. The GPU Delegate for Android uses OpenGL ES 3.1+ and Vulkan.

On iOS, acceleration is provided through the Core ML Delegate, which translates TFLite operations into Core ML format and executes them on the Apple Neural Engine. According to Apple ML Benchmarking, using the Core ML Delegate speeds up inference on iPhone 15 Pro by up to 4x compared to CPU. The GPU Delegate for iOS uses Metal Performance Shaders.

XNNPACK Delegate is a cross-platform solution for ARM CPU, optimized for mobile processors. XNNPACK supports FP32, FP16, and INT8 operations and does not require special hardware. It is recommended as a baseline delegate for all devices.

Deploying a Model with TFLite

The deployment process includes three steps. First — converting the model to .tflite via TFLite Converter. Second — including the .tflite file in the application resources. For Android, the file is placed in assets; for iOS, in the app bundle via Xcode. Third — initializing the Interpreter and performing inference.

For dynamic model updates, Google recommends using Firebase ML Model Downloading or a custom cloud download mechanism. The updated .tflite file is saved to local storage and loaded into the Interpreter on the next launch.

When deploying, it is important to consider the .tflite file size. Google Play limits APK size to 200 MB. For models larger than 50 MB, it is recommended to use Android App Bundle or download the model separately via Play Asset Delivery.

Examples of Using TFLite in Code

Example of loading and running a model on Android with Java API. Use Interpreter.create() to load .tflite from assets and run() for inference. Input and output data are passed as multidimensional arrays or ByteBuffer:

import org.tensorflow.lite.Interpreter;
import java.nio.MappedByteBuffer;
import java.io.FileInputStream;
import java.nio.channels.FileChannel;

MappedByteBuffer model = new FileInputStream(
    getAssets().openFd("model.tflite")
).getChannel().map(
    FileChannel.MapMode.READ_ONLY, 0, fc.size()
);

Interpreter interpreter = new Interpreter.create(model);
float[][] input = new float[1][224 * 224 * 3];
float[][] output = new float[1][1000];
interpreter.run(input, output);
interpreter.close();

Example of using GPU Delegate on Android for hardware acceleration. Add the dependency com.android.support:tensorflow-lite-gpu and specify the delegate when creating the Interpreter:

import org.tensorflow.lite.Interpreter;
import org.tensorflow.lite.gpu.GpuDelegate;

GpuDelegate gpuDelegate = new GpuDelegate.create();
Interpreter.Options options = new Interpreter.Options().addDelegate(gpuDelegate);
Interpreter interpreter = new Interpreter.create(model, options);

// Run inference on the input data
interpreter.run(input, output);

// Release delegate resources after inference
gpuDelegate.close();
interpreter.close();