Video Encoding in Mobile Applications: What It Is, Codecs and Parameters

What Is Video Encoding?

Video encoding is the process of converting an uncompressed video signal into a digital format using compression algorithms. These algorithms, implemented in video codecs, remove redundant information both in the spatial domain (within a single frame) and the temporal domain (between successive frames).

Uncompressed video is a sequence of frames, each stored as an array of pixels in RGB or YUV format. Without compression, one second of 1080p video at 30 FPS requires about 1.5 GB of memory. The H.264 codec compresses this stream to 3–5 Mbps, which is 2500 times smaller than the original size.

Video Compression Principles

Video codecs use two types of compression: intra-frame and inter-frame. Intra-frame compression is applied to each frame independently, using algorithms similar to JPEG: discrete cosine transform (DCT), quantization, and entropy coding.

Inter-frame compression analyzes the difference between successive frames and encodes only the changes. If the background is static, the codec transmits a full frame only once (I-frame), and then only the differences (P- and B-frames), which significantly reduces data volume for low-motion scenes.

Video Codecs: H.264, H.265, VP9, AV1

Video codec selection determines the quality-to-size ratio, device compatibility, and licensing fees. Four main codecs dominate the mobile application market: H.264 (AVC), H.265 (HEVC), VP9, and AV1.

H.264 — The Universal Standard

H.264 (AVC — Advanced Video Coding) remains the most compatible video codec: it is supported by all mobile devices, browsers, and Smart TVs since 2005. Hardware-accelerated H.264 decoding is available even on budget devices, making it ideal for a universal stream.

The main drawback of H.264 is relatively low compression efficiency compared to modern codecs. At the same bitrate, H.265 and AV1 provide noticeably higher quality, which is especially important for HDR and 4K content.

H.265 — Efficiency for 4K

H.265 (HEVC — High Efficiency Video Coding) provides 40–50% better compression than H.264 at the same visual quality. This is achieved through more flexible frame partitioning into blocks (CTU up to 64×64 pixels instead of 16×16) and improved motion prediction algorithms.

HEVC is supported on all Apple devices (starting from iPhone 6), on Android (starting from version 5.0 with hardware decoding), and on most Smart TVs. HEVC licensing fees are higher than H.264 and are managed by two patent pools, which complicates legal usage.

AV1 — The Open Standard of the Future

AV1 is a video codec developed by the Alliance for Open Media (AOM), a consortium led by Google, Netflix, Amazon, Intel, and Mozilla. AV1 provides 50–60% better compression than H.264 and is completely royalty-free, making it attractive for streaming services.

The main drawback of AV1 is high computational complexity of encoding. AV1 encoding can be 5–10 times slower than H.264 on the same hardware, although hardware decoding is already appearing on 2024–2025 devices (Snapdragon 8 Gen 3, Apple A17 Pro).

Audio Codecs: AAC, MP3, Opus

Audio codecs in mobile applications are no less important than video codecs. Streaming video requires audio with minimal latency, high quality, and compatibility with MP4 and WebM containers. Three main audio codecs cover 99% of mobile streaming applications.

AAC — The Apple and YouTube Standard

AAC (Advanced Audio Coding) is the successor to MP3, developed by MPEG in 1997 and became the standard audio codec for iTunes, YouTube, and Android. AAC provides the same quality as MP3 at 30% lower bitrate and supports multi-channel audio up to 7.1.

For mobile applications, AAC-LC at 128–192 kbps for stereo and AAC-HE for low bitrates of 32–64 kbps are recommended. Hardware AAC decoding is available on all modern devices, minimizing power consumption.

Opus — The Modern Audio Codec

Opus is an open audio codec from IETF (RFC 6716), providing the best quality among all audio codecs at bitrates from 6 to 510 kbps. Opus combines SILK (an algorithm for speech) and CELT (for music) and automatically switches modes depending on the content.

Opus is used in WebRTC for video calls, in Spotify and Discord applications, and is supported in HLS starting from iOS 11 and Android 5.0. Opus low latency (5–26 ms vs 40–100 ms for AAC) makes it the preferred choice for live broadcasts and video conferencing.

Encoding Parameters: Bitrate, Resolution, FPS

Encoding parameters determine the final file size, video quality, and device compatibility. A mobile application developer must choose optimal settings for each type of content: movies, sports broadcasts, user-generated content.

Bitrate and Its Types

Bitrate is the number of bits used to encode one second of video. There are CBR (Constant Bitrate) for live broadcasts, VBR (Variable Bitrate) with target quality for archival storage, and ABR (Average Bitrate) for streaming.

CRF — Quality Control

CRF (Constant Rate Factor) is an encoding parameter in x264 and x265 that sets target quality instead of a fixed bitrate. The CRF scale ranges from 0 (lossless) to 51 (worst quality), with 18–28 being the working range. For mobile applications, CRF 23 for x264 and CRF 25 for x265 are recommended.

CRF 18–20 provides high quality for presentation content and offline viewing. CRF 24–28 is suitable for streaming and user-generated content where file size matters more than absolute quality.

Codec Profiles and Levels

The H.264 standard defines profiles (Baseline, Main, High) and levels (Level 1–5.2) that limit the maximum video parameters. Baseline Profile is compatible with all devices but does not support B-frames. High Profile provides maximum quality and is used for 1080p content.

For 4K streaming, Level 5.1 and above is required, which is supported by most mobile devices starting from 2018. The codec level determines the maximum resolution, bitrate, and buffer size, ensuring correct decoding on all certified devices.

Encoding for Streaming

Encoding for streaming differs from archival encoding in that it requires creating multiple quality variants (ABR ladder) and optimizing for network transmission. Content providers use real-time transcoding or pre-encoding for VOD.

A typical ABR ladder for mobile applications includes 4–6 variants: from 144p (200 kbps) for weak networks to 1080p (8 Mbps) for Wi-Fi. Recommended resolutions: 144p, 240p, 360p, 480p, 720p, 1080p. For 4K content, 1440p and 2160p are added.

Transcoding on a Mobile Device

val mediaCodec = MediaCodec.createEncoderByType(
    MediaFormat.MIMETYPE_VIDEO_AVC
)
val format = MediaFormat.createVideoFormat(
    MediaFormat.MIMETYPE_VIDEO_AVC,
    1280, 720
).apply {
    setInteger(MediaFormat.KEY_BIT_RATE, 3_000_000)
    setInteger(MediaFormat.KEY_FRAME_RATE, 30)
    setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 2)
    setInteger(MediaFormat.KEY_PROFILE,
        MediaCodecInfo.CodecProfileLevel.AVCProfileHigh)
    setInteger(MediaFormat.KEY_LEVEL,
        MediaCodecInfo.CodecProfileLevel.AVCLevel31)
}
mediaCodec.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE)

Encoding for Live Broadcasts

For live broadcasts, low latency is especially important. Encoding settings should minimize codec input/output delay: a GOP size of 1–2 seconds (every 30–60 frames) is recommended, B-frames should be disabled or minimized, and Baseline Profile is used for H.264.

Modern low-latency protocols — LL-HLS and CMAF-Low Latency — require fragmented encoding with 1–2 second segments and support for partial downloads (Partial Segments). The codec must support moof-box in each segment for fragmented MP4.

Optimizing the Encoding Pipeline

• Hardware encoding via MediaCodec (Android) or VideoToolbox (iOS) is mandatory for real-time performance.
• Multi-threaded encoding on the server using libx264/libx265 in ultrafast or veryfast mode.
• Encoding presets: placebo — worst for VOD (maximum compression), ultrafast — for live broadcasts.
• Two-pass VBR — the preferred mode for VOD, providing predictable size and quality.