Media and Streaming in Mobile Development: What It Is, Protocols, and How It Works

HLS and DASH Protocols

HTTP Live Streaming (HLS) is a protocol for streaming in mobile apps developed by Apple. It splits video into small segments (usually 6–10 seconds) and transmits them over regular HTTP. The player downloads a playlist (.m3u8) and sequentially loads segments, adapting quality to connection speed. HLS provides reliable streaming in mobile apps on all Apple devices and is the de facto standard for iOS.

HLS vs DASH

DASH (Dynamic Adaptive Streaming over HTTP) is an open MPEG standard for delivering media to mobile apps. Unlike HLS, DASH uses MP4 format for segments and an XML manifest (MPD) instead of .m3u8. DASH provides more control over codecs and supports more formats. Choosing between HLS and DASH is a key decision when developing streaming for mobile devices. The protocol comparison is shown in the table below.

In practice, iOS apps choose HLS — AVPlayer works with it natively. For Android, DASH is preferred thanks to ExoPlayer. Both protocols are suitable for media streaming on mobile devices. At IT Sectr, we use both technologies and recommend choosing the protocol based on the target platform.

Adaptive Bitrate and Codecs

Adaptive Bitrate (ABR) is a mechanism where the player automatically selects video quality based on connection speed. The stream is encoded at multiple bitrates (360p, 720p, 1080p, 4K). The player switches between them without pauses. Codecs: H.264 (AVC) — baseline, H.265 (HEVC) — 50% more efficient, VP9 and AV1 — open alternatives for Android.

For hardware decoding in mobile apps, it is important to choose the right codec. Hardware Decoding uses the device's GPU/video processor and consumes less battery. Software Decoding runs on the CPU — used when hardware decoding is unavailable. IT Sectr recommends always checking codec support via MediaCodec (Android) or AVAssetTrack (iOS) for optimal media playback on mobile devices.

Players: ExoPlayer and AVPlayer

ExoPlayer is a media player from Google for Android, optimized for media playback in mobile apps. Unlike MediaPlayer, ExoPlayer is not tied to hardware decoding and supports custom schemes through extension modules. It works with HLS, DASH, SmoothStreaming, and RTSP. ExoPlayer is the main player for YouTube, Google TV, and Android TV.

ExoPlayer vs AVPlayerViewController

AVPlayer and AVPlayerViewController are Apple players for iOS, macOS, tvOS. AVPlayer is a low-level API, AVPlayerViewController is a ready-made UI wrapper with PiP and AirPlay. Key differences: ExoPlayer gives full control over buffering and rendering, AVPlayerViewController offers quick integration with minimal code. The choice of player directly affects the quality of streaming in mobile apps.

// ExoPlayer — setting up a basic player on Android
val player = ExoPlayer.Builder(this)
    .setMediaSource(ProgressiveMediaSource.Factory(DefaultHttpDataSource.Factory()))
    .build()
val mediaItem = MediaItem.fromUri("https://example.com/stream.m3u8")
player.setMediaItem(mediaItem)
player.prepare()
player.playWhenReady = true

The code shows minimal ExoPlayer initialization: creating an instance, setting the media item, and starting playback. ExoPlayer automatically handles the Activity lifecycle through player.release() in onStop. For Android TV or PiP, additional lifecycle handling via PlayerView is required — this is especially important when setting up streaming for mobile apps.

Transcoding and Video Processing

Transcoding is converting video from one format to another. It is used to optimize media streams for different mobile devices (e.g., H.264 → H.265). On mobile devices, transcoding is possible via MediaCodec (Android) or AVAssetExportSession (iOS). Transcoding is resource-intensive and drains the battery — do it in the background using WorkManager or BGTaskScheduler.

DRM: Widevine, FairPlay, and PlayReady

Digital Rights Management (DRM) are systems for protecting media content from unauthorized copying. Three main technologies are used for streaming premium media in mobile apps: Widevine (Google), FairPlay (Apple), and PlayReady (Microsoft). Without DRM, premium services (Netflix, Spotify, Disney+) cannot distribute content.

DRM System Comparison

Widevine is the DRM protection standard for Android, available in three levels: L3 (software encryption), L2, L1 (hardware encryption, Trusted Execution Environment only). FairPlay is Apple's exclusive system, using SVP (Secure Video Pipeline) at the chip level. PlayReady is Microsoft's system, prevalent on Xbox and Windows devices.

Multi-DRM is an approach where you use a single platform to manage all DRM keys. For example: EZDRM or Verimatrix. They issue Widevine, FairPlay, and PlayReady licenses through a unified API. Multi-DRM is especially relevant for premium media on mobile devices. At IT Sectr, we implement multi-DRM through aggregators — this saves time and money compared to integrating each DRM separately.

Audio Playback: Audio Focus and Audio Session

Audio Focus (Android) is an audio stream management system for media playback in mobile apps. When your app requests Audio Focus, other apps (music, podcasts) pause or lower their volume (ducking). Without Audio Focus, two apps would play simultaneously, creating noise.

Audio Focus on Android

Android divides audio streams into types: MUSIC, VOICE_CALL, ALARM, NOTIFICATION. For a media player, you need to request AudioManager.AUDIOFOCUS_GAIN. When focus is lost (incoming call), you must pause media playback and resume after focus returns. Implementation via AudioManager.OnAudioFocusChangeListener.

Audio Session on iOS

Audio Session (iOS) is the equivalent of Audio Focus for media apps on Apple mobile devices. It is configured via AVAudioSession.sharedInstance(). Categories: .playback (required for video), .soloAmbient (default, silent when locked), .playAndRecord (for recording). For streaming, use .playback with the .allowAirPlay option.

PiP and AirPlay

Picture-in-Picture (PiP) is a mode where media content continues playing in a small floating window when the app is minimized or the user navigates to another screen. PiP has become standard for media apps on mobile devices — iOS (since iPadOS 13) and Android (since Android 8.0).

PiP on Android

On Android, PiP is enabled via Activity.isInPictureInPictureMode() and PictureInPictureParams.Builder. The app must be in multi-window mode. MediaSession is required to control media playback from the PiP window. In the manifest, specify android:supportsPictureInPicture="true".

AirPlay and Chromecast

AirPlay is Apple's protocol for streaming media to Apple TV, HomePod, and compatible Smart TVs. On iOS, it is implemented via AVRoutePickerView or MPNowPlayingInfoCenter. On Android, the equivalent is Google Cast SDK (Chromecast). Both technologies require configuration in the binary file and player support. High-quality streaming in mobile apps is impossible without one of them.