Transcoding in Mobile Development — What It Is, How It Works, and Use Cases

What Is Transcoding?

Transcoding is the process of converting a media file from one compression format to another by fully decoding the source stream into an intermediate uncompressed PCM format and then encoding it with new parameters. If the source file uses H.264 codec with a bitrate of 10 Mbps and the output needs H.265 with a bitrate of 3 Mbps — that is transcoding.

Transcoding differs from simple repackaging (transmuxing), where only the container changes (e.g., MP4 to MKV) while the compressed bitstream remains unchanged. Transcoding involves computationally expensive operations: decoding each frame, applying filters (scaling, color correction, cropping), and re-encoding with new parameters. This makes transcoding one of the most resource-intensive operations when working with media.

Transcoding is used in a wide range of tasks: adapting video to bandwidth constraints, converting to a format with hardware decoding support on the target device, creating multiple versions for HLS/DASH streaming, extracting audio tracks into a separate file. OTT services (Netflix, YouTube, Twitch) transcode every uploaded file into dozens of variants with different bitrates, resolutions, and codecs to provide adaptive streaming to millions of users.

How Does Transcoding Work?

The transcoding process consists of three main stages: decoding, processing, and encoding. Each stage can be performed on either CPU or GPU/hardware blocks depending on availability and required performance.

Transcoding Stages

The first stage is decoding the source stream. The source file is read from the container (MP4, MOV, MKV), after which compressed video packets are sent to the decoder. Decoding can be hardware-based (if the codec is supported) or software-based via FFmpeg. The output of decoding is uncompressed frames in YUV420 or BGRA format — this is the stage where transcoding differs from simple remultiplexing.

The second stage is filtering and processing. Decoded frames pass through a chain of filters: scaling to the target resolution, changing frame rate, color correction, overlaying text or graphics. FFmpeg filter chain is built as a graph where each filter is a separate processing module. For example, filter scale=1280:720 changes resolution, fps=30 changes frame rate, and yadif performs deinterlacing. All operations are performed on uncompressed frames, making the second stage the most resource-intensive.

The third stage is encoding into the target format. Processed frames are fed into the encoder, which compresses them according to the target codec algorithm. The encoder can be hardware-based (VideoToolbox on iOS, MediaCodec on Android) or software-based (libx264, libx265). Encoding parameters: CRF (Constant Rate Factor) for constant quality, bitrate for CBR/VBR, profile and level for compatibility with target devices.

// FFmpeg transcoding pipeline diagram
AVFormatContext *inputCtx, *outputCtx;
AVCodecContext *decoderCtx, *encoderCtx;
AVFrame *frame = av_frame_alloc();
AVPacket packet;

while (av_read_frame(inputCtx, &packet) >= 0) {
    // 1. Decode
    avcodec_send_packet(decoderCtx, &packet);
    avcodec_receive_frame(decoderCtx, frame);
    
    // 2. Filter (scale + fps change)
    sws_scale(swsCtx, frame->data, frame->linesize,
              0, frame->height, scaledFrame->data,
              scaledFrame->linesize);
    
    // 3. Encode
    avcodec_send_frame(encoderCtx, scaledFrame);
    avcodec_receive_packet(encoderCtx, &outPacket);
    av_interleaved_write_frame(outputCtx, &outPacket);
}

The above pipeline demonstrates the classic transcoding cycle. The av_read_frame function reads compressed packets from the input file, avcodec_send_packet decodes them into frames, sws_scale performs scaling, and avcodec_send_frame encodes the processed frame into the output format. This three-stage cycle repeats for each frame or group of frames (GOP), depending on encoder settings.

Transcoding vs Transmuxing

The difference between transcoding and transmuxing is one of the most common points of confusion in media engineering. Understanding this difference is critical for choosing the right media processing strategy.

Transmuxing is repackaging a compressed stream into a different container without decoding and re-encoding. If video is already compressed with H.265 codec in an MP4 container and needs to be placed into MOV or MKV container — transmuxing simply copies bitstream packets from one container to another. Quality is not affected, processing time is minimal since no frame decoding is required. FFmpeg performs transmuxing with the -codec copy flag.

Transcoding, on the other hand, fully decodes and recompresses the media stream. Every time a video goes through transcoding, generation loss may occur — a slight quality degradation due to repeated lossy compression. Even at the same bitrate, the third generation of transcoding is usually worse than the first. This is why professionals recommend storing master copies in uncompressed or minimally compressed formats (ProRes, DNxHR) and transcoding only final delivery versions.

Transcoding Tools

The choice of transcoding tool depends on the platform, performance requirements, and use case. Both native APIs and cross-platform libraries are available for mobile development.

FFmpeg — Universal Tool

FFmpeg is the de facto standard for transcoding on all platforms. The FFmpeg command line allows performing virtually any conversion: codec change, bitrate adjustment, trimming, concatenation, filter application. For mobile applications, FFmpeg is integrated via the libavformat, libavcodec, and libavfilter libraries. Example of a typical transcoding command: ffmpeg -i input.mp4 -c:v libx265 -crf 23 -c:a aac -b:a 128k output.mp4.

Platform APIs

On iOS, transcoding is performed via AVAssetWriter and AVAssetReader. AVAssetReader decodes the source file by reading uncompressed frames, and AVAssetWriter encodes them into the target format. This approach automatically uses VideoToolbox hardware encoders, ensuring maximum performance. On Android, similar functionality is available through MediaCodec paired with MediaExtractor and MediaMuxer — MediaExtractor extracts compressed packets, MediaCodec decodes and encodes, MediaMuxer writes the result.

Cloud Services

For server-side transcoding in production environments, cloud services are used: AWS Elemental MediaConvert, Azure Media Services, Google Transcoder API. These services automatically scale under load, support all popular formats, and can transcode a single input file into dozens of output variants for adaptive streaming (HLS, DASH). For mobile applications, cloud transcoding is the optimal solution as it does not burden the user's device and allows asynchronous content preparation.

Transcoding Code Examples

Let us examine practical transcoding examples on mobile platforms using hardware acceleration and configuration of key quality parameters.

H.264 to H.265 Transcoding on iOS

import AVFoundation

func transcodeVideo(sourceURL: URL, destURL: URL) {
    let asset = AVAsset(url: sourceURL)
    let preset = AVAssetExportPresetHEVCHighestQuality
    
    AVAssetExportSession(asset: asset, presetName: preset)?
        .exportAsynchronously {
            switch assetExportSession?.status {
            case .completed:
                print("Transcoding finished")
            case .failed:
                print("Error: " + assetExportSession.error.localizedDescription)
            default:
                break
            }
        }
    
    // Manual transcoding with AVAssetReader + AVAssetWriter
    let reader = try AVAssetReader(asset: asset)
    let writer = try AVAssetWriter(url: destURL,
        fileType: .mp4)
    
    let outputSettings: [String: Any] = [
        AVVideoCodecKey: AVVideoCodecType.hevc,
        AVVideoWidthKey: 1920,
        AVVideoHeightKey: 1080,
        AVVideoCompressionPropertiesKey: [
            AVVideoAverageBitRateKey: 4_000_000,
            AVVideoProfileLevelKey: AVVideoProfileLevelH265Main10
        ]
    ]
    
    let adaptor = AVAssetWriterInput(
        mediaType: .video,
        outputSettings: outputSettings
    )
    writer.add(adaptor)
}

The example shows two approaches to transcoding on iOS. AVAssetExportSession is a simple way with quality presets (HEVCHighestQuality for H.265). The manual pipeline via AVAssetReader + AVAssetWriter provides full control over parameters: bitrate, profile, level. The AVVideoProfileLevelH265Main10 parameter enables the HDR profile Main10 with 10-bit color depth, which is important for modern HDR content.

Transcoding on Android with MediaCodec

class Transcoder(private val context: Context) {
    
    fun transcodeToHevc(inputUri: Uri, outputFile: File) {
        val extractor = MediaExtractor()
        extractor.setDataSource(context, inputUri, null)
        
        val trackFormat = extractor.getTrackFormat(videoTrackIndex)
        val mime = trackFormat.getString(MediaFormat.KEY_MIME)
        
        val decoder = MediaCodec.createDecoderByType(mime!!)
        val encoder = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_VIDEO_HEVC)
        
        val outputFormat = MediaFormat.createVideoFormat(
            MediaFormat.MIMETYPE_VIDEO_HEVC, 1920, 1080
        ).apply {
            setInteger(MediaFormat.KEY_BIT_RATE, 4_000_000)
            setInteger(MediaFormat.KEY_FRAME_RATE, 30)
            setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 2)
        }
        
        encoder.configure(outputFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE)
        encoder.start()
    }
}

The Android code creates a pipeline of MediaExtractor — MediaCodec decoder — MediaCodec encoder — MediaMuxer. MediaExtractor determines the codec type from the input file and selects the corresponding decoder. The encoder is configured for H.265 (HEVC) with a bitrate of 4 Mbps and a key frame interval of 2 seconds, which is optimal for streaming. Important: MediaCodec encoder works synchronously, so for realtime transcoding a loop with correct PTS timestamp handling for each frame must be implemented.

Optimizing Transcoding for Mobile Devices

Transcoding on mobile devices is a task that requires careful optimization due to limited CPU, GPU resources and thermal constraints. Several strategies help perform transcoding efficiently.

Using Hardware Encoding

The key performance factor is the hardware encoder. On iOS, VideoToolbox provides hardware H.264 and H.265 encoding at speeds 5–10 times faster than software libx264. On Android, MediaCodec uses hardware OMX components if available. Enabling hardware encoding reduces the transcoding time of a 10-minute video from 30–40 minutes (software) to 3–5 minutes (hardware) on a flagship device.

Choosing Quality Parameters

For mobile transcoding, the balance between quality, size, and processing time is critical. For H.265 on mobile devices, a bitrate of 4–8 Mbps is recommended for 1080p video at 30 FPS. CRF mode (Constant Rate Factor) in libx265 allows setting quality directly, where 23–28 provides good visual quality at a moderate file size. For hardware encoders, use CBR mode with a target bitrate since CRF is not supported in hardware.

Thermal Management

Continuous transcoding on a mobile device causes significant heating. After 5–7 minutes of intensive 4K video encoding, the processor temperature can reach 50–55 degrees, after which throttling kicks in. The solution is transcoding with pauses or reducing the frame rate to 30 FPS. If the application requires batch transcoding (e.g., a video editor), it is better to process in 2–3 minute batches with cooling intervals. For production scenarios, it is optimal to offload transcoding to the server side and use cloud services.