Software Decoding: What It Is, How It Works, and Use Cases

What Is Software Decoding?

Software decoding is a method of decompressing media data where all computational operations are performed on the general-purpose CPU cores. Unlike hardware decoding, where each codec has a dedicated physical block, a software decoder is ordinary code that executes the same algorithms using processor instructions.

Software decoders are written in C/C++ using optimizations for specific CPU architectures: ARM NEON SIMD instructions for mobile devices, Intel SSE/AVX for desktops. The libavcodec library from FFmpeg contains tens of thousands of lines of optimized assembly code for different platforms, allowing software decoding to achieve decent performance even for heavy formats like AV1 on powerful CPUs.

The main advantage of software decoding is versatility. If a hardware decoder supports only 4–5 main formats (H.264, H.265, VP9, AV1), FFmpeg can decode more than 200 codecs: from modern AV1 and H.265 to archival Sorenson Spark, RealVideo, and Motion JPEG. This makes software decoding an indispensable tool for applications working with non-standard media data — such as professional video editors, video surveillance systems, and specialized players.

How Does Software Decoding Work?

Software decoding follows the same stages as hardware decoding, but on a general-purpose CPU. Each stage is implemented as functions that are called sequentially for each macroblock or frame. The key difference is flexibility: the developer can modify the pipeline, add filters, and post-processing between decoding stages.

Software Decoder Architecture

A typical software decoder consists of modules that implement individual algorithm stages. The entropy decoding module reads the bitstream and reconstructs quantized DCT coefficients. For H.264, this module implements CABAC (Context-Adaptive Binary Arithmetic Coding) — a complex algorithm with conditional branches that is difficult to accelerate in hardware but runs efficiently on a CPU with a good branch predictor.

The inverse quantization module multiplies the coefficients by the quantization step, and the inverse DCT module applies the discrete cosine transform. The software implementation of inverse DCT uses the fast Chen algorithm or the Loeffler algorithm, which reduce the number of multiply-accumulate operations from 4096 to 256 for an 8x8 block. NEON SIMD instructions (ARM) or SSE (x86) allow processing 4–8 coefficients per instruction, providing 4–8x speedup compared to scalar code.

The motion compensation module is the most memory-intensive. It extracts regions from reference frames according to motion vectors and applies subpixel interpolation. For H.265, interpolation accuracy reaches 1/8 pixel, requiring 8-tap FIR filter for luma and 4-tap for chroma. The software implementation must load large amounts of reference frame data from cache, making motion compensation a bottleneck when decoding high resolutions on CPU.

Decoding Specifics on Mobile CPUs

Modern mobile processors, such as the Apple A17 or Qualcomm Snapdragon 8 Gen 2, have 6–8 cores with enough performance for software decoding of 1080p H.264 without frame drops. However, for 4K content, especially in H.265 and AV1 formats, software decoding on CPU may struggle: typical load across all cores reaches 70–90%, which is critical for multitasking. Big cores (Apple Performance, Qualcomm Kryo Prime) provide ~4–5x performance compared to small efficiency cores, but consume proportionally more power.

Popular Software Decoders

The software decoder market features several key libraries, each optimized for its niche. The choice of decoder depends on the required formats, platform, and licensing constraints.

FFmpeg / libavcodec

FFmpeg is the de facto standard for software decoding in the industry. The libavcodec library includes decoders for all major and most rare codecs, supports all containers (MP4, MKV, AVI, MOV, WebM), and runs on all platforms. FFmpeg is licensed under LGPL/GPL, which requires compliance with licensing terms for commercial use. On mobile devices, FFmpeg is used through wrappers: ffmpeg-kit for iOS and Android, mobile-ffmpeg for React Native.

Dav1d — Optimized AV1 Decoder

Dav1d is a software AV1 decoder from VideoLAN (developers of VLC), written in C with SIMD optimizations. Its main goal is the fastest possible software decoding of AV1 on CPUs without hardware support. Dav1d is 30–50% faster than the reference libaom decoder from the Alliance for Open Media thanks to aggressive optimizations: manual cache management, JIT compilation for post-processing filters, and vectorization of critical functions.

On mobile devices, dav1d can decode 1080p AV1 in real time on flagship SoCs (Apple A16+, Snapdragon 8 Gen 2+), but 4K requires a powerful CPU. For example, on Apple M1, software dav1d achieves ~60 FPS for 4K AV1, while on Snapdragon 8 Gen 2 it reaches ~35 FPS. For stable 4K AV1 playback on mobile devices, hardware support is still recommended.

Software vs Hardware Decoding Comparison

The choice between software and hardware decoding is a trade-off between compatibility and efficiency. The table below provides a detailed comparison of key characteristics.

Software decoding provides maximum flexibility: the developer can modify algorithms, add custom filters, and implement custom processing pipelines. For example, in video editing applications, each decoding stage can be redirected to GPU for color correction or effect overlays — this is only possible with software control over decoding.

However, the price of flexibility is power consumption. For mobile devices with a 3000–5000 mAh battery, continuous software decoding reduces viewing time from 10–15 hours (hardware) to 2–4 hours. CPU heating to 45–50 degrees can also cause throttling — a reduction in processor frequency to prevent overheating, leading to frame drops and degraded user experience.

Software Decoding Code Examples

Let's look at a practical implementation of software decoding on both mobile platforms. On iOS, software decoding is used through FFmpeg, and on Android — through the same library with a Java/Kotlin wrapper.

Software Decoding with FFmpeg in C

extern "C" {
    #include <libavcodec/avcodec.h>
    #include <libavformat/avformat.h>
    #include <libswscale/swscale.h>
}

class SoftwareDecoder {
    AVCodecContext* codecCtx;
    
public:
    bool init(const char* filename) {
        AVFormatContext* fmtCtx = nullptr;
        avformat_open_input(&fmtCtx, filename, nullptr, nullptr);
        avformat_find_stream_info(fmtCtx, nullptr);
        
        int videoStream = av_find_best_stream(
            fmtCtx, AVMEDIA_TYPE_VIDEO, -1, -1, nullptr, 0
        );
        
        AVCodec* decoder = avcodec_find_decoder(
            fmtCtx->streams[videoStream]->codecpar->codec_id
        );
        
        codecCtx = avcodec_alloc_context3(decoder);
        avcodec_parameters_to_context(codecCtx,
            fmtCtx->streams[videoStream]->codecpar);
        avcodec_open2(codecCtx, decoder, nullptr);
        return true;
    }
    
    AVFrame* decodePacket(AVPacket* packet) {
        avcodec_send_packet(codecCtx, packet);
        AVFrame* frame = av_frame_alloc();
        int ret = avcodec_receive_frame(codecCtx, frame);
        return (ret >= 0) ? frame : nullptr;
    }
};

The code demonstrates a minimal FFmpeg pipeline for software decoding. avformat_open_input opens the file and determines the container format, avcodec_find_decoder automatically finds a suitable decoder for any codec. The decodePacket method uses the new API (avcodec_send_packet / avcodec_receive_frame), which supports multithreaded decoding with the AV_CODEC_FLAG_LOW_DELAY flag enabled for real-time applications.

Software Decoding in Kotlin with mobile-ffmpeg

class SoftwareDecoder(private val context: Context) {
    fun decodeVideo(inputPath: String, outputFolder: String) {
        val cmd = "-i $inputPath -vf fps=1 $outputFolder/frame_%04d.jpg"
        FFmpegExecutor(context).executeCommand(cmd) { rc ->
            Log.d("Decoder", "Finished with rc: $rc")
        }
    }
    
    fun getFrameCount(filePath: String): Int {
        val probe = MediaMetadataRetriever()
        probe.setDataSource(filePath)
        val duration = probe.extractMetadata(
            MediaMetadataRetriever.METADATA_KEY_DURATION
        )?.toIntOrNull() ?: 0
        val fps = probe.extractMetadata(
            MediaMetadataRetriever.METADATA_KEY_VIDEO_FRAME_COUNT
        )?.toIntOrNull() ?: 0
        probe.release()
        return fps
    }
}

The Kotlin example uses FFmpegExecutor to extract one frame per second from a video. The parameter -vf fps=1 creates a filter that skips 59 out of 60 frames, reducing CPU load. This approach is useful for creating previews and placeholders in mobile applications. For real-time software decoding, it is recommended to use the low-level libavcodec API directly through JNI.

Software AV1 Decoding with dav1d

#include <dav1d/dav1d.h>

int decode_av1_frame(const uint8_t* data, size_t size) {
    Dav1dContext* ctx = nullptr;
    Dav1dSettings settings = { 0 };
    dav1d_default_settings(&settings);
    settings.n_threads = 4;
    dav1d_open(&ctx, &settings);
    
    Dav1dData dav1d_data = { 0 };
    dav1d_data_wrap(&dav1d_data, data, size, nullptr, nullptr);
    
    Dav1dPicture pic = { 0 };
    if (dav1d_send_data(ctx, &dav1d_data) == 0) {
        dav1d_get_picture(ctx, &pic);
    }
    
    dav1d_close(&ctx);
    return pic.p.w;
}

Dav1d provides a minimalistic API: dav1d_open creates a decoder context with a specified number of threads, dav1d_send_data accepts the compressed bitstream, dav1d_get_picture returns the decoded frame in YUV420 format. For mobile devices, the optimal number of threads (n_threads) is the number of performance CPU cores minus one, to leave resources for the UI thread. Dav1d also supports Dav1dPicAllocator for memory management and avoiding unnecessary copies when transferring frames to the GPU.

When to Use Software Decoding

Despite higher power consumption, software decoding is indispensable in several scenarios where hardware decoding cannot provide the required functionality. Understanding these scenarios helps developers make architectural decisions.

Rare and Legacy Formats

Hardware decoders only support modern formats. If the application works with archival recordings, video surveillance (MJPEG, H.263), professional codecs (ProRes, DNxHD, CineForm), or content from third-party sources — software decoding through FFmpeg will be the only option. ProRes is decoded software-only on all devices except Apple A13+ chips with hardware support. For H.263, there is no hardware support on any modern SoC — only software decoding.

Custom Post-Processing

Software decoding provides full access to each frame processing stage. This is critical for applications that need to apply filters (blur, noise reduction, sharpening) directly on decoded data before output. FFmpeg filters allow building complex chains: decoding -> color correction -> scaling -> subtitle overlay -> encoding — all within a single library without transferring data between different APIs.

Fallback When Hardware Support Is Absent

The recommended media player architecture is hybrid: hardware decoding as primary, software decoding as fallback. Before playback, the application checks the availability of a hardware decoder for the given codec. If no decoder is found, software decoding is started through FFmpeg. This strategy ensures maximum compatibility without sacrificing performance for main formats. Availability checks should be performed at each launch, as hardware support may vary even on devices of the same model due to different SoC revisions.