CoreImage: Filters and Image Processing

What is CoreImage

CoreImage is an Apple framework for raster image and video processing, first introduced in macOS 10.4 Tiger and ported to iOS 5.0. Unlike Core Graphics or UIKit, CoreImage works at the pixel shader level, using the GPU for parallel computations. This allows applying complex effects — blur, color correction, morphing — to megapixel-resolution images in milliseconds.

CoreImage architecture is built on the concept of lazy evaluation: CIFilter does not perform processing until rendering in CIContext. The developer builds a chain of multiple CIFilters, connecting the output of one filter to the input of another, and only the render call launches the full GPU pipeline. This minimizes the number of passes through the graphics card and maintains performance.

CoreImage supports wide color gamut (P3, sRGB, linear sRGB), HDR images and working with 16-bit and 32-bit pixel formats. On devices with Neural Engine, CoreImage can combine GPU shaders with Core ML for intelligent processing — for example, automatic color correction based on machine learning, as implemented in the Photos app on iOS 17.

CoreImage Architecture: CIImage, CIFilter, CIContext

Three main classes form the core of CoreImage: CIImage (data), CIFilter (operation) and CIContext (execution environment). Each class is responsible for its own pipeline phase and does not mix responsibilities.

CIImage — Immutable Source

CIImage is a lightweight object that describes an image but does not directly contain pixel data. It can be created from UIImage, CGImage, Metal texture, GL texture or a file on disk. CIImage is immutable — any filter application creates a new CIImage without modifying the original. This allows building transformation chains without copying data between steps.

An important feature: CIImage is not tied to screen resolution or color space until rendering. The same CIImage can be rendered into a UIImage for screen and into a TIFF file for printing with different parameters — CoreImage automatically adapts processing to the target context.

CIFilter — Processing Operation

CIFilter is the main class for applying effects. Each CIFilter is a shader program with input parameters (inputImage, inputIntensity, inputRadius, etc.) and one output image. Apple ships over 200 built-in CIFilters, grouped by categories: CICategoryColorEffect (color correction), CICategoryBlur (blur), CICategoryStylize (stylization), CICategoryGeometryAdjust (geometric transformations).

To create a custom CIFilter, the developer writes a Metal shader (CIKernel) — a fragment shader in the Metal Shading Language that processes each pixel independently. After compilation, the shader is registered as a CIFilter subclass and can be used in chains alongside Apple’s built-in filters.

CIContext — Rendering Environment

CIContext is a heavyweight object that manages GPU resource allocation, shader compilation and buffering. Creating a CIContext is an expensive operation, so Apple recommends creating it once per application lifecycle and reusing it. The context is tied to a specific GPU (Metal, OpenGL) or CPU and determines the color space of the output image.

CIFilter: Built-in Filters and Custom Metal Shaders

The CIFilter library is constantly expanding: iOS 17 added new filters for HDR processing and machine learning. All built-in filters are documented in the Apple Developer Documentation and can be searched by categories or names. To discover available filters on the device, use CIFilter.filterNames(inCategory:).

Chain of Multiple Filters

The main capability of CoreImage is composing multiple CIFilters into a chain. The output of one filter is fed to the input of another through the inputImage parameter. CoreImage optimizes the chain at the rendering stage: adjacent filters working in the same color space are combined into a single GPU pass (kernel fusion). This provides a performance gain without developer intervention.

Custom Filters via CIKernel

To create a custom effect, the developer writes a Metal function with the [[kernel]] annotation. The function receives pixel coordinates and returns a color. CoreImage passes the shader to the GPU, where it executes in parallel for all image pixels. Custom filters are especially useful for unique visual effects: sepia with a nonlinear curve, dispersion blur, or film grain simulation.

#include <CoreImage/CoreImage.h>

[[kernel]] float4 customFilter(sample_t s, float intensity) {
    float4 result;
    result.rgb = s.rgb * (1.0 - intensity) + s.r * intensity;
    result.a = s.a;
    return result;
}

Performance and Caching

CoreImage automatically manages the intermediate results cache. If the same filter chain is applied to different images, CoreImage can reuse compiled shaders. For maximum performance, avoid creating a CIContext for each frame — use one context and reset the filter’s input parameters.

CIDetector: Face, Text and Object Detection

CIDetector is a CoreImage subsystem for detecting semantic objects in an image without using Vision or Core ML. CIDetector supports four detection types: faces (CIDetectorTypeFace), text (CIDetectorTypeText), QR codes (CIDetectorTypeQRCode) and rectangles (CIDetectorTypeRectangle). The detector works on the CPU with optimization for NEON instructions, ensuring stable speed on devices without a powerful GPU.

Face Detection

The face detector CIDetectorTypeFace returns an array of CIFaceFeature with face bounds coordinates, eye and mouth positions. Since iOS 12, CIDetector also detects smiles (hasSmile) and left/right eye position (leftEyeClosed, rightEyeClosed). Detection accuracy is sufficient for most AR and photo applications, but for critical scenarios Apple recommends Vision.framework.

Rectangle Detection

CIDetectorTypeRectangle finds rectangular areas in an image — documents, business cards, device screens. The detector returns CIRectangleFeature with four points forming the rectangle contour. According to WWDC 2024, the rectangle detector uses the same neural network architecture as Vision, but with lower memory consumption. This makes it preferred for real-time frame stream processing, for example, in document scanner applications.

Code Examples: Filters and Detection in Swift

Practical examples will help solidify understanding of CoreImage. Let’s look at two typical scenarios: applying a color filter to a photo and face detection with bounding box drawing.

Applying Sepia to an Image

A basic example of the CISepiaTone filter, which converts a color image to sepia tones. The filter accepts the inputIntensity parameter from 0 (original) to 1 (maximum effect). Note the lazy evaluation: the CIImage returned by filter.outputImage does not contain pixels — rendering only happens when context.createCGImage is called.

import CoreImage

guard let inputImage = CIImage(image: uiImage) else { return }
let filter = CIFilter(name: "CISepiaTone")
filter?.setValue(inputImage, forKey: kCIInputImageKey)
filter?.setValue(0.8, forKey: kCIInputIntensityKey)

let context = CIContext()
let outputImage = filter?.outputImage
let cgImage = context.createCGImage(outputImage!, from: inputImage.extent)
let resultImage = UIImage(cgImage: cgImage!)

Face Detection with Bounding Box Drawing

The following example uses CIDetector to find faces in an image and draw a rectangle around each detected face. CIFaceFeature contains coordinates in the image coordinate system (origin at bottom-left), so when drawing on UIKit, coordinate transformation via affine transform is required.

let detector = CIDetector(
    ofType: CIDetectorTypeFace,
    context: context,
    options: [CIDetectorAccuracy: CIDetectorAccuracyHigh]
)
let features = detector?.features(in: inputImage) as? [CIFaceFeature]

for face in features ?? [] {
    print("Face bounds: \(face.bounds)")
    if face.hasSmile {
        print("Person is smiling")
    }
}

Composing Multiple Effects

The final example demonstrates a chain of three filters: color correction, vignetting and texture overlay. Note that the outputImage of the first filter is passed as inputImage to the second — without intermediate rendering. When createCGImage is called, CoreImage combines all three operations into a single GPU pass.

let colorFilter = CIFilter(name: "CIColorControls")
colorFilter?.setValue(inputImage, forKey: kCIInputImageKey)
colorFilter?.setValues([
    kCIInputBrightnessKey: 0.1,
    kCIInputContrastKey: 1.2
])

let vignetteFilter = CIFilter(name: "CIVignette")
vignetteFilter?.setValue(colorFilter?.outputImage, forKey: kCIInputImageKey)
vignetteFilter?.setValue(2.0, forKey: "inputRadius")

let finalImage = vignetteFilter?.outputImage
let result = context.createCGImage(finalImage!, from: inputImage.extent)