Golden Test — What It Is, How Snapshot Testing Works and Its Application

What Is Golden Test and How Does It Work?

Golden Test is an automated check of a component’s visual appearance by pixel-by-pixel comparison with a reference. The process: (1) the developer or tester creates the first snapshot of the component — this is the “golden” (reference). (2) The golden file is saved in the repository next to the test. (3) On subsequent runs, the test re-renders the component and compares it with the saved golden. (4) If the images match — the test passes. If they differ — the test fails with a diff. Decision: either the changes are expected (update the golden) or it’s a bug.

How the golden is generated — the library renders the component into an off-screen buffer (Android: Canvas, iOS: UIGraphicsImageRenderer) without a real display. This means golden tests work on CI without an emulator screen (virtual display), speeding up execution. Paparazzi on Android uses Layoutlib from Android Studio — the same engine as the Layout Editor. iOSSnapshotTestCase uses UIKit rendering into a CGImage. The result is a PNG file of a fixed size.

Golden File Size and Storage Management

Golden files — a PNG snapshot of one screen (1080x1920) takes 200–800 KB depending on complexity. For a project with 500 golden tests, that’s ~100–400 MB in the repository. Solutions: (1) store golden files in Git LFS. (2) Use PNG compression (pngcrush, oxipng). (3) Store golden files on a separate storage (S3) and fetch them during build. At IT Sectr, we store golden files in Git LFS with a 1 MB per file threshold — this is enough for 90% of tests.

Flaky Golden Tests and How to Fix Them

Flaky golden tests — the main problem with golden testing. Different GPUs, font versions, and anti-aliasing produce micro-differences in pixels. Solutions: threshold (allowable percentage of differing pixels), fuzzy comparison, and running on identical CI agents (same GPU, OS, emulator version). Paparazzi uses pixel-perfect comparison, so CI agents must be identical.

Golden Test vs Screenshot Test: What’s the Difference?

Golden Test is a type of screenshot testing with a fixed reference. The term “golden” means the reference has been approved by the team and is stored in the repository. Any image change requires a conscious decision from the developer: update the golden or fix the code. Golden test works at the level of individual components (Composable, UIView) and does not require a real device.

Screenshot Test is a broader concept. A screenshot test can capture a full screen with real data, navigation, system status bar, and animations. Screenshot tests are often run on real devices or emulators via UI Automator (Android) or XCUITest (iOS). Golden tests run in a unit-test environment (JVM, XCTest) without an emulator and only capture a single component.

Coverage Strategy: Golden vs Screenshot

Golden vs Screenshot — golden tests for checking individual UI components (button, card, dialog) on every commit. Screenshot tests for E2E checks of entire screens before release. Golden tests provide fast feedback to the developer, screenshot tests provide confidence in the integrity of the entire application. At IT Sectr, we use golden tests for Pull Requests (3–5 minutes) and screenshot tests nightly (30–60 minutes).

Paparazzi and Roborazzi: Snapshot Testing on Android

Paparazzi — a library from Cash App (Square) that renders Android View and Jetpack Compose components into PNG without an emulator. It uses Layoutlib (the same engine as Android Studio Preview). Setup: add the Gradle plugin, write a test with @Test and @RunWith(PaparazziRule::class), call paparazzi.snapshot(view). Paparazzi does not support animations, video, or Real Device — only static component rendering.

// build.gradle.kts (module)
plugins {
    id("app.cash.paparazzi") version "1.3.1"
}

// Golden test for a Compose component
class ButtonGoldenTest {

    @get:Rule
    val paparazzi = Paparazzi(
        Paparazzi.PaparazziSnapshotConfig(
            deviceConfig = DeviceConfig.PIXEL_6,
            theme = "android:Theme.Material.Light.NoActionBar"
        )
    )

    @Test
    fun primary_button() {
        paparazzi.snapshot {
            Button(
                onClick = { },
                modifier = Modifier.width(200.dp)
            ) {
                Text("Submit")
            }
        }
    }
}

Roborazzi — an alternative to Paparazzi with support for Compose, View, and image comparison. Difference: Roborazzi works through Robolectric and supports a threshold (allowable pixel difference percentage). This reduces flakiness with different GPUs on CI. Roborazzi can also create GIF animations of changes (before/after/diff), which is convenient for code review. Golden file format: PNG + JSON metadata.

Updating the golden — after an intentional UI change, the developer deletes the old golden files and runs tests with the record flag. Paparazzi recreates all golden files. Then the developer commits the new golden files along with the code change. In Code Review, the reviewer sees the diff of old and new golden files. If the changes are approved — the PR is merged. If not — the developer fixes the code and restarts the tests. Never update golden files automatically on CI — only locally.

SwiftSnapshotTesting and iOSSnapshotTestCase on iOS

SwiftSnapshotTesting — a library from pointfree.co, creators of Composable Architecture. Supports UIView, UIViewController, CALayer, and SwiftUI View. Principle: assertSnapshot(matching: view, as: .image). On first run, the golden is created automatically. On subsequent runs, it’s compared. If the difference exceeds the allowable threshold, the test fails. SwiftSnapshotTesting works through UIGraphicsImageRenderer, which is compatible with CI (Xcode Cloud, GitHub Actions).

import SnapshotTesting
import XCTest

final class ProfileCardSnapshotTests: XCTestCase {

    func test_profile_card_default() {
        let card = ProfileCard(
            name: "Alice",
            avatar: UIImage.testImage(),
            badge: "Pro"
        )
        let controller = UIHostingController(rootView: card)

        assertSnapshot(
            matching: controller,
            as: .image(on: .iPhoneSe),
            record: ProcessInfo.processInfo
                .environment["RECORD"] != nil
        )
    }
}

iOSSnapshotTestCase (formerly FBSnapshotTestCase) — a library from Uber for UIKit. Unlike SwiftSnapshotTesting, iOSSnapshotTestCase requires specifying the screen size and orientation. Golden files are PNGs in the ReferenceImages folder. Advantage: works with UIKit without SwiftUI and supports iOS 12+. Disadvantage: does not update golden files automatically — must be run with the record flag. SwiftSnapshotTesting is more modern and recommended for new projects.

Device-specific golden — golden files differ for different screen sizes and orientations. The standard approach: name golden files as TestName@3x~iPhone14.png. SwiftSnapshotTesting automatically adds a device suffix if the .image(on: .iPhoneSe) parameter is specified. On Android, Paparazzi uses DeviceConfig to set the size. Store golden files for each supported device form factor separately. Do not use one golden for different sizes — this will lead to flaky tests.

Working with Golden Files in CI and Managing Updates

CI pipeline — golden tests should run on every Pull Request. If a test fails, CI shows the diff image as a build artifact. The developer reviews the diff and makes a decision. Important: golden files generated on CI are never committed automatically. Only local generation by the developer after an intentional change. GitHub Actions and GitLab CI support uploading artifacts (png, html) for viewing diffs in the browser.

Repository size — golden files grow quickly. 500 tests = 100–400 MB of PNGs. Solutions: (1) Git LFS — each golden is stored in LFS, cloned only on checkout. (2) Store golden files in a separate repository and include them as a submodule. (3) S3 + caching — golden files on S3, CI downloads only changed files by checksum. At IT Sectr, we use Git LFS with track *.png filter=lfs diff=lfs merge=lfs text=false. Locally, golden files reside in src/test/goldens/.

Code Review of golden files — regular git diff does not show PNG changes. Solutions: (1) GitHub opens PNG images on click. (2) Use Review Apps where golden diffs are visible in the browser. (3) Generate an HTML report with before/after/diff columns side by side. Paparazzi creates an HTML report with three columns: actual, expected, diff. The report is attached to CI artifacts. Reviewers view the report without downloading files locally.

When to update the golden — only after a conscious UI change. Changing a font, color, padding, icon — the golden must be updated. Adding a new button, rearranging elements — the golden must be updated. A bug fix that changes the visual appearance — the golden must be updated. Refactoring without UI changes — the golden should not change. If a golden changes without UI code changes — it’s a flaky test caused by environment, look for the cause in CI agents or dependency versions.