Rasterization — what it is, stages and operating principle

What Is Rasterization?

Rasterization is a stage of the graphics pipeline that converts geometric primitives (triangles, lines) into a set of fragments for further processing by the Fragment Shader. The input consists of triangle vertices in screen coordinates and their attributes (color, normals, UV). The output is a set of fragments with interpolated attributes.

Unlike programmable shaders (Vertex and Fragment Shader), rasterization is a fixed (non-programmable) stage. The developer cannot change the rasterization algorithm but can control its parameters: culling mode, line width, polygon offset, and multisampling.

The Role of Rasterization in Real-Time Graphics

Virtually all modern real-time graphics — games, UI, AR/VR — use rasterization. It is the most performant rendering method, capable of processing millions of triangles per frame at 60 FPS. GPU rasterizers are optimized for parallel processing of thousands of triangles simultaneously through specialized ROP (Render Output Units) blocks.

How Does Rasterization Work?

Triangle rasterization begins after the Vertex Shader has transformed the vertices into screen coordinates. The GPU takes the three vertices of a triangle and determines which screen pixels lie inside that triangle.

Stages of Triangle Rasterization

The process includes three sequential steps: determining the triangle’s bounding box in pixels — checking each pixel inside the bounding box for triangle membership (edge function) — computing interpolated attributes (barycentric coordinates) for each covered pixel. Edge function is a linear equation that determines which side of the triangle edge a point lies on.

Rasterization Algorithms

Rasterization can be implemented using various algorithms, each with its own trade-offs between performance and quality. Modern GPUs use a combination of several approaches.

Scanline Algorithm

The classic rasterization algorithm developed for early GPUs. The triangle is processed line by line (scanline): for each screen line, the entry and exit points of the triangle are determined, and all pixels between them are filled. Scanline conversion is efficient for large triangles but requires complex logic for triangles of arbitrary shape.

Half-Space (Edge Function) Method

Modern GPUs use the half-space rasterization method: for each side of the triangle, an edge function is computed — a linear equation that is positive on one side and negative on the other. A pixel belongs to the triangle if all three edge functions are positive. Half-space rasterization is easily parallelizable and well-suited for SIMD GPU architectures.

Tile-Based Rasterization (TBR)

Mobile GPUs (Qualcomm Adreno, ARM Mali) use Tile-Based Rendering (TBR). The screen is divided into small tiles (16x16 or 32x32 pixels), and rasterization is performed for each tile separately. This minimizes memory accesses by storing the framebuffer in fast local memory (tile memory). TBR reduces power consumption, which is critical for mobile devices.

Rasterization vs Ray Tracing

Rasterization and ray tracing are two fundamentally different approaches to 3D graphics rendering. Rasterization dominates real-time applications, while ray tracing is used in offline rendering and is gradually entering real-time (on RTX-compatible GPUs).

MSAA and Anti-Aliasing in Rasterization

Rasterization introduces the problem of aliasing — jagged edges (jaggies) at triangle boundaries. This occurs because a pixel is discrete and cannot be partially covered by a triangle. Multisampling (MSAA) is used to combat aliasing.

How MSAA Works

MSAA (Multisample Anti-Aliasing) performs rasterization at an increased sampling rate (2x, 4x, 8x). Instead of one sample per pixel, multiple sub-samples are used, each tested for triangle coverage. MSAA 4x processes 4 sub-samples per pixel, providing noticeable edge smoothing without fully increasing the rendering resolution.

FXAA and Other Post-Process Methods

Unlike MSAA, post-process anti-aliasing methods (FXAA, SMAA, TAA) work after rasterization by analyzing the final image. FXAA (Fast Approximate Anti-Aliasing) detects image edges and applies blur for smoothing, working significantly faster than MSAA but with less quality.

Rasterization on GPU and Optimization

Rasterization on modern GPUs is performed by specialized hardware blocks — ROP (Render Output Units). The number of ROPs directly affects the fill rate — the number of pixels the GPU can process per second.

Rasterization Performance Factors

• Triangle count — the number of triangles per frame. Each triangle requires overhead for setup and edge testing.
• Fill rate — the number of pixels processed by ROP blocks per second. Depends on GPU clock frequency and the number of ROPs.
• Overdraw — repeated rasterization of the same pixel due to triangle overlap. Optimized by sorting objects by depth.

Rasterization Optimization for Mobile GPUs

For mobile GPUs (Tile-Based Rendering), critical factors include: minimizing overdraw through early-Z rejection, using back-face culling to remove invisible triangles, and occlusion culling to reject triangles fully covered by other objects.