Sync Engine: Key Concepts, Types and Working Mechanisms

What is a Sync Engine?

Sync Engine — is an architectural layer between the local database and a remote API that manages data flow in both directions. Its tasks: track changes, send them to the server, receive server changes and resolve conflicts. The user interacts with local data, while the Sync Engine seamlessly synchronizes it with the server.

Sync Engines can be built-in (Firebase Firestore, Couchbase Lite, Realm) or custom — written for specific business logic. Built-in engines offer ready-made offline-first functionality and conflict resolution. Custom engines provide full control over data format, sync protocol and conflict policy.

According to Sravana Karthik (2024), author of «Mobile Sync Engine Design Patterns», a custom Sync Engine is justified for apps with complex business logic (finance, healthcare, IoT) where custom merge rules are critical. For typical scenarios (notes, chats, feeds), a built-in Firestore or Realm is sufficient.

interface SyncEngine {
    suspend fun pull(lastSyncTimestamp: Long): SyncResult
    suspend fun push(operations: List<QueuedOperation>): PushResult
    suspend fun resolve(conflicts: List<Conflict>): ResolutionResult
    fun observeSyncState(): Flow<SyncState>
}

This interface describes the minimal Sync Engine contract: pull (loading server changes), push (sending local changes), resolve (handling conflicts) and observe (monitoring sync state). Such an abstraction allows changing the implementation without modifying the Presentation layer.

Sync Types: Full, Incremental and Push

Full sync — each session loads the entire dataset from the server. Simple to implement, but unacceptable for large volumes: downloading 10,000 records every time you open the app consumes traffic and battery. Full sync is justified for reference data (country list) with rare updates.

Incremental sync — only records changed since the last sync are transferred. The server stores the timestamp of the last change for each record or the entire set. The client sends lastSyncTimestamp and receives only records with updated_at > that value. According to Instagram Engineering (2024), incremental sync reduces data transfer volume by 97% compared to full sync.

Push sync (server-initiated) — the server itself notifies the client about the need to sync via FCM (Firebase Cloud Messaging), WebSocket or SSE (Server-Sent Events). The client does not waste resources on periodic polling. Push sync is the optimal choice for real-time apps: chats, notifications, likes. Google Firebase Firestore uses WebSocket for real-time sync with automatic fallback to HTTP polling.

Hybrid approach — a combination of types: full sync for baseline data at app startup, then incremental sync for updates, and for critical events — push sync via FCM. This provides both speed and resource savings.

Incremental Sync — How Checkpoints and Deltas Work

Checkpoint — a value that the client stores between sync sessions. Usually this is the updated_at of the last successfully synced record. On the next sync, the client sends the checkpoint to the server, and the server returns all records with updated_at after the checkpoint. Cursor-based pagination — an advanced version where the server returns a cursor (pointer to the next page) along with the data.

Delta sync — the server computes the diff between the current data state and the snapshot the client last saw. Instead of sending all records, only operations (insert, update, delete) are transferred. This is especially effective for large datasets where only a few records have changed. Google Drive API (2025) uses changes.list with pageToken for file delta sync.

«Deferred deltas» strategy — on the mobile client, changes are not sent immediately but buffered in the Offline Queue. When the threshold is reached (10 operations or 30 seconds), a delta package is formed and sent to the server. According to Dropbox Mobile Engineering (2024), delta batching reduced the number of HTTP requests by 65% and reduced battery consumption by 12%.

data class SyncCheckpoint(
    val lastUpdated: Long,
    val pageToken: String?,
    val version: Int
)

suspend fun syncIncremental(checkpoint: SyncCheckpoint): SyncResult =
    api.pullChanges(
        since = checkpoint.lastUpdated,
        token = checkpoint.pageToken
    )

SyncCheckpoint stores both the timestamp and the pagination cursor for long lists. Two-parameter checkpoint guarantees that no record is skipped or duplicated when syncing large datasets.

Push Sync — Instant Sync via WebSocket and FCM

WebSocket — a persistent bidirectional connection between the client and server. The server sends updates immediately when data changes. WebSocket is optimal for real-time apps: chats, streaming, collaborative work. Downside: battery and traffic consumption for maintaining the connection (heartbeat). OkHttp WebSocket on Android and URLSessionWebSocketTask on iOS — built-in implementations.

Firebase Cloud Messaging (FCM) — push notifications that the server sends not for user display but to trigger synchronization. Upon receiving a silent push (data message), the app wakes up and starts the Sync Engine. FCM does not require a permanent connection and is more economical than WebSocket for rare notifications.

SSE (Server-Sent Events) — a one-way channel through which the server sends events to the client. Simpler than WebSocket to implement, but does not support bidirectional communication. EventSource API (JavaScript) and OkHttp SSE (Android) — popular libraries. SSE is suitable for notifications about new data when the client does not need to send data back through the same channel.

According to WhatsApp Engineering (2024), their Sync Engine uses a combination of WebSocket for an active session and FCM for waking the app in the background: WebSocket disconnects after 5 minutes of inactivity, and subsequent updates are delivered via silent push.

Snapshot Sync and Data Versioning

Snapshot-based sync — the server periodically creates a full data snapshot and assigns it a version. The client stores the current version number. If it is outdated, it downloads a new snapshot. This is a simple and reliable strategy, but inefficient for frequent changes — the entire dataset is downloaded each time.

Per-record versioning — each record has a version field. During sync, the client sends the versions of all records, and the server returns only those whose version has changed. This is more efficient than snapshot sync but requires storing versions on the client. Vector Clocks — an advanced technique for distributed systems where each node assigns its own version and conflicts are resolved by partial order.

Snapshot with incremental diff — a hybrid approach: rare full snapshots (once a day) + incremental sync in between. Upon startup after a long absence, the client loads a snapshot, and during frequent syncs — only deltas. Git-like approach — each data commit has a hash, and the client knows which commit to base off of. This is implemented in Couchbase Lite Sync Gateway (2024) and is the gold standard for reliability.

data class VersionedEntryT(
    val id: String,
    val data: T,
    val version: Long,
    val deleted: Boolean
)

fun SyncEngine.resolveVersion(local: VersionedEntry*, remote: VersionedEntry*): VersionedEntry* =
    when {
        local.version > remote.version -> local
        remote.version > local.version -> remote
        else -> resolveConflict(local, remote)
    }

Version resolution rule: if versions match — no changes exist. If the local version is newer — local wins. If the server version is newer — server wins. Only when versions are equal but data differs — the conflict resolver is called. Last Write Wins with a version flag — the simplest but reliable strategy.

How to Build a Sync Engine for a Mobile App

Step 1: Define the data model — which entities are synchronized, how often they change, and their volume. For each entity, define the strategy (incremental / full / push) and acceptable sync delay.

Step 2: Choose a protocol — REST with checkpoints, GraphQL with Subscriptions, or gRPC with bidirectional stream. GraphQL Subscriptions — a popular choice for modern apps: one protocol for both pull and push. Apollo Client (2025) supports offline sync via device cache.

Step 3: Implement an Offline Queue — local change storage with idempotency keys (see the article «Offline Queue»). The queue is the foundation of a reliable Sync Engine: without it, synchronization does not guarantee change delivery.

Step 4: Choose a conflict resolver — LWW for simple cases, CRDT for collaborative editing, Custom merge for business logic. Rule: the resolver must be idempotent — re-applying the same operation must produce the same result.

Step 5: Monitoring and metrics — log each sync: record count, execution time, conflict count, errors. Firebase Crashlytics or Sentry (2025) allow tracking sync errors in real time.

According to Realm Team (2024), a typical mobile app Sync Engine processes 100–500 synchronizations per day per device, transferring an average of 50–200 KB of data per session. Protocol optimization — using Protobuf compression instead of JSON — reduces data transfer volume by another 40–60%.