Device Farm: What It Is, Cloud Services, and Real Devices

What is Device Farm

Device Farm is an infrastructure solution for testing mobile applications that provides remote access to physical and virtual devices over the internet. Unlike purchasing and maintaining your own device lab, developers rent time on third-party devices, paying only for actual usage.

Why Device Farm is Needed

The mobile device market includes over 24,000 unique Android smartphone models according to OpenSignal, 2025. Physically acquiring all popular models for testing is impossible — it would require millions of dollars in investment and constant fleet updates. Device Farm solves this problem by providing access to current devices on a subscription basis.

Physical Devices vs Emulators

Emulators and simulators cover only 60–70% of testing scenarios. Real devices are necessary for testing sensors, camera, GPS, battery level, performance on weak hardware, and behavior during incoming calls. Device Farm combines both approaches: emulators for quick smoke tests and physical devices for full verification.

Who Device Farm is For

Device Farm is suitable for startups launching their first app, mature product teams with regular releases, and enterprise projects with dozens of applications. The value increases with the number of target devices and release frequency — the more configurations need to be tested, the more cost-effective the cloud model becomes.

How Device Farm Works

A typical workflow with Device Farm consists of five stages: uploading the build, selecting devices, configuring tests, running them, and analyzing results. Each stage is automated via API, allowing Device Farm to be embedded into a CI/CD pipeline without manual intervention.

Uploading the Application

The developer uploads the compiled APK (Android), AAB, or IPA (iOS) to the provider’s cloud storage. Some services, such as Firebase Test Lab, also accept Xcode or Android Studio projects directly. File size is limited: AWS Device Farm supports up to 200 MB, BrowserStack up to 500 MB.

Selecting Configurations

Target models and OS versions are selected from the device catalog. Modern Device Farms support grouping: “All Flagships 2024–2025” or “Budget Android with Android 13–14.” Parallel execution allows running one test simultaneously on 10–50 devices, reducing total time to minutes.

Running and Monitoring

Tests execute on real devices in the provider’s data centers. The developer monitors progress through a real-time web console: logs, screenshots, and test run videos. When a test fails, Device Farm takes a screenshot and collects a crash log for diagnosis.

// Example of a Device Farm configuration in a Jenkinsfile
pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                sh './gradlew assembleDebug'
            }
        }
        stage('Device Farm Tests') {
            steps {
                sh '''aws device-farm schedule-run
                    --project-arn arn:aws:devicefarm:us-west-2:123:project:1
                    --app-arn arn:aws:devicefarm:us-west-2:123:app:1
                    --device-pool-arn arn:aws:devicefarm:us-west-2:123:pool:1
                    --test-arn arn:aws:devicefarm:us-west-2:123:test:1'''
            }
        }
    }
}

Key Features of Device Farm

Modern Device Farms provide not only test execution but also supporting tools for diagnostics, performance monitoring, and integration with other development services. The feature set varies between providers, but the core remains common.

Automated Testing

All major frameworks are supported: Espresso and UI Automator for Android, XCTest and XCUITest for iOS, Appium and Calabash for cross-platform projects. Device Farm runs tests without code modification — simply specify the test framework during configuration.

Manual Interactive Testing

The developer gets remote access to a real device through a browser: touches, swipes, screen rotation, button presses — all actions are transmitted in real time. Interactive mode is indispensable for reproducing rare bugs not covered by automated tests. AWS Device Farm provides up to 60 minutes of manual sessions at no extra charge.

Diagnostic Data Collection

After each run, Device Farm generates a report: video recording of the test, step-by-step screenshots, device logs (logcat for Android, syslog for iOS), performance data (CPU, memory, network), and crash stacks. This data helps localize issues without reproducing them on a local device.

CI/CD Integration

All major Device Farms provide plugins for Jenkins, GitLab CI, GitHub Actions, and CircleCI. Webhook notifications alert the team about test results via Slack, Telegram, or email. Setting up CI/CD integration is standard practice for teams practicing continuous delivery.

Popular Device Farm Providers

The Device Farm market is represented by three main categories: general-purpose cloud providers (AWS), specialized testing services (BrowserStack), and platform ecosystem tools (Firebase). The choice depends on the technology stack and team budget.

AWS Device Farm

Integrated with the Amazon Web Services ecosystem. Developers already using AWS for hosting and CI/CD get a single contract and seamless integration with CodePipeline and CodeBuild. AWS Device Farm supports testing Android, iOS, and web applications on physical devices and emulators.

BrowserStack App Automate

The largest independent provider with a fleet of over 3,000 real devices. BrowserStack is known for its user-friendly web interface, fast device provisioning (10–15 seconds), and support for Appium without complex configuration. The service automatically updates its device fleet within a week of a new model’s release.

Firebase Test Lab

A free service from Google for Android applications with limited iOS testing support. Firebase Test Lab is especially attractive for startups: basic testing scenarios are available without a subscription. Integration with Firebase Console and Crashlytics allows linking test failures to real crashes in production.

Other Providers

The market includes Samsung Remote Test Lab (free access to Samsung devices), Kobiton (enterprise-oriented with on-premise deployment), Perfecto (acquired by Perforce, focused on the financial sector), and Sauce Labs (one of the oldest services, now part of Tricentis).

Example of Running Tests on Device Farm

Let’s look at integrating Firebase Test Lab with the command line for an Android app. Firebase Test Lab runs tests via the gcloud CLI, allowing integration into any CI pipeline without additional plugins. The example demonstrates a minimal smoke test configuration on three devices.

# Installing the gcloud CLI and authorization
gcloud auth login
gcloud config set project my-android-app

# Uploading the APK to Firebase Test Lab
gcloud firebase test android run \
    --app app-debug.apk \
    --test app-debug-test.apk \
    --type instrumentation \
    --device model=Pixel7,version=34 \
    --device model=SamsungS23,version=33 \
    --device model=OnePlus12,version=34 \
    --timeout 30m \
    --results-bucket gs://my-app-test-results

The command uploads two files: the build itself and the test APK with Espresso instrumentation tests. Firebase Test Lab automatically selects the latest available OS version for each model if version is not explicitly specified. Results are saved to Google Cloud Storage in XML and HTML format for browser viewing.

GitHub Actions Integration

For automatic execution on every push, use GitHub Actions. The workflow builds the app, runs Firebase Test Lab, and returns the check status.

name: Android CI with Device Farm
on: [push]
jobs:
  test-on-devices:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: 17
      - run: ./gradlew assembleDebug assembleDebugAndroidTest
      - uses: google-github-actions/auth@v2
        with:
          credentials_json: ${{ secrets.GCP_SA_KEY }}
      - run: gcloud firebase test android run
          --app app/build/outputs/apk/debug/app-debug.apk
          --test app/build/outputs/apk/androidTest/debug/app-debug-test.apk
          --device model=Pixel8,version=35

Device Farm Pricing Models

The cost of Device Farm depends on several parameters: number of parallel sessions, device type (physical or emulator), test duration, and additional features. It is important to understand the cost structure before choosing a provider to avoid unexpected expenses when scaling.

Testing Minutes

Most providers charge for actual device usage time. Physical devices cost 2–5 times more than emulators. AWS Device Farm offers 1000 free minutes per month for new accounts, BrowserStack — 60 minutes trial period. The average cost per minute on a physical Android device is $0.05–0.17.

Team Subscription

For regular testing, a fixed subscription is more cost-effective. BrowserStack offers a Team plan at $299/month for 3 users with unlimited minutes on physical devices. AWS Device Farm does not have a flat subscription — payment is per minute, which is convenient for irregular workloads but more expensive under intensive use.

Hidden Costs

When budgeting, consider: storage of testing artifacts (videos, screenshots, logs) — AWS Device Farm stores results for up to 90 days at no extra charge, BrowserStack up to 30 days; network traffic when uploading large APK/IPA files; number of parallel sessions — increasing parallelism may require a more expensive plan.