Closed and Open Beta — What It Is, How App Beta Testing Works

What Is App Beta Testing

Beta testing is a stage of testing an application on real users before the official release. Unlike Internal Testing, where developers and QA engineers test, beta tests are conducted on an external audience that uses the application in real-world conditions with their own devices, data, and scenarios.

Beta testing is divided into two types: Closed Beta and Open Beta. In Google Play, both tracks are available through the Developer Console; in App Store — via TestFlight. The main difference is in the access method: Closed Beta requires an invitation, Open Beta is available via a public link or through store search.

Why Beta Testing Is Needed

According to a Google Play Console study, beta tests reveal up to 40% of critical bugs that were not detected during Internal Testing. Real users use different device models, OS versions, and network conditions that are impossible to reproduce in a test environment. Beta testing also gathers qualitative feedback on UX/UI and new features.

Beta Testing Stages in the Pipeline

A typical pipeline looks like this: Internal Testing → Closed Beta → Open Beta → Production. After stabilization on the Internal track, the build is published to Closed Beta for a limited external audience. After collecting feedback and fixing bugs — to Open Beta for everyone. The final Production release is performed after confirming stability on Open Beta.

Closed Beta: Features and Setup

Closed Beta is an invitation-only testing track. The developer specifies a list of email addresses or creates a Google Group whose members get access to the beta version. In Google Play, Closed Beta supports up to 10,000 testers, which significantly exceeds the Internal Testing limit of 100 people.

Setting Up Closed Beta in Google Play

To create a Closed Beta track, go to Google Play Console → Release → Testing → Closed Beta. Create a tester group and specify the method of adding: by email, via Google Group, or through an invitation link. After uploading the build and its verification by Google Play, the system sends invitations to group members.

// Fastlane — publishing to Closed Beta track
lane :closed_beta_release do
    gradle(task: ":app:assembleRelease")
    
    upload_to_play_store(
        track: "beta",
        release_status: "draft",
        rollout: 1.0
    )
    
    promote_to_play_store(
        track: "beta",
        release_status: "completed"
    )
end

Version Management on Closed Beta

The Closed Beta track uses a separate version code number. It is recommended to allocate a versionCode range that does not overlap with Internal Testing and Production. For example, for version 2.4.0: Internal → versionCode 24000, Closed Beta → 24001, Open Beta → 24002, Production → 24003. This prevents conflicts when promoting a build between tracks.

Open Beta: Public Testing

Open Beta is a track available to all users without an invitation. In Google Play, Open Beta appears in the store as a separate app card with a Beta label. Any user can join the testing through a public link or by finding the app in Google Play and clicking Become a Tester.

Benefits of Open Beta

Open Beta provides maximum audience coverage for testing. Unlike Closed Beta, where the sample is determined by the developer, Open Beta attracts users with diverse devices, habits, and scenarios. This gives the most complete picture of application stability before release. Feedback is collected through Google Play Rating and in-app surveys.

Open Beta Limitations in Google Play

Open Beta is available to any developer account, but requires moderation approval before publication. Google Play checks the build for compliance with basic requirements, just like a production release. After approval, the track is published in the store, and any user can subscribe to it. You can cancel Open Beta at any time without losing current installs.

Beta Testing in App Store via TestFlight

In the Apple ecosystem, external beta testing is performed through TestFlight External Testing. The maximum number of external testers is 10,000 people. Unlike Google Play, TestFlight does not support a full Open Beta with store display — access is distributed only via an invitation link or through an Apple public page.

TestFlight External Testing: Process

To publish a build to TestFlight External Testing, the developer uploads an IPA via Xcode or Transporter, after which Beta App Review begins. Apple checks the build for basic requirements — unlike a full App Review, the check takes 1–2 days. After approval, the build is available for distribution via link for up to 90 days. To extend the period, a new build must be uploaded.

Collecting Feedback via TestFlight

TestFlight has built-in support for collecting screenshots and device logs. When the tester shakes the device, a report is sent to the developer through App Store Connect. Each report contains a stack trace, screenshot, build version, and device information. This simplifies reproducing and fixing bugs without lengthy correspondence with the tester.

Setting Up Beta Tracks in Google Play Console

Setting up Closed and Open Beta in Google Play Console is done in the Release → Testing section. The process takes 15–30 minutes and requires a one-time track configuration before first use. Let us go through step-by-step instructions for both types of beta testing.

Publishing to Closed Beta

In Google Play Console, create a tester group and upload the build to the Closed Beta track. The system checks basic requirements and within 5–15 minutes the build becomes available to group members. Members receive an email with an invitation and installation instructions through Google Play.

Publishing to Open Beta

Select the Open Beta track and upload the build. Unlike Closed Beta, Open Beta undergoes moderation (like a production release), which takes 24–48 hours. After approval, the app card appears in Google Play with a Beta label. Users can join testing through the Become a Tester button.

Best Practices for Beta Testing

The effectiveness of beta testing directly depends on the quality of process organization. Below are proven practices based on the experience of major developers and Google Play Console recommendations. Following these rules increases the bug detection rate by 40–60%.

• Start with Closed Beta on a trusted audience of 100–500 people
• Collect stability metrics: ANR, crashes, freeze frequency
• Use built-in feedback tools (Firebase, Crashlytics, TestFlight)
• Conduct A/B testing of new features in Closed Beta before Open Beta
• Set a beta test duration: 7–14 days for Closed, 14–30 days for Open

Analyzing Feedback and Metrics

After completing the beta test, collect all reports, classify bugs by priority, and hand them over to development. Bugs discovered in Open Beta must be fixed before the production release. Users who participated in the beta test often become the first active users after the official launch.

Communicating with Beta Testers

Keep testers informed of updates. Use built-in notifications in Google Play and TestFlight to announce new builds. Maintain a changelog with descriptions of fixes and new features. Respond to feedback in the Resolution Center (TestFlight) or on the app page (Google Play) — this increases tester engagement.

Beta Test Performance Metrics

Key metrics for evaluating a beta test: number of active testers, percentage reporting bugs, average time to first report, and Coverage Rate — the percentage of devices and OS versions covered by testing. If the Coverage Rate is below 40%, add testers with missing configurations through targeted invitation emails.