QA Testing (Android)

Android testing automation with Espresso, UI Automator, Compose Testing, screenshot tests, and adaptive UI validation.

Core References: Android Testing Docs, Build-Managed Devices, Compose Testing, UI Automator, Screenshot Testing, Accessibility Checking

Quick Reference

Task	Command
List emulators	emulator -list-avds
Start emulator	emulator @<avd_name>
List devices	adb devices
Install APK	adb install -r <path-to-apk>
Run unit tests	./gradlew test
Run instrumented tests (connected)	./gradlew connectedAndroidTest
Run instrumented tests (GMD)	./gradlew <device><variant>AndroidTest
Run screenshot tests	./gradlew validateDebugScreenshotTest
List GMD tasks	`./gradlew tasks --all
Clear app data	adb shell pm clear <applicationId>

Platform Versions

Look up current versions before pinning; this table keeps only the compatibility rules.

Component	Where to check the current version	Stable notes
Android / API level	Android developer release notes	The newest OS version is not automatically the target-API requirement; see "Google Play Target API Policy" below
AGP	https://developer.android.com/build/releases/about-agp	Select the wrapper/JDK from the compatibility table for the installed AGP minor; major-version matching alone is insufficient
Robolectric	https://github.com/robolectric/robolectric/releases	Robolectric 4.17 added SDK 37 support. JDK 21 is required for SDK 36+
UI Automator	https://developer.android.com/jetpack/androidx/releases/test-uiautomator	The 2.4 API boundary adds the uiAutomator {} DSL; select the project-approved release
Compose Preview Screenshot Testing	Android Studio / AGP release notes	Check plugin vs AGP test-suite setup and CLI vs IDE requirements separately; see screenshot-testing.md
ATD images	Emulator system-image list	Check supported API images in the managed-device guide and SDK Manager; ATD disables hardware rendering
Maestro	Maestro release notes	New API-level support can reach the local CLI before Maestro Cloud; check before targeting a new Android version

Google Play Target API Policy (check before every release cutover)

Before choosing the release matrix, read the target-API requirements for the app's form factor, submission type, deadline and extension status. New submissions and existing-app availability have different rules; private apps and Wear/TV/Automotive/XR can have exceptions. Record the applicable requirement and verify the app's targetSdk in the built artifact.

Quick Start

• Prefer build-managed devices (GMD) for CI, using ATD images where the suite and supported API levels allow it; use connectedAndroidTest for local ad-hoc runs.
• Select an ATD API from the supported image list; use standard images when the required API or rendering path is unavailable.
• Enable Android Test Orchestrator and clearPackageData for instrumented suites that need strong isolation.
• Disable animations in Gradle, not with ad-hoc ADB steps inside the test flow.
• Treat screenshot tests as a separate fast regression layer: Compose Preview Screenshot Testing for @Preview coverage, Paparazzi or Roborazzi for JVM rendering, device-based snapshots only when hardware fidelity matters.
• Cover adaptive layouts explicitly: compact and expanded layouts, tablet or foldable paths when supported, and orientation changes where the product depends on them.
• Add accessibility checks to existing UI tests early instead of treating accessibility as a manual-only phase.

Recommended Gradle defaults for stable instrumented tests:

android {
    testOptions {
        animationsDisabled = true
        execution = "ANDROIDX_TEST_ORCHESTRATOR"

        emulatorSnapshots {
            enableForTestFailures = true
            maxSnapshotsForTestFailures = 2
        }
    }
}

dependencies {
    androidTestUtil(libs.androidx.test.orchestrator)
}

If you rely on test isolation between instrumented tests, also set runner args such as clearPackageData=true in your Gradle or CI wiring.

Inputs to Gather

• UI stack: Views, Compose, or mixed
• Test layer: unit, Robolectric, instrumented UI, screenshot, UI Automator, macrobenchmark
• CI target: PR gate vs nightly vs release; managed emulator vs device farm
• Device matrix: min supported API, target API, screen classes, locales, dark mode, form factors
• Flake symptoms: timeouts, missing nodes, idling or sync issues, renderer-only diffs, device-only failures
• Accessibility requirements: touch targets, labels, contrast, TalkBack-critical flows
• App seams: DI hooks for fakes, feature flags, test accounts, deterministic clocks, image loaders

Testing Layers

Layer	Framework	Scope
Unit	JUnit + Mockito	JVM, no Android
Unit (Android)	Robolectric	JVM, simulated framework
UI (Views)	Espresso	Instrumented
UI (Compose)	Compose Testing	Instrumented; JVM with explicit Robolectric setup
Adaptive UI	Espresso Device API + DeviceConfigurationOverride	Instrumented or host-assisted
Screenshot	Compose Preview Screenshot Testing, Paparazzi, Roborazzi	JVM or instrumented
System	UI Automator	Cross-app, system UI, benchmarking drivers

Core Principles (Stability)

Device Matrix

• Default: emulators for PR gates; real devices or device farms for release-critical journeys.
• Cover at least min supported API and target API, then add tablet or foldable coverage if the product exposes expanded layouts.
• Model adaptive coverage by layout behavior, not by device marketing names alone.

Flake Control

• Prefer testOptions { animationsDisabled = true } for instrumented tests.
• Use Android Test Orchestrator when shared app state or process crashes cause cross-test leakage.
• Use IdlingResources, Compose synchronization, waitUntil, or UI Automator conditions instead of sleeps.
• Mock network with MockWebServer or DI fakes; avoid live backends in CI.
• Keep screenshot tests deterministic: fixed fonts, locale, clocks, network images, and seeded data.
• Use test-failure snapshots or screenshots in CI so failures have artifacts, not just stack traces.

Accessibility And Adaptive Coverage

• Run accessibility checks in the same suite as high-value Espresso or Compose journeys.
• Validate compact and expanded layout states intentionally; do not assume a phone-only test matrix covers responsive layouts.
• Prefer stable selectors: withId() for Views, testTag for Compose, and resource-id or content descriptions for UI Automator.

Expert Judgment (What A Checklist Misses)

Emulator vs. Physical Device — the actual decision

A checklist says "use emulators for CI, real devices for release." The judgment call is which real-device signals are worth paying for:

• Emulators (ideally build-managed, ATD where hardware fidelity is not required) are correct for PR gates because they are deterministic, disposable, and free of thermal/battery/OEM noise that would otherwise get misattributed to app bugs.
• Physical devices or a device farm earn their cost only for journeys where OEM or hardware behavior is the actual risk: camera capture, biometric auth, NFC/BLE, background-work reliability under real Doze/App Standby, foldable hinge/posture transitions, and payment SDKs that behave differently under real Play Integrity checks. If none of those are in the flow, a device farm run is theater, not signal.
• A common anti-pattern is running the entire regression suite nightly on a device farm "for confidence." This burns budget without changing the failure mode you'd actually catch — most of what fails there also fails, faster and cheaper, on a managed emulator. Reserve device-farm minutes for the handful of hardware-coupled journeys above and gate them separately from the general regression suite.
• Emulator system-image choice is itself a judgment call, not just "google vs aosp": for services-dependent flows, verify the image supplies the required services and distribution channel; anything asserting on hardware-rendered pixels needs a non-ATD image; use ATD where its supported API and reduced system surface fit the journey, otherwise a standard image. Google API images do not prove Play Store/distribution-dependent behavior; check that channel separately.

Android Flake Taxonomy (diagnose by category, not by symptom)

Generic "flaky test" triage wastes time re-running instead of classifying. Android UI-test flakiness clusters into a small number of root-cause families — identify which one you're looking at before reaching for retries:

1. Synchronization gaps — an async operation Espresso/Compose does not know how to wait for. Signature: passes on a fast local machine, fails under CI load; NoMatchingViewException or an assertion firing against a stale loading state. Root cause is almost always an unregistered IdlingResource (Espresso) or a missing waitUntil/synchronized TestDispatcher (Compose/coroutines) — see references/espresso-patterns.md for the registration trap.
2. Animation/transition timing — window or property animations still running when Espresso samples the view tree. Prefer the Gradle-level testOptions { animationsDisabled = true } for CI. Only fall back to the per-test adb-settings DisableAnimationsRule pattern in references/espresso-patterns.md when the Gradle flag is unavailable (for example, a local module or Gradle version that predates it) — label it "legacy" in your own suite, since the two mechanisms can silently disagree if both are present.
3. Semantics-tree merging (Compose-specific) — a child testTag inside a Material component (e.g. the Text inside a Button) is invisible to onNodeWithTag because merged descendants collapse into the parent node; or a multi-value assertion like assertTextEquals("Save") fails against a merged node's full Text list. Text finders such as onNodeWithText still match the merged parent by default — this is not a "finds nothing" bug, and not a timing issue at all; do not "fix" it with a wait loop. See the corrected merged-tree trap in references/compose-testing.md. A v1-to-v2 migration can instead expose queued coroutine work; classify that under synchronization gaps (#1), using the migration section in references/compose-testing.md.
4. State leakage across tests — a prior test left SharedPreferences, a DB row, or a singleton in a state the next test doesn't expect. Symptom is order-dependent failures that disappear when the failing test is run alone. Fix with Test Orchestrator + clearPackageData, not by reordering tests.
5. Doze / App Standby / background restrictions — tests that rely on background work (WorkManager, FCM, foreground services) can behave differently under device power-management states than in a freshly booted emulator. This shows up almost exclusively on physical devices or long-running CI machines, rarely on ephemeral managed devices — if a background-work test is flaky only in one environment, suspect power-management state before suspecting the test.
6. Renderer/fidelity mismatches — screenshot diffs that are real but meaningless: ATD disabling hardware rendering, GPU driver differences between CI and local, or font substitution differences across OS images. Distinguish this from an actual visual regression by re-running the same golden generation pipeline on the same image type used to record it.
7. Non-deterministic test data or clocks — live network calls, real System.currentTimeMillis(), or unseeded random data leaking into assertions or screenshot goldens. Always mockable; the fact that it's still happening usually means a fake wasn't reused when a new screen was added.

When a test is flaky, name which of the seven it is before touching the test — the fix for #1 (register/await) will not touch #4 (isolate/clear state), and applying #2's fix (disable animations) to a #5 issue (Doze) does nothing.

Test Pyramid Reality For Compose Apps

The classic pyramid (many unit tests, some integration, few E2E) still holds, but Compose changes where the middle layer sits:

• createComposeRule() still hosts your content in a ComponentActivity under the hood and runs instrumented on a device or emulator unless the module also runs it under Robolectric in test/ — it is not automatically device-free. What it does avoid is your app's own Activity, DI graph, and navigation. Prefer it over createAndroidComposeRule<Activity>() whenever the composable doesn't need real navigation, DI graph, or activity lifecycle, and run it under Robolectric (Roborazzi or a @RunWith(AndroidJUnit4) + isIncludeAndroidResources JVM test) when you actually want a fast, unit-test-like run. See references/compose-testing.md for the setup.
• This shifts a large share of what used to be slow instrumented Espresso coverage down into fast, parallelizable JVM-adjacent tests — but only if ViewModels are tested independently (Turbine/MockK against StateFlow, not through the UI). A Compose app that only has "instrumented Compose tests that also exercise the ViewModel" has recreated the old inverted pyramid with new tools.
• Screenshot tests (Compose Preview Screenshot Testing / Paparazzi / Roborazzi) form a separate, parallel layer, not a replacement for either unit or instrumented behavioral tests — they catch visual regressions that behavioral assertions cannot see (spacing, overlap, dark-mode contrast) but say nothing about correctness of state transitions.
• Reserve UI Automator and Maestro-style E2E flows for the top of the pyramid: a handful of true cross-app or full-stack smoke journeys (onboarding, checkout, deep-link entry), run on nightly/release pipelines, not PR gates.

Device / API Fragmentation Strategy

• Always cover the app's actual min supported API and its current target API — these two are non-negotiable regardless of market share.
• Beyond that pair, prioritize by behavioral risk, not by device popularity: API levels that changed permission models, background-execution limits, or storage scoping (e.g., scoped storage, notification runtime permission, foreground-service type declarations) deserve dedicated coverage on the OS version that introduced the change, even if that OS version is not the most common one in your install base.
• Treat any specific device/API market-share number as unverified unless sourced from your own analytics (Play Console's own device catalog and Android Vitals for your app) — public aggregate stats go stale within months and vary enormously by app category and geography. Do not hardcode "X% of users are on API N" into test-planning docs; pull it fresh from the app's own Play Console dashboard each planning cycle.
• Foldable/tablet/expanded-layout coverage is now a fragmentation axis in its own right, independent of API level — a phone-only test matrix increasingly under-covers real usage on large-screen and foldable devices even when API coverage looks complete.

When To Invest In Screenshot Testing

Screenshot testing has real setup and maintenance cost (goldens go stale, false positives from font/renderer drift, review burden on every intentional UI change). It is worth that cost when:

• The team ships UI changes frequently enough that manual visual review does not scale (multiple PRs/day touching shared components).
• The app has a design system or component library where a single regression fans out across many screens — screenshot tests catch that fan-out cheaply; behavioral tests do not.
• Dark mode, large font scale, or RTL layouts are supported and have historically regressed silently.

It is a poor early investment for a small team still iterating rapidly on visual design — churn in intentional goldens will dominate signal from real regressions. Start with host-side tools (Compose Preview Screenshot Testing, Paparazzi, Roborazzi) for cheap iteration; only add device-based snapshot testing (Shot, or ATD-excluded device runs) once a specific rendering-sensitive surface (WebView, Maps, camera preview, custom Canvas/GPU work) has already caused a shipped visual bug that host-side rendering could not have caught.

Writing Tests

• Espresso (Views): open references/espresso-patterns.md
• Compose: open references/compose-testing.md
• UI Automator (system or cross-app): open references/uiautomator.md
• Screenshot tests: open references/screenshot-testing.md
• Adaptive screen-size testing: open references/adaptive-screen-testing.md
• Accessibility checks: open references/accessibility-checks.md

Compose Test v2 Migration

The v1 Compose test APIs (androidx.compose.ui.test.junit4.createComposeRule) are deprecated. v2 (...junit4.v2.createComposeRule, ...test.v2.runComposeUiTest, from ui-test-junit4 1.11.0-alpha03+) switches the default dispatcher from UnconfinedTestDispatcher to StandardTestDispatcher and forces touch input mode by default. Standard UI assertions already synchronize; direct state reads and manual clock control can need waitForIdle()/runOnIdle {} or scheduler advancement (taxonomy item #1). See references/compose-testing.md for the full migration guide, failure-class breakdown, and code samples.

Workflow

Add a New UI Test (Instrumented)

• Pick the narrowest framework that can observe the behavior: Espresso or Compose first, UI Automator at the system boundary.
• Add stable selectors: View id, Compose Modifier.testTag, system resource-id or content description.
• Control externals: fake network, deterministic data, fixed time and locale, and stable image sources.
• Add waits through framework synchronization or explicit conditions; avoid Thread.sleep().
• Run locally with connectedAndroidTest or a single managed-device task before widening the matrix.

Add Screenshot Coverage

• Use Compose Preview Screenshot Testing for @Preview-driven Compose UI states.
• Use Paparazzi or Roborazzi when you need JVM-fast coverage across Compose and legacy Views.
• Keep goldens small, reviewable, and tied to stable UI states instead of whole-screen snapshots of volatile content.
• Capture HTML or diff artifacts in CI on every failure.

Diagnose a Flaky Instrumented Test

• Reproduce locally or on one managed device before widening the matrix.
• Remove nondeterminism: network, locale, clock, feature flags, image loading, renderer differences.
• Replace sleeps with idling, waitUntil, onElement, or watcher-based synchronization.
• Capture logcat, screenshot, screen recording, and any managed-device failure snapshot.
• If still flaky, isolate app state further with Orchestrator or runner args and bisect the interaction steps.

Add A CI Gate (Preferred: Build-Managed Devices)

• Configure build-managed devices plus ATD images when hardware rendering is not required and the API level is supported by the image.
• Add the GitHub Actions GPU flag when your runner lacks hardware rendering support.
• Keep PR gates small and deterministic; expand device groups or sharding on nightly and release pipelines.
• Upload reports from build/reports/androidTests/, screenshot reports, diff images, logcat, and managed-device outputs.
• Report build, install, launch, interaction, and distribution evidence separately. Record APK/AAB identity, variant, device/API, locale, exact test task, and artifacts; a Gradle build proves packaging, while only an outcome assertion on the named device proves the exercised journey.
• Keep notifications, billing, deep links, background work, and OEM-specific flows unverified until the relevant real-device or distribution-channel check runs.

ADB Commands (Triage)

# Screenshot
adb exec-out screencap -p > screenshot.png

# Screen recording
adb shell screenrecord /sdcard/demo.mp4

# Pull managed-device test artifacts after a local failure
adb pull /sdcard/Android/media ./device-artifacts

CI Integration

Preferred: build-managed devices. See references/gradle-managed-devices.md and references/android-ci-optimization.md.

# .github/workflows/android.yml
name: Android CI
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6  # or later — verify at actions/checkout releases
      - uses: actions/setup-java@v5  # or later
        with:
          java-version: '21'  # SDK 36 unit tests need JDK 21
          distribution: 'temurin'
      - uses: gradle/actions/setup-gradle@v5  # or later
      - run: ./gradlew testDebugUnitTest <device><api>DebugAndroidTest -Pandroid.testoptions.manageddevices.emulator.gpu=swiftshader_indirect

Navigation

The reference guides are intentionally large; search within them instead of loading everything:

• rg -n "^## " skills/universal/qa-testing-android/references/compose-testing.md
• rg -n "Idling|waitUntil|Synchronization" skills/universal/qa-testing-android/references/compose-testing.md
• rg -n "DisplaySizeRule|DeviceConfigurationOverride|fold" skills/universal/qa-testing-android/references/adaptive-screen-testing.md
• rg -n "PreviewTest|Paparazzi|Roborazzi|Shot" skills/universal/qa-testing-android/references/screenshot-testing.md

Do / Avoid

Do

• Prefer build-managed devices plus ATD images for stable CI
• Use Android Test Orchestrator only where isolation solves a real state-leak or crash problem
• Add screenshot and accessibility checks to high-value UI flows
• Run a small matrix on PRs and widen via groups or shards on nightly or release pipelines
• Use a MainDispatcherRule + TestDispatcher (runTest { }, advanceUntilIdle(), kotlinx-coroutines-test) for every ViewModel and StateFlow test — choose eager UnconfinedTestDispatcher only when that scheduling behavior is intentional, because it can mask ordering races
• Run a release-variant smoke test in CI that exercises at least one @Serializable endpoint per data class, so R8 full-mode stripping of kotlinx-serialization $serializer classes is caught before release (see software-android-runtime-debugging/references/proguard-r8-triage.md)
• For Compose tests, assert on semantic state — onNodeWithTag(...).assertTextEquals(...) — not on reference equality of UI state objects, because Strong Skipping Mode means the UI may or may not receive the same instance across emissions

Avoid

• Thread.sleep() for synchronization
• Tests that depend on live backends or time-sensitive external content
• Flaky selectors such as localized text or positional-only nodes
• Device-wide screenshot assertions on ATD when hardware-rendered fidelity is required
• Asserting StateFlow<UiState> reference equality across emissions (assertThat(state).isSameInstanceAs(previous)) — new data class instances from copy() have different references but structurally equal content; test semantic equality, not identity
• Running serialization-dependent tests only on debug builds — R8 is off in debug, so debug-only suites cannot catch serialization keep-rule failures

Resources

Resource	Purpose
references/espresso-patterns.md	Espresso matchers and actions
references/compose-testing.md	Compose testing guide
references/uiautomator.md	UI Automator patterns for system UI and benchmarking
references/gradle-managed-devices.md	Managed device setup and CI
references/screenshot-testing.md	Visual regression testing
references/adaptive-screen-testing.md	Screen-size and foldable coverage
references/accessibility-checks.md	Accessibility checks for Espresso and Compose
references/test-orchestrator-patterns.md	AndroidX Test Orchestrator patterns
references/android-ci-optimization.md	CI pipeline optimization
references/modern-test-tooling.md	JUnit 5, MockK, Turbine, Robolectric, Maestro
data/sources.json	Curated external sources

Templates

Template	Purpose
assets/template-android-test-checklist.md	Stability checklist

Related Skills

Skill	Purpose
software-mobile	Android development
qa-testing-strategy	Test strategy
qa-testing-mobile	Cross-platform mobile
software-android-native	Native Android implementation and agent workflows
software-android-runtime-debugging	Build/install/launch proof and stale-build triage
software-android-design	Native Android visual design and Material 3 review

Learnings Loop

When prior decisions or pitfalls are relevant, consult learnings.consolidated.md if present; use learnings.md only for needed history or as the available fallback. Otherwise skip both.

After applying it, if you encountered a pattern worth remembering, a mistake worth preventing, or a domain fact that surprised you, append one dated bullet to learnings.md via agents-skills-feedback-loop/scripts/append_learning.py. Do not modify SKILL.md itself.