Benchmarking a custom Android emulator on your desk

Setting up an Android emulator with a custom ROM for performance benchmarks is more than choosing a wallpaper and preinstalled apps. A custom build can alter the framework, system services, graphics drivers and build configuration, all of which can influence startup time, memory use and frame pacing. A useful benchmark therefore measures the complete build rather than treating the emulator as an interchangeable test device.

The goal is not simply to make the image boot. The goal is to create a reproducible test bed where one variable, usually the ROM, can be compared with a known baseline. That means recording the host machine, virtual hardware profile, workload, warm-up policy and collection tools. It also means accepting that emulator numbers describe software execution through virtualisation, not the speed of a physical handset.

For Australian developers, this setup works well when the rig is treated like a dedicated lab bench. Local developer meetups in Sydney, Melbourne, Brisbane and Perth give teams an easy way to exchange build notes, and a concise configuration file is easier to share than a room full of screenshots. Australian hardware listings normally include GST in the advertised AUD price, so the cost of another SSD or more RAM should be included when deciding whether the machine is viable.

Start with a stock system image of the same API level, then build the custom image against an explicitly documented toolchain. Measure cold boot and warmed application paths separately, and publish the raw metrics as well as the averages. The following method keeps the exercise practical, repeatable and honest about what the figures represent.

Define a comparable benchmark

Choose a repeatable workload before touching the ROM. A representative screen or workflow is usually more useful than a synthetic score, especially when the custom build changes framework code. Use the same APK, account state, input sequence and network conditions for every run. Record launch latency, frame timing, CPU time and resident memory as primary measures.

The host rig needs its own baseline. Record the processor model, enabled cores, RAM, storage type, operating system version and whether hardware virtualisation is available. Keep the same Android Virtual Device (AVD) profile, core allocation and power settings for the stock and custom images. An upgrade mid-series invalidates the comparison, even if the new machine feels faster.

Give each image a traceable identity. Record the source commit, build date, release tag, target API, ABI and a checksum for every image used. Include the output of adb shell getprop for the relevant properties, because a matching application can still run against different framework and vendor configurations.

Build an emulator-compatible ROM

Only an emulator-compatible build belongs in this workflow. Start with an AOSP-derived source tree and produce the architecture accepted by the emulator and host—commonly x86_64 on Intel or AMD hosts and arm64-v8a on Apple silicon—along with matching system, vendor and product properties. A ROM compiled only for the opposite architecture may boot on some physical Android devices yet fail inside the emulator because its native libraries do not match the virtual CPU.

Treat the source tree as a controlled environment. The practical lesson from an indoor herb garden guide is to change one condition at a time and record the result. Apply that discipline to the ROM: alter one subsystem, rebuild from a known commit, and retain the previous image for comparison rather than overwriting it.

Package the output in the layout expected by the emulator, including the system image, compatible kernel and ramdisk, vendor files and any required firmware. Install or register it as a custom system image, then create an AVD with the same device profile used for the stock baseline. Before benchmarking, check adb shell getprop ro.build.fingerprint and adb shell getprop ro.product.cpu.abi to confirm that the emulator booted the intended build.

Configure the emulator for stable runs

Create two clearly labelled workflows: a clean run and a warm run. Disable automatic snapshot loading for cold measurements so stale processes and caches cannot masquerade as startup performance. Start the cold-boot clock when the emulator process launches and stop it when sys.boot_completed becomes 1. If snapshot state is intentionally studied, save it under a separate name and record whether it was restored before the application launch.

The emulator command line should make the policy obvious. A typical cold pass uses options such as -no-snapshot-load, -wipe-data and -no-boot-anim, while a warmed test restores the declared snapshot only after boot completes. Headless runs may also use -no-window and a software renderer, but that creates a different graphics scenario from host GPU acceleration and must be labelled as such.

Environmental control matters as much as emulator flags. Keep the laptop on a hard surface, retain the same cooling path and avoid background downloads during measurement. Brisbane developers should be especially careful with hot days, since a throttled laptop can erase a small ROM gain before the application starts. Across Australia’s widely separated time zones, a shared benchmark window also prevents teams in Perth and Sydney from running competing background workloads at the same moment.

Automate workloads and capture evidence

Script the user journey rather than driving it by hand. The harness should create a fresh profile or reset the known account state, install the same APK, clear logs, launch the target screen, wait for a stable frame and then collect results. Run each scenario in both cold and warm modes, and reset animation scales to documented values before and after the suite.

Keep the benchmark runner separate from any local backend used by the Android workload. For a harness built around a Ruby service, the process-management approach in these Puma migration notes offers a useful parallel: make startup, health checks and termination explicit, then let the runner record timing boundaries without embedding policy inside the workload.

Use several tools because no single metric explains performance. adb shell am start -W captures launch components; dumpsys gfxinfo exposes legacy frame statistics; and Perfetto traces can show scheduler, graphics and system-service delays on supported images. Pair those traces with dumpsys meminfo and dumpsys cpuinfo. Store raw files with timestamps so averages cannot hide a delayed background job or one unusually slow launch.

Compare runs without chasing noise

Repeat each condition enough times to expose variance. Five clean starts and twenty warmed launches are practical minimums for an initial comparison, although heavier workloads may need more samples. Discard a run only when a documented setup failure occurred; do not silently remove the slowest result. Report the median, range and 95th percentile, then annotate thermal throttling, background activity or a changed input state.

The comparison matrix separates startup modes and build variants so their results are not blended.

Configuration	Measurement	Primary use	Main limitation
Stock AOSP image, clean boot	Cold boot and first launch	Establishes the framework baseline	Contains no custom ROM changes
Custom ROM image, clean boot	Cold boot and first launch	Directly tests startup effects	Has greater run-to-run variance
Stock AOSP image, warmed state	Repeated application launch	Establishes steady-state behaviour	Conceals cold-start costs
Custom ROM image, warmed state	Repeated application launch	Measures custom steady-state behaviour	Can hide startup regressions
Same journey on both images	End-to-end workflow	Confirms practical consistency	Depends on identical account and network state

Read the comparison by workload, not by the single fastest number. A custom ROM may trim boot time yet worsen sustained rendering, or improve a service-heavy screen while leaving package installation untouched. Profile the affected subsystem before promoting a build. The final report should also state that virtual CPU scheduling and graphics translation can amplify differences that would behave differently on physical Android hardware.

Publish the evidence and keep testing

Turn the setup into a reproducible artefact rather than a folder of unexplained binaries. Publish a small manifest containing the source commit, toolchain version, build flags, image checksum, AVD configuration and exact test commands. Keep logs and traces in a compressed archive, then add a short explanation of every metric and any known emulator limitation.

Australian teams can use the material at a local developer meetup, an internal engineering forum or a shared Git repository. Clear terminology matters across a geographically distributed group: use Australian English such as “optimisation” and “customisation”, state whether timings use wall clock or monotonic time, and identify the timezone for scheduled runs. The result should be understandable by another engineer, not just reproducible on the original desk.

Build the stock baseline today, preserve its image and raw results, then create the custom ROM with one deliberate optimisation at a time. Publish the configuration and measurements beside a reproducible script, and invite other Australian developers to run the same protocol on their own rigs.