Build an iOS Scroll Performance Regression Gate with XCTMetric

After adding a gradient layer, image decoding, or complex shadows to a product list, all functional tests may still pass while scrolling starts dropping frames. Dragging through the list a few times by hand makes it difficult to tell whether the issue is a one-off stutter or a code regression. A more reliable approach is to standardize the simulator, dataset, and gestures on a cloud Mac, use XCTMetric to repeatedly measure duration, CPU, and memory, and configure continuous integration to block only changes that exceed the baseline.

Control the Variables That Affect Results

The first step in building a performance gate is not writing assertions, but reducing environmental noise. Pin the Xcode and macOS versions, simulator model, system language, display orientation, and test data. The test account should open the target screen directly so that login requests, image downloads, and server response times do not contaminate the scrolling metrics.

Add a launch argument that is enabled only for UI tests, such as -UITestSeed fixed. It should load local fixtures, generate a fixed number of list items in a deterministic order, and disable random animations. The list also needs stable accessibility identifiers such as feed.list and feed.reset; test code should not locate elements by localized text.

The first run usually includes app installation, dynamic library loading, font initialization, and cache warm-up. Run one unmeasured test before collecting production samples, but do not delete caches to create an artificial “completely cold start” environment that does not reflect normal usage.

A performance baseline describes a complete environment, not a universal millisecond value that can be applied to every Mac and every simulator.

Create a Repeatable Scrolling Path

A single swipeUp() is too sensitive to the starting point and list length. A more reliable test should first return to the top, verify that the first anchor is visible, and then perform a fixed number of gestures. Reset the list after each measurement iteration so that the next iteration does not begin at the bottom.

import XCTest

final class FeedScrollPerformanceTests: XCTestCase {
    func testFeedScrollPerformance() {
        let app = XCUIApplication()
        app.launchArguments += ["-UITestSeed", "fixed"]
        app.launch()

        let list = app.collectionViews["feed.list"]
        XCTAssertTrue(list.waitForExistence(timeout: 10))

        app.buttons["feed.reset"].tap()
        XCTAssertTrue(app.cells["feed.item.0"].waitForExistence(timeout: 5))

        let options = XCTMeasureOptions()
        options.iterationCount = 5

        measure(
            metrics: [XCTClockMetric(), XCTCPUMetric(), XCTMemoryMetric()],
            options: options
        ) {
            app.buttons["feed.reset"].tap()
            for _ in 0..<6 {
                list.swipeUp(velocity: .fast)
            }
        }
    }
}

The reset button may appear only in test mode, but it must perform deterministic scrolling rather than fetch the data again. If the screen uses paginated loading, replace the network layer with local responses or measure pagination latency in a separate test.

Run on a Fixed Destination and Archive the Results

First list the available simulators to ensure that continuous integration has not silently switched devices. Then specify a unique result bundle path for the test. Any existing result bundle must be removed beforehand because xcodebuild will refuse to overwrite it.

set -euo pipefail

RESULT="$PWD/artifacts/FeedScroll.xcresult"
rm -rf "$RESULT"
mkdir -p "$PWD/artifacts"

xcodebuild test \
  -workspace App.xcworkspace \
  -scheme AppUITests \
  -destination 'platform=iOS Simulator,name=iPhone 16,OS=latest' \
  -only-testing:AppUITests/FeedScrollPerformanceTests \
  -resultBundlePath "$RESULT"

OS=latest works well for pipelines that always track the current toolchain. For long-term comparisons, pin a specific runtime and rebuild the baseline after upgrading Xcode or the operating system. Jobs running on OVPS nodes should also record the host configuration, commit hash, Xcode version, and simulator runtime so that environment upgrades are not mistaken for code regressions.

Before using the xcresulttool bundled with the current Xcode version, check its help output because subcommands may change between toolchains. Whether results are processed with a script or a test report parser, the original xcresult should be retained as an attachment when a test fails.

Evaluate Results with Medians and Relative Thresholds

Do not block a merge because of a single slow sample. Run five to seven iterations for each commit, discard the warm-up iteration, and then calculate the median. The baseline can be the median of several recent stable runs on the main branch rather than the first measurement preserved indefinitely.

Monitor three categories of change separately. Total duration reflects the user’s wait, CPU metrics can reveal repeated layout work or excessive drawing, and memory metrics can expose problems with image caches and view reuse. If a commit increases duration while CPU and memory remain stable, check simulator load first. A simultaneous deterioration in all three metrics is more likely to indicate an application code regression.

Use proportional thresholds together with a minimum absolute difference. For example, fail only when the median duration rises by more than 12% over the baseline and the absolute increase exceeds a team-defined noise floor. Threshold values must be based on historical variation in the same environment rather than copied from another project.

Preserve Evidence and Narrow the Scope After a Failure

When the gate fails, do not overwrite the baseline immediately. First rerun the same commit once. If it still fails, preserve the xcresult, complete logs, test data version, operating system and toolchain versions, and summaries of CPU, memory, and duration. Then narrow down the commit range and focus on image decoding, shadows and blurs, Auto Layout constraints, synchronous I/O on the main thread, list reuse, and logging.

It is also important to distinguish metric degradation from a change in the test path. If an accessibility identifier no longer works or the reset action does not return to the top, the test may still run while measuring a different part of the screen. Adding visibility assertions for critical starting and ending points is more effective than simply increasing the retry count.

Baseline updates should be submitted as separate changes that explain the environment upgrade or acceptable interaction adjustment and include samples from before and after the update. This keeps the performance gate from being triggered repeatedly by incidental noise without allowing casual baseline refreshes to render it ineffective.

Frequently asked questions