Test Data and Flakiness

Test data is generated, synthetic, or approved and masked. Tests must not depend on execution order or shared mutable records. Unique run identifiers prevent collisions across parallel jobs.

A flaky test is a defect. Quarantine is allowed only with an owner, linked issue, expiry, and continued visibility. Retries may gather diagnostics but must not turn repeated instability into a green release signal.

Track failure rate, quarantine age, runtime, and top unstable tests. Teams fix root causes such as asynchronous races, eventual consistency assumptions, weak cleanup, and environment saturation.