The Chromium project utilizes release block labels to help define each milestone's critical path. The following labels are available:
Release block labels must be used in conjunction with a milestone (M=##) label as well as an OS (e.g. OS-Android). The combination of blocking label, milestone, and OS determine which releases we hold; e.g. OS=Android M=59 ReleaseBlock-Beta means we will not ship M59 Android builds to either the beta or stable channel until the bug is addressed. Android M59 dev releases, or releases on any other platform, would not be blocked.
Because these labels are used to manage critical path, they should not be used unless we genuinely believe we cannot ship to a given channel with the issue present. Do not mark an issue as a blocker “to ensure someone looks at it.” Priority and milestone labels should be used for this purpose.
These rules apply to bugs and regressions; teams may use criteria for blocking due to other work (e.g. tasks) as they see fit.
Issues should be evaluated for release blocking status using the following matrix based on the issue's severity and prevalence:
| Low Impact | Medium | High | Critical | |
|---|---|---|---|---|
| Few Users | ReleaseBlock-Stable | ReleaseBlock-Dev | ||
| Some | ReleaseBlock-Stable | ReleaseBlock-Beta | ReleaseBlock-Dev | |
| Most | ReleaseBlock-Stable | ReleaseBlock-Beta | ReleaseBlock-Beta | ReleaseBlock-Dev |
| All | ReleaseBlock-Stable | ReleaseBlock-Beta | ReleaseBlock-Dev | ReleaseBlock-Dev |
Severity is defined as the impact to a user who experiences the bug.
Prevalence is defined as the volume of users who will experience the bug.
Note that prevalence should be evaluated based on the population of users they affect - e.g. a bug affecting all Android users (but not Windows users) would still be considered to affect all users, and a bug affecting all Enterprise Windows users (but not all consumer Windows users) could also be considered to affect all users.
In practice, the data available for assessing severity and prevalence of bugs is usually imperfect, so best judgement and rules of thumb need to be employed.
Engineers assessing bugs can use their knowledge of the underlying system to intuit whether the observed symptom might be the “tip of an iceberg” of a wider bug which might have much wider severity and prevalence. The evaluation isn't required to be limited to the so-far exactly observed symptoms, but should also be biased upward on the basis of well-founded fears. For example, scary race conditions or symptoms that indicate a core system with many dependencies is being undermined.
A rule of thumb is that such scary “iceberg” problems are more likely for changes which have not yet been exposed to a large population of users -- especially, bugs in Dev channel, or bugs in Beta channel affecting a relatively small or quiet population (for example, Android WebView has a tiny beta population, and non-English-speaking users have more difficulty getting their feedback heard). On the other hand, if a bug already has been present on 100% of stable channel for weeks or months before it was first noticed, that's evidence that the problem is not so scary or urgent after all. Therefore, recent regressions should have an upward bias in severity/prevalence assessment, while nonrecent ones should have a downward one.
If this sort of consideration is a factor, that should be explicitly mentioned in the bug update.
The definitions provided above are examples; teams are encouraged to customize where it makes sense, e.g. the web platform team may consider developer impact for severity and feature usage for prevalence.
Everyone should feel free to add, modify, or remove release blocking labels where appropriate, so long as you follow the guidelines below. If a TPM or test engineer has marked a bug as a release blocker, but a developer knows for sure that the issue should not block the release, the developer should remove the release blocking label; similarly anyone should feel free to add a release blocking label to a bug they feel warrants holding a release. That said, there are some general guidelines to follow:
Any bugs related to a feature that is behind a flag and is not enabled by default should never block a release (they should be disabled instead). Features enabled by default should follow the proposals listed above.
Regressions should follow the same guidelines as listed above; an issue should not be tagged as a release blocker simply because it is a regression. While we'd like to prevent regressions in general, there is a large backlog of bugs we need to address, and we should focus on the most important. To ensure we maintain a high bar for product quality, we should track the number of introduced versus escaped regressions, and follow up if the number starts to rise.
In practice, it is still expected that the majority of release blockers filed will be recent regressions, because on average they have higher severity, prevalence and uncertainty than longstanding bugs.