breadcrumbs: Chromium Commit Queue > page_name: integration-with-rietveld title: ‘Design: 3-way Integration with Rietveld and the Try Server’
Drastically reduce the perceived time (wall-clock time) from asking the infrastructure to commit a change list (CL) to the time it is committed. This reduces the time a developer has to care about a particular CL, by getting it in or refusing it as fast as possible.
The Chromium team uses the Commit Queue, which in turn uses the Try Server, to perform automated testing of their pending CL on multiple platforms in multiple configurations. Rietveld is used for code reviews and the Chromium team uses a slightly forked version to better integrate it with the rest of the Chromium infrastructure.
Before the integration, the Commit Queue is awaken as soon as the CL author, or one of his reviewer, checks the infamous Commit checkbox on Rietveld. The Commit Queue polls updates from Rietveld at a rate of 0.1hz to get new CLs to test.
This means that pre-commit verifications, which include code styling, OWNERS check, a proper LGTM from at least one Chromium Committer, properly testing the patches on the Try Server, only starts at the moment the checkbox is set. This means a maximal time is spent between the moment the checkbox is checked and the change is committed.
Also, before the integration, because of the way try jobs are triggered, the Commit Queue cannot trust the try jobs triggered by anyone but itself. The “previous way” of triggering try jobs is to commit a raw diff into subversion. Let's just say that it is suboptimal.
This project is about implementing techniques in the integration so the Commit Queue can efficiently run presubmit checks and Try Jobs on the Try Server before the CL author request the commit to be merged into the source tree;
The Commit Queue is also trusting the try jobs triggered by others than itself. The new way involves the Try Server polling Rietveld directly to fetch the diff directly from the code review. This is the reason the try jobs can be “trusted”.
Rietveld acts as a datastore to map a Verification steps to a Patchset.
The infrastructure consist of:
The first task is to make sure Try Jobs results on Rietveld can be trusted. This is done by differentiating try jobs triggered directly from Rietveld from any other try jobs. The polling code is in try_job_rietveld.py and sets the build property try_job_key
that is then used to certify this try jobs was indeed triggered from Rietveld in view_chromium.py. The json packets are pushed via Buildbot‘s HttpStatusPush implementation from the Try Server back into Rietveld’s /status_listener URL as HTTP POST requests.
Rietveld already has a mechanism to search for issues with specific criteria. For example,
While polling it really not optimal, it permits reusing the current API. The /search
interface supports specifying an opaque datastore cursor
so only new issues are returned. The cursor to use on the next request is returned inside the json data.
For each new CL modified, the CQ must decide if it reacts to the CL. To enable preemptive CL verification, the following steps are used on each new CL;
Then for commit time;
This project is driven primary by rogerta@ and maruel@. It benefits immediately from Try Server enhancement projects like the Swarm integration project but it is still designed to work without Try Server performance boost and work around its general sluggishness.
The whole purpose of this project is to reduce the latency of the pre-commit checks. The latency will boils down to
At that point, we are not aiming at optimizing the commit delay below 30 seconds, lower than that is a nice to have.
The Commit Queue is a single threaded process. As such, it is bound by its slowest operations. This currently include the PRESUBMIT checks and applying/commiting CLs. The goal is to make checks happen earlier and reuse the results from previous checks whenever they are trusted, not to make the checks faster.
There are multiple single points of failure;
The security is using the following assumptions;
The integration itself, being an integration project, is mostly tested directly “on prod”. While there are a few unit tests to check basic functionality, most of the bugs are at integration points. This being a productivity tool for developers, it is expected that they have some clue about what's going wrong.