Enterprise system tests are resource intensive. So much so that a single test suite takes over an entire GCP project while running. Assuming we manage to build our entire test suite such that the properties listed under the TEST section are met, then it would be possible for multiple test suites to run at the same time in a single GCP project.
In practice, this isn‘t perfect since there’s too much room for two tests to interact with each other. For example, a change made to an IIS Web Application can affect a test suite from a prior revision that's running concurrently. Fortunately, such changes should be infrequent and the lab should be able to parallelize multiple test session as long as their ASSET MANIFESTs are the same.
When a change is detected to the ASSET MANIFEST, the GREETER can request all other test suites to be drained so that the ambient asset definitions can be rotated to the new version. Once this is done, all new test runs based on the new asset definition can once again be parallelized.
This process is a bit complicated and is something we can implement as we move further along in the implementation. It‘s not something that’s being considered for the initial deployment.
A more reliable (less flaky or complicated) means of scaling would be to allow multiple deployments of the lab, each into its own GCP project. A GREETER from each instance could be registered with Luci Isolate effectively providing load balancing for enterprise tests.
The number of such labs that can be integrated with the waterfall is limited by the availability of physical resources as described in On-Premise Fixtures. Each instance of the lab must have a set of identically configured physical devices for tests to be reliable.
