Buildbot master authentication to Cloud Endpoints

TL;DR Each master may have a service account assigned to it. Two modules auth.py and deferred_resource.py together can be used to make authenticated Cloud Endpoints API calls.

Master service accounts

Each master may have a service account assigned:

# master_site_config.py
class MyMaster(Master3):
  # other stuff
  service_account_file = 'service-account-myproject.json'

Service account files should always be named “service-account-[shortname].json”. If service_account_file attribute is set, service_account_path attribute is resolved to [CREDENTIALS_DIR]/[service_account_file]. In order to request a service account for your master, file a bug.

One service account can be used to authorize calls to different GAE apps.

Making authenticated API calls

TL;DR Use deferred_resource.py to generate an API client for Twisted code at runtime. Use auth.py to sign it.

from master import auth
from master import deferred_resource

MY_SERVICE_HOSTNAME = 'my_service.appspot.com'
MY_SERVICE_DISCOVERY_URL = (
  '%s/_ah/api/discovery/v1/apis/{api}/{apiVersion}/rest' %
  MY_SERVICE_HOSTNAME
)

@defer.inlineCallbacks
def greet(active_master):
  # active_master is master configuration,
  # what is normally called ActiveMaster in master.cfg

  # Create a signed httplib.Http2 factory.
  http_factory = lambda: auth.create_http(active_master)

  # Create API client for Twisted.
  my_service = yield deferred_resource.DeferredResource.build(
      'my_service',
      'v1',
      http_factory=http_factory,
      discoveryServiceUrl=MY_SERVICE_DISCOVERY_URL)

  # Call API.
  res = yield my_service.api.greet('John', body={'message': 'hi'})

Using gsutil.py from a recipe

api.gsutil also requires authentication through a service account, however this is managed differently. To use api.gsutil do the following:

1. Add mdb.chrome-troopers@google.com as an editor on the cloud project associated with the cloud storage bucket you wish to upload to.
2. File an issue at http://crbug.com/ and add the Infra-Labs label. Mention your the account owning the cloud storage bucket as well as the master/slaves you wish to have authenticated.
3. Infra Labs will create the necessary service accounts and add them to the slaves in question. They will also be able to roll credentials when necessary.

Creating a service account - guide for admins

See Pupper service accounts