This document gives you some idea of where to find which code, and also a high level overview of how we've organized the code.
The toolchain is written primarily in Go. Hence the organization of the directories follows what's required by Go.
go: All Go sources, test files, immediate dependencies of tests.
schema: ProtoBuf schema in the form of .proto files. These are used to generate Go code stubs that are then placed under the go/ tree.
docs: Documentation.
examples: Examples. Not used by the build, but are there for your perusal.
resources: Resources that are embedded into the toolchain.
build: Code used by build.py.
scripts: Example and miscellaneous scripts that are not directly used by the build system nor the toolchain.
In addition to the above, at after a building you'll find a few additional directories as follows:
out: Directory containing build artifacts.
vendor: Directory containing external dependencies that were pulled down by the Go Dep tool.
third_party: Additional dependencies that are not managed by the Go toolchain.
The entire purpose of the codebase is to produce two binaries and their supporting documentation.
The two binaries are:
cel_ctl: The commandline control utility. This is the primary (at the moment) interface for users to interact with the toolchain.
cel_agent: This is the agent binary that is installed and run on every VM that's created within the lab. Multiple of these may be built during a single build to account for the different architectures of the VMs.
Assets vs Host Environment: These two concepts are described in detail in ASSET MANIFEST and HOST ENVIRONMENT respectively. In code we strive to maintain a package-wise separation between the two.
The toolchain currently only supports Google Cloud Platform as a backend for hosting labs. This may change in the near future (already changed and the documentation is out-of-date? Let us know! File a bug here). For mobility in this space and for general code health, we want to maintain a separation between the ASSET MANIFEST and the HOST ENVIRONMENT code.
What does “package-wise separation” mean?
Package-wise separation means that the code pertaining to HOST ENVIRONMENT must need to access code pertainng to ASSET MANIFEST. Since Go imports can't be cyclical, this implies that code pertaining to ASSET MANIFEST must necessarily be independent of the HOST ENVIRONMENT. This way the Go toolchain will help us maintain code health.
Deployer vs Agent: The deployer is the primary task of the cel_ctl tool. The Agent is primarily the cel_agent binary. Code that‘s specific to one should be kept in separate Go packages from code that’s specific to the other.
In code, we maintain deployer/agent separation via subdirectories named deploy and agent.
GCP vs Not-GCP: The goal here is to isolate GCP code into its own packages such that only those packages can import GCP API code. This, once again, is for maintaining code health by preventing unnecessary dependencies on GCP by packages that shouldn't.
In code, this manifests itself as the go/gcp directory which contains all the GCP specific code.
The subdirectories under the top level go/ directory are:
cmd: Commands.
cmd/cel_ctl: The CEL controller tool. This directory should only contain the logic needed to understand commandline options. Code in the contained main package can import any other Go package other than those under tools, and other subdirectories of cmd.
One of the primary duties of the cel_ctl command is to manage the deployement of assets into a lab. The code for doing that is in go/cel/deploy as described below.
cmd/cel_agent: The CEL agent tool. This directory should only contain the logic needed to understand commandline options. Code in the contained main package can import any other Go package other than those under tools, and other subdirectories of cmd.
common: Common logic that's shared between many of the components of the CEL toolchain. There are deploy and agent subdirectories under here for deployer and agent specific code.
cel: High level workflow. Once again deploy and agent subdirectories further separate out the workflow logic.
asset: Asset Manifest specific code.
host: Host Environment specific code.
gcp: Go code that depends directly on Google Cloud Platform APIs and abstractions. Subdirectories therein have generated code for dealing with GCP API specific schemas (e.g.: go/gcp/compute contains generate code for managing data structures needed to talk to the Google Compute Engine API.
There are deploy and agent subdirectories under here for deployer and agent specific code.
tools: These tools are indirect dependencies for cel_ctl and cel_agent.
