Clone this repo:
  1. 5c0b137 Make BQWrite accept a write disposition by Katie Thomas · 3 months ago deployed master
  2. 63f7c1c Add README to packages/dataflow/common by Katie Thomas · 3 months ago
  3. c59b89b Add more fields to cq_attempts by Katie Thomas · 3 months ago
  4. f0614eb Fix bug in cq_attempts by Katie Thomas · 3 months ago
  5. ba8ccb2 Bump dataflow package version by Katie Dillon · 3 months ago

Workflow Testing

To test that your pipeline will run remotely:

python <path-to-dataflow-job> --job_name <pick-a-job-name> \
--project chrome-infra-events --runner DataflowRunner --setup_file \
<infra-checkout-path>/packages/dataflow/setup.py \
--staging_location gs://dataflow-chrome-infra-events/staging --temp_location \
gs://dataflow-chrome-infra-events/temp --save_main_session

Job names should match the regular expression [a-z]([-a-z0-9]{0,38}[a-z0-9]). Navigate to the Dataflow console in your browser (project: chrome-infra-events) and you should see your job running. Wait until it succeeds.