| commit | 6d7644871eb4da2d6a50db13cb3f57e82f21088d | [log] [tgz] |
|---|---|---|
| author | Katie Thomas <katthomas@google.com> | Wed Jan 03 13:40:22 2018 |
| committer | Commit Bot <commit-bot@chromium.org> | Wed Jan 03 17:48:17 2018 |
| tree | 475cc34d91494957c07d7caf912145cf56e53d47 | |
| parent | 894fe71f159591b068e558ad0e22e567604f00f3 [diff] |
[packages/dataflow] remove version from setup.py We don't do releases, so this doesn't make sense. Bug:798439 Change-Id: I9ed746f77f0672dd663f09e50c886ecf1c54404b Reviewed-on: https://chromium-review.googlesource.com/848742 Reviewed-by: Aaron Gable <agable@chromium.org> Commit-Queue: Katie Thomas <katthomas@google.com> Cr-Mirrored-From: https://chromium.googlesource.com/infra/infra Cr-Mirrored-Commit: fa2f32ca126b84e711400e41d2a74d64d8913837
To test that your pipeline will run remotely, you can use the command below.
You must first create Google Storage buckets to pass with the --staging_location and --temp_location options. The name is not important, but for example you could use gs://my-dataflow-job/staging.
python <path-to-dataflow-job> --job_name <pick-a-job-name> \ --project <project> --runner DataflowRunner \ --setup_file <infra-checkout-path>/packages/dataflow/setup.py \ --staging_location <staging bucket> \ --temp_location <temp bucket> --save_main_session
Job names should match the regular expression [a-z]([-a-z0-9]{0,38}[a-z0-9]). Navigate to the Dataflow console for your project in your browser and you should see your job running. Wait until it succeeds.