This page describes the current status and how to migrate code.
crbug.com/941669 tracks the overall migration.
It is now safe to write new code using Python3 as long as it isn't called from other Python2 files. See the Python style guide for the latest on this.
As of the time of writing (2021-07-19), we're in the following state:
This document will not attempt to replicate the general information in the Python.org Porting HOWTO or the many, many other guides on the Internet.
However, here are a few notes that may be helpful in a Chromium context:
Most of our Python code is pretty straightforward and moves easily from Python2 to Python3, so don't stress out about this!
When migrating code, please make the Right Changes, rather than the minimum needed to get things to work. For example, make sure your code is clear about whether something should be a byte string or a unicode string, rather than trying to handle both “to be compatible”. (If you really do need the code to handle both, though, that can be okay.)
Do not assume you can use vpython regardless of context! vpython has performance issues in some situations and so we don't want to use it yet for things invoked by gclient, PRESUBMITs, or Ninja. However, all tests are run under vpython and so you can assume it there.
Some people find the 2to3 tool to be useful to partially or completely automate the migration of existing files, and the six module to shim compatibility across the two. The six module is available in vpython and in //third_party/six/src/six.py.
shebang lines mostly don‘t matter. A “shebang line” is the line at the beginning of many unix scripts, like #!/usr/bin/env python. They are not that important for us because most of our python invocations come from tools like Ninja or Swarming and invoke python directly. So, while you should keep them accurate where they are useful, we can’t rely on them to tell which code has been migrated and which hasn't.
The major gotchas for us tend to have to do with processing output from a subprocess (e.g., subprocess.check_output()). By default output is returned as a binary string, so get in the habit of calling .check_output().decode('utf-8') instead. This is compatible across 2 and 3. ‘utf-8’ is the default in Python3 (ASCII was the default in Python2), but being explicitly is probably a good idea until we have migrated everything.
Be aware that filter and map return generators in Python3, which are one-shot objects. If you reference them inside another loop, e.g.,
foo = [ ... ]
bar = filter(some_function, [ ...])
for x in foo:
for y in bar:
do_something(x, y)
this won't work right, because on the second and subsequent iterations, bar will be an empty list.
Best practice is to use a list comprehension instead of map or filter, but you can also explicitly cast the results of map or filter to a list if the list comprehension is too awkward.
Some modules (like urllib2) were renamed and/or moved around in Python 3. A Google search will usually quickly tell you the new location.
Watch out for places where you reference basestring or unicode directly. six provides some compatibility types to help here.
Generally speaking, test your changes the same way we do everything else: make a change locally, and rely on the CQ and CI bots to catch problems.
However, here are some specific guidelines for the different contexts where we use Python:
To switch a gclient hook from Python2 to Python3, simply change python to python3 in the DEPS file, and make sure things still run :).
All targets in the build use Python3 now, and anything declared via an action() rule in GN will use Python3.
Test targets that run by invoking python scripts (like telemetry_unittests or blink_web_tests) should eventually migrate to using the script_test GN templates. Once you do that, they will use Python3 by default. However, some tests may specify run_under_python2 = true as a template variable to use Python2, so when you're ready to test Python3, just delete that line.
Some tests still need to be migrated to script_test() (crbug.com/1208648). The process for doing that is not yet well documented, so ask on python@chromium.org (or ask dpranke@ directly).
There is no general mechanism for migrating tests that are C++ executables that launch python via subprocesses, so you're on your own for dealing with that.
Presubmit checks are run using Python 2 by default. To run them using Python3, add the line USE_PYTHON3 = True to the PRESUBMIT.py file in question (effectively creating a global variable).