Doc/whatsnew/3.16.rst

****************************
  What's new in Python 3.16
****************************

:Editor: TBD

.. Rules for maintenance:

   * Anyone can add text to this document.  Do not spend very much time
   on the wording of your changes, because your text will probably
   get rewritten to some degree.

   * The maintainer will go through Misc/NEWS periodically and add
   changes; it's therefore more important to add your changes to
   Misc/NEWS than to this file.

   * This is not a complete list of every single change; completeness
   is the purpose of Misc/NEWS.  Some changes I consider too small
   or esoteric to include.  If such a change is added to the text,
   I'll just remove it.  (This is another reason you shouldn't spend
   too much time on writing your addition.)

   * If you want to draw your new text to the attention of the
   maintainer, add 'XXX' to the beginning of the paragraph or
   section.

   * It's OK to just add a fragmentary note about a change.  For
   example: "XXX Describe the transmogrify() function added to the
   socket module."  The maintainer will research the change and
   write the necessary text.

   * You can comment out your additions if you like, but it's not
   necessary (especially when a final release is some months away).

   * Credit the author of a patch or bugfix.   Just the name is
   sufficient; the e-mail address isn't necessary.

   * It's helpful to add the issue number as a comment:

   XXX Describe the transmogrify() function added to the socket
   module.
   (Contributed by P.Y. Developer in :gh:`12345`.)

   This saves the maintainer the effort of going through the VCS log
   when researching a change.

This article explains the new features in Python 3.16, compared to 3.15.

For full details, see the :ref:`changelog <changelog>`.

.. note::

   Prerelease users should be aware that this document is currently in draft
   form. It will be updated substantially as Python 3.16 moves towards release,
   so it's worth checking back even after reading earlier versions.


Summary --- release highlights
==============================

.. This section singles out the most important changes in Python 3.16.
   Brevity is key.


.. PEP-sized items next.



New features
============



Other language changes
======================

* :meth:`memoryview.cast` now allows casting a multidimensional
  F-contiguous view to a one-dimensional view.
  (Contributed by Jaemin Park in :gh:`91484`.)

* :meth:`memoryview.cast` now accepts an *order* parameter.  With
  ``order='F'`` it returns a zero-copy Fortran-contiguous (column-major)
  view of a flat buffer, which is useful for interfacing with column-major
  libraries.
  (Contributed by Serhiy Storchaka in :gh:`78959`.)

* :ref:`Frame objects <frame-objects>` now support :mod:`weak references
  <weakref>`.  This allows associating extra data with active frames,
  for example in debuggers, without keeping the frames (and everything
  they reference) alive indefinitely.
  (Contributed by Łukasz Langa in :gh:`102960`.)


New modules
===========

* None yet.


Improved modules
================

asyncio
-------

* Add the *mode* parameter to :meth:`asyncio.loop.create_unix_server` and
  :func:`asyncio.start_unix_server` to set the permissions of the Unix
  socket file created for *path*.
  (Contributed by Sam Bull in :gh:`94984`.)


codecs
------

* On platforms that provide the C library's :manpage:`iconv(3)` function,
  every encoding known to ``iconv`` for which Python has no built-in codec
  is now available (for example ``cp1133``).
  Prefixing an encoding name with ``iconv:`` forces the ``iconv``-based codec
  even when a built-in codec of the same name exists.
  (Contributed by Serhiy Storchaka in :gh:`152997`.)

csv
---

* :meth:`csv.Sniffer.sniff` now deduces the dialect by trial parsing
  instead of heuristics based on matching isolated fragments
  and on character frequencies,
  so the result is consistent with how :func:`csv.reader` will parse the sample.
  It can now detect the *escapechar* parameter,
  accepts non-ASCII delimiters in the *delimiters* argument,
  no longer misdetects the delimiter
  when the sample contains delimiter characters inside quoted fields,
  and no longer takes quadratic time on quoted samples.
  The results may differ from those of earlier Python versions.
  (Contributed by Serhiy Storchaka in :gh:`83273`.)

* :meth:`csv.Sniffer.sniff` now detects the *lineterminator* parameter
  by a majority vote among the line endings of the sample,
  instead of always returning ``'\r\n'``.
  (Contributed by Serhiy Storchaka in :gh:`75008`.)

curses
------

* The :mod:`curses` character-cell window methods now accept a full character
  cell --- a spacing character optionally followed by combining characters ---
  in addition to a single integer or byte character.  This affects
  :meth:`~curses.window.addch`, :meth:`~curses.window.bkgd`,
  :meth:`~curses.window.bkgdset`, :meth:`~curses.window.border`,
  :meth:`~curses.window.box`, :meth:`~curses.window.echochar`,
  :meth:`~curses.window.hline`, :meth:`~curses.window.insch` and
  :meth:`~curses.window.vline`.
  Also add the wide-character read methods :meth:`~curses.window.get_wstr` and
  :meth:`~curses.window.in_wstr`, the counterparts of
  :meth:`~curses.window.getstr` and :meth:`~curses.window.instr` that return a
  :class:`str` rather than :class:`bytes`,
  and the module functions :func:`curses.erasewchar`, :func:`curses.killwchar`
  and :func:`curses.wunctrl`, the wide-character counterparts of
  :func:`curses.erasechar`, :func:`curses.killchar` and :func:`curses.unctrl`.
  On a narrow (non-ncursesw) build the character cell holds a single character
  without combining marks, representable as one byte in the window's encoding,
  and :meth:`~curses.window.in_wstr` returns its decoded text.
  (Contributed by Serhiy Storchaka in :gh:`151757`.)

* Add the :class:`curses.complexchar` type, representing a styled
  character cell (its text, attributes and color pair), and the window
  methods :meth:`~curses.window.in_wch` and :meth:`~curses.window.getbkgrnd`
  that return one --- the counterparts of
  :meth:`~curses.window.inch` and :meth:`~curses.window.getbkgd`.  The
  character-cell methods, such as :meth:`~curses.window.addch` and
  :meth:`~curses.window.border`, now also accept a
  :class:`~curses.complexchar`.  These work whether or not Python was built
  against a wide-character-aware curses library; on a narrow build a cell holds a
  single character representable as one byte in the window's encoding (so only
  8-bit locales are supported).
  (Contributed by Serhiy Storchaka in :gh:`152233`.)

* Add the :class:`curses.complexstr` type, an immutable run of styled cells
  (the string counterpart of :class:`~curses.complexchar`), and the window
  method :meth:`~curses.window.in_wchstr` that returns one.  The string-cell
  methods :meth:`~curses.window.addstr`, :meth:`~curses.window.addnstr`,
  :meth:`~curses.window.insstr` and :meth:`~curses.window.insnstr` now also
  accept a :class:`~curses.complexstr`.  Like :class:`~curses.complexchar`, it
  works whether or not Python was built against a wide-character-aware curses
  library.
  (Contributed by Serhiy Storchaka in :gh:`152233`.)

* The wide-character :mod:`curses` functions and methods
  :meth:`~curses.window.get_wch`, :meth:`~curses.window.get_wstr`,
  :func:`curses.unget_wch`, :func:`curses.erasewchar`,
  :func:`curses.killwchar` and :func:`curses.wunctrl` now also work when Python
  is not built against a wide-character-aware curses library, on an 8-bit
  locale, where each character is a single byte in the relevant encoding.
  :func:`curses.ungetch` now also accepts a one-character string, like
  :func:`curses.unget_wch`; on a wide-character build it can be any character
  (previously a multibyte character raised :exc:`OverflowError`).
  (Contributed by Serhiy Storchaka in :gh:`152470`.)

* Add support for multiple terminals to the :mod:`curses` module:
  the new functions :func:`curses.newterm`, :func:`curses.set_term`
  and :func:`curses.new_prescr`,
  the corresponding :ref:`screen <curses-screen-objects>` object,
  and the :meth:`window.use() <curses.window.use>` method.
  (Contributed by Serhiy Storchaka in :gh:`90092`.)

* Add the :mod:`curses` window methods :meth:`~curses.window.attr_get`,
  :meth:`~curses.window.attr_set`, :meth:`~curses.window.attr_on`,
  :meth:`~curses.window.attr_off` and :meth:`~curses.window.color_set`, which
  pass the color pair as a separate argument instead of packing it into the
  attribute value, and the corresponding ``WA_*`` attribute constants.
  (Contributed by Serhiy Storchaka in :gh:`152219`.)

* Add the :func:`curses.term_attrs` function, which returns the supported
  video attributes as :ref:`WA_* <curses-wa-constants>` values, the
  counterpart of :func:`curses.termattrs`.
  (Contributed by Serhiy Storchaka in :gh:`152332`.)

* Add the ``WACS_*`` constants to the :mod:`curses` module, the counterparts of
  the :ref:`ACS_* <curses-acs-codes>` line-drawing codes as
  :class:`curses.complexchar` cells.
  (Contributed by Serhiy Storchaka in :gh:`155863`.)

* Add the :mod:`curses` functions :func:`curses.alloc_pair`,
  :func:`curses.find_pair`, :func:`curses.free_pair` and
  :func:`curses.reset_color_pairs` for dynamic color-pair management,
  available when built against a wide-character ncurses with extended-color
  support.
  (Contributed by Serhiy Storchaka in :gh:`151774`.)

* Add the :mod:`curses` functions :func:`~curses.scr_dump`,
  :func:`~curses.scr_restore`, :func:`~curses.scr_init` and
  :func:`~curses.scr_set`, which dump the whole screen to a file and restore it.
  (Contributed by Serhiy Storchaka in :gh:`152260`.)

* Add the :mod:`curses` window method :meth:`~curses.window.dupwin`, which
  returns a new window that is an independent duplicate of an existing one.
  (Contributed by Serhiy Storchaka in :gh:`152258`.)

* Add the soft-label-key functions to the :mod:`curses` module, which manage a
  row of labels along the bottom line of the screen:
  :func:`~curses.slk_init`, :func:`~curses.slk_set`, :func:`~curses.slk_label`,
  :func:`~curses.slk_refresh`, :func:`~curses.slk_noutrefresh`,
  :func:`~curses.slk_clear`, :func:`~curses.slk_restore`,
  :func:`~curses.slk_touch`, the attribute functions
  :func:`~curses.slk_attron`, :func:`~curses.slk_attroff`,
  :func:`~curses.slk_attrset`, :func:`~curses.slk_attr`,
  :func:`~curses.slk_attr_on`, :func:`~curses.slk_attr_off`,
  :func:`~curses.slk_attr_set`, and :func:`~curses.slk_color`.
  (Contributed by Serhiy Storchaka in :gh:`152263`.)

* Add the :mod:`curses` key-management functions :func:`~curses.define_key`,
  :func:`~curses.key_defined` and :func:`~curses.keyok`, available when built
  against an ncurses with ``NCURSES_EXT_FUNCS``.
  (Contributed by Serhiy Storchaka in :gh:`152334`.)

* Add the :func:`curses.has_mouse` function and the
  :meth:`curses.window.mouse_trafo` method, completing the :mod:`curses`
  mouse interface.
  (Contributed by Serhiy Storchaka in :gh:`152325`.)

* Add :mod:`curses` functions and window methods that report state which could
  previously only be set, such as :meth:`curses.window.is_keypad`,
  :meth:`curses.window.getparent` and :func:`curses.is_cbreak`,
  available when built against an ncurses with ``NCURSES_EXT_FUNCS``.
  (Contributed by Serhiy Storchaka in :gh:`151776`.)

* Add :func:`curses.nofilter`, which undoes the effect of :func:`curses.filter`.
  (Contributed by Serhiy Storchaka in :gh:`151744`.)

* :class:`curses.textpad.Textbox` now supports entering and reading back the
  full Unicode range, including combining characters, when curses is built with
  wide-character support.
  (Contributed by Serhiy Storchaka in :gh:`133031`.)


ctypes
------

* Add :func:`ctypes.util.struct` for generating :class:`~ctypes.Structure` types
  from an annotation-based syntax, similar to how the :mod:`dataclasses` module
  is used.
  (Contributed by Peter Bierma in :gh:`104533`.)
* Add :func:`ctypes.util.wrap_dll_function` for generating function pointers
  through a function signature.
  (Contributed by Peter Bierma in :gh:`153903`.)


concurrent.futures
------------------

* The iterator returned by :meth:`concurrent.futures.Executor.map` is no longer
  automatically closed if a function call raises an exception.
  Use method :meth:`!close` to explicitly close the iterator.
  (Contributed by xzmeng and Serhiy Storchaka in :gh:`108518`.)


encodings
---------

* Add the ``utf-7-imap`` codec, implementing the modified UTF-7 encoding
  used for international IMAP4 mailbox names (:rfc:`3501`).
  (Contributed by Serhiy Storchaka in :gh:`66788`.)


gzip
----

* :func:`gzip.open` now accepts an optional argument ``mtime``
  which is passed on to the constructor of the :class:`~gzip.GzipFile` class.
  (Contributed by Marin Misur in :gh:`91372`.)

io
--

* Add :meth:`io.BytesIO.peek` method to read without advancing position.
  (Contributed by Marcel Martin in :gh:`90533`.)


imaplib
-------

* Add the :meth:`~imaplib.IMAP4.id` method,
  a wrapper for the ``ID`` command (:rfc:`2971`).
  (Contributed by Serhiy Storchaka in :gh:`98092`.)

* Add the :meth:`~imaplib.IMAP4.move` method,
  a wrapper for the ``MOVE`` command (:rfc:`6851`).
  (Contributed by Serhiy Storchaka in :gh:`77508`.)

* Add the :meth:`~imaplib.IMAP4.login_plain` method, which authenticates
  using the ``PLAIN`` SASL mechanism (:rfc:`4616`) and supports non-ASCII
  user names and passwords.
  (Contributed by Przemysław Buczkowski and Serhiy Storchaka in :gh:`89869`.)

* Non-ASCII mailbox names are now automatically encoded as modified UTF-7
  (:rfc:`3501`, section 5.1.3) in the default mode, so international mailbox
  names can be passed as ordinary :class:`str` without enabling ``UTF8=ACCEPT``
  (under which they are sent as UTF-8).
  (Contributed by Serhiy Storchaka in :gh:`49555`.)

* The :meth:`~imaplib.IMAP4.copy`, :meth:`~imaplib.IMAP4.move`,
  :meth:`~imaplib.IMAP4.fetch`, :meth:`~imaplib.IMAP4.store`,
  :meth:`~imaplib.IMAP4.search`, :meth:`~imaplib.IMAP4.sort`,
  :meth:`~imaplib.IMAP4.thread` and :meth:`~imaplib.IMAP4.expunge` methods
  now accept a keyword-only *uid* argument that selects the corresponding
  ``UID`` command, as a more convenient alternative to
  :meth:`~imaplib.IMAP4.uid`.
  (Contributed by Serhiy Storchaka in :gh:`153502`.)

* :meth:`~imaplib.IMAP4.search`, :meth:`~imaplib.IMAP4.sort`
  and :meth:`~imaplib.IMAP4.thread` (and the corresponding ``uid`` commands)
  now encode :class:`str` search criteria to the declared *charset*,
  so international search text can be passed as an ordinary :class:`str`.
  When *charset* is ``None`` (as it must be under ``UTF8=ACCEPT``),
  the criteria are sent using the connection encoding instead.
  (Contributed by Serhiy Storchaka in :gh:`153494`.)

* Command methods now accept structured arguments,
  so the module takes care of quoting instead of the caller.
  A *message_set* and lists of flags or other atoms
  can be passed as sequences instead of preformatted strings,
  and the :meth:`~imaplib.IMAP4.search`, :meth:`~imaplib.IMAP4.fetch`,
  :meth:`~imaplib.IMAP4.sort`, :meth:`~imaplib.IMAP4.thread` and
  :meth:`~imaplib.IMAP4.uid` methods accept a *params* keyword argument
  that substitutes and quotes ``?`` placeholders,
  in the manner of :mod:`sqlite3` parameter substitution.
  (Contributed by Serhiy Storchaka in :gh:`153521`.)


ipaddress
---------

Add :meth:`~ipaddress.IPv4Network.next_network` and
:meth:`~ipaddress.IPv6Network.next_network` methods to find the next nearest
network with a specific prefix size.


logging
-------

* :class:`~logging.handlers.TimedRotatingFileHandler` now uses the creation
  time instead of the last modification time of an existing log file as
  the basis for the first rotation after handler creation, if supported by
  the OS and file system.
  This allows it to be used in short-running programs that start and end
  before the rotation interval expires.
  (Contributed by Iván Márton and Serhiy Storchaka in :gh:`84649`.)


lzma
----

* Add support of new BCJ filters ARM64 and RISC-V via
  :const:`!lzma.FILTER_ARM64` and :const:`!lzma.FILTER_RISCV`.  Note that the
  new filters will work only if the runtime library supports them. ARM64 filter
  requires ``lzma`` 5.4.0 or newer while RISC-V requires 5.6.0 or newer.
  (Contributed by Chien Wong in :gh:`115988`.)


math
----

* Added trigonometric functions that work in units of half turns, rather than
  radians. The new functions :func:`math.acospi`, :func:`math.asinpi`,
  :func:`math.atanpi`, and :func:`math.atan2pi` return half-turn angles. The
  new functions :func:`math.cospi`, :func:`math.sinpi`, and :func:`math.tanpi`
  take half-turn angle arguments. These functions are recommended by IEEE
  754-2019 and standardized in C23.
  (Contributed by Jeff Epler in :gh:`150534`.)


multiprocessing
---------------

* Add the optional ``buffersize`` parameter to
  :meth:`multiprocessing.pool.Pool.imap` and
  :meth:`multiprocessing.pool.Pool.imap_unordered` to limit the number of
  submitted tasks whose results have not yet been yielded. If the buffer is
  full, iteration over the *iterables* pauses until a result is yielded from
  the buffer. To fully utilize pool's capacity when using this feature, set
  *buffersize* at least to the number of processes in pool (to consume
  *iterable* as you go), or even higher (to prefetch the next
  ``N=buffersize-processes`` arguments).

  (Contributed by Oleksandr Baltian in :gh:`136871`.)


os
--

* Add :func:`os.pidfd_getfd` for duplicating a file descriptor from another
  process via a pidfd.  Available on Linux 5.6+.
  (Contributed by Maurycy Pawłowski-Wieroński in :gh:`149464`.)


pydoc
-----

* Modernize the HTML output of :mod:`pydoc`.  The pages generated by the
  :mod:`pydoc` HTTP server now use semantic HTML5 markup and a new style
  sheet based on the python-docs-theme used by `docs.python.org
  <https://docs.python.org/>`__, with dark mode support.  Class members
  inherited from other classes are collapsed by default.
  (Contributed by Serhiy Storchaka in :gh:`153906`.)


re
--

* :mod:`re` now supports set operations and nested sets in character classes,
  as described in `Unicode Technical Standard #18
  <https://unicode.org/reports/tr18/>`__: set difference (``[A--B]``),
  intersection (``[A&&B]``) and union (``[A||B]``), where an operand may be a
  nested set written in square brackets.  For example, ``[a-z--[aeiou]]``
  matches an ASCII lowercase consonant.
  (Contributed by Serhiy Storchaka in :gh:`152100`.)

* Regular expressions now support Unicode property escapes ``\p{...}`` and
  ``\P{...}``, which match a character by a Unicode property -- for example
  ``\p{Lu}`` (an uppercase letter), ``\p{Cased}`` or ``\p{ASCII}``.  See
  :ref:`the regular expression syntax <re-syntax>` for the supported
  properties.
  (Contributed by Serhiy Storchaka in :gh:`95555`.)


shlex
-----

* Add keyword-only parameter *force* to :func:`shlex.quote` to force quoting
  a string, even if it is already safe for a shell without being quoted.
  (Contributed by Jay Berry in :gh:`148846`.)

sqlite3
-------

* :class:`sqlite3.Blob` now supports negative-step slices for reading and
  writing (e.g. ``blob[9:0:-2]``).  Previously, such slices would raise
  :exc:`SystemError` or :exc:`ValueError`.
  (Contributed by Jiseok CHOI in :gh:`150449`.)

symtable
--------

* :func:`symtable.symtable` now accepts an AST object,
  like the builtin :func:`compile`.
  (Contributed by Serhiy Storchaka in :gh:`153844`.)


tkinter
-------

* Added many :class:`tkinter.ttk.Treeview` methods wrapping the enhanced
  ``ttk::treeview`` widget commands from Tk 9.1, such as
  :meth:`~tkinter.ttk.Treeview.sort`, :meth:`~tkinter.ttk.Treeview.search`,
  :meth:`~tkinter.ttk.Treeview.expand`, :meth:`~tkinter.ttk.Treeview.collapse`,
  :meth:`~tkinter.ttk.Treeview.hide`, :meth:`~tkinter.ttk.Treeview.unhide`, and
  methods for cell focus, selection and tagging.  The
  :meth:`~tkinter.ttk.Treeview.expand` and :meth:`~tkinter.ttk.Treeview.collapse`
  methods (without recursion) also work on Tk older than 9.1.
  (Contributed by Serhiy Storchaka in :gh:`151910`.)

* Added new :class:`!tkinter.Text` methods :meth:`~tkinter.Text.edit_canundo`
  and :meth:`~tkinter.Text.edit_canredo` which return whether an undo or redo
  is possible.
  (Contributed by Serhiy Storchaka in :gh:`151674`.)

* Added new :class:`!tkinter.Text` methods :meth:`~tkinter.Text.sync` and
  :meth:`~tkinter.Text.pendingsync` which control and report the
  synchronization of the displayed view with the underlying text.
  (Contributed by Serhiy Storchaka in :gh:`151675`.)

* Added the :meth:`ttk.Style.theme_styles
  <tkinter.ttk.Style.theme_styles>` method which returns the list of styles
  defined in a theme.
  (Contributed by Serhiy Storchaka in :gh:`151920`.)

* Added new :class:`!tkinter.Canvas` methods :meth:`~tkinter.Canvas.rchars`
  which replaces the text or coordinates of canvas items, and
  :meth:`~tkinter.Canvas.rotate` which rotates the coordinates of canvas items.
  (Contributed by Serhiy Storchaka in :gh:`151876`.)

* Added a :meth:`!validate` method to the :class:`!tkinter.Entry` and
  :class:`!tkinter.Spinbox` widgets, which forces an evaluation of the
  validation command.
  (Contributed by Serhiy Storchaka in :gh:`151878`.)

* Added the :meth:`tkinter.Menu.postcascade` method, and the
  :meth:`~tkinter.Misc.tk_scaling` and :meth:`~tkinter.Misc.tk_inactive`
  methods which respectively query or set the display scaling factor and
  report the user idle time.
  (Contributed by Serhiy Storchaka in :gh:`151881`.)

* Added the :meth:`!tk_print` method to :class:`tkinter.Canvas` and
  :class:`tkinter.Text` which prints the contents of the widget using the
  native print dialog.  It requires Tk 8.7/9.0 or newer.
  (Contributed by Serhiy Storchaka in :gh:`153256`.)

* Added new window-management methods :meth:`~tkinter.Misc.winfo_isdark`
  (dark mode detection), :meth:`~tkinter.Wm.wm_iconbadge` (application icon
  badge) and :meth:`~tkinter.Wm.wm_stackorder` (toplevel stacking order).
  (Contributed by Serhiy Storchaka in :gh:`151874`.)

* Added the :meth:`~tkinter.Misc.tk_appname`,
  :meth:`~tkinter.Misc.tk_useinputmethods` and :meth:`~tkinter.Misc.tk_caret`
  methods, exposing the application send name, the X Input Methods state and
  the input method caret location.
  (Contributed by Serhiy Storchaka in :gh:`151886`.)

* Added support for more options in :class:`!tkinter.PhotoImage` methods: the
  *format* parameter of :meth:`~tkinter.PhotoImage.put`, the *metadata*
  parameter of :meth:`~tkinter.PhotoImage.put`, :meth:`~tkinter.PhotoImage.read`,
  :meth:`~tkinter.PhotoImage.write` and :meth:`~tkinter.PhotoImage.data`, and
  the *withalpha* parameter of :meth:`~tkinter.PhotoImage.get`.
  (Contributed by Serhiy Storchaka in :gh:`151890`.)

* Added the :meth:`~tkinter.PhotoImage.redither` method which recalculates the
  dithered image when its data was supplied in pieces.
  (Contributed by Serhiy Storchaka in :gh:`151888`.)

* :class:`tkinter.OptionMenu` now accepts arbitrary :class:`!tkinter.Menubutton`
  options as keyword arguments, which can also override its default appearance.
  (Contributed by Serhiy Storchaka in :gh:`101284`.)

* The :mod:`tkinter.simpledialog` dialogs were modernized to match the look
  and feel of the native Tk dialogs.
  :class:`!tkinter.simpledialog.SimpleDialog` and the
  :func:`~tkinter.simpledialog.askinteger`,
  :func:`~tkinter.simpledialog.askfloat` and
  :func:`~tkinter.simpledialog.askstring` dialogs are now built from the themed
  :mod:`tkinter.ttk` widgets instead of the classic :mod:`tkinter` widgets;
  the :class:`!tkinter.simpledialog.Dialog` base class still defaults to the
  classic widgets for compatibility.  Both :class:`!Dialog` and
  :class:`!SimpleDialog` gained a *use_ttk* parameter that selects between the
  classic Tk widgets and the themed ttk widgets.  :class:`!SimpleDialog` also
  gained *bitmap* and
  *detail* parameters, draws the standard icons with themed images in the
  ttk version, and accepts mappings of button options as *buttons* entries.
  (Contributed by Serhiy Storchaka in :gh:`59396`.)

* The :class:`!tkinter.filedialog.FileDialog` dialog and its
  :class:`!tkinter.filedialog.LoadFileDialog` and
  :class:`!tkinter.filedialog.SaveFileDialog` subclasses are now built from the
  themed :mod:`tkinter.ttk` widgets by default instead of the classic
  :mod:`tkinter` widgets, and gained a *use_ttk* parameter to select between
  them.
  (Contributed by Serhiy Storchaka in :gh:`59396`.)

* Added the :mod:`tkinter.systray` module which provides the
  :class:`~tkinter.systray.SysTrayIcon` class as an interface to the system
  tray icon and the :func:`~tkinter.systray.notify` function which sends a
  desktop notification.  They require Tk 8.7/9.0 or newer.
  (Contributed by Serhiy Storchaka in :gh:`153259`.)

* :class:`tkinter.scrolledtext.ScrolledText` gained a *use_ttk* parameter to use
  the themed :mod:`tkinter.ttk` frame and scroll bar instead of the classic
  :mod:`tkinter` widgets.
  (Contributed by Serhiy Storchaka in :gh:`59396`.)

* :class:`tkinter.font.Font` can now wrap a font description without creating a
  new named font, by passing it as *font* with ``exists=True`` and no *name*.
  This avoids a loss of precision in :meth:`~tkinter.font.Font.actual`,
  :meth:`~tkinter.font.Font.measure` and :meth:`~tkinter.font.Font.metrics`.
  (Contributed by Serhiy Storchaka in :gh:`143990`.)

* Added the :mod:`tkinter.fontchooser` module which provides the
  :class:`~tkinter.fontchooser.FontChooser` class as an interface to the
  native font selection dialog.
  (Contributed by Serhiy Storchaka in :gh:`72880`.)

* Values of several Tcl object types returned by :mod:`tkinter` are now
  converted to the corresponding Python type instead of being wrapped in a
  :class:`!_tkinter.Tcl_Obj`: ``index``, ``window``, ``nsName`` and
  ``parsedVarName`` objects to :class:`str`, and ``pixel`` screen distances
  with no unit suffix to :class:`int` or :class:`float`.
  (Contributed by Serhiy Storchaka in :gh:`153513`.)

xml
---

* Add support for multiple multi-byte encodings in the :mod:`XML parser
  <xml.parsers.expat>`: "cp932", "cp949", "cp950", "Big5", "EUC-JP",
  "GB2312", "GBK", "johab", and "Shift_JIS".
  Add partial support (only BMP characters) for multi-byte encodings
  "Big5-HKSCS", "EUC_JIS-2004", "EUC_JISX0213", "Shift_JIS-2004",
  "Shift_JISX0213", "utf-8-sig" and non-standard aliases like "UTF8"
  (without hyphen).
  The parser now raises :exc:`ValueError` for known unsupported
  multi-byte encodings such as "ISO-2022-JP" or "raw-unicode-escape"
  instead of failing later, when encountering non-ASCII data.
  (Contributed by Serhiy Storchaka in :gh:`62259`.)

zipfile
-------

* Add :meth:`ZipFile.remove() <zipfile.ZipFile.remove>` to remove a member
  from an archive's central directory, and
  :meth:`ZipFile.repack() <zipfile.ZipFile.repack>` to reclaim the space used
  by the local file entries of removed members.
  (Contributed by Danny Lin in :gh:`51067`.)

.. Add improved modules above alphabetically, not here at the end.

Optimizations
=============

re
--

* Character class escapes (``\d``, ``\D``, ``\s``, ``\S``, ``\w`` and ``\W``)
  outside a character set, and character sets containing a single such escape
  (such as ``[\d]`` or ``[^\s]``), are now compiled to a single ``CATEGORY``
  opcode instead of being wrapped in an ``IN`` block.  This speeds up matching
  of patterns such as ``\d+`` and reduces the size of the compiled byte code.
  (Contributed by Serhiy Storchaka in :gh:`152033` and Pieter Eendebak in
  :gh:`152056`.)

module_name
-----------

* TODO



Removed
========

annotationlib
-------------

* The :meth:`!annotationlib.ForwardRef._evaluate` method
  which has been deprecated since Python 3.14.
  Use :meth:`annotationlib.ForwardRef.evaluate`
  or :func:`typing.evaluate_forward_ref` instead.

array
-----

* The ``'u'`` format code (:c:type:`wchar_t`) which has been deprecated in
  documentation since Python 3.3 and at runtime since Python 3.13.
  Use ``'w'`` format code instead (:c:type:`Py_UCS4`, always 4 bytes).

asyncio
-------

* The :func:`!asyncio.iscoroutinefunction`
  which has been deprecated since Python 3.14.
  Use :func:`inspect.iscoroutinefunction` instead.

* The event loop policy system, including the
  :class:`!asyncio.AbstractEventLoopPolicy`,
  :class:`!asyncio.DefaultEventLoopPolicy`,
  :class:`!asyncio.WindowsSelectorEventLoopPolicy` and
  :class:`!asyncio.WindowsProactorEventLoopPolicy` classes and the
  :func:`!asyncio.get_event_loop_policy` and
  :func:`!asyncio.set_event_loop_policy` functions,
  which have been deprecated since Python 3.14.
  Use :func:`asyncio.run` or :class:`asyncio.Runner` with a custom
  *loop_factory* instead.

functools
---------

* Calling the Python implementation of :func:`functools.reduce` with *function*
  or *sequence* as keyword arguments has been deprecated since Python 3.14.

logging
-------

* Support for custom logging handlers with the *strm* argument is deprecated
  and scheduled for removal in Python 3.16. Define handlers with the *stream*
  argument instead.

mimetypes
---------

* Valid extensions start with a '.' or are empty for
  :meth:`mimetypes.MimeTypes.add_type`.
  Undotted extensions now raise a :exc:`ValueError`.

shutil
------

* The :exc:`!ExecError` exception which has been deprecated since Python 3.14.
  It has not been used by any function in :mod:`!shutil` since Python 3.4.
  (Contributed by Stan Ulbrych in :gh:`149567`.)

symtable
--------

* The :meth:`!symtable.Class.get_methods` method
  which has been deprecated since Python 3.14.

sys
---

* The :func:`!_enablelegacywindowsfsencoding` function
  which has been deprecated since Python 3.13.
  Use the :envvar:`PYTHONLEGACYWINDOWSFSENCODING` environment variable instead.
  (Contributed by Stan Ulbrych in :gh:`149595`.)

sysconfig
---------

* The :func:`!sysconfig.expand_makefile_vars` function
  which has been deprecated since Python 3.14.
  Use the ``vars`` argument of :func:`sysconfig.get_paths` instead.
  (Contributed by Stan Ulbrych in :gh:`149499`.)

tarfile
-------

* The undocumented and unused :attr:`!tarfile.TarFile.tarfile` attribute
  has been deprecated since Python 3.13.

xml.etree.ElementTree
---------------------

* The undocumented :meth:`!xml.etree.ElementTree.ElementTree.write_c14n` method
  and the support of ``method="c14n"`` in
  :meth:`~xml.etree.ElementTree.ElementTree.write` and
  :func:`~xml.etree.ElementTree.tostring`.
  They never worked and always raised :exc:`ValueError`.
  Use :func:`xml.etree.ElementTree.canonicalize` instead.
  (Contributed by Daniel Hillier in :gh:`118318`.)

.. Add removals above alphabetically, not here at the end.


Deprecated
==========

New deprecations
----------------

* :mod:`abc`

  * Soft-deprecated since Python 3.3 :class:`abc.abstractclassmethod`,
    :class:`abc.abstractstaticmethod`, and :class:`abc.abstractproperty`
    now raise a :exc:`DeprecationWarning`.
    These classes will be removed in Python 3.21, instead
    use :func:`abc.abstractmethod` with :func:`classmethod`,
    :func:`staticmethod`, and :class:`property` respectively.

* :mod:`ast`:

  * Classes ``slice``, ``Index``, ``ExtSlice``, ``Suite``, ``Param``,
    ``AugLoad`` and ``AugStore``, deprecated since Python 3.9, are no longer
    imported by ``from ast import *`` and issue a deprecation warning on
    use. The classes are slated for removal in Python 3.21. These types are not
    generated by the parser or accepted by the code generator.
  * The ``dims`` property of ``ast.Tuple`` objects, deprecated since Python
    3.9, now issues a deprecation warning on use. This property is slated for
    removal in 3.21. Use ``ast.Tuple.elts`` instead.

* :mod:`struct`:

  * Soft-deprecated since Python 3.15, using ``'F'`` and ``'D'`` type codes are now
    deprecated.  These codes will be removed in Python 3.21.  Use instead
    two-letter forms ``'Zf'`` and ``'Zd'``.
    (Contributed by Sergey B Kirpichev in :gh:`121249`.)

* :mod:`tempfile`

  * The private ``tempfile._TemporaryFileWrapper`` name is deprecated
    and is slated for removal in Python 3.21.
    Use the new public :class:`tempfile.TemporaryFileWrapper` instead,
    which is the return type of :func:`tempfile.NamedTemporaryFile`.

* :mod:`tkinter`:

  * :func:`tkinter.filedialog.askopenfiles` is deprecated and slated for
    removal in Python 3.19.  Opening several files at once is error-prone, and
    the returned list cannot be used in a :keyword:`with` statement.  Iterate
    over the names returned by :func:`~tkinter.filedialog.askopenfilenames` and
    open them one by one instead.
    (Contributed by Serhiy Storchaka in :gh:`152638`.)

.. Add deprecations above alphabetically, not here at the end.

.. include:: ../deprecations/pending-removal-in-3.17.rst

.. include:: ../deprecations/pending-removal-in-3.18.rst

.. include:: ../deprecations/pending-removal-in-3.19.rst

.. include:: ../deprecations/pending-removal-in-3.20.rst

.. include:: ../deprecations/pending-removal-in-3.21.rst

.. include:: ../deprecations/pending-removal-in-future.rst


Porting to Python 3.16
======================

This section lists previously described changes and other bugfixes
that may require changes to your code.

* In :mod:`tkinter`, the *name* parameter of the
  :meth:`~tkinter.Misc.wait_variable`, :meth:`~tkinter.Misc.setvar` and
  :meth:`~tkinter.Misc.getvar` methods and the *value* parameter of
  :meth:`!setvar` are now required.  Calling these methods without
  them, which formerly defaulted to ``'PY_VAR'`` and ``'1'``, now raises
  :exc:`TypeError`.
  (Contributed by Serhiy Storchaka in :gh:`152587`.)

* On Windows, seeking a pipe now fails instead of silently appearing to
  succeed: :func:`os.lseek` and :meth:`~io.IOBase.seek` raise :exc:`OSError`,
  and :meth:`~io.IOBase.seekable` returns ``False``.  As a consequence,
  opening a pipe in a read-write binary mode (``'r+b'`` or ``'w+b'``) now
  raises :exc:`io.UnsupportedOperation` unless buffering is disabled.
  (Contributed by An Long in :gh:`86768`.)


Build changes
=============

* Remove the bundled copy of the libmpdec_ decimal library from the CPython source tree
  to simplify maintenance and updates. The :mod:`decimal` module will now
  unconditionally use the system's libmpdec decimal library. Also remove the
  now unused :option:`!--with-system-libmpdec` :program:`configure` flag.
  This change has no impact on binary releases of Python, which have been
  built against a separate copy of libmpdec for the past several releases.

  (Contributed by Sergey B Kirpichev in :gh:`115119`.)

  .. _libmpdec: https://www.bytereef.org/mpdecimal/

* Add a :option:`--with-build-details-suffix` configure flag to allow
  Linux distributions that co-install multiple versions of Python in the
  same tree to avoid ``build-details.json`` clashes.

  (Contributed by Stefano Rivera in :gh:`131372`.)

* Add the :option:`--with-curses` :program:`configure` option to select the
  curses backend for the :mod:`curses` and :mod:`curses.panel` modules.
  In addition to ``ncursesw`` and ``ncurses``, it can now build against the
  system's native ``curses`` library (for example on NetBSD or Solaris), with
  wide-character support when the library provides it.  :option:`--without-curses`
  excludes the modules from the build.

  (Contributed by Serhiy Storchaka in :gh:`136687`.)

* Add the :option:`--with-zlib` and :option:`--with-bzip2` :program:`configure`
  options to select the zlib and bzip2 implementations used to build the
  :mod:`zlib` and :mod:`bz2` modules. In addition to the default system
  libraries, they can build against `zlib-rs
  <https://github.com/trifectatechfoundation/zlib-rs/>`__ and `libbzip2-rs
  <https://github.com/trifectatechfoundation/libbzip2-rs>`__, and
  ``--with-zlib=zlib-ng`` verifies that the detected zlib is `zlib-ng
  <https://github.com/zlib-ng/zlib-ng>`__. :option:`--without-zlib` and
  :option:`--without-bzip2` exclude the modules from the build.

  (Contributed by Stan Ulbrych in :gh:`139314`.)


C API changes
=============

New features
------------

* TODO

Porting to Python 3.16
----------------------

* :c:func:`PyType_ClearCache` is now a no-op as the type cache is now
  implemented per-type. It still returns the current version tag.

Deprecated C APIs
-----------------

* :c:func:`PyGen_New`, :c:func:`PyGen_NewWithQualName`, :c:func:`PyCoro_New`,
  and :c:func:`PyAsyncGen_New` are deprecated.
  They are scheduled for removal in 3.18.

.. Add C API deprecations above alphabetically, not here at the end.

.. include:: ../deprecations/c-api-pending-removal-in-3.18.rst

.. include:: ../deprecations/c-api-pending-removal-in-3.19.rst

.. include:: ../deprecations/c-api-pending-removal-in-3.20.rst

.. include:: ../deprecations/c-api-pending-removal-in-future.rst

Removed C APIs
--------------

* The :c:var:`!PyUnstable_ExecutableKinds` array, as well as the macros
  :c:macro:`!PyUnstable_EXECUTABLE_KIND_SKIP`,
  :c:macro:`!PyUnstable_EXECUTABLE_KIND_PY_FUNCTION`,
  :c:macro:`!PyUnstable_EXECUTABLE_KIND_BUILTIN_FUNCTION`,
  :c:macro:`!PyUnstable_EXECUTABLE_KIND_METHOD_DESCRIPTOR` and
  :c:macro:`!PyUnstable_EXECUTABLE_KINDS`, were removed.

  (Contributed by Peters Bierma and Viktorin in :gh:`152105`.)