text:
PEP: 410 Title: Use decimal.Decimal type for timestamps Version:
$Revision$ Last-Modified: $Date$ Author: Victor Stinner
<EMAIL_ADDRESS>Status: Rejected Type: Standards Track
Content-Type: text/x-rst Created: 01-Feb-2012 Python-Version: 3.3
Resolution:
https://mail.python.org/pipermail/python-dev/2012-February/116837.html

Rejection Notice

This PEP is rejected. See
https://mail.python.org/pipermail/python-dev/2012-February/116837.html.

Abstract

Decimal becomes the official type for high-resolution timestamps to make
Python support new functions using a nanosecond resolution without loss
of precision.

Rationale

Python 2.3 introduced float timestamps to support sub-second
resolutions. os.stat() uses float timestamps by default since Python
2.5. Python 3.3 introduced functions supporting nanosecond resolutions:

-   os module: futimens(), utimensat()
-   time module: clock_gettime(), clock_getres(), monotonic(),
    wallclock()

os.stat() reads nanosecond timestamps but returns timestamps as float.

The Python float type uses binary64 format of the IEEE 754 standard.
With a resolution of one nanosecond (10⁻⁹), float timestamps lose
precision for values bigger than 2²⁴ seconds (194 days: 1970-07-14 for
an Epoch timestamp).

Nanosecond resolution is required to set the exact modification time on
filesystems supporting nanosecond timestamps (e.g. ext4, btrfs, NTFS,
...). It helps also to compare the modification time to check if a file
is newer than another file. Use cases: copy the modification time of a
file using shutil.copystat(), create a TAR archive with the tarfile
module, manage a mailbox with the mailbox module, etc.

An arbitrary resolution is preferred over a fixed resolution (like
nanosecond) to not have to change the API when a better resolution is
required. For example, the NTP protocol uses fractions of 2³² seconds
(approximately 2.3 × 10⁻¹⁰ second), whereas the NTP protocol version 4
uses fractions of 2⁶⁴ seconds (5.4 × 10⁻²⁰ second).

Note

With a resolution of 1 microsecond (10⁻⁶), float timestamps lose
precision for values bigger than 2³³ seconds (272 years: 2242-03-16 for
an Epoch timestamp). With a resolution of 100 nanoseconds (10⁻⁷,
resolution used on Windows), float timestamps lose precision for values
bigger than 2²⁹ seconds (17 years: 1987-01-05 for an Epoch timestamp).

Specification

Add decimal.Decimal as a new type for timestamps. Decimal supports any
timestamp resolution, support arithmetic operations and is comparable.
It is possible to coerce a Decimal to float, even if the conversion may
lose precision. The clock resolution can also be stored in a Decimal
object.

Add an optional timestamp argument to:

-   os module: fstat(), fstatat(), lstat(), stat() (st_atime, st_ctime
    and st_mtime fields of the stat structure), sched_rr_get_interval(),
    times(), wait3() and wait4()
-   resource module: ru_utime and ru_stime fields of getrusage()
-   signal module: getitimer(), setitimer()
-   time module: clock(), clock_gettime(), clock_getres(), monotonic(),
    time() and wallclock()

The timestamp argument value can be float or Decimal, float is still the
default for backward compatibility. The following functions support
Decimal as input:

-   datetime module: date.fromtimestamp(), datetime.fromtimestamp() and
    datetime.utcfromtimestamp()
-   os module: futimes(), futimesat(), lutimes(), utime()
-   select module: epoll.poll(), kqueue.control(), select()
-   signal module: setitimer(), sigtimedwait()
-   time module: ctime(), gmtime(), localtime(), sleep()

The os.stat_float_times() function is deprecated: use an explicit cast
using int() instead.

Note

The decimal module is implemented in Python and is slower than float,
but there is a new C implementation which is almost ready for inclusion
in CPython.

Backwards Compatibility

The default timestamp type (float) is unchanged, so there is no impact
on backward compatibility nor on performances. The new timestamp type,
decimal.Decimal, is only returned when requested explicitly.

Objection: clocks accuracy

Computer clocks and operating systems are inaccurate and fail to provide
nanosecond accuracy in practice. A nanosecond is what it takes to
execute a couple of CPU instructions. Even on a real-time operating
system, a nanosecond-precise measurement is already obsolete when it
starts being processed by the higher-level application. A single cache
miss in the CPU will make the precision worthless.

Note

Linux actually is able to measure time in nanosecond precision, even
though it is not able to keep its clock synchronized to UTC with a
nanosecond accuracy.

Alternatives: Timestamp types

To support timestamps with an arbitrary or nanosecond resolution, the
following types have been considered:

-   decimal.Decimal
-   number of nanoseconds
-   128-bits float
-   datetime.datetime
-   datetime.timedelta
-   tuple of integers
-   timespec structure

Criteria:

-   Doing arithmetic on timestamps must be possible
-   Timestamps must be comparable
-   An arbitrary resolution, or at least a resolution of one nanosecond
    without losing precision
-   It should be possible to coerce the new timestamp to float for
    backward compatibility

A resolution of one nanosecond is enough to support all current C
functions.

The best resolution used by operating systems is one nanosecond. In
practice, most clock accuracy is closer to microseconds than
nanoseconds. So it sounds reasonable to use a fixed resolution of one
nanosecond.

Number of nanoseconds (int)

A nanosecond resolution is enough for all current C functions and so a
timestamp can simply be a number of nanoseconds, an integer, not a
float.

The number of nanoseconds format has been rejected because it would
require to add new specialized functions for this format because it not
possible to differentiate a number of nanoseconds and a number of
seconds just by checking the object type.

128-bits float

Add a new IEEE 754-2008 quad-precision binary float type. The IEEE
754-2008 quad precision float has 1 sign bit, 15 bits of exponent and
112 bits of mantissa. 128-bits float is supported by GCC (4.3), Clang
and ICC compilers.

Python must be portable and so cannot rely on a type only available on
some platforms. For example, Visual C++ 2008 doesn't support 128-bits
float, whereas it is used to build the official Windows executables.
Another example: GCC 4.3 does not support __float128 in 32-bit mode on
x86 (but GCC 4.4 does).

There is also a license issue: GCC uses the MPFR library for 128-bits
float, library distributed under the GNU LGPL license. This license is
not compatible with the Python license.

Note

The x87 floating point unit of Intel CPU supports 80-bit floats. This
format is not supported by the SSE instruction set, which is now
preferred over float, especially on x86_64. Other CPU vendors don't
support 80-bit float.

datetime.datetime

The datetime.datetime type is the natural choice for a timestamp because
it is clear that this type contains a timestamp, whereas int, float and
Decimal are raw numbers. It is an absolute timestamp and so is well
defined. It gives direct access to the year, month, day, hours, minutes
and seconds. It has methods related to time like methods to format the
timestamp as string (e.g. datetime.datetime.strftime).

The major issue is that except os.stat(), time.time() and
time.clock_gettime(time.CLOCK_GETTIME), all time functions have an
unspecified starting point and no timezone information, and so cannot be
converted to datetime.datetime.

datetime.datetime has also issues with timezone. For example, a datetime
object without timezone (unaware) and a datetime with a timezone (aware)
cannot be compared. There is also an ordering issues with daylight
saving time (DST) in the duplicate hour of switching from DST to normal
time.

datetime.datetime has been rejected because it cannot be used for
functions using an unspecified starting point like os.times() or
time.clock().

For time.time() and time.clock_gettime(time.CLOCK_GETTIME): it is
already possible to get the current time as a datetime.datetime object
using:

    datetime.datetime.now(datetime.timezone.utc)

For os.stat(), it is simple to create a datetime.datetime object from a
decimal.Decimal timestamp in the UTC timezone:

    datetime.datetime.fromtimestamp(value, datetime.timezone.utc)

Note

datetime.datetime only supports microsecond resolution, but can be
enhanced to support nanosecond.

datetime.timedelta

datetime.timedelta is the natural choice for a relative timestamp
because it is clear that this type contains a timestamp, whereas int,
float and Decimal are raw numbers. It can be used with datetime.datetime
to get an absolute timestamp when the starting point is known.

datetime.timedelta has been rejected because it cannot be coerced to
float and has a fixed resolution. One new standard timestamp type is
enough, Decimal is preferred over datetime.timedelta. Converting a
datetime.timedelta to float requires an explicit call to the
datetime.timedelta.total_seconds() method.

Note

datetime.timedelta only supports microsecond resolution, but can be
enhanced to support nanosecond.

Tuple of integers

To expose C functions in Python, a tuple of integers is the natural
choice to store a timestamp because the C language uses structures with
integers fields (e.g. timeval and timespec structures). Using only
integers avoids the loss of precision (Python supports integers of
arbitrary length). Creating and parsing a tuple of integers is simple
and fast.

Depending of the exact format of the tuple, the precision can be
arbitrary or fixed. The precision can be choose as the loss of precision
is smaller than an arbitrary limit like one nanosecond.

Different formats have been proposed:

-   A: (numerator, denominator)
    -   value = numerator / denominator
    -   resolution = 1 / denominator
    -   denominator > 0
-   B: (seconds, numerator, denominator)
    -   value = seconds + numerator / denominator
    -   resolution = 1 / denominator
    -   0 <= numerator < denominator
    -   denominator > 0
-   C: (intpart, floatpart, base, exponent)
    -   value = intpart + floatpart / base^(exponent)
    -   resolution = 1 / base ^(exponent)
    -   0 <= floatpart < base ^(exponent)
    -   base > 0
    -   exponent >= 0
-   D: (intpart, floatpart, exponent)
    -   value = intpart + floatpart / 10^(exponent)
    -   resolution = 1 / 10 ^(exponent)
    -   0 <= floatpart < 10 ^(exponent)
    -   exponent >= 0
-   E: (sec, nsec)
    -   value = sec + nsec × 10⁻⁹
    -   resolution = 10 ⁻⁹ (nanosecond)
    -   0 <= nsec < 10 ⁹

All formats support an arbitrary resolution, except of the format (E).

The format (D) may not be able to store the exact value (may loss of
precision) if the clock frequency is arbitrary and cannot be expressed
as a power of 10. The format (C) has a similar issue, but in such case,
it is possible to use base=frequency and exponent=1.

The formats (C), (D) and (E) allow optimization for conversion to float
if the base is 2 and to decimal.Decimal if the base is 10.

The format (A) is a simple fraction. It supports arbitrary precision, is
simple (only two fields), only requires a simple division to get the
floating point value, and is already used by float.as_integer_ratio().

To simplify the implementation (especially the C implementation to avoid
integer overflow), a numerator bigger than the denominator can be
accepted. The tuple may be normalized later.

Tuple of integers have been rejected because they don't support
arithmetic operations.

Note

On Windows, the QueryPerformanceCounter() clock uses the frequency of
the processor which is an arbitrary number and so may not be a power or
2 or 10. The frequency can be read using QueryPerformanceFrequency().

timespec structure

timespec is the C structure used to store timestamp with a nanosecond
resolution. Python can use a type with the same structure: (seconds,
nanoseconds). For convenience, arithmetic operations on timespec are
supported.

Example of an incomplete timespec type supporting addition, subtraction
and coercion to float:

    class timespec(tuple):
        def __new__(cls, sec, nsec):
            if not isinstance(sec, int):
                raise TypeError
            if not isinstance(nsec, int):
                raise TypeError
            asec, nsec = divmod(nsec, 10 ** 9)
            sec += asec
            obj = tuple.__new__(cls, (sec, nsec))
            obj.sec = sec
            obj.nsec = nsec
            return obj

        def __float__(self):
            return self.sec + self.nsec * 1e-9

        def total_nanoseconds(self):
            return self.sec * 10 ** 9 + self.nsec

        def __add__(self, other):
            if not isinstance(other, timespec):
                raise TypeError
            ns_sum = self.total_nanoseconds() + other.total_nanoseconds()
            return timespec(*divmod(ns_sum, 10 ** 9))

        def __sub__(self, other):
            if not isinstance(other, timespec):
                raise TypeError
            ns_diff = self.total_nanoseconds() - other.total_nanoseconds()
            return timespec(*divmod(ns_diff, 10 ** 9))

        def __str__(self):
            if self.sec < 0 and self.nsec:
                sec = abs(1 + self.sec)
                nsec = 10**9 - self.nsec
                return '-%i.%09u' % (sec, nsec)
            else:
                return '%i.%09u' % (self.sec, self.nsec)

        def __repr__(self):
            return '<timespec(%s, %s)>' % (self.sec, self.nsec)

The timespec type is similar to the format (E) of tuples of integer,
except that it supports arithmetic and coercion to float.

The timespec type was rejected because it only supports nanosecond
resolution and requires to implement each arithmetic operation, whereas
the Decimal type is already implemented and well tested.

Alternatives: API design

Add a string argument to specify the return type

Add a string argument to function returning timestamps, example:
time.time(format="datetime"). A string is more extensible than a type:
it is possible to request a format that has no type, like a tuple of
integers.

This API was rejected because it was necessary to import implicitly
modules to instantiate objects (e.g. import datetime to create
datetime.datetime). Importing a module may raise an exception and may be
slow, such behaviour is unexpected and surprising.

Add a global flag to change the timestamp type

A global flag like os.stat_decimal_times(), similar to
os.stat_float_times(), can be added to set globally the timestamp type.

A global flag may cause issues with libraries and applications expecting
float instead of Decimal. Decimal is not fully compatible with float.
float+Decimal raises a TypeError for example. The os.stat_float_times()
case is different because an int can be coerced to float and int+float
gives float.

Add a protocol to create a timestamp

Instead of hard coding how timestamps are created, a new protocol can be
added to create a timestamp from a fraction.

For example, time.time(timestamp=type) would call the class method
type.__fromfraction__(numerator, denominator) to create a timestamp
object of the specified type. If the type doesn't support the protocol,
a fallback is used: type(numerator) / type(denominator).

A variant is to use a "converter" callback to create a timestamp.
Example creating a float timestamp:

    def timestamp_to_float(numerator, denominator):
        return float(numerator) / float(denominator)

Common converters can be provided by time, datetime and other modules,
or maybe a specific "hires" module. Users can define their own
converters.

Such protocol has a limitation: the timestamp structure has to be
decided once and cannot be changed later. For example, adding a timezone
or the absolute start of the timestamp would break the API.

The protocol proposition was as being excessive given the requirements,
but that the specific syntax proposed (time.time(timestamp=type)) allows
this to be introduced later if compelling use cases are discovered.

Note

Other formats may be used instead of a fraction: see the tuple of
integers section for example.

Add new fields to os.stat

To get the creation, modification and access time of a file with a
nanosecond resolution, three fields can be added to os.stat() structure.

The new fields can be timestamps with nanosecond resolution (e.g.
Decimal) or the nanosecond part of each timestamp (int).

If the new fields are timestamps with nanosecond resolution, populating
the extra fields would be time-consuming. Any call to os.stat() would be
slower, even if os.stat() is only called to check if a file exists. A
parameter can be added to os.stat() to make these fields optional, the
structure would have a variable number of fields.

If the new fields only contain the fractional part (nanoseconds),
os.stat() would be efficient. These fields would always be present and
so set to zero if the operating system does not support sub-second
resolution. Splitting a timestamp in two parts, seconds and nanoseconds,
is similar to the timespec type and tuple of integers, and so have the
same drawbacks.

Adding new fields to the os.stat() structure does not solve the
nanosecond issue in other modules (e.g. the time module).

Add a boolean argument

Because we only need one new type (Decimal), a simple boolean flag can
be added. Example: time.time(decimal=True) or time.time(hires=True).

Such flag would require to do a hidden import which is considered as a
bad practice.

The boolean argument API was rejected because it is not "pythonic".
Changing the return type with a parameter value is preferred over a
boolean parameter (a flag).

Add new functions

Add new functions for each type, examples:

-   time.clock_decimal()
-   time.time_decimal()
-   os.stat_decimal()
-   os.stat_timespec()
-   etc.

Adding a new function for each function creating timestamps duplicate a
lot of code and would be a pain to maintain.

Add a new hires module

Add a new module called "hires" with the same API than the time module,
except that it would return timestamp with high resolution, e.g.
decimal.Decimal. Adding a new module avoids to link low-level modules
like time or os to the decimal module.

This idea was rejected because it requires to duplicate most of the code
of the time module, would be a pain to maintain, and timestamps are used
modules other than the time module. Examples: signal.sigtimedwait(),
select.select(), resource.getrusage(), os.stat(), etc. Duplicate the
code of each module is not acceptable.

Links

Python:

-   Issue #7652: Merge C version of decimal into py3k (cdecimal)
-   Issue #11457: os.stat(): add new fields to get timestamps as Decimal
    objects with nanosecond resolution
-   Issue #13882: PEP 410: Use decimal.Decimal type for timestamps
-   [Python-Dev] Store timestamps as decimal.Decimal objects

Other languages:

-   Ruby (1.9.3), the Time class supports picosecond (10⁻¹²)
-   .NET framework, DateTime type: number of 100-nanosecond intervals
    that have elapsed since 12:00:00 midnight, January 1, 0001.
    DateTime.Ticks uses a signed 64-bit integer.
-   Java (1.5), System.nanoTime(): wallclock with an unspecified
    starting point as a number of nanoseconds, use a signed 64 bits
    integer (long).
-   Perl, Time::Hiref module: use float so has the same loss of
    precision issue with nanosecond resolution than Python float
    timestamps

Copyright

This document has been placed in the public domain.


text:
PEP: 8 Title: Style Guide for Python Code Author: Guido van Rossum
<EMAIL_ADDRESS>Barry Warsaw<EMAIL_ADDRESS>Alyssa Coghlan
<EMAIL_ADDRESS>Status: Active Type: Process Created: 05-Jul-2001
Post-History: 05-Jul-2001, 01-Aug-2013

Introduction

This document gives coding conventions for the Python code comprising
the standard library in the main Python distribution. Please see the
companion informational PEP describing style guidelines for the C code
in the C implementation of Python <7>.

This document and PEP 257 (Docstring Conventions) were adapted from
Guido's original Python Style Guide essay, with some additions from
Barry's style guide[1].

This style guide evolves over time as additional conventions are
identified and past conventions are rendered obsolete by changes in the
language itself.

Many projects have their own coding style guidelines. In the event of
any conflicts, such project-specific guides take precedence for that
project.

A Foolish Consistency is the Hobgoblin of Little Minds

One of Guido's key insights is that code is read much more often than it
is written. The guidelines provided here are intended to improve the
readability of code and make it consistent across the wide spectrum of
Python code. As PEP 20 says, "Readability counts".

A style guide is about consistency. Consistency with this style guide is
important. Consistency within a project is more important. Consistency
within one module or function is the most important.

However, know when to be inconsistent -- sometimes style guide
recommendations just aren't applicable. When in doubt, use your best
judgment. Look at other examples and decide what looks best. And don't
hesitate to ask!

In particular: do not break backwards compatibility just to comply with
this PEP!

Some other good reasons to ignore a particular guideline:

1.  When applying the guideline would make the code less readable, even
    for someone who is used to reading code that follows this PEP.
2.  To be consistent with surrounding code that also breaks it (maybe
    for historic reasons) -- although this is also an opportunity to
    clean up someone else's mess (in true XP style).
3.  Because the code in question predates the introduction of the
    guideline and there is no other reason to be modifying that code.
4.  When the code needs to remain compatible with older versions of
    Python that don't support the feature recommended by the style
    guide.

Code Lay-out

Indentation

Use 4 spaces per indentation level.

Continuation lines should align wrapped elements either vertically using
Python's implicit line joining inside parentheses, brackets and braces,
or using a hanging indent[2]. When using a hanging indent the following
should be considered; there should be no arguments on the first line and
further indentation should be used to clearly distinguish itself as a
continuation line:

    # Correct:

    # Aligned with opening delimiter.
    foo = long_function_name(var_one, var_two,
                             var_three, var_four)

    # Add 4 spaces (an extra level of indentation) to distinguish arguments from the rest.
    def long_function_name(
            var_one, var_two, var_three,
            var_four):
        print(var_one)

    # Hanging indents should add a level.
    foo = long_function_name(
        var_one, var_two,
        var_three, var_four)

    # Wrong:

    # Arguments on first line forbidden when not using vertical alignment.
    foo = long_function_name(var_one, var_two,
        var_three, var_four)

    # Further indentation required as indentation is not distinguishable.
    def long_function_name(
        var_one, var_two, var_three,
        var_four):
        print(var_one)

The 4-space rule is optional for continuation lines.

Optional:

    # Hanging indents *may* be indented to other than 4 spaces.
    foo = long_function_name(
      var_one, var_two,
      var_three, var_four)

When the conditional part of an if-statement is long enough to require
that it be written across multiple lines, it's worth noting that the
combination of a two character keyword (i.e. if), plus a single space,
plus an opening parenthesis creates a natural 4-space indent for the
subsequent lines of the multiline conditional. This can produce a visual
conflict with the indented suite of code nested inside the if-statement,
which would also naturally be indented to 4 spaces. This PEP takes no
explicit position on how (or whether) to further visually distinguish
such conditional lines from the nested suite inside the if-statement.
Acceptable options in this situation include, but are not limited to:

    # No extra indentation.
    if (this_is_one_thing and
        that_is_another_thing):
        do_something()

    # Add a comment, which will provide some distinction in editors
    # supporting syntax highlighting.
    if (this_is_one_thing and
        that_is_another_thing):
        # Since both conditions are true, we can frobnicate.
        do_something()

    # Add some extra indentation on the conditional continuation line.
    if (this_is_one_thing
            and that_is_another_thing):
        do_something()

(Also see the discussion of whether to break before or after binary
operators below.)

The closing brace/bracket/parenthesis on multiline constructs may either
line up under the first non-whitespace character of the last line of
list, as in:

    my_list = [
        1, 2, 3,
        4, 5, 6,
        ]
    result = some_function_that_takes_arguments(
        'a', 'b', 'c',
        'd', 'e', 'f',
        )

or it may be lined up under the first character of the line that starts
the multiline construct, as in:

    my_list = [
        1, 2, 3,
        4, 5, 6,
    ]
    result = some_function_that_takes_arguments(
        'a', 'b', 'c',
        'd', 'e', 'f',
    )

Tabs or Spaces?

Spaces are the preferred indentation method.

Tabs should be used solely to remain consistent with code that is
already indented with tabs.

Python disallows mixing tabs and spaces for indentation.

Maximum Line Length

Limit all lines to a maximum of 79 characters.

For flowing long blocks of text with fewer structural restrictions
(docstrings or comments), the line length should be limited to 72
characters.

Limiting the required editor window width makes it possible to have
several files open side by side, and works well when using code review
tools that present the two versions in adjacent columns.

The default wrapping in most tools disrupts the visual structure of the
code, making it more difficult to understand. The limits are chosen to
avoid wrapping in editors with the window width set to 80, even if the
tool places a marker glyph in the final column when wrapping lines. Some
web based tools may not offer dynamic line wrapping at all.

Some teams strongly prefer a longer line length. For code maintained
exclusively or primarily by a team that can reach agreement on this
issue, it is okay to increase the line length limit up to 99 characters,
provided that comments and docstrings are still wrapped at 72
characters.

The Python standard library is conservative and requires limiting lines
to 79 characters (and docstrings/comments to 72).

The preferred way of wrapping long lines is by using Python's implied
line continuation inside parentheses, brackets and braces. Long lines
can be broken over multiple lines by wrapping expressions in
parentheses. These should be used in preference to using a backslash for
line continuation.

Backslashes may still be appropriate at times. For example, long,
multiple with-statements could not use implicit continuation before
Python 3.10, so backslashes were acceptable for that case:

    with open('/path/to/some/file/you/want/to/read') as file_1, \
         open('/path/to/some/file/being/written', 'w') as file_2:
        file_2.write(file_1.read())

(See the previous discussion on multiline if-statements for further
thoughts on the indentation of such multiline with-statements.)

Another such case is with assert statements.

Make sure to indent the continued line appropriately.

Should a Line Break Before or After a Binary Operator?

For decades the recommended style was to break after binary operators.
But this can hurt readability in two ways: the operators tend to get
scattered across different columns on the screen, and each operator is
moved away from its operand and onto the previous line. Here, the eye
has to do extra work to tell which items are added and which are
subtracted:

    # Wrong:
    # operators sit far away from their operands
    income = (gross_wages +
              taxable_interest +
              (dividends - qualified_dividends) -
              ira_deduction -
              student_loan_interest)

To solve this readability problem, mathematicians and their publishers
follow the opposite convention. Donald Knuth explains the traditional
rule in his Computers and Typesetting series: "Although formulas within
a paragraph always break after binary operations and relations,
displayed formulas always break before binary operations"[3].

Following the tradition from mathematics usually results in more
readable code:

    # Correct:
    # easy to match operators with operands
    income = (gross_wages
              + taxable_interest
              + (dividends - qualified_dividends)
              - ira_deduction
              - student_loan_interest)

In Python code, it is permissible to break before or after a binary
operator, as long as the convention is consistent locally. For new code
Knuth's style is suggested.

Blank Lines

Surround top-level function and class definitions with two blank lines.

Method definitions inside a class are surrounded by a single blank line.

Extra blank lines may be used (sparingly) to separate groups of related
functions. Blank lines may be omitted between a bunch of related
one-liners (e.g. a set of dummy implementations).

Use blank lines in functions, sparingly, to indicate logical sections.

Python accepts the control-L (i.e. ^L) form feed character as
whitespace; many tools treat these characters as page separators, so you
may use them to separate pages of related sections of your file. Note,
some editors and web-based code viewers may not recognize control-L as a
form feed and will show another glyph in its place.

Source File Encoding

Code in the core Python distribution should always use UTF-8, and should
not have an encoding declaration.

In the standard library, non-UTF-8 encodings should be used only for
test purposes. Use non-ASCII characters sparingly, preferably only to
denote places and human names. If using non-ASCII characters as data,
avoid noisy Unicode characters like z̯̯͡a̧͎̺l̡͓̫g̹̲o̡̼̘ and byte order marks.

All identifiers in the Python standard library MUST use ASCII-only
identifiers, and SHOULD use English words wherever feasible (in many
cases, abbreviations and technical terms are used which aren't English).

Open source projects with a global audience are encouraged to adopt a
similar policy.

Imports

-   Imports should usually be on separate lines:

        # Correct:
        import os
        import sys

        # Wrong:
        import sys, os

    It's okay to say this though:

        # Correct:
        from subprocess import Popen, PIPE

-   Imports are always put at the top of the file, just after any module
    comments and docstrings, and before module globals and constants.

    Imports should be grouped in the following order:

    1.  Standard library imports.
    2.  Related third party imports.
    3.  Local application/library specific imports.

    You should put a blank line between each group of imports.

-   Absolute imports are recommended, as they are usually more readable
    and tend to be better behaved (or at least give better error
    messages) if the import system is incorrectly configured (such as
    when a directory inside a package ends up on sys.path):

        import mypkg.sibling
        from mypkg import sibling
        from mypkg.sibling import example

    However, explicit relative imports are an acceptable alternative to
    absolute imports, especially when dealing with complex package
    layouts where using absolute imports would be unnecessarily verbose:

        from . import sibling
        from .sibling import example

    Standard library code should avoid complex package layouts and
    always use absolute imports.

-   When importing a class from a class-containing module, it's usually
    okay to spell this:

        from myclass import MyClass
        from foo.bar.yourclass import YourClass

    If this spelling causes local name clashes, then spell them
    explicitly:

        import myclass
        import foo.bar.yourclass

    and use myclass.MyClass and foo.bar.yourclass.YourClass.

-   Wildcard imports (from <module> import *) should be avoided, as they
    make it unclear which names are present in the namespace, confusing
    both readers and many automated tools. There is one defensible use
    case for a wildcard import, which is to republish an internal
    interface as part of a public API (for example, overwriting a pure
    Python implementation of an interface with the definitions from an
    optional accelerator module and exactly which definitions will be
    overwritten isn't known in advance).

    When republishing names this way, the guidelines below regarding
    public and internal interfaces still apply.

Module Level Dunder Names

Module level "dunders" (i.e. names with two leading and two trailing
underscores) such as __all__, __author__, __version__, etc. should be
placed after the module docstring but before any import statements
except from __future__ imports. Python mandates that future-imports must
appear in the module before any other code except docstrings:

    """This is the example module.

    This module does stuff.
    """

    from __future__ import barry_as_FLUFL

    __all__ = ['a', 'b', 'c']
    __version__ = '0.1'
    __author__ = 'Cardinal Biggles'

    import os
    import sys

String Quotes

In Python, single-quoted strings and double-quoted strings are the same.
This PEP does not make a recommendation for this. Pick a rule and stick
to it. When a string contains single or double quote characters,
however, use the other one to avoid backslashes in the string. It
improves readability.

For triple-quoted strings, always use double quote characters to be
consistent with the docstring convention in PEP 257.

Whitespace in Expressions and Statements

Pet Peeves

Avoid extraneous whitespace in the following situations:

-   Immediately inside parentheses, brackets or braces:

        # Correct:
        spam(ham[1], {eggs: 2})

        # Wrong:
        spam( ham[ 1 ], { eggs: 2 } )

-   Between a trailing comma and a following close parenthesis:

        # Correct:
        foo = (0,)

        # Wrong:
        bar = (0, )

-   Immediately before a comma, semicolon, or colon:

        # Correct:
        if x == 4: print(x, y); x, y = y, x

        # Wrong:
        if x == 4 : print(x , y) ; x , y = y , x

-   However, in a slice the colon acts like a binary operator, and
    should have equal amounts on either side (treating it as the
    operator with the lowest priority). In an extended slice, both
    colons must have the same amount of spacing applied. Exception: when
    a slice parameter is omitted, the space is omitted:

        # Correct:
        ham[1:9], ham[1:9:3], ham[:9:3], ham[1::3], ham[1:9:]
        ham[lower:upper], ham[lower:upper:], ham[lower::step]
        ham[lower+offset : upper+offset]
        ham[: upper_fn(x) : step_fn(x)], ham[:: step_fn(x)]
        ham[lower + offset : upper + offset]

        # Wrong:
        ham[lower + offset:upper + offset]
        ham[1: 9], ham[1 :9], ham[1:9 :3]
        ham[lower : : step]
        ham[ : upper]

-   Immediately before the open parenthesis that starts the argument
    list of a function call:

        # Correct:
        spam(1)

        # Wrong:
        spam (1)

-   Immediately before the open parenthesis that starts an indexing or
    slicing:

        # Correct:
        dct['key'] = lst[index]

        # Wrong:
        dct ['key'] = lst [index]

-   More than one space around an assignment (or other) operator to
    align it with another:

        # Correct:
        x = 1
        y = 2
        long_variable = 3

        # Wrong:
        x             = 1
        y             = 2
        long_variable = 3

Other Recommendations

-   Avoid trailing whitespace anywhere. Because it's usually invisible,
    it can be confusing: e.g. a backslash followed by a space and a
    newline does not count as a line continuation marker. Some editors
    don't preserve it and many projects (like CPython itself) have
    pre-commit hooks that reject it.

-   Always surround these binary operators with a single space on either
    side: assignment (=), augmented assignment (+=, -= etc.),
    comparisons (==, <, >, !=, <>, <=, >=, in, not in, is, is not),
    Booleans (and, or, not).

-   If operators with different priorities are used, consider adding
    whitespace around the operators with the lowest priority(ies). Use
    your own judgment; however, never use more than one space, and
    always have the same amount of whitespace on both sides of a binary
    operator:

        # Correct:
        i = i + 1
        submitted += 1
        x = x*2 - 1
        hypot2 = x*x + y*y
        c = (a+b) * (a-b)

        # Wrong:
        i=i+1
        submitted +=1
        x = x * 2 - 1
        hypot2 = x * x + y * y
        c = (a + b) * (a - b)

-   Function annotations should use the normal rules for colons and
    always have spaces around the -> arrow if present. (See Function
    Annotations below for more about function annotations.):

        # Correct:
        def munge(input: AnyStr): ...
        def munge() -> PosInt: ...

        # Wrong:
        def munge(input:AnyStr): ...
        def munge()->PosInt: ...

-   Don't use spaces around the = sign when used to indicate a keyword
    argument, or when used to indicate a default value for an
    unannotated function parameter:

        # Correct:
        def complex(real, imag=0.0):
            return magic(r=real, i=imag)

        # Wrong:
        def complex(real, imag = 0.0):
            return magic(r = real, i = imag)

    When combining an argument annotation with a default value, however,
    do use spaces around the = sign:

        # Correct:
        def munge(sep: AnyStr = None): ...
        def munge(input: AnyStr, sep: AnyStr = None, limit=1000): ...

        # Wrong:
        def munge(input: AnyStr=None): ...
        def munge(input: AnyStr, limit = 1000): ...

-   Compound statements (multiple statements on the same line) are
    generally discouraged:

        # Correct:
        if foo == 'blah':
            do_blah_thing()
        do_one()
        do_two()
        do_three()

    Rather not:

        # Wrong:
        if foo == 'blah': do_blah_thing()
        do_one(); do_two(); do_three()

-   While sometimes it's okay to put an if/for/while with a small body
    on the same line, never do this for multi-clause statements. Also
    avoid folding such long lines!

    Rather not:

        # Wrong:
        if foo == 'blah': do_blah_thing()
        for x in lst: total += x
        while t < 10: t = delay()

    Definitely not:

        # Wrong:
        if foo == 'blah': do_blah_thing()
        else: do_non_blah_thing()

        try: something()
        finally: cleanup()

        do_one(); do_two(); do_three(long, argument,
                                     list, like, this)

        if foo == 'blah': one(); two(); three()

When to Use Trailing Commas

Trailing commas are usually optional, except they are mandatory when
making a tuple of one element. For clarity, it is recommended to
surround the latter in (technically redundant) parentheses:

    # Correct:
    FILES = ('setup.cfg',)

    # Wrong:
    FILES = 'setup.cfg',

When trailing commas are redundant, they are often helpful when a
version control system is used, when a list of values, arguments or
imported items is expected to be extended over time. The pattern is to
put each value (etc.) on a line by itself, always adding a trailing
comma, and add the close parenthesis/bracket/brace on the next line.
However it does not make sense to have a trailing comma on the same line
as the closing delimiter (except in the above case of singleton tuples):

    # Correct:
    FILES = [
        'setup.cfg',
        'tox.ini',
        ]
    initialize(FILES,
               error=True,
               )

    # Wrong:
    FILES = ['setup.cfg', 'tox.ini',]
    initialize(FILES, error=True,)

Comments

Comments that contradict the code are worse than no comments. Always
make a priority of keeping the comments up-to-date when the code
changes!

Comments should be complete sentences. The first word should be
capitalized, unless it is an identifier that begins with a lower case
letter (never alter the case of identifiers!).

Block comments generally consist of one or more paragraphs built out of
complete sentences, with each sentence ending in a period.

You should use one or two spaces after a sentence-ending period in
multi-sentence comments, except after the final sentence.

Ensure that your comments are clear and easily understandable to other
speakers of the language you are writing in.

Python coders from non-English speaking countries: please write your
comments in English, unless you are 120% sure that the code will never
be read by people who don't speak your language.

Block Comments

Block comments generally apply to some (or all) code that follows them,
and are indented to the same level as that code. Each line of a block
comment starts with a # and a single space (unless it is indented text
inside the comment).

Paragraphs inside a block comment are separated by a line containing a
single #.

Inline Comments

Use inline comments sparingly.

An inline comment is a comment on the same line as a statement. Inline
comments should be separated by at least two spaces from the statement.
They should start with a # and a single space.

Inline comments are unnecessary and in fact distracting if they state
the obvious. Don't do this:

    x = x + 1                 # Increment x

But sometimes, this is useful:

    x = x + 1                 # Compensate for border

Documentation Strings

Conventions for writing good documentation strings (a.k.a. "docstrings")
are immortalized in PEP 257.

-   Write docstrings for all public modules, functions, classes, and
    methods. Docstrings are not necessary for non-public methods, but
    you should have a comment that describes what the method does. This
    comment should appear after the def line.

-   PEP 257 describes good docstring conventions. Note that most
    importantly, the """ that ends a multiline docstring should be on a
    line by itself:

        """Return a foobang

        Optional plotz says to frobnicate the bizbaz first.
        """

-   For one liner docstrings, please keep the closing """ on the same
    line:

        """Return an ex-parrot."""

Naming Conventions

The naming conventions of Python's library are a bit of a mess, so we'll
never get this completely consistent -- nevertheless, here are the
currently recommended naming standards. New modules and packages
(including third party frameworks) should be written to these standards,
but where an existing library has a different style, internal
consistency is preferred.

Overriding Principle

Names that are visible to the user as public parts of the API should
follow conventions that reflect usage rather than implementation.

Descriptive: Naming Styles

There are a lot of different naming styles. It helps to be able to
recognize what naming style is being used, independently from what they
are used for.

The following naming styles are commonly distinguished:

-   b (single lowercase letter)

-   B (single uppercase letter)

-   lowercase

-   lower_case_with_underscores

-   UPPERCASE

-   UPPER_CASE_WITH_UNDERSCORES

-   CapitalizedWords (or CapWords, or CamelCase -- so named because of
    the bumpy look of its letters[4]). This is also sometimes known as
    StudlyCaps.

    Note: When using acronyms in CapWords, capitalize all the letters of
    the acronym. Thus HTTPServerError is better than HttpServerError.

-   mixedCase (differs from CapitalizedWords by initial lowercase
    character!)

-   Capitalized_Words_With_Underscores (ugly!)

There's also the style of using a short unique prefix to group related
names together. This is not used much in Python, but it is mentioned for
completeness. For example, the os.stat() function returns a tuple whose
items traditionally have names like st_mode, st_size, st_mtime and so
on. (This is done to emphasize the correspondence with the fields of the
POSIX system call struct, which helps programmers familiar with that.)

The X11 library uses a leading X for all its public functions. In
Python, this style is generally deemed unnecessary because attribute and
method names are prefixed with an object, and function names are
prefixed with a module name.

In addition, the following special forms using leading or trailing
underscores are recognized (these can generally be combined with any
case convention):

-   _single_leading_underscore: weak "internal use" indicator. E.g.
    from M import * does not import objects whose names start with an
    underscore.

-   single_trailing_underscore_: used by convention to avoid conflicts
    with Python keyword, e.g. :

        tkinter.Toplevel(master, class_='ClassName')

-   __double_leading_underscore: when naming a class attribute, invokes
    name mangling (inside class FooBar, __boo becomes _FooBar__boo; see
    below).

-   __double_leading_and_trailing_underscore__: "magic" objects or
    attributes that live in user-controlled namespaces. E.g. __init__,
    __import__ or __file__. Never invent such names; only use them as
    documented.

Prescriptive: Naming Conventions

Names to Avoid

Never use the characters 'l' (lowercase letter el), 'O' (uppercase
letter oh), or 'I' (uppercase letter eye) as single character variable
names.

In some fonts, these characters are indistinguishable from the numerals
one and zero. When tempted to use 'l', use 'L' instead.

ASCII Compatibility

Identifiers used in the standard library must be ASCII compatible as
described in the policy section <3131#policy-specification> of PEP 3131.

Package and Module Names

Modules should have short, all-lowercase names. Underscores can be used
in the module name if it improves readability. Python packages should
also have short, all-lowercase names, although the use of underscores is
discouraged.

When an extension module written in C or C++ has an accompanying Python
module that provides a higher level (e.g. more object oriented)
interface, the C/C++ module has a leading underscore (e.g. _socket).

Class Names

Class names should normally use the CapWords convention.

The naming convention for functions may be used instead in cases where
the interface is documented and used primarily as a callable.

Note that there is a separate convention for builtin names: most builtin
names are single words (or two words run together), with the CapWords
convention used only for exception names and builtin constants.

Type Variable Names

Names of type variables introduced in PEP 484 should normally use
CapWords preferring short names: T, AnyStr, Num. It is recommended to
add suffixes _co or _contra to the variables used to declare covariant
or contravariant behavior correspondingly:

    from typing import TypeVar

    VT_co = TypeVar('VT_co', covariant=True)
    KT_contra = TypeVar('KT_contra', contravariant=True)

Exception Names

Because exceptions should be classes, the class naming convention
applies here. However, you should use the suffix "Error" on your
exception names (if the exception actually is an error).

Global Variable Names

(Let's hope that these variables are meant for use inside one module
only.) The conventions are about the same as those for functions.

Modules that are designed for use via from M import * should use the
__all__ mechanism to prevent exporting globals, or use the older
convention of prefixing such globals with an underscore (which you might
want to do to indicate these globals are "module non-public").

Function and Variable Names

Function names should be lowercase, with words separated by underscores
as necessary to improve readability.

Variable names follow the same convention as function names.

mixedCase is allowed only in contexts where that's already the
prevailing style (e.g. threading.py), to retain backwards compatibility.

Function and Method Arguments

Always use self for the first argument to instance methods.

Always use cls for the first argument to class methods.

If a function argument's name clashes with a reserved keyword, it is
generally better to append a single trailing underscore rather than use
an abbreviation or spelling corruption. Thus class_ is better than clss.
(Perhaps better is to avoid such clashes by using a synonym.)

Method Names and Instance Variables

Use the function naming rules: lowercase with words separated by
underscores as necessary to improve readability.

Use one leading underscore only for non-public methods and instance
variables.

To avoid name clashes with subclasses, use two leading underscores to
invoke Python's name mangling rules.

Python mangles these names with the class name: if class Foo has an
attribute named __a, it cannot be accessed by Foo.__a. (An insistent
user could still gain access by calling Foo._Foo__a.) Generally, double
leading underscores should be used only to avoid name conflicts with
attributes in classes designed to be subclassed.

Note: there is some controversy about the use of __names (see below).

Constants

Constants are usually defined on a module level and written in all
capital letters with underscores separating words. Examples include
MAX_OVERFLOW and TOTAL.

Designing for Inheritance

Always decide whether a class's methods and instance variables
(collectively: "attributes") should be public or non-public. If in
doubt, choose non-public; it's easier to make it public later than to
make a public attribute non-public.

Public attributes are those that you expect unrelated clients of your
class to use, with your commitment to avoid backwards incompatible
changes. Non-public attributes are those that are not intended to be
used by third parties; you make no guarantees that non-public attributes
won't change or even be removed.

We don't use the term "private" here, since no attribute is really
private in Python (without a generally unnecessary amount of work).

Another category of attributes are those that are part of the "subclass
API" (often called "protected" in other languages). Some classes are
designed to be inherited from, either to extend or modify aspects of the
class's behavior. When designing such a class, take care to make
explicit decisions about which attributes are public, which are part of
the subclass API, and which are truly only to be used by your base
class.

With this in mind, here are the Pythonic guidelines:

-   Public attributes should have no leading underscores.

-   If your public attribute name collides with a reserved keyword,
    append a single trailing underscore to your attribute name. This is
    preferable to an abbreviation or corrupted spelling. (However,
    notwithstanding this rule, 'cls' is the preferred spelling for any
    variable or argument which is known to be a class, especially the
    first argument to a class method.)

    Note 1: See the argument name recommendation above for class
    methods.

-   For simple public data attributes, it is best to expose just the
    attribute name, without complicated accessor/mutator methods. Keep
    in mind that Python provides an easy path to future enhancement,
    should you find that a simple data attribute needs to grow
    functional behavior. In that case, use properties to hide functional
    implementation behind simple data attribute access syntax.

    Note 1: Try to keep the functional behavior side-effect free,
    although side-effects such as caching are generally fine.

    Note 2: Avoid using properties for computationally expensive
    operations; the attribute notation makes the caller believe that
    access is (relatively) cheap.

-   If your class is intended to be subclassed, and you have attributes
    that you do not want subclasses to use, consider naming them with
    double leading underscores and no trailing underscores. This invokes
    Python's name mangling algorithm, where the name of the class is
    mangled into the attribute name. This helps avoid attribute name
    collisions should subclasses inadvertently contain attributes with
    the same name.

    Note 1: Note that only the simple class name is used in the mangled
    name, so if a subclass chooses both the same class name and
    attribute name, you can still get name collisions.

    Note 2: Name mangling can make certain uses, such as debugging and
    __getattr__(), less convenient. However the name mangling algorithm
    is well documented and easy to perform manually.

    Note 3: Not everyone likes name mangling. Try to balance the need to
    avoid accidental name clashes with potential use by advanced
    callers.

Public and Internal Interfaces

Any backwards compatibility guarantees apply only to public interfaces.
Accordingly, it is important that users be able to clearly distinguish
between public and internal interfaces.

Documented interfaces are considered public, unless the documentation
explicitly declares them to be provisional or internal interfaces exempt
from the usual backwards compatibility guarantees. All undocumented
interfaces should be assumed to be internal.

To better support introspection, modules should explicitly declare the
names in their public API using the __all__ attribute. Setting __all__
to an empty list indicates that the module has no public API.

Even with __all__ set appropriately, internal interfaces (packages,
modules, classes, functions, attributes or other names) should still be
prefixed with a single leading underscore.

An interface is also considered internal if any containing namespace
(package, module or class) is considered internal.

Imported names should always be considered an implementation detail.
Other modules must not rely on indirect access to such imported names
unless they are an explicitly documented part of the containing module's
API, such as os.path or a package's __init__ module that exposes
functionality from submodules.

Programming Recommendations

-   Code should be written in a way that does not disadvantage other
    implementations of Python (PyPy, Jython, IronPython, Cython, Psyco,
    and such).

    For example, do not rely on CPython's efficient implementation of
    in-place string concatenation for statements in the form a += b or
    a = a + b. This optimization is fragile even in CPython (it only
    works for some types) and isn't present at all in implementations
    that don't use refcounting. In performance sensitive parts of the
    library, the ''.join() form should be used instead. This will ensure
    that concatenation occurs in linear time across various
    implementations.

-   Comparisons to singletons like None should always be done with is or
    is not, never the equality operators.

    Also, beware of writing if x when you really mean if x is not None
    -- e.g. when testing whether a variable or argument that defaults to
    None was set to some other value. The other value might have a type
    (such as a container) that could be false in a boolean context!

-   Use is not operator rather than not ... is. While both expressions
    are functionally identical, the former is more readable and
    preferred:

        # Correct:
        if foo is not None:

        # Wrong:
        if not foo is None:

-   When implementing ordering operations with rich comparisons, it is
    best to implement all six operations (__eq__, __ne__, __lt__,
    __le__, __gt__, __ge__) rather than relying on other code to only
    exercise a particular comparison.

    To minimize the effort involved, the functools.total_ordering()
    decorator provides a tool to generate missing comparison methods.

    PEP 207 indicates that reflexivity rules are assumed by Python.
    Thus, the interpreter may swap y > x with x < y, y >= x with x <= y,
    and may swap the arguments of x == y and x != y. The sort() and
    min() operations are guaranteed to use the < operator and the max()
    function uses the > operator. However, it is best to implement all
    six operations so that confusion doesn't arise in other contexts.

-   Always use a def statement instead of an assignment statement that
    binds a lambda expression directly to an identifier:

        # Correct:
        def f(x): return 2*x

        # Wrong:
        f = lambda x: 2*x

    The first form means that the name of the resulting function object
    is specifically 'f' instead of the generic '<lambda>'. This is more
    useful for tracebacks and string representations in general. The use
    of the assignment statement eliminates the sole benefit a lambda
    expression can offer over an explicit def statement (i.e. that it
    can be embedded inside a larger expression)

-   Derive exceptions from Exception rather than BaseException. Direct
    inheritance from BaseException is reserved for exceptions where
    catching them is almost always the wrong thing to do.

    Design exception hierarchies based on the distinctions that code
    catching the exceptions is likely to need, rather than the locations
    where the exceptions are raised. Aim to answer the question "What
    went wrong?" programmatically, rather than only stating that "A
    problem occurred" (see PEP 3151 for an example of this lesson being
    learned for the builtin exception hierarchy)

    Class naming conventions apply here, although you should add the
    suffix "Error" to your exception classes if the exception is an
    error. Non-error exceptions that are used for non-local flow control
    or other forms of signaling need no special suffix.

-   Use exception chaining appropriately. raise X from Y should be used
    to indicate explicit replacement without losing the original
    traceback.

    When deliberately replacing an inner exception (using
    raise X from None), ensure that relevant details are transferred to
    the new exception (such as preserving the attribute name when
    converting KeyError to AttributeError, or embedding the text of the
    original exception in the new exception message).

-   When catching exceptions, mention specific exceptions whenever
    possible instead of using a bare except: clause:

        try:
            import platform_specific_module
        except ImportError:
            platform_specific_module = None

    A bare except: clause will catch SystemExit and KeyboardInterrupt
    exceptions, making it harder to interrupt a program with Control-C,
    and can disguise other problems. If you want to catch all exceptions
    that signal program errors, use except Exception: (bare except is
    equivalent to except BaseException:).

    A good rule of thumb is to limit use of bare 'except' clauses to two
    cases:

    1.  If the exception handler will be printing out or logging the
        traceback; at least the user will be aware that an error has
        occurred.
    2.  If the code needs to do some cleanup work, but then lets the
        exception propagate upwards with raise. try...finally can be a
        better way to handle this case.

-   When catching operating system errors, prefer the explicit exception
    hierarchy introduced in Python 3.3 over introspection of errno
    values.

-   Additionally, for all try/except clauses, limit the try clause to
    the absolute minimum amount of code necessary. Again, this avoids
    masking bugs:

        # Correct:
        try:
            value = collection[key]
        except KeyError:
            return key_not_found(key)
        else:
            return handle_value(value)

        # Wrong:
        try:
            # Too broad!
            return handle_value(collection[key])
        except KeyError:
            # Will also catch KeyError raised by handle_value()
            return key_not_found(key)

-   When a resource is local to a particular section of code, use a with
    statement to ensure it is cleaned up promptly and reliably after
    use. A try/finally statement is also acceptable.

-   Context managers should be invoked through separate functions or
    methods whenever they do something other than acquire and release
    resources:

        # Correct:
        with conn.begin_transaction():
            do_stuff_in_transaction(conn)

        # Wrong:
        with conn:
            do_stuff_in_transaction(conn)

    The latter example doesn't provide any information to indicate that
    the __enter__ and __exit__ methods are doing something other than
    closing the connection after a transaction. Being explicit is
    important in this case.

-   Be consistent in return statements. Either all return statements in
    a function should return an expression, or none of them should. If
    any return statement returns an expression, any return statements
    where no value is returned should explicitly state this as
    return None, and an explicit return statement should be present at
    the end of the function (if reachable):

        # Correct:

        def foo(x):
            if x >= 0:
                return math.sqrt(x)
            else:
                return None

        def bar(x):
            if x < 0:
                return None
            return math.sqrt(x)

        # Wrong:

        def foo(x):
            if x >= 0:
                return math.sqrt(x)

        def bar(x):
            if x < 0:
                return
            return math.sqrt(x)

-   Use ''.startswith() and ''.endswith() instead of string slicing to
    check for prefixes or suffixes.

    startswith() and endswith() are cleaner and less error prone:

        # Correct:
        if foo.startswith('bar'):

        # Wrong:
        if foo[:3] == 'bar':

-   Object type comparisons should always use isinstance() instead of
    comparing types directly:

        # Correct:
        if isinstance(obj, int):

        # Wrong:
        if type(obj) is type(1):

-   For sequences, (strings, lists, tuples), use the fact that empty
    sequences are false:

        # Correct:
        if not seq:
        if seq:

        # Wrong:
        if len(seq):
        if not len(seq):

-   Don't write string literals that rely on significant trailing
    whitespace. Such trailing whitespace is visually indistinguishable
    and some editors (or more recently, reindent.py) will trim them.

-   Don't compare boolean values to True or False using ==:

        # Correct:
        if greeting:

        # Wrong:
        if greeting == True:

    Worse:

        # Wrong:
        if greeting is True:

-   Use of the flow control statements return/break/continue within the
    finally suite of a try...finally, where the flow control statement
    would jump outside the finally suite, is discouraged. This is
    because such statements will implicitly cancel any active exception
    that is propagating through the finally suite:

        # Wrong:
        def foo():
            try:
                1 / 0
            finally:
                return 42

Function Annotations

With the acceptance of PEP 484, the style rules for function annotations
have changed.

-   Function annotations should use PEP 484 syntax (there are some
    formatting recommendations for annotations in the previous section).

-   The experimentation with annotation styles that was recommended
    previously in this PEP is no longer encouraged.

-   However, outside the stdlib, experiments within the rules of PEP 484
    are now encouraged. For example, marking up a large third party
    library or application with PEP 484 style type annotations,
    reviewing how easy it was to add those annotations, and observing
    whether their presence increases code understandability.

-   The Python standard library should be conservative in adopting such
    annotations, but their use is allowed for new code and for big
    refactorings.

-   For code that wants to make a different use of function annotations
    it is recommended to put a comment of the form:

        # type: ignore

    near the top of the file; this tells type checkers to ignore all
    annotations. (More fine-grained ways of disabling complaints from
    type checkers can be found in PEP 484.)

-   Like linters, type checkers are optional, separate tools. Python
    interpreters by default should not issue any messages due to type
    checking and should not alter their behavior based on annotations.

-   Users who don't want to use type checkers are free to ignore them.
    However, it is expected that users of third party library packages
    may want to run type checkers over those packages. For this purpose
    PEP 484 recommends the use of stub files: .pyi files that are read
    by the type checker in preference of the corresponding .py files.
    Stub files can be distributed with a library, or separately (with
    the library author's permission) through the typeshed repo[5].

Variable Annotations

PEP 526 introduced variable annotations. The style recommendations for
them are similar to those on function annotations described above:

-   Annotations for module level variables, class and instance
    variables, and local variables should have a single space after the
    colon.

-   There should be no space before the colon.

-   If an assignment has a right hand side, then the equality sign
    should have exactly one space on both sides:

        # Correct:

        code: int

        class Point:
            coords: Tuple[int, int]
            label: str = '<unknown>'

        # Wrong:

        code:int  # No space after colon
        code : int  # Space before colon

        class Test:
            result: int=0  # No spaces around equality sign

-   Although the PEP 526 is accepted for Python 3.6, the variable
    annotation syntax is the preferred syntax for stub files on all
    versions of Python (see PEP 484 for details).

Footnotes

References

Copyright

This document has been placed in the public domain.

[1] Barry's GNU Mailman style guide
http://barry.warsaw.us/software/STYLEGUIDE.txt

[2] Hanging indentation is a type-setting style where all the lines in a
paragraph are indented except the first line. In the context of Python,
the term is used to describe a style where the opening parenthesis of a
parenthesized statement is the last non-whitespace character of the
line, with subsequent lines being indented until the closing
parenthesis.

[3] Donald Knuth's The TeXBook, pages 195 and 196.

[4] http://www.wikipedia.com/wiki/Camel_case

[5] Typeshed repo https://github.com/python/typeshed

license: Public Domain


text:
PEP: 623 Title: Remove wstr from Unicode Author: Inada Naoki
<EMAIL_ADDRESS>BDFL-Delegate: Victor Stinner
<EMAIL_ADDRESS>Discussions-To:
<EMAIL_ADDRESS>Status: Final Type: Standards Track Content-Type: text/x-rst Created:
25-Jun-2020 Python-Version: 3.10 Resolution:
<EMAIL_ADDRESS>
Abstract

PEP 393 deprecated some unicode APIs, and introduced wchar_t *wstr, and
Py_ssize_t wstr_length in the Unicode structure to support these
deprecated APIs.

This PEP is planning removal of wstr, and wstr_length with deprecated
APIs using these members by Python 3.12.

Deprecated APIs which doesn't use the members are out of scope because
they can be removed independently.

Motivation

Memory usage

str is one of the most used types in Python. Even most simple ASCII
strings have a wstr member. It consumes 8 bytes per string on 64-bit
systems.

Runtime overhead

To support legacy Unicode object, many Unicode APIs must call
PyUnicode_READY().

We can remove this overhead too by dropping support of legacy Unicode
object.

Simplicity

Supporting legacy Unicode object makes the Unicode implementation more
complex. Until we drop legacy Unicode object, it is very hard to try
other Unicode implementation like UTF-8 based implementation in PyPy.

Rationale

Python 4.0 is not scheduled yet

PEP 393 introduced efficient internal representation of Unicode and
removed border between "narrow" and "wide" build of Python.

PEP 393 was implemented in Python 3.3 which is released in 2012. Old
APIs were deprecated since then, and the removal was scheduled in Python
4.0.

Python 4.0 was expected as next version of Python 3.9 when PEP 393 was
accepted. But the next version of Python 3.9 is Python 3.10, not 4.0.
This is why this PEP schedule the removal plan again.

Python 2 reached EOL

Since Python 2 didn't have PEP 393 Unicode implementation, legacy APIs
might help C extension modules supporting both of Python 2 and 3.

But Python 2 reached the EOL in 2020. We can remove legacy APIs kept for
compatibility with Python 2.

Plan

Python 3.9

These macros and functions are marked as deprecated, using Py_DEPRECATED
macro.

-   Py_UNICODE_WSTR_LENGTH()
-   PyUnicode_GET_SIZE()
-   PyUnicode_GetSize()
-   PyUnicode_GET_DATA_SIZE()
-   PyUnicode_AS_UNICODE()
-   PyUnicode_AS_DATA()
-   PyUnicode_AsUnicode()
-   _PyUnicode_AsUnicode()
-   PyUnicode_AsUnicodeAndSize()
-   PyUnicode_FromUnicode()

Python 3.10

-   Following macros, enum members are marked as deprecated.
    Py_DEPRECATED(3.10) macro are used as possible. But they are
    deprecated only in comment and document if the macro can not be used
    easily.
    -   PyUnicode_WCHAR_KIND
    -   PyUnicode_READY()
    -   PyUnicode_IS_READY()
    -   PyUnicode_IS_COMPACT()
-   PyUnicode_FromUnicode(NULL, size) and
    PyUnicode_FromStringAndSize(NULL, size) emit DeprecationWarning when
    size > 0.
-   PyArg_ParseTuple() and PyArg_ParseTupleAndKeywords() emit
    DeprecationWarning when u, u#, Z, and Z# formats are used.

Python 3.12

-   Following members are removed from the Unicode structures:
    -   wstr
    -   wstr_length
    -   state.compact
    -   state.ready
-   The PyUnicodeObject structure is removed.
-   Following macros and functions, and enum members are removed:
    -   Py_UNICODE_WSTR_LENGTH()
    -   PyUnicode_GET_SIZE()
    -   PyUnicode_GetSize()
    -   PyUnicode_GET_DATA_SIZE()
    -   PyUnicode_AS_UNICODE()
    -   PyUnicode_AS_DATA()
    -   PyUnicode_AsUnicode()
    -   _PyUnicode_AsUnicode()
    -   PyUnicode_AsUnicodeAndSize()
    -   PyUnicode_FromUnicode()
    -   PyUnicode_WCHAR_KIND
    -   PyUnicode_READY()
    -   PyUnicode_IS_READY()
    -   PyUnicode_IS_COMPACT()
-   PyUnicode_FromStringAndSize(NULL, size)) raises RuntimeError when
    size > 0.
-   PyArg_ParseTuple() and PyArg_ParseTupleAndKeywords() raise
    SystemError when u, u#, Z, and Z# formats are used, as other
    unsupported format character.

Discussion

-   Draft PEP: Remove wstr from Unicode
-   When can we remove wchar_t* cache from string?
-   PEP 623: Remove wstr from Unicode object #1462

References

-   bpo-38604: Schedule Py_UNICODE API removal
-   bpo-36346: Prepare for removing the legacy Unicode C API
-   bpo-30863: Rewrite PyUnicode_AsWideChar() and
    PyUnicode_AsWideCharString(): They no longer cache the wchar_t*
    representation of string objects.

Copyright

This document has been placed in the public domain.


text:
PEP: 3000 Title: Python 3000 Version: $Revision$ Last-Modified: $Date$
Author: Guido van Rossum<EMAIL_ADDRESS>Status: Final Type: Process
Content-Type: text/x-rst Created: 05-Apr-2006 Post-History:

Abstract

This PEP sets guidelines for Python 3000 development. Ideally, we first
agree on the process, and start discussing features only after the
process has been decided and specified. In practice, we'll be discussing
features and process simultaneously; often the debate about a particular
feature will prompt a process discussion.

Naming

Python 3000, Python 3.0 and Py3K are all names for the same thing. The
project is called Python 3000, or abbreviated to Py3k. The actual Python
release will be referred to as Python 3.0, and that's what "python3.0
-V" will print; the actual file names will use the same naming
convention we use for Python 2.x. I don't want to pick a new name for
the executable or change the suffix for Python source files.

PEP Numbering

Python 3000 PEPs are numbered starting at PEP 3000. PEPs 3000-3099 are
meta-PEPs -- these can be either process or informational PEPs. PEPs
3100-3999 are feature PEPs. PEP 3000 itself (this PEP) is special; it is
the meta-PEP for Python 3000 meta-PEPs (IOW it describe the process to
define processes). PEP 3100 is also special; it's a laundry list of
features that were selected for (hopeful) inclusion in Python 3000
before we started the Python 3000 process for real. PEP 3099, finally,
is a list of features that will not change.

Timeline

See PEP 361, which contains the release schedule for Python 2.6 and 3.0.
These versions will be released in lockstep.

Note: standard library development is expected to ramp up after 3.0a1 is
released.

I expect that there will be parallel Python 2.x and 3.x releases for
some time; the Python 2.x releases will continue for a longer time than
the traditional 2.x.y bugfix releases. Typically, we stop releasing
bugfix versions for 2.x once version 2.(x+1) has been released. But I
expect there to be at least one or two new 2.x releases even after 3.0
(final) has been released, probably well into 3.1 or 3.2. This will to
some extent depend on community demand for continued 2.x support,
acceptance and stability of 3.0, and volunteer stamina.

I expect that Python 3.1 and 3.2 will be released much sooner after 3.0
than has been customary for the 2.x series. The 3.x release pattern will
stabilize once the community is happy with 3.x.

Compatibility and Transition

Python 3.0 will break backwards compatibility with Python 2.x.

There is no requirement that Python 2.6 code will run unmodified on
Python 3.0. Not even a subset. (Of course there will be a tiny subset,
but it will be missing major functionality.)

Python 2.6 will support forward compatibility in the following two ways:

-   It will support a "Py3k warnings mode" which will warn dynamically
    (i.e. at runtime) about features that will stop working in Python
    3.0, e.g. assuming that range() returns a list.
-   It will contain backported versions of many Py3k features, either
    enabled through __future__ statements or simply by allowing old and
    new syntax to be used side-by-side (if the new syntax would be a
    syntax error in 2.x).

Instead, and complementary to the forward compatibility features in 2.6,
there will be a separate source code conversion tool[1]. This tool can
do a context-free source-to-source translation. For example, it can
translate apply(f, args) into f(*args). However, the tool cannot do data
flow analysis or type inferencing, so it simply assumes that apply in
this example refers to the old built-in function.

The recommended development model for a project that needs to support
Python 2.6 and 3.0 simultaneously is as follows:

0.  You should have excellent unit tests with close to full coverage.
1.  Port your project to Python 2.6.
2.  Turn on the Py3k warnings mode.
3.  Test and edit until no warnings remain.
4.  Use the 2to3 tool to convert this source code to 3.0 syntax. Do not
    manually edit the output!
5.  Test the converted source code under 3.0.
6.  If problems are found, make corrections to the 2.6 version of the
    source code and go back to step 3.
7.  When it's time to release, release separate 2.6 and 3.0 tarballs (or
    whatever archive form you use for releases).

It is recommended not to edit the 3.0 source code until you are ready to
reduce 2.6 support to pure maintenance (i.e. the moment when you would
normally move the 2.6 code to a maintenance branch anyway).

PS. We need a meta-PEP to describe the transitional issues in detail.

Implementation Language

Python 3000 will be implemented in C, and the implementation will be
derived as an evolution of the Python 2 code base. This reflects my
views (which I share with Joel Spolsky[2]) on the dangers of complete
rewrites. Since Python 3000 as a language is a relatively mild
improvement on Python 2, we can gain a lot by not attempting to
reimplement the language from scratch. I am not against parallel
from-scratch implementation efforts, but my own efforts will be directed
at the language and implementation that I know best.

Meta-Contributions

Suggestions for additional text for this PEP are gracefully accepted by
the author. Draft meta-PEPs for the topics above and additional topics
are even more welcome!

References

Copyright

This document has been placed in the public domain.

  Local Variables: mode: indented-text indent-tabs-mode: nil
  sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

[1] The 2to3 tool, in the subversion sandbox
http://svn.python.org/view/sandbox/trunk/2to3/

[2] Joel on Software: Things You Should Never Do, Part I
http://www.joelonsoftware.com/articles/fog0000000069.html