text:
PEP: 207 Title: Rich Comparisons Author: Guido van Rossum
<EMAIL_ADDRESS>David Ascher<EMAIL_ADDRESS>Status: Final
Type: Standards Track Content-Type: text/x-rst Created: 25-Jul-2000
Python-Version: 2.1 Post-History:

Abstract

This PEP proposes several new features for comparisons:

-   Allow separately overloading of <, >, <=, >=, ==, !=, both in
    classes and in C extensions.
-   Allow any of those overloaded operators to return something else
    besides a Boolean result.

Motivation

The main motivation comes from NumPy, whose users agree that A<B should
return an array of elementwise comparison outcomes; they currently have
to spell this as less(A,B) because A<B can only return a Boolean result
or raise an exception.

An additional motivation is that frequently, types don't have a natural
ordering, but still need to be compared for equality. Currently such a
type must implement comparison and thus define an arbitrary ordering,
just so that equality can be tested.

Also, for some object types an equality test can be implemented much
more efficiently than an ordering test; for example, lists and
dictionaries that differ in length are unequal, but the ordering
requires inspecting some (potentially all) items.

Previous Work

Rich Comparisons have been proposed before; in particular by David
Ascher, after experience with Numerical Python:

  http://starship.python.net/crew/da/proposals/richcmp.html

It is also included below as an Appendix. Most of the material in this
PEP is derived from David's proposal.

Concerns

1.  Backwards compatibility, both at the Python level (classes using
    __cmp__ need not be changed) and at the C level (extensions defining
    tp_comparea need not be changed, code using PyObject_Compare() must
    work even if the compared objects use the new rich comparison
    scheme).
2.  When A<B returns a matrix of elementwise comparisons, an easy
    mistake to make is to use this expression in a Boolean context.
    Without special precautions, it would always be true. This use
    should raise an exception instead.
3.  If a class overrides x==y but nothing else, should x!=y be computed
    as not(x==y), or fail? What about the similar relationship between <
    and >=, or between > and <=?
4.  Similarly, should we allow x<y to be calculated from y>x? And x<=y
    from not(x>y)? And x==y from y==x, or x!=y from y!=x?
5.  When comparison operators return elementwise comparisons, what to do
    about shortcut operators like A<B<C, A<B and C<D, A<B or C<D?
6.  What to do about min() and max(), the 'in' and 'not in' operators,
    list.sort(), dictionary key comparison, and other uses of
    comparisons by built-in operations?

Proposed Resolutions

1.  Full backwards compatibility can be achieved as follows. When an
    object defines tp_compare() but not tp_richcompare(), and a rich
    comparison is requested, the outcome of tp_compare() is used in the
    obvious way. E.g. if "<" is requested, an exception if tp_compare()
    raises an exception, the outcome is 1 if tp_compare() is negative,
    and 0 if it is zero or positive. Etc.

    Full forward compatibility can be achieved as follows. When a
    classic comparison is requested on an object that implements
    tp_richcompare(), up to three comparisons are used: first == is
    tried, and if it returns true, 0 is returned; next, < is tried and
    if it returns true, -1 is returned; next, > is tried and if it
    returns true, +1 is returned. If any operator tried returns a
    non-Boolean value (see below), the exception raised by conversion to
    Boolean is passed through. If none of the operators tried returns
    true, the classic comparison fallbacks are tried next.

    (I thought long and hard about the order in which the three
    comparisons should be tried. At one point I had a convincing
    argument for doing it in this order, based on the behavior of
    comparisons for cyclical data structures. But since that code has
    changed again, I'm not so sure that it makes a difference any more.)

2.  Any type that returns a collection of Booleans instead of a single
    boolean should define nb_nonzero() to raise an exception. Such a
    type is considered a non-Boolean.

3.  The == and != operators are not assumed to be each other's
    complement (e.g. IEEE 754 floating point numbers do not satisfy
    this). It is up to the type to implement this if desired. Similar
    for < and >=, or > and <=; there are lots of examples where these
    assumptions aren't true (e.g. tabnanny).

4.  The reflexivity rules are assumed by Python. Thus, the interpreter
    may swap y>x with x<y, y>=x with x<=y, and may swap the arguments of
    x==y and x!=y. (Note: Python currently assumes that x==x is always
    true and x!=x is never true; this should not be assumed.)

5.  In the current proposal, when A<B returns an array of elementwise
    comparisons, this outcome is considered non-Boolean, and its
    interpretation as Boolean by the shortcut operators raises an
    exception. David Ascher's proposal tries to deal with this; I don't
    think this is worth the additional complexity in the code generator.
    Instead of A<B<C, you can write (A<B)&(B<C).

6.  The min() and list.sort() operations will only use the < operator;
    max() will only use the > operator. The 'in' and 'not in' operators
    and dictionary lookup will only use the == operator.

Implementation Proposal

This closely follows David Ascher's proposal.

C API

-   New functions:

        PyObject *PyObject_RichCompare(PyObject *, PyObject *, int)

    This performs the requested rich comparison, returning a Python
    object or raising an exception. The 3rd argument must be one of
    Py_LT, Py_LE, Py_EQ, Py_NE, Py_GT or Py_GE.

        int PyObject_RichCompareBool(PyObject *, PyObject *, int)

    This performs the requested rich comparison, returning a Boolean: -1
    for exception, 0 for false, 1 for true. The 3rd argument must be one
    of Py_LT, Py_LE, Py_EQ, Py_NE, Py_GT or Py_GE. Note that when
    PyObject_RichCompare() returns a non-Boolean object,
    PyObject_RichCompareBool() will raise an exception.

-   New typedef:

        typedef PyObject *(*richcmpfunc) (PyObject *, PyObject *, int);

-   New slot in type object, replacing spare tp_xxx7:

        richcmpfunc tp_richcompare;

    This should be a function with the same signature as
    PyObject_RichCompare(), and performing the same comparison. At least
    one of the arguments is of the type whose tp_richcompare slot is
    being used, but the other may have a different type. If the function
    cannot compare the particular combination of objects, it should
    return a new reference to Py_NotImplemented.

-   PyObject_Compare() is changed to try rich comparisons if they are
    defined (but only if classic comparisons aren't defined).

Changes to the interpreter

-   Whenever PyObject_Compare() is called with the intent of getting the
    outcome of a particular comparison (e.g. in list.sort(), and of
    course for the comparison operators in ceval.c), the code is changed
    to call PyObject_RichCompare() or PyObject_RichCompareBool()
    instead; if the C code needs to know the outcome of the comparison,
    PyObject_IsTrue() is called on the result (which may raise an
    exception).
-   Most built-in types that currently define a comparison will be
    modified to define a rich comparison instead. (This is optional;
    I've converted lists, tuples, complex numbers, and arrays so far,
    and am not sure whether I will convert others.)

Classes

-   Classes can define new special methods __lt__, __le__, __eq__,
    __ne__, __gt__, __ge__ to override the corresponding operators.
    (I.e., <, <=, ==, !=, >, >=. You gotta love the Fortran heritage.)
    If a class defines __cmp__ as well, it is only used when __lt__ etc.
    have been tried and return NotImplemented.

Copyright

This document has been placed in the public domain.

Appendix

Here is most of David Ascher's original proposal (version 0.2.1, dated
Wed Jul 22 16:49:28 1998; I've left the Contents, History and Patches
sections out). It addresses almost all concerns above.

Abstract

A new mechanism allowing comparisons of Python objects to return values
other than -1, 0, or 1 (or raise exceptions) is proposed. This mechanism
is entirely backwards compatible, and can be controlled at the level of
the C PyObject type or of the Python class definition. There are three
cooperating parts to the proposed mechanism:

-   the use of the last slot in the type object structure to store a
    pointer to a rich comparison function
-   the addition of special methods for classes
-   the addition of an optional argument to the builtin cmp() function.

Motivation

The current comparison protocol for Python objects assumes that any two
Python objects can be compared (as of Python 1.5, object comparisons can
raise exceptions), and that the return value for any comparison should
be -1, 0 or 1. -1 indicates that the first argument to the comparison
function is less than the right one, +1 indicating the contrapositive,
and 0 indicating that the two objects are equal. While this mechanism
allows the establishment of an order relationship (e.g. for use by the
sort() method of list objects), it has proven to be limited in the
context of Numeric Python (NumPy).

Specifically, NumPy allows the creation of multidimensional arrays,
which support most of the numeric operators. Thus:

    x = array((1,2,3,4))        y = array((2,2,4,4))

are two NumPy arrays. While they can be added elementwise,:

    z = x + y   # z == array((3,4,7,8))

they cannot be compared in the current framework - the released version
of NumPy compares the pointers, (thus yielding junk information) which
was the only solution before the recent addition of the ability (in 1.5)
to raise exceptions in comparison functions.

Even with the ability to raise exceptions, the current protocol makes
array comparisons useless. To deal with this fact, NumPy includes
several functions which perform the comparisons: less(), less_equal(),
greater(), greater_equal(), equal(), not_equal(). These functions return
arrays with the same shape as their arguments (modulo broadcasting),
filled with 0's and 1's depending on whether the comparison is true or
not for each element pair. Thus, for example, using the arrays x and y
defined above:

    less(x,y)

would be an array containing the numbers (1,0,0,0).

The current proposal is to modify the Python object interface to allow
the NumPy package to make it so that x < y returns the same thing as
less(x,y). The exact return value is up to the NumPy package -- what
this proposal really asks for is changing the Python core so that
extension objects have the ability to return something other than -1, 0,
1, should their authors choose to do so.

Current State of Affairs

The current protocol is, at the C level, that each object type defines a
tp_compare slot, which is a pointer to a function which takes two
PyObject* references and returns -1, 0, or 1. This function is called by
the PyObject_Compare() function defined in the C API. PyObject_Compare()
is also called by the builtin function cmp() which takes two arguments.

Proposed Mechanism

1.  Changes to the C structure for type objects

    The last available slot in the PyTypeObject, reserved up to now for
    future expansion, is used to optionally store a pointer to a new
    comparison function, of type richcmpfunc defined by:

        typedef PyObject *(*richcmpfunc)
             Py_PROTO((PyObject *, PyObject *, int));

    This function takes three arguments. The first two are the objects
    to be compared, and the third is an integer corresponding to an
    opcode (one of LT, LE, EQ, NE, GT, GE). If this slot is left NULL,
    then rich comparison for that object type is not supported (except
    for class instances whose class provide the special methods
    described below).

    The above opcodes need to be added to the published Python/C API
    (probably under the names Py_LT, Py_LE, etc.)

2.  Additions of special methods for classes

    Classes wishing to support the rich comparison mechanisms must add
    one or more of the following new special methods:

        def __lt__(self, other):
           ...
        def __le__(self, other):
           ...
        def __gt__(self, other):
           ...
        def __ge__(self, other):
           ...
        def __eq__(self, other):
           ...
        def __ne__(self, other):
           ...

    Each of these is called when the class instance is the on the
    left-hand-side of the corresponding operators (<, <=, >, >=, ==, and
    != or <>). The argument other is set to the object on the right side
    of the operator. The return value of these methods is up to the
    class implementor (after all, that's the entire point of the
    proposal).

    If the object on the left side of the operator does not define an
    appropriate rich comparison operator (either at the C level or with
    one of the special methods, then the comparison is reversed, and the
    right hand operator is called with the opposite operator, and the
    two objects are swapped. This assumes that a < b and b > a are
    equivalent, as are a <= b and b >= a, and that == and != are
    commutative (e.g. a == b if and only if b == a).

    For example, if obj1 is an object which supports the rich comparison
    protocol and x and y are objects which do not support the rich
    comparison protocol, then obj1 < x will call the __lt__ method of
    obj1 with x as the second argument. x < obj1 will call obj1's __gt__
    method with x as a second argument, and x < y will just use the
    existing (non-rich) comparison mechanism.

    The above mechanism is such that classes can get away with not
    implementing either __lt__ and __le__ or __gt__ and __ge__. Further
    smarts could have been added to the comparison mechanism, but this
    limited set of allowed "swaps" was chosen because it doesn't require
    the infrastructure to do any processing (negation) of return values.
    The choice of six special methods was made over a single (e.g.
    __richcmp__) method to allow the dispatching on the opcode to be
    performed at the level of the C implementation rather than the
    user-defined method.

3.  Addition of an optional argument to the builtin cmp()

    The builtin cmp() is still used for simple comparisons. For rich
    comparisons, it is called with a third argument, one of "<", "<=",
    ">", ">=", "==", "!=", "<>" (the last two have the same meaning).
    When called with one of these strings as the third argument, cmp()
    can return any Python object. Otherwise, it can only return -1, 0 or
    1 as before.

Chained Comparisons

Problem

It would be nice to allow objects for which the comparison returns
something other than -1, 0, or 1 to be used in chained comparisons, such
as:

    x < y < z

Currently, this is interpreted by Python as:

    temp1 = x < y
    if temp1:
      return y < z
    else:
      return temp1

Note that this requires testing the truth value of the result of
comparisons, with potential "shortcutting" of the right-side comparison
testings. In other words, the truth-value of the result of the result of
the comparison determines the result of a chained operation. This is
problematic in the case of arrays, since if x, y and z are three arrays,
then the user expects:

    x < y < z

to be an array of 0's and 1's where 1's are in the locations
corresponding to the elements of y which are between the corresponding
elements in x and z. In other words, the right-hand side must be
evaluated regardless of the result of x < y, which is incompatible with
the mechanism currently in use by the parser.

Solution

Guido mentioned that one possible way out would be to change the code
generated by chained comparisons to allow arrays to be chained-compared
intelligently. What follows is a mixture of his idea and my suggestions.
The code generated for x < y < z would be equivalent to:

    temp1 = x < y
    if temp1:
      temp2 = y < z
      return boolean_combine(temp1, temp2)
    else:
      return temp1

where boolean_combine is a new function which does something like the
following:

    def boolean_combine(a, b):
        if hasattr(a, '__boolean_and__') or \
           hasattr(b, '__boolean_and__'):
            try:
                return a.__boolean_and__(b)
            except:
                return b.__boolean_and__(a)
        else: # standard behavior
            if a:
                return b
            else:
                return 0

where the __boolean_and__ special method is implemented for C-level
types by another value of the third argument to the richcmp function.
This method would perform a boolean comparison of the arrays (currently
implemented in the umath module as the logical_and ufunc).

Thus, objects returned by rich comparisons should always test true, but
should define another special method which creates boolean combinations
of them and their argument.

This solution has the advantage of allowing chained comparisons to work
for arrays, but the disadvantage that it requires comparison arrays to
always return true (in an ideal world, I'd have them always raise an
exception on truth testing, since the meaning of testing "if a>b:" is
massively ambiguous.

The inlining already present which deals with integer comparisons would
still apply, resulting in no performance cost for the most common cases.

authors:
David Ascher
Guido van Rossum


text:
PEP: 3108 Title: Standard Library Reorganization Version: $Revision$
Last-Modified: $Date$ Author: Brett Cannon<EMAIL_ADDRESS>Status:
Final Type: Standards Track Content-Type: text/x-rst Created:
01-Jan-2007 Python-Version: 3.0 Post-History: 28-Apr-2008

Note

The merging of profile/cProfile as of Python 3.3 did not occur, and thus
is considered abandoned (although it would be acceptable to do in the
future).

Abstract

Just like the language itself, Python's standard library (stdlib) has
grown over the years to be very rich. But over time some modules have
lost their need to be included with Python. There has also been an
introduction of a naming convention for modules since Python's inception
that not all modules follow.

Python 3.0 presents a chance to remove modules that do not have long
term usefulness. This chance also allows for the renaming of modules so
that they follow the Python style guide <8>. This PEP lists modules that
should not be included in Python 3.0 or which need to be renamed.

Modules to Remove

Guido pronounced that "silly old stuff" is to be deleted from the stdlib
for Py3K[1]. This is open-ended on purpose. Each module to be removed
needs to have a justification as to why it should no longer be
distributed with Python. This can range from the module being deprecated
in Python 2.x to being for a platform that is no longer widely used.

This section of the PEP lists the various modules to be removed. Each
subsection represents a different reason for modules to be removed. Each
module must have a specific justification on top of being listed in a
specific subsection so as to make sure only modules that truly deserve
to be removed are in fact removed.

When a reason mentions how long it has been since a module has been
"uniquely edited", it is in reference to how long it has been since a
checkin was done specifically for the module and not for a change that
applied universally across the entire stdlib. If an edit time is not
denoted as "unique" then it is the last time the file was edited,
period.

Previously deprecated [done]

PEP 4 lists all modules that have been deprecated in the stdlib. The
specified motivations mirror those listed in PEP 4. All modules listed
in the PEP at the time of the first alpha release of Python 3.0 will be
removed.

The entire contents of lib-old will also be removed. These modules have
already been removed from being imported but are kept in the
distribution for Python for users that rely upon the code.

-   cfmfile
    -   Documented as deprecated since Python 2.4 without an explicit
        reason.
-   cl
    -   Documented as obsolete since Python 2.0 or earlier.
    -   Interface to SGI hardware.
-   md5
    -   Supplanted by the hashlib module.
-   mimetools
    -   Documented as obsolete in a previous version.
    -   Supplanted by the email package.
-   MimeWriter
    -   Supplanted by the email package.
-   mimify
    -   Supplanted by the email package.
-   multifile
    -   Supplanted by the email package.
-   posixfile
    -   Locking is better done by fcntl.lockf().
-   rfc822
    -   Supplanted by the email package.
-   sha
    -   Supplanted by the hashlib package.
-   sv
    -   Documented as obsolete since Python 2.0 or earlier.
    -   Interface to obsolete SGI Indigo hardware.
-   timing
    -   Documented as obsolete since Python 2.0 or earlier.
    -   time.clock() gives better time resolution.

Platform-specific with minimal use [done]

Python supports many platforms, some of which are not widely used or
maintained. And on some of these platforms there are modules that have
limited use to people on those platforms. Because of their limited
usefulness it would be better to no longer burden the Python development
team with their maintenance.

The modules mentioned below are documented. All undocumented modules for
the specified platforms will also be removed.

IRIX

The IRIX operating system is no longer produced[2]. Removing all modules
from the plat-irix[56] directory has been deemed reasonable because of
this fact.

-   AL/al
    -   Provides sound support on Indy and Indigo workstations.
    -   Both workstations are no longer available.
    -   Code has not been uniquely edited in three years.
-   cd/CD
    -   CD drive control for SGI systems.
    -   SGI no longer sells machines with IRIX on them.
    -   Code has not been uniquely edited in 14 years.
-   cddb
    -   Undocumented.
-   cdplayer
    -   Undocumented.
-   cl/CL/CL_old
    -   Compression library for SGI systems.
    -   SGI no longer sells machines with IRIX on them.
    -   Code has not been uniquely edited in 14 years.
-   DEVICE/GL/gl/cgen/cgensuport
    -   GL access, which is the predecessor to OpenGL.
    -   Has not been edited in at least eight years.
    -   Third-party libraries provide better support (PyOpenGL[3]).
-   ERRNO
    -   Undocumented.
-   FILE
    -   Undocumented.
-   FL/fl/flp
    -   Wrapper for the FORMS library[4]
    -   FORMS has not been edited in 12 years.
    -   Library is not widely used.
    -   First eight hits on Google are for Python docs for fl.
-   fm
    -   Wrapper to the IRIS Font Manager library.
    -   Only available on SGI machines which no longer come with IRIX.
-   GET
    -   Undocumented.
-   GLWS
    -   Undocumented.
-   imgfile
    -   Wrapper for SGI libimage library for imglib image files (.rgb
        files).
    -   Python Imaging Library provides read-only support[5].
    -   Not uniquely edited in 13 years.
-   IN
    -   Undocumented.
-   IOCTL
    -   Undocumented.
-   jpeg
    -   Wrapper for JPEG (de)compressor.
    -   Code not uniquely edited in nine years.
    -   Third-party libraries provide better support (Python Imaging
        Library[6]).
-   panel
    -   Undocumented.
-   panelparser
    -   Undocumented.
-   readcd
    -   Undocumented.
-   SV
    -   Undocumented.
-   torgb
    -   Undocumented.
-   WAIT
    -   Undocumented.

Mac-specific modules

The Mac-specific modules are not well-maintained (e.g., the bgen tool
used to auto-generate many of the modules has never been updated to
support UCS-4). It is also not Python's place to maintain such a large
amount of OS-specific modules. Thus all modules under Lib/plat-mac and
Mac are to be removed.

A stub module for proxy access will be provided for use by urllib.

-   _builtinSuites
    -   Undocumented.
    -   Package under lib-scriptpackages.
-   Audio_mac
    -   Undocumented.
-   aepack
    -   OSA support is better through third-party modules.
        -   Appscript[7].
    -   Hard-coded endianness which breaks on Intel Macs.
    -   Might need to rename if Carbon package dependent.
-   aetools
    -   See aepack.
-   aetypes
    -   See aepack.
-   applesingle
    -   Undocumented.
    -   AppleSingle is a binary file format for A/UX.
    -   A/UX no longer distributed.
-   appletrawmain
    -   Undocumented.
-   appletrunner
    -   Undocumented.
-   argvemulator
    -   Undocumented.
-   autoGIL
    -   Very bad model for using Python with the CFRunLoop.
-   bgenlocations
    -   Undocumented.
-   buildtools
    -   Documented as deprecated since Python 2.3 without an explicit
        reason.
-   bundlebuilder
    -   Undocumented.
-   Carbon
    -   Carbon development has stopped.
    -   Does not support 64-bit systems completely.
    -   Dependent on bgen which has never been updated to support UCS-4
        Unicode builds of Python.
-   CodeWarrior
    -   Undocumented.
    -   Package under lib-scriptpackages.
-   ColorPicker
    -   Better to use Cocoa for GUIs.
-   EasyDialogs
    -   Better to use Cocoa for GUIs.
-   Explorer
    -   Undocumented.
    -   Package under lib-scriptpackages.
-   Finder
    -   Undocumented.
    -   Package under lib-scriptpackages.
-   findertools
    -   No longer useful.
-   FrameWork
    -   Poorly documented.
    -   Not updated to support Carbon Events.
-   gensuitemodule
    -   See aepack.
-   ic
-   icglue
-   icopen
    -   Not needed on OS X.
    -   Meant to replace 'open' which is usually a bad thing to do.
-   macerrors
    -   Undocumented.
-   MacOS
    -   Would also mean the removal of binhex.
-   macostools
-   macresource
    -   Undocumented.
-   MiniAEFrame
    -   See aepack.
-   Nav
    -   Undocumented.
-   Netscape
    -   Undocumented.
    -   Package under lib-scriptpackages.
-   OSATerminology
-   pimp
    -   Undocumented.
-   PixMapWrapper
    -   Undocumented.
-   StdSuites
    -   Undocumented.
    -   Package under lib-scriptpackages.
-   SystemEvents
    -   Undocumented.
    -   Package under lib-scriptpackages.
-   Terminal
    -   Undocumented.
    -   Package under lib-scriptpackages.
-   terminalcommand
    -   Undocumented.
-   videoreader
    -   No longer used.
-   W
    -   No longer distributed with Python.

Solaris

-   SUNAUDIODEV/sunaudiodev
    -   Access to the sound card on Sun machines.
    -   Code not uniquely edited in over eight years.

Hardly used [done]

Some platform-independent modules are rarely used. There are a number of
possible explanations for this, including, ease of reimplementation,
very small audience or lack of adherence to more modern standards.

-   audiodev
    -   Undocumented.
    -   Not edited in five years.
-   imputil
    -   Undocumented.
    -   Never updated to support absolute imports.
-   mutex
    -   Easy to implement using a semaphore and a queue.
    -   Cannot block on a lock attempt.
    -   Not uniquely edited since its addition 15 years ago.
    -   Only useful with the 'sched' module.
    -   Not thread-safe.
-   stringold
    -   Function versions of the methods on string objects.
    -   Obsolete since Python 1.6.
    -   Any functionality not in the string object or module will be
        moved to the string module (mostly constants).
-   sunaudio
    -   Undocumented.
    -   Not edited in over seven years.
    -   The sunau module provides similar abilities.
-   toaiff
    -   Undocumented.
    -   Requires sox library to be installed on the system.
-   user
    -   Easily handled by allowing the application specify its own
        module name, check for existence, and import if found.
-   new
    -   Just a rebinding of names from the 'types' module.
    -   Can also call type built-in to get most types easily.
    -   Docstring states the module is no longer useful as of revision
        27241 (2002-06-15).
-   pure
    -   Written before Pure Atria was bought by Rational which was then
        bought by IBM (in other words, very old).
-   test.testall
    -   From the days before regrtest.

Obsolete

Becoming obsolete signifies that either another module in the stdlib or
a widely distributed third-party library provides a better solution for
what the module is meant for.

-   Bastion/rexec [done]
    -   Restricted execution / security.
    -   Turned off in Python 2.3.
    -   Modules deemed unsafe.
-   bsddb185 [done]
    -   Superseded by bsddb3
    -   Not built by default.
    -   Documentation specifies that the "module should never be used
        directly in new code".
    -   Available externally from PyPI.
-   Canvas [done]
    -   Marked as obsolete in a comment by Guido since 2000 (see
        http://bugs.python.org/issue210677).
    -   Better to use the Tkinter.Canvas class.
-   commands [done]
    -   subprocess module replaces it (PEP 324).
    -   Remove getstatus(), move rest to subprocess.
-   compiler [done]
    -   Having to maintain both the built-in compiler and the stdlib
        package is redundant[8].
    -   The AST created by the compiler is available[9].
    -   Mechanism to compile from an AST needs to be added.
-   dircache [done]
    -   Negligible use.
    -   Easily replicated.
-   dl [done]
    -   ctypes provides better support for same functionality.
-   fpformat [done]
    -   All functionality is supported by string interpolation.
-   htmllib [done]
    -   Superseded by HTMLParser.
-   ihooks [done]
    -   Undocumented.
    -   For use with rexec which has been turned off since Python 2.3.
-   imageop [done]
    -   Better support by third-party libraries (Python Imaging
        Library[10]).

    -

        Unit tests relied on rgbimg and imgfile.

            -   rgbimg was removed in Python 2.6.
            -   imgfile slated for removal in this PEP.
-   linuxaudiodev [done]
    -   Replaced by ossaudiodev.
-   mhlib [done]
    -   Should be removed as an individual module; use mailbox instead.
-   popen2 [done]
    -   subprocess module replaces it (PEP 324).
-   sgmllib [done]
    -   Does not fully parse SGML.
    -   In the stdlib for support to htmllib which is slated for
        removal.
-   sre [done]
    -   Previously deprecated; import re instead.
-   stat [TODO need to move all uses over to os.stat()]
    -   os.stat() now returns a tuple with attributes.
    -   Functions in the module should be made into methods for the
        object returned by os.stat.
-   statvfs [done]
    -   os.statvfs now returns a tuple with attributes.
-   thread [done]
    -   People should use 'threading' instead.
        -   Rename 'thread' to _thread.
        -   Deprecate dummy_thread and rename _dummy_thread.
        -   Move thread.get_ident over to threading.
    -   Guido has previously supported the deprecation [11].
-   urllib [done]
    -   Superseded by urllib2.
    -   Functionality unique to urllib will be kept in the urllib
        package.
-   UserDict [done: 3.0] [TODO handle 2.6]
    -   Not as useful since types can be a superclass.
    -   Useful bits moved to the 'collections' module.
-   UserList/UserString [done]
    -   Not useful since types can be a superclass.
    -   Moved to the 'collections' module.

Maintenance Burden

Over the years, certain modules have become a heavy burden upon
python-dev to maintain. In situations like this, it is better for the
module to be given to the community to maintain to free python-dev to
focus more on language support and other modules in the standard library
that do not take up an undue amount of time and effort.

-   bsddb3
    -   Externally maintained at
        http://www.jcea.es/programacion/pybsddb.htm .
    -   Consistent testing instability.
    -   Berkeley DB follows a different release schedule than Python,
        leading to the bindings not necessarily being in sync with what
        is available.

Modules to Rename

Many modules existed in the stdlib before PEP 8 came into existence.
This has led to some naming inconsistencies and namespace bloat that
should be addressed.

PEP 8 violations [done]

PEP 8 specifies that modules "should have short, all-lowercase names"
where "underscores can be used ... if it improves readability". The use
of underscores is discouraged in package names. The following modules
violate PEP 8 and are not somehow being renamed by being moved to a
package.

  Current Name   Replacement Name
  -------------- ------------------
  _winreg        winreg
  ConfigParser   configparser
  copy_reg       copyreg
  Queue          queue
  SocketServer   socketserver

Merging C and Python implementations of the same interface

Several interfaces have both a Python and C implementation. While it is
great to have a C implementation for speed with a Python implementation
as fallback, there is no need to expose the two implementations
independently in the stdlib. For Python 3.0 all interfaces with two
implementations will be merged into a single public interface.

The C module is to be given a leading underscore to delineate the fact
that it is not the reference implementation (the Python implementation
is). This means that any semantic difference between the C and Python
versions must be dealt with before Python 3.0 or else the C
implementation will be removed until it can be fixed.

One interface that is not listed below is xml.etree.ElementTree. This is
an externally maintained module and thus is not under the direct control
of the Python development team for renaming. See Open Issues for a
discussion on this.

-   pickle/cPickle [done]
    -   Rename cPickle to _pickle.
    -   Semantic completeness of C implementation not verified.
-   profile/cProfile [TODO]
    -   Rename cProfile to _profile.
    -   Semantic completeness of C implementation not verified.
-   StringIO/cStringIO [done]
    -   Add the class to the 'io' module.

No public, documented interface [done]

There are several modules in the stdlib that have no defined public
interface. These modules exist as support code for other modules that
are exposed. Because they are not meant to be used directly they should
be renamed to reflect this fact.

  Current Name   Replacement Name
  -------------- ------------------
  markupbase     _markupbase

Poorly chosen names [done]

A few modules have names that were poorly chosen in hindsight. They
should be renamed so as to prevent their bad name from perpetuating
beyond the 2.x series.

  Current Name        Replacement Name
  ------------------- ------------------
  repr                reprlib
  test.test_support   test.support

Grouping of modules [done]

As the stdlib has grown, several areas within it have expanded to
include multiple modules (e.g., support for database files). It thus
makes sense to group related modules into packages.

dbm package

  Current Name   Replacement Name
  -------------- ------------------
  anydbm         dbm.__init__[12]
  dbhash         dbm.bsd
  dbm            dbm.ndbm
  dumbdm         dbm.dumb
  gdbm           dbm.gnu
  whichdb        dbm.__init__[13]

html package

  Current Name     Replacement Name
  ---------------- ------------------
  HTMLParser       html.parser
  htmlentitydefs   html.entities

http package

  Current Name       Replacement Name
  ------------------ ------------------
  httplib            http.client
  BaseHTTPServer     http.server[14]
  CGIHTTPServer      http.server[15]
  SimpleHTTPServer   http.server[16]
  Cookie             http.cookies
  cookielib          http.cookiejar

tkinter package

  Current Name     Replacement Name
  ---------------- --------------------------
  Dialog           tkinter.dialog
  FileDialog       tkinter.filedialog[17]
  FixTk            tkinter._fix
  ScrolledText     tkinter.scrolledtext
  SimpleDialog     tkinter.simpledialog[18]
  Tix              tkinter.tix
  Tkconstants      tkinter.constants
  Tkdnd            tkinter.dnd
  Tkinter          tkinter.__init__
  tkColorChooser   tkinter.colorchooser
  tkCommonDialog   tkinter.commondialog
  tkFileDialog     tkinter.filedialog[19]
  tkFont           tkinter.font
  tkMessageBox     tkinter.messagebox
  tkSimpleDialog   tkinter.simpledialog[20]
  turtle           tkinter.turtle

urllib package

Originally this new package was to be named url, but because of the
common use of the name as a variable, it has been deemed better to keep
the name urllib and instead shift existing modules around into a new
package.

  Current Name   Replacement Name
  -------------- ------------------------------------------------
  urllib2        urllib.request, urllib.error
  urlparse       urllib.parse
  urllib         urllib.parse, urllib.request, urllib.error[21]
  robotparser    urllib.robotparser

xmlrpc package

  Current Name         Replacement Name
  -------------------- -------------------
  xmlrpclib            xmlrpc.client
  DocXMLRPCServer      xmlrpc.server[22]
  SimpleXMLRPCServer   xmlrpc.server[23]

Transition Plan

Issues

Issues related to this PEP:

-   Issue 2775: Master tracking issue
-   Issue 2828: clean up undoc.rst

For modules to be removed

For module removals, it is easiest to remove the module first in Python
3.0 to see where dependencies exist. This makes finding code that
(possibly) requires the suppression of the DeprecationWarning easier.

In Python 3.0

1.  Remove the module.
2.  Remove related tests.
3.  Remove all documentation (typically the module's documentation file
    and its entry in a file for the Library Reference).
4.  Edit Modules/Setup.dist and setup.py if needed.
5.  Run the regression test suite (using -uall); watch out for tests
    that are skipped because an import failed for the removed module.
6.  Check in the change (with an appropriate Misc/NEWS entry).
7.  Update this PEP noting that the 3.0 step is done.

In Python 2.6

1.  Add the following code to the deprecated module if it is implemented
    in Python as the first piece of executed code (adjusting the module
    name and the warnings import and needed):

        from warnings import warnpy3k
        warnpy3k("the XXX module has been removed in Python 3.0",
                 stacklevel=2)
        del warnpy3k

    or the following if it is an extension module:

        if (PyErr_WarnPy3k("the XXX module has been removed in "
                           "Python 3.0", 2) < 0)
            return;

    (the Python-Dev TextMate bundle, available from Misc/TextMate,
    contains a command that will generate all of this for you).

2.  Update the documentation. For modules with their own documentation
    file, use the :deprecated: option with the module directive along
    with the deprecated directive, stating the deprecation is occurring
    in 2.6, but is for the module's removal in 3.0.:

        .. deprecated:: 2.6
           The :mod:`XXX` module has been removed in Python 3.0.

    For modules simply listed in a file (e.g., undoc.rst), use the
    warning directive.

3.  Add the module to the module deletion test in test_py3kwarn.

4.

    Suppress the warning in the module's test code using

        test.test_support.import_module(name, deprecated=True).

5.  Check in the change w/ appropriate Misc/NEWS entry (block this
    checkin in py3k!).

6.  Update this PEP noting that the 2.6 step is done.

Renaming of modules

Support in the 2to3 refactoring tool for renames will be used to help
people transition to new module names [24]. Import statements will be
rewritten so that only the import statement and none of the rest of the
code needs to be touched. This will be accomplished by using the as
keyword in import statements to bind in the module namespace to the old
name while importing based on the new name (when the keyword is not
already used, otherwise the reassigned name should be left alone and
only the module that is imported needs to be changed). The fix_imports
fixer is an example of how to approach this.

Python 3.0

1.  Update 2to3 in the sandbox to support the rename.
2.  Use svn move to rename the module.
3.  Update all import statements in the stdlib to use the new name (use
    2to3's fix_imports fixer for the easiest solution).
4.  Rename the module in its own documentation.
5.  Update all references in the documentation from the old name to the
    new name.
6.  Run regrtest.py -uall to verify the rename worked.
7.  Add an entry in Misc/NEWS.
8.  Commit the changes.

Python 2.6

1.  In the module's documentation, add a note mentioning that the module
    is renamed in Python 3.0:

        .. note::
           The :mod:`OLDNAME` module has been renamed to :mod:`NEWNAME` in
           Python 3.0.

2.  Commit the documentation change.

3.  Block the revision in py3k.

Open Issues

Renaming of modules maintained outside of the stdlib

xml.etree.ElementTree not only does not meet PEP 8 naming standards but
it also has an exposed C implementation. It is an externally maintained
package, though PEP 360. A request will be made for the maintainer to
change the name so that it matches PEP 8 and hides the C implementation.

Rejected Ideas

Modules that were originally suggested for removal

-   asynchat/asyncore
    -   Josiah Carlson has said he will maintain the modules.
-   audioop/sunau/aifc
    -   Audio modules where the formats are still used.
-   base64/quopri/uu
    -   All still widely used.
    -   'codecs' module does not provide as nice of an API for basic
        usage.
-   fileinput
    -   Useful when having to work with stdin.
-   linecache
    -   Used internally in several places.
-   nis
    -   Testimonials from people that new installations of NIS are still
        occurring
-   getopt
    -   Simpler than optparse.
-   repr
    -   Useful as a basis for overriding.
    -   Used internally.
-   sched
    -   Useful for simulations.
-   symtable/_symtable
    -   Docs were written.
-   telnetlib
    -   Really handy for quick-and-dirty remote access.
    -   Some hardware supports using telnet for configuration and
        querying.
-   Tkinter
    -   Would prevent IDLE from existing.
    -   No GUI toolkit would be available out of the box.

Introducing a new top-level package

It has been suggested that the entire stdlib be placed within its own
package. This PEP will not address this issue as it has its own design
issues (naming, does it deserve special consideration in import
semantics, etc.). Everything within this PEP can easily be handled if a
new top-level package is introduced.

Introducing new packages to contain theme-related modules

During the writing of this PEP it was noticed that certain themes
appeared in the stdlib. In the past people have suggested introducing
new packages to help collect modules that share a similar theme (e.g.,
audio). An Open Issue was created to suggest some new packages to
introduce.

In the end, though, not enough support could be pulled together to
warrant moving forward with the idea. Instead name simplification has
been chosen as the guiding force for PEPs to create.

References

Copyright

This document has been placed in the public domain.

  Local Variables: mode: indented-text indent-tabs-mode: nil
  sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

[1] Python-Dev email: "Py3k release schedule worries"
(https://mail.python.org/pipermail/python-3000/2006-December/005130.html)

[2] SGI Press Release: End of General Availability for MIPS IRIX
Products -- December 2006 (http://www.sgi.com/support/mips_irix.html)

[3] PyOpenGL (http://pyopengl.sourceforge.net/)

[4] FORMS Library by Mark Overmars (ftp://ftp.cs.ruu.nl/pub/SGI/FORMS)

[5] Python Imaging Library (PIL)
(http://www.pythonware.com/products/pil/)

[6] Python Imaging Library (PIL)
(http://www.pythonware.com/products/pil/)

[7] appscript (http://appscript.sourceforge.net/)

[8] python-dev email: getting compiler package failures
(https://mail.python.org/pipermail/python-3000/2007-May/007615.html)

[9] _ast module (http://docs.python.org/library/ast.html)

[10] Python Imaging Library (PIL)
(http://www.pythonware.com/products/pil/)

[11] Python-Dev email: Autoloading?
(https://mail.python.org/pipermail/python-dev/2005-October/057244.html)

[12] dbm.__init__ can combine anybdbm and whichdb since the public API
for both modules has no name conflict and the two modules have closely
related usage.

[13] dbm.__init__ can combine anybdbm and whichdb since the public API
for both modules has no name conflict and the two modules have closely
related usage.

[14] The http.server module can combine the specified modules safely as
they have no naming conflicts.

[15] The http.server module can combine the specified modules safely as
they have no naming conflicts.

[16] The http.server module can combine the specified modules safely as
they have no naming conflicts.

[17] tkinter.filedialog can safely combine FileDialog and tkFileDialog
as there are no naming conflicts.

[18] tkinter.simpledialog can safely combine SimpleDialog and
tkSimpleDialog have no naming conflicts.

[19] tkinter.filedialog can safely combine FileDialog and tkFileDialog
as there are no naming conflicts.

[20] tkinter.simpledialog can safely combine SimpleDialog and
tkSimpleDialog have no naming conflicts.

[21] The quoting-related functions from urllib will be added to
urllib.parse. urllib.URLOpener and urllib.FancyUrlOpener will be added
to urllib.request as long as the documentation for both modules is
updated.

[22] The modules being combined into xmlrpc.server have no naming
conflicts and thus can safely be merged.

[23] The modules being combined into xmlrpc.server have no naming
conflicts and thus can safely be merged.

[24] 2to3 refactoring tool
(http://svn.python.org/view/sandbox/trunk/2to3/)

license: Public Domain


text:
PEP: 457 Title: Notation For Positional-Only Parameters Version:
$Revision$ Last-Modified: $Date$ Author: Larry Hastings
<EMAIL_ADDRESS>Discussions-To<EMAIL_ADDRESS>Status: Final
Type: Informational Content-Type: text/x-rst Created: 08-Oct-2013

Overview

This PEP proposes a notation for positional-only parameters in Python.
Positional-only parameters are parameters without an externally-usable
name; when a function accepting positional-only parameters is called,
positional arguments are mapped to these parameters based solely on
their position.

This PEP is an Informational PEP describing the notation for use when
describing APIs that use positional-only parameters (e.g. in Argument
Clinic, or in the string representation of inspect.Signature objects). A
separate PEP, PEP 570, proposes elevation of this notation to full
Python syntax.

Rationale

Python has always supported positional-only parameters. Early versions
of Python lacked the concept of specifying parameters by name, so
naturally all parameters were positional-only. This changed around
Python 1.0, when all parameters suddenly became positional-or-keyword.
But, even in current versions of Python, many CPython "builtin"
functions still only accept positional-only arguments.

Functions implemented in modern Python can accept an arbitrary number of
positional-only arguments, via the variadic *args parameter. However,
there is no Python syntax to specify accepting a specific number of
positional-only parameters. Put another way, there are many builtin
functions whose signatures are simply not expressible with Python
syntax.

This PEP proposes a notation for such signatures that could form the
basis of a backwards-compatible syntax that should permit implementing
any builtin in pure Python code (see PEP 570 for that proposal).

Positional-Only Parameter Semantics In Current Python

There are many, many examples of builtins that only accept
positional-only parameters. The resulting semantics are easily
experienced by the Python programmer--just try calling one, specifying
its arguments by name:

    >>> pow(x=5, y=3)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: pow() takes no keyword arguments

In addition, there are some functions with particularly interesting
semantics:

-   range(), which accepts an optional parameter to the left of its
    required parameter.[1]
-   dict(), whose mapping/iterator parameter is optional and
    semantically must be positional-only. Any externally visible name
    for this parameter would occlude that name going into the **kwarg
    keyword variadic parameter dict![2]

Obviously one can simulate any of these in pure Python code by accepting
(*args, **kwargs) and parsing the arguments by hand. But this results in
a disconnect between the Python function's signature and what it
actually accepts, not to mention the work of implementing said argument
parsing.

Motivation

This PEP does not propose we implement positional-only parameters in
Python. The goal of this PEP is simply to define the syntax, so that:

-   Documentation can clearly, unambiguously, and consistently express
    exactly how the arguments for a function will be interpreted.
-   The syntax is reserved for future use, in case the community decides
    someday to add positional-only parameters to the language.
-   Argument Clinic can use a variant of the syntax as part of its input
    when defining the arguments for built-in functions.

The Current State Of Documentation For Positional-Only Parameters

The documentation for positional-only parameters is incomplete and
inconsistent:

-   Some functions denote optional groups of positional-only arguments
    by enclosing them in nested square brackets.[3]
-   Some functions denote optional groups of positional-only arguments
    by presenting multiple prototypes with varying numbers of
    arguments.[4]
-   Some functions use both of the above approaches.[5][6]

One more important idea to consider: currently in the documentation
there's no way to tell whether a function takes positional-only
parameters. open() accepts keyword arguments, ord() does not, but there
is no way of telling just by reading the documentation that this is
true.

Syntax And Semantics

From the "ten-thousand foot view", and ignoring *args and **kwargs for
now, the grammar for a function definition currently looks like this:

    def name(positional_or_keyword_parameters, *, keyword_only_parameters):

Building on that perspective, the new syntax for functions would look
like this:

    def name(positional_only_parameters, /, positional_or_keyword_parameters,
             *, keyword_only_parameters):

All parameters before the / are positional-only. If / is not specified
in a function signature, that function does not accept any
positional-only parameters.

Positional-only parameters can have a default value, and if they do they
are optional. Positional-only parameters that don't have a default value
are "required" positional-only parameters.

More semantics of positional-only parameters:

-   Although positional-only parameter technically have names, these
    names are internal-only; positional-only parameters are never
    externally addressable by name. (Similarly to *args and **kwargs.)
-   If there are arguments after the /, then you must specify a comma
    after the /, just as there is a comma after the * denoting the shift
    to keyword-only parameters.
-   This syntax has no effect on *args or **kwargs.

Additional Limitations

Argument Clinic uses a form of this syntax for specifying builtins. It
imposes further limitations that are theoretically unnecessary but make
the implementation easier. Specifically:

-   A function that has positional-only parameters currently cannot have
    any other kind of parameter. (This will probably be relaxed slightly
    in the near future.)
-   Argument Clinic supports an additional syntax called "optional
    groups". An "optional group" is a sequential set of positional-only
    parameters that must be specified or not-specified as a group. If,
    for example, you define a function in Argument Clinic that takes
    four parameters, and all of them are positional-only and in one
    optional group, then when calling the function you must specify
    either zero arguments or four arguments. This is necessary to cover
    more of Python's legacy library, but is outside the scope of this
    PEP, and is not recommended for actual inclusion in the Python
    language.

Notes For A Future Implementor

If we decide to implement positional-only parameters in a future version
of Python, we'd have to do some additional work to preserve their
semantics. The problem: how do we inform a parameter that no value was
passed in for it when the function was called?

The obvious solution: add a new singleton constant to Python that is
passed in when a parameter is not mapped to an argument. I propose that
the value be called undefined, and be a singleton of a special class
called Undefined. If a positional-only parameter did not receive an
argument when called, its value would be set to undefined.

But this raises a further problem. How do can we tell the difference
between "this positional-only parameter did not receive an argument" and
"the caller passed in undefined for this parameter"?

It'd be nice to make it illegal to pass undefined in as an argument to a
function--to, say, raise an exception. But that would slow Python down,
and the "consenting adults" rule appears applicable here. So making it
illegal should probably be strongly discouraged but not outright
prevented.

However, it should be allowed (and encouraged) for user functions to
specify undefined as a default value for parameters.

Unresolved Questions

There are three types of parameters in Python:

1.  positional-only parameters,
2.  positional-or-keyword parameters, and
3.  keyword-only parameters.

Python allows functions to have both 2 and 3. And some builtins (e.g.
range) have both 1 and 3. Does it make sense to have functions that have
both 1 and 2? Or all of the above?

Thanks

Credit for the use of '/' as the separator between positional-only and
positional-or-keyword parameters goes to Guido van Rossum, in a proposal
from 2012.[7]

Credit for making left option groups higher precedence goes to Alyssa
Coghlan. (Conversation in person at PyCon US 2013.)

Copyright

This document has been placed in the public domain.

  Local Variables: mode: indented-text indent-tabs-mode: nil
  sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

[1] http://docs.python.org/3/library/functions.html#func-range

[2] http://docs.python.org/3/library/stdtypes.html#dict

[3] http://docs.python.org/3/library/curses.html#curses.window.border

[4] http://docs.python.org/3/library/os.html#os.sendfile

[5] http://docs.python.org/3/library/functions.html#func-range

[6] http://docs.python.org/3/library/curses.html#curses.window.addch

[7] Guido van Rossum, posting to python-ideas, March 2012:
https://mail.python.org/pipermail/python-ideas/2012-March/014364.html
and
https://mail.python.org/pipermail/python-ideas/2012-March/014378.html
and
https://mail.python.org/pipermail/python-ideas/2012-March/014417.html


text:
PEP: 275 Title: Switching on Multiple Values Author: Marc-André Lemburg
<EMAIL_ADDRESS>Status: Rejected Type: Standards Track Content-Type:
text/x-rst Created: 10-Nov-2001 Python-Version: 2.6 Post-History:

Rejection Notice

A similar PEP for Python 3000, PEP 3103, was already rejected, so this
proposal has no chance of being accepted either.

Abstract

This PEP proposes strategies to enhance Python's performance with
respect to handling switching on a single variable having one of
multiple possible values.

Problem

Up to Python 2.5, the typical way of writing multi-value switches has
been to use long switch constructs of the following type:

    if x == 'first state':
        ...
    elif x == 'second state':
        ...
    elif x == 'third state':
        ...
    elif x == 'fourth state':
        ...
    else:
        # default handling
        ...

This works fine for short switch constructs, since the overhead of
repeated loading of a local (the variable x in this case) and comparing
it to some constant is low (it has a complexity of O(n) on average).
However, when using such a construct to write a state machine such as is
needed for writing parsers the number of possible states can easily
reach 10 or more cases.

The current solution to this problem lies in using a dispatch table to
find the case implementing method to execute depending on the value of
the switch variable (this can be tuned to have a complexity of O(1) on
average, e.g. by using perfect hash tables). This works well for state
machines which require complex and lengthy processing in the different
case methods. It does not perform well for ones which only process one
or two instructions per case, e.g.

    def handle_data(self, data):
        self.stack.append(data)

A nice example of this is the state machine implemented in pickle.py
which is used to serialize Python objects. Other prominent cases include
XML SAX parsers and Internet protocol handlers.

Proposed Solutions

This PEP proposes two different but not necessarily conflicting
solutions:

1.  Adding an optimization to the Python compiler and VM which detects
    the above if-elif-else construct and generates special opcodes for
    it which use a read-only dictionary for storing jump offsets.
2.  Adding new syntax to Python which mimics the C style switch
    statement.

The first solution has the benefit of not relying on adding new keywords
to the language, while the second looks cleaner. Both involve some
run-time overhead to assure that the switching variable is immutable and
hashable.

Both solutions use a dictionary lookup to find the right jump location,
so they both share the same problem space in terms of requiring that
both the switch variable and the constants need to be compatible to the
dictionary implementation (hashable, comparable, a==b =>
hash(a)==hash(b)).

Solution 1: Optimizing if-elif-else

Implementation:

It should be possible for the compiler to detect an if-elif-else
construct which has the following signature:

    if x == 'first':...
    elif x == 'second':...
    else:...

i.e. the left hand side always references the same variable, the right
hand side a hashable immutable builtin type. The right hand sides need
not be all of the same type, but they should be comparable to the type
of the left hand switch variable.

The compiler could then setup a read-only (perfect) hash table, store it
in the constants and add an opcode SWITCH in front of the standard
if-elif-else byte code stream which triggers the following run-time
behaviour:

At runtime, SWITCH would check x for being one of the well-known
immutable types (strings, unicode, numbers) and use the hash table for
finding the right opcode snippet. If this condition is not met, the
interpreter should revert to the standard if-elif-else processing by
simply skipping the SWITCH opcode and proceeding with the usual
if-elif-else byte code stream.

Issues:

The new optimization should not change the current Python semantics (by
reducing the number of __cmp__ calls and adding __hash__ calls in
if-elif-else constructs which are affected by the optimization). To
assure this, switching can only safely be implemented either if a "from
__future__" style flag is used, or the switching variable is one of the
builtin immutable types: int, float, string, unicode, etc. (not
subtypes, since it's not clear whether these are still immutable or not)

To prevent post-modifications of the jump-table dictionary (which could
be used to reach protected code), the jump-table will have to be a
read-only type (e.g. a read-only dictionary).

The optimization should only be used for if-elif-else constructs which
have a minimum number of n cases (where n is a number which has yet to
be defined depending on performance tests).

Solution 2: Adding a switch statement to Python

New Syntax

    switch EXPR:
        case CONSTANT:
            SUITE
        case CONSTANT:
            SUITE
        ...
        else:
            SUITE

(modulo indentation variations)

The "else" part is optional. If no else part is given and none of the
defined cases matches, no action is taken and the switch statement is
ignored. This is in line with the current if-behaviour. A user who wants
to signal this situation using an exception can define an else-branch
which then implements the intended action.

Note that the constants need not be all of the same type, but they
should be comparable to the type of the switch variable.

Implementation

The compiler would have to compile this into byte code similar to this:

    def whatis(x):
        switch(x):
            case 'one':
                print '1'
            case 'two':
                print '2'
            case 'three':
                print '3'
            else:
                print "D'oh!"

into (omitting POP_TOP's and SET_LINENO's):

    6  LOAD_FAST         0 (x)
    9  LOAD_CONST        1 (switch-table-1)
    12  SWITCH            26 (to 38)

    14  LOAD_CONST        2 ('1')
    17  PRINT_ITEM
    18  PRINT_NEWLINE
    19  JUMP 43

    22  LOAD_CONST        3 ('2')
    25  PRINT_ITEM
    26  PRINT_NEWLINE
    27  JUMP 43

    30  LOAD_CONST        4 ('3')
    33  PRINT_ITEM
    34  PRINT_NEWLINE
    35  JUMP 43

    38  LOAD_CONST        5 ("D'oh!")
    41  PRINT_ITEM
    42  PRINT_NEWLINE

    >>43  LOAD_CONST        0 (None)
    46  RETURN_VALUE

Where the 'SWITCH' opcode would jump to 14, 22, 30 or 38 depending on
'x'.

Thomas Wouters has written a patch which demonstrates the above. You can
download it from[1].

Issues

The switch statement should not implement fall-through behaviour (as
does the switch statement in C). Each case defines a complete and
independent suite; much like in an if-elif-else statement. This also
enables using break in switch statements inside loops.

If the interpreter finds that the switch variable x is not hashable, it
should raise a TypeError at run-time pointing out the problem.

There have been other proposals for the syntax which reuse existing
keywords and avoid adding two new ones ("switch" and "case"). Others
have argued that the keywords should use new terms to avoid confusion
with the C keywords of the same name but slightly different semantics
(e.g. fall-through without break). Some of the proposed variants:

    case EXPR:
        of CONSTANT:
            SUITE
        of CONSTANT:
            SUITE
        else:
            SUITE

    case EXPR:
        if CONSTANT:
             SUITE
        if CONSTANT:
            SUITE
        else:
            SUITE

    when EXPR:
        in CONSTANT_TUPLE:
            SUITE
        in CONSTANT_TUPLE:
            SUITE
        ...
    else:
         SUITE

The switch statement could be extended to allow multiple values for one
section (e.g. case 'a', 'b', 'c': ...). Another proposed extension would
allow ranges of values (e.g. case 10..14: ...). These should probably be
post-poned, but already kept in mind when designing and implementing a
first version.

Examples

The following examples all use a new syntax as proposed by solution 2.
However, all of these examples would work with solution 1 as well.

    switch EXPR:                   switch x:
        case CONSTANT:                 case "first":
            SUITE                          print x
        case CONSTANT:                 case "second":
            SUITE                          x = x**2
        ...                                print x
        else:                          else:
            SUITE                          print "whoops!"

    case EXPR:                     case x:
        of CONSTANT:                   of "first":
            SUITE                          print x
        of CONSTANT:                   of "second":
            SUITE                          print x**2
        else:                          else:
            SUITE                          print "whoops!"

    case EXPR:                     case state:
        if CONSTANT:                   if "first":
             SUITE                         state = "second"
        if CONSTANT:                   if "second":
            SUITE                          state = "third"
        else:                          else:
            SUITE                          state = "first"

    when EXPR:                     when state:
        in CONSTANT_TUPLE:             in ("first", "second"):
            SUITE                          print state
        in CONSTANT_TUPLE:                 state = next_state(state)
            SUITE                      in ("seventh",):
        ...                                print "done"
    else:                                  break    # out of loop!
         SUITE                     else:
                                       print "middle state"
                                       state = next_state(state)

Here's another nice application found by Jack Jansen (switching on
argument types):

    switch type(x).__name__:
        case 'int':
            SUITE
        case 'string':
            SUITE

Scope

XXX Explain "from __future__ import switch"

Credits

-   Martin von Löwis (issues with the optimization idea)
-   Thomas Wouters (switch statement + byte code compiler example)
-   Skip Montanaro (dispatching ideas, examples)
-   Donald Beaudry (switch syntax)
-   Greg Ewing (switch syntax)
-   Jack Jansen (type switching examples)

References

Copyright

This document has been placed in the public domain.

[1] https://sourceforge.net/tracker/index.php?func=detail&aid=481118&group_id=5470&atid=305470


text:
PEP: 3150 Title: Statement local namespaces (aka "given" clause)
Version: $Revision$ Last-Modified: $Date$ Author: Alyssa Coghlan
<EMAIL_ADDRESS>Status: Deferred Type: Standards Track
Content-Type: text/x-rst Created: 09-Jul-2010 Python-Version: 3.4
Post-History: 14-Jul-2010, 21-Apr-2011, 13-Jun-2011

Abstract

This PEP proposes the addition of an optional given clause to several
Python statements that do not currently have an associated code suite.
This clause will create a statement local namespace for additional names
that are accessible in the associated statement, but do not become part
of the containing namespace.

Adoption of a new symbol, ?, is proposed to denote a forward reference
to the namespace created by running the associated code suite. It will
be a reference to a types.SimpleNamespace object.

The primary motivation is to enable a more declarative style of
programming, where the operation to be performed is presented to the
reader first, and the details of the necessary subcalculations are
presented in the following indented suite. As a key example, this would
elevate ordinary assignment statements to be on par with class and def
statements where the name of the item to be defined is presented to the
reader in advance of the details of how the value of that item is
calculated. It also allows named functions to be used in a "multi-line
lambda" fashion, where the name is used solely as a placeholder in the
current expression and then defined in the following suite.

A secondary motivation is to simplify interim calculations in module and
class level code without polluting the resulting namespaces.

The intent is that the relationship between a given clause and a
separate function definition that performs the specified operation will
be similar to the existing relationship between an explicit while loop
and a generator that produces the same sequence of operations as that
while loop.

The specific proposal in this PEP has been informed by various
explorations of this and related concepts over the years
(e.g.[1],[2],[3],[4], [5]), and is inspired to some degree by the where
and let clauses in Haskell. It avoids some problems that have been
identified in past proposals, but has not yet itself been subject to the
test of implementation.

Proposal

This PEP proposes the addition of an optional given clause to the syntax
for simple statements which may contain an expression, or may substitute
for such a statement for purely syntactic purposes. The current list of
simple statements that would be affected by this addition is as follows:

-   expression statement
-   assignment statement
-   augmented assignment statement
-   del statement
-   return statement
-   yield statement
-   raise statement
-   assert statement
-   pass statement

The given clause would allow subexpressions to be referenced by name in
the header line, with the actual definitions following in the indented
clause. As a simple example:

    sorted_data = sorted(data, key=?.sort_key) given:
        def sort_key(item):
            return item.attr1, item.attr2

The new symbol ? is used to refer to the given namespace. It would be a
types.SimpleNamespace instance, so ?.sort_key functions as a forward
reference to a name defined in the given clause.

A docstring would be permitted in the given clause, and would be
attached to the result namespace as its __doc__ attribute.

The pass statement is included to provide a consistent way to skip
inclusion of a meaningful expression in the header line. While this is
not an intended use case, it isn't one that can be prevented as multiple
alternatives (such as ... and ()) remain available even if pass itself
is disallowed.

The body of the given clause will execute in a new scope, using normal
function closure semantics. To support early binding of loop variables
and global references, as well as to allow access to other names defined
at class scope, the given clause will also allow explicit binding
operations in the header line:

    # Explicit early binding via given clause
    seq = []
    for i in range(10):
        seq.append(?.f) given i=i in:
            def f():
                return i
    assert [f() for f in seq] == list(range(10))

Semantics

The following statement:

    op(?.f, ?.g) given bound_a=a, bound_b=b in:
        def f():
            return bound_a + bound_b
        def g():
            return bound_a - bound_b

Would be roughly equivalent to the following code (__var denotes a
hidden compiler variable or simply an entry on the interpreter stack):

    __arg1 = a
    __arg2 = b
    def __scope(bound_a, bound_b):
        def f():
            return bound_a + bound_b
        def g():
            return bound_a - bound_b
       return types.SimpleNamespace(**locals())
    __ref = __scope(__arg1, __arg2)
    __ref.__doc__ = __scope.__doc__
    op(__ref.f, __ref.g)

A given clause is essentially a nested function which is created and
then immediately executed. Unless explicitly passed in, names are looked
up using normal scoping rules, and thus names defined at class scope
will not be visible. Names declared as forward references are returned
and used in the header statement, without being bound locally in the
surrounding namespace.

Syntax Change

Current:

    expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) |
                 ('=' (yield_expr|testlist_star_expr))*)
    del_stmt: 'del' exprlist
    pass_stmt: 'pass'
    return_stmt: 'return' [testlist]
    yield_stmt: yield_expr
    raise_stmt: 'raise' [test ['from' test]]
    assert_stmt: 'assert' test [',' test]

New:

    expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) |
                 ('=' (yield_expr|testlist_star_expr))*) [given_clause]
    del_stmt: 'del' exprlist [given_clause]
    pass_stmt: 'pass' [given_clause]
    return_stmt: 'return' [testlist] [given_clause]
    yield_stmt: yield_expr [given_clause]
    raise_stmt: 'raise' [test ['from' test]] [given_clause]
    assert_stmt: 'assert' test [',' test] [given_clause]
    given_clause: "given" [(NAME '=' test)+ "in"]":" suite

(Note that expr_stmt in the grammar is a slight misnomer, as it covers
assignment and augmented assignment in addition to simple expression
statements)

Note

These proposed grammar changes don't yet cover the forward reference
expression syntax for accessing names defined in the statement local
namespace.

The new clause is added as an optional element of the existing
statements rather than as a new kind of compound statement in order to
avoid creating an ambiguity in the grammar. It is applied only to the
specific elements listed so that nonsense like the following is
disallowed:

    break given:
        a = b = 1

    import sys given:
        a = b = 1

However, the precise Grammar change described above is inadequate, as it
creates problems for the definition of simple_stmt (which allows
chaining of multiple single line statements with ";" rather than "\n").

So the above syntax change should instead be taken as a statement of
intent. Any actual proposal would need to resolve the simple_stmt
parsing problem before it could be seriously considered. This would
likely require a non-trivial restructuring of the grammar, breaking up
small_stmt and flow_stmt to separate the statements that potentially
contain arbitrary subexpressions and then allowing a single one of those
statements with a given clause at the simple_stmt level. Something along
the lines of:

    stmt: simple_stmt | given_stmt | compound_stmt
    simple_stmt: small_stmt (';' (small_stmt | subexpr_stmt))* [';'] NEWLINE
    small_stmt: (pass_stmt | flow_stmt | import_stmt |
                 global_stmt | nonlocal_stmt)
    flow_stmt: break_stmt | continue_stmt
    given_stmt: subexpr_stmt (given_clause |
                  (';' (small_stmt | subexpr_stmt))* [';']) NEWLINE
    subexpr_stmt: expr_stmt | del_stmt | flow_subexpr_stmt | assert_stmt
    flow_subexpr_stmt: return_stmt | raise_stmt | yield_stmt
    given_clause: "given" (NAME '=' test)* ":" suite

For reference, here are the current definitions at that level:

    stmt: simple_stmt | compound_stmt
    simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE
    small_stmt: (expr_stmt | del_stmt | pass_stmt | flow_stmt |
                 import_stmt | global_stmt | nonlocal_stmt | assert_stmt)
    flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt

In addition to the above changes, the definition of atom would be
changed to also allow ?. The restriction of this usage to statements
with an associated given clause would be handled by a later stage of the
compilation process (likely AST construction, which already enforces
other restrictions where the grammar is overly permissive in order to
simplify the initial parsing step).

New PEP 8 Guidelines

As discussed on python-ideas ([6],[7]) new PEP 8 guidelines would also
need to be developed to provide appropriate direction on when to use the
given clause over ordinary variable assignments.

Based on the similar guidelines already present for try statements, this
PEP proposes the following additions for given statements to the
"Programming Conventions" section of PEP 8:

-   for code that could reasonably be factored out into a separate
    function, but is not currently reused anywhere, consider using a
    given clause. This clearly indicates which variables are being used
    only to define subcomponents of another statement rather than to
    hold algorithm or application state. This is an especially useful
    technique when passing multi-line functions to operations which take
    callable arguments.
-   keep given clauses concise. If they become unwieldy, either break
    them up into multiple steps or else move the details into a separate
    function.

Rationale

Function and class statements in Python have a unique property relative
to ordinary assignment statements: to some degree, they are declarative.
They present the reader of the code with some critical information about
a name that is about to be defined, before proceeding on with the
details of the actual definition in the function or class body.

The name of the object being declared is the first thing stated after
the keyword. Other important information is also given the honour of
preceding the implementation details:

-   decorators (which can greatly affect the behaviour of the created
    object, and were placed ahead of even the keyword and name as a
    matter of practicality more so than aesthetics)
-   the docstring (on the first line immediately following the header
    line)
-   parameters, default values and annotations for function definitions
-   parent classes, metaclass and optionally other details (depending on
    the metaclass) for class definitions

This PEP proposes to make a similar declarative style available for
arbitrary assignment operations, by permitting the inclusion of a
"given" suite following any simple assignment statement:

    TARGET = [TARGET2 = ... TARGETN =] EXPR given:
        SUITE

By convention, code in the body of the suite should be oriented solely
towards correctly defining the assignment operation carried out in the
header line. The header line operation should also be adequately
descriptive (e.g. through appropriate choices of variable names) to give
a reader a reasonable idea of the purpose of the operation without
reading the body of the suite.

However, while they are the initial motivating use case, limiting this
feature solely to simple assignments would be overly restrictive. Once
the feature is defined at all, it would be quite arbitrary to prevent
its use for augmented assignments, return statements, yield expressions,
comprehensions and arbitrary expressions that may modify the application
state.

The given clause may also function as a more readable alternative to
some uses of lambda expressions and similar constructs when passing
one-off functions to operations like sorted() or in callback based
event-driven programming.

In module and class level code, the given clause will serve as a clear
and reliable replacement for usage of the del statement to keep interim
working variables from polluting the resulting namespace.

One potentially useful way to think of the proposed clause is as a
middle ground between conventional in-line code and separation of an
operation out into a dedicated function, just as an inline while loop
may eventually be factored out into a dedicated generator.

Design Discussion

Keyword Choice

This proposal initially used where based on the name of a similar
construct in Haskell. However, it has been pointed out that there are
existing Python libraries (such as Numpy[8]) that already use where in
the SQL query condition sense, making that keyword choice potentially
confusing.

While given may also be used as a variable name (and hence would be
deprecated using the usual __future__ dance for introducing new
keywords), it is associated much more strongly with the desired "here
are some extra variables this expression may use" semantics for the new
clause.

Reusing the with keyword has also been proposed. This has the advantage
of avoiding the addition of a new keyword, but also has a high potential
for confusion as the with clause and with statement would look similar
but do completely different things. That way lies C++ and Perl :)

Relation to PEP 403

PEP 403 (General Purpose Decorator Clause) attempts to achieve the main
goals of this PEP using a less radical language change inspired by the
existing decorator syntax.

Despite having the same author, the two PEPs are in direct competition
with each other. PEP 403 represents a minimalist approach that attempts
to achieve useful functionality with a minimum of change from the status
quo. This PEP instead aims for a more flexible standalone statement
design, which requires a larger degree of change to the language.

Note that where PEP 403 is better suited to explaining the behaviour of
generator expressions correctly, this PEP is better able to explain the
behaviour of decorator clauses in general. Both PEPs support adequate
explanations for the semantics of container comprehensions.

Explaining Container Comprehensions and Generator Expressions

One interesting feature of the proposed construct is that it can be used
as a primitive to explain the scoping and execution order semantics of
container comprehensions:

    seq2 = [x for x in y if q(x) for y in seq if p(y)]

    # would be equivalent to

    seq2 = ?.result given seq=seq:
        result = []
        for y in seq:
            if p(y):
                for x in y:
                    if q(x):
                        result.append(x)

The important point in this expansion is that it explains why
comprehensions appear to misbehave at class scope: only the outermost
iterator is evaluated at class scope, while all predicates, nested
iterators and value expressions are evaluated inside a nested scope.

Not that, unlike PEP 403, the current version of this PEP cannot provide
a precisely equivalent expansion for a generator expression. The closest
it can get is to define an additional level of scoping:

    seq2 = ?.g(seq) given:
        def g(seq):
            for y in seq:
                if p(y):
                    for x in y:
                        if q(x):
                            yield x

This limitation could be remedied by permitting the given clause to be a
generator function, in which case ? would refer to a generator-iterator
object rather than a simple namespace:

    seq2 = ? given seq=seq in:
        for y in seq:
            if p(y):
                for x in y:
                    if q(x):
                        yield x

However, this would make the meaning of "?" quite ambiguous, even more
so than is already the case for the meaning of def statements (which
will usually have a docstring indicating whether or not a function
definition is actually a generator)

Explaining Decorator Clause Evaluation and Application

The standard explanation of decorator clause evaluation and application
has to deal with the idea of hidden compiler variables in order to show
steps in their order of execution. The given statement allows a
decorated function definition like:

    @classmethod
    def classname(cls):
        return cls.__name__

To instead be explained as roughly equivalent to:

    classname = .d1(classname) given:
        d1 = classmethod
        def classname(cls):
            return cls.__name__

Anticipated Objections

Two Ways To Do It

A lot of code may now be written with values defined either before the
expression where they are used or afterwards in a given clause, creating
two ways to do it, perhaps without an obvious way of choosing between
them.

On reflection, I feel this is a misapplication of the "one obvious way"
aphorism. Python already offers lots of ways to write code. We can use a
for loop or a while loop, a functional style or an imperative style or
an object oriented style. The language, in general, is designed to let
people write code that matches the way they think. Since different
people think differently, the way they write their code will change
accordingly.

Such stylistic questions in a code base are rightly left to the
development group responsible for that code. When does an expression get
so complicated that the subexpressions should be taken out and assigned
to variables, even though those variables are only going to be used
once? When should an inline while loop be replaced with a generator that
implements the same logic? Opinions differ, and that's OK.

However, explicit PEP 8 guidance will be needed for CPython and the
standard library, and that is discussed in the proposal above.

Out of Order Execution

The given clause makes execution jump around a little strangely, as the
body of the given clause is executed before the simple statement in the
clause header. The closest any other part of Python comes to this is the
out of order evaluation in list comprehensions, generator expressions
and conditional expressions and the delayed application of decorator
functions to the function they decorate (the decorator expressions
themselves are executed in the order they are written).

While this is true, the syntax is intended for cases where people are
themselves thinking about a problem out of sequence (at least as far as
the language is concerned). As an example of this, consider the
following thought in the mind of a Python user:

  I want to sort the items in this sequence according to the values of
  attr1 and attr2 on each item.

If they're comfortable with Python's lambda expressions, then they might
choose to write it like this:

    sorted_list = sorted(original, key=(lambda v: v.attr1, v.attr2))

That gets the job done, but it hardly reaches the standard of
executable pseudocode that fits Python's reputation.

If they don't like lambda specifically, the operator module offers an
alternative that still allows the key function to be defined inline:

    sorted_list = sorted(original,
                         key=operator.attrgetter(v. 'attr1', 'attr2'))

Again, it gets the job done, but even the most generous of readers would
not consider that to be "executable pseudocode".

If they think both of the above options are ugly and confusing, or they
need logic in their key function that can't be expressed as an
expression (such as catching an exception), then Python currently forces
them to reverse the order of their original thought and define the
sorting criteria first:

    def sort_key(item):
        return item.attr1, item.attr2

    sorted_list = sorted(original, key=sort_key)

"Just define a function" has been the rote response to requests for
multi-line lambda support for years. As with the above options, it gets
the job done, but it really does represent a break between what the user
is thinking and what the language allows them to express.

I believe the proposal in this PEP would finally let Python get close to
the "executable pseudocode" bar for the kind of thought expressed above:

    sorted_list = sorted(original, key=?.key) given:
        def key(item):
            return item.attr1, item.attr2

Everything is in the same order as it was in the user's original
thought, and they don't even need to come up with a name for the sorting
criteria: it is possible to reuse the keyword argument name directly.

A possible enhancement to those proposal would be to provide a
convenient shorthand syntax to say "use the given clause contents as
keyword arguments". Even without dedicated syntax, that can be written
simply as **vars(?).

Harmful to Introspection

Poking around in module and class internals is an invaluable tool for
white-box testing and interactive debugging. The given clause will be
quite effective at preventing access to temporary state used during
calculations (although no more so than current usage of del statements
in that regard).

While this is a valid concern, design for testability is an issue that
cuts across many aspects of programming. If a component needs to be
tested independently, then a given statement should be refactored in to
separate statements so that information is exposed to the test suite.
This isn't significantly different from refactoring an operation hidden
inside a function or generator out into its own function purely to allow
it to be tested in isolation.

Lack of Real World Impact Assessment

The examples in the current PEP are almost all relatively small "toy"
examples. The proposal in this PEP needs to be subjected to the test of
application to a large code base (such as the standard library or a
large Twisted application) in a search for examples where the
readability of real world code is genuinely enhanced.

This is more of a deficiency in the PEP rather than the idea, though. If
it wasn't a real world problem, we wouldn't get so many complaints about
the lack of multi-line lambda support and Ruby's block construct
probably wouldn't be quite so popular.

Open Questions

Syntax for Forward References

The ? symbol is proposed for forward references to the given namespace
as it is short, currently unused and suggests "there's something missing
here that will be filled in later".

The proposal in the PEP doesn't neatly parallel any existing Python
feature, so reusing an already used symbol has been deliberately
avoided.

Handling of nonlocal and global

nonlocal and global are explicitly disallowed in the given clause suite
and will be syntax errors if they occur. They will work normally if they
appear within a def statement within that suite.

Alternatively, they could be defined as operating as if the anonymous
functions were defined as in the expansion above.

Handling of break and continue

break and continue will operate as if the anonymous functions were
defined as in the expansion above. They will be syntax errors if they
occur in the given clause suite but will work normally if they appear
within a for or while loop as part of that suite.

Handling of return and yield

return and yield are explicitly disallowed in the given clause suite and
will be syntax errors if they occur. They will work normally if they
appear within a def statement within that suite.

Examples

Defining callbacks for event driven programming:

    # Current Python (definition before use)
    def cb(sock):
        # Do something with socket
    def eb(exc):
        logging.exception(
            "Failed connecting to %s:%s", host, port)
    loop.create_connection((host, port), cb, eb) given:

    # Becomes:
    loop.create_connection((host, port), ?.cb, ?.eb) given:
        def cb(sock):
            # Do something with socket
        def eb(exc):
            logging.exception(
                "Failed connecting to %s:%s", host, port)

Defining "one-off" classes which typically only have a single instance:

    # Current Python (instantiation after definition)
    class public_name():
      ... # However many lines
    public_name = public_name(*params)

    # Current Python (custom decorator)
    def singleton(*args, **kwds):
        def decorator(cls):
            return cls(*args, **kwds)
        return decorator

    @singleton(*params)
    class public_name():
      ... # However many lines

    # Becomes:
    public_name = ?.MeaningfulClassName(*params) given:
      class MeaningfulClassName():
        ... # Should trawl the stdlib for an example of doing this

Calculating attributes without polluting the local namespace (from
os.py):

    # Current Python (manual namespace cleanup)
    def _createenviron():
      ... # 27 line function

    environ = _createenviron()
    del _createenviron

    # Becomes:
    environ = ?._createenviron() given:
        def _createenviron():
          ... # 27 line function

Replacing default argument hack (from functools.lru_cache):

    # Current Python (default argument hack)
    def decorating_function(user_function,
                   tuple=tuple, sorted=sorted, len=len, KeyError=KeyError):
      ... # 60 line function
    return decorating_function

    # Becomes:
    return ?.decorating_function given:
      # Cell variables rather than locals, but should give similar speedup
      tuple, sorted, len, KeyError = tuple, sorted, len, KeyError
      def decorating_function(user_function):
        ... # 60 line function

    # This example also nicely makes it clear that there is nothing in the
    # function after the nested function definition. Due to additional
    # nested functions, that isn't entirely clear in the current code.

Possible Additions

-   The current proposal allows the addition of a given clause only for
    simple statements. Extending the idea to allow the use of compound
    statements would be quite possible (by appending the given clause as
    an independent suite at the end), but doing so raises serious
    readability concerns (as values defined in the given clause may be
    used well before they are defined, exactly the kind of readability
    trap that other features like decorators and with statements are
    designed to eliminate)
-   The "explicit early binding" variant may be applicable to the
    discussions on python-ideas on how to eliminate the default argument
    hack. A given clause in the header line for functions (after the
    return type annotation) may be the answer to that question.

Rejected Alternatives

-   An earlier version of this PEP allowed implicit forward references
    to the names in the trailing suite, and also used implicit early
    binding semantics. Both of these ideas substantially complicated the
    proposal without providing a sufficient increase in expressive
    power. The current proposing with explicit forward references and
    early binding brings the new construct into line with existing
    scoping semantics, greatly improving the chances the idea can
    actually be implemented.
-   In addition to the proposals made here, there have also been
    suggestions of two suite "in-order" variants which provide the
    limited scoping of names without supporting out-of-order execution.
    I believe these suggestions largely miss the point of what people
    are complaining about when they ask for multi-line lambda support -
    it isn't that coming up with a name for the subexpression is
    especially difficult, it's that naming the function before the
    statement that uses it means the code no longer matches the way the
    developer thinks about the problem at hand.
-   I've made some unpublished attempts to allow direct references to
    the closure implicitly created by the given clause, while still
    retaining the general structure of the syntax as defined in this PEP
    (For example, allowing a subexpression like ?given or :given to be
    used in expressions to indicate a direct reference to the implied
    closure, thus preventing it from being called automatically to
    create the local namespace). All such attempts have appeared
    unattractive and confusing compared to the simpler
    decorator-inspired proposal in PEP 403.

Reference Implementation

None as yet. If you want a crash course in Python namespace semantics
and code compilation, feel free to try ;)

TO-DO

-   Mention PEP 359 and possible uses for locals() in the given clause
-   Figure out if this can be used internally to make the implementation
    of zero-argument super() calls less awful

References

-   The "Status quo wins a stalemate" design principle
-   Multi-line lambdas (again!)

Copyright

This document has been placed in the public domain.

[1] Explicitation lines in Python

[2] 'where' statement in Python

[3] Where-statement (Proposal for function expressions)

[4] Assignments in list/generator expressions

[5] Discussion of PEP 403 (statement local function definition)

[6] Possible PEP 3150 style guidelines (#1)

[7] Possible PEP 3150 style guidelines (#2)

[8] Name conflict with NumPy for 'where' keyword choice


text:
PEP: 160 Title: Python 1.6 Release Schedule Version: $Revision$
Last-Modified: $Date$ Author: Fred L. Drake, Jr<EMAIL_ADDRESS>Status: Final Type: Informational Topic: Release Content-Type:
text/x-rst Created: 25-Jul-2000 Python-Version: 1.6 Post-History:

Introduction

This PEP describes the Python 1.6 release schedule. The CVS revision
history of this file contains the definitive historical record.

This release will be produced by BeOpen PythonLabs staff for the
Corporation for National Research Initiatives (CNRI).

Schedule

-   August 1: 1.6 beta 1 release (planned).
-   August 3: 1.6 beta 1 release (actual).
-   August 15: 1.6 final release (planned).
-   September 5: 1.6 final release (actual).

Features

A number of features are required for Python 1.6 in order to fulfill the
various promises that have been made. The following are required to be
fully operational, documented, and forward compatible with the plans for
Python 2.0:

-   Unicode support: The Unicode object defined for Python 2.0 must be
    provided, including all methods and codec support.
-   SRE: Fredrik Lundh's new regular expression engine will be used to
    provide support for both 8-bit strings and Unicode strings. It must
    pass the regression test used for the pcre-based version of the re
    module.
-   The curses module was in the middle of a transformation to a
    package, so the final form was adopted.

Mechanism

The release will be created as a branch from the development tree rooted
at CNRI's close of business on 16 May 2000. Patches required from more
recent checkins will be merged in by moving the branch tag on individual
files whenever possible in order to reduce mailing list clutter and
avoid divergent and incompatible implementations.

The branch tag is "cnri-16-start".

Patches and features will be merged to the extent required to pass
regression tests in effect on 16 May 2000.

The beta release is tagged "r16b1" in the CVS repository, and the final
Python 1.6 release is tagged "release16" in the repository.

Copyright

This document has been placed in the public domain.

authors:
Fred L. Drake
Jr.