text:
PEP: 422 Title: Simpler customisation of class creation Version:
$Revision$ Last-Modified: $Date$ Author: Alyssa Coghlan
<EMAIL_ADDRESS>Daniel Urban<EMAIL_ADDRESS>Status:
Withdrawn Type: Standards Track Content-Type: text/x-rst Created:
05-Jun-2012 Python-Version: 3.5 Post-History: 05-Jun-2012, 10-Feb-2013

Abstract

Currently, customising class creation requires the use of a custom
metaclass. This custom metaclass then persists for the entire lifecycle
of the class, creating the potential for spurious metaclass conflicts.

This PEP proposes to instead support a wide range of customisation
scenarios through a new namespace parameter in the class header, and a
new __autodecorate__ hook in the class body.

The new mechanism should be easier to understand and use than
implementing a custom metaclass, and thus should provide a gentler
introduction to the full power Python's metaclass machinery.

PEP Withdrawal

This proposal has been withdrawn in favour of Martin Teichmann's
proposal in PEP 487, which achieves the same goals through a simpler,
easier to use __init_subclass__ hook that simply isn't invoked for the
base class that defines the hook.

Background

For an already created class cls, the term "metaclass" has a clear
meaning: it is the value of type(cls).

During class creation, it has another meaning: it is also used to refer
to the metaclass hint that may be provided as part of the class
definition. While in many cases these two meanings end up referring to
one and the same object, there are two situations where that is not the
case:

-   If the metaclass hint refers to an instance of type, then it is
    considered as a candidate metaclass along with the metaclasses of
    all of the parents of the class being defined. If a more appropriate
    metaclass is found amongst the candidates, then it will be used
    instead of the one given in the metaclass hint.
-   Otherwise, an explicit metaclass hint is assumed to be a factory
    function and is called directly to create the class object. In this
    case, the final metaclass will be determined by the factory function
    definition. In the typical case (where the factory functions just
    calls type, or, in Python 3.3 or later, types.new_class) the actual
    metaclass is then determined based on the parent classes.

It is notable that only the actual metaclass is inherited - a factory
function used as a metaclass hook sees only the class currently being
defined, and is not invoked for any subclasses.

In Python 3, the metaclass hint is provided using the metaclass=Meta
keyword syntax in the class header. This allows the __prepare__ method
on the metaclass to be used to create the locals() namespace used during
execution of the class body (for example, specifying the use of
collections.OrderedDict instead of a regular dict).

In Python 2, there was no __prepare__ method (that API was added for
Python 3 by PEP 3115). Instead, a class body could set the __metaclass__
attribute, and the class creation process would extract that value from
the class namespace to use as the metaclass hint. There is published
code that makes use of this feature.

Another new feature in Python 3 is the zero-argument form of the super()
builtin, introduced by PEP 3135. This feature uses an implicit __class__
reference to the class being defined to replace the "by name" references
required in Python 2. Just as code invoked during execution of a Python
2 metaclass could not call methods that referenced the class by name (as
the name had not yet been bound in the containing scope), similarly,
Python 3 metaclasses cannot call methods that rely on the implicit
__class__ reference (as it is not populated until after the metaclass
has returned control to the class creation machinery).

Finally, when a class uses a custom metaclass, it can pose additional
challenges to the use of multiple inheritance, as a new class cannot
inherit from parent classes with unrelated metaclasses. This means that
it is impossible to add a metaclass to an already published class: such
an addition is a backwards incompatible change due to the risk of
metaclass conflicts.

Proposal

This PEP proposes that a new mechanism to customise class creation be
added to Python 3.4 that meets the following criteria:

1.  Integrates nicely with class inheritance structures (including
    mixins and multiple inheritance)
2.  Integrates nicely with the implicit __class__ reference and
    zero-argument super() syntax introduced by PEP 3135
3.  Can be added to an existing base class without a significant risk of
    introducing backwards compatibility problems
4.  Restores the ability for class namespaces to have some influence on
    the class creation process (above and beyond populating the
    namespace itself), but potentially without the full flexibility of
    the Python 2 style __metaclass__ hook

One mechanism that can achieve this goal is to add a new implicit class
decoration hook, modelled directly on the existing explicit class
decorators, but defined in the class body or in a parent class, rather
than being part of the class definition header.

Specifically, it is proposed that class definitions be able to provide a
class initialisation hook as follows:

    class Example:
        def __autodecorate__(cls):
            # This is invoked after the class is created, but before any
            # explicit decorators are called
            # The usual super() mechanisms are used to correctly support
            # multiple inheritance. The class decorator style signature helps
            # ensure that invoking the parent class is as simple as possible.
            cls = super().__autodecorate__()
            return cls

To simplify the cooperative multiple inheritance case, object will gain
a default implementation of the hook that returns the class unmodified:

    class object:
        def __autodecorate__(cls):
            return cls

If a metaclass wishes to block implicit class decoration for some
reason, it must arrange for cls.__autodecorate__ to trigger
AttributeError.

If present on the created object, this new hook will be called by the
class creation machinery after the __class__ reference has been
initialised. For types.new_class(), it will be called as the last step
before returning the created class object. __autodecorate__ is
implicitly converted to a class method when the class is created (prior
to the hook being invoked).

Note, that when __autodecorate__ is called, the name of the class is not
yet bound to the new class object. As a consequence, the two argument
form of super() cannot be used to call methods (e.g.,
super(Example, cls) wouldn't work in the example above). However, the
zero argument form of super() works as expected, since the __class__
reference is already initialised.

This general proposal is not a new idea (it was first suggested for
inclusion in the language definition more than 10 years ago, and a
similar mechanism has long been supported by Zope's ExtensionClass), but
the situation has changed sufficiently in recent years that the idea is
worth reconsidering for inclusion as a native language feature.

In addition, the introduction of the metaclass __prepare__ method in PEP
3115 allows a further enhancement that was not possible in Python 2:
this PEP also proposes that type.__prepare__ be updated to accept a
factory function as a namespace keyword-only argument. If present, the
value provided as the namespace argument will be called without
arguments to create the result of type.__prepare__ instead of using a
freshly created dictionary instance. For example, the following will use
an ordered dictionary as the class namespace:

    class OrderedExample(namespace=collections.OrderedDict):
        def __autodecorate__(cls):
            # cls.__dict__ is still a read-only proxy to the class namespace,
            # but the underlying storage is an OrderedDict instance

Note

This PEP, along with the existing ability to use __prepare__ to share a
single namespace amongst multiple class objects, highlights a possible
issue with the attribute lookup caching: when the underlying mapping is
updated by other means, the attribute lookup cache is not invalidated
correctly (this is a key part of the reason class __dict__ attributes
produce a read-only view of the underlying storage).

Since the optimisation provided by that cache is highly desirable, the
use of a preexisting namespace as the class namespace may need to be
declared as officially unsupported (since the observed behaviour is
rather strange when the caches get out of sync).

Key Benefits

Easier use of custom namespaces for a class

Currently, to use a different type (such as collections.OrderedDict) for
a class namespace, or to use a pre-populated namespace, it is necessary
to write and use a custom metaclass. With this PEP, using a custom
namespace becomes as simple as specifying an appropriate factory
function in the class header.

Easier inheritance of definition time behaviour

Understanding Python's metaclasses requires a deep understanding of the
type system and the class construction process. This is legitimately
seen as challenging, due to the need to keep multiple moving parts (the
code, the metaclass hint, the actual metaclass, the class object,
instances of the class object) clearly distinct in your mind. Even when
you know the rules, it's still easy to make a mistake if you're not
being extremely careful. An earlier version of this PEP actually
included such a mistake: it stated "subclass of type" for a constraint
that is actually "instance of type".

Understanding the proposed implicit class decoration hook only requires
understanding decorators and ordinary method inheritance, which isn't
quite as daunting a task. The new hook provides a more gradual path
towards understanding all of the phases involved in the class definition
process.

Reduced chance of metaclass conflicts

One of the big issues that makes library authors reluctant to use
metaclasses (even when they would be appropriate) is the risk of
metaclass conflicts. These occur whenever two unrelated metaclasses are
used by the desired parents of a class definition. This risk also makes
it very difficult to add a metaclass to a class that has previously been
published without one.

By contrast, adding an __autodecorate__ method to an existing type poses
a similar level of risk to adding an __init__ method: technically, there
is a risk of breaking poorly implemented subclasses, but when that
occurs, it is recognised as a bug in the subclass rather than the
library author breaching backwards compatibility guarantees. In fact,
due to the constrained signature of __autodecorate__, the risk in this
case is actually even lower than in the case of __init__.

Integrates cleanly with PEP 3135

Unlike code that runs as part of the metaclass, code that runs as part
of the new hook will be able to freely invoke class methods that rely on
the implicit __class__ reference introduced by PEP 3135, including
methods that use the zero argument form of super().

Replaces many use cases for dynamic setting of __metaclass__

For use cases that don't involve completely replacing the defined class,
Python 2 code that dynamically set __metaclass__ can now dynamically set
__autodecorate__ instead. For more advanced use cases, introduction of
an explicit metaclass (possibly made available as a required base class)
will still be necessary in order to support Python 3.

Design Notes

Determining if the class being decorated is the base class

In the body of an __autodecorate__ method, as in any other class method,
__class__ will be bound to the class declaring the method, while the
value passed in may be a subclass.

This makes it relatively straightforward to skip processing the base
class if necessary:

    class Example:
        def __autodecorate__(cls):
            cls = super().__autodecorate__()
            # Don't process the base class
            if cls is __class__:
                return
            # Process subclasses here
            ...

Replacing a class with a different kind of object

As an implicit decorator, __autodecorate__ is able to relatively easily
replace the defined class with a different kind of object. Technically
custom metaclasses and even __new__ methods can already do this
implicitly, but the decorator model makes such code much easier to
understand and implement.

    class BuildDict:
        def __autodecorate__(cls):
            cls = super().__autodecorate__()
            # Don't process the base class
            if cls is __class__:
                return
            # Convert subclasses to ordinary dictionaries
            return cls.__dict__.copy()

It's not clear why anyone would ever do this implicitly based on
inheritance rather than just using an explicit decorator, but the
possibility seems worth noting.

Open Questions

Is the namespace concept worth the extra complexity?

Unlike the new __autodecorate__ hook the proposed namespace keyword
argument is not automatically inherited by subclasses. Given the way
this proposal is currently written , the only way to get a special
namespace used consistently in subclasses is still to write a custom
metaclass with a suitable __prepare__ implementation.

Changing the custom namespace factory to also be inherited would
significantly increase the complexity of this proposal, and introduce a
number of the same potential base class conflict issues as arise with
the use of custom metaclasses.

Eric Snow has put forward a separate proposal to instead make the
execution namespace for class bodies an ordered dictionary by default,
and capture the class attribute definition order for future reference as
an attribute (e.g. __definition_order__) on the class object.

Eric's suggested approach may be a better choice for a new default
behaviour for type that combines well with the proposed __autodecorate__
hook, leaving the more complex configurable namespace factory idea to a
custom metaclass like the one shown below.

New Ways of Using Classes

The new namespace keyword in the class header enables a number of
interesting options for controlling the way a class is initialised,
including some aspects of the object models of both Javascript and Ruby.

All of the examples below are actually possible today through the use of
a custom metaclass:

    class CustomNamespace(type):
        @classmethod
        def __prepare__(meta, name, bases, *, namespace=None, **kwds):
            parent_namespace = super().__prepare__(name, bases, **kwds)
            return namespace() if namespace is not None else parent_namespace
        def __new__(meta, name, bases, ns, *, namespace=None, **kwds):
            return super().__new__(meta, name, bases, ns, **kwds)
        def __init__(cls, name, bases, ns, *, namespace=None, **kwds):
            return super().__init__(name, bases, ns, **kwds)

The advantage of implementing the new keyword directly in
type.__prepare__ is that the only persistent effect is then the change
in the underlying storage of the class attributes. The metaclass of the
class remains unchanged, eliminating many of the drawbacks typically
associated with these kinds of customisations.

Order preserving classes

    class OrderedClass(namespace=collections.OrderedDict):
        a = 1
        b = 2
        c = 3

Prepopulated namespaces

    seed_data = dict(a=1, b=2, c=3)
    class PrepopulatedClass(namespace=seed_data.copy):
        pass

Cloning a prototype class

    class NewClass(namespace=Prototype.__dict__.copy):
        pass

Extending a class

Note

Just because the PEP makes it possible to do this relatively cleanly
doesn't mean anyone should do this!

    from collections import MutableMapping

    # The MutableMapping + dict combination should give something that
    # generally behaves correctly as a mapping, while still being accepted
    # as a class namespace
    class ClassNamespace(MutableMapping, dict):
        def __init__(self, cls):
            self._cls = cls
        def __len__(self):
            return len(dir(self._cls))
        def __iter__(self):
            for attr in dir(self._cls):
                yield attr
        def __contains__(self, attr):
            return hasattr(self._cls, attr)
        def __getitem__(self, attr):
            return getattr(self._cls, attr)
        def __setitem__(self, attr, value):
            setattr(self._cls, attr, value)
        def __delitem__(self, attr):
            delattr(self._cls, attr)

    def extend(cls):
        return lambda: ClassNamespace(cls)

    class Example:
        pass

    class ExtendedExample(namespace=extend(Example)):
        a = 1
        b = 2
        c = 3

    >>> Example.a, Example.b, Example.c
    (1, 2, 3)

Rejected Design Options

Calling __autodecorate__ from type.__init__

Calling the new hook automatically from type.__init__, would achieve
most of the goals of this PEP. However, using that approach would mean
that __autodecorate__ implementations would be unable to call any
methods that relied on the __class__ reference (or used the
zero-argument form of super()), and could not make use of those features
themselves.

The current design instead ensures that the implicit decorator hook is
able to do anything an explicit decorator can do by running it after the
initial class creation is already complete.

Calling the automatic decoration hook __init_class__

Earlier versions of the PEP used the name __init_class__ for the name of
the new hook. There were three significant problems with this name:

-   it was hard to remember if the correct spelling was __init_class__
    or __class_init__
-   the use of "init" in the name suggested the signature should match
    that of type.__init__, which is not the case
-   the use of "init" in the name suggested the method would be run as
    part of initial class object creation, which is not the case

The new name __autodecorate__ was chosen to make it clear that the new
initialisation hook is most usefully thought of as an implicitly invoked
class decorator, rather than as being like an __init__ method.

Requiring an explicit decorator on __autodecorate__

Originally, this PEP required the explicit use of @classmethod on the
__autodecorate__ decorator. It was made implicit since there's no
sensible interpretation for leaving it out, and that case would need to
be detected anyway in order to give a useful error message.

This decision was reinforced after noticing that the user experience of
defining __prepare__ and forgetting the @classmethod method decorator is
singularly incomprehensible (particularly since PEP 3115 documents it as
an ordinary method, and the current documentation doesn't explicitly say
anything one way or the other).

Making __autodecorate__ implicitly static, like __new__

While it accepts the class to be instantiated as the first argument,
__new__ is actually implicitly treated as a static method rather than as
a class method. This allows it to be readily extracted from its defining
class and called directly on a subclass, rather than being coupled to
the class object it is retrieved from.

Such behaviour initially appears to be potentially useful for the new
__autodecorate__ hook, as it would allow __autodecorate__ methods to
readily be used as explicit decorators on other classes.

However, that apparent support would be an illusion as it would only
work correctly if invoked on a subclass, in which case the method can
just as readily be retrieved from the subclass and called that way.
Unlike __new__, there's no issue with potentially changing method
signatures at different points in the inheritance chain.

Passing in the namespace directly rather than a factory function

At one point, this PEP proposed that the class namespace be passed
directly as a keyword argument, rather than passing a factory function.
However, this encourages an unsupported behaviour (that is, passing the
same namespace to multiple classes, or retaining direct write access to
a mapping used as a class namespace), so the API was switched to the
factory function version.

Reference Implementation

A reference implementation for __autodecorate__ has been posted to the
issue tracker. It uses the original __init_class__ naming. does not yet
allow the implicit decorator to replace the class with a different
object and does not implement the suggested namespace parameter for
type.__prepare__.

TODO

-   address the 5 points in
    https://mail.python.org/pipermail/python-dev/2013-February/123970.html

References

Copyright

This document has been placed in the public domain.


text:
PEP: 297 Title: Support for System Upgrades Author: Marc-André Lemburg
<EMAIL_ADDRESS>Status: Rejected Type: Standards Track Content-Type:
text/x-rst Created: 19-Jul-2001 Python-Version: 2.6 Post-History:

Rejection Notice

This PEP is rejected for failure to generate significant interest.

Abstract

This PEP proposes strategies to allow the Python standard library to be
upgraded in parts without having to reinstall the complete distribution
or having to wait for a new patch level release.

Problem

Python currently does not allow overriding modules or packages in the
standard library per default. Even though this is possible by defining a
PYTHONPATH environment variable (the paths defined in this variable are
prepended to the Python standard library path), there is no standard way
of achieving this without changing the configuration.

Since Python's standard library is starting to host packages which are
also available separately, e.g. the distutils, email and PyXML packages,
which can also be installed independently of the Python distribution, it
is desirable to have an option to upgrade these packages without having
to wait for a new patch level release of the Python interpreter to bring
along the changes.

On some occasions, it may also be desirable to update modules of the
standard library without going through the whole Python release cycle,
e.g. in order to provide hot-fixes for security problems.

Proposed Solutions

This PEP proposes two different but not necessarily conflicting
solutions:

1.  Adding a new standard search path to sys.path:
    $stdlibpath/system-packages just before the $stdlibpath entry. This
    complements the already existing entry for site add-ons
    $stdlibpath/site-packages which is appended to the sys.path at
    interpreter startup time.

    To make use of this new standard location, distutils will need to
    grow support for installing certain packages in
    $stdlibpath/system-packages rather than the standard location for
    third-party packages $stdlibpath/site-packages.

2.  Tweaking distutils to install directly into $stdlibpath for the
    system upgrades rather than into $stdlibpath/site-packages.

The first solution has a few advantages over the second:

-   upgrades can be easily identified (just look in
    $stdlibpath/system-packages)
-   upgrades can be de-installed without affecting the rest of the
    interpreter installation
-   modules can be virtually removed from packages; this is due to the
    way Python imports packages: once it finds the top-level package
    directory it stay in this directory for all subsequent package
    submodule imports
-   the approach has an overall much cleaner design than the hackish
    install on top of an existing installation approach

The only advantages of the second approach are that the Python
interpreter does not have to changed and that it works with older Python
versions.

Both solutions require changes to distutils. These changes can also be
implemented by package authors, but it would be better to define a
standard way of switching on the proposed behaviour.

Scope

Solution 1: Python 2.6 and up

Solution 2: all Python versions supported by distutils

Credits

None

References

None

Copyright

This document has been placed in the public domain.

license: Public Domain


text:
PEP: 368 Title: Standard image protocol and class Version: $Revision$
Last-Modified: $Date$ Author: Lino Mastrodomenico
<EMAIL_ADDRESS>Status: Deferred Type: Standards Track
Content-Type: text/x-rst Created: 28-Jun-2007 Python-Version: 2.6, 3.0
Post-History:

Abstract

The current situation of image storage and manipulation in the Python
world is extremely fragmented: almost every library that uses image
objects has implemented its own image class, incompatible with everyone
else's and often not very pythonic. A basic RGB image class exists in
the standard library (Tkinter.PhotoImage), but is pretty much unusable,
and unused, for anything except Tkinter programming.

This fragmentation not only takes up valuable space in the developers
minds, but also makes the exchange of images between different libraries
(needed in relatively common use cases) slower and more complex than it
needs to be.

This PEP proposes to improve the situation by defining a simple and
pythonic image protocol/interface that can be hopefully accepted and
implemented by existing image classes inside and outside the standard
library without breaking backward compatibility with their existing user
bases. In practice this is a definition of how a minimal image-like
object should look and act (in a similar way to the read() and write()
methods in file-like objects).

The inclusion in the standard library of a class that provides basic
image manipulation functionality and implements the new protocol is also
proposed, together with a mixin class that helps adding support for the
protocol to existing image classes.

PEP Deferral

Further exploration of the concepts covered in this PEP has been
deferred for lack of a current champion interested in promoting the
goals of the PEP and collecting and incorporating feedback, and with
sufficient available time to do so effectively.

Rationale

A good way to have high quality modules ready for inclusion in the
Python standard library is to simply wait for natural selection among
competing external libraries to provide a clear winner with useful
functionality and a big user base. Then the de facto standard can be
officially sanctioned by including it in the standard library.

Unfortunately this approach hasn't worked well for the creation of a
dominant image class in the Python world: almost every third-party
library that requires an image object creates its own class incompatible
with the ones from other libraries. This is a real problem because it's
entirely reasonable for a program to create and manipulate an image
using, e.g., PIL (the Python Imaging Library) and then display it using
wxPython or pygame. But these libraries have different and incompatible
image classes, and the usual solution is to manually "export" an image
from the source to a (width, height, bytes_string) tuple and "import" it
creating a new instance in the target format. This approach works, but
is both uglier and slower than it needs to be.

Another "solution" that has been sometimes used is the creation of
specific adapters and/or converters from a class to another (e.g. PIL
offers the ImageTk module for converting PIL images to a class
compatible with the Tkinter one). But this approach doesn't scale well
with the number of libraries involved and it's still annoying for the
user: if I have a perfectly good image object why should I convert
before passing it to the next method, why can't it simply accept my
image as-is?

The problem isn't by any stretch limited to the three mentioned
libraries and has probably multiple causes, including two that IMO are
very important to understand before solving it:

-   in today's computing world an image is a basic type not strictly
    tied to a specific domain. This is why there will never be a clear
    winner between the image classes from the three libraries mentioned
    above (PIL, wxPython and pygame): they cover different domains and
    don't really compete with each other;
-   the Python standard library has never provided a good image class
    that can be adopted or imitated by third part modules.
    Tkinter.PhotoImage provides basic RGB functionality, but it's by far
    the slowest and ugliest of the bunch and it can be instantiated only
    after the Tkinter root window has been created.

This PEP tries to improve this situation in four ways:

1.  It defines a simple and pythonic image protocol/interface (both on
    the Python and the C side) that can be hopefully accepted and
    implemented by existing image classes inside and outside the
    standard library without breaking backward compatibility with their
    existing user bases.
2.  It proposes the inclusion in the standard library of three new
    classes:
    -   ImageMixin provides almost everything necessary to implement the
        new protocol; its main purpose is to make as simple as possible
        to support this interface for existing libraries, in some cases
        as simple as adding it to the list of base classes and doing
        minor additions to the constructor.
    -   Image is a subclass of ImageMixin and will add a constructor
        that can resize and/or convert an image between different pixel
        formats. This is intended to provide a fast and efficient
        default implementation of the new protocol.
    -   ImageSize is a minor helper class. See below for details.
3.  Tkinter.PhotoImage will implement the new protocol (mostly through
    the ImageMixin class) and all the Tkinter methods that can receive
    an image will be modified the accept any object that implements the
    interface. As an aside the author of this PEP will collaborate with
    the developers of the most common external libraries to achieve the
    same goal (supporting the protocol in their classes and accepting
    any class that implements it).
4.  New PyImage_* functions will be added to the CPython C API: they
    implement the C side of the protocol and accept as first parameter
    any object that supports it, even if it isn't an instance of the
    Image/ImageMixin classes.

The main effects for the end user will be a simplification of the
interchange of images between different libraries (if everything goes
well, any Python library will accept images from any other library) and
the out-of-the-box availability of the new Image class. The new class is
intended to cover simple but common use cases like cropping and/or
resizing a photograph to the desired size and passing it an appropriate
widget for displaying it on a window, or darkening a texture and passing
it to a 3D library.

The Image class is not intended to replace or compete with PIL,
Pythonmagick or NumPy, even if it provides a (very small) subset of the
functionality of these three libraries. In particular PIL offers very
rich image manipulation features with dozens of classes, filters,
transformations and file formats. The inclusion of PIL (or something
similar) in the standard library may, or may not, be a worthy goal but
it's completely outside the scope of this PEP.

Specification

The imageop module is used as the default location for the new classes
and objects because it has for a long time hosted functions that
provided a somewhat similar functionality, but a new module may be
created if preferred (e.g. a new "image" or "media" module; the latter
may eventually include other multimedia classes).

MODES is a new module level constant: it is a set of the pixel formats
supported by the Image class. Any image object that implements the new
protocol is guaranteed to be formatted in one of these modes, but
libraries that accept images are allowed to support only a subset of
them.

These modes are in turn also available as module level constants (e.g.
imageop.RGB).

The following table is a summary of the modes currently supported and
their properties:

+-------------+-------------+-------------+-------------+-------------+
|   Name      |   Component | Bits per    | Subsampling |   Valid     |
|             |   names     | component   |             |   intervals |
+-------------+-------------+-------------+-------------+-------------+
| ---------   | ---------   | ---------   | ---------   | ---------   |
+-------------+-------------+-------------+-------------+-------------+
| L           | l           | 8           | no          | full range  |
|             | (lowercase  |             |             |             |
|             | L)          |             |             |             |
+-------------+-------------+-------------+-------------+-------------+
| L16         | l           | 16          | no          | full range  |
+-------------+-------------+-------------+-------------+-------------+
| L32         | l           | 32          | no          | full range  |
+-------------+-------------+-------------+-------------+-------------+
| LA          | l, a        | 8           | no          | full range  |
+-------------+-------------+-------------+-------------+-------------+
| LA32        | l, a        | 16          | no          | full range  |
+-------------+-------------+-------------+-------------+-------------+
| RGB         | r, g, b     | 8           | no          | full range  |
+-------------+-------------+-------------+-------------+-------------+
| RGB48       | r, g, b     | 16          | no          | full range  |
+-------------+-------------+-------------+-------------+-------------+
| RGBA        | r, g, b, a  | 8           | no          | full range  |
+-------------+-------------+-------------+-------------+-------------+
| RGBA64      | r, g, b, a  | 16          | no          | full range  |
+-------------+-------------+-------------+-------------+-------------+
| YV12        | y, cr, cb   | 8           | 1, 2, 2     | 16-235,     |
|             |             |             |             | 16-240,     |
|             |             |             |             | 16-240      |
+-------------+-------------+-------------+-------------+-------------+
| JPEG_YV12   | y, cr, cb   | 8           | 1, 2, 2     | full range  |
+-------------+-------------+-------------+-------------+-------------+
| CMYK        | c, m, y, k  | 8           | no          | full range  |
+-------------+-------------+-------------+-------------+-------------+
| CMYK64      | c, m, y, k  | 16          | no          | full range  |
+-------------+-------------+-------------+-------------+-------------+

When the name of a mode ends with a number, it represents the average
number of bits per pixel. All the other modes simply use a byte per
component per pixel.

No palette modes or modes with less than 8 bits per component are
supported. Welcome to the 21st century.

Here's a quick description of the modes and the rationale for their
inclusion; there are four groups of modes:

1.  grayscale (L* modes): they are heavily used in scientific computing
    (those people may also need a very high dynamic range and precision,
    hence L32, the only mode with 32 bits per component) and sometimes
    it can be useful to consider a single component of a color image as
    a grayscale image (this is used by the individual planes of the
    planar images, see YV12 below); the name of the component ('l',
    lowercase letter L) stands for luminance, the second optional
    component ('a') is the alpha value and represents the opacity of the
    pixels: alpha = 0 means full transparency, alpha = 255/65535
    represents a fully opaque pixel;

2.  RGB* modes: the garden variety color images. The optional alpha
    component has the same meaning as in grayscale modes;

3.  YCbCr, a.k.a. YUV (*YV12 modes). These modes are planar (i.e. the
    values of all the pixel for each component are stored in a
    consecutive memory area, instead of the usual arrangement where all
    the components of a pixel reside in consecutive bytes) and use a 1,
    2, 2 (a.k.a. 4:2:0) subsampling (i.e. each pixel has its own Y
    value, but the Cb and Cr components are shared between groups of 2x2
    adjacent pixels) because this is the format that's by far the most
    common for YCbCr images. Please note that the V (Cr) plane is stored
    before the U (Cb) plane.

    YV12 is commonly used for MPEG2 (including DVDs), MPEG4 (both
    ASP/DivX and AVC/H.264) and Theora video frames. Valid values for Y
    are in range(16, 236) (excluding 236), and valid values for Cb and
    Cr are in range(16, 241). JPEG_YV12 is similar to YV12, but the
    three components can have the full range of 256 values. It's the
    native format used by almost all JPEG/JFIF files and by MJPEG video
    frames. The "strangeness" of these two wrt all the other supported
    modes derives from the fact that they are widely used that way by a
    lot of existing libraries and applications; this is also the reason
    why they are included (and the fact that they can't losslessly
    converted to RGB because YCbCr is a bigger color space); the funny
    4:2:0 planar arrangement of the pixel values is relatively easy to
    support because in most cases the three planes can be considered
    three separate grayscale images;

4.  CMYK* modes (cyan, magenta, yellow and black) are subtractive color
    modes, used for printing color images on dead trees. Professional
    designers love to pretend that they can't live without them, so here
    they are.

Python API

See the examples below.

In Python 2.x, all the new classes defined here are new-style classes.

Mode Objects

The mode objects offer a number of attributes and methods that can be
used for implementing generic algorithms that work on different types of
images:

components

  The number of components per pixel (e.g. 4 for an RGBA image).

component_names

  A tuple of strings; see the column "Component names" in the above
  table.

bits_per_component

  8, 16 or 32; see "Bits per component" in the above table.

bytes_per_pixel

  components * bits_per_component // 8, only available for non planar
  modes (see below).

planar

  Boolean; True if the image components reside each in a separate plane.
  Currently this happens if and only if the mode uses subsampling.

subsampling

  A tuple that for each component in the mode contains a tuple of two
  integers that represent the amount of downsampling in the horizontal
  and vertical direction, respectively. In practice it's
  ((1, 1), (2, 2), (2, 2)) for YV12 and JPEG_YV12 and
  ((1, 1),) * components for everything else.

x_divisor

  max(x for x, y in subsampling); the width of an image that uses this
  mode must be divisible for this value.

y_divisor

  max(y for x, y in subsampling); the height of an image that uses this
  mode must be divisible for this value.

intervals

  A tuple that for each component in the mode contains a tuple of two
  integers: the minimum and maximum valid value for the component. Its
  value is ((16, 235), (16, 240), (16, 240)) for YV12 and
  ((0, 2 ** bits_per_component - 1),) * components for everything else.

get_length(iterable[integer]) -> int

  The parameter must be an iterable that contains two integers: the
  width and height of an image; it returns the number of bytes needed to
  store an image of these dimensions with this mode.

Implementation detail: the modes are instances of a subclass of str and
have a value equal to their name (e.g. imageop.RGB == 'RGB') except for
L32 that has value 'I'. This is only intended for backward compatibility
with existing PIL users; new code that uses the image protocol proposed
here should not rely on this detail.

Image Protocol

Any object that supports the image protocol must provide the following
methods and attributes:

mode

  The format and the arrangement of the pixels in this image; it's one
  of the constants in the MODES set.

size

  An instance of the ImageSize class; it's a named tuple of two
  integers: the width and the height of the image in pixels; both of
  them must be >= 1 and can also be accessed as the width and height
  attributes of size.

buffer

  A sequence of integers between 0 and 255; they are the actual bytes
  used for storing the image data (i.e. modifying their values affects
  the image pixels and vice versa); the data has a
  row-major/C-contiguous order without padding and without any special
  memory alignment, even when there are more than 8 bits per component.
  The only supported methods are __len__, __getitem__/__setitem__ (with
  both integers and slice indexes) and __iter__; on the C side it
  implements the buffer protocol.

  This is a pretty low level interface to the image and the user is
  responsible for using the correct (native) byte order for modes with
  more than 8 bit per component and the correct value ranges for YV12
  images. A buffer may or may not keep a reference to its image, but
  it's still safe (if useless) to use the buffer even after the
  corresponding image has been destroyed by the garbage collector (this
  will require changes to the image class of wxPython and possibly other
  libraries). Implementation detail: this can be an array('B'), a
  bytes() object or a specialized fixed-length type.

info

  A dict object that can contain arbitrary metadata associated with the
  image (e.g. DPI, gamma, ICC profile, exposure time...); the
  interpretation of this data is beyond the scope of this PEP and
  probably depends on the library used to create and/or to save the
  image; if a method of the image returns a new image, it can copy or
  adapt metadata from its own info attribute (the ImageMixin
  implementation always creates a new image with an empty info
  dictionary).

bits_per_component
bytes_per_pixel
component_names
components
intervals
planar
subsampling

  Shortcuts for the corresponding mode.* attributes.

map(function[, function...]) -> None

  For every pixel in the image, maps each component through the
  corresponding function. If only one function is passed, it is used
  repeatedly for each component. This method modifies the image in place
  and is usually very fast (most of the time the functions are called
  only a small number of times, possibly only once for simple functions
  without branches), but it imposes a number of restrictions on the
  function(s) passed:

  -   it must accept a single integer argument and return a number (map
      will round the result to the nearest integer and clip it to
      range(0, 2 ** bits_per_component), if necessary);
  -   it must not try to intercept any BaseException, Exception or any
      unknown subclass of Exception raised by any operation on the
      argument (implementations may try to optimize the speed by passing
      funny objects, so even a simple "if n == 10:" may raise an
      exception: simply ignore it, map will take care of it); catching
      any other exception is fine;
  -   it should be side-effect free and its result should not depend on
      values (other than the argument) that may change during a single
      invocation of map.

rotate90() -> image
rotate180() -> image
rotate270() -> image

  Return a copy of the image rotated 90, 180 or 270 degrees
  counterclockwise around its center.

clip() -> None

  Saturates invalid component values in YV12 images to the minimum or
  the maximum allowed (see mode.intervals), for other image modes this
  method does nothing, very fast; libraries that save/export YV12 images
  are encouraged to always call this method, since intermediate
  operations (e.g. the map method) may assign to pixels values outside
  the valid intervals.

split() -> tuple[image]

  Returns a tuple of L, L16 or L32 images corresponding to the
  individual components in the image.

Planar images also supports attributes with the same names defined in
component_names: they contain grayscale (mode L) images that offer a
view on the pixel values for the corresponding component; any change to
the subimages is immediately reflected on the parent image and vice
versa (their buffers refer to the same memory location).

Non-planar images offer the following additional methods:

pixels() -> iterator[pixel]

  Returns an iterator that iterates over all the pixels in the image,
  starting from the top line and scanning each line from left to right.
  See below for a description of the pixel objects.

__iter__() -> iterator[line]

  Returns an iterator that iterates over all the lines in the image,
  from top to bottom. See below for a description of the line objects.

__len__() -> int

  Returns the number of lines in the image (size.height).

__getitem__(integer) -> line

  Returns the line at the specified (y) position.

__getitem__(tuple[integer]) -> pixel

  The parameter must be a tuple of two integers; they are interpreted
  respectively as x and y coordinates in the image (0, 0 is the top left
  corner) and a pixel object is returned.

__getitem__(slice | tuple[integer | slice]) -> image

  The parameter must be a slice or a tuple that contains two slices or
  an integer and a slice; the selected area of the image is copied and a
  new image is returned; image[x:y:z] is equivalent to image[:, x:y:z].

__setitem__(tuple[integer], integer | iterable[integer]) -> None

  Modifies the pixel at specified position; image[x, y] = integer is a
  shortcut for image[x, y] = (integer,) for images with a single
  component.

__setitem__(slice | tuple[integer | slice], image) -> None

  Selects an area in the same way as the corresponding form of the
  __getitem__ method and assigns to it a copy of the pixels from the
  image in the second argument, that must have exactly the same mode as
  this image and the same size as the specified area; the alpha
  component, if present, is simply copied and doesn't affect the other
  components of the image (i.e. no alpha compositing is performed).

The mode, size and buffer (including the address in memory of the
buffer) never change after an image is created.

It is expected that, if PEP 3118 is accepted, all the image objects will
support the new buffer protocol, however this is beyond the scope of
this PEP.

Image and ImageMixin Classes

The ImageMixin class implements all the methods and attributes described
above except mode, size, buffer and info. Image is a subclass of
ImageMixin that adds support for these four attributes and offers the
following constructor (please note that the constructor is not part of
the image protocol):

__init__(mode, size, color, source)

  mode must be one of the constants in the MODES set, size is a sequence
  of two integers (width and height of the new image); color is a
  sequence of integers, one for each component of the image, used to
  initialize all the pixels to the same value; source can be a sequence
  of integers of the appropriate size and format that is copied as-is in
  the buffer of the new image or an existing image; in Python 2.x source
  can also be an instance of str and is interpreted as a sequence of
  bytes. color and source are mutually exclusive and if they are both
  omitted the image is initialized to transparent black (all the bytes
  in the buffer have value 16 in the YV12 mode, 255 in the CMYK* modes
  and 0 for everything else). If source is present and is an image, mode
  and/or size can be omitted; if they are specified and are different
  from the source mode and/or size, the source image is converted.

  The exact algorithms used for resizing and doing color space
  conversions may differ between Python versions and implementations,
  but they always give high quality results (e.g.: a cubic spline
  interpolation can be used for upsampling and an antialias filter can
  be used for downsampling images); any combination of mode conversion
  is supported, but the algorithm used for conversions to and from the
  CMYK* modes is pretty naïve: if you have the exact color profiles of
  your devices you may want to use a good color management tool such as
  LittleCMS. The new image has an empty info dict.

Line Objects

The line objects (returned, e.g., when iterating over an image) support
the following attributes and methods:

mode

  The mode of the image from where this line comes.

__iter__() -> iterator[pixel]

  Returns an iterator that iterates over all the pixels in the line,
  from left to right. See below for a description of the pixel objects.

__len__() -> int

  Returns the number of pixels in the line (the image width).

__getitem__(integer) -> pixel

  Returns the pixel at the specified (x) position.

__getitem__(slice) -> image

  The selected part of the line is copied and a new image is returned;
  the new image will always have height 1.

__setitem__(integer, integer | iterable[integer]) -> None

  Modifies the pixel at the specified position; line[x] = integer is a
  shortcut for line[x] = (integer,) for images with a single component.

__setitem__(slice, image) -> None

  Selects a part of the line and assigns to it a copy of the pixels from
  the image in the second argument, that must have height 1, a width
  equal to the specified slice and the same mode as this line; the alpha
  component, if present, is simply copied and doesn't affect the other
  components of the image (i.e. no alpha compositing is performed).

Pixel Objects

The pixel objects (returned, e.g., when iterating over a line) support
the following attributes and methods:

mode

  The mode of the image from where this pixel comes.

value

  A tuple of integers, one for each component. Any iterable of the
  correct length can be assigned to value (it will be automagically
  converted to a tuple), but you can't assign to it an integer, even if
  the mode has only a single component: use, e.g., pixel.l = 123
  instead.

r, g, b, a, l, c, m, y, k

  The integer values of each component; only those applicable for the
  current mode (in mode.component_names) will be available.

__iter__() -> iterator[int]
__len__() -> int
__getitem__(integer | slice) -> int | tuple[int]
__setitem__(integer | slice, integer | iterable[integer]) ->                                                              None

  These four methods emulate a fixed length list of integers, one for
  each pixel component.

ImageSize Class

ImageSize is a named tuple, a class identical to tuple except that:

-   its constructor only accepts two integers, width and height; they
    are converted in the constructor using their __index__() methods, so
    all the ImageSize objects are guaranteed to contain only int (or
    possibly long, in Python 2.x) instances;
-   it has a width and a height property that are equivalent to the
    first and the second number in the tuple, respectively;
-   the string returned by its __repr__ method is
    'imageop.ImageSize(width=%d, height=%d)' % (width, height).

ImageSize is not usually instantiated by end-users, but can be used when
creating a new class that implements the image protocol, since the size
attribute must be an ImageSize instance.

C API

The available image modes are visible at the C level as PyImage_*
constants of type PyObject * (e.g.: PyImage_RGB is imageop.RGB).

The following functions offer a C-friendly interface to mode and image
objects (all the functions return NULL or -1 on failure):

int PyImageMode_Check(PyObject *obj)

  Returns true if the object obj is a valid image mode.

int PyImageMode_GetComponents(PyObject *mode)
PyObject* PyImageMode_GetComponentNames(PyObject *mode)
int PyImageMode_GetBitsPerComponent(PyObject *mode)
int PyImageMode_GetBytesPerPixel(PyObject *mode)
int PyImageMode_GetPlanar(PyObject *mode)
PyObject* PyImageMode_GetSubsampling(PyObject *mode)
int PyImageMode_GetXDivisor(PyObject *mode)
int PyImageMode_GetYDivisor(PyObject *mode)
Py_ssize_t PyImageMode_GetLength(PyObject *mode, Py_ssize_t width,                                     Py_ssize_t height)

  These functions are equivalent to their corresponding Python
  attributes or methods.

int PyImage_Check(PyObject *obj)

  Returns true if the object obj is an Image object or an instance of a
  subtype of the Image type; see also PyObject_CheckImage below.

int PyImage_CheckExact(PyObject *obj)

  Returns true if the object obj is an Image object, but not an instance
  of a subtype of the Image type.

PyObject* PyImage_New(PyObject *mode, Py_ssize_t width,                          Py_ssize_t height)

  Returns a new Image instance, initialized to transparent black (see
  Image.__init__ above for the details).

PyObject* PyImage_FromImage(PyObject *image, PyObject *mode,                                Py_ssize_t width, Py_ssize_t height)

  Returns a new Image instance, initialized with the contents of the
  image object rescaled and converted to the specified mode, if
  necessary.

PyObject* PyImage_FromBuffer(PyObject *buffer, PyObject *mode,                                 Py_ssize_t width,                                 Py_ssize_t height)

  Returns a new Image instance, initialized with the contents of the
  buffer object.

int PyObject_CheckImage(PyObject *obj)

  Returns true if the object obj implements a sufficient subset of the
  image protocol to be accepted by the functions defined below, even if
  its class is not a subclass of ImageMixin and/or Image. Currently it
  simply checks for the existence and correctness of the attributes
  mode, size and buffer.

PyObject* PyImage_GetMode(PyObject *image)
Py_ssize_t PyImage_GetWidth(PyObject *image)
Py_ssize_t PyImage_GetHeight(PyObject *image)
int PyImage_Clip(PyObject *image)
PyObject* PyImage_Split(PyObject *image)
PyObject* PyImage_GetBuffer(PyObject *image)
int PyImage_AsBuffer(PyObject *image, const void **buffer,                         Py_ssize_t *buffer_len)

  These functions are equivalent to their corresponding Python
  attributes or methods; the image memory can be accessed only with the
  GIL and a reference to the image or its buffer held, and extra care
  should be taken for modes with more than 8 bits per component: the
  data is stored in native byte order and it can be not aligned on 2 or
  4 byte boundaries.

Examples

A few examples of common operations with the new Image class and
protocol:

    # create a new black RGB image of 6x9 pixels
    rgb_image = imageop.Image(imageop.RGB, (6, 9))

    # same as above, but initialize the image to bright red
    rgb_image = imageop.Image(imageop.RGB, (6, 9), color=(255, 0, 0))

    # convert the image to YCbCr
    yuv_image = imageop.Image(imageop.JPEG_YV12, source=rgb_image)

    # read the value of a pixel and split it into three ints
    r, g, b = rgb_image[x, y]

    # modify the magenta component of a pixel in a CMYK image
    cmyk_image[x, y].m = 13

    # modify the Y (luma) component of a pixel in a *YV12 image and
    # its corresponding subsampled Cr (red chroma)
    yuv_image.y[x, y] = 42
    yuv_image.cr[x // 2, y // 2] = 54

    # iterate over an image
    for line in rgb_image:
        for pixel in line:
            # swap red and blue, and set green to 0
            pixel.value = pixel.b, 0, pixel.r

    # find the maximum value of the red component in the image
    max_red = max(pixel.r for pixel in rgb_image.pixels())

    # count the number of colors in the image
    num_of_colors = len(set(tuple(pixel) for pixel in image.pixels()))

    # copy a block of 4x2 pixels near the upper right corner of an
    # image and paste it into the lower left corner of the same image
    image[:4, -2:] = image[-6:-2, 1:3]

    # create a copy of the image, except that the new image can have a
    # different (usually empty) info dict
    new_image = image[:]

    # create a mirrored copy of the image, with the left and right
    # sides flipped
    flipped_image = image[::-1, :]

    # downsample an image to half its original size using a fast, low
    # quality operation and a slower, high quality one:
    low_quality_image = image[::2, ::2]
    new_size = image.size.width // 2, image.size.height // 2
    high_quality_image = imageop.Image(size=new_size, source=image)

    # direct buffer access
    rgb_image[0, 0] = r, g, b
    assert tuple(rgb_image.buffer[:3]) == (r, g, b)

Backwards Compatibility

There are three areas touched by this PEP where backwards compatibility
should be considered:

-   Python 2.6: new classes and objects are added to the imageop module
    without touching the existing module contents; new methods and
    attributes will be added to Tkinter.PhotoImage and its __getitem__
    and __setitem__ methods will be modified to accept integers, tuples
    and slices (currently they only accept strings). All the changes
    provide a superset of the existing functionality, so no major
    compatibility issues are expected.
-   Python 3.0: the legacy contents of the imageop module will be
    deleted, according to PEP 3108; everything defined in this proposal
    will work like in Python 2.x with the exception of the usual 2.x/3.0
    differences (e.g. support for long integers and for interpreting str
    instances as sequences of bytes will be dropped).
-   external libraries: the names and the semantics of the standard
    image methods and attributes are carefully chosen to allow some
    external libraries that manipulate images (including at least PIL,
    wxPython and pygame) to implement the new protocol in their image
    classes without breaking compatibility with existing code. The only
    blatant conflicts between the image protocol and NumPy arrays are
    the value of the size attribute and the coordinates order in the
    image[x, y] expression.

Reference Implementation

If this PEP is accepted, the author will provide a reference
implementation of the new classes in pure Python (that can run in
CPython, PyPy, Jython and IronPython) and a second one optimized for
speed in Python and C, suitable for inclusion in the CPython standard
library. The author will also submit the required Tkinter patches. For
all the code will be available a version for Python 2.x and a version
for Python 3.0 (it is expected that the two version will be very similar
and the Python 3.0 one will probably be generated almost completely
automatically).

Acknowledgments

The implementation of this PEP, if accepted, is sponsored by Google
through the Google Summer of Code program.

Copyright

This document has been placed in the public domain.

  Local Variables: mode: indented-text indent-tabs-mode: nil
  sentence-end-double-space: t fill-column: 70 coding: utf-8 End:


text:
PEP: 320 Title: Python 2.4 Release Schedule Version: $Revision$
Last-Modified: $Date$ Author: Barry Warsaw, Raymond Hettinger, Anthony
Baxter Status: Final Type: Informational Topic: Release Content-Type:
text/x-rst Created: 29-Jul-2003 Python-Version: 2.4 Post-History:
01-Dec-2004

Abstract

This document describes the development and release schedule for Python
2.4. The schedule primarily concerns itself with PEP-sized items. Small
features may be added up to and including the first beta release. Bugs
may be fixed until the final release.

There will be at least two alpha releases, two beta releases, and one
release candidate. The release date was 30th November, 2004.

Release Manager

Anthony Baxter

Martin von Lowis is building the Windows installers, Fred the doc
packages, Sean the RPMs.

Release Schedule

-   July 9: alpha 1 [completed]
-   August 5/6: alpha 2 [completed]
-   Sept 3: alpha 3 [completed]
-   October 15: beta 1 [completed]
-   November 3: beta 2 [completed]
-   November 18: release candidate 1 [completed]
-   November 30: final [completed]

Completed features for 2.4

-   PEP 218 Builtin Set Objects.
-   PEP 289 Generator expressions.
-   PEP 292 Simpler String Substitutions to be implemented as a module.
-   PEP 318: Function/method decorator syntax, using @syntax
-   PEP 322 Reverse Iteration.
-   PEP 327: A Decimal package for fixed precision arithmetic.
-   PEP 328: Multi-line Imports
-   Encapsulate the decorate-sort-undecorate pattern in a keyword for
    list.sort().
-   Added a builtin called sorted() which may be used in expressions.
-   The itertools module has two new functions, tee() and groupby().
-   Add a collections module with a deque() object.
-   Add two statistical/reduction functions, nlargest() and nsmallest()
    to the heapq module.
-   Python's windows installer now uses MSI

Deferred until 2.5

-   Deprecate and/or remove the modules listed in PEP 4 (posixfile,
    gopherlib, pre, others)
-   Remove support for platforms as described in PEP 11.
-   Finish implementing the Distutils bdist_dpkg command. (AMK)
-   Add support for reading shadow passwords[1]
-   It would be nice if the built-in SSL socket type could be used for
    non-blocking SSL I/O. Currently packages such as Twisted which
    implement async servers using SSL have to require third-party
    packages such as pyopenssl.
-   AST-based compiler: this branch was not completed in time for 2.4,
    but will land on the trunk some time after 2.4 final is out, for
    inclusion in 2.5.
-   reST is going to be used a lot in Zope3. Maybe it could become a
    standard library module? (Since reST's author thinks it's too
    instable, I'm inclined not to do this.)

Ongoing tasks

The following are ongoing TO-DO items which we should attempt to work on
without hoping for completion by any particular date.

-   Documentation: complete the distribution and installation manuals.
-   Documentation: complete the documentation for new-style classes.
-   Look over the Demos/ directory and update where required (Andrew
    Kuchling has done a lot of this)
-   New tests.
-   Fix doc bugs on SF.
-   Remove use of deprecated features in the core.
-   Document deprecated features appropriately.
-   Mark deprecated C APIs with Py_DEPRECATED.
-   Deprecate modules which are unmaintained, or perhaps make a new
    category for modules 'Unmaintained'
-   In general, lots of cleanup so it is easier to move forward.

Open issues

None at this time.

Carryover features from Python 2.3

-   The import lock could use some redesign.[2]

-   A nicer API to open text files, replacing the ugly (in some people's
    eyes) "U" mode flag. There's a proposal out there to have a new
    built-in type textfile(filename, mode, encoding). (Shouldn't it have
    a bufsize argument too?)

-   New widgets for Tkinter???

    Has anyone gotten the time for this? Are there any new widgets in Tk
    8.4? Note that we've got better Tix support already (though not on
    Windows yet).

-   PEP 304 (Controlling Generation of Bytecode Files by Montanaro)
    seems to have lost steam.

-   For a class defined inside another class, the __name__ should be
    "outer.inner", and pickling should work. ([3]. I'm no longer certain
    this is easy or even right.)

-   Decide on a clearer deprecation policy (especially for modules) and
    act on it. For a start, see this message from Neal Norwitz[4]. There
    seems insufficient interest in moving this further in an organized
    fashion, and it's not particularly important.

-   Provide alternatives for common uses of the types module; Skip
    Montanaro has posted a proto-PEP for this idea[5]. There hasn't been
    any progress on this, AFAICT.

-   Use pending deprecation for the types and string modules. This
    requires providing alternatives for the parts that aren't covered
    yet (e.g. string.whitespace and types.TracebackType). It seems we
    can't get consensus on this.

-   PEP 262 Database of Installed Python Packages (Kuchling)

    This turns out to be useful for Jack Jansen's Python installer, so
    the database is worth implementing. Code will go in sandbox/pep262.

-   PEP 269 Pgen Module for Python (Riehl)

    (Some necessary changes are in; the pgen module itself needs to
    mature more.)

-   PEP 266 Optimizing Global Variable/Attribute Access (Montanaro)

    PEP 267 Optimized Access to Module Namespaces (Hylton)

    PEP 280 Optimizing access to globals (van Rossum)

    These are basically three friendly competing proposals. Jeremy has
    made a little progress with a new compiler, but it's going slowly
    and the compiler is only the first step. Maybe we'll be able to
    refactor the compiler in this release. I'm tempted to say we won't
    hold our breath.

-   Lazily tracking tuples?[6][7] Not much enthusiasm I believe.

-   PEP 286 Enhanced Argument Tuples (von Loewis)

    I haven't had the time to review this thoroughly. It seems a deep
    optimization hack (also makes better correctness guarantees though).

-   Make 'as' a keyword. It has been a pseudo-keyword long enough. Too
    much effort to bother.

References

Copyright

This document has been placed in the public domain.

[1] Shadow Password Support Module https://bugs.python.org/issue579435

[2] PyErr_Warn may cause import deadlock
https://bugs.python.org/issue683658

[3] Nested class __name__ https://bugs.python.org/issue633930

[4] Neal Norwitz, random vs whrandom
https://mail.python.org/pipermail/python-dev/2002-April/023165.html

[5] Skip Montanaro, python/dist/src/Lib types.py,1.26,1.27
https://mail.python.org/pipermail/python-dev/2002-May/024346.html

[6] Daniel Dunbar, Lazily GC tracking tuples
https://mail.python.org/pipermail/python-dev/2002-May/023926.html

[7] GC: untrack simple objects https://bugs.python.org/issue558745

authors:
Anthony Baxter
Barry Warsaw
Raymond Hettinger