text:
PEP: 3101 Title: Advanced String Formatting Version: $Revision$
Last-Modified: $Date$ Author: Talin<EMAIL_ADDRESS>Status: Final
Type: Standards Track Content-Type: text/x-rst Created: 16-Apr-2006
Python-Version: 3.0 Post-History: 28-Apr-2006, 06-May-2006, 10-Jun-2007,
14-Aug-2007, 14-Sep-2008

Abstract

This PEP proposes a new system for built-in string formatting
operations, intended as a replacement for the existing '%' string
formatting operator.

Rationale

Python currently provides two methods of string interpolation:

-   The '%' operator for strings.[1]
-   The string.Template module.[2]

The primary scope of this PEP concerns proposals for built-in string
formatting operations (in other words, methods of the built-in string
type).

The '%' operator is primarily limited by the fact that it is a binary
operator, and therefore can take at most two arguments. One of those
arguments is already dedicated to the format string, leaving all other
variables to be squeezed into the remaining argument. The current
practice is to use either a dictionary or a tuple as the second
argument, but as many people have commented [3], this lacks flexibility.
The "all or nothing" approach (meaning that one must choose between only
positional arguments, or only named arguments) is felt to be overly
constraining.

While there is some overlap between this proposal and string.Template,
it is felt that each serves a distinct need, and that one does not
obviate the other. This proposal is for a mechanism which, like '%', is
efficient for small strings which are only used once, so, for example,
compilation of a string into a template is not contemplated in this
proposal, although the proposal does take care to define format strings
and the API in such a way that an efficient template package could reuse
the syntax and even some of the underlying formatting code.

Specification

The specification will consist of the following parts:

-   Specification of a new formatting method to be added to the built-in
    string class.
-   Specification of functions and flag values to be added to the string
    module, so that the underlying formatting engine can be used with
    additional options.
-   Specification of a new syntax for format strings.
-   Specification of a new set of special methods to control the
    formatting and conversion of objects.
-   Specification of an API for user-defined formatting classes.
-   Specification of how formatting errors are handled.

Note on string encodings: When discussing this PEP in the context of
Python 3.0, it is assumed that all strings are unicode strings, and that
the use of the word 'string' in the context of this document will
generally refer to a Python 3.0 string, which is the same as Python 2.x
unicode object.

In the context of Python 2.x, the use of the word 'string' in this
document refers to an object which may either be a regular string or a
unicode object. All of the function call interfaces described in this
PEP can be used for both strings and unicode objects, and in all cases
there is sufficient information to be able to properly deduce the output
string type (in other words, there is no need for two separate APIs). In
all cases, the type of the format string dominates - that is, the result
of the conversion will always result in an object that contains the same
representation of characters as the input format string.

String Methods

The built-in string class (and also the unicode class in 2.6) will gain
a new method, 'format', which takes an arbitrary number of positional
and keyword arguments:

    "The story of {0}, {1}, and {c}".format(a, b, c=d)

Within a format string, each positional argument is identified with a
number, starting from zero, so in the above example, 'a' is argument 0
and 'b' is argument 1. Each keyword argument is identified by its
keyword name, so in the above example, 'c' is used to refer to the third
argument.

There is also a global built-in function, 'format' which formats a
single value:

    print(format(10.0, "7.3g"))

This function is described in a later section.

Format Strings

Format strings consist of intermingled character data and markup.

Character data is data which is transferred unchanged from the format
string to the output string; markup is not transferred from the format
string directly to the output, but instead is used to define
'replacement fields' that describe to the format engine what should be
placed in the output string in place of the markup.

Brace characters ('curly braces') are used to indicate a replacement
field within the string:

    "My name is {0}".format('Fred')

The result of this is the string:

    "My name is Fred"

Braces can be escaped by doubling:

    "My name is {0} :-{{}}".format('Fred')

Which would produce:

    "My name is Fred :-{}"

The element within the braces is called a 'field'. Fields consist of a
'field name', which can either be simple or compound, and an optional
'format specifier'.

Simple and Compound Field Names

Simple field names are either names or numbers. If numbers, they must be
valid base-10 integers; if names, they must be valid Python identifiers.
A number is used to identify a positional argument, while a name is used
to identify a keyword argument.

A compound field name is a combination of multiple simple field names in
an expression:

    "My name is {0.name}".format(open('out.txt', 'w'))

This example shows the use of the 'getattr' or 'dot' operator in a field
expression. The dot operator allows an attribute of an input value to be
specified as the field value.

Unlike some other programming languages, you cannot embed arbitrary
expressions in format strings. This is by design - the types of
expressions that you can use is deliberately limited. Only two operators
are supported: the '.' (getattr) operator, and the '[]' (getitem)
operator. The reason for allowing these operators is that they don't
normally have side effects in non-pathological code.

An example of the 'getitem' syntax:

    "My name is {0[name]}".format(dict(name='Fred'))

It should be noted that the use of 'getitem' within a format string is
much more limited than its conventional usage. In the above example, the
string 'name' really is the literal string 'name', not a variable named
'name'. The rules for parsing an item key are very simple. If it starts
with a digit, then it is treated as a number, otherwise it is used as a
string.

Because keys are not quote-delimited, it is not possible to specify
arbitrary dictionary keys (e.g., the strings "10" or ":-]") from within
a format string.

Implementation note: The implementation of this proposal is not required
to enforce the rule about a simple or dotted name being a valid Python
identifier. Instead, it will rely on the getattr function of the
underlying object to throw an exception if the identifier is not legal.
The str.format() function will have a minimalist parser which only
attempts to figure out when it is "done" with an identifier (by finding
a '.' or a ']', or '}', etc.).

Format Specifiers

Each field can also specify an optional set of 'format specifiers' which
can be used to adjust the format of that field. Format specifiers follow
the field name, with a colon (':') character separating the two:

    "My name is {0:8}".format('Fred')

The meaning and syntax of the format specifiers depends on the type of
object that is being formatted, but there is a standard set of format
specifiers used for any object that does not override them.

Format specifiers can themselves contain replacement fields. For
example, a field whose field width is itself a parameter could be
specified via:

    "{0:{1}}".format(a, b)

These 'internal' replacement fields can only occur in the format
specifier part of the replacement field. Internal replacement fields
cannot themselves have format specifiers. This implies also that
replacement fields cannot be nested to arbitrary levels.

Note that the doubled '}' at the end, which would normally be escaped,
is not escaped in this case. The reason is because the '{{' and '}}'
syntax for escapes is only applied when used outside of a format field.
Within a format field, the brace characters always have their normal
meaning.

The syntax for format specifiers is open-ended, since a class can
override the standard format specifiers. In such cases, the str.format()
method merely passes all of the characters between the first colon and
the matching brace to the relevant underlying formatting method.

Standard Format Specifiers

If an object does not define its own format specifiers, a standard set
of format specifiers is used. These are similar in concept to the format
specifiers used by the existing '%' operator, however there are also a
number of differences.

The general form of a standard format specifier is:

    [[fill]align][sign][#][0][minimumwidth][.precision][type]

The brackets ([]) indicate an optional element.

Then the optional align flag can be one of the following:

    '<' - Forces the field to be left-aligned within the available
          space (This is the default.)
    '>' - Forces the field to be right-aligned within the
          available space.
    '=' - Forces the padding to be placed after the sign (if any)
          but before the digits.  This is used for printing fields
          in the form '+000000120'. This alignment option is only
          valid for numeric types.
    '^' - Forces the field to be centered within the available
          space.

Note that unless a minimum field width is defined, the field width will
always be the same size as the data to fill it, so that the alignment
option has no meaning in this case.

The optional 'fill' character defines the character to be used to pad
the field to the minimum width. The fill character, if present, must be
followed by an alignment flag.

The 'sign' option is only valid for numeric types, and can be one of the
following:

    '+'  - indicates that a sign should be used for both
           positive as well as negative numbers
    '-'  - indicates that a sign should be used only for negative
           numbers (this is the default behavior)
    ' '  - indicates that a leading space should be used on
           positive numbers

If the '#' character is present, integers use the 'alternate form' for
formatting. This means that binary, octal, and hexadecimal output will
be prefixed with '0b', '0o', and '0x', respectively.

'width' is a decimal integer defining the minimum field width. If not
specified, then the field width will be determined by the content.

If the width field is preceded by a zero ('0') character, this enables
zero-padding. This is equivalent to an alignment type of '=' and a fill
character of '0'.

The 'precision' is a decimal number indicating how many digits should be
displayed after the decimal point in a floating point conversion. For
non-numeric types the field indicates the maximum field size - in other
words, how many characters will be used from the field content. The
precision is ignored for integer conversions.

Finally, the 'type' determines how the data should be presented.

The available integer presentation types are:

    'b' - Binary. Outputs the number in base 2.
    'c' - Character. Converts the integer to the corresponding
          Unicode character before printing.
    'd' - Decimal Integer. Outputs the number in base 10.
    'o' - Octal format. Outputs the number in base 8.
    'x' - Hex format. Outputs the number in base 16, using
          lower-case letters for the digits above 9.
    'X' - Hex format. Outputs the number in base 16, using
          upper-case letters for the digits above 9.
    'n' - Number. This is the same as 'd', except that it uses the
          current locale setting to insert the appropriate
          number separator characters.
    '' (None) - the same as 'd'

The available floating point presentation types are:

    'e' - Exponent notation. Prints the number in scientific
          notation using the letter 'e' to indicate the exponent.
    'E' - Exponent notation. Same as 'e' except it converts the
          number to uppercase.
    'f' - Fixed point. Displays the number as a fixed-point
          number.
    'F' - Fixed point. Same as 'f' except it converts the number
          to uppercase.
    'g' - General format. This prints the number as a fixed-point
          number, unless the number is too large, in which case
          it switches to 'e' exponent notation.
    'G' - General format. Same as 'g' except switches to 'E'
          if the number gets to large.
    'n' - Number. This is the same as 'g', except that it uses the
          current locale setting to insert the appropriate
          number separator characters.
    '%' - Percentage. Multiplies the number by 100 and displays
          in fixed ('f') format, followed by a percent sign.
    '' (None) - similar to 'g', except that it prints at least one
          digit after the decimal point.

Objects are able to define their own format specifiers to replace the
standard ones. An example is the 'datetime' class, whose format
specifiers might look something like the arguments to the strftime()
function:

    "Today is: {0:%a %b %d %H:%M:%S %Y}".format(datetime.now())

For all built-in types, an empty format specification will produce the
equivalent of str(value). It is recommended that objects defining their
own format specifiers follow this convention as well.

Explicit Conversion Flag

The explicit conversion flag is used to transform the format field value
before it is formatted. This can be used to override the type-specific
formatting behavior, and format the value as if it were a more generic
type. Currently, two explicit conversion flags are recognized:

    !r - convert the value to a string using repr().
    !s - convert the value to a string using str().

These flags are placed before the format specifier:

    "{0!r:20}".format("Hello")

In the preceding example, the string "Hello" will be printed, with
quotes, in a field of at least 20 characters width.

A custom Formatter class can define additional conversion flags. The
built-in formatter will raise a ValueError if an invalid conversion flag
is specified.

Controlling Formatting on a Per-Type Basis

Each Python type can control formatting of its instances by defining a
__format__ method. The __format__ method is responsible for interpreting
the format specifier, formatting the value, and returning the resulting
string.

The new, global built-in function 'format' simply calls this special
method, similar to how len() and str() simply call their respective
special methods:

    def format(value, format_spec):
        return value.__format__(format_spec)

It is safe to call this function with a value of "None" (because the
"None" value in Python is an object and can have methods.)

Several built-in types, including 'str', 'int', 'float', and 'object'
define __format__ methods. This means that if you derive from any of
those types, your class will know how to format itself.

The object.__format__ method is the simplest: It simply converts the
object to a string, and then calls format again:

    class object:
        def __format__(self, format_spec):
            return format(str(self), format_spec)

The __format__ methods for 'int' and 'float' will do numeric formatting
based on the format specifier. In some cases, these formatting
operations may be delegated to other types. So for example, in the case
where the 'int' formatter sees a format type of 'f' (meaning 'float') it
can simply cast the value to a float and call format() again.

Any class can override the __format__ method to provide custom
formatting for that type:

    class AST:
        def __format__(self, format_spec):
            ...

Note for Python 2.x: The 'format_spec' argument will be either a string
object or a unicode object, depending on the type of the original format
string. The __format__ method should test the type of the specifiers
parameter to determine whether to return a string or unicode object. It
is the responsibility of the __format__ method to return an object of
the proper type.

Note that the 'explicit conversion' flag mentioned above is not passed
to the __format__ method. Rather, it is expected that the conversion
specified by the flag will be performed before calling __format__.

User-Defined Formatting

There will be times when customizing the formatting of fields on a
per-type basis is not enough. An example might be a spreadsheet
application, which displays hash marks '#' when a value is too large to
fit in the available space.

For more powerful and flexible formatting, access to the underlying
format engine can be obtained through the 'Formatter' class that lives
in the 'string' module. This class takes additional options which are
not accessible via the normal str.format method.

An application can subclass the Formatter class to create its own
customized formatting behavior.

The PEP does not attempt to exactly specify all methods and properties
defined by the Formatter class; instead, those will be defined and
documented in the initial implementation. However, this PEP will specify
the general requirements for the Formatter class, which are listed
below.

Although string.format() does not directly use the Formatter class to do
formatting, both use the same underlying implementation. The reason that
string.format() does not use the Formatter class directly is because
"string" is a built-in type, which means that all of its methods must be
implemented in C, whereas Formatter is a Python class. Formatter
provides an extensible wrapper around the same C functions as are used
by string.format().

Formatter Methods

The Formatter class takes no initialization arguments:

    fmt = Formatter()

The public API methods of class Formatter are as follows:

    -- format(format_string, *args, **kwargs)
    -- vformat(format_string, args, kwargs)

'format' is the primary API method. It takes a format template, and an
arbitrary set of positional and keyword arguments. 'format' is just a
wrapper that calls 'vformat'.

'vformat' is the function that does the actual work of formatting. It is
exposed as a separate function for cases where you want to pass in a
predefined dictionary of arguments, rather than unpacking and repacking
the dictionary as individual arguments using the *args and **kwds
syntax. 'vformat' does the work of breaking up the format template
string into character data and replacement fields. It calls the
'get_positional' and 'get_index' methods as appropriate (described
below.)

Formatter defines the following overridable methods:

    -- get_value(key, args, kwargs)
    -- check_unused_args(used_args, args, kwargs)
    -- format_field(value, format_spec)

'get_value' is used to retrieve a given field value. The 'key' argument
will be either an integer or a string. If it is an integer, it
represents the index of the positional argument in 'args'; If it is a
string, then it represents a named argument in 'kwargs'.

The 'args' parameter is set to the list of positional arguments to
'vformat', and the 'kwargs' parameter is set to the dictionary of
positional arguments.

For compound field names, these functions are only called for the first
component of the field name; subsequent components are handled through
normal attribute and indexing operations.

So for example, the field expression '0.name' would cause 'get_value' to
be called with a 'key' argument of 0. The 'name' attribute will be
looked up after 'get_value' returns by calling the built-in 'getattr'
function.

If the index or keyword refers to an item that does not exist, then an
IndexError/KeyError should be raised.

'check_unused_args' is used to implement checking for unused arguments
if desired. The arguments to this function is the set of all argument
keys that were actually referred to in the format string (integers for
positional arguments, and strings for named arguments), and a reference
to the args and kwargs that was passed to vformat. The set of unused
args can be calculated from these parameters. 'check_unused_args' is
assumed to throw an exception if the check fails.

'format_field' simply calls the global 'format' built-in. The method is
provided so that subclasses can override it.

To get a better understanding of how these functions relate to each
other, here is pseudocode that explains the general operation of
vformat:

    def vformat(format_string, args, kwargs):

      # Output buffer and set of used args
      buffer = StringIO.StringIO()
      used_args = set()

      # Tokens are either format fields or literal strings
      for token in self.parse(format_string):
        if is_format_field(token):
          # Split the token into field value and format spec
          field_spec, _, format_spec = token.partition(":")

          # Check for explicit type conversion
          explicit, _, field_spec  = field_spec.rpartition("!")

          # 'first_part' is the part before the first '.' or '['
          # Assume that 'get_first_part' returns either an int or
          # a string, depending on the syntax.
          first_part = get_first_part(field_spec)
          value = self.get_value(first_part, args, kwargs)

          # Record the fact that we used this arg
          used_args.add(first_part)

          # Handle [subfield] or .subfield. Assume that 'components'
          # returns an iterator of the various subfields, not including
          # the first part.
          for comp in components(field_spec):
            value = resolve_subfield(value, comp)

          # Handle explicit type conversion
          if explicit == 'r':
            value = repr(value)
          elif explicit == 's':
            value = str(value)

          # Call the global 'format' function and write out the converted
          # value.
          buffer.write(self.format_field(value, format_spec))

        else:
          buffer.write(token)

      self.check_unused_args(used_args, args, kwargs)
      return buffer.getvalue()

Note that the actual algorithm of the Formatter class (which will be
implemented in C) may not be the one presented here. (It's likely that
the actual implementation won't be a 'class' at all - rather, vformat
may just call a C function which accepts the other overridable methods
as arguments.) The primary purpose of this code example is to illustrate
the order in which overridable methods are called.

Customizing Formatters

This section describes some typical ways that Formatter objects can be
customized.

To support alternative format-string syntax, the 'vformat' method can be
overridden to alter the way format strings are parsed.

One common desire is to support a 'default' namespace, so that you don't
need to pass in keyword arguments to the format() method, but can
instead use values in a pre-existing namespace. This can easily be done
by overriding get_value() as follows:

    class NamespaceFormatter(Formatter):
       def __init__(self, namespace={}):
           Formatter.__init__(self)
           self.namespace = namespace

       def get_value(self, key, args, kwds):
           if isinstance(key, str):
               try:
                   # Check explicitly passed arguments first
                   return kwds[key]
               except KeyError:
                   return self.namespace[key]
           else:
               Formatter.get_value(key, args, kwds)

One can use this to easily create a formatting function that allows
access to global variables, for example:

    fmt = NamespaceFormatter(globals())

    greeting = "hello"
    print(fmt.format("{greeting}, world!"))

A similar technique can be done with the locals() dictionary to gain
access to the locals dictionary.

It would also be possible to create a 'smart' namespace formatter that
could automatically access both locals and globals through snooping of
the calling stack. Due to the need for compatibility with the different
versions of Python, such a capability will not be included in the
standard library, however it is anticipated that someone will create and
publish a recipe for doing this.

Another type of customization is to change the way that built-in types
are formatted by overriding the 'format_field' method. (For non-built-in
types, you can simply define a __format__ special method on that type.)
So for example, you could override the formatting of numbers to output
scientific notation when needed.

Error handling

There are two classes of exceptions which can occur during formatting:
exceptions generated by the formatter code itself, and exceptions
generated by user code (such as a field object's 'getattr' function).

In general, exceptions generated by the formatter code itself are of the
"ValueError" variety -- there is an error in the actual "value" of the
format string. (This is not always true; for example, the
string.format() function might be passed a non-string as its first
parameter, which would result in a TypeError.)

The text associated with these internally generated ValueError
exceptions will indicate the location of the exception inside the format
string, as well as the nature of the exception.

For exceptions generated by user code, a trace record and dummy frame
will be added to the traceback stack to help in determining the location
in the string where the exception occurred. The inserted traceback will
indicate that the error occurred at:

    File "<format_string>;", line XX, in column_YY

where XX and YY represent the line and character position information in
the string, respectively.

Alternate Syntax

Naturally, one of the most contentious issues is the syntax of the
format strings, and in particular the markup conventions used to
indicate fields.

Rather than attempting to exhaustively list all of the various
proposals, I will cover the ones that are most widely used already.

-   Shell variable syntax: $name and $(name) (or in some variants,
    ${name}). This is probably the oldest convention out there, and is
    used by Perl and many others. When used without the braces, the
    length of the variable is determined by lexically scanning until an
    invalid character is found.

    This scheme is generally used in cases where interpolation is
    implicit - that is, in environments where any string can contain
    interpolation variables, and no special substitution function need
    be invoked. In such cases, it is important to prevent the
    interpolation behavior from occurring accidentally, so the '$'
    (which is otherwise a relatively uncommonly-used character) is used
    to signal when the behavior should occur.

    It is the author's opinion, however, that in cases where the
    formatting is explicitly invoked, that less care needs to be taken
    to prevent accidental interpolation, in which case a lighter and
    less unwieldy syntax can be used.

-   printf and its cousins ('%'), including variations that add a field
    index, so that fields can be interpolated out of order.

-   Other bracket-only variations. Various MUDs (Multi-User Dungeons)
    such as MUSH have used brackets (e.g. [name]) to do string
    interpolation. The Microsoft .Net libraries uses braces ({}), and a
    syntax which is very similar to the one in this proposal, although
    the syntax for format specifiers is quite different.[4]

-   Backquoting. This method has the benefit of minimal syntactical
    clutter, however it lacks many of the benefits of a function call
    syntax (such as complex expression arguments, custom formatters,
    etc.).

-   Other variations include Ruby's #{}, PHP's {$name}, and so on.

Some specific aspects of the syntax warrant additional comments:

1) Backslash character for escapes. The original version of this PEP
used backslash rather than doubling to escape a bracket. This worked
because backslashes in Python string literals that don't conform to a
standard backslash sequence such as \n are left unmodified. However,
this caused a certain amount of confusion, and led to potential
situations of multiple recursive escapes, i.e. \\\\{ to place a literal
backslash in front of a bracket.

2) The use of the colon character (':') as a separator for format
specifiers. This was chosen simply because that's what .Net uses.

Alternate Feature Proposals

Restricting attribute access: An earlier version of the PEP restricted
the ability to access attributes beginning with a leading underscore,
for example "{0}._private". However, this is a useful ability to have
when debugging, so the feature was dropped.

Some developers suggested that the ability to do 'getattr' and 'getitem'
access should be dropped entirely. However, this is in conflict with the
needs of another set of developers who strongly lobbied for the ability
to pass in a large dict as a single argument (without flattening it into
individual keyword arguments using the **kwargs syntax) and then have
the format string refer to dict entries individually.

There has also been suggestions to expand the set of expressions that
are allowed in a format string. However, this was seen to go against the
spirit of TOOWTDI, since the same effect can be achieved in most cases
by executing the same expression on the parameter before it's passed in
to the formatting function. For cases where the format string is being
use to do arbitrary formatting in a data-rich environment, it's
recommended to use a template engine specialized for this purpose, such
as Genshi[5] or Cheetah[6].

Many other features were considered and rejected because they could
easily be achieved by subclassing Formatter instead of building the
feature into the base implementation. This includes alternate syntax,
comments in format strings, and many others.

Security Considerations

Historically, string formatting has been a common source of security
holes in web-based applications, particularly if the string formatting
system allows arbitrary expressions to be embedded in format strings.

The best way to use string formatting in a way that does not create
potential security holes is to never use format strings that come from
an untrusted source.

Barring that, the next best approach is to ensure that string formatting
has no side effects. Because of the open nature of Python, it is
impossible to guarantee that any non-trivial operation has this
property. What this PEP does is limit the types of expressions in format
strings to those in which visible side effects are both rare and
strongly discouraged by the culture of Python developers. So for
example, attribute access is allowed because it would be considered
pathological to write code where the mere access of an attribute has
visible side effects (whether the code has invisible side effects - such
as creating a cache entry for faster lookup - is irrelevant.)

Sample Implementation

An implementation of an earlier version of this PEP was created by
Patrick Maupin and Eric V. Smith, and can be found in the pep3101
sandbox at:

  http://svn.python.org/view/sandbox/trunk/pep3101/

Backwards Compatibility

Backwards compatibility can be maintained by leaving the existing
mechanisms in place. The new system does not collide with any of the
method names of the existing string formatting techniques, so both
systems can co-exist until it comes time to deprecate the older system.

References

Copyright

This document has been placed in the public domain.

[1] Python Library Reference - String formatting operations
http://docs.python.org/library/stdtypes.html#string-formatting-operations

[2] Python Library References - Template strings
http://docs.python.org/library/string.html#string.Template

[3] [Python-3000] String formatting operations in python 3k
https://mail.python.org/pipermail/python-3000/2006-April/000285.html

[4] Composite Formatting - [.Net Framework Developer's Guide]
http://msdn.microsoft.com/library/en-us/cpguide/html/cpconcompositeformatting.asp?frame=true

[5] Genshi templating engine. http://genshi.edgewall.org/

[6] Cheetah - The Python-Powered Template Engine.
http://www.cheetahtemplate.org/


text:
PEP: 103 Title: Collecting information about git Author: Oleg Broytman
<EMAIL_ADDRESS>Status: Withdrawn Type: Informational Created:
01-Jun-2015 Post-History: 12-Sep-2015

It is too generic and doesn't really deal with Python development. It is
no longer updated.

The content was moved to Python Wiki. Make further updates in the wiki.

Abstract

This Informational PEP collects information about git. There is, of
course, a lot of documentation for git, so the PEP concentrates on more
complex (and more related to Python development) issues, scenarios and
examples.

The plan is to extend the PEP in the future collecting information about
equivalence of Mercurial and git scenarios to help migrating Python
development from Mercurial to git.

The author of the PEP doesn't currently plan to write a Process PEP on
migration Python development from Mercurial to git.

Documentation

Git is accompanied with a lot of documentation, both online and offline.

Documentation for starters

Git Tutorial: part 1, part 2.

Git User's manual. Everyday GIT With 20 Commands Or So. Git workflows.

Advanced documentation

Git Magic, with a number of translations.

Pro Git. The Book about git. Buy it at Amazon or download in PDF, mobi,
or ePub form. It has translations to many different languages. Download
Russian translation from GArik.

Git Wiki.

Git Buch (German).

Offline documentation

Git has builtin help: run git help $TOPIC. For example, run git help git
or git help help.

Quick start

Download and installation

Unix users: download and install using your package manager.

Microsoft Windows: download git-for-windows.

MacOS X: use git installed with XCode or download from MacPorts or
git-osx-installer or install git with Homebrew: brew install git.

git-cola (repository) is a Git GUI written in Python and GPL licensed.
Linux, Windows, MacOS X.

TortoiseGit is a Windows Shell Interface to Git based on TortoiseSVN;
open source.

Initial configuration

This simple code is often appears in documentation, but it is important
so let repeat it here. Git stores author and committer names/emails in
every commit, so configure your real name and preferred email:

    $ git config --global user.name "User Name"
    $ git config --global user.email<EMAIL_ADDRESS>
Examples in this PEP

Examples of git commands in this PEP use the following approach. It is
supposed that you, the user, works with a local repository named python
that has an upstream remote repo named origin. Your local repo has two
branches v1 and master. For most examples the currently checked out
branch is master. That is, it's assumed you have done something like
that:

    $ git clone https://git.python.org/python.git
    $ cd python
    $ git branch v1 origin/v1

The first command clones remote repository into local directory python,
creates a new local branch master, sets remotes/origin/master as its
upstream remote-tracking branch and checks it out into the working
directory.

The last command creates a new local branch v1 and sets
remotes/origin/v1 as its upstream remote-tracking branch.

The same result can be achieved with commands:

    $ git clone -b v1 https://git.python.org/python.git
    $ cd python
    $ git checkout --track origin/master

The last command creates a new local branch master, sets
remotes/origin/master as its upstream remote-tracking branch and checks
it out into the working directory.

Branches and branches

Git terminology can be a bit misleading. Take, for example, the term
"branch". In git it has two meanings. A branch is a directed line of
commits (possibly with merges). And a branch is a label or a pointer
assigned to a line of commits. It is important to distinguish when you
talk about commits and when about their labels. Lines of commits are by
itself unnamed and are usually only lengthening and merging. Labels, on
the other hand, can be created, moved, renamed and deleted freely.

Remote repositories and remote branches

Remote-tracking branches are branches (pointers to commits) in your
local repository. They are there for git (and for you) to remember what
branches and commits have been pulled from and pushed to what remote
repos (you can pull from and push to many remotes). Remote-tracking
branches live under remotes/$REMOTE namespaces, e.g.
remotes/origin/master.

To see the status of remote-tracking branches run:

    $ git branch -rv

To see local and remote-tracking branches (and tags) pointing to
commits:

    $ git log --decorate

You never do your own development on remote-tracking branches. You
create a local branch that has a remote branch as upstream and do
development on that local branch. On push git pushes commits to the
remote repo and updates remote-tracking branches, on pull git fetches
commits from the remote repo, updates remote-tracking branches and
fast-forwards, merges or rebases local branches.

When you do an initial clone like this:

    $ git clone -b v1 https://git.python.org/python.git

git clones remote repository https://git.python.org/python.git to
directory python, creates a remote named origin, creates remote-tracking
branches, creates a local branch v1, configure it to track upstream
remotes/origin/v1 branch and checks out v1 into the working directory.

Some commands, like git status --branch and git branch --verbose, report
the difference between local and remote branches. Please remember they
only do comparison with remote-tracking branches in your local
repository, and the state of those remote-tracking branches can be
outdated. To update remote-tracking branches you either fetch and merge
(or rebase) commits from the remote repository or update remote-tracking
branches without updating local branches.

Updating local and remote-tracking branches

To update remote-tracking branches without updating local branches run
git remote update [$REMOTE...]. For example:

    $ git remote update
    $ git remote update origin

Fetch and pull

There is a major difference between

    $ git fetch $REMOTE $BRANCH

and

    $ git fetch $REMOTE $BRANCH:$BRANCH

The first command fetches commits from the named $BRANCH in the $REMOTE
repository that are not in your repository, updates remote-tracking
branch and leaves the id (the hash) of the head commit in file
.git/FETCH_HEAD.

The second command fetches commits from the named $BRANCH in the $REMOTE
repository that are not in your repository and updates both the local
branch $BRANCH and its upstream remote-tracking branch. But it refuses
to update branches in case of non-fast-forward. And it refuses to update
the current branch (currently checked out branch, where HEAD is pointing
to).

The first command is used internally by git pull.

    $ git pull $REMOTE $BRANCH

is equivalent to

    $ git fetch $REMOTE $BRANCH
    $ git merge FETCH_HEAD

Certainly, $BRANCH in that case should be your current branch. If you
want to merge a different branch into your current branch first update
that non-current branch and then merge:

    $ git fetch origin v1:v1  # Update v1
    $ git pull --rebase origin master  # Update the current branch master
                                       # using rebase instead of merge
    $ git merge v1

If you have not yet pushed commits on v1, though, the scenario has to
become a bit more complex. Git refuses to update non-fast-forwardable
branch, and you don't want to do force-pull because that would remove
your non-pushed commits and you would need to recover. So you want to
rebase v1 but you cannot rebase non-current branch. Hence, checkout v1
and rebase it before merging:

    $ git checkout v1
    $ git pull --rebase origin v1
    $ git checkout master
    $ git pull --rebase origin master
    $ git merge v1

It is possible to configure git to make it fetch/pull a few branches or
all branches at once, so you can simply run

    $ git pull origin

or even

    $ git pull

Default remote repository for fetching/pulling is origin. Default set of
references to fetch is calculated using matching algorithm: git fetches
all branches having the same name on both ends.

Push

Pushing is a bit simpler. There is only one command push. When you run

    $ git push origin v1 master

git pushes local v1 to remote v1 and local master to remote master. The
same as:

    $ git push origin v1:v1 master:master

Git pushes commits to the remote repo and updates remote-tracking
branches. Git refuses to push commits that aren't fast-forwardable. You
can force-push anyway, but please remember - you can force-push to your
own repositories but don't force-push to public or shared repos. If you
find git refuses to push commits that aren't fast-forwardable, better
fetch and merge commits from the remote repo (or rebase your commits on
top of the fetched commits), then push. Only force-push if you know what
you do and why you do it. See the section Commit editing and caveats
below.

It is possible to configure git to make it push a few branches or all
branches at once, so you can simply run

    $ git push origin

or even

    $ git push

Default remote repository for pushing is origin. Default set of
references to push in git before 2.0 is calculated using matching
algorithm: git pushes all branches having the same name on both ends.
Default set of references to push in git 2.0+ is calculated using simple
algorithm: git pushes the current branch back to its @{upstream}.

To configure git before 2.0 to the new behaviour run:

    $ git config push.default simple

To configure git 2.0+ to the old behaviour run:

    $ git config push.default matching

Git doesn't allow to push a branch if it's the current branch in the
remote non-bare repository: git refuses to update remote working
directory. You really should push only to bare repositories. For
non-bare repositories git prefers pull-based workflow.

When you want to deploy code on a remote host and can only use push
(because your workstation is behind a firewall and you cannot pull from
it) you do that in two steps using two repositories: you push from the
workstation to a bare repo on the remote host, ssh to the remote host
and pull from the bare repo to a non-bare deployment repo.

That changed in git 2.3, but see the blog post for caveats; in 2.4 the
push-to-deploy feature was further improved.

Tags

Git automatically fetches tags that point to commits being fetched
during fetch/pull. To fetch all tags (and commits they point to) run
git fetch --tags origin. To fetch some specific tags fetch them
explicitly:

    $ git fetch origin tag $TAG1 tag $TAG2...

For example:

    $ git fetch origin tag 1.4.2
    $ git fetch origin v1:v1 tag 2.1.7

Git doesn't automatically pushes tags. That allows you to have private
tags. To push tags list them explicitly:

    $ git push origin tag 1.4.2
    $ git push origin v1 master tag 2.1.7

Or push all tags at once:

    $ git push --tags origin

Don't move tags with git tag -f or remove tags with git tag -d after
they have been published.

Private information

When cloning/fetching/pulling/pushing git copies only database objects
(commits, trees, files and tags) and symbolic references (branches and
lightweight tags). Everything else is private to the repository and
never cloned, updated or pushed. It's your config, your hooks, your
private exclude file.

If you want to distribute hooks, copy them to the working tree, add,
commit, push and instruct the team to update and install the hooks
manually.

Commit editing and caveats

A warning not to edit published (pushed) commits also appears in
documentation but it's repeated here anyway as it's very important.

It is possible to recover from a forced push but it's PITA for the
entire team. Please avoid it.

To see what commits have not been published yet compare the head of the
branch with its upstream remote-tracking branch:

    $ git log origin/master..  # from origin/master to HEAD (of master)
    $ git log origin/v1..v1  # from origin/v1 to the head of v1

For every branch that has an upstream remote-tracking branch git
maintains an alias @{upstream} (short version @{u}), so the commands
above can be given as:

    $ git log @{u}..
    $ git log<EMAIL_ADDRESS>
To see the status of all branches:

    $ git branch -avv

To compare the status of local branches with a remote repo:

    $ git remote show origin

Read how to recover from upstream rebase. It is in git help rebase.

On the other hand, don't be too afraid about commit editing. You can
safely edit, reorder, remove, combine and split commits that haven't
been pushed yet. You can even push commits to your own (backup) repo,
edit them later and force-push edited commits to replace what have
already been pushed. Not a problem until commits are in a public or
shared repository.

Undo

Whatever you do, don't panic. Almost anything in git can be undone.

git checkout: restore file's content

git checkout, for example, can be used to restore the content of file(s)
to that one of a commit. Like this:

    git checkout HEAD~ README

The commands restores the contents of README file to the last but one
commit in the current branch. By default the commit ID is simply HEAD;
i.e. git checkout README restores README to the latest commit.

(Do not use git checkout to view a content of a file in a commit, use
git cat-file -p; e.g. git cat-file -p HEAD~:path/to/README).

git reset: remove (non-pushed) commits

git reset moves the head of the current branch. The head can be moved to
point to any commit but it's often used to remove a commit or a few
(preferably, non-pushed ones) from the top of the branch - that is, to
move the branch backward in order to undo a few (non-pushed) commits.

git reset has three modes of operation - soft, hard and mixed. Default
is mixed. ProGit explains the difference very clearly. Bare repositories
don't have indices or working trees so in a bare repo only soft reset is
possible.

Unstaging

Mixed mode reset with a path or paths can be used to unstage changes
-that is, to remove from index changes added with git add for
committing. See The Book for details about unstaging and other undo
tricks.

git reflog: reference log

Removing commits with git reset or moving the head of a branch sounds
dangerous and it is. But there is a way to undo: another reset back to
the original commit. Git doesn't remove commits immediately;
unreferenced commits (in git terminology they are called "dangling
commits") stay in the database for some time (default is two weeks) so
you can reset back to it or create a new branch pointing to the original
commit.

For every move of a branch's head - with git commit, git checkout,
git fetch, git pull, git rebase, git reset and so on - git stores a
reference log (reflog for short). For every move git stores where the
head was. Command git reflog can be used to view (and manipulate) the
log.

In addition to the moves of the head of every branch git stores the
moves of the HEAD - a symbolic reference that (usually) names the
current branch. HEAD is changed with git checkout $BRANCH.

By default git reflog shows the moves of the HEAD, i.e. the command is
equivalent to git reflog HEAD. To show the moves of the head of a branch
use the command git reflog $BRANCH.

So to undo a git reset lookup the original commit in git reflog, verify
it with git show or git log and run git reset $COMMIT_ID. Git stores the
move of the branch's head in reflog, so you can undo that undo later
again.

In a more complex situation you'd want to move some commits along with
resetting the head of the branch. Cherry-pick them to the new branch.
For example, if you want to reset the branch master back to the original
commit but preserve two commits created in the current branch do
something like:

    $ git branch save-master  # create a new branch saving master
    $ git reflog  # find the original place of master
    $ git reset $COMMIT_ID
    $ git cherry-pick save-master~ save-master
    $ git branch -D save-master  # remove temporary branch

git revert: revert a commit

git revert reverts a commit or commits, that is, it creates a new commit
or commits that revert(s) the effects of the given commits. It's the
only way to undo published commits (git commit --amend, git rebase and
git reset change the branch in non-fast-forwardable ways so they should
only be used for non-pushed commits.)

There is a problem with reverting a merge commit. git revert can undo
the code created by the merge commit but it cannot undo the fact of
merge. See the discussion How to revert a faulty merge.

One thing that cannot be undone

Whatever you undo, there is one thing that cannot be undone -overwritten
uncommitted changes. Uncommitted changes don't belong to git so git
cannot help preserving them.

Most of the time git warns you when you're going to execute a command
that overwrites uncommitted changes. Git doesn't allow you to switch
branches with git checkout. It stops you when you're going to rebase
with non-clean working tree. It refuses to pull new commits over
non-committed files.

But there are commands that do exactly that - overwrite files in the
working tree. Commands like git checkout $PATHs or git reset --hard
silently overwrite files including your uncommitted changes.

With that in mind you can understand the stance "commit early, commit
often". Commit as often as possible. Commit on every save in your editor
or IDE. You can edit your commits before pushing - edit commit messages,
change commits, reorder, combine, split, remove. But save your changes
in git database, either commit changes or at least stash them with
git stash.

Merge or rebase?

Internet is full of heated discussions on the topic: "merge or rebase?"
Most of them are meaningless. When a DVCS is being used in a big team
with a big and complex project with many branches there is simply no way
to avoid merges. So the question's diminished to "whether to use rebase,
and if yes - when to use rebase?" Considering that it is very much
recommended not to rebase published commits the question's diminished
even further: "whether to use rebase on non-pushed commits?"

That small question is for the team to decide. To preserve the beauty of
linear history it's recommended to use rebase when pulling, i.e. do
git pull --rebase or even configure automatic setup of rebase for every
new branch:

    $ git config branch.autosetuprebase always

and configure rebase for existing branches:

    $ git config branch.$NAME.rebase true

For example:

    $ git config branch.v1.rebase true
    $ git config branch.master.rebase true

After that git pull origin master becomes equivalent to
git pull --rebase origin master.

It is recommended to create new commits in a separate feature or topic
branch while using rebase to update the mainline branch. When the topic
branch is ready merge it into mainline. To avoid a tedious task of
resolving large number of conflicts at once you can merge the topic
branch to the mainline from time to time and switch back to the topic
branch to continue working on it. The entire workflow would be something
like:

    $ git checkout -b issue-42  # create a new issue branch and switch to it
        ...edit/test/commit...
    $ git checkout master
    $ git pull --rebase origin master  # update master from the upstream
    $ git merge issue-42
    $ git branch -d issue-42  # delete the topic branch
    $ git push origin master

When the topic branch is deleted only the label is removed, commits are
stayed in the database, they are now merged into master:

    o--o--o--o--o--M--< master - the mainline branch
        \         /
         --*--*--*             - the topic branch, now unnamed

The topic branch is deleted to avoid cluttering branch namespace with
small topic branches. Information on what issue was fixed or what
feature was implemented should be in the commit messages.

But even that small amount of rebasing could be too big in case of
long-lived merged branches. Imagine you're doing work in both v1 and
master branches, regularly merging v1 into master. After some time you
will have a lot of merge and non-merge commits in master. Then you want
to push your finished work to a shared repository and find someone has
pushed a few commits to v1. Now you have a choice of two equally bad
alternatives: either you fetch and rebase v1 and then have to recreate
all you work in master (reset master to the origin, merge v1 and
cherry-pick all non-merge commits from the old master); or merge the new
v1 and loose the beauty of linear history.

Null-merges

Git has a builtin merge strategy for what Python core developers call
"null-merge":

    $ git merge -s ours v1  # null-merge v1 into master

Branching models

Git doesn't assume any particular development model regarding branching
and merging. Some projects prefer to graduate patches from the oldest
branch to the newest, some prefer to cherry-pick commits backwards, some
use squashing (combining a number of commits into one). Anything is
possible.

There are a few examples to start with. git help workflows describes how
the very git authors develop git.

ProGit book has a few chapters devoted to branch management in different
projects: Git Branching - Branching Workflows and Distributed Git -
Contributing to a Project.

There is also a well-known article A successful Git branching model by
Vincent Driessen. It recommends a set of very detailed rules on creating
and managing mainline, topic and bugfix branches. To support the model
the author implemented git flow extension.

Advanced configuration

Line endings

Git has builtin mechanisms to handle line endings between platforms with
different end-of-line styles. To allow git to do CRLF conversion assign
text attribute to files using .gitattributes. For files that have to
have specific line endings assign eol attribute. For binary files the
attribute is, naturally, binary.

For example:

    $ cat .gitattributes
    *.py text
    *.txt text
    *.png binary
    /readme.txt eol=CRLF

To check what attributes git uses for files use git check-attr command.
For example:

    $ git check-attr -a -- \*.py

Useful assets

GitAlias (repository) is a big collection of aliases. A careful
selection of aliases for frequently used commands could save you a lot
of keystrokes!

GitIgnore and https://github.com/github/gitignore are collections of
.gitignore files for all kinds of IDEs and programming languages. Python
included!

pre-commit (repositories) is a framework for managing and maintaining
multi-language pre-commit hooks. The framework is written in Python and
has a lot of plugins for many programming languages.

Advanced topics

Staging area

Staging area aka index aka cache is a distinguishing feature of git.
Staging area is where git collects patches before committing them.
Separation between collecting patches and commit phases provides a very
useful feature of git: you can review collected patches before commit
and even edit them - remove some hunks, add new hunks and review again.

To add files to the index use git add. Collecting patches before
committing means you need to do that for every change, not only to add
new (untracked) files. To simplify committing in case you just want to
commit everything without reviewing run git commit --all (or just -a) -
the command adds every changed tracked file to the index and then
commit. To commit a file or files regardless of patches collected in the
index run git commit [--only|-o] -- $FILE....

To add hunks of patches to the index use git add --patch (or just -p).
To remove collected files from the index use git reset HEAD -- $FILE...
To add/inspect/remove collected hunks use git add --interactive (-i).

To see the diff between the index and the last commit (i.e., collected
patches) use git diff --cached. To see the diff between the working tree
and the index (i.e., uncollected patches) use just git diff. To see the
diff between the working tree and the last commit (i.e., both collected
and uncollected patches) run git diff HEAD.

See WhatIsTheIndex and IndexCommandQuickref in Git Wiki.

Root

Git switches to the root (top-level directory of the project where .git
subdirectory exists) before running any command. Git remembers though
the directory that was current before the switch. Some programs take
into account the current directory. E.g., git status shows file paths of
changed and unknown files relative to the current directory; git grep
searches below the current directory; git apply applies only those hunks
from the patch that touch files below the current directory.

But most commands run from the root and ignore the current directory.
Imagine, for example, that you have two work trees, one for the branch
v1 and the other for master. If you want to merge v1 from a subdirectory
inside the second work tree you must write commands as if you're in the
top-level dir. Let take two work trees, project-v1 and project, for
example:

    $ cd project/subdirectory
    $ git fetch ../project-v1 v1:v1
    $ git merge v1

Please note the path in git fetch ../project-v1 v1:v1 is ../project-v1
and not ../../project-v1 despite the fact that we run the commands from
a subdirectory, not from the root.

ReReRe

Rerere is a mechanism that helps to resolve repeated merge conflicts.
The most frequent source of recurring merge conflicts are topic branches
that are merged into mainline and then the merge commits are removed;
that's often performed to test the topic branches and train rerere;
merge commits are removed to have clean linear history and finish the
topic branch with only one last merge commit.

Rerere works by remembering the states of tree before and after a
successful commit. That way rerere can automatically resolve conflicts
if they appear in the same files.

Rerere can be used manually with git rerere command but most often it's
used automatically. Enable rerere with these commands in a working tree:

    $ git config rerere.enabled true
    $ git config rerere.autoupdate true

You don't need to turn rerere on globally - you don't want rerere in
bare repositories or single-branch repositories; you only need rerere in
repos where you often perform merges and resolve merge conflicts.

See Rerere in The Book.

Database maintenance

Git object database and other files/directories under .git require
periodic maintenance and cleanup. For example, commit editing left
unreferenced objects (dangling objects, in git terminology) and these
objects should be pruned to avoid collecting cruft in the DB. The
command git gc is used for maintenance. Git automatically runs
git gc --auto as a part of some commands to do quick maintenance. Users
are recommended to run git gc --aggressive from time to time;
git help gc recommends to run it every few hundred changesets; for more
intensive projects it should be something like once a week and less
frequently (biweekly or monthly) for lesser active projects.

git gc --aggressive not only removes dangling objects, it also repacks
object database into indexed and better optimized pack(s); it also packs
symbolic references (branches and tags). Another way to do it is to run
git repack.

There is a well-known message from Linus Torvalds regarding "stupidity"
of git gc --aggressive. The message can safely be ignored now. It is old
and outdated, git gc --aggressive became much better since that time.

For those who still prefer git repack over git gc --aggressive the
recommended parameters are git repack -a -d -f --depth=20 --window=250.
See this detailed experiment for explanation of the effects of these
parameters.

From time to time run git fsck [--strict] to verify integrity of the
database. git fsck may produce a list of dangling objects; that's not an
error, just a reminder to perform regular maintenance.

Tips and tricks

Command-line options and arguments

git help cli recommends not to combine short options/flags. Most of the
times combining works: git commit -av works perfectly, but there are
situations when it doesn't. E.g., git log -p -5 cannot be combined as
git log -p5.

Some options have arguments, some even have default arguments. In that
case the argument for such option must be spelled in a sticky way:
-Oarg, never -O arg because for an option that has a default argument
the latter means "use default value for option -O and pass arg further
to the option parser". For example, git grep has an option -O that
passes a list of names of the found files to a program; default program
for -O is a pager (usually less), but you can use your editor:

    $ git grep -Ovim  # but not -O vim

BTW, if git is instructed to use less as the pager (i.e., if pager is
not configured in git at all it uses less by default, or if it gets less
from GIT_PAGER or PAGER environment variables, or if it was configured
with git config [--global] core.pager less, or less is used in the
command git grep -Oless) git grep passes +/$pattern option to less which
is quite convenient. Unfortunately, git grep doesn't pass the pattern if
the pager is not exactly less, even if it's less with parameters
(something like git config [--global] core.pager less -FRSXgimq);
fortunately, git grep -Oless always passes the pattern.

bash/zsh completion

It's a bit hard to type
git rebase --interactive --preserve-merges HEAD~5 manually even for
those who are happy to use command-line, and this is where shell
completion is of great help. Bash/zsh come with programmable completion,
often automatically installed and enabled, so if you have bash/zsh and
git installed, chances are you are already done - just go and use it at
the command-line.

If you don't have necessary bits installed, install and enable
bash_completion package. If you want to upgrade your git completion to
the latest and greatest download necessary file from git contrib.

Git-for-windows comes with git-bash for which bash completion is
installed and enabled.

bash/zsh prompt

For command-line lovers shell prompt can carry a lot of useful
information. To include git information in the prompt use git-prompt.sh.
Read the detailed instructions in the file.

Search the Net for "git prompt" to find other prompt variants.

SSH connection sharing

SSH connection sharing is a feature of OpenSSH and perhaps derivatives
like PuTTY. SSH connection sharing is a way to decrease ssh client
startup time by establishing one connection and reusing it for all
subsequent clients connecting to the same server. SSH connection sharing
can be used to speedup a lot of short ssh sessions like scp, sftp, rsync
and of course git over ssh. If you regularly fetch/pull/push from/to
remote repositories accessible over ssh then using ssh connection
sharing is recommended.

To turn on ssh connection sharing add something like this to your
~/.ssh/config:

    Host *
    ControlMaster auto
    ControlPath ~/.ssh/mux-%r@%h:%p
    ControlPersist 600

See OpenSSH wikibook and search for more information.

SSH connection sharing can be used at GitHub, GitLab and SourceForge
repositories, but please be advised that BitBucket doesn't allow it and
forcibly closes master connection after a short inactivity period so you
will see errors like this from ssh: "Connection to bitbucket.org closed
by remote host."

git on server

The simplest way to publish a repository or a group of repositories is
git daemon. The daemon provides anonymous access, by default it is
read-only. The repositories are accessible by git protocol (git://
URLs). Write access can be enabled but the protocol lacks any
authentication means, so it should be enabled only within a trusted LAN.
See git help daemon for details.

Git over ssh provides authentication and repo-level authorisation as
repositories can be made user- or group-writeable (see parameter
core.sharedRepository in git help config). If that's too permissive or
too restrictive for some project's needs there is a wrapper gitolite
that can be configured to allow access with great granularity; gitolite
is written in Perl and has a lot of documentation.

Web interface to browse repositories can be created using gitweb or
cgit. Both are CGI scripts (written in Perl and C). In addition to web
interface both provide read-only dumb http access for git (http(s)://
URLs). Klaus is a small and simple WSGI web server that implements both
web interface and git smart HTTP transport; supports Python 2 and Python
3, performs syntax highlighting.

There are also more advanced web-based development environments that
include ability to manage users, groups and projects; private,
group-accessible and public repositories; they often include issue
trackers, wiki pages, pull requests and other tools for development and
communication. Among these environments are Kallithea and pagure, both
are written in Python; pagure was written by Fedora developers and is
being used to develop some Fedora projects. GitPrep is yet another
GitHub clone, written in Perl. Gogs is written in Go. GitBucket is
written in Scala.

And last but not least, GitLab. It's perhaps the most advanced web-based
development environment for git. Written in Ruby, community edition is
free and open source (MIT license).

From Mercurial to git

There are many tools to convert Mercurial repositories to git. The most
famous are, probably, hg-git and fast-export (many years ago it was
known under the name hg2git).

But a better tool, perhaps the best, is git-remote-hg. It provides
transparent bidirectional (pull and push) access to Mercurial
repositories from git. Its author wrote a comparison of alternatives
that seems to be mostly objective.

To use git-remote-hg, install or clone it, add to your PATH (or copy
script git-remote-hg to a directory that's already in PATH) and prepend
hg:: to Mercurial URLs. For example:

    $ git clone https://github.com/felipec/git-remote-hg.git
    $ PATH=$PATH:"`pwd`"/git-remote-hg
    $ git clone hg::https://hg.python.org/peps/ PEPs

To work with the repository just use regular git commands including
git fetch/pull/push.

To start converting your Mercurial habits to git see the page Mercurial
for Git users at Mercurial wiki. At the second half of the page there is
a table that lists corresponding Mercurial and git commands. Should work
perfectly in both directions.

Python Developer's Guide also has a chapter Mercurial for git developers
that documents a few differences between git and hg.

Git and GitHub

gitsome - Git/GitHub command line interface (CLI). Written in Python,
work on MacOS, Unix, Windows. Git/GitHub CLI with autocomplete, includes
many GitHub integrated commands that work with all shells, builtin xonsh
with Python REPL to run Python commands alongside shell commands,
command history, customizable highlighting, thoroughly documented.

Copyright

This document has been placed in the public domain.

license: Public Domain


text:
PEP: 618 Title: Add Optional Length-Checking To zip Version: $Revision$
Last-Modified: $Date$ Author: Brandt Bucher<EMAIL_ADDRESS>Sponsor:
Antoine Pitrou<EMAIL_ADDRESS>BDFL-Delegate: Guido van Rossum
<EMAIL_ADDRESS>Status: Final Type: Standards Track Content-Type:
text/x-rst Created: 01-May-2020 Python-Version: 3.10 Post-History:
01-May-2020, 10-May-2020, 16-Jun-2020 Resolution:
<EMAIL_ADDRESS>
Abstract

This PEP proposes adding an optional strict boolean keyword parameter to
the built-in zip. When enabled, a ValueError is raised if one of the
arguments is exhausted before the others.

Motivation

It is clear from the author's personal experience and a survey of the
standard library that much (if not most) zip usage involves iterables
that must be of equal length. Sometimes this invariant is proven true
from the context of the surrounding code, but often the data being
zipped is passed from the caller, sourced separately, or generated in
some fashion. In any of these cases, the default behavior of zip means
that faulty refactoring or logic errors could easily result in silently
losing data. These bugs are not only difficult to diagnose, but
difficult to even detect at all.

It is easy to come up with simple cases where this could be a problem.
For example, the following code may work fine when items is a sequence,
but silently start producing shortened, mismatched results if items is
refactored by the caller to be a consumable iterator:

    def apply_calculations(items):
        transformed = transform(items)
        for i, t in zip(items, transformed):
            yield calculate(i, t)

There are several other ways in which zip is commonly used. Idiomatic
tricks are especially susceptible, because they are often employed by
users who lack a complete understanding of how the code works. One
example is unpacking into zip to lazily "unzip" or "transpose" nested
iterables:

    >>> x = [[1, 2, 3], ["one" "two" "three"]]
    >>> xt = list(zip(*x))

Another is "chunking" data into equal-sized groups:

    >>> n = 3
    >>> x = range(n ** 2),
    >>> xn = list(zip(*[iter(x)] * n))

In the first case, non-rectangular data is usually a logic error. In the
second case, data with a length that is not a multiple of n is often an
error as well. However, both of these idioms will silently omit the
tail-end items of malformed input.

Perhaps most convincingly, the use of zip in the standard-library ast
module created a bug in literal_eval which silently dropped parts of
malformed nodes:

    >>> from ast import Constant, Dict, literal_eval
    >>> nasty_dict = Dict(keys=[Constant(None)], values=[])
    >>> literal_eval(nasty_dict)  # Like eval("{None: }")
    {}

In fact, the author has counted dozens of other call sites in Python's
standard library and tooling where it would be appropriate to enable
this new feature immediately.

Rationale

Some critics assert that constant boolean switches are a "code-smell",
or go against Python's design philosophy. However, Python currently
contains several examples of boolean keyword parameters on built-in
functions which are typically called with compile-time constants:

-   compile(..., dont_inherit=True)
-   open(..., closefd=False)
-   print(..., flush=True)
-   sorted(..., reverse=True)

Many more exist in the standard library.

The idea and name for this new parameter were originally proposed by Ram
Rachum. The thread received over 100 replies, with the alternative
"equal" receiving a similar amount of support.

The author does not have a strong preference between the two choices,
though "equal equals" is a bit awkward in prose. It may also (wrongly)
imply some notion of "equality" between the zipped items:

    >>> z = zip([2.0, 4.0, 6.0], [2, 4, 8], equal=True)

Specification

When the built-in zip is called with the keyword-only argument
strict=True, the resulting iterator will raise a ValueError if the
arguments are exhausted at differing lengths. This error will occur at
the point when iteration would normally stop today.

Backward Compatibility

This change is fully backward-compatible. zip currently takes no keyword
arguments, and the "non-strict" default behavior when strict is omitted
remains unchanged.

Reference Implementation

The author has drafted a C implementation.

An approximate Python translation is:

    def zip(*iterables, strict=False):
        if not iterables:
            return
        iterators = tuple(iter(iterable) for iterable in iterables)
        try:
            while True:
                items = []
                for iterator in iterators:
                    items.append(next(iterator))
                yield tuple(items)
        except StopIteration:
            if not strict:
                return
        if items:
            i = len(items)
            plural = " " if i == 1 else "s 1-"
            msg = f"zip() argument {i+1} is shorter than argument{plural}{i}"
            raise ValueError(msg)
        sentinel = object()
        for i, iterator in enumerate(iterators[1:], 1):
            if next(iterator, sentinel) is not sentinel:
                plural = " " if i == 1 else "s 1-"
                msg = f"zip() argument {i+1} is longer than argument{plural}{i}"
                raise ValueError(msg)

Rejected Ideas

Add itertools.zip_strict

This is the alternative with the most support on the Python-Ideas
mailing list, so it deserves to be discussed in some detail here. It
does not have any disqualifying flaws, and could work well enough as a
substitute if this PEP is rejected.

With that in mind, this section aims to outline why adding an optional
parameter to zip is a smaller change that ultimately does a better job
of solving the problems motivating this PEP.

Precedent

It seems that a great deal of the motivation driving this alternative is
that zip_longest already exists in itertools. However, zip_longest is in
many ways a much more complicated, specialized utility: it takes on the
responsibility of filling in missing values, a job neither of the other
variants needs to concern themselves with.

If both zip and zip_longest lived alongside each other in itertools or
as builtins, then adding zip_strict in the same location would indeed be
a much stronger argument. However, the new "strict" variant is
conceptually much closer to zip in interface and behavior than
zip_longest, while still not meeting the high bar of being its own
builtin. Given this situation, it seems most natural for zip to grow
this new option in-place.

Usability

If zip is capable of preventing this class of bug, it becomes much
simpler for users to enable the check at call sites with this property.
Compare this with importing a drop-in replacement for a built-in
utility, which feels somewhat heavy just to check a tricky condition
that should "always" be true.

Some have also argued that a new function buried in the standard library
is somehow more "discoverable" than a keyword parameter on the built-in
itself. The author does not agree with this assessment.

Maintenance Cost

While implementation should only be a secondary concern when making
usability improvements, it is important to recognize that adding a new
utility is significantly more complicated than modifying an existing
one. The CPython implementation accompanying this PEP is simple and has
no measurable performance impact on default zip behavior, while adding
an entirely new utility to itertools would require either:

-   Duplicating much of the existing zip logic, as zip_longest already
    does.
-   Significantly refactoring either zip, zip_longest, or both to share
    a common or inherited implementation (which may impact performance).

Add Several "Modes" To Switch Between

This option only makes more sense than a binary flag if we anticipate
having three or more modes. The "obvious" three choices for these
enumerated or constant modes would be "shortest" (the current zip
behavior), "strict" (the proposed behavior), and "longest" (the
itertools.zip_longest behavior).

However, it doesn't seem like adding behaviors other than the current
default and the proposed "strict" mode is worth the additional
complexity. The clearest candidate, "longest", would require a new
fillvalue parameter (which is meaningless for both other modes). This
mode is also already handled perfectly by itertools.zip_longest, and
adding it would create two ways of doing the same thing. It's not clear
which would be the "obvious" choice: the mode parameter on the built-in
zip, or the long-lived namesake utility in itertools.

Add A Method Or Alternate Constructor To The zip Type

Consider the following two options, which have both been proposed:

    >>> zm = zip(*iters).strict()
    >>> zd = zip.strict(*iters)

It's not obvious which one will succeed, or how the other will fail. If
zip.strict is implemented as a method, zm will succeed, but zd will fail
in one of several confusing ways:

-   Yield results that aren't wrapped in a tuple (if iters contains just
    one item, a zip iterator).
-   Raise a TypeError for an incorrect argument type (if iters contains
    just one item, not a zip iterator).
-   Raise a TypeError for an incorrect number of arguments (otherwise).

If zip.strict is implemented as a classmethod or staticmethod, zd will
succeed, and zm will silently yield nothing (which is the problem we are
trying to avoid in the first place).

This proposal is further complicated by the fact that CPython's actual
zip type is currently an undocumented implementation detail. This means
that choosing one of the above behaviors will effectively "lock in" the
current implementation (or at least require it to be emulated) going
forward.

Change The Default Behavior Of zip

There is nothing "wrong" with the default behavior of zip, since there
are many cases where it is indeed the correct way to handle
unequally-sized inputs. It's extremely useful, for example, when dealing
with infinite iterators.

itertools.zip_longest already exists to service those cases where the
"extra" tail-end data is still needed.

Accept A Callback To Handle Remaining Items

While able to do basically anything a user could need, this solution
makes handling the more common cases (like rejecting mismatched lengths)
unnecessarily complicated and non-obvious.

Raise An AssertionError

There are no built-in functions or types that raise an AssertionError as
part of their API. Further, the official documentation simply reads (in
its entirety):

  Raised when an assert statement fails.

Since this feature has nothing to do with Python's assert statement,
raising an AssertionError here would be inappropriate. Users desiring a
check that is disabled in optimized mode (like an assert statement) can
use strict=__debug__ instead.

Add A Similar Feature to map

This PEP does not propose any changes to map, since the use of map with
multiple iterable arguments is quite rare. However, this PEP's ruling
shall serve as precedent such a future discussion (should it occur).

If rejected, the feature is realistically not worth pursuing. If
accepted, such a change to map should not require its own PEP (though,
like all enhancements, its usefulness should be carefully considered).
For consistency, it should follow same API and semantics debated here
for zip.

Do Nothing

This option is perhaps the least attractive.

Silently truncated data is a particularly nasty class of bug, and
hand-writing a robust solution that gets this right isn't trivial. The
real-world motivating examples from Python's own standard library are
evidence that it's very easy to fall into the sort of trap that this
feature aims to avoid.

References

Examples

Note

This listing is not exhaustive.

-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/_pydecimal.py#L3394
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/_pydecimal.py#L3418
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/_pydecimal.py#L3435
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L94-L95
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L1184
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L1275
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L1363
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/ast.py#L1391
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/copy.py#L217
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/csv.py#L142
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/dis.py#L462
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/filecmp.py#L142
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/filecmp.py#L143
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/inspect.py#L1440
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/inspect.py#L2095
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/os.py#L510
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/plistlib.py#L577
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/tarfile.py#L1317
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/tarfile.py#L1323
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/tarfile.py#L1339
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/turtle.py#L3015
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/turtle.py#L3071
-   https://github.com/python/cpython/blob/27c0d9b54abaa4112d5a317b8aa78b39ad60a808/Lib/turtle.py#L3901

Copyright

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.


text:
PEP: 283 Title: Python 2.3 Release Schedule Version: $Revision$
Last-Modified: $Date$ Author: Guido van Rossum Status: Final Type:
Informational Topic: Release Content-Type: text/x-rst Created:
27-Feb-2002 Python-Version: 2.3 Post-History: 27-Feb-2002

Abstract

This document describes the development and release schedule for Python
2.3. The schedule primarily concerns itself with PEP-sized items. Small
features may be added up to and including the first beta release. Bugs
may be fixed until the final release.

There will be at least two alpha releases, two beta releases, and one
release candidate. Alpha and beta releases will be spaced at least 4
weeks apart (except if an emergency release must be made to correct a
blunder in the previous release; then the blunder release does not
count). Release candidates will be spaced at least one week apart
(excepting again blunder corrections).

  ------------- -------------
  alpha 1       31 Dec 2002
  alpha 2       19 Feb 2003
  beta 1        25 Apr 2003
  beta 2        29 Jun 2003
  candidate 1   18 Jul 2003
  candidate 2   24 Jul 2003
  final         29 Jul 2003
  ------------- -------------

Release Manager

Barry Warsaw, Jeremy Hylton, Tim Peters

Completed features for 2.3

This list is not complete. See Doc/whatsnew/whatsnew23.tex in CVS for
more, and of course Misc/NEWS for the full list.

-   Tk 8.4 update.

-   The bool type and its constants, True and False (PEP 285).

-   PyMalloc was greatly enhanced and is enabled by default.

-   Universal newline support (PEP 278).

-   PEP 263 Defining Python Source Code Encodings, Lemburg

    Implemented (at least phase 1, which is all that's planned for 2.3).

-   Extended slice notation for all built-in sequences. The patch by
    Michael Hudson is now all checked in.

-   Speed up list iterations by filling tp_iter and other tweaks. See
    https://bugs.python.org/issue560736; also done for xrange and
    tuples.

-   Timeout sockets. https://bugs.python.org/issue555085

-   Stage B0 of the int/long integration (PEP 237). This means issuing a
    FutureWarning about situations where hex or oct conversions or left
    shifts returns a different value for an int than for a long with the
    same value. The semantics do not change in Python 2.3; that will
    happen in Python 2.4.

-   Nuke SET_LINENO from all code objects (providing a different way to
    set debugger breakpoints). This can boost pystone by >5%.
    https://bugs.python.org/issue587993, now checked in. (Unfortunately
    the pystone boost didn't happen. What happened?)

-   Write a pymemcompat.h that people can bundle with their extensions
    and then use the 2.3 memory interface with all Pythons in the range
    1.5.2 to 2.3. (Michael Hudson checked in Misc/pymemcompat.h.)

-   Add a new concept, "pending deprecation", with associated warning
    PendingDeprecationWarning. This warning is normally suppressed, but
    can be enabled by a suitable -W option. Only a few things use this
    at this time.

-   Warn when an extension type's tp_compare returns anything except -1,
    0 or 1. https://bugs.python.org/issue472523

-   Warn for assignment to None (in various forms).

-   PEP 218 Adding a Built-In Set Object Type, Wilson

    Alex Martelli contributed a new version of Greg Wilson's prototype,
    and I've reworked that quite a bit. It's in the standard library now
    as the module sets, although some details may still change until the
    first beta release. (There are no plans to make this a built-in
    type, for now.)

-   PEP 293 Codec error handling callbacks, Dörwald

    Fully implemented. Error handling in unicode.encode or str.decode
    can now be customized.

-   PEP 282 A Logging System, Mick

    Vinay Sajip's implementation has been packagized and imported.
    (Documentation and unit tests still pending.)
    https://bugs.python.org/issue578494

-   A modified MRO (Method Resolution Order) algorithm. Consensus is
    that we should adopt C3. Samuele Pedroni has contributed a draft
    implementation in C, see https://bugs.python.org/issue619475 This
    has now been checked in.

-   A new command line option parser. Greg Ward's Optik package
    (http://optik.sf.net) has been adopted, converted to a single module
    named optparse. See also http://www.python.org/sigs/getopt-sig/

-   A standard datetime type. This started as a wiki:
    http://www.zope.org/Members/fdrake/DateTimeWiki/FrontPage. A
    prototype was coded in nondist/sandbox/datetime/. Tim Peters has
    finished the C implementation and checked it in.

-   PEP 273 Import Modules from Zip Archives, Ahlstrom

    Implemented as a part of the PEP 302 implementation work.

-   PEP 302 New Import Hooks, JvR

    Implemented (though the 2.3a1 release contained some bugs that have
    been fixed post-release).

-   A new pickling protocol. See PEP 307.

-   PEP 305 (CSV File API, by Skip Montanaro et al.) is in; this is the
    csv module.

-   Raymond Hettinger's itertools module is in.

-   PEP 311 (Simplified GIL Acquisition for Extensions, by Mark Hammond)
    has been included in beta 1.

-   Two new PyArg_Parse*() format codes, 'k' returns an unsigned C long
    int that receives the lower LONG_BIT bits of the Python argument,
    truncating without range checking. 'K' returns an unsigned C long
    long int that receives the lower LONG_LONG_BIT bits, truncating
    without range checking. (SF 595026; Thomas Heller did this work.)

-   A new version of IDLE was imported from the IDLEfork project
    (http://idlefork.sf.net). The code now lives in the idlelib package
    in the standard library and the idle script is installed by
    setup.py.

Planned features for 2.3

Too late for anything more to get done here.

Ongoing tasks

The following are ongoing TO-DO items which we should attempt to work on
without hoping for completion by any particular date.

-   Documentation: complete the distribution and installation manuals.
-   Documentation: complete the documentation for new-style classes.
-   Look over the Demos/ directory and update where required (Andrew
    Kuchling has done a lot of this)
-   New tests.
-   Fix doc bugs on SF.
-   Remove use of deprecated features in the core.
-   Document deprecated features appropriately.
-   Mark deprecated C APIs with Py_DEPRECATED.
-   Deprecate modules which are unmaintained, or perhaps make a new
    category for modules 'Unmaintained'
-   In general, lots of cleanup so it is easier to move forward.

Open issues

There are some issues that may need more work and/or thought before the
final release (and preferably before the first beta release): No issues
remaining.

Features that did not make it into Python 2.3

-   The import lock could use some redesign. (SF 683658.)

-   Set API issues; is the sets module perfect?

    I expect it's good enough to stop polishing it until we've had more
    widespread user experience.

-   A nicer API to open text files, replacing the ugly (in some people's
    eyes) "U" mode flag. There's a proposal out there to have a new
    built-in type textfile(filename, mode, encoding). (Shouldn't it have
    a bufsize argument too?)

    Ditto.

-   New widgets for Tkinter???

    Has anyone gotten the time for this? Are there any new widgets in Tk
    8.4? Note that we've got better Tix support already (though not on
    Windows yet).

-   Fredrik Lundh's basetime proposal:

    http://effbot.org/ideas/time-type.htm

    I believe this is dead now.

-   PEP 304 (Controlling Generation of Bytecode Files by Montanaro)
    seems to have lost steam.

-   For a class defined inside another class, the __name__ should be
    "outer.inner", and pickling should work. (SF 633930. I'm no longer
    certain this is easy or even right.)

-   reST is going to be used a lot in Zope3. Maybe it could become a
    standard library module? (Since reST's author thinks it's too
    unstable, I'm inclined not to do this.)

-   Decide on a clearer deprecation policy (especially for modules) and
    act on it. For a start, see this message from Neal Norwitz:
    https://mail.python.org/pipermail/python-dev/2002-April/023165.html
    There seems insufficient interest in moving this further in an
    organized fashion, and it's not particularly important.

-   Provide alternatives for common uses of the types module;

    Skip Montanaro has posted a proto-PEP for this idea:
    https://mail.python.org/pipermail/python-dev/2002-May/024346.html

    There hasn't been any progress on this, AFAICT.

-   Use pending deprecation for the types and string modules. This
    requires providing alternatives for the parts that aren't covered
    yet (e.g. string.whitespace and types.TracebackType). It seems we
    can't get consensus on this.

-   Deprecate the buffer object.

    -   https://mail.python.org/pipermail/python-dev/2002-July/026388.html
    -   https://mail.python.org/pipermail/python-dev/2002-July/026408.html

    It seems that this is never going to be resolved.

-   PEP 269 Pgen Module for Python, Riehl

    (Some necessary changes are in; the pgen module itself needs to
    mature more.)

-   Add support for the long-awaited Python catalog. Kapil Thangavelu
    has a Zope-based implementation that he demoed at OSCON 2002. Now
    all we need is a place to host it and a person to champion it. (Some
    changes to distutils to support this are in, at least.)

-   PEP 266 Optimizing Global Variable/Attribute Access, Montanaro

    PEP 267 Optimized Access to Module Namespaces, Hylton

    PEP 280 Optimizing access to globals, van Rossum

    These are basically three friendly competing proposals. Jeremy has
    made a little progress with a new compiler, but it's going slow and
    the compiler is only the first step. Maybe we'll be able to refactor
    the compiler in this release. I'm tempted to say we won't hold our
    breath. In the meantime, Oren Tirosh has a much simpler idea that
    may give a serious boost to the performance of accessing globals and
    built-ins, by optimizing and inlining the dict access:
    http://tothink.com/python/fastnames/

-   Lazily tracking tuples?

    -   https://mail.python.org/pipermail/python-dev/2002-May/023926.html
    -   https://bugs.python.org/issue558745

    Not much enthusiasm I believe.

-   PEP 286 Enhanced Argument Tuples, von Loewis

    I haven't had the time to review this thoroughly. It seems a deep
    optimization hack (also makes better correctness guarantees though).

-   Make 'as' a keyword. It has been a pseudo-keyword long enough. Too
    much effort to bother.

Copyright

This document has been placed in the public domain.