text:
PEP: 572 Title: Assignment Expressions Author: Chris Angelico
<EMAIL_ADDRESS>Tim Peters<EMAIL_ADDRESS>Guido van Rossum
<EMAIL_ADDRESS>Status: Final Type: Standards Track Content-Type:
text/x-rst Created: 28-Feb-2018 Python-Version: 3.8 Post-History:
28-Feb-2018, 02-Mar-2018, 23-Mar-2018, 04-Apr-2018, 17-Apr-2018,
25-Apr-2018, 09-Jul-2018, 05-Aug-2019 Resolution:
https://mail.python.org/pipermail/python-dev/2018-July/154601.html

Abstract

This is a proposal for creating a way to assign to variables within an
expression using the notation NAME := expr.

As part of this change, there is also an update to dictionary
comprehension evaluation order to ensure key expressions are executed
before value expressions (allowing the key to be bound to a name and
then re-used as part of calculating the corresponding value).

During discussion of this PEP, the operator became informally known as
"the walrus operator". The construct's formal name is "Assignment
Expressions" (as per the PEP title), but they may also be referred to as
"Named Expressions" (e.g. the CPython reference implementation uses that
name internally).

Rationale

Naming the result of an expression is an important part of programming,
allowing a descriptive name to be used in place of a longer expression,
and permitting reuse. Currently, this feature is available only in
statement form, making it unavailable in list comprehensions and other
expression contexts.

Additionally, naming sub-parts of a large expression can assist an
interactive debugger, providing useful display hooks and partial
results. Without a way to capture sub-expressions inline, this would
require refactoring of the original code; with assignment expressions,
this merely requires the insertion of a few name := markers. Removing
the need to refactor reduces the likelihood that the code be
inadvertently changed as part of debugging (a common cause of
Heisenbugs), and is easier to dictate to another programmer.

The importance of real code

During the development of this PEP many people (supporters and critics
both) have had a tendency to focus on toy examples on the one hand, and
on overly complex examples on the other.

The danger of toy examples is twofold: they are often too abstract to
make anyone go "ooh, that's compelling", and they are easily refuted
with "I would never write it that way anyway".

The danger of overly complex examples is that they provide a convenient
strawman for critics of the proposal to shoot down ("that's
obfuscated").

Yet there is some use for both extremely simple and extremely complex
examples: they are helpful to clarify the intended semantics. Therefore,
there will be some of each below.

However, in order to be compelling, examples should be rooted in real
code, i.e. code that was written without any thought of this PEP, as
part of a useful application, however large or small. Tim Peters has
been extremely helpful by going over his own personal code repository
and picking examples of code he had written that (in his view) would
have been clearer if rewritten with (sparing) use of assignment
expressions. His conclusion: the current proposal would have allowed a
modest but clear improvement in quite a few bits of code.

Another use of real code is to observe indirectly how much value
programmers place on compactness. Guido van Rossum searched through a
Dropbox code base and discovered some evidence that programmers value
writing fewer lines over shorter lines.

Case in point: Guido found several examples where a programmer repeated
a subexpression, slowing down the program, in order to save one line of
code, e.g. instead of writing:

    match = re.match(data)
    group = match.group(1) if match else None

they would write:

    group = re.match(data).group(1) if re.match(data) else None

Another example illustrates that programmers sometimes do more work to
save an extra level of indentation:

    match1 = pattern1.match(data)
    match2 = pattern2.match(data)
    if match1:
        result = match1.group(1)
    elif match2:
        result = match2.group(2)
    else:
        result = None

This code tries to match pattern2 even if pattern1 has a match (in which
case the match on pattern2 is never used). The more efficient rewrite
would have been:

    match1 = pattern1.match(data)
    if match1:
        result = match1.group(1)
    else:
        match2 = pattern2.match(data)
        if match2:
            result = match2.group(2)
        else:
            result = None

Syntax and semantics

In most contexts where arbitrary Python expressions can be used, a named
expression can appear. This is of the form NAME := expr where expr is
any valid Python expression other than an unparenthesized tuple, and
NAME is an identifier.

The value of such a named expression is the same as the incorporated
expression, with the additional side-effect that the target is assigned
that value:

    # Handle a matched regex
    if (match := pattern.search(data)) is not None:
        # Do something with match

    # A loop that can't be trivially rewritten using 2-arg iter()
    while chunk := file.read(8192):
       process(chunk)

    # Reuse a value that's expensive to compute
    [y := f(x), y**2, y**3]

    # Share a subexpression between a comprehension filter clause and its output
    filtered_data = [y for x in data if (y := f(x)) is not None]

Exceptional cases

There are a few places where assignment expressions are not allowed, in
order to avoid ambiguities or user confusion:

-   Unparenthesized assignment expressions are prohibited at the top
    level of an expression statement. Example:

        y := f(x)  # INVALID
        (y := f(x))  # Valid, though not recommended

    This rule is included to simplify the choice for the user between an
    assignment statement and an assignment expression -- there is no
    syntactic position where both are valid.

-   Unparenthesized assignment expressions are prohibited at the top
    level of the right hand side of an assignment statement. Example:

        y0 = y1 := f(x)  # INVALID
        y0 = (y1 := f(x))  # Valid, though discouraged

    Again, this rule is included to avoid two visually similar ways of
    saying the same thing.

-   Unparenthesized assignment expressions are prohibited for the value
    of a keyword argument in a call. Example:

        foo(x = y := f(x))  # INVALID
        foo(x=(y := f(x)))  # Valid, though probably confusing

    This rule is included to disallow excessively confusing code, and
    because parsing keyword arguments is complex enough already.

-   Unparenthesized assignment expressions are prohibited at the top
    level of a function default value. Example:

        def foo(answer = p := 42):  # INVALID
            ...
        def foo(answer=(p := 42)):  # Valid, though not great style
            ...

    This rule is included to discourage side effects in a position whose
    exact semantics are already confusing to many users (cf. the common
    style recommendation against mutable default values), and also to
    echo the similar prohibition in calls (the previous bullet).

-   Unparenthesized assignment expressions are prohibited as annotations
    for arguments, return values and assignments. Example:

        def foo(answer: p := 42 = 5):  # INVALID
            ...
        def foo(answer: (p := 42) = 5):  # Valid, but probably never useful
            ...

    The reasoning here is similar to the two previous cases; this
    ungrouped assortment of symbols and operators composed of : and = is
    hard to read correctly.

-   Unparenthesized assignment expressions are prohibited in lambda
    functions. Example:

        (lambda: x := 1) # INVALID
        lambda: (x := 1) # Valid, but unlikely to be useful
        (x := lambda: 1) # Valid
        lambda line: (m := re.match(pattern, line)) and m.group(1) # Valid

    This allows lambda to always bind less tightly than :=; having a
    name binding at the top level inside a lambda function is unlikely
    to be of value, as there is no way to make use of it. In cases where
    the name will be used more than once, the expression is likely to
    need parenthesizing anyway, so this prohibition will rarely affect
    code.

-   Assignment expressions inside of f-strings require parentheses.
    Example:

        >>> f'{(x:=10)}'  # Valid, uses assignment expression
        '10'
        >>> x = 10
        >>> f'{x:=10}'    # Valid, passes '=10' to formatter
        '        10'

    This shows that what looks like an assignment operator in an
    f-string is not always an assignment operator. The f-string parser
    uses : to indicate formatting options. To preserve backwards
    compatibility, assignment operator usage inside of f-strings must be
    parenthesized. As noted above, this usage of the assignment operator
    is not recommended.

Scope of the target

An assignment expression does not introduce a new scope. In most cases
the scope in which the target will be bound is self-explanatory: it is
the current scope. If this scope contains a nonlocal or global
declaration for the target, the assignment expression honors that. A
lambda (being an explicit, if anonymous, function definition) counts as
a scope for this purpose.

There is one special case: an assignment expression occurring in a list,
set or dict comprehension or in a generator expression (below
collectively referred to as "comprehensions") binds the target in the
containing scope, honoring a nonlocal or global declaration for the
target in that scope, if one exists. For the purpose of this rule the
containing scope of a nested comprehension is the scope that contains
the outermost comprehension. A lambda counts as a containing scope.

The motivation for this special case is twofold. First, it allows us to
conveniently capture a "witness" for an any() expression, or a
counterexample for all(), for example:

    if any((comment := line).startswith('#') for line in lines):
        print("First comment:", comment)
    else:
        print("There are no comments")

    if all((nonblank := line).strip() == '' for line in lines):
        print("All lines are blank")
    else:
        print("First non-blank line:", nonblank)

Second, it allows a compact way of updating mutable state from a
comprehension, for example:

    # Compute partial sums in a list comprehension
    total = 0
    partial_sums = [total := total + v for v in values]
    print("Total:", total)

However, an assignment expression target name cannot be the same as a
for-target name appearing in any comprehension containing the assignment
expression. The latter names are local to the comprehension in which
they appear, so it would be contradictory for a contained use of the
same name to refer to the scope containing the outermost comprehension
instead.

For example, [i := i+1 for i in range(5)] is invalid: the for i part
establishes that i is local to the comprehension, but the i := part
insists that i is not local to the comprehension. The same reason makes
these examples invalid too:

    [[(j := j) for i in range(5)] for j in range(5)] # INVALID
    [i := 0 for i, j in stuff]                       # INVALID
    [i+1 for i in (i := stuff)]                      # INVALID

While it's technically possible to assign consistent semantics to these
cases, it's difficult to determine whether those semantics actually make
sense in the absence of real use cases. Accordingly, the reference
implementation[1] will ensure that such cases raise SyntaxError, rather
than executing with implementation defined behaviour.

This restriction applies even if the assignment expression is never
executed:

    [False and (i := 0) for i, j in stuff]     # INVALID
    [i for i, j in stuff if True or (j := 1)]  # INVALID

For the comprehension body (the part before the first "for" keyword) and
the filter expression (the part after "if" and before any nested "for"),
this restriction applies solely to target names that are also used as
iteration variables in the comprehension. Lambda expressions appearing
in these positions introduce a new explicit function scope, and hence
may use assignment expressions with no additional restrictions.

Due to design constraints in the reference implementation (the symbol
table analyser cannot easily detect when names are re-used between the
leftmost comprehension iterable expression and the rest of the
comprehension), named expressions are disallowed entirely as part of
comprehension iterable expressions (the part after each "in", and before
any subsequent "if" or "for" keyword):

    [i+1 for i in (j := stuff)]                    # INVALID
    [i+1 for i in range(2) for j in (k := stuff)]  # INVALID
    [i+1 for i in [j for j in (k := stuff)]]       # INVALID
    [i+1 for i in (lambda: (j := stuff))()]        # INVALID

A further exception applies when an assignment expression occurs in a
comprehension whose containing scope is a class scope. If the rules
above were to result in the target being assigned in that class's scope,
the assignment expression is expressly invalid. This case also raises
SyntaxError:

    class Example:
        [(j := i) for i in range(5)]  # INVALID

(The reason for the latter exception is the implicit function scope
created for comprehensions -- there is currently no runtime mechanism
for a function to refer to a variable in the containing class scope, and
we do not want to add such a mechanism. If this issue ever gets resolved
this special case may be removed from the specification of assignment
expressions. Note that the problem already exists for using a variable
defined in the class scope from a comprehension.)

See Appendix B for some examples of how the rules for targets in
comprehensions translate to equivalent code.

Relative precedence of :=

The := operator groups more tightly than a comma in all syntactic
positions where it is legal, but less tightly than all other operators,
including or, and, not, and conditional expressions (A if C else B). As
follows from section "Exceptional cases" above, it is never allowed at
the same level as =. In case a different grouping is desired,
parentheses should be used.

The := operator may be used directly in a positional function call
argument; however it is invalid directly in a keyword argument.

Some examples to clarify what's technically valid or invalid:

    # INVALID
    x := 0

    # Valid alternative
    (x := 0)

    # INVALID
    x = y := 0

    # Valid alternative
    x = (y := 0)

    # Valid
    len(lines := f.readlines())

    # Valid
    foo(x := 3, cat='vector')

    # INVALID
    foo(cat=category := 'vector')

    # Valid alternative
    foo(cat=(category := 'vector'))

Most of the "valid" examples above are not recommended, since human
readers of Python source code who are quickly glancing at some code may
miss the distinction. But simple cases are not objectionable:

    # Valid
    if any(len(longline := line) >= 100 for line in lines):
        print("Extremely long line:", longline)

This PEP recommends always putting spaces around :=, similar to PEP 8's
recommendation for = when used for assignment, whereas the latter
disallows spaces around = used for keyword arguments.)

Change to evaluation order

In order to have precisely defined semantics, the proposal requires
evaluation order to be well-defined. This is technically not a new
requirement, as function calls may already have side effects. Python
already has a rule that subexpressions are generally evaluated from left
to right. However, assignment expressions make these side effects more
visible, and we propose a single change to the current evaluation order:

-   In a dict comprehension {X: Y for ...}, Y is currently evaluated
    before X. We propose to change this so that X is evaluated before Y.
    (In a dict display like {X: Y} this is already the case, and also in
    dict((X, Y) for ...) which should clearly be equivalent to the dict
    comprehension.)

Differences between assignment expressions and assignment statements

Most importantly, since := is an expression, it can be used in contexts
where statements are illegal, including lambda functions and
comprehensions.

Conversely, assignment expressions don't support the advanced features
found in assignment statements:

-   Multiple targets are not directly supported:

        x = y = z = 0  # Equivalent: (z := (y := (x := 0)))

-   Single assignment targets other than a single NAME are not
    supported:

        # No equivalent
        a[i] = x
        self.rest = []

-   Priority around commas is different:

        x = 1, 2  # Sets x to (1, 2)
        (x := 1, 2)  # Sets x to 1

-   Iterable packing and unpacking (both regular or extended forms) are
    not supported:

        # Equivalent needs extra parentheses
        loc = x, y  # Use (loc := (x, y))
        info = name, phone, *rest  # Use (info := (name, phone, *rest))

        # No equivalent
        px, py, pz = position
        name, phone, email, *other_info = contact

-   Inline type annotations are not supported:

        # Closest equivalent is "p: Optional[int]" as a separate declaration
        p: Optional[int] = None

-   Augmented assignment is not supported:

        total += tax  # Equivalent: (total := total + tax)

Specification changes during implementation

The following changes have been made based on implementation experience
and additional review after the PEP was first accepted and before Python
3.8 was released:

-   for consistency with other similar exceptions, and to avoid locking
    in an exception name that is not necessarily going to improve
    clarity for end users, the originally proposed TargetScopeError
    subclass of SyntaxError was dropped in favour of just raising
    SyntaxError directly.[2]
-   due to a limitation in CPython's symbol table analysis process, the
    reference implementation raises SyntaxError for all uses of named
    expressions inside comprehension iterable expressions, rather than
    only raising them when the named expression target conflicts with
    one of the iteration variables in the comprehension. This could be
    revisited given sufficiently compelling examples, but the extra
    complexity needed to implement the more selective restriction
    doesn't seem worthwhile for purely hypothetical use cases.

Examples

Examples from the Python standard library

site.py

env_base is only used on these lines, putting its assignment on the if
moves it as the "header" of the block.

-   Current:

        env_base = os.environ.get("PYTHONUSERBASE", None)
        if env_base:
            return env_base

-   Improved:

        if env_base := os.environ.get("PYTHONUSERBASE", None):
            return env_base

_pydecimal.py

Avoid nested if and remove one indentation level.

-   Current:

        if self._is_special:
            ans = self._check_nans(context=context)
            if ans:
                return ans

-   Improved:

        if self._is_special and (ans := self._check_nans(context=context)):
            return ans

copy.py

Code looks more regular and avoid multiple nested if. (See Appendix A
for the origin of this example.)

-   Current:

        reductor = dispatch_table.get(cls)
        if reductor:
            rv = reductor(x)
        else:
            reductor = getattr(x, "__reduce_ex__", None)
            if reductor:
                rv = reductor(4)
            else:
                reductor = getattr(x, "__reduce__", None)
                if reductor:
                    rv = reductor()
                else:
                    raise Error(
                        "un(deep)copyable object of type %s" % cls)

-   Improved:

        if reductor := dispatch_table.get(cls):
            rv = reductor(x)
        elif reductor := getattr(x, "__reduce_ex__", None):
            rv = reductor(4)
        elif reductor := getattr(x, "__reduce__", None):
            rv = reductor()
        else:
            raise Error("un(deep)copyable object of type %s" % cls)

datetime.py

tz is only used for s += tz, moving its assignment inside the if helps
to show its scope.

-   Current:

        s = _format_time(self._hour, self._minute,
                         self._second, self._microsecond,
                         timespec)
        tz = self._tzstr()
        if tz:
            s += tz
        return s

-   Improved:

        s = _format_time(self._hour, self._minute,
                         self._second, self._microsecond,
                         timespec)
        if tz := self._tzstr():
            s += tz
        return s

sysconfig.py

Calling fp.readline() in the while condition and calling .match() on the
if lines make the code more compact without making it harder to
understand.

-   Current:

        while True:
            line = fp.readline()
            if not line:
                break
            m = define_rx.match(line)
            if m:
                n, v = m.group(1, 2)
                try:
                    v = int(v)
                except ValueError:
                    pass
                vars[n] = v
            else:
                m = undef_rx.match(line)
                if m:
                    vars[m.group(1)] = 0

-   Improved:

        while line := fp.readline():
            if m := define_rx.match(line):
                n, v = m.group(1, 2)
                try:
                    v = int(v)
                except ValueError:
                    pass
                vars[n] = v
            elif m := undef_rx.match(line):
                vars[m.group(1)] = 0

Simplifying list comprehensions

A list comprehension can map and filter efficiently by capturing the
condition:

    results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0]

Similarly, a subexpression can be reused within the main expression, by
giving it a name on first use:

    stuff = [[y := f(x), x/y] for x in range(5)]

Note that in both cases the variable y is bound in the containing scope
(i.e. at the same level as results or stuff).

Capturing condition values

Assignment expressions can be used to good effect in the header of an if
or while statement:

    # Loop-and-a-half
    while (command := input("> ")) != "quit":
        print("You entered:", command)

    # Capturing regular expression match objects
    # See, for instance, Lib/pydoc.py, which uses a multiline spelling
    # of this effect
    if match := re.search(pat, text):
        print("Found:", match.group(0))
    # The same syntax chains nicely into 'elif' statements, unlike the
    # equivalent using assignment statements.
    elif match := re.search(otherpat, text):
        print("Alternate found:", match.group(0))
    elif match := re.search(third, text):
        print("Fallback found:", match.group(0))

    # Reading socket data until an empty string is returned
    while data := sock.recv(8192):
        print("Received data:", data)

Particularly with the while loop, this can remove the need to have an
infinite loop, an assignment, and a condition. It also creates a smooth
parallel between a loop which simply uses a function call as its
condition, and one which uses that as its condition but also uses the
actual value.

Fork

An example from the low-level UNIX world:

    if pid := os.fork():
        # Parent code
    else:
        # Child code

Rejected alternative proposals

Proposals broadly similar to this one have come up frequently on
python-ideas. Below are a number of alternative syntaxes, some of them
specific to comprehensions, which have been rejected in favour of the
one given above.

Changing the scope rules for comprehensions

A previous version of this PEP proposed subtle changes to the scope
rules for comprehensions, to make them more usable in class scope and to
unify the scope of the "outermost iterable" and the rest of the
comprehension. However, this part of the proposal would have caused
backwards incompatibilities, and has been withdrawn so the PEP can focus
on assignment expressions.

Alternative spellings

Broadly the same semantics as the current proposal, but spelled
differently.

1.  EXPR as NAME:

        stuff = [[f(x) as y, x/y] for x in range(5)]

    Since EXPR as NAME already has meaning in import, except and with
    statements (with different semantics), this would create unnecessary
    confusion or require special-casing (e.g. to forbid assignment
    within the headers of these statements).

    (Note that with EXPR as VAR does not simply assign the value of EXPR
    to VAR -- it calls EXPR.__enter__() and assigns the result of that
    to VAR.)

    Additional reasons to prefer := over this spelling include:

    -   In if f(x) as y the assignment target doesn't jump out at you --
        it just reads like if f x blah blah and it is too similar
        visually to if f(x) and y.

    -   In all other situations where an as clause is allowed, even
        readers with intermediary skills are led to anticipate that
        clause (however optional) by the keyword that starts the line,
        and the grammar ties that keyword closely to the as clause:

        -   import foo as bar
        -   except Exc as var
        -   with ctxmgr() as var

        To the contrary, the assignment expression does not belong to
        the if or while that starts the line, and we intentionally allow
        assignment expressions in other contexts as well.

    -   The parallel cadence between

        -   NAME = EXPR
        -   if NAME := EXPR

        reinforces the visual recognition of assignment expressions.

2.  EXPR -> NAME:

        stuff = [[f(x) -> y, x/y] for x in range(5)]

    This syntax is inspired by languages such as R and Haskell, and some
    programmable calculators. (Note that a left-facing arrow y <- f(x)
    is not possible in Python, as it would be interpreted as less-than
    and unary minus.) This syntax has a slight advantage over 'as' in
    that it does not conflict with with, except and import, but
    otherwise is equivalent. But it is entirely unrelated to Python's
    other use of -> (function return type annotations), and compared to
    := (which dates back to Algol-58) it has a much weaker tradition.

3.  Adorning statement-local names with a leading dot:

        stuff = [[(f(x) as .y), x/.y] for x in range(5)] # with "as"
        stuff = [[(.y := f(x)), x/.y] for x in range(5)] # with ":="

    This has the advantage that leaked usage can be readily detected,
    removing some forms of syntactic ambiguity. However, this would be
    the only place in Python where a variable's scope is encoded into
    its name, making refactoring harder.

4.  Adding a where: to any statement to create local name bindings:

        value = x**2 + 2*x where:
            x = spam(1, 4, 7, q)

    Execution order is inverted (the indented body is performed first,
    followed by the "header"). This requires a new keyword, unless an
    existing keyword is repurposed (most likely with:). See PEP 3150 for
    prior discussion on this subject (with the proposed keyword being
    given:).

5.  TARGET from EXPR:

        stuff = [[y from f(x), x/y] for x in range(5)]

    This syntax has fewer conflicts than as does (conflicting only with
    the raise Exc from Exc notation), but is otherwise comparable to it.
    Instead of paralleling with expr as target: (which can be useful but
    can also be confusing), this has no parallels, but is evocative.

Special-casing conditional statements

One of the most popular use-cases is if and while statements. Instead of
a more general solution, this proposal enhances the syntax of these two
statements to add a means of capturing the compared value:

    if re.search(pat, text) as match:
        print("Found:", match.group(0))

This works beautifully if and ONLY if the desired condition is based on
the truthiness of the captured value. It is thus effective for specific
use-cases (regex matches, socket reads that return '' when done), and
completely useless in more complicated cases (e.g. where the condition
is f(x) < 0 and you want to capture the value of f(x)). It also has no
benefit to list comprehensions.

Advantages: No syntactic ambiguities. Disadvantages: Answers only a
fraction of possible use-cases, even in if/while statements.

Special-casing comprehensions

Another common use-case is comprehensions (list/set/dict, and genexps).
As above, proposals have been made for comprehension-specific solutions.

1.  where, let, or given:

        stuff = [(y, x/y) where y = f(x) for x in range(5)]
        stuff = [(y, x/y) let y = f(x) for x in range(5)]
        stuff = [(y, x/y) given y = f(x) for x in range(5)]

    This brings the subexpression to a location in between the 'for'
    loop and the expression. It introduces an additional language
    keyword, which creates conflicts. Of the three, where reads the most
    cleanly, but also has the greatest potential for conflict (e.g.
    SQLAlchemy and numpy have where methods, as does tkinter.dnd.Icon in
    the standard library).

2.  with NAME = EXPR:

        stuff = [(y, x/y) with y = f(x) for x in range(5)]

    As above, but reusing the with keyword. Doesn't read too badly, and
    needs no additional language keyword. Is restricted to
    comprehensions, though, and cannot as easily be transformed into
    "longhand" for-loop syntax. Has the C problem that an equals sign in
    an expression can now create a name binding, rather than performing
    a comparison. Would raise the question of why "with NAME = EXPR:"
    cannot be used as a statement on its own.

3.  with EXPR as NAME:

        stuff = [(y, x/y) with f(x) as y for x in range(5)]

    As per option 2, but using as rather than an equals sign. Aligns
    syntactically with other uses of as for name binding, but a simple
    transformation to for-loop longhand would create drastically
    different semantics; the meaning of with inside a comprehension
    would be completely different from the meaning as a stand-alone
    statement, while retaining identical syntax.

Regardless of the spelling chosen, this introduces a stark difference
between comprehensions and the equivalent unrolled long-hand form of the
loop. It is no longer possible to unwrap the loop into statement form
without reworking any name bindings. The only keyword that can be
repurposed to this task is with, thus giving it sneakily different
semantics in a comprehension than in a statement; alternatively, a new
keyword is needed, with all the costs therein.

Lowering operator precedence

There are two logical precedences for the := operator. Either it should
bind as loosely as possible, as does statement-assignment; or it should
bind more tightly than comparison operators. Placing its precedence
between the comparison and arithmetic operators (to be precise: just
lower than bitwise OR) allows most uses inside while and if conditions
to be spelled without parentheses, as it is most likely that you wish to
capture the value of something, then perform a comparison on it:

    pos = -1
    while pos := buffer.find(search_term, pos + 1) >= 0:
        ...

Once find() returns -1, the loop terminates. If := binds as loosely as =
does, this would capture the result of the comparison (generally either
True or False), which is less useful.

While this behaviour would be convenient in many situations, it is also
harder to explain than "the := operator behaves just like the assignment
statement", and as such, the precedence for := has been made as close as
possible to that of = (with the exception that it binds tighter than
comma).

Allowing commas to the right

Some critics have claimed that the assignment expressions should allow
unparenthesized tuples on the right, so that these two would be
equivalent:

    (point := (x, y))
    (point := x, y)

(With the current version of the proposal, the latter would be
equivalent to ((point := x), y).)

However, adopting this stance would logically lead to the conclusion
that when used in a function call, assignment expressions also bind less
tight than comma, so we'd have the following confusing equivalence:

    foo(x := 1, y)
    foo(x := (1, y))

The less confusing option is to make := bind more tightly than comma.

Always requiring parentheses

It's been proposed to just always require parentheses around an
assignment expression. This would resolve many ambiguities, and indeed
parentheses will frequently be needed to extract the desired
subexpression. But in the following cases the extra parentheses feel
redundant:

    # Top level in if
    if match := pattern.match(line):
        return match.group(1)

    # Short call
    len(lines := f.readlines())

Frequently Raised Objections

Why not just turn existing assignment into an expression?

C and its derivatives define the = operator as an expression, rather
than a statement as is Python's way. This allows assignments in more
contexts, including contexts where comparisons are more common. The
syntactic similarity between if (x == y) and if (x = y) belies their
drastically different semantics. Thus this proposal uses := to clarify
the distinction.

With assignment expressions, why bother with assignment statements?

The two forms have different flexibilities. The := operator can be used
inside a larger expression; the = statement can be augmented to += and
its friends, can be chained, and can assign to attributes and
subscripts.

Why not use a sublocal scope and prevent namespace pollution?

Previous revisions of this proposal involved sublocal scope (restricted
to a single statement), preventing name leakage and namespace pollution.
While a definite advantage in a number of situations, this increases
complexity in many others, and the costs are not justified by the
benefits. In the interests of language simplicity, the name bindings
created here are exactly equivalent to any other name bindings,
including that usage at class or module scope will create
externally-visible names. This is no different from for loops or other
constructs, and can be solved the same way: del the name once it is no
longer needed, or prefix it with an underscore.

(The author wishes to thank Guido van Rossum and Christoph Groth for
their suggestions to move the proposal in this direction.[3])

Style guide recommendations

As expression assignments can sometimes be used equivalently to
statement assignments, the question of which should be preferred will
arise. For the benefit of style guides such as PEP 8, two
recommendations are suggested.

1.  If either assignment statements or assignment expressions can be
    used, prefer statements; they are a clear declaration of intent.
2.  If using assignment expressions would lead to ambiguity about
    execution order, restructure it to use statements instead.

Acknowledgements

The authors wish to thank Alyssa Coghlan and Steven D'Aprano for their
considerable contributions to this proposal, and members of the
core-mentorship mailing list for assistance with implementation.

Appendix A: Tim Peters's findings

Here's a brief essay Tim Peters wrote on the topic.

I dislike "busy" lines of code, and also dislike putting conceptually
unrelated logic on a single line. So, for example, instead of:

    i = j = count = nerrors = 0

I prefer:

    i = j = 0
    count = 0
    nerrors = 0

instead. So I suspected I'd find few places I'd want to use assignment
expressions. I didn't even consider them for lines already stretching
halfway across the screen. In other cases, "unrelated" ruled:

    mylast = mylast[1]
    yield mylast[0]

is a vast improvement over the briefer:

    yield (mylast := mylast[1])[0]

The original two statements are doing entirely different conceptual
things, and slamming them together is conceptually insane.

In other cases, combining related logic made it harder to understand,
such as rewriting:

    while True:
        old = total
        total += term
        if old == total:
            return total
        term *= mx2 / (i*(i+1))
        i += 2

as the briefer:

    while total != (total := total + term):
        term *= mx2 / (i*(i+1))
        i += 2
    return total

The while test there is too subtle, crucially relying on strict
left-to-right evaluation in a non-short-circuiting or method-chaining
context. My brain isn't wired that way.

But cases like that were rare. Name binding is very frequent, and
"sparse is better than dense" does not mean "almost empty is better than
sparse". For example, I have many functions that return None or 0 to
communicate "I have nothing useful to return in this case, but since
that's expected often I'm not going to annoy you with an exception".
This is essentially the same as regular expression search functions
returning None when there is no match. So there was lots of code of the
form:

    result = solution(xs, n)
    if result:
        # use result

I find that clearer, and certainly a bit less typing and
pattern-matching reading, as:

    if result := solution(xs, n):
        # use result

It's also nice to trade away a small amount of horizontal whitespace to
get another _line of surrounding code on screen. I didn't give much
weight to this at first, but it was so very frequent it added up, and I
soon enough became annoyed that I couldn't actually run the briefer
code. That surprised me!

There are other cases where assignment expressions really shine. Rather
than pick another from my code, Kirill Balunov gave a lovely example
from the standard library's copy() function in copy.py:

    reductor = dispatch_table.get(cls)
    if reductor:
        rv = reductor(x)
    else:
        reductor = getattr(x, "__reduce_ex__", None)
        if reductor:
            rv = reductor(4)
        else:
            reductor = getattr(x, "__reduce__", None)
            if reductor:
                rv = reductor()
            else:
                raise Error("un(shallow)copyable object of type %s" % cls)

The ever-increasing indentation is semantically misleading: the logic is
conceptually flat, "the first test that succeeds wins":

    if reductor := dispatch_table.get(cls):
        rv = reductor(x)
    elif reductor := getattr(x, "__reduce_ex__", None):
        rv = reductor(4)
    elif reductor := getattr(x, "__reduce__", None):
        rv = reductor()
    else:
        raise Error("un(shallow)copyable object of type %s" % cls)

Using easy assignment expressions allows the visual structure of the
code to emphasize the conceptual flatness of the logic; ever-increasing
indentation obscured it.

A smaller example from my code delighted me, both allowing to put
inherently related logic in a single line, and allowing to remove an
annoying "artificial" indentation level:

    diff = x - x_base
    if diff:
        g = gcd(diff, n)
        if g > 1:
            return g

became:

    if (diff := x - x_base) and (g := gcd(diff, n)) > 1:
        return g

That if is about as long as I want my lines to get, but remains easy to
follow.

So, in all, in most lines binding a name, I wouldn't use assignment
expressions, but because that construct is so very frequent, that leaves
many places I would. In most of the latter, I found a small win that
adds up due to how often it occurs, and in the rest I found a moderate
to major win. I'd certainly use it more often than ternary if, but
significantly less often than augmented assignment.

A numeric example

I have another example that quite impressed me at the time.

Where all variables are positive integers, and a is at least as large as
the n'th root of x, this algorithm returns the floor of the n'th root of
x (and roughly doubling the number of accurate bits per iteration):

    while a > (d := x // a**(n-1)):
        a = ((n-1)*a + d) // n
    return a

It's not obvious why that works, but is no more obvious in the "loop and
a half" form. It's hard to prove correctness without building on the
right insight (the "arithmetic mean - geometric mean inequality"), and
knowing some non-trivial things about how nested floor functions behave.
That is, the challenges are in the math, not really in the coding.

If you do know all that, then the assignment-expression form is easily
read as "while the current guess is too large, get a smaller guess",
where the "too large?" test and the new guess share an expensive
sub-expression.

To my eyes, the original form is harder to understand:

    while True:
        d = x // a**(n-1)
        if a <= d:
            break
        a = ((n-1)*a + d) // n
    return a

Appendix B: Rough code translations for comprehensions

This appendix attempts to clarify (though not specify) the rules when a
target occurs in a comprehension or in a generator expression. For a
number of illustrative examples we show the original code, containing a
comprehension, and the translation, where the comprehension has been
replaced by an equivalent generator function plus some scaffolding.

Since [x for ...] is equivalent to list(x for ...) these examples all
use list comprehensions without loss of generality. And since these
examples are meant to clarify edge cases of the rules, they aren't
trying to look like real code.

Note: comprehensions are already implemented via synthesizing nested
generator functions like those in this appendix. The new part is adding
appropriate declarations to establish the intended scope of assignment
expression targets (the same scope they resolve to as if the assignment
were performed in the block containing the outermost comprehension). For
type inference purposes, these illustrative expansions do not imply that
assignment expression targets are always Optional (but they do indicate
the target binding scope).

Let's start with a reminder of what code is generated for a generator
expression without assignment expression.

-   Original code (EXPR usually references VAR):

        def f():
            a = [EXPR for VAR in ITERABLE]

-   Translation (let's not worry about name conflicts):

        def f():
            def genexpr(iterator):
                for VAR in iterator:
                    yield EXPR
            a = list(genexpr(iter(ITERABLE)))

Let's add a simple assignment expression.

-   Original code:

        def f():
            a = [TARGET := EXPR for VAR in ITERABLE]

-   Translation:

        def f():
            if False:
                TARGET = None  # Dead code to ensure TARGET is a local variable
            def genexpr(iterator):
                nonlocal TARGET
                for VAR in iterator:
                    TARGET = EXPR
                    yield TARGET
            a = list(genexpr(iter(ITERABLE)))

Let's add a global TARGET declaration in f().

-   Original code:

        def f():
            global TARGET
            a = [TARGET := EXPR for VAR in ITERABLE]

-   Translation:

        def f():
            global TARGET
            def genexpr(iterator):
                global TARGET
                for VAR in iterator:
                    TARGET = EXPR
                    yield TARGET
            a = list(genexpr(iter(ITERABLE)))

Or instead let's add a nonlocal TARGET declaration in f().

-   Original code:

        def g():
            TARGET = ...
            def f():
                nonlocal TARGET
                a = [TARGET := EXPR for VAR in ITERABLE]

-   Translation:

        def g():
            TARGET = ...
            def f():
                nonlocal TARGET
                def genexpr(iterator):
                    nonlocal TARGET
                    for VAR in iterator:
                        TARGET = EXPR
                        yield TARGET
                a = list(genexpr(iter(ITERABLE)))

Finally, let's nest two comprehensions.

-   Original code:

        def f():
            a = [[TARGET := i for i in range(3)] for j in range(2)]
            # I.e., a = [[0, 1, 2], [0, 1, 2]]
            print(TARGET)  # prints 2

-   Translation:

        def f():
            if False:
                TARGET = None
            def outer_genexpr(outer_iterator):
                nonlocal TARGET
                def inner_generator(inner_iterator):
                    nonlocal TARGET
                    for i in inner_iterator:
                        TARGET = i
                        yield i
                for j in outer_iterator:
                    yield list(inner_generator(range(3)))
            a = list(outer_genexpr(range(2)))
            print(TARGET)

Appendix C: No Changes to Scope Semantics

Because it has been a point of confusion, note that nothing about
Python's scoping semantics is changed. Function-local scopes continue to
be resolved at compile time, and to have indefinite temporal extent at
run time ("full closures"). Example:

    a = 42
    def f():
        # `a` is local to `f`, but remains unbound
        # until the caller executes this genexp:
        yield ((a := i) for i in range(3))
        yield lambda: a + 100
        print("done")
        try:
            print(f"`a` is bound to {a}")
            assert False
        except UnboundLocalError:
            print("`a` is not yet bound")

Then:

    >>> results = list(f()) # [genexp, lambda]
    done
    `a` is not yet bound
    # The execution frame for f no longer exists in CPython,
    # but f's locals live so long as they can still be referenced.
    >>> list(map(type, results))
    [<class 'generator'>, <class 'function'>]
    >>> list(results[0])
    [0, 1, 2]
    >>> results[1]()
    102
    >>> a
    42

References

Copyright

This document has been placed in the public domain.

[1] Proof of concept implementation
(https://github.com/Rosuav/cpython/tree/assignment-expressions)

[2] Discussion of PEP 572 TargetScopeError
<EMAIL_ADDRESS>
[3] Pivotal post regarding inline assignment semantics
(https://mail.python.org/pipermail/python-ideas/2018-March/049409.html)

authors:
Chris Angelico
Tim Peters


text:
PEP: 378 Title: Format Specifier for Thousands Separator Version:
$Revision$ Last-Modified: $Date$ Author: Raymond Hettinger
<EMAIL_ADDRESS>Status: Final Type: Standards Track Content-Type:
text/x-rst Created: 12-Mar-2009 Python-Version: 2.7, 3.1 Post-History:
12-Mar-2009

Motivation

Provide a simple, non-locale aware way to format a number with a
thousands separator.

Adding thousands separators is one of the simplest ways to humanize a
program's output, improving its professional appearance and readability.

In the finance world, output with thousands separators is the norm.
Finance users and non-professional programmers find the locale approach
to be frustrating, arcane and non-obvious.

The locale module presents two other challenges. First, it is a global
setting and not suitable for multi-threaded apps that need to serve-up
requests in multiple locales. Second, the name of a relevant locale
(such as "de_DE") can vary from platform to platform or may not be
defined at all. The docs for the locale module describe these and many
other challenges in detail.

It is not the goal to replace the locale module, to perform
internationalization tasks, or accommodate every possible convention.
Such tasks are better suited to robust tools like Babel. Instead, the
goal is to make a common, everyday task easier for many users.

Main Proposal (from Alyssa Coghlan, originally called Proposal I)

A comma will be added to the format() specifier mini-language:

    [[fill]align][sign][#][0][width][,][.precision][type]

The ',' option indicates that commas should be included in the output as
a thousands separator. As with locales which do not use a period as the
decimal point, locales which use a different convention for digit
separation will need to use the locale module to obtain appropriate
formatting.

The proposal works well with floats, ints, and decimals. It also allows
easy substitution for other separators. For example:

    format(n, "6,d").replace(",", "_")

This technique is completely general but it is awkward in the one case
where the commas and periods need to be swapped:

    format(n, "6,f").replace(",", "X").replace(".", ",").replace("X", ".")

The width argument means the total length including the commas and
decimal point:

    format(1234, "08,d")     -->    '0001,234'
    format(1234.5, "08,.1f") -->    '01,234.5'

The ',' option is defined as shown above for types 'd', 'e', 'f', 'g',
'E', 'G', '%', 'F' and ''. To allow future extensions, it is undefined
for other types: binary, octal, hex, character, etc.

This proposal has the virtue of being simpler than the alternative
proposal but is much less flexible and meets the needs of fewer users
right out of the box. It is expected that some other solution will arise
for specifying alternative separators.

Current Version of the Mini-Language

-   Python 2.6 docs
-   PEP 3101 Advanced String Formatting

Research into what Other Languages Do

Scanning the web, I've found that thousands separators are usually one
of COMMA, DOT, SPACE, APOSTROPHE or UNDERSCORE.

C-Sharp provides both styles (picture formatting and type specifiers).
The type specifier approach is locale aware. The picture formatting only
offers a COMMA as a thousands separator:

    String.Format("{0:n}", 12400)     ==>    "12,400"
    String.Format("{0:0,0}", 12400)   ==>    "12,400"

Common Lisp uses a COLON before the ~D decimal type specifier to emit a
COMMA as a thousands separator. The general form of ~D is
~mincol,padchar,commachar,commaintervalD. The padchar defaults to SPACE.
The commachar defaults to COMMA. The commainterval defaults to three.

    (format nil "~:D" 229345007)   =>   "229,345,007"

-   The ADA language allows UNDERSCORES in its numeric literals.

Visual Basic and its brethren (like MS Excel) use a completely different
style and have ultra-flexible custom format specifiers like:

    "_($* #,##0_)".

COBOL uses picture clauses like:

    PICTURE $***,**9.99CR

Java offers a Decimal.Format Class that uses picture patterns (one for
positive numbers and an optional one for negatives) such as:
"#,##0.00;(#,##0.00)". It allows arbitrary groupings including hundreds
and ten-thousands and uneven groupings. The special pattern characters
are non-localized (using a DOT for a decimal separator and a COMMA for a
grouping separator). The user can supply an alternate set of symbols
using the formatter's DecimalFormatSymbols object.

Alternative Proposal (from Eric Smith, originally called Proposal II)

Make both the thousands separator and decimal separator user specifiable
but not locale aware. For simplicity, limit the choices to a COMMA, DOT,
SPACE, APOSTROPHE or UNDERSCORE. The SPACE can be either U+0020 or
U+00A0.

Whenever a separator is followed by a precision, it is a decimal
separator and an optional separator preceding it is a thousands
separator. When the precision is absent, a lone specifier means a
thousands separator:

    [[fill]align][sign][#][0][width][tsep][dsep precision][type]

Examples:

    format(1234, "8.1f")     -->    '  1234.0'
    format(1234, "8,1f")     -->    '  1234,0'
    format(1234, "8.,1f")    -->    ' 1.234,0'
    format(1234, "8 ,f")     -->    ' 1 234,0'
    format(1234, "8d")       -->    '    1234'
    format(1234, "8,d")      -->    '   1,234'
    format(1234, "8_d")      -->    '   1_234'

This proposal meets mosts needs, but it comes at the expense of taking a
bit more effort to parse. Not every possible convention is covered, but
at least one of the options (spaces or underscores) should be readable,
understandable, and useful to folks from many diverse backgrounds.

As shown in the examples, the width argument means the total length
including the thousands separators and decimal separators.

No change is proposed for the locale module.

The thousands separator is defined as shown above for types 'd', 'e',
'f', 'g', '%', 'E', 'G' and 'F'. To allow future extensions, it is
undefined for other types: binary, octal, hex, character, etc.

The drawback to this alternative proposal is the difficulty of mentally
parsing whether a single separator is a thousands separator or decimal
separator. Perhaps it is too arcane to link the decimal separator with
the precision specifier.

Commentary

-   Some commenters do not like the idea of format strings at all and
    find them to be unreadable. Suggested alternatives include the COBOL
    style PICTURE approach or a convenience function with keyword
    arguments for every possible combination.
-   Some newsgroup respondants think there is no place for any scripts
    that are not internationalized and that it is a step backwards to
    provide a simple way to hardwire a particular choice (thus reducing
    incentive to use a locale sensitive approach).
-   Another thought is that embedding some particular convention in
    individual format strings makes it hard to change that convention
    later. No workable alternative was suggested but the general idea is
    to set the convention once and have it apply everywhere (others
    commented that locale already provides a way to do this).
-   There are some precedents for grouping digits in the fractional part
    of a floating point number, but this PEP does not venture into that
    territory. Only digits to the left of the decimal point are grouped.
    This does not preclude future extensions; it just focuses on a
    single, generally useful extension to the formatting language.
-   James Knight observed that Indian/Pakistani numbering systems group
    by hundreds. Ben Finney noted that Chinese group by ten-thousands.
    Eric Smith pointed-out that these are already handled by the "n"
    specifier in the locale module (albeit only for integers). This PEP
    does not attempt to support all of those possibilities. It focuses
    on a single, relatively common grouping convention that offers a
    quick way to improve readability in many (though not all) contexts.

Copyright

This document has been placed in the public domain.

  Local Variables: mode: indented-text indent-tabs-mode: nil
  sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

license: Public Domain