text:
PEP: 474 Title: Creating forge.python.org Version: $Revision$
Last-Modified: $Date$ Author: Alyssa Coghlan<EMAIL_ADDRESS>Status: Withdrawn Type: Process Content-Type: text/x-rst Created:
19-Jul-2014 Post-History: 19-Jul-2014, 08-Jan-2015, 01-Feb-2015

Abstract

This PEP proposes setting up a new PSF provided resource,
forge.python.org, as a location for maintaining various supporting
repositories (such as the repository for Python Enhancement Proposals)
in a way that is more accessible to new contributors, and easier to
manage for core developers.

This PEP does not propose any changes to the core development workflow
for CPython itself (see PEP 462 in relation to that).

PEP Withdrawal

This PEP has been withdrawn by the author in favour of the GitLab based
proposal in PEP 507.

If anyone else would like to take over championing this PEP, contact the
core-workflow mailing list

Proposal

This PEP proposes that an instance of the self-hosted Kallithea code
repository management system be deployed as "forge.python.org".

Individual repositories (such as the developer guide or the PEPs
repository) may then be migrated from the existing hg.python.org
infrastructure to the new forge.python.org infrastructure on a
case-by-case basis. Each migration will need to decide whether to retain
a read-only mirror on hg.python.org, or whether to just migrate
wholesale to the new location.

In addition to supporting read-only mirrors on hg.python.org,
forge.python.org will also aim to support hosting mirrors on popular
proprietary hosting sites like GitHub and BitBucket. The aim will be to
allow users familiar with these sites to submit and discuss pull
requests using their preferred workflow, with forge.python.org
automatically bringing those contributions over to the master
repository.

Given the availability and popularity of commercially backed "free for
open source projects" repository hosting services, this would not be a
general purpose hosting site for arbitrary Python projects. The initial
focus will be specifically on CPython and other repositories currently
hosted on hg.python.org. In the future, this could potentially be
expanded to consolidating other PSF managed repositories that are
currently externally hosted to gain access to a pull request based
workflow, such as the repository for the python.org Django application.
As with the initial migrations, any such future migrations would be
considered on a case-by-case basis, taking into account the preferences
of the primary users of each repository.

Rationale

Currently, hg.python.org hosts more than just the core CPython
repository, it also hosts other repositories such as those for the
CPython developer guide and for Python Enhancement Proposals, along with
various "sandbox" repositories for core developer experimentation.

While the simple "pull request" style workflow made popular by code
hosting sites like GitHub and BitBucket isn't adequate for the complex
branching model needed for parallel maintenance and development of the
various CPython releases, it's a good fit for several of the ancillary
projects that surround CPython that we don't wish to move to a
proprietary hosting site.

The key requirements proposed for a PSF provided software forge are:

-   MUST support simple "pull request" style workflows
-   MUST support online editing for simple changes
-   MUST be backed by an active development organisation (community or
    commercial)
-   MUST support self-hosting of the master repository on PSF
    infrastructure without ongoing fees

Additional recommended requirements that are satisfied by this proposal,
but may be negotiable if a sufficiently compelling alternative is
presented:

-   SHOULD be a fully open source application written in Python
-   SHOULD support Mercurial (for consistency with existing tooling)
-   SHOULD support Git (to provide that option to users that prefer it)
-   SHOULD allow users of git and Mercurial clients to transparently
    collaborate on the same repository
-   SHOULD allow users of GitHub and BitBucket to submit proposed
    changes using the standard pull request workflows offered by those
    tools
-   SHOULD be open to customisation to meet the needs of CPython core
    development, including providing a potential path forward for the
    proposed migration to a core reviewer model in PEP 462

The preference for self-hosting without ongoing fees rules out the
free-as-in-beer providers like GitHub and BitBucket, in addition to the
various proprietary source code management offerings.

The preference for Mercurial support not only rules out GitHub, but also
other Git-only solutions like GitLab and Gitorious.

The hard requirement for online editing support rules out the Apache
Allura/HgForge combination.

The preference for a fully open source solution rules out RhodeCode.

Of the various options considered by the author of this proposal, that
leaves Kallithea SCM as the proposed foundation for a forge.python.org
service.

Kallithea is a full GPLv3 application (derived from the clearly and
unambiguously GPLv3 licensed components of RhodeCode) that is being
developed under the auspices of the Software Freedom Conservancy. The
Conservancy has affirmed that the Kallithea codebase is completely and
validly licensed under GPLv3. In addition to their role in building the
initial Kallithea community, the Conservancy is also the legal home of
both the Mercurial and Git projects. Other SFC member projects that may
be familiar to Python users include Twisted, Gevent, BuildBot and PyPy.

Intended Benefits

The primary benefit of deploying Kallithea as forge.python.org is that
supporting repositories such as the developer guide and the PEP repo
could potentially be managed using pull requests and online editing.
This would be much simpler than the current workflow which requires PEP
editors and other core developers to act as intermediaries to apply
updates suggested by other users.

The richer administrative functionality would also make it substantially
easier to grant users access to particular repositories for
collaboration purposes, without having to grant them general access to
the entire installation. This helps lower barriers to entry, as trust
can more readily be granted and earned incrementally, rather than being
an all-or-nothing decision around granting core developer access.

Sustaining Engineering Considerations

Even with its current workflow, CPython itself remains one of the
largest open source projects in the world (in the top 2% of projects
tracked on OpenHub). Unfortunately, we have been significantly less
effective at encouraging contributions to the projects that make up
CPython's workflow infrastructure, including ensuring that our
installations track upstream, and that wherever feasible, our own
customisations are contributed back to the original project.

As such, a core component of this proposal is to actively engage with
the upstream Kallithea community to lower the barriers to working with
and on the Kallithea SCM, as well as with the PSF Infrastructure team to
ensure the forge.python.org service integrates cleanly with the PSF's
infrastructure automation.

This approach aims to provide a number of key benefits:

-   allowing those of us contributing to maintenance of this service to
    be as productive as possible in the time we have available
-   offering a compelling professional development opportunity to those
    volunteers that choose to participate in maintenance of this service
-   making the Kallithea project itself more attractive to other
    potential users by making it as easy as possible to adopt, deploy
    and manage
-   as a result of the above benefits, attracting sufficient
    contributors both in the upstream Kallithea community, and within
    the CPython infrastructure community, to allow the forge.python.org
    service to evolve effectively to meet changing developer
    expectations

Some initial steps have already been taken to address these sustaining
engineering concerns:

-   Tymoteusz Jankowski has been working with Donald Stufft to work out
    what would be involved in deploying Kallithea using the PSF's Salt
    based infrastructure automation.
-   Graham Dumpleton and I have been working on making it easy to deploy
    demonstration Kallithea instances to the free tier of Red Hat's open
    source hosting service, OpenShift Online. (See the comments on that
    post, or the quickstart issue tracker for links to Graham's follow
    on work)

The next major step to be undertaken is to come up with a local
development workflow that allows contributors on Windows, Mac OS X and
Linux to run the Kallithea tests locally, without interfering with the
operation of their own system. The currently planned approach for this
is to focus on Vagrant, which is a popular automated virtual machine
management system specifically aimed at developers running local VMs for
testing purposes. The Vagrant based development guidelines for OpenShift
Origin provide an extended example of the kind of workflow this approach
enables. It's also worth noting that Vagrant is one of the options for
working with a local build of the main python.org website.

If these workflow proposals end up working well for Kallithea, they may
also be worth proposing for use by the upstream projects backing other
PSF and CPython infrastructure services, including Roundup, BuildBot,
and the main python.org web site.

Personal Motivation

As of July 2015, I now work for Red Hat as a software development
workflow designer and process architect, focusing on the upstream
developer experience in Fedora. Two of the key pieces of that experience
will be familiar to many web service developers: Docker for local
container management, and Vagrant for cross-platform local development
VM management. Spending time applying these technologies in multiple
upstream contexts helps provide additional insight into what works well
and what still needs further improvement to provide a good software
development experience that is well integrated on Fedora, but also
readily available on other Linux distributions, Windows, Mac OS X.

In relation to code review workflows in particular, the primary code
review workflow management tools I've used in my career are Gerrit (for
multi-step code review with fine-grained access control), GitHub and
BitBucket (for basic pull request based workflows), and Rietveld (for
CPython's optional pre-commit reviews).

Kallithea is interesting as a base project to build, as it's currently a
combined repo hosting and code review management platform, but doesn't
directly integrate the two by offering online merges. This creates the
opportunity to blend the low barrier to entry benefits of the
GitHub/BitBucket pull request model with the mentoring and task hand-off
benefits of Gerrit in defining an online code merging model for
Kallithea in collaboration with the upstream Kallithea developers.

Technical Concerns and Challenges

Introducing a new service into the CPython infrastructure presents a
number of interesting technical concerns and challenges. This section
covers several of the most significant ones.

Service hosting

The default position of this PEP is that the new forge.python.org
service will be integrated into the existing PSF Salt infrastructure and
hosted on the PSF's Rackspace cloud infrastructure.

However, other hosting options will also be considered, in particular,
possible deployment as a Kubernetes hosted web service on either Google
Container Engine or the next generation of Red Hat's OpenShift Online
service, by using either GCEPersistentDisk or the open source GlusterFS
distributed filesystem to hold the source code repositories.

Ongoing infrastructure maintenance

Ongoing infrastructure maintenance is an area of concern within the PSF,
as we currently lack a system administrator mentorship program
equivalent to the Fedora Infrastructure Apprentice or GNOME
Infrastructure Apprentice programs.

Instead, systems tend to be maintained largely by developers as a
part-time activity on top of their development related contributions,
rather than seeking to recruit folks that are more interested in
operations (i.e. keeping existing systems running well) than they are in
development (i.e. making changes to the services to provide new features
or a better user experience, or to address existing issues).

While I'd personally like to see the PSF operating such a program at
some point in the future, I don't consider setting one up to be a
feasible near term goal. However, I do consider it feasible to continue
laying the groundwork for such a program by extending the PSF's existing
usage of modern infrastructure technologies like OpenStack and Salt to
cover more services, as well as starting to explore the potential
benefits of containers and container platforms when it comes to
maintaining and enhancing PSF provided services.

I also plan to look into the question of whether or not an open source
cloud management platform like ManageIQ may help us bring our emerging
"cloud sprawl" problem across Rackspace, Google, Amazon and other
services more under control.

User account management

Ideally we'd like to be able to offer a single account that spans all
python.org services, including Kallithea, Roundup/Rietveld, PyPI and the
back end for the new python.org site, but actually implementing that
would be a distinct infrastructure project, independent of this PEP.
(It's also worth noting that the fine-grained control of ACLs offered by
such a capability is a prerequisite for setting up an effective system
administrator mentorship program)

For the initial rollout of forge.python.org, we will likely create yet
another identity silo within the PSF infrastructure. A potentially
superior alternative would be to add support for python-social-auth to
Kallithea, but actually doing so would not be a requirement for the
initial rollout of the service (the main technical concern there is that
Kallithea is a Pylons application that has not yet been ported to
Pyramid, so integration will require either adding a Pylons backend to
python-social-auth, or else embarking on the Pyramid migration in
Kallithea).

Breaking existing SSH access and links for Mercurial repositories

This PEP proposes leaving the existing hg.python.org installation alone,
and setting up Kallithea on a new host. This approach minimises the risk
of interfering with the development of CPython itself (and any other
projects that don't migrate to the new software forge), but does make
any migrations of existing repos more disruptive (since existing
checkouts will break).

Integration with Roundup

Kallithea provides configurable issue tracker integration. This will
need to be set up appropriately to integrate with the Roundup issue
tracker at bugs.python.org before the initial rollout of the
forge.python.org service.

Accepting pull requests on GitHub and BitBucket

The initial rollout of forge.python.org would support publication of
read-only mirrors, both on hg.python.org and other services, as that is
a relatively straightforward operation that can be implemented in a
commit hook.

While a highly desirable feature, accepting pull requests on external
services, and mirroring them as submissions to the master repositories
on forge.python.org is a more complex problem, and would likely not be
included as part of the initial rollout of the forge.python.org service.

Transparent Git and Mercurial interoperability

Kallithea's native support for both Git and Mercurial offers an
opportunity to make it relatively straightforward for developers to use
the client of their choice to interact with repositories hosted on
forge.python.org.

This transparent interoperability does not exist yet, but running our
own multi-VCS repository hosting service provides the opportunity to
make this capability a reality, rather than passively waiting for a
proprietary provider to deign to provide a feature that likely isn't in
their commercial interest. There's a significant misalignment of
incentives between open source communities and commercial providers in
this particular area, as even though offering VCS client choice can
significantly reduce community friction by eliminating the need for
projects to make autocratic decisions that force particular tooling
choices on potential contributors, top down enforcement of tool
selection (regardless of developer preference) is currently still the
norm in the corporate and other organisational environments that produce
GitHub and Atlassian's paying customers.

Prior to acceptance, in the absence of transparent interoperability,
this PEP should propose specific recommendations for inclusion in the
CPython developer's guide section for git users for creating pull
requests against forge.python.org hosted Mercurial repositories.

Pilot Objectives and Timeline

[TODO: Update this section for Brett's revised timeline, which aims to
have a CPython demo repository online by October 31st, to get a better
indication of future capabilities once CPython itself migrates over to
the new system, rather than just the support repos]

This proposal is part of Brett Cannon's current evaluation of
improvement proposals for various aspects of the CPython development
workflow. Key dates in that timeline are:

-   Feb 1: Draft proposal published (for Kallithea, this PEP)
-   Apr 8: Discussion of final proposals at Python Language Summit
-   May 1: Brett's decision on which proposal to accept
-   Sep 13: Python 3.5 released, adopting new workflows for Python 3.6

If this proposal is selected for further development, it is proposed to
start with the rollout of the following pilot deployment:

-   a reference implementation operational at
    kallithea-pilot.python.org, containing at least the developer guide
    and PEP repositories. This will be a "throwaway" instance, allowing
    core developers and other contributors to experiment freely without
    worrying about the long term consequences for the repository
    history.
-   read-only live mirrors of the Kallithea hosted repositories on
    GitHub and BitBucket. As with the pilot service itself, these would
    be temporary repos, to be discarded after the pilot period ends.
-   clear documentation on using those mirrors to create pull requests
    against Kallithea hosted Mercurial repositories (for the pilot, this
    will likely not include using the native pull request workflows of
    those hosted services)
-   automatic linking of issue references in code review comments and
    commit messages to the corresponding issues on bugs.python.org
-   draft updates to PEP 1 explaining the Kallithea-based PEP editing
    and submission workflow

The following items would be needed for a production migration, but
there doesn't appear to be an obvious way to trial an updated
implementation as part of the pilot:

-   adjusting the PEP publication process and the developer guide
    publication process to be based on the relocated Mercurial repos

The following items would be objectives of the overall workflow
improvement process, but are considered "desirable, but not essential"
for the initial adoption of the new service in September (if this
proposal is the one selected and the proposed pilot deployment is
successful):

-   allowing the use of python-social-auth to authenticate against the
    PSF hosted Kallithea instance
-   allowing the use of the GitHub and BitBucket pull request workflows
    to submit pull requests to the main Kallithea repo
-   allowing easy triggering of forced BuildBot runs based on Kallithea
    hosted repos and pull requests (prior to the implementation of PEP
    462, this would be intended for use with sandbox repos rather than
    the main CPython repo)

Future Implications for CPython Core Development

The workflow requirements for the main CPython development repository
are significantly more complex than those for the repositories being
discussed in this PEP. These concerns are covered in more detail in PEP
462.

Given Guido's recommendation to replace Rietveld with a more actively
maintained code review system, my current plan is to rewrite that PEP to
use Kallithea as the proposed glue layer, with enhanced Kallithea pull
requests eventually replacing the current practice of uploading patche
files directly to the issue tracker.

I've also started working with Pierre Yves-David on a custom Mercurial
extension that automates some aspects of the CPython core development
workflow.

Copyright

This document has been placed in the public domain.


text:
PEP: 648 Title: Extensible customizations of the interpreter at startup
Author: Mario Corchero<EMAIL_ADDRESS>Sponsor: Pablo Galindo
Discussions-To:
https://discuss.python.org/t/pep-648-extensible-customizations-of-the-interpreter-at-startup/6403
Status: Rejected Type: Standards Track Content-Type: text/x-rst Created:
30-Dec-2020 Python-Version: 3.11 Post-History: 16-Dec-2020, 18-Dec-2020

Abstract

This PEP proposes supporting extensible customization of the interpreter
by allowing users to install files that will be executed at startup.

PEP Rejection

PEP 648 was rejected by the steering council as it has a limited number
of use cases and further complicates the startup sequence.

Motivation

System administrators, tools that repackage the interpreter and some
libraries need to customize aspects of the interpreter at startup time.

This is usually achieved via sitecustomize.py for system administrators
whilst libraries rely on exploiting pth files. This PEP proposes a way
of achieving the same functionality in a more user-friendly and
structured way.

Limitations of pth files

If a library needs to perform any customization before an import or that
relates to the general working of the interpreter, they often rely on
the fact that pth files, which are loaded at startup and implemented via
the site module[1], can include Python code that will be executed when
the pth file is evaluated.

Note that pth files were originally developed to just add additional
directories to sys.path, but they may also contain lines which start
with "import", which will be passed to exec(). Users have exploited this
feature to allow the customizations that they needed. See setuptools [2]
or betterexceptions[3] as examples.

Using pth files for this purpose is far from ideal for library
developers, as they need to inject code into a single line preceded by
an import, making it rather unreadable. Library developers following
that practice will usually create a module that performs all actions on
import, as done by betterexceptions[4], but the approach is still not
really user friendly.

Additionally, it is also non-ideal for users of the interpreter if they
want to inspect what is being executed at Python startup as they need to
review all the pth files for potential code execution which can be
spread across all site paths. Most of those pth files will be
"legitimate" pth files that just modify the path, answering the question
of "what is changing my interpreter at startup" a rather complex one.

Lastly, there have been multiple suggestions for removing code execution
from pth files, see[5] and[6].

Limitations of sitecustomize.py

Whilst sitecustomize is an acceptable solution, it assumes a single
person is in charge of the system and the interpreter. If both the
system administrator and the responsibility of provisioning the
interpreter want to add customizations at the interpreter startup they
need to agree on the contents of the file and combine all the changes.
This is not a major limitation though, and it is not the main driver of
this change. Should the change happen, it will also improve the
situation for these users, as rather than having a sitecustomize.py
which performs all those actions, they can have custom isolated files
named after the features they want to enhance. As an example, Ubuntu
could change their current sitecustomize.py to just be
ubuntu_apport_python_hook. This not only better represents its intent
but also gives users of the interpreter a better understanding of the
modifications happening on their interpreter.

Rationale

This PEP proposes supporting extensible customization of the interpreter
at startup by executing all files discovered in directories named
__sitecustomize__ in sitepackages[7] or usersitepackages[8] at startup
time.

Why __sitecustomize__

The name aims to follow the already existing concept of
sitecustomize.py. As the directory will be within sys.path, given that
it is located in site paths, we choose to use double underscore around
its name, to prevent colliding with the already existing
sitecustomize.py.

Discovering the new __sitecustomize__ directories

The Python interpreter will look at startup for directory named
__sitecustomize__ within any of the standard site-packages path.

These are commonly the Python system location and the user location, but
are ultimately defined by the site module logic.

Users can use site.sitepackages[9] and site.usersitepackages[10] to know
the paths where the interpreter can discover __sitecustomize__
directories.

Time of __sitecustomize__ discovery

The __sitecustomize__ directories will be discovered exactly after pth
files are discovered in a site-packages path as part of site.addsitedir
[11].

These is repeated for each of the site-packages path in the exact same
order that is being followed today for pth files.

Order of execution within __sitecustomize__

The implementation will execute the files within __sitecustomize__ by
sorting them by name when discovering each of the __sitecustomize__
directories. We discourage users to rely on the order of execution
though.

We considered executing them in random order, but that could result in
different results depending on how the interpreter chooses to pick up
those files. So even if it won't be a good practice to rely on other
files being executed, we think that is better than having randomly
different results on interpreter startup. We chose to run the files
after the pth files in case a user needs to add items to the path before
running a files.

Interaction with pth files

pth files can be used to add paths into sys.path, but this should not
affect the __sitecustomize__ discovery process, as those directories are
looked up exclusively in site-packages paths.

Execution of files within __sitecustomize__

When a __sitecustomize__ directory is discovered, all of the files that
have a .py extension within it will be read with io.open_code and
executed by using exec[12].

An empty dictionary will be passed as globals to the exec function to
prevent unexpected interactions between different files.

Failure handling

Any error on the execution of any of the files will not be logged unless
the interpreter is run in verbose mode and it should not stop the
evaluation of other files. The user will receive a message in stderr
saying that the file failed to be executed and that verbose mode can be
used to get more information. This behaviour mimics the one existing for
sitecustomize.py.

Interaction with virtual environments

The customizations applied to an interpreter via the new
__sitecustomize__ solutions will continue to work when a user creates a
virtual environment the same way that sitecustomize.py interact with
virtual environments.

This is a difference when compared to pth files, which are not
propagated into virtual environments unless include-system-site-packages
is enabled.

If library maintainers have features installed via __sitecustomize__
that they do not want to propagate into virtual environments, they
should detect if they are running within a virtual environment by
checking sys.prefix == sys.base_prefix. This behavior is similar to
packages that modify the global sitecustomize.py.

Interaction with sitecustomize.py and usercustomize.py

Until removed, sitecustomize and usercustomize will be executed after
__sitecustomize__ similar to pth files. See the Backward compatibility
section for information on removal plans for sitecustomize and
usercustomize.

Identifying all installed files

To facilitate debugging of the Python startup, if the site module is
invoked it will print the __sitecustomize__ directories that will be
discovered on startup.

Files naming convention

Packages will be encouraged to include the name of the package within
the name of the file to avoid collisions between packages. But the only
requirement on the filename is that it ends in .py for the interpreter
to execute them.

Disabling start files

In some scenarios, like when the startup time is key, it might be
desired to disable this option altogether. The already existing flag
-S[13] will disable all site-related manipulation, including this new
feature. If the flag is passed in, __sitecustomize__ directories will
not be discovered.

Additionally, to allow for starting the interpreter disabling only this
new feature a new option will be added under -X: disablesitecustomize,
which will disable the discovery of __sitecustomize__ exclusively.

Lastly, the user can disable the discovery of __sitecustomize__
directories only in the user site by disabling the user site via any of
the multiple options in the site.py module.

Support in build backends

Whilst build backends can choose to provide an option to facilitate the
installation of these files into a __sitecustomize__ directory, this PEP
does not address that directly. Similar to pth files, build backends can
choose to not provide an easy-to-configure mechanism for
__sitecustomize__ files and let users hook into the installation process
to include such files. We do not think build backends enhanced support
as a requirement for this PEP.

Impact on startup time

A concern in this implementation is how Python interpreter startup time
can be affected by this addition. We expect the performance impact to be
highly coupled to the logic in the files that a user or sysadmin
installs in the Python environment being tested.

If the interpreter has any files in their __sitecustomize__ directory,
the file execution time plus a call reading the code will be added to
the startup time. This is similar to how code execution is impacting
startup time through sitecustomize.py, usercustomize.py and code in pth
files. We will therefore focus here on comparing this solution against
those three, as otherwise the actual time added to startup is highly
dependent on the code that is being executed in those files.

Results were gathered by running "./python.exe -c pass" with perf on 50
iterations, repeating 50 times the command on each iteration and getting
the geometric mean of all the results. The file used to run those
benchmarks is checked in in the reference implementation[14].

The benchmark was run with 3.10 alpha 7 compiled with PGO and LTO with
the following parameters and system state:

-   Perf event: Max sample rate set to 1 per second
-   CPU Frequency: Minimum frequency of CPU 17,35 set to the maximum
    frequency
-   Turbo Boost (MSR): Turbo Boost disabled on CPU 17: MSR 0x1a0 set to
    0x4000850089
-   IRQ affinity: Set default affinity to CPU 0-16,18-34
-   IRQ affinity: Set affinity of IRQ
    1,3-16,21,25-31,56-59,68-85,87,89-90,92-93,95-104 to CPU 0-16,18-34
-   CPU: use 2 logical CPUs: 17,35
-   Perf event: Maximum sample rate: 1 per second
-   ASLR: Full randomization
-   Linux scheduler: Isolated CPUs (2/36): 17,35
-   Linux scheduler: RCU disabled on CPUs (2/36): 17,35
-   CPU Frequency: 0-16,18-34=min=1200 MHz, max=3600 MHz;
    17,35=min=max=3600 MHz
-   Turbo Boost (MSR): CPU 17,35: disabled

The code placed to be executed in pth files, sitecustomize.py,
usercustomize.py and files within __sitecustomize__ is the following:

  import time; x = time.time() ** 5

The file is aimed at execution a simple operation but still expected to
be negligible. This is to put the experiment in a situation where we
make visible any hit on performance due to the mechanism whilst still
making it relatively realistic. Additionally, it starts with an import
and is a single line to be able to be used in pth files.

+--------------------------+--------------------------+--------------+
| Test                     | # of files               | Time (us)    |
+==========================+==========================+==============+
|   #                      | sitecustomize.py         | Run 1 Run 2  |
|                          | usercustomize.py pth     |              |
|                          | __sitecustomize__        |              |
+--------------------------+--------------------------+--------------+
| ----------------------   | ====================     | ====== ===== |
|                          | ====================     |              |
|                          | =======                  |              |
|                          | =====================    |              |
+--------------------------+--------------------------+--------------+
|   1                      | 0 0 0 Dir not created    | 13884 13897  |
+--------------------------+--------------------------+--------------+
|   2                      | 0 0 0 0                  | 13871 13818  |
+--------------------------+--------------------------+--------------+
|   3                      | 0 0 1 0                  | 13964 13924  |
+--------------------------+--------------------------+--------------+
|   4                      | 0 0 0 1                  | 13940 13939  |
+--------------------------+--------------------------+--------------+
|   5                      | 1 1 0 0                  | 13990 13993  |
+--------------------------+--------------------------+--------------+
|   6                      | 0 0 0 2 (system + user)  | 14063 14040  |
+--------------------------+--------------------------+--------------+
|   7                      | 0 0 50 0                 | 16011 16014  |
+--------------------------+--------------------------+--------------+
|   8                      | 0 0 0 50                 | 15456 15448  |
+--------------------------+--------------------------+--------------+

Results can be reproduced with run-benchmark.py script provided in the
reference implementation[15].

We interpret the following from these results:

-   Using two __sitecustomize__ scripts compared to sitecustomize.py and
    usercustomize.py slows down the interpreter by 0.3%. We expect this
    slowdown until sitecustomize.py and usercustomize.py are removed in
    a future release as even if the user does not create the files, the
    interpreter will still attempt to import them.
-   With the arbitrary 50 pth files with code tested, moving those to
    __sitecustomize__ produces a speedup of ~3.5% in startup. Which is
    likely related to the simpler logic to evaluate __sitecustomize__
    files compared to pth file execution.
-   In general all measurements show that there is a low impact on
    startup time with this addition.

Audit Event

A new audit event will be added and triggered on __sitecustomize__
execution to facilitate security inspection by calling sys.audit [16]
with "sitecustimze.exec_file" as name and the filename as argument.

Security implications

This PEP aims to move all code execution from pth files to files within
a __sitecustomize__ directory. We think this is an improvement to system
admins for the following reasons:

-   Allows to quickly identify the code being executed at startup time
    by the interpreter by looking into a single directory rather than
    having to scan all pth files.
-   Allows to track usage of this feature through the new proposed audit
    event.
-   Gives finer grain control by allowing to tune permissions on the
    __sitecustomize__ directory, potentially allowing users to install
    only packages that does not change the interpreter startup.

In short, whilst this allows for a malicious users to drop a file that
will be executed at startup, it's an improvement compared to the
existing pth files.

How to teach this

This can be documented and taught as simple as saying that the
interpreter will try to look for the __sitecustomize__ directory at
startup in its site paths and if it finds any files with .py extension,
it will then execute it one by one.

For system administrators and tools that package the interpreter, we can
now recommend placing files in __sitecustomize__ as they used to place
sitecustomize.py. Being more comfortable on that their content won't be
overridden by the next person, as they can provide with specific files
to handle the logic they want to customize.

Library developers should be able to specify a new argument on tools
like setuptools that will inject those new files. Something like
sitecustomize_files=["scripts/betterexceptions.py"], which allows them
to add those. Should the build backend not support that, they can
manually install them as they used to do with pth files. We will
recommend them to include the name of the package as part of the file's
name.

Backward compatibility

This PEP adds a deprecation warning on sitecustomize.py,
usercustomize.py and pth code execution in 3.11, 3.12 and 3.13. With
plans on removing those features by 3.14. The migration from those
solutions to __sitecustomize__ should ideally be just moving the logic
into a different file.

Whilst the existing sitecustomize.py mechanism was created targeting
System Administrators that placed it in a site path, the file could be
actually placed anywhere in the path at the time that the interpreter
was starting up. The new mechanism does not allow for users to place
__sitecustomize__ directories anywhere in the path, but only in site
paths. System administrators can recover a similar behavior to
sitecustomize.py by adding a custom file in __sitecustomize__ which just
imports sitecustomize as a migration path.

Reference Implementation

An initial implementation that passes the CPython test suite is
available for evaluation[17].

This implementation is just for the reviewer to play with and check
potential issues that this PEP could generate.

Rejected Ideas

Do nothing

Whilst the current status "works" it presents the issues listed in the
motivation. After analyzing the impact of this change, we believe it is
worth it, given the enhanced experience it brings.

Formalize using pth files

Another option would be to just glorify and document the usage of pth
files to inject code at startup code, but that is a suboptimal
experience for users as listed in the motivation.

Making __sitecustomize__ a namespace package

We considered making the directory a namespace package and just import
all the modules within it, which allowed searching across all paths in
sys.path at initialization time and provided a way to declare
dependencies between files by importing each other. This was rejected
for multiple reasons:

1.  This was unnecessarily broadening the list of paths where arbitrary
    files are executed.
2.  The logic brought additional complexity, like what to do if a
    package were to install an __init__.py file in one of the locations.
3.  It's cheaper to search for __sitecustomize__ as we are looking for
    pth files already in the site paths compared to performing an actual
    import of a namespace package.

Support for shutdown customization

init.d users might be tempted to implement this feature in a way that
users could also add code at shutdown, but extra support for that is not
needed, as Python users can already do that via atexit.

Using entry_points

We considered extending the use of entry points to allow specifying
files that should be executed at startup but we discarded that solution
due to two main reasons. The first one being impact on startup time.
This approach will require scanning all packages distribution
information to just execute a handful of files. This has an impact on
performance even if the user is not using the feature and such impact
growths linearly with the number of packages installed in the
environment. The second reason was that the proposed implementation in
this PEP offers a single solution for startup customization for packages
and system administrators. Additionally, if the main objective of entry
points is to make it easy for libraries to install files at startup,
that can still be added and make the build backends just install the
files within the __sitecustomize__ directory.

Copyright

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.

Acknowledgements

Thanks Pablo Galindo for contributing to this PEP and offering his PC to
run the benchmark.

References

[1] https://docs.python.org/3/library/site.html

[2] https://github.com/pypa/setuptools/blob/b6bbe236ed0689f50b5148f1172510b975687e62/setup.py#L100

[3] https://github.com/Qix-/better-exceptions/blob/7b417527757d555faedc354c86d3b6fe449200c2/better_exceptions_hook.pth#L1

[4] https://github.com/Qix-/better-exceptions/blob/7b417527757d555faedc354c86d3b6fe449200c2/better_exceptions_hook.pth#L1

[5] https://bugs.python.org/issue24534

[6] https://bugs.python.org/issue33944

[7] https://docs.python.org/3/library/site.html?highlight=site#site.getsitepackages

[8] https://docs.python.org/3/library/site.html?highlight=site#site.getusersitepackages

[9] https://docs.python.org/3/library/site.html?highlight=site#site.getsitepackages

[10] https://docs.python.org/3/library/site.html?highlight=site#site.getusersitepackages

[11] https://github.com/python/cpython/blob/5787ba4a45492e232f5470c7d2e93763198e4b22/Lib/site.py#L207

[12] https://docs.python.org/3/library/functions.html#exec

[13] https://docs.python.org/3/using/cmdline.html#id3

[14] https://github.com/mariocj89/cpython/tree/pu/__sitecustomize__

[15] https://github.com/mariocj89/cpython/tree/pu/__sitecustomize__

[16] https://docs.python.org/3/library/sys.html#sys.audit

[17] https://github.com/mariocj89/cpython/tree/pu/__sitecustomize__

license: Public Domain


text:
PEP: 229 Title: Using Distutils to Build Python Version: $Revision$
Last-Modified: $Date$ Author: A.M. Kuchling<EMAIL_ADDRESS>Status: Final
Type: Standards Track Content-Type: text/x-rst Created: 16-Nov-2000
Python-Version: 2.1 Post-History:

Introduction

The Modules/Setup mechanism has some flaws:

-   People have to remember to uncomment bits of Modules/Setup in order
    to get all the possible modules.
-   Moving Setup to a new version of Python is tedious; new modules have
    been added, so you can't just copy the older version, but have to
    reconcile the two versions.
-   Users have to figure out where the needed libraries, such as zlib,
    are installed.

Proposal

Use the Distutils to build the modules that come with Python.

The changes can be broken up into several pieces:

1.  The Distutils needs some Python modules to be able to build modules.
    Currently I believe the minimal list is posix, _sre, and string.

    These modules will have to be built before the Distutils can be
    used, so they'll simply be hardwired into Modules/Makefile and be
    automatically built.

2.  A top-level setup.py script will be written that checks the
    libraries installed on the system and compiles as many modules as
    possible.

3.  Modules/Setup will be kept and settings in it will override
    setup.py's usual behavior, so you can disable a module known to be
    buggy, or specify particular compilation or linker flags. However,
    in the common case where setup.py works correctly, everything in
    Setup will remain commented out. The other Setup.* become
    unnecessary, since nothing will be generating Setup automatically.

The patch was checked in for Python 2.1, and has been subsequently
modified.

Implementation

Patch #102588 on SourceForge contains the proposed patch. Currently the
patch tries to be conservative and to change as few files as possible,
in order to simplify backing out the patch. For example, no attempt is
made to rip out the existing build mechanisms. Such simplifications can
wait for later in the beta cycle, when we're certain the patch will be
left in, or they can wait for Python 2.2.

The patch makes the following changes:

-   Makes some required changes to distutils/sysconfig (these will be
    checked in separately)
-   In the top-level Makefile.in, the "sharedmods" target simply runs
    "./python setup.py build", and "sharedinstall" runs
    "./python setup.py install". The "clobber" target also deletes the
    build/ subdirectory where Distutils puts its output.
-   Modules/Setup.config.in only contains entries for the gc and thread
    modules; the readline, curses, and db modules are removed because
    it's now setup.py's job to handle them.
-   Modules/Setup.dist now contains entries for only 3 modules --_sre,
    posix, and strop.
-   The configure script builds setup.cfg from setup.cfg.in. This is
    needed for two reasons: to make building in subdirectories work, and
    to get the configured installation prefix.
-   Adds setup.py to the top directory of the source tree. setup.py is
    the largest piece of the puzzle, though not the most complicated.
    setup.py contains a subclass of the BuildExt class, and extends it
    with a detect_modules() method that does the work of figuring out
    when modules can be compiled, and adding them to the 'exts' list.

Unresolved Issues

Do we need to make it possible to disable the 3 hard-wired modules
without manually hacking the Makefiles? [Answer: No.]

The Distutils always compile modules as shared libraries. How do we
support compiling them statically into the resulting Python binary?

[Answer: building a Python binary with the Distutils should be feasible,
though no one has implemented it yet. This should be done someday, but
isn't a pressing priority as messing around with the top-level
Makefile.pre.in is good enough.]

Copyright

This document has been placed in the public domain.


text:
PEP: 568 Title: Generator-sensitivity for Context Variables Author:
Nathaniel J. Smith<EMAIL_ADDRESS>Status: Deferred Type: Standards
Track Content-Type: text/x-rst Created: 04-Jan-2018 Python-Version: 3.8
Post-History:

Abstract

Context variables provide a generic mechanism for tracking dynamic,
context-local state, similar to thread-local storage but generalized to
cope work with other kinds of thread-like contexts, such as asyncio
Tasks. PEP 550 proposed a mechanism for context-local state that was
also sensitive to generator context, but this was pretty complicated, so
the BDFL requested it be simplified. The result was PEP 567, which is
targeted for inclusion in 3.7. This PEP then extends PEP 567's machinery
to add generator context sensitivity.

This PEP is starting out in the "deferred" status, because there isn't
enough time to give it proper consideration before the 3.7 feature
freeze. The only goal right now is to understand what would be required
to add generator context sensitivity in 3.8, so that we can avoid
shipping something in 3.7 that would rule it out by accident. (Ruling it
out on purpose can wait until 3.8 ;-).)

Rationale

[Currently the point of this PEP is just to understand how this would
work, with discussion of whether it's a good idea deferred until after
the 3.7 feature freeze. So rationale is TBD.]

High-level summary

Instead of holding a single Context, the threadstate now holds a
ChainMap of Contexts. ContextVar.get and ContextVar.set are backed by
the ChainMap. Generators and async generators each have an associated
Context that they push onto the ChainMap while they're running to
isolate their context-local changes from their callers, though this can
be overridden in cases like @contextlib.contextmanager where "leaking"
context changes from the generator into its caller is desirable.

Specification

Review of PEP 567

Let's start by reviewing how PEP 567 works, and then in the next section
we'll describe the differences.

In PEP 567, a Context is a Mapping from ContextVar objects to arbitrary
values. In our pseudo-code here we'll pretend that it uses a dict for
backing storage. (The real implementation uses a HAMT, which is
semantically equivalent to a dict but with different performance
trade-offs.):

    class Context(collections.abc.Mapping):
        def __init__(self):
            self._data = {}
            self._in_use = False

        def __getitem__(self, key):
            return self._data[key]

        def __iter__(self):
            return iter(self._data)

        def __len__(self):
            return len(self._data)

At any given moment, the threadstate holds a current Context
(initialized to an empty Context when the threadstate is created); we
can use Context.run to temporarily switch the current Context:

    # Context.run
    def run(self, fn, *args, **kwargs):
        if self._in_use:
            raise RuntimeError("Context already in use")
        tstate = get_thread_state()
        old_context = tstate.current_context
        tstate.current_context = self
        self._in_use = True
        try:
            return fn(*args, **kwargs)
        finally:
            state.current_context = old_context
            self._in_use = False

We can fetch a shallow copy of the current Context by calling
copy_context; this is commonly used when spawning a new task, so that
the child task can inherit context from its parent:

    def copy_context():
        tstate = get_thread_state()
        new_context = Context()
        new_context._data = dict(tstate.current_context)
        return new_context

In practice, what end users generally work with is ContextVar objects,
which also provide the only way to mutate a Context. They work with a
utility class Token, which can be used to restore a ContextVar to its
previous value:

    class Token:
        MISSING = sentinel_value()

        # Note: constructor is private
        def __init__(self, context, var, old_value):
            self._context = context
            self.var = var
            self.old_value = old_value

        # XX: PEP 567 currently makes this a method on ContextVar, but
        # I'm going to propose it switch to this API because it's simpler.
        def reset(self):
            # XX: should we allow token reuse?
            # XX: should we allow tokens to be used if the saved
            # context is no longer active?
            if self.old_value is self.MISSING:
                del self._context._data[self.context_var]
            else:
                self._context._data[self.context_var] = self.old_value

    # XX: the handling of defaults here uses the simplified proposal from
    # https://mail.python.org/pipermail/python-dev/2018-January/151596.html
    # This can be updated to whatever we settle on, it was just less
    # typing this way :-)
    class ContextVar:
        def __init__(self, name, *, default=None):
            self.name = name
            self.default = default

        def get(self):
            context = get_thread_state().current_context
            return context.get(self, self.default)

        def set(self, new_value):
            context = get_thread_state().current_context
            token = Token(context, self, context.get(self, Token.MISSING))
            context._data[self] = new_value
            return token

Changes from PEP 567 to this PEP

In general, Context remains the same. However, now instead of holding a
single Context object, the threadstate stores a stack of them. This
stack acts just like a collections.ChainMap, so we'll use that in our
pseudocode. Context.run then becomes:

    # Context.run
    def run(self, fn, *args, **kwargs):
        if self._in_use:
            raise RuntimeError("Context already in use")
        tstate = get_thread_state()
        old_context_stack = tstate.current_context_stack
        tstate.current_context_stack = ChainMap([self])     # changed
        self._in_use = True
        try:
            return fn(*args, **kwargs)
        finally:
            state.current_context_stack = old_context_stack
            self._in_use = False

Aside from some updated variables names (e.g., tstate.current_context →
tstate.current_context_stack), the only change here is on the marked
line, which now wraps the context in a ChainMap before stashing it in
the threadstate.

We also add a Context.push method, which is almost exactly like
Context.run, except that it temporarily pushes the Context onto the
existing stack, instead of temporarily replacing the whole stack:

    # Context.push
    def push(self, fn, *args, **kwargs):
        if self._in_use:
            raise RuntimeError("Context already in use")
        tstate = get_thread_state()
        tstate.current_context_stack.maps.insert(0, self)  # different from run
        self._in_use = True
        try:
            return fn(*args, **kwargs)
        finally:
            tstate.current_context_stack.maps.pop(0)       # different from run
            self._in_use = False

In most cases, we don't expect push to be used directly; instead, it
will be used implicitly by generators. Specifically, every generator
object and async generator object gains a new attribute .context. When
an (async) generator object is created, this attribute is initialized to
an empty Context (self.context = Context()). This is a mutable
attribute; it can be changed by user code. But trying to set it to
anything that isn't a Context object or None will raise an error.

Whenever we enter an generator via __next__, send, throw, or close, or
enter an async generator by calling one of those methods on its
__anext__, asend, athrow, or aclose coroutines, then its .context
attribute is checked, and if non-None, is automatically pushed:

    # GeneratorType.__next__
    def __next__(self):
        if self.context is not None:
            return self.context.push(self.__real_next__)
        else:
            return self.__real_next__()

While we don't expect people to use Context.push often, making it a
public API preserves the principle that a generator can always be
rewritten as an explicit iterator class with equivalent semantics.

Also, we modify contextlib.(async)contextmanager to always set its
(async) generator objects' .context attribute to None:

    # contextlib._GeneratorContextManagerBase.__init__
    def __init__(self, func, args, kwds):
        self.gen = func(*args, **kwds)
        self.gen.context = None                  # added
        ...

This makes sure that code like this continues to work as expected:

    @contextmanager
    def decimal_precision(prec):
        with decimal.localcontext() as ctx:
            ctx.prec = prec
            yield

    with decimal_precision(2):
        ...

The general idea here is that by default, every generator object gets
its own local context, but if users want to explicitly get some other
behavior then they can do that.

Otherwise, things mostly work as before, except that we go through and
swap everything to use the threadstate ChainMap instead of the
threadstate Context. In full detail:

The copy_context function now returns a flattened copy of the
"effective" context. (As an optimization, the implementation might
choose to do this flattening lazily, but if so this will be made
invisible to the user.) Compared to our previous implementation above,
the only change here is that tstate.current_context has been replaced
with tstate.current_context_stack:

    def copy_context() -> Context:
        tstate = get_thread_state()
        new_context = Context()
        new_context._data = dict(tstate.current_context_stack)
        return new_context

Token is unchanged, and the changes to ContextVar.get are trivial:

    # ContextVar.get
    def get(self):
        context_stack = get_thread_state().current_context_stack
        return context_stack.get(self, self.default)

ContextVar.set is a little more interesting: instead of going through
the ChainMap machinery like everything else, it always mutates the top
Context in the stack, and – crucially! – sets up the returned Token to
restore its state later. This allows us to avoid accidentally
"promoting" values between different levels in the stack, as would
happen if we did old = var.get(); ...; var.set(old):

    # ContextVar.set
    def set(self, new_value):
        top_context = get_thread_state().current_context_stack.maps[0]
        token = Token(top_context, self, top_context.get(self, Token.MISSING))
        top_context._data[self] = new_value
        return token

And finally, to allow for introspection of the full context stack, we
provide a new function contextvars.get_context_stack:

    def get_context_stack() -> List[Context]:
        return list(get_thread_state().current_context_stack.maps)

That's all.

Comparison to PEP 550

The main difference from PEP 550 is that it reified what we're calling
"contexts" and "context stacks" as two different concrete types
(LocalContext and ExecutionContext respectively). This led to lots of
confusion about what the differences were, and which object should be
used in which places. This proposal simplifies things by only reifying
the Context, which is "just a dict", and makes the "context stack" an
unnamed feature of the interpreter's runtime state – though it is still
possible to introspect it using get_context_stack, for debugging and
other purposes.

Implementation notes

Context will continue to use a HAMT-based mapping structure under the
hood instead of dict, since we expect that calls to copy_context are
much more common than ContextVar.set. In almost all cases, copy_context
will find that there's only one Context in the stack (because it's rare
for generators to spawn new tasks), and can simply re-use it directly;
in other cases HAMTs are cheap to merge and this can be done lazily.

Rather than using an actual ChainMap object, we'll represent the context
stack using some appropriate structure – the most appropriate options
are probably either a bare list with the "top" of the stack being the
end of the list so we can use push/pop, or else an intrusive linked list
(PyThreadState → Context → Context → ...), with the "top" of the stack
at the beginning of the list to allow efficient push/pop.

A critical optimization in PEP 567 is the caching of values inside
ContextVar. Switching from a single context to a context stack makes
this a little bit more complicated, but not too much. Currently, we
invalidate the cache whenever the threadstate's current Context changes
(on thread switch, and when entering/exiting Context.run). The simplest
approach here would be to invalidate the cache whenever stack changes
(on thread switch, when entering/exiting Context.run, and when
entering/leaving Context.push). The main effect of this is that
iterating a generator will invalidate the cache. It seems unlikely that
this will cause serious problems, but if it does, then I think it can be
avoided with a cleverer cache key that recognizes that pushing and then
popping a Context returns the threadstate to its previous state. (Idea:
store the cache key for a particular stack configuration in the topmost
Context.)

It seems unavoidable in this design that uncached get will be O(n),
where n is the size of the context stack. However, n will generally be
very small – it's roughly the number of nested generators, so usually
n=1, and it will be extremely rare to see n greater than, say, 5. At
worst, n is bounded by the recursion limit. In addition, we can expect
that in most cases of deep generator recursion, most of the Contexts in
the stack will be empty, and thus can be skipped extremely quickly
during lookup. And for repeated lookups the caching mechanism will kick
in. So it's probably possible to construct some extreme case where this
causes performance problems, but ordinary code should be essentially
unaffected.

Copyright

This document has been placed in the public domain.


text:
PEP: 626 Title: Precise line numbers for debugging and other tools.
Author: Mark Shannon<EMAIL_ADDRESS>BDFL-Delegate: Pablo Galindo
<EMAIL_ADDRESS>Status: Final Type: Standards Track Content-Type:
text/x-rst Created: 15-Jul-2020 Python-Version: 3.10 Post-History:
17-Jul-2020

Abstract

Python should guarantee that when tracing is turned on, "line" tracing
events are generated for all lines of code executed and only for lines
of code that are executed.

The f_lineno attribute of frame objects should always contain the
expected line number. During frame execution, the expected line number
is the line number of source code currently being executed. After a
frame has completed, either by returning or by raising an exception, the
expected line number is the line number of the last line of source that
was executed.

A side effect of ensuring correct line numbers, is that some bytecodes
will need to be marked as artificial, and not have a meaningful line
number. To assist tools, a new co_lines attribute will be added that
describes the mapping from bytecode to source.

Motivation

Users of sys.settrace and associated tools should be able to rely on
tracing events being generated for all lines of code, and only for
actual code. They should also be able to assume that the line number in
f_lineno is correct.

The current implementation mostly does this, but fails in a few cases.
This requires workarounds in tooling and is a nuisance for alternative
Python implementations.

Having this guarantee also benefits implementers of CPython in the long
term, as the current behaviour is not obvious and has some odd corner
cases.

Rationale

In order to guarantee that line events are generated when expected, the
co_lnotab attribute, in its current form, can no longer be the source of
truth for line number information.

Rather than attempt to fix the co_lnotab attribute, a new method
co_lines() will be added, which returns an iterator over bytecode
offsets and source code lines.

Ensuring that the bytecode is annotated correctly to enable accurate
line number information means that some bytecodes must be marked as
artificial, and not have a line number.

Some care must be taken not to break existing tooling. To minimize
breakage, the co_lnotab attribute will be retained, but lazily generated
on demand.

Specification

Line events and the f_lineno attribute should act as an experienced
Python user would expect in all cases.

Tracing

Tracing generates events for calls, returns, exceptions, lines of source
code executed, and, under some circumstances, instructions executed.

Only line events are covered by this PEP.

When tracing is turned on, line events will be generated when:

-   A new line of source code is reached.
-   A backwards jump occurs, even if it jumps to the same line, as may
    happen in list comprehensions.

Additionally, line events will never be generated for source code lines
that are not executed.

What is considered to be code for the purposes of tracing

All expressions and parts of expressions are considered to be executable
code.

In general, all statements are also considered to be executable code.
However, when a statement is spread over several lines, we must consider
which parts of a statement are considered to be executable code.

Statements are made up of keywords and expressions. Not all keywords
have a direct runtime effect, so not all keywords are considered to be
executable code. For example, else, is a necessary part of an if
statement, but there is no runtime effect associated with an else.

For the purposes of tracing, the following keywords will not be
considered to be executable code:

-   del -- The expression to be deleted is treated as the executable
    code.
-   else -- No runtime effect
-   finally -- No runtime effect
-   global -- Purely declarative
-   nonlocal -- Purely declarative

All other keywords are considered to be executable code.

Example event sequences

In the following examples, events are listed as "name", f_lineno pairs.

The code

    1.     global x
    2.     x = a

generates the following event:

    "line" 2

The code

    1.     try:
    2.        pass
    3.     finally:
    4.        pass

generates the following events:

    "line" 1
    "line" 2
    "line" 4

The code

    1.      for (
    2.          x) in [1]:
    3.          pass
    4.      return

generates the following events:

    "line" 2       # evaluate [1]
    "line" 1       # for
    "line" 2       # store to x
    "line" 3       # pass
    "line" 1       # for
    "line" 4       # return
    "return" 1

The