Merge branch 'main' into main

CONTRIBUTING.md

@@ -1,90 +1,118
 ## Triaging Issues
 
 On the IPython repository,  we strive to trust users and give them responsibility.
 By using one of our bots, any user can close issues or add/remove
 labels by mentioning the bot and asking it to do things on your behalf.
 
 To close an issue (or PR), even if you did not create it, use the following:
 
 > @meeseeksdev close
 
 This command can be in the middle of another comment, but must start on its
 own line. 
 
 To add labels to an issue, ask the bot to `tag` with a comma-separated list of
 tags to add:
 
 > @meeseeksdev tag windows, documentation
 
 Only already pre-created tags can be added.  So far, the list is limited to:
 `async/await`, `backported`, `help wanted`, `documentation`, `notebook`,
 `tab-completion`, `windows`
 
 To remove a label, use the `untag` command:
 
 > @meeseeksdev untag windows, documentation
 
 We'll be adding additional capabilities for the bot and will share them here
 when they are ready to be used.
 
 ## Opening an Issue
 
 When opening a new Issue, please take the following steps:
 
 1. Search GitHub and/or Google for your issue to avoid duplicate reports.
    Keyword searches for your error messages are most helpful.
 2. If possible, try updating to main and reproducing your issue,
    because we may have already fixed it.
 3. Try to include a minimal reproducible test case.
 4. Include relevant system information.  Start with the output of:
 
         python -c "import IPython; print(IPython.sys_info())"
 
    And include any relevant package versions, depending on the issue, such as
    matplotlib, numpy, Qt, Qt bindings (PyQt/PySide), tornado, web browser, etc.
 
 ## Pull Requests
 
 Some guidelines on contributing to IPython:
 
 * All work is submitted via Pull Requests.
 * Pull Requests can be submitted as soon as there is code worth discussing.
   Pull Requests track the branch, so you can continue to work after the PR is submitted.
   Review and discussion can begin well before the work is complete,
   and the more discussion the better.
   The worst case is that the PR is closed.
 * Pull Requests should generally be made against main
 * Pull Requests should be tested, if feasible:
     - bugfixes should include regression tests.
     - new behavior should at least get minimal exercise.
 * New features and backwards-incompatible changes should be documented by adding
   a new file to the [pr](docs/source/whatsnew/pr) directory, see [the README.md
   there](docs/source/whatsnew/pr/README.md) for details.
 * Don't make 'cleanup' pull requests just to change code style.
   We don't follow any style guide strictly, and we consider formatting changes
   unnecessary noise.
   If you're making functional changes, you can clean up the specific pieces of
   code you're working on.
 
-[Travis](http://travis-ci.org/#!/ipython/ipython) does a pretty good job testing
-IPython and Pull Requests, but it may make sense to manually perform tests,
+[GitHub Actions](https://github.com/ipython/ipython/actions/workflows/test.yml) does
+a pretty good job testing IPython and Pull Requests,
+but it may make sense to manually perform tests,
 particularly for PRs that affect `IPython.parallel` or Windows.
 
 For more detailed information, see our [GitHub Workflow](https://github.com/ipython/ipython/wiki/Dev:-GitHub-workflow).
 
 ## Running Tests
 
 All the tests can be run by using
 ```shell
 pytest
 ```
 
 All the tests for a single module (for example **test_alias**) can be run by using the fully qualified path to the module.
 ```shell
 pytest IPython/core/tests/test_alias.py
 ```
 
 Only a single test (for example **test_alias_lifecycle**) within a single file can be run by adding the specific test after a `::` at the end:
 ```shell
 pytest IPython/core/tests/test_alias.py::test_alias_lifecycle
 ```
+
+## Code style
+
+* Before committing, run `darker -r 60625f241f298b5039cb2debc365db38aa7bb522 <file path>` to apply selective `black` formatting on modified regions using [darker](https://github.com/akaihola/darker).
+* For newly added modules or refactors, please enable static typing analysis with `mypy` for the modified module by adding the file path in [`mypy.yml`](https://github.com/ipython/ipython/blob/main/.github/workflows/mypy.yml) workflow.
+* As described in the pull requests section, please avoid excessive formatting changes; if a formatting-only commit is necessary, consider adding its hash to [`.git-blame-ignore-revs`](https://github.com/ipython/ipython/blob/main/.git-blame-ignore-revs) file.
+
+## Documentation
+
+Sphinx documentation can be built locally using standard sphinx `make` commands. To build HTML documentation from the root of the project, execute:
+
+```shell
+pip install -r docs/requirements.txt   # only needed once
+make -C docs/ html SPHINXOPTS="-W"
+```
+
+To force update of the API documentation, precede the `make` command with:
+
+```shell
+python3 docs/autogen_api.py
+```
+
+Similarly, to force-update the configuration, run:
+
+```shell
+python3 docs/autogen_config.py
+```

README.rst

@@ -1,174 +1,174
 .. image:: https://codecov.io/github/ipython/ipython/coverage.svg?branch=main
     :target: https://codecov.io/github/ipython/ipython?branch=main
 
 .. image:: https://img.shields.io/pypi/v/IPython.svg
     :target: https://pypi.python.org/pypi/ipython
 
 .. image:: https://github.com/ipython/ipython/actions/workflows/test.yml/badge.svg
-    :target: https://github.com/ipython/ipython/actions/workflows/test.yml)
+    :target: https://github.com/ipython/ipython/actions/workflows/test.yml
 
 .. image:: https://www.codetriage.com/ipython/ipython/badges/users.svg
     :target: https://www.codetriage.com/ipython/ipython/
 
 .. image:: https://raster.shields.io/badge/Follows-NEP29-brightgreen.png
     :target: https://numpy.org/neps/nep-0029-deprecation_policy.html
 
 .. image:: https://tidelift.com/badges/package/pypi/ipython?style=flat
     :target: https://tidelift.com/subscription/pkg/pypi-ipython
 
 
 ===========================================
  IPython: Productive Interactive Computing
 ===========================================
 
 Overview
 ========
 
 Welcome to IPython.  Our full documentation is available on `ipython.readthedocs.io
 <https://ipython.readthedocs.io/en/stable/>`_ and contains information on how to install, use, and
 contribute to the project.
 IPython (Interactive Python) is a command shell for interactive computing in multiple programming languages, originally developed for the Python programming language, that offers introspection, rich media, shell syntax, tab completion, and history.
 
 **IPython versions and Python Support**
 
 Starting with IPython 7.10, IPython follows `NEP 29 <https://numpy.org/neps/nep-0029-deprecation_policy.html>`_
 
 **IPython 7.17+** requires Python version 3.7 and above.
 
 **IPython 7.10+** requires Python version 3.6 and above.
 
 **IPython 7.0** requires Python version 3.5 and above.
 
 **IPython 6.x** requires Python version 3.3 and above.
 
 **IPython 5.x LTS** is the compatible release for Python 2.7.
 If you require Python 2 support, you **must** use IPython 5.x LTS. Please
 update your project configurations and requirements as necessary.
 
 
 The Notebook, Qt console and a number of other pieces are now parts of *Jupyter*.
 See the `Jupyter installation docs <https://jupyter.readthedocs.io/en/latest/install.html>`__
 if you want to use these.
 
 Main features of IPython
 ========================
 Comprehensive object introspection.
 
 Input history, persistent across sessions.
 
 Caching of output results during a session with automatically generated references.
 
 Extensible tab completion, with support by default for completion of python variables and keywords, filenames and function keywords.
 
 Extensible system of ‘magic’ commands for controlling the environment and performing many tasks related to IPython or the operating system.
 
 A rich configuration system with easy switching between different setups (simpler than changing $PYTHONSTARTUP environment variables every time).
 
 Session logging and reloading.
 
 Extensible syntax processing for special purpose situations.
 
 Access to the system shell with user-extensible alias system.
 
 Easily embeddable in other Python programs and GUIs.
 
 Integrated access to the pdb debugger and the Python profiler.
 
 
 Development and Instant running
 ===============================
 
 You can find the latest version of the development documentation on `readthedocs
 <https://ipython.readthedocs.io/en/latest/>`_.
 
 You can run IPython from this directory without even installing it system-wide
 by typing at the terminal::
 
    $ python -m IPython
 
 Or see the `development installation docs
 <https://ipython.readthedocs.io/en/latest/install/install.html#installing-the-development-version>`_
 for the latest revision on read the docs.
 
 Documentation and installation instructions for older version of IPython can be
 found on the `IPython website <https://ipython.org/documentation.html>`_
 
 
 
 IPython requires Python version 3 or above
 ==========================================
 
 Starting with version 6.0, IPython does not support Python 2.7, 3.0, 3.1, or
 3.2.
 
 For a version compatible with Python 2.7, please install the 5.x LTS Long Term
 Support version.
 
 If you are encountering this error message you are likely trying to install or
 use IPython from source. You need to checkout the remote 5.x branch. If you are
 using git the following should work::
 
   $ git fetch origin
   $ git checkout 5.x
 
 If you encounter this error message with a regular install of IPython, then you
 likely need to update your package manager, for example if you are using `pip`
 check the version of pip with::
 
   $ pip --version
 
 You will need to update pip to the version 9.0.1 or greater. If you are not using
 pip, please inquiry with the maintainers of the package for your package
 manager.
 
 For more information see one of our blog posts:
 
     https://blog.jupyter.org/release-of-ipython-5-0-8ce60b8d2e8e
 
 As well as the following Pull-Request for discussion:
 
     https://github.com/ipython/ipython/pull/9900
 
 This error does also occur if you are invoking ``setup.py`` directly – which you
 should not – or are using ``easy_install`` If this is the case, use ``pip
 install .`` instead of ``setup.py install`` , and ``pip install -e .`` instead
 of ``setup.py develop`` If you are depending on IPython as a dependency you may
 also want to have a conditional dependency on IPython depending on the Python
 version::
 
     install_req = ['ipython']
     if sys.version_info[0] < 3 and 'bdist_wheel' not in sys.argv:
         install_req.remove('ipython')
         install_req.append('ipython<6')
 
     setup(
         ...
         install_requires=install_req
     )
 
 Alternatives to IPython
 =======================
 
 IPython may not be to your taste; if that's the case there might be similar
 project that you might want to use:
 
 - The classic Python REPL.
 - `bpython <https://bpython-interpreter.org/>`_
 - `mypython <https://www.asmeurer.com/mypython/>`_
 - `ptpython and ptipython <https://pypi.org/project/ptpython/>`_
 - `Xonsh <https://xon.sh/>`_
 
 Ignoring commits with git blame.ignoreRevsFile
 ==============================================
 
 As of git 2.23, it is possible to make formatting changes without breaking
 ``git blame``. See the `git documentation
 <https://git-scm.com/docs/git-config#Documentation/git-config.txt-blameignoreRevsFile>`_
 for more details.
 
 To use this feature you must:
 
 - Install git >= 2.23
 - Configure your local git repo by running:
    - POSIX: ``tools\configure-git-blame-ignore-revs.sh``
    - Windows:  ``tools\configure-git-blame-ignore-revs.bat``

tools/release_helper.sh

@@ -1,251 +1,252
 # Simple tool to help for release
 # when releasing with bash, simple source it to get asked questions. 
 
 # misc check before starting
 
 python -c 'import keyring'
 python -c 'import twine'
 python -c 'import sphinx'
 python -c 'import sphinx_rtd_theme'
 python -c 'import pytest'
+python -c 'import build'
 
 
 BLACK=$(tput setaf 1)
 RED=$(tput setaf 1)
 GREEN=$(tput setaf 2)
 YELLOW=$(tput setaf 3)
 BLUE=$(tput setaf 4)
 MAGENTA=$(tput setaf 5)
 CYAN=$(tput setaf 6)
 WHITE=$(tput setaf 7)
 NOR=$(tput sgr0)
 
 
 echo "Will use $BLUE'$EDITOR'$NOR to edit files when necessary"
 echo -n "PREV_RELEASE (X.y.z) [$PREV_RELEASE]: "
 read input
 PREV_RELEASE=${input:-$PREV_RELEASE}
 echo -n "MILESTONE (X.y) [$MILESTONE]: "
 read input
 MILESTONE=${input:-$MILESTONE}
 echo -n "VERSION (X.y.z) [$VERSION]:"
 read input
 VERSION=${input:-$VERSION}
 echo -n "BRANCH (main|X.y) [$BRANCH]:"
 read input
 BRANCH=${input:-$BRANCH}
 
 ask_section(){
     echo
     echo $BLUE"$1"$NOR
     echo -n $GREEN"Press Enter to continue, S to skip: "$NOR
     if [ "$ZSH_NAME" = "zsh" ] ; then
         read -k1 value
         value=${value%$'\n'}
     else
        read -n1 value
     fi
     if [ -z "$value" ] || [ $value = 'y' ]; then
         return 0
     fi
     return 1
 }
 
 
 maybe_edit(){
     echo
     echo $BLUE"$1"$NOR 
     echo -n $GREEN"Press ${BLUE}e$GREEN to Edit ${BLUE}$1$GREEN, any other keys to skip: "$NOR
     if [ "$ZSH_NAME" = "zsh" ] ; then
         read -k1 value
         value=${value%$'\n'}
     else
        read -n1 value
     fi
 
     echo 
     if [ $value = 'e' ]  ; then
         $=EDITOR $1
     fi
 }
 
 
 
 echo 
 if ask_section "Updating what's new with information from docs/source/whatsnew/pr"
 then
     python tools/update_whatsnew.py
 
     echo
     echo $BLUE"please move the contents of "docs/source/whatsnew/development.rst" to version-X.rst"$NOR
     echo $GREEN"Press enter to continue"$NOR
     read
 fi
 
 if ask_section "Gen Stats, and authors"
 then
 
     echo 
     echo $BLUE"here are all the authors that contributed to this release:"$NOR
     git log --format="%aN <%aE>" $PREV_RELEASE... | sort -u -f
 
     echo 
     echo $BLUE"If you see any duplicates cancel (Ctrl-C), then edit .mailmap."
     echo $GREEN"Press enter to continue:"$NOR
     read
 
     echo $BLUE"generating stats"$NOR
     python tools/github_stats.py --milestone $MILESTONE > stats.rst
 
     echo $BLUE"stats.rst files generated."$NOR
     echo $GREEN"Please merge it with the right file (github-stats-X.rst) and commit."$NOR
     echo $GREEN"press enter to continue."$NOR
     read
 
 fi
 
 if ask_section "Generate API difference (using frapuccino)"
 then
     echo $BLUE"Checking out $PREV_RELEASE"$NOR
     git checkout $PREV_RELEASE
     sleep 1
     echo $BLUE"Saving API to file $PREV_RELEASE"$NOR
     frappuccino IPython IPython.kernel IPython.lib IPython.qt IPython.lib.kernel IPython.html IPython.frontend IPython.external --save IPython-$PREV_RELEASE.json
     echo $BLUE"coming back to $BRANCH"$NOR
     git checkout $BRANCH
     sleep 1
     echo $BLUE"comparing ..."$NOR
     frappuccino IPython IPython.kernel IPython.lib --compare IPython-$PREV_RELEASE.json
     echo $GREEN"Use the above guideline to write an API changelog ..."$NOR
     echo $GREEN"Press any keys to continue"$NOR
     read
 fi
 
 echo "Cleaning repository"
 git clean -xfdi
 
 echo $GREEN"please update version number in ${RED}IPython/core/release.py${NOR} , Do not commit yet – we'll do it later."$NOR
 echo $GREEN"I tried ${RED}sed -i bkp -e '/Uncomment/s/^# //g' IPython/core/release.py${NOR}"
 sed -i bkp -e '/Uncomment/s/^# //g' IPython/core/release.py
 rm IPython/core/release.pybkp
 git diff | cat
 maybe_edit IPython/core/release.py
 
 echo $GREEN"Press enter to continue"$NOR
 read
 
 if ask_section "Build the documentation ?"
 then
     make html -C docs
     echo 
     echo $GREEN"Check the docs, press enter to continue"$NOR
     read
 
 fi
 
 if ask_section "Should we commit, tag, push... etc ? "
 then
    echo
    echo $BLUE"Let's commit : git commit -am \"release $VERSION\" -S"
    echo $GREEN"Press enter to commit"$NOR
    read
    git commit -am "release $VERSION" -S
    
    echo
    echo $BLUE"git push origin \$BRANCH ($BRANCH)?"$NOR
    echo $GREEN"Make sure you can push"$NOR
    echo $GREEN"Press enter to continue"$NOR
    read
    git push origin $BRANCH
    
    echo
    echo "Let's tag : git tag -am \"release $VERSION\" \"$VERSION\" -s"
    echo $GREEN"Press enter to tag commit"$NOR
    read
    git tag -am "release $VERSION" "$VERSION" -s
    
    echo
    echo $BLUE"And push the tag: git push origin \$VERSION ?"$NOR
    echo $GREEN"Press enter to continue"$NOR
    read
    git push origin $VERSION
    
    
    echo $GREEN"please update version number and back to .dev in ${RED}IPython/core/release.py"
    echo $GREEN"I tried ${RED}sed -i bkp -e '/Uncomment/s/^/# /g' IPython/core/release.py${NOR}"
    sed -i bkp -e '/Uncomment/s/^/# /g' IPython/core/release.py
    rm IPython/core/release.pybkp
    git diff | cat
    echo $GREEN"Please bump ${RED}the minor version number${NOR}"
    maybe_edit IPython/core/release.py
    echo ${BLUE}"Do not commit yet – we'll do it later."$NOR
 
    
    echo $GREEN"Press enter to continue"$NOR
    read
    
    echo
    echo "Let's commit : "$BLUE"git commit -am \"back to dev\""$NOR
    echo $GREEN"Press enter to commit"$NOR
    read
    git commit -am "back to dev"
 
    echo
    echo $BLUE"git push origin \$BRANCH ($BRANCH)?"$NOR
    echo $GREEN"Press enter to continue"$NOR
    read
    git push origin $BRANCH
 
    
    echo
    echo $BLUE"let's : git checkout $VERSION"$NOR
    echo $GREEN"Press enter to continue"$NOR
    read
    git checkout $VERSION
 fi
 
 if ask_section "Should we build and release ?"
 then
     
     echo $BLUE"going to set SOURCE_DATE_EPOCH"$NOR
     echo $BLUE'export SOURCE_DATE_EPOCH=$(git show -s --format=%ct HEAD)'$NOR
     echo $GREEN"Press enter to continue"$NOR
     read
 
     export SOURCE_DATE_EPOCH=$(git show -s --format=%ct HEAD)
 
     echo $BLUE"SOURCE_DATE_EPOCH set to $SOURCE_DATE_EPOCH"$NOR
     echo $GREEN"Press enter to continue"$NOR
     read
 
 
 
     echo
     echo $BLUE"Attempting to build package..."$NOR
 
     tools/release
 
 
     echo $RED'$ shasum -a 256 dist/*'
     shasum -a 256 dist/*
     echo $NOR
 
     echo $BLUE"We are going to rebuild, node the hash above, and compare them to the rebuild"$NOR
     echo $GREEN"Press enter to continue"$NOR
     read
 
     echo
     echo $BLUE"Attempting to build package..."$NOR
 
     tools/release
 
     echo $RED"Check the shasum for SOURCE_DATE_EPOCH=$SOURCE_DATE_EPOCH"
     echo $RED'$ shasum -a 256 dist/*'
     shasum -a 256 dist/*
     echo $NOR
 
     if ask_section "upload packages ?"
     then 
        tools/release upload
     fi
 fi