##// END OF EJS Templates
fix release instructions
Matthias Bussonnier -
Show More
@@ -1,321 +1,321 b''
1 .. _core_developer_guide:
1 .. _core_developer_guide:
2
2
3 =================================
3 =================================
4 Guide for IPython core Developers
4 Guide for IPython core Developers
5 =================================
5 =================================
6
6
7 This guide documents the development of IPython itself. Alternatively,
7 This guide documents the development of IPython itself. Alternatively,
8 developers of third party tools and libraries that use IPython should see the
8 developers of third party tools and libraries that use IPython should see the
9 :doc:`../development/index`.
9 :doc:`../development/index`.
10
10
11
11
12 For instructions on how to make a developer install see :ref:`devinstall`.
12 For instructions on how to make a developer install see :ref:`devinstall`.
13
13
14 Backporting Pull requests
14 Backporting Pull requests
15 -------------------------
15 -------------------------
16
16
17 All pull requests should usually be made against ``master``, if a Pull Request
17 All pull requests should usually be made against ``master``, if a Pull Request
18 need to be backported to an earlier release; then it should be tagged with the
18 need to be backported to an earlier release; then it should be tagged with the
19 correct ``milestone``.
19 correct ``milestone``.
20
20
21 If you tag a pull request with a milestone **before** merging the pull request,
21 If you tag a pull request with a milestone **before** merging the pull request,
22 and the base ref is `master`, then our backport bot should automatically create
22 and the base ref is `master`, then our backport bot should automatically create
23 a corresponding pull-request that backport on the correct branch.
23 a corresponding pull-request that backport on the correct branch.
24
24
25 If you are an admin on the IPython repository you can also just mention the
25 If you are an admin on the IPython repository you can also just mention the
26 **backport bot** to do the work for you. The bot is evolving so instructions may
26 **backport bot** to do the work for you. The bot is evolving so instructions may
27 be different. At the time of this writing you can use::
27 be different. At the time of this writing you can use::
28
28
29 @meeseeksdev[bot] backport [to <branchname>]
29 @meeseeksdev[bot] backport [to <branchname>]
30
30
31 The bot will attempt to backport the current pull-request and issue a PR if
31 The bot will attempt to backport the current pull-request and issue a PR if
32 possible. If the milestone is set on the issue you can omit the branch to
32 possible. If the milestone is set on the issue you can omit the branch to
33 backport to.
33 backport to.
34
34
35 .. note::
35 .. note::
36
36
37 The ``@`` and ``[dev]`` when mentioning the bot should be optional and can
37 The ``@`` and ``[bot]`` when mentioning the bot should be optional and can
38 be omitted.
38 be omitted.
39
39
40 If the pull request cannot be automatically backported, the bot should tell you
40 If the pull request cannot be automatically backported, the bot should tell you
41 so on the PR and apply a "Need manual backport" tag to the origin PR.
41 so on the PR and apply a "Need manual backport" tag to the origin PR.
42
42
43
43
44 Backport with ghpro
44 Backport with ghpro
45 -------------------
45 -------------------
46
46
47 We can also use `ghpro <https://pypi.python.org/pypi/ghpro>`
47 We can also use `ghpro <https://pypi.python.org/pypi/ghpro>`_
48 to automatically list and apply the PR on other branches. For example:
48 to automatically list and apply the PR on other branches. For example:
49
49
50 .. code-block:: bash
50 .. code-block:: bash
51
51
52 $ backport-pr todo --milestone 5.2
52 $ backport-pr todo --milestone 5.2
53 [...snip..]
53 [...snip..]
54 The following PRs have been backported
54 The following PRs have been backported
55 9848
55 9848
56 9851
56 9851
57 9953
57 9953
58 9955
58 9955
59 The following PRs should be backported:
59 The following PRs should be backported:
60 9417
60 9417
61 9863
61 9863
62 9925
62 9925
63 9947
63 9947
64
64
65 $ backport-pr apply 5.x 9947
65 $ backport-pr apply 5.x 9947
66 [...snip...]
66 [...snip...]
67
67
68
68
69 .. _release_process:
69 .. _release_process:
70
70
71 =======================
71 =======================
72 IPython release process
72 IPython release process
73 =======================
73 =======================
74
74
75 This document contains the process that is used to create an IPython release.
75 This document contains the process that is used to create an IPython release.
76
76
77 Conveniently, the ``release`` script in the ``tools`` directory of the ``IPython``
77 Conveniently, the ``release`` script in the ``tools`` directory of the ``IPython``
78 repository automates most of the release process. This document serves as a
78 repository automates most of the release process. This document serves as a
79 handy reminder and checklist for the release manager.
79 handy reminder and checklist for the release manager.
80
80
81 During the release process, you might need the extra following dependencies:
81 During the release process, you might need the extra following dependencies:
82
82
83 - ``keyring`` to access your GitHub authentication tokens
83 - ``keyring`` to access your GitHub authentication tokens
84 - ``graphviz`` to generate some graphs in the documentation
84 - ``graphviz`` to generate some graphs in the documentation
85 - ``ghpro`` to generate the stats
85 - ``ghpro`` to generate the stats
86
86
87 Make sure you have all the required dependencies to run the tests as well.
87 Make sure you have all the required dependencies to run the tests as well.
88
88
89
89
90 1. Set Environment variables
90 1. Set Environment variables
91 ----------------------------
91 ----------------------------
92
92
93 Set environment variables to document previous release tag, current
93 Set environment variables to document previous release tag, current
94 release milestone, current release version, and git tag.
94 release milestone, current release version, and git tag.
95
95
96 These variables may be used later to copy/paste as answers to the script
96 These variables may be used later to copy/paste as answers to the script
97 questions instead of typing the appropriate command when the time comes. These
97 questions instead of typing the appropriate command when the time comes. These
98 variables are not used by the scripts directly; therefore, there is no need to
98 variables are not used by the scripts directly; therefore, there is no need to
99 ``export`` them. The format for bash is as follows, but note that these values
99 ``export`` them. The format for bash is as follows, but note that these values
100 are just an example valid only for the 5.0 release; you'll need to update them
100 are just an example valid only for the 5.0 release; you'll need to update them
101 for the release you are actually making::
101 for the release you are actually making::
102
102
103 PREV_RELEASE=4.2.1
103 PREV_RELEASE=4.2.1
104 MILESTONE=5.0
104 MILESTONE=5.0
105 VERSION=5.0.0
105 VERSION=5.0.0
106 BRANCH=master
106 BRANCH=master
107
107
108
108
109 2. Create GitHub stats and finish release note
109 2. Create GitHub stats and finish release note
110 ----------------------------------------------
110 ----------------------------------------------
111
111
112 .. note::
112 .. note::
113
113
114 This step is optional if making a Beta or RC release.
114 This step is optional if making a Beta or RC release.
115
115
116 .. note::
116 .. note::
117
117
118 Before generating the GitHub stats, verify that all closed issues and pull
118 Before generating the GitHub stats, verify that all closed issues and pull
119 requests have `appropriate milestones
119 requests have `appropriate milestones
120 <https://github.com/ipython/ipython/wiki/Dev:-GitHub-workflow#milestones>`_.
120 <https://github.com/ipython/ipython/wiki/Dev:-GitHub-workflow#milestones>`_.
121 `This search
121 `This search
122 <https://github.com/ipython/ipython/issues?q=is%3Aclosed+no%3Amilestone+is%3Aissue>`_
122 <https://github.com/ipython/ipython/issues?q=is%3Aclosed+no%3Amilestone+is%3Aissue>`_
123 should return no results before creating the GitHub stats.
123 should return no results before creating the GitHub stats.
124
124
125 If a major release:
125 If a major release:
126
126
127 - merge any pull request notes into what's new::
127 - merge any pull request notes into what's new::
128
128
129 python tools/update_whatsnew.py
129 python tools/update_whatsnew.py
130
130
131 - update ``docs/source/whatsnew/development.rst``, to ensure it covers
131 - update ``docs/source/whatsnew/development.rst``, to ensure it covers
132 the major release features
132 the major release features
133
133
134 - move the contents of ``development.rst`` to ``versionX.rst`` where ``X`` is
134 - move the contents of ``development.rst`` to ``versionX.rst`` where ``X`` is
135 the numerical release version
135 the numerical release version
136
136
137 - generate summary of GitHub contributions, which can be done with::
137 - generate summary of GitHub contributions, which can be done with::
138
138
139 python tools/github_stats.py --milestone $MILESTONE > stats.rst
139 python tools/github_stats.py --milestone $MILESTONE > stats.rst
140
140
141 which may need some manual cleanup of ``stats.rst``. Add the cleaned
141 which may need some manual cleanup of ``stats.rst``. Add the cleaned
142 ``stats.rst`` results to ``docs/source/whatsnew/github-stats-X.rst``
142 ``stats.rst`` results to ``docs/source/whatsnew/github-stats-X.rst``
143 where ``X`` is the numerical release version (don't forget to add it to
143 where ``X`` is the numerical release version (don't forget to add it to
144 the git repository as well). If creating a major release, make a new
144 the git repository as well). If creating a major release, make a new
145 ``github-stats-X.rst`` file; if creating a minor release, the content
145 ``github-stats-X.rst`` file; if creating a minor release, the content
146 from ``stats.rst`` may simply be added to the top of an existing
146 from ``stats.rst`` may simply be added to the top of an existing
147 ``github-stats-X.rst`` file.
147 ``github-stats-X.rst`` file.
148
148
149 - Edit ``docs/source/whatsnew/index.rst`` to list the new ``github-stats-X``
149 - Edit ``docs/source/whatsnew/index.rst`` to list the new ``github-stats-X``
150 file you just created.
150 file you just created.
151
151
152 - Remove temporarily the first entry called ``development`` (you'll need to
152 - Remove temporarily the first entry called ``development`` (you'll need to
153 add it back after release).
153 add it back after release).
154
154
155 Make sure that the stats file has a header or it won't be rendered in
155 Make sure that the stats file has a header or it won't be rendered in
156 the final documentation.
156 the final documentation.
157
157
158 To find duplicates and update `.mailmap`, use::
158 To find duplicates and update `.mailmap`, use::
159
159
160 git log --format="%aN <%aE>" $PREV_RELEASE... | sort -u -f
160 git log --format="%aN <%aE>" $PREV_RELEASE... | sort -u -f
161
161
162 If a minor release you might need to do some of the above points manually, and
162 If a minor release you might need to do some of the above points manually, and
163 forward port the changes.
163 forward port the changes.
164
164
165 3. Make sure the repository is clean
165 3. Make sure the repository is clean
166 ------------------------------------
166 ------------------------------------
167
167
168 of any file that could be problematic.
168 of any file that could be problematic.
169 Remove all non-tracked files with:
169 Remove all non-tracked files with:
170
170
171 .. code::
171 .. code::
172
172
173 git clean -xfdi
173 git clean -xfdi
174
174
175 This will ask for confirmation before removing all untracked files. Make
175 This will ask for confirmation before removing all untracked files. Make
176 sure the ``dist/`` folder is clean to avoid any stale builds from
176 sure the ``dist/`` folder is clean to avoid any stale builds from
177 previous build attempts.
177 previous build attempts.
178
178
179
179
180 4. Update the release version number
180 4. Update the release version number
181 ------------------------------------
181 ------------------------------------
182
182
183 Edit ``IPython/core/release.py`` to have the current version.
183 Edit ``IPython/core/release.py`` to have the current version.
184
184
185 in particular, update version number and ``_version_extra`` content in
185 in particular, update version number and ``_version_extra`` content in
186 ``IPython/core/release.py``.
186 ``IPython/core/release.py``.
187
187
188 Step 5 will validate your changes automatically, but you might still want to
188 Step 5 will validate your changes automatically, but you might still want to
189 make sure the version number matches pep440.
189 make sure the version number matches pep440.
190
190
191 In particular, ``rc`` and ``beta`` are not separated by ``.`` or the ``sdist``
191 In particular, ``rc`` and ``beta`` are not separated by ``.`` or the ``sdist``
192 and ``bdist`` will appear as different releases. For example, a valid version
192 and ``bdist`` will appear as different releases. For example, a valid version
193 number for a release candidate (rc) release is: ``1.3rc1``. Notice that there
193 number for a release candidate (rc) release is: ``1.3rc1``. Notice that there
194 is no separator between the '3' and the 'r'. Check the environment variable
194 is no separator between the '3' and the 'r'. Check the environment variable
195 ``$VERSION`` as well.
195 ``$VERSION`` as well.
196
196
197 You will likely just have to modify/comment/uncomment one of the lines setting
197 You will likely just have to modify/comment/uncomment one of the lines setting
198 ``_version_extra``
198 ``_version_extra``
199
199
200
200
201 5. Run the `tools/build_release` script
201 5. Run the `tools/build_release` script
202 ---------------------------------------
202 ---------------------------------------
203
203
204 Running ``tools/build_release`` does all the file checking and building that
204 Running ``tools/build_release`` does all the file checking and building that
205 the real release script will do. This makes test installations, checks that
205 the real release script will do. This makes test installations, checks that
206 the build procedure runs OK, and tests other steps in the release process.
206 the build procedure runs OK, and tests other steps in the release process.
207
207
208 The ``build_release`` script will in particular verify that the version number
208 The ``build_release`` script will in particular verify that the version number
209 match PEP 440, in order to avoid surprise at the time of build upload.
209 match PEP 440, in order to avoid surprise at the time of build upload.
210
210
211 We encourage creating a test build of the docs as well.
211 We encourage creating a test build of the docs as well.
212
212
213 6. Create and push the new tag
213 6. Create and push the new tag
214 ------------------------------
214 ------------------------------
215
215
216 Commit the changes to release.py::
216 Commit the changes to release.py::
217
217
218 git commit -am "release $VERSION"
218 git commit -am "release $VERSION"
219 git push origin $BRANCH
219 git push origin $BRANCH
220
220
221 Create and push the tag::
221 Create and push the tag::
222
222
223 git tag -am "release $VERSION" "$VERSION"
223 git tag -am "release $VERSION" "$VERSION"
224 git push origin --tags
224 git push origin --tags
225
225
226 Update release.py back to ``x.y-dev`` or ``x.y-maint``, and re-add the
226 Update release.py back to ``x.y-dev`` or ``x.y-maint``, and re-add the
227 ``development`` entry in ``docs/source/whatsnew/index.rst`` and push::
227 ``development`` entry in ``docs/source/whatsnew/index.rst`` and push::
228
228
229 git commit -am "back to development"
229 git commit -am "back to development"
230 git push origin $BRANCH
230 git push origin $BRANCH
231
231
232 Now checkout the tag we just made::
232 Now checkout the tag we just made::
233
233
234 git checkout $VERSION
234 git checkout $VERSION
235
235
236 7. Run the release script
236 7. Run the release script
237 -------------------------
237 -------------------------
238
238
239 Run the ``release`` script, this step requires having a current wheel, Python
239 Run the ``release`` script, this step requires having a current wheel, Python
240 >=3.4 and Python 2.7.::
240 >=3.4 and Python 2.7.::
241
241
242 ./tools/release
242 ./tools/release
243
243
244 This makes the tarballs and wheels, and puts them under the ``dist/``
244 This makes the tarballs and wheels, and puts them under the ``dist/``
245 folder. Be sure to test the ``wheels`` and the ``sdist`` locally before
245 folder. Be sure to test the ``wheels`` and the ``sdist`` locally before
246 uploading them to PyPI. We do not use an universal wheel as each wheel
246 uploading them to PyPI. We do not use an universal wheel as each wheel
247 installs an ``ipython2`` or ``ipython3`` script, depending on the version of
247 installs an ``ipython2`` or ``ipython3`` script, depending on the version of
248 Python it is built for. Using an universal wheel would prevent this.
248 Python it is built for. Using an universal wheel would prevent this.
249
249
250 Use the following to actually upload the result of the build::
250 Use the following to actually upload the result of the build::
251
251
252 ./tools/release upload
252 ./tools/release upload
253
253
254 It should posts them to ``archive.ipython.org`` and to PyPI.
254 It should posts them to ``archive.ipython.org`` and to PyPI.
255
255
256 PyPI/Warehouse will automatically hide previous releases. If you are uploading
256 PyPI/Warehouse will automatically hide previous releases. If you are uploading
257 a non-stable version, make sure to log-in to PyPI and un-hide previous version.
257 a non-stable version, make sure to log-in to PyPI and un-hide previous version.
258
258
259
259
260 8. Draft a short release announcement
260 8. Draft a short release announcement
261 -------------------------------------
261 -------------------------------------
262
262
263 The announcement should include:
263 The announcement should include:
264
264
265 - release highlights
265 - release highlights
266 - a link to the html version of the *What's new* section of the documentation
266 - a link to the html version of the *What's new* section of the documentation
267 - a link to upgrade or installation tips (if necessary)
267 - a link to upgrade or installation tips (if necessary)
268
268
269 Post the announcement to the mailing list and or blog, and link from Twitter.
269 Post the announcement to the mailing list and or blog, and link from Twitter.
270
270
271 .. note::
271 .. note::
272
272
273 If you are doing a RC or Beta, you can likely skip the next steps.
273 If you are doing a RC or Beta, you can likely skip the next steps.
274
274
275 9. Update milestones on GitHub
275 9. Update milestones on GitHub
276 -------------------------------
276 -------------------------------
277
277
278 These steps will bring milestones up to date:
278 These steps will bring milestones up to date:
279
279
280 - close the just released milestone
280 - close the just released milestone
281 - open a new milestone for the next release (x, y+1), if the milestone doesn't
281 - open a new milestone for the next release (x, y+1), if the milestone doesn't
282 exist already
282 exist already
283
283
284 10. Update the IPython website
284 10. Update the IPython website
285 ------------------------------
285 ------------------------------
286
286
287 The IPython website should document the new release:
287 The IPython website should document the new release:
288
288
289 - add release announcement (news, announcements)
289 - add release announcement (news, announcements)
290 - update current version and download links
290 - update current version and download links
291 - update links on the documentation page (especially if a major release)
291 - update links on the documentation page (especially if a major release)
292
292
293 11. Update readthedocs
293 11. Update readthedocs
294 ----------------------
294 ----------------------
295
295
296 Make sure to update readthedocs and set the latest tag as stable, as well as
296 Make sure to update readthedocs and set the latest tag as stable, as well as
297 checking that previous release is still building under its own tag.
297 checking that previous release is still building under its own tag.
298
298
299 12. Update the Conda-Forge feedstock
299 12. Update the Conda-Forge feedstock
300 ------------------------------------
300 ------------------------------------
301
301
302 Follow the instructions on `the repository <https://github.com/conda-forge/ipython-feedstock>`_
302 Follow the instructions on `the repository <https://github.com/conda-forge/ipython-feedstock>`_
303
303
304 13. Celebrate!
304 13. Celebrate!
305 --------------
305 --------------
306
306
307 Celebrate the release and please thank the contributors for their work. Great
307 Celebrate the release and please thank the contributors for their work. Great
308 job!
308 job!
309
309
310
310
311
311
312 Old Documentation
312 Old Documentation
313 =================
313 =================
314
314
315 Out of date documentation is still available and have been kept for archival purposes.
315 Out of date documentation is still available and have been kept for archival purposes.
316
316
317 .. note::
317 .. note::
318
318
319 Developers documentation used to be on the IPython wiki, but are now out of
319 Developers documentation used to be on the IPython wiki, but are now out of
320 date. The wiki is though still available for historical reasons: `Old IPython
320 date. The wiki is though still available for historical reasons: `Old IPython
321 GitHub Wiki. <https://github.com/ipython/ipython/wiki/Dev:-Index>`_
321 GitHub Wiki. <https://github.com/ipython/ipython/wiki/Dev:-Index>`_
General Comments 0
You need to be logged in to leave comments. Login now