##// END OF EJS Templates
add info to sign releases
Matthias Bussonnier -
Show More
@@ -1,321 +1,327 b''
1 .. _core_developer_guide:
1 .. _core_developer_guide:
2
2
3 =================================
3 =================================
4 Guide for IPython core Developers
4 Guide for IPython core Developers
5 =================================
5 =================================
6
6
7 This guide documents the development of IPython itself. Alternatively,
7 This guide documents the development of IPython itself. Alternatively,
8 developers of third party tools and libraries that use IPython should see the
8 developers of third party tools and libraries that use IPython should see the
9 :doc:`../development/index`.
9 :doc:`../development/index`.
10
10
11
11
12 For instructions on how to make a developer install see :ref:`devinstall`.
12 For instructions on how to make a developer install see :ref:`devinstall`.
13
13
14 Backporting Pull requests
14 Backporting Pull requests
15 =========================
15 =========================
16
16
17 All pull requests should usually be made against ``master``, if a Pull Request
17 All pull requests should usually be made against ``master``, if a Pull Request
18 need to be backported to an earlier release; then it should be tagged with the
18 need to be backported to an earlier release; then it should be tagged with the
19 correct ``milestone``.
19 correct ``milestone``.
20
20
21 If you tag a pull request with a milestone **before** merging the pull request,
21 If you tag a pull request with a milestone **before** merging the pull request,
22 and the base ref is ``master``, then our backport bot should automatically create
22 and the base ref is ``master``, then our backport bot should automatically create
23 a corresponding pull-request that backport on the correct branch.
23 a corresponding pull-request that backport on the correct branch.
24
24
25 If you have write access to the IPython repository you can also just mention the
25 If you have write access to the IPython repository you can also just mention the
26 **backport bot** to do the work for you. The bot is evolving so instructions may
26 **backport bot** to do the work for you. The bot is evolving so instructions may
27 be different. At the time of this writing you can use::
27 be different. At the time of this writing you can use::
28
28
29 @meeseeksdev[bot] backport [to] <branchname>
29 @meeseeksdev[bot] backport [to] <branchname>
30
30
31 The bot will attempt to backport the current pull-request and issue a PR if
31 The bot will attempt to backport the current pull-request and issue a PR if
32 possible.
32 possible.
33
33
34 .. note::
34 .. note::
35
35
36 The ``@`` and ``[bot]`` when mentioning the bot should be optional and can
36 The ``@`` and ``[bot]`` when mentioning the bot should be optional and can
37 be omitted.
37 be omitted.
38
38
39 If the pull request cannot be automatically backported, the bot should tell you
39 If the pull request cannot be automatically backported, the bot should tell you
40 so on the PR and apply a "Need manual backport" tag to the origin PR.
40 so on the PR and apply a "Need manual backport" tag to the origin PR.
41
41
42
42
43 Backport with ghpro
43 Backport with ghpro
44 -------------------
44 -------------------
45
45
46 We can also use `ghpro <https://pypi.python.org/pypi/ghpro>`_
46 We can also use `ghpro <https://pypi.python.org/pypi/ghpro>`_
47 to automatically list and apply the PR on other branches. For example:
47 to automatically list and apply the PR on other branches. For example:
48
48
49 .. code-block:: bash
49 .. code-block:: bash
50
50
51 $ backport-pr todo --milestone 5.2
51 $ backport-pr todo --milestone 5.2
52 [...snip..]
52 [...snip..]
53 The following PRs have been backported
53 The following PRs have been backported
54 9848
54 9848
55 9851
55 9851
56 9953
56 9953
57 9955
57 9955
58 The following PRs should be backported:
58 The following PRs should be backported:
59 9417
59 9417
60 9863
60 9863
61 9925
61 9925
62 9947
62 9947
63
63
64 $ backport-pr apply 5.x 9947
64 $ backport-pr apply 5.x 9947
65 [...snip...]
65 [...snip...]
66
66
67
67
68 .. _release_process:
68 .. _release_process:
69
69
70 IPython release process
70 IPython release process
71 =======================
71 =======================
72
72
73 This document contains the process that is used to create an IPython release.
73 This document contains the process that is used to create an IPython release.
74
74
75 Conveniently, the ``release`` script in the ``tools`` directory of the ``IPython``
75 Conveniently, the ``release`` script in the ``tools`` directory of the ``IPython``
76 repository automates most of the release process. This document serves as a
76 repository automates most of the release process. This document serves as a
77 handy reminder and checklist for the release manager.
77 handy reminder and checklist for the release manager.
78
78
79 During the release process, you might need the extra following dependencies:
79 During the release process, you might need the extra following dependencies:
80
80
81 - ``keyring`` to access your GitHub authentication tokens
81 - ``keyring`` to access your GitHub authentication tokens
82 - ``graphviz`` to generate some graphs in the documentation
82 - ``graphviz`` to generate some graphs in the documentation
83 - ``ghpro`` to generate the stats
83 - ``ghpro`` to generate the stats
84
84
85 Make sure you have all the required dependencies to run the tests as well.
85 Make sure you have all the required dependencies to run the tests as well.
86
86
87
87
88 1. Set Environment variables
88 1. Set Environment variables
89 ----------------------------
89 ----------------------------
90
90
91 Set environment variables to document previous release tag, current
91 Set environment variables to document previous release tag, current
92 release milestone, current release version, and git tag.
92 release milestone, current release version, and git tag.
93
93
94 These variables may be used later to copy/paste as answers to the script
94 These variables may be used later to copy/paste as answers to the script
95 questions instead of typing the appropriate command when the time comes. These
95 questions instead of typing the appropriate command when the time comes. These
96 variables are not used by the scripts directly; therefore, there is no need to
96 variables are not used by the scripts directly; therefore, there is no need to
97 ``export`` them. The format for bash is as follows, but note that these values
97 ``export`` them. The format for bash is as follows, but note that these values
98 are just an example valid only for the 5.0 release; you'll need to update them
98 are just an example valid only for the 5.0 release; you'll need to update them
99 for the release you are actually making::
99 for the release you are actually making::
100
100
101 PREV_RELEASE=4.2.1
101 PREV_RELEASE=4.2.1
102 MILESTONE=5.0
102 MILESTONE=5.0
103 VERSION=5.0.0
103 VERSION=5.0.0
104 BRANCH=master
104 BRANCH=master
105
105
106
106
107 2. Create GitHub stats and finish release note
107 2. Create GitHub stats and finish release note
108 ----------------------------------------------
108 ----------------------------------------------
109
109
110 .. note::
110 .. note::
111
111
112 This step is optional if making a Beta or RC release.
112 This step is optional if making a Beta or RC release.
113
113
114 .. note::
114 .. note::
115
115
116 Before generating the GitHub stats, verify that all closed issues and pull
116 Before generating the GitHub stats, verify that all closed issues and pull
117 requests have `appropriate milestones
117 requests have `appropriate milestones
118 <https://github.com/ipython/ipython/wiki/Dev:-GitHub-workflow#milestones>`_.
118 <https://github.com/ipython/ipython/wiki/Dev:-GitHub-workflow#milestones>`_.
119 `This search
119 `This search
120 <https://github.com/ipython/ipython/issues?q=is%3Aclosed+no%3Amilestone+is%3Aissue>`_
120 <https://github.com/ipython/ipython/issues?q=is%3Aclosed+no%3Amilestone+is%3Aissue>`_
121 should return no results before creating the GitHub stats.
121 should return no results before creating the GitHub stats.
122
122
123 If a major release:
123 If a major release:
124
124
125 - merge any pull request notes into what's new::
125 - merge any pull request notes into what's new::
126
126
127 python tools/update_whatsnew.py
127 python tools/update_whatsnew.py
128
128
129 - update ``docs/source/whatsnew/development.rst``, to ensure it covers
129 - update ``docs/source/whatsnew/development.rst``, to ensure it covers
130 the major release features
130 the major release features
131
131
132 - move the contents of ``development.rst`` to ``versionX.rst`` where ``X`` is
132 - move the contents of ``development.rst`` to ``versionX.rst`` where ``X`` is
133 the numerical release version
133 the numerical release version
134
134
135 - generate summary of GitHub contributions, which can be done with::
135 - generate summary of GitHub contributions, which can be done with::
136
136
137 python tools/github_stats.py --milestone $MILESTONE > stats.rst
137 python tools/github_stats.py --milestone $MILESTONE > stats.rst
138
138
139 which may need some manual cleanup of ``stats.rst``. Add the cleaned
139 which may need some manual cleanup of ``stats.rst``. Add the cleaned
140 ``stats.rst`` results to ``docs/source/whatsnew/github-stats-X.rst``
140 ``stats.rst`` results to ``docs/source/whatsnew/github-stats-X.rst``
141 where ``X`` is the numerical release version (don't forget to add it to
141 where ``X`` is the numerical release version (don't forget to add it to
142 the git repository as well). If creating a major release, make a new
142 the git repository as well). If creating a major release, make a new
143 ``github-stats-X.rst`` file; if creating a minor release, the content
143 ``github-stats-X.rst`` file; if creating a minor release, the content
144 from ``stats.rst`` may simply be added to the top of an existing
144 from ``stats.rst`` may simply be added to the top of an existing
145 ``github-stats-X.rst`` file.
145 ``github-stats-X.rst`` file.
146
146
147 - Edit ``docs/source/whatsnew/index.rst`` to list the new ``github-stats-X``
147 - Edit ``docs/source/whatsnew/index.rst`` to list the new ``github-stats-X``
148 file you just created.
148 file you just created.
149
149
150 - You do not need to temporarily remove the first entry called
150 - You do not need to temporarily remove the first entry called
151 ``development``, nor re-add it after the release, it will automatically be
151 ``development``, nor re-add it after the release, it will automatically be
152 hidden when releasing a stable version of IPython (if ``_version_extra``
152 hidden when releasing a stable version of IPython (if ``_version_extra``
153 in ``release.py`` is an empty string.
153 in ``release.py`` is an empty string.
154
154
155 Make sure that the stats file has a header or it won't be rendered in
155 Make sure that the stats file has a header or it won't be rendered in
156 the final documentation.
156 the final documentation.
157
157
158 To find duplicates and update `.mailmap`, use::
158 To find duplicates and update `.mailmap`, use::
159
159
160 git log --format="%aN <%aE>" $PREV_RELEASE... | sort -u -f
160 git log --format="%aN <%aE>" $PREV_RELEASE... | sort -u -f
161
161
162 If a minor release you might need to do some of the above points manually, and
162 If a minor release you might need to do some of the above points manually, and
163 forward port the changes.
163 forward port the changes.
164
164
165 3. Make sure the repository is clean
165 3. Make sure the repository is clean
166 ------------------------------------
166 ------------------------------------
167
167
168 of any file that could be problematic.
168 of any file that could be problematic.
169 Remove all non-tracked files with:
169 Remove all non-tracked files with:
170
170
171 .. code::
171 .. code::
172
172
173 git clean -xfdi
173 git clean -xfdi
174
174
175 This will ask for confirmation before removing all untracked files. Make
175 This will ask for confirmation before removing all untracked files. Make
176 sure the ``dist/`` folder is clean to avoid any stale builds from
176 sure the ``dist/`` folder is clean to avoid any stale builds from
177 previous build attempts.
177 previous build attempts.
178
178
179
179
180 4. Update the release version number
180 4. Update the release version number
181 ------------------------------------
181 ------------------------------------
182
182
183 Edit ``IPython/core/release.py`` to have the current version.
183 Edit ``IPython/core/release.py`` to have the current version.
184
184
185 in particular, update version number and ``_version_extra`` content in
185 in particular, update version number and ``_version_extra`` content in
186 ``IPython/core/release.py``.
186 ``IPython/core/release.py``.
187
187
188 Step 5 will validate your changes automatically, but you might still want to
188 Step 5 will validate your changes automatically, but you might still want to
189 make sure the version number matches pep440.
189 make sure the version number matches pep440.
190
190
191 In particular, ``rc`` and ``beta`` are not separated by ``.`` or the ``sdist``
191 In particular, ``rc`` and ``beta`` are not separated by ``.`` or the ``sdist``
192 and ``bdist`` will appear as different releases. For example, a valid version
192 and ``bdist`` will appear as different releases. For example, a valid version
193 number for a release candidate (rc) release is: ``1.3rc1``. Notice that there
193 number for a release candidate (rc) release is: ``1.3rc1``. Notice that there
194 is no separator between the '3' and the 'r'. Check the environment variable
194 is no separator between the '3' and the 'r'. Check the environment variable
195 ``$VERSION`` as well.
195 ``$VERSION`` as well.
196
196
197 You will likely just have to modify/comment/uncomment one of the lines setting
197 You will likely just have to modify/comment/uncomment one of the lines setting
198 ``_version_extra``
198 ``_version_extra``
199
199
200
200
201 5. Run the `tools/build_release` script
201 5. Run the `tools/build_release` script
202 ---------------------------------------
202 ---------------------------------------
203
203
204 Running ``tools/build_release`` does all the file checking and building that
204 Running ``tools/build_release`` does all the file checking and building that
205 the real release script will do. This makes test installations, checks that
205 the real release script will do. This makes test installations, checks that
206 the build procedure runs OK, and tests other steps in the release process.
206 the build procedure runs OK, and tests other steps in the release process.
207
207
208 The ``build_release`` script will in particular verify that the version number
208 The ``build_release`` script will in particular verify that the version number
209 match PEP 440, in order to avoid surprise at the time of build upload.
209 match PEP 440, in order to avoid surprise at the time of build upload.
210
210
211 We encourage creating a test build of the docs as well.
211 We encourage creating a test build of the docs as well.
212
212
213 6. Create and push the new tag
213 6. Create and push the new tag
214 ------------------------------
214 ------------------------------
215
215
216 Commit the changes to release.py::
216 Commit the changes to release.py::
217
217
218 git commit -am "release $VERSION"
218 git commit -am "release $VERSION" -S
219 git push origin $BRANCH
219 git push origin $BRANCH
220
220
221 (omit the ``-S`` if you are no signing the package)
222
221 Create and push the tag::
223 Create and push the tag::
222
224
223 git tag -am "release $VERSION" "$VERSION"
225 git tag -am "release $VERSION" "$VERSION" -S
224 git push origin $VERSION
226 git push origin $VERSION
225
227
228 (omit the ``-S`` if you are no signing the package)
229
226 Update release.py back to ``x.y-dev`` or ``x.y-maint``, and re-add the
230 Update release.py back to ``x.y-dev`` or ``x.y-maint``, and re-add the
227 ``development`` entry in ``docs/source/whatsnew/index.rst`` and push::
231 ``development`` entry in ``docs/source/whatsnew/index.rst`` and push::
228
232
229 git commit -am "back to development"
233 git commit -am "back to development" -S
230 git push origin $BRANCH
234 git push origin $BRANCH
231
235
236 (omit the ``-S`` if you are no signing the package)
237
232 Now checkout the tag we just made::
238 Now checkout the tag we just made::
233
239
234 git checkout $VERSION
240 git checkout $VERSION
235
241
236 7. Run the release script
242 7. Run the release script
237 -------------------------
243 -------------------------
238
244
239 Run the ``release`` script, this step requires having a current wheel, Python
245 Run the ``release`` script, this step requires having a current wheel, Python
240 >=3.4 and Python 2.7.::
246 >=3.4 and Python 2.7.::
241
247
242 ./tools/release
248 ./tools/release
243
249
244 This makes the tarballs and wheels, and puts them under the ``dist/``
250 This makes the tarballs and wheels, and puts them under the ``dist/``
245 folder. Be sure to test the ``wheels`` and the ``sdist`` locally before
251 folder. Be sure to test the ``wheels`` and the ``sdist`` locally before
246 uploading them to PyPI. We do not use an universal wheel as each wheel
252 uploading them to PyPI. We do not use an universal wheel as each wheel
247 installs an ``ipython2`` or ``ipython3`` script, depending on the version of
253 installs an ``ipython2`` or ``ipython3`` script, depending on the version of
248 Python it is built for. Using an universal wheel would prevent this.
254 Python it is built for. Using an universal wheel would prevent this.
249
255
250 Use the following to actually upload the result of the build::
256 Use the following to actually upload the result of the build::
251
257
252 ./tools/release upload
258 ./tools/release upload
253
259
254 It should posts them to ``archive.ipython.org`` and to PyPI.
260 It should posts them to ``archive.ipython.org`` and to PyPI.
255
261
256 PyPI/Warehouse will automatically hide previous releases. If you are uploading
262 PyPI/Warehouse will automatically hide previous releases. If you are uploading
257 a non-stable version, make sure to log-in to PyPI and un-hide previous version.
263 a non-stable version, make sure to log-in to PyPI and un-hide previous version.
258
264
259
265
260 8. Draft a short release announcement
266 8. Draft a short release announcement
261 -------------------------------------
267 -------------------------------------
262
268
263 The announcement should include:
269 The announcement should include:
264
270
265 - release highlights
271 - release highlights
266 - a link to the html version of the *What's new* section of the documentation
272 - a link to the html version of the *What's new* section of the documentation
267 - a link to upgrade or installation tips (if necessary)
273 - a link to upgrade or installation tips (if necessary)
268
274
269 Post the announcement to the mailing list and or blog, and link from Twitter.
275 Post the announcement to the mailing list and or blog, and link from Twitter.
270
276
271 .. note::
277 .. note::
272
278
273 If you are doing a RC or Beta, you can likely skip the next steps.
279 If you are doing a RC or Beta, you can likely skip the next steps.
274
280
275 9. Update milestones on GitHub
281 9. Update milestones on GitHub
276 -------------------------------
282 -------------------------------
277
283
278 These steps will bring milestones up to date:
284 These steps will bring milestones up to date:
279
285
280 - close the just released milestone
286 - close the just released milestone
281 - open a new milestone for the next release (x, y+1), if the milestone doesn't
287 - open a new milestone for the next release (x, y+1), if the milestone doesn't
282 exist already
288 exist already
283
289
284 10. Update the IPython website
290 10. Update the IPython website
285 ------------------------------
291 ------------------------------
286
292
287 The IPython website should document the new release:
293 The IPython website should document the new release:
288
294
289 - add release announcement (news, announcements)
295 - add release announcement (news, announcements)
290 - update current version and download links
296 - update current version and download links
291 - update links on the documentation page (especially if a major release)
297 - update links on the documentation page (especially if a major release)
292
298
293 11. Update readthedocs
299 11. Update readthedocs
294 ----------------------
300 ----------------------
295
301
296 Make sure to update readthedocs and set the latest tag as stable, as well as
302 Make sure to update readthedocs and set the latest tag as stable, as well as
297 checking that previous release is still building under its own tag.
303 checking that previous release is still building under its own tag.
298
304
299 12. Update the Conda-Forge feedstock
305 12. Update the Conda-Forge feedstock
300 ------------------------------------
306 ------------------------------------
301
307
302 Follow the instructions on `the repository <https://github.com/conda-forge/ipython-feedstock>`_
308 Follow the instructions on `the repository <https://github.com/conda-forge/ipython-feedstock>`_
303
309
304 13. Celebrate!
310 13. Celebrate!
305 --------------
311 --------------
306
312
307 Celebrate the release and please thank the contributors for their work. Great
313 Celebrate the release and please thank the contributors for their work. Great
308 job!
314 job!
309
315
310
316
311
317
312 Old Documentation
318 Old Documentation
313 =================
319 =================
314
320
315 Out of date documentation is still available and have been kept for archival purposes.
321 Out of date documentation is still available and have been kept for archival purposes.
316
322
317 .. note::
323 .. note::
318
324
319 Developers documentation used to be on the IPython wiki, but are now out of
325 Developers documentation used to be on the IPython wiki, but are now out of
320 date. The wiki is though still available for historical reasons: `Old IPython
326 date. The wiki is though still available for historical reasons: `Old IPython
321 GitHub Wiki. <https://github.com/ipython/ipython/wiki/Dev:-Index>`_
327 GitHub Wiki. <https://github.com/ipython/ipython/wiki/Dev:-Index>`_
General Comments 0
You need to be logged in to leave comments. Login now