##// END OF EJS Templates
Cleany seprate steps to make sure none is forgotten
Matthias Bussonnier -
Show More
@@ -1,318 +1,321 b''
1 .. _core_developer_guide:
1 .. _core_developer_guide:
2
2
3 =================================
3 =================================
4 Guide for IPython core Developers
4 Guide for IPython core Developers
5 =================================
5 =================================
6
6
7 This guide documents the development of IPython itself. Alternatively,
7 This guide documents the development of IPython itself. Alternatively,
8 developers of third party tools and libraries that use IPython should see the
8 developers of third party tools and libraries that use IPython should see the
9 :doc:`../development/index`.
9 :doc:`../development/index`.
10
10
11
11
12 For instructions on how to make a developer install see :ref:`devinstall`.
12 For instructions on how to make a developer install see :ref:`devinstall`.
13
13
14 Backporting Pull requests
14 Backporting Pull requests
15 -------------------------
15 -------------------------
16
16
17 All pull requests should usually be made against ``master``, if a Pull Request
17 All pull requests should usually be made against ``master``, if a Pull Request
18 need to be backported to an earlier release; then it should be tagged with the
18 need to be backported to an earlier release; then it should be tagged with the
19 correct ``milestone``.
19 correct ``milestone``.
20
20
21 If you tag a pull request with a milestone **before** merging the pull request,
21 If you tag a pull request with a milestone **before** merging the pull request,
22 and the base ref is `master`, then our backport bot should automatically create
22 and the base ref is `master`, then our backport bot should automatically create
23 a corresponding pull-request that backport on the correct branch.
23 a corresponding pull-request that backport on the correct branch.
24
24
25 If you are an admin on the IPython repository you can also just mention the
25 If you are an admin on the IPython repository you can also just mention the
26 **backport bot** to do the work for you. The bot is evolving so instructions may
26 **backport bot** to do the work for you. The bot is evolving so instructions may
27 be different. At the time of this writing you can use::
27 be different. At the time of this writing you can use::
28
28
29 @meeseeksdev[bot] backport [to <branchname>]
29 @meeseeksdev[bot] backport [to <branchname>]
30
30
31 The bot will attempt to backport the current pull-request and issue a PR if
31 The bot will attempt to backport the current pull-request and issue a PR if
32 possible. If the milestone is set on the issue you can omit the branch to
32 possible. If the milestone is set on the issue you can omit the branch to
33 backport to.
33 backport to.
34
34
35 .. note::
35 .. note::
36
36
37 The ``@`` and ``[dev]`` when mentioning the bot should be optional and can
37 The ``@`` and ``[dev]`` when mentioning the bot should be optional and can
38 be omitted.
38 be omitted.
39
39
40 If the pull request cannot be automatically backported, the bot should tell you
40 If the pull request cannot be automatically backported, the bot should tell you
41 so on the PR and apply a "Need manual backport" tag to the origin PR.
41 so on the PR and apply a "Need manual backport" tag to the origin PR.
42
42
43
43
44 Backport with ghpro
44 Backport with ghpro
45 -------------------
45 -------------------
46
46
47 We can also use `ghpro <https://pypi.python.org/pypi/ghpro>`
47 We can also use `ghpro <https://pypi.python.org/pypi/ghpro>`
48 to automatically list and apply the PR on other branches. For example:
48 to automatically list and apply the PR on other branches. For example:
49
49
50 .. code-block:: bash
50 .. code-block:: bash
51
51
52 $ backport-pr todo --milestone 5.2
52 $ backport-pr todo --milestone 5.2
53 [...snip..]
53 [...snip..]
54 The following PRs have been backported
54 The following PRs have been backported
55 9848
55 9848
56 9851
56 9851
57 9953
57 9953
58 9955
58 9955
59 The following PRs should be backported:
59 The following PRs should be backported:
60 9417
60 9417
61 9863
61 9863
62 9925
62 9925
63 9947
63 9947
64
64
65 $ backport-pr apply 5.x 9947
65 $ backport-pr apply 5.x 9947
66 [...snip...]
66 [...snip...]
67
67
68
68
69 .. _release_process:
69 .. _release_process:
70
70
71 =======================
71 =======================
72 IPython release process
72 IPython release process
73 =======================
73 =======================
74
74
75 This document contains the process that is used to create an IPython release.
75 This document contains the process that is used to create an IPython release.
76
76
77 Conveniently, the ``release`` script in the ``tools`` directory of the ``IPython``
77 Conveniently, the ``release`` script in the ``tools`` directory of the ``IPython``
78 repository automates most of the release process. This document serves as a
78 repository automates most of the release process. This document serves as a
79 handy reminder and checklist for the release manager.
79 handy reminder and checklist for the release manager.
80
80
81 During the release process, you might need the extra following dependencies:
81 During the release process, you might need the extra following dependencies:
82
82
83 - ``keyring`` to access your GitHub authentication tokens
83 - ``keyring`` to access your GitHub authentication tokens
84 - ``graphviz`` to generate some graphs in the documentation
84 - ``graphviz`` to generate some graphs in the documentation
85 - ``ghpro`` to generate the stats
85 - ``ghpro`` to generate the stats
86
86
87 Make sure you have all the required dependencies to run the tests as well.
87 Make sure you have all the required dependencies to run the tests as well.
88
88
89
89
90 1. Set Environment variables
90 1. Set Environment variables
91 ----------------------------
91 ----------------------------
92
92
93 Set environment variables to document previous release tag, current
93 Set environment variables to document previous release tag, current
94 release milestone, current release version, and git tag.
94 release milestone, current release version, and git tag.
95
95
96 These variables may be used later to copy/paste as answers to the script
96 These variables may be used later to copy/paste as answers to the script
97 questions instead of typing the appropriate command when the time comes. These
97 questions instead of typing the appropriate command when the time comes. These
98 variables are not used by the scripts directly; therefore, there is no need to
98 variables are not used by the scripts directly; therefore, there is no need to
99 ``export`` them. The format for bash is as follows, but note that these values
99 ``export`` them. The format for bash is as follows, but note that these values
100 are just an example valid only for the 5.0 release; you'll need to update them
100 are just an example valid only for the 5.0 release; you'll need to update them
101 for the release you are actually making::
101 for the release you are actually making::
102
102
103 PREV_RELEASE=4.2.1
103 PREV_RELEASE=4.2.1
104 MILESTONE=5.0
104 MILESTONE=5.0
105 VERSION=5.0.0
105 VERSION=5.0.0
106 BRANCH=master
106 BRANCH=master
107
107
108
108
109 2. Create GitHub stats and finish release note
109 2. Create GitHub stats and finish release note
110 ----------------------------------------------
110 ----------------------------------------------
111
111
112 .. note::
112 .. note::
113
113
114 This step is optional if making a Beta or RC release.
114 This step is optional if making a Beta or RC release.
115
115
116 .. note::
116 .. note::
117
117
118 Before generating the GitHub stats, verify that all closed issues and pull
118 Before generating the GitHub stats, verify that all closed issues and pull
119 requests have `appropriate milestones
119 requests have `appropriate milestones
120 <https://github.com/ipython/ipython/wiki/Dev:-GitHub-workflow#milestones>`_.
120 <https://github.com/ipython/ipython/wiki/Dev:-GitHub-workflow#milestones>`_.
121 `This search
121 `This search
122 <https://github.com/ipython/ipython/issues?q=is%3Aclosed+no%3Amilestone+is%3Aissue>`_
122 <https://github.com/ipython/ipython/issues?q=is%3Aclosed+no%3Amilestone+is%3Aissue>`_
123 should return no results before creating the GitHub stats.
123 should return no results before creating the GitHub stats.
124
124
125 If a major release:
125 If a major release:
126
126
127 - merge any pull request notes into what's new::
127 - merge any pull request notes into what's new::
128
128
129 python tools/update_whatsnew.py
129 python tools/update_whatsnew.py
130
130
131 - update ``docs/source/whatsnew/development.rst``, to ensure it covers
131 - update ``docs/source/whatsnew/development.rst``, to ensure it covers
132 the major release features
132 the major release features
133
133
134 - move the contents of ``development.rst`` to ``versionX.rst`` where ``X`` is
134 - move the contents of ``development.rst`` to ``versionX.rst`` where ``X`` is
135 the numerical release version
135 the numerical release version
136
136
137 - generate summary of GitHub contributions, which can be done with::
137 - generate summary of GitHub contributions, which can be done with::
138
138
139 python tools/github_stats.py --milestone $MILESTONE > stats.rst
139 python tools/github_stats.py --milestone $MILESTONE > stats.rst
140
140
141 which may need some manual cleanup of ``stats.rst``. Add the cleaned
141 which may need some manual cleanup of ``stats.rst``. Add the cleaned
142 ``stats.rst`` results to ``docs/source/whatsnew/github-stats-X.rst``
142 ``stats.rst`` results to ``docs/source/whatsnew/github-stats-X.rst``
143 where ``X`` is the numerical release version (don't forget to add it to
143 where ``X`` is the numerical release version (don't forget to add it to
144 the git repo as well). If creating a major release, make a new
144 the git repository as well). If creating a major release, make a new
145 ``github-stats-X.rst`` file; if creating a minor release, the content
145 ``github-stats-X.rst`` file; if creating a minor release, the content
146 from ``stats.rst`` may simply be added to the top of an existing
146 from ``stats.rst`` may simply be added to the top of an existing
147 ``github-stats-X.rst`` file. Finally, edit
147 ``github-stats-X.rst`` file.
148 ``docs/source/whatsnew/index.rst`` to list the new ``github-stats-X``
148
149 file you just created and remove temporarily the first entry called
149 - Edit ``docs/source/whatsnew/index.rst`` to list the new ``github-stats-X``
150 ``development`` (you'll need to add it back after release).
150 file you just created.
151
152 - Remove temporarily the first entry called ``development`` (you'll need to
153 add it back after release).
151
154
152 Make sure that the stats file has a header or it won't be rendered in
155 Make sure that the stats file has a header or it won't be rendered in
153 the final documentation.
156 the final documentation.
154
157
155 To find duplicates and update `.mailmap`, use::
158 To find duplicates and update `.mailmap`, use::
156
159
157 git log --format="%aN <%aE>" $PREV_RELEASE... | sort -u -f
160 git log --format="%aN <%aE>" $PREV_RELEASE... | sort -u -f
158
161
159 If a minor release you might need to do some of the above points manually, and
162 If a minor release you might need to do some of the above points manually, and
160 forward port the changes.
163 forward port the changes.
161
164
162 3. Make sure the repository is clean
165 3. Make sure the repository is clean
163 ------------------------------------
166 ------------------------------------
164
167
165 of any file that could be problematic.
168 of any file that could be problematic.
166 Remove all non-tracked files with:
169 Remove all non-tracked files with:
167
170
168 .. code::
171 .. code::
169
172
170 git clean -xfdi
173 git clean -xfdi
171
174
172 This will ask for confirmation before removing all untracked files. Make
175 This will ask for confirmation before removing all untracked files. Make
173 sure the ``dist/`` folder is clean to avoid any stale builds from
176 sure the ``dist/`` folder is clean to avoid any stale builds from
174 previous build attempts.
177 previous build attempts.
175
178
176
179
177 4. Update the release version number
180 4. Update the release version number
178 ------------------------------------
181 ------------------------------------
179
182
180 Edit ``IPython/core/release.py`` to have the current version.
183 Edit ``IPython/core/release.py`` to have the current version.
181
184
182 in particular, update version number and ``_version_extra`` content in
185 in particular, update version number and ``_version_extra`` content in
183 ``IPython/core/release.py``.
186 ``IPython/core/release.py``.
184
187
185 Step 5 will validate your changes automatically, but you might still want to
188 Step 5 will validate your changes automatically, but you might still want to
186 make sure the version number matches pep440.
189 make sure the version number matches pep440.
187
190
188 In particular, ``rc`` and ``beta`` are not separated by ``.`` or the ``sdist``
191 In particular, ``rc`` and ``beta`` are not separated by ``.`` or the ``sdist``
189 and ``bdist`` will appear as different releases. For example, a valid version
192 and ``bdist`` will appear as different releases. For example, a valid version
190 number for a release candidate (rc) release is: ``1.3rc1``. Notice that there
193 number for a release candidate (rc) release is: ``1.3rc1``. Notice that there
191 is no separator between the '3' and the 'r'. Check the environment variable
194 is no separator between the '3' and the 'r'. Check the environment variable
192 ``$VERSION`` as well.
195 ``$VERSION`` as well.
193
196
194 You will likely just have to modify/comment/uncomment one of the lines setting
197 You will likely just have to modify/comment/uncomment one of the lines setting
195 ``_version_extra``
198 ``_version_extra``
196
199
197
200
198 5. Run the `tools/build_release` script
201 5. Run the `tools/build_release` script
199 ---------------------------------------
202 ---------------------------------------
200
203
201 Running ``tools/build_release`` does all the file checking and building that
204 Running ``tools/build_release`` does all the file checking and building that
202 the real release script will do. This makes test installations, checks that
205 the real release script will do. This makes test installations, checks that
203 the build procedure runs OK, and tests other steps in the release process.
206 the build procedure runs OK, and tests other steps in the release process.
204
207
205 The ``build_release`` script will in particular verify that the version number
208 The ``build_release`` script will in particular verify that the version number
206 match PEP 440, in order to avoid surprise at the time of build upload.
209 match PEP 440, in order to avoid surprise at the time of build upload.
207
210
208 We encourage creating a test build of the docs as well.
211 We encourage creating a test build of the docs as well.
209
212
210 6. Create and push the new tag
213 6. Create and push the new tag
211 ------------------------------
214 ------------------------------
212
215
213 Commit the changes to release.py::
216 Commit the changes to release.py::
214
217
215 git commit -am "release $VERSION"
218 git commit -am "release $VERSION"
216 git push origin $BRANCH
219 git push origin $BRANCH
217
220
218 Create and push the tag::
221 Create and push the tag::
219
222
220 git tag -am "release $VERSION" "$VERSION"
223 git tag -am "release $VERSION" "$VERSION"
221 git push origin --tags
224 git push origin --tags
222
225
223 Update release.py back to ``x.y-dev`` or ``x.y-maint``, and re-add the
226 Update release.py back to ``x.y-dev`` or ``x.y-maint``, and re-add the
224 ``development`` entry in ``docs/source/whatsnew/index.rst`` and push::
227 ``development`` entry in ``docs/source/whatsnew/index.rst`` and push::
225
228
226 git commit -am "back to development"
229 git commit -am "back to development"
227 git push origin $BRANCH
230 git push origin $BRANCH
228
231
229 Now checkout the tag we just made::
232 Now checkout the tag we just made::
230
233
231 git checkout $VERSION
234 git checkout $VERSION
232
235
233 7. Run the release script
236 7. Run the release script
234 -------------------------
237 -------------------------
235
238
236 Run the ``release`` script, this step requires having a current wheel, Python
239 Run the ``release`` script, this step requires having a current wheel, Python
237 >=3.4 and Python 2.7.::
240 >=3.4 and Python 2.7.::
238
241
239 ./tools/release
242 ./tools/release
240
243
241 This makes the tarballs and wheels, and puts them under the ``dist/``
244 This makes the tarballs and wheels, and puts them under the ``dist/``
242 folder. Be sure to test the ``wheels`` and the ``sdist`` locally before
245 folder. Be sure to test the ``wheels`` and the ``sdist`` locally before
243 uploading them to PyPI. We do not use an universal wheel as each wheel
246 uploading them to PyPI. We do not use an universal wheel as each wheel
244 installs an ``ipython2`` or ``ipython3`` script, depending on the version of
247 installs an ``ipython2`` or ``ipython3`` script, depending on the version of
245 Python it is built for. Using an universal wheel would prevent this.
248 Python it is built for. Using an universal wheel would prevent this.
246
249
247 Use the following to actually upload the result of the build::
250 Use the following to actually upload the result of the build::
248
251
249 ./tools/release upload
252 ./tools/release upload
250
253
251 It should posts them to ``archive.ipython.org`` and to PyPI.
254 It should posts them to ``archive.ipython.org`` and to PyPI.
252
255
253 PyPI/Warehouse will automatically hide previous releases. If you are uploading
256 PyPI/Warehouse will automatically hide previous releases. If you are uploading
254 a non-stable version, make sure to log-in to PyPI and un-hide previous version.
257 a non-stable version, make sure to log-in to PyPI and un-hide previous version.
255
258
256
259
257 8. Draft a short release announcement
260 8. Draft a short release announcement
258 -------------------------------------
261 -------------------------------------
259
262
260 The announcement should include:
263 The announcement should include:
261
264
262 - release highlights
265 - release highlights
263 - a link to the html version of the *What's new* section of the documentation
266 - a link to the html version of the *What's new* section of the documentation
264 - a link to upgrade or installation tips (if necessary)
267 - a link to upgrade or installation tips (if necessary)
265
268
266 Post the announcement to the mailing list and or blog, and link from Twitter.
269 Post the announcement to the mailing list and or blog, and link from Twitter.
267
270
268 .. note::
271 .. note::
269
272
270 If you are doing a RC or Beta, you can likely skip the next steps.
273 If you are doing a RC or Beta, you can likely skip the next steps.
271
274
272 9. Update milestones on GitHub
275 9. Update milestones on GitHub
273 -------------------------------
276 -------------------------------
274
277
275 These steps will bring milestones up to date:
278 These steps will bring milestones up to date:
276
279
277 - close the just released milestone
280 - close the just released milestone
278 - open a new milestone for the next release (x, y+1), if the milestone doesn't
281 - open a new milestone for the next release (x, y+1), if the milestone doesn't
279 exist already
282 exist already
280
283
281 10. Update the IPython website
284 10. Update the IPython website
282 ------------------------------
285 ------------------------------
283
286
284 The IPython website should document the new release:
287 The IPython website should document the new release:
285
288
286 - add release announcement (news, announcements)
289 - add release announcement (news, announcements)
287 - update current version and download links
290 - update current version and download links
288 - update links on the documentation page (especially if a major release)
291 - update links on the documentation page (especially if a major release)
289
292
290 11. Update readthedocs
293 11. Update readthedocs
291 ----------------------
294 ----------------------
292
295
293 Make sure to update readthedocs and set the latest tag as stable, as well as
296 Make sure to update readthedocs and set the latest tag as stable, as well as
294 checking that previous release is still building under its own tag.
297 checking that previous release is still building under its own tag.
295
298
296 12. Update the Conda-Forge feedstock
299 12. Update the Conda-Forge feedstock
297 ------------------------------------
300 ------------------------------------
298
301
299 Follow the instructions on `the repository <https://github.com/conda-forge/ipython-feedstock>`_
302 Follow the instructions on `the repository <https://github.com/conda-forge/ipython-feedstock>`_
300
303
301 13. Celebrate!
304 13. Celebrate!
302 --------------
305 --------------
303
306
304 Celebrate the release and please thank the contributors for their work. Great
307 Celebrate the release and please thank the contributors for their work. Great
305 job!
308 job!
306
309
307
310
308
311
309 Old Documentation
312 Old Documentation
310 =================
313 =================
311
314
312 Out of date documentation is still available and have been kept for archival purposes.
315 Out of date documentation is still available and have been kept for archival purposes.
313
316
314 .. note::
317 .. note::
315
318
316 Developers documentation used to be on the IPython wiki, but are now out of
319 Developers documentation used to be on the IPython wiki, but are now out of
317 date. The wiki is though still available for historical reasons: `Old IPython
320 date. The wiki is though still available for historical reasons: `Old IPython
318 GitHub Wiki. <https://github.com/ipython/ipython/wiki/Dev:-Index>`_
321 GitHub Wiki. <https://github.com/ipython/ipython/wiki/Dev:-Index>`_
@@ -1,47 +1,46 b''
1 .. Developers should add in this file, during each release cycle, information
1 .. Developers should add in this file, during each release cycle, information
2 .. about important changes they've made, in a summary format that's meant for
2 .. about important changes they've made, in a summary format that's meant for
3 .. end users. For each release we normally have three sections: features, bug
3 .. end users. For each release we normally have three sections: features, bug
4 .. fixes and api breakage.
4 .. fixes and api breakage.
5 .. Please remember to credit the authors of the contributions by name,
5 .. Please remember to credit the authors of the contributions by name,
6 .. especially when they are new users or developers who do not regularly
6 .. especially when they are new users or developers who do not regularly
7 .. participate in IPython's development.
7 .. participate in IPython's development.
8
8
9 .. _whatsnew_index:
9 .. _whatsnew_index:
10
10
11 =====================
11 =====================
12 What's new in IPython
12 What's new in IPython
13 =====================
13 =====================
14
14
15 This section documents the changes that have been made in various versions of
15 This section documents the changes that have been made in various versions of
16 IPython. Users should consult these pages to learn about new features, bug
16 IPython. Users should consult these pages to learn about new features, bug
17 fixes and backwards incompatibilities. Developers should summarize the
17 fixes and backwards incompatibilities. Developers should summarize the
18 development work they do here in a user friendly format.
18 development work they do here in a user friendly format.
19
19
20 .. toctree::
20 .. toctree::
21 :maxdepth: 1
21 :maxdepth: 1
22
22
23 development
24 version6
23 version6
25 github-stats-6
24 github-stats-6
26 version5
25 version5
27 github-stats-5
26 github-stats-5
28 version4
27 version4
29 github-stats-4
28 github-stats-4
30 version3
29 version3
31 github-stats-3
30 github-stats-3
32 version3_widget_migration
31 version3_widget_migration
33 version2.0
32 version2.0
34 github-stats-2.0
33 github-stats-2.0
35 version1.0
34 version1.0
36 github-stats-1.0
35 github-stats-1.0
37 version0.13
36 version0.13
38 github-stats-0.13
37 github-stats-0.13
39 version0.12
38 version0.12
40 github-stats-0.12
39 github-stats-0.12
41 version0.11
40 version0.11
42 github-stats-0.11
41 github-stats-0.11
43 version0.10
42 version0.10
44 version0.9
43 version0.9
45 version0.8
44 version0.8
46
45
47
46
General Comments 0
You need to be logged in to leave comments. Login now