Show More
@@ -1,316 +1,317 b'' | |||
|
1 | 1 | .. _contributing: |
|
2 | 2 | |
|
3 | 3 | ========================= |
|
4 | 4 | Contributing to Kallithea |
|
5 | 5 | ========================= |
|
6 | 6 | |
|
7 | 7 | Kallithea is developed and maintained by its users. Please join us and scratch |
|
8 | 8 | your own itch. |
|
9 | 9 | |
|
10 | 10 | |
|
11 | 11 | Infrastructure |
|
12 | 12 | -------------- |
|
13 | 13 | |
|
14 | 14 | The main repository is hosted on Our Own Kallithea (aka OOK) at |
|
15 | 15 | https://kallithea-scm.org/repos/kallithea/, our self-hosted instance |
|
16 | 16 | of Kallithea. |
|
17 | 17 | |
|
18 | 18 | For now, we use Bitbucket_ for `pull requests`_ and `issue tracking`_. The |
|
19 | 19 | issue tracker is for tracking bugs, not for support, discussion, or ideas -- |
|
20 | 20 | please use the `mailing list`_ or :ref:`IRC <readme>` to reach the community. |
|
21 | 21 | |
|
22 | 22 | We use Weblate_ to translate the user interface messages into languages other |
|
23 | 23 | than English. Join our project on `Hosted Weblate`_ to help us. |
|
24 | 24 | To register, you can use your Bitbucket or GitHub account. See :ref:`translations` |
|
25 | 25 | for more details. |
|
26 | 26 | |
|
27 | 27 | |
|
28 | 28 | Getting started |
|
29 | 29 | --------------- |
|
30 | 30 | |
|
31 | 31 | To get started with Kallithea development:: |
|
32 | 32 | |
|
33 | 33 | hg clone https://kallithea-scm.org/repos/kallithea |
|
34 | 34 | cd kallithea |
|
35 | 35 | virtualenv ../kallithea-venv |
|
36 | 36 | source ../kallithea-venv/bin/activate |
|
37 | 37 | pip install --upgrade pip setuptools |
|
38 | 38 | pip install --upgrade -e . |
|
39 | 39 | pip install --upgrade -r dev_requirements.txt |
|
40 | 40 | npm install # install dependencies - both tools and data |
|
41 | 41 | npm run less # for generating css from less |
|
42 | 42 | gearbox make-config my.ini |
|
43 | 43 | gearbox setup-db -c my.ini --user=user --email=user@example.com --password=password --repos=/tmp |
|
44 | 44 | gearbox serve -c my.ini --reload & |
|
45 | 45 | firefox http://127.0.0.1:5000/ |
|
46 | 46 | |
|
47 | 47 | If you plan to use Bitbucket_ for sending contributions, you can also fork |
|
48 | 48 | Kallithea on Bitbucket_ first (https://bitbucket.org/conservancy/kallithea) and |
|
49 | 49 | then replace the clone step above by a clone of your fork. In this case, please |
|
50 | 50 | see :ref:`contributing-guidelines` below for configuring your fork correctly. |
|
51 | 51 | |
|
52 | 52 | |
|
53 | 53 | Contribution flow |
|
54 | 54 | ----------------- |
|
55 | 55 | |
|
56 | 56 | Starting from an existing Kallithea clone, make sure it is up to date with the |
|
57 | 57 | latest upstream changes:: |
|
58 | 58 | |
|
59 | 59 | hg pull |
|
60 | 60 | hg update |
|
61 | 61 | |
|
62 | 62 | Review the :ref:`contributing-guidelines` and :ref:`coding-guidelines`. |
|
63 | 63 | |
|
64 | 64 | If you are new to Mercurial, refer to Mercurial `Quick Start`_ and `Beginners |
|
65 | 65 | Guide`_ on the Mercurial wiki. |
|
66 | 66 | |
|
67 | 67 | Now, make some changes and test them (see :ref:`contributing-tests`). Don't |
|
68 | 68 | forget to add new tests to cover new functionality or bug fixes. |
|
69 | 69 | |
|
70 | 70 | For documentation changes, run ``make html`` from the ``docs`` directory to |
|
71 | 71 | generate the HTML result, then review them in your browser. |
|
72 | 72 | |
|
73 | 73 | Before submitting any changes, run the cleanup script:: |
|
74 | 74 | |
|
75 | 75 | ./scripts/run-all-cleanup |
|
76 | 76 | |
|
77 | 77 | When you are completely ready, you can send your changes to the community for |
|
78 | 78 | review and inclusion. Most commonly used methods are sending patches to the |
|
79 | 79 | mailing list (via ``hg email``) or by creating a pull request on Bitbucket_. |
|
80 | 80 | |
|
81 | 81 | .. _contributing-tests: |
|
82 | 82 | |
|
83 | 83 | |
|
84 | 84 | Running tests |
|
85 | 85 | ------------- |
|
86 | 86 | |
|
87 | 87 | After finishing your changes make sure all tests pass cleanly. Run the testsuite |
|
88 | 88 | by invoking ``py.test`` from the project root:: |
|
89 | 89 | |
|
90 | 90 | py.test |
|
91 | 91 | |
|
92 | 92 | Note that testing on Python 2.6 also requires ``unittest2``. |
|
93 | 93 | |
|
94 | 94 | Note that on unix systems, the temporary directory (``/tmp`` or where |
|
95 | 95 | ``$TMPDIR`` points) must allow executable files; Git hooks must be executable, |
|
96 | 96 | and the test suite creates repositories in the temporary directory. Linux |
|
97 | 97 | systems with /tmp mounted noexec will thus fail. |
|
98 | 98 | |
|
99 | 99 | You can also use ``tox`` to run the tests with all supported Python versions |
|
100 | 100 | (currently Python 2.6--2.7). |
|
101 | 101 | |
|
102 |
When running tests, Kallithea |
|
|
103 | SQLite database specified there. | |
|
102 | When running tests, Kallithea generates a `test.ini` based on template values | |
|
103 | in `kallithea/tests/conftest.py` and populates the SQLite database specified | |
|
104 | there. | |
|
104 | 105 | |
|
105 | 106 | It is possible to avoid recreating the full test database on each invocation of |
|
106 | 107 | the tests, thus eliminating the initial delay. To achieve this, run the tests as:: |
|
107 | 108 | |
|
108 |
gearbox serve -c kallithea |
|
|
109 | gearbox serve -c /tmp/kallithea-test-XXX/test.ini --pid-file=test.pid --daemon | |
|
109 | 110 | KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test |
|
110 | 111 | kill -9 $(cat test.pid) |
|
111 | 112 | |
|
112 | 113 | In these commands, the following variables are used:: |
|
113 | 114 | |
|
114 | 115 | KALLITHEA_WHOOSH_TEST_DISABLE=1 - skip whoosh index building and tests |
|
115 | 116 | KALLITHEA_NO_TMP_PATH=1 - disable new temp path for tests, used mostly for testing_vcs_operations |
|
116 | 117 | |
|
117 | 118 | You can run individual tests by specifying their path as argument to py.test. |
|
118 | 119 | py.test also has many more options, see `py.test -h`. Some useful options |
|
119 | 120 | are:: |
|
120 | 121 | |
|
121 | 122 | -k EXPRESSION only run tests which match the given substring |
|
122 | 123 | expression. An expression is a python evaluable |
|
123 | 124 | expression where all names are substring-matched |
|
124 | 125 | against test names and their parent classes. Example: |
|
125 | 126 | -x, --exitfirst exit instantly on first error or failed test. |
|
126 | 127 | --lf rerun only the tests that failed at the last run (or |
|
127 | 128 | all if none failed) |
|
128 | 129 | --ff run all tests but run the last failures first. This |
|
129 | 130 | may re-order tests and thus lead to repeated fixture |
|
130 | 131 | setup/teardown |
|
131 | 132 | --pdb start the interactive Python debugger on errors. |
|
132 | 133 | -s, --capture=no don't capture stdout (any stdout output will be |
|
133 | 134 | printed immediately) |
|
134 | 135 | |
|
135 | 136 | Performance tests |
|
136 | 137 | ^^^^^^^^^^^^^^^^^ |
|
137 | 138 | |
|
138 | 139 | A number of performance tests are present in the test suite, but they are |
|
139 | 140 | not run in a standard test run. These tests are useful to |
|
140 | 141 | evaluate the impact of certain code changes with respect to performance. |
|
141 | 142 | |
|
142 | 143 | To run these tests:: |
|
143 | 144 | |
|
144 | 145 | env TEST_PERFORMANCE=1 py.test kallithea/tests/performance |
|
145 | 146 | |
|
146 | 147 | To analyze performance, you could install pytest-profiling_, which enables the |
|
147 | 148 | --profile and --profile-svg options to py.test. |
|
148 | 149 | |
|
149 | 150 | .. _pytest-profiling: https://github.com/manahl/pytest-plugins/tree/master/pytest-profiling |
|
150 | 151 | |
|
151 | 152 | .. _contributing-guidelines: |
|
152 | 153 | |
|
153 | 154 | |
|
154 | 155 | Contribution guidelines |
|
155 | 156 | ----------------------- |
|
156 | 157 | |
|
157 | 158 | Kallithea is GPLv3 and we assume all contributions are made by the |
|
158 | 159 | committer/contributor and under GPLv3 unless explicitly stated. We do care a |
|
159 | 160 | lot about preservation of copyright and license information for existing code |
|
160 | 161 | that is brought into the project. |
|
161 | 162 | |
|
162 | 163 | Contributions will be accepted in most formats -- such as pull requests on |
|
163 | 164 | Bitbucket, something hosted on your own Kallithea instance, or patches sent by |
|
164 | 165 | email to the `kallithea-general`_ mailing list. |
|
165 | 166 | |
|
166 | 167 | When contributing via Bitbucket, please make your fork of |
|
167 | 168 | https://bitbucket.org/conservancy/kallithea/ `non-publishing`_ -- it is one of |
|
168 | 169 | the settings on "Repository details" page. This ensures your commits are in |
|
169 | 170 | "draft" phase and makes it easier for you to address feedback and for project |
|
170 | 171 | maintainers to integrate your changes. |
|
171 | 172 | |
|
172 | 173 | .. _non-publishing: https://www.mercurial-scm.org/wiki/Phases#Publishing_Repository |
|
173 | 174 | |
|
174 | 175 | Make sure to test your changes both manually and with the automatic tests |
|
175 | 176 | before posting. |
|
176 | 177 | |
|
177 | 178 | We care about quality and review and keeping a clean repository history. We |
|
178 | 179 | might give feedback that requests polishing contributions until they are |
|
179 | 180 | "perfect". We might also rebase and collapse and make minor adjustments to your |
|
180 | 181 | changes when we apply them. |
|
181 | 182 | |
|
182 | 183 | We try to make sure we have consensus on the direction the project is taking. |
|
183 | 184 | Everything non-sensitive should be discussed in public -- preferably on the |
|
184 | 185 | mailing list. We aim at having all non-trivial changes reviewed by at least |
|
185 | 186 | one other core developer before pushing. Obvious non-controversial changes will |
|
186 | 187 | be handled more casually. |
|
187 | 188 | |
|
188 | 189 | There is a main development branch ("default") which is generally stable so that |
|
189 | 190 | it can be (and is) used in production. There is also a "stable" branch that is |
|
190 | 191 | almost exclusively reserved for bug fixes or trivial changes. Experimental |
|
191 | 192 | changes should live elsewhere (for example in a pull request) until they are |
|
192 | 193 | ready. |
|
193 | 194 | |
|
194 | 195 | .. _coding-guidelines: |
|
195 | 196 | |
|
196 | 197 | |
|
197 | 198 | Coding guidelines |
|
198 | 199 | ----------------- |
|
199 | 200 | |
|
200 | 201 | We don't have a formal coding/formatting standard. We are currently using a mix |
|
201 | 202 | of Mercurial's (https://www.mercurial-scm.org/wiki/CodingStyle), pep8, and |
|
202 | 203 | consistency with existing code. Run ``scripts/run-all-cleanup`` before |
|
203 | 204 | committing to ensure some basic code formatting consistency. |
|
204 | 205 | |
|
205 | 206 | We support both Python 2.6.x and 2.7.x and nothing else. For now we don't care |
|
206 | 207 | about Python 3 compatibility. |
|
207 | 208 | |
|
208 | 209 | We try to support the most common modern web browsers. IE9 is still supported |
|
209 | 210 | to the extent it is feasible, IE8 is not. |
|
210 | 211 | |
|
211 | 212 | We primarily support Linux and OS X on the server side but Windows should also work. |
|
212 | 213 | |
|
213 | 214 | HTML templates should use 2 spaces for indentation ... but be pragmatic. We |
|
214 | 215 | should use templates cleverly and avoid duplication. We should use reasonable |
|
215 | 216 | semantic markup with element classes and IDs that can be used for styling and testing. |
|
216 | 217 | We should only use inline styles in places where it really is semantic (such as |
|
217 | 218 | ``display: none``). |
|
218 | 219 | |
|
219 | 220 | JavaScript must use ``;`` between/after statements. Indentation 4 spaces. Inline |
|
220 | 221 | multiline functions should be indented two levels -- one for the ``()`` and one for |
|
221 | 222 | ``{}``. |
|
222 | 223 | Variables holding jQuery objects should be named with a leading ``$``. |
|
223 | 224 | |
|
224 | 225 | Commit messages should have a leading short line summarizing the changes. For |
|
225 | 226 | bug fixes, put ``(Issue #123)`` at the end of this line. |
|
226 | 227 | |
|
227 | 228 | Use American English grammar and spelling overall. Use `English title case`_ for |
|
228 | 229 | page titles, button labels, headers, and 'labels' for fields in forms. |
|
229 | 230 | |
|
230 | 231 | .. _English title case: https://en.wikipedia.org/wiki/Capitalization#Title_case |
|
231 | 232 | |
|
232 | 233 | Template helpers (that is, everything in ``kallithea.lib.helpers``) |
|
233 | 234 | should only be referenced from templates. If you need to call a |
|
234 | 235 | helper from the Python code, consider moving the function somewhere |
|
235 | 236 | else (e.g. to the model). |
|
236 | 237 | |
|
237 | 238 | Notes on the SQLAlchemy session |
|
238 | 239 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
|
239 | 240 | |
|
240 | 241 | Each HTTP request runs inside an independent SQLAlchemy session (as well |
|
241 | 242 | as in an independent database transaction). ``Session`` is the session manager |
|
242 | 243 | and factory. ``Session()`` will create a new session on-demand or return the |
|
243 | 244 | current session for the active thread. Many database operations are methods on |
|
244 | 245 | such session instances - only ``Session.remove()`` should be called directly on |
|
245 | 246 | the manager. |
|
246 | 247 | |
|
247 | 248 | Database model objects |
|
248 | 249 | (almost) always belong to a particular SQLAlchemy session, which means |
|
249 | 250 | that SQLAlchemy will ensure that they're kept in sync with the database |
|
250 | 251 | (but also means that they cannot be shared across requests). |
|
251 | 252 | |
|
252 | 253 | Objects can be added to the session using ``Session().add``, but this is |
|
253 | 254 | rarely needed: |
|
254 | 255 | |
|
255 | 256 | * When creating a database object by calling the constructor directly, |
|
256 | 257 | it must explicitly be added to the session. |
|
257 | 258 | |
|
258 | 259 | * When creating an object using a factory function (like |
|
259 | 260 | ``create_repo``), the returned object has already (by convention) |
|
260 | 261 | been added to the session, and should not be added again. |
|
261 | 262 | |
|
262 | 263 | * When getting an object from the session (via ``Session().query`` or |
|
263 | 264 | any of the utility functions that look up objects in the database), |
|
264 | 265 | it's already part of the session, and should not be added again. |
|
265 | 266 | SQLAlchemy monitors attribute modifications automatically for all |
|
266 | 267 | objects it knows about and syncs them to the database. |
|
267 | 268 | |
|
268 | 269 | SQLAlchemy also flushes changes to the database automatically; manually |
|
269 | 270 | calling ``Session().flush`` is usually only necessary when the Python |
|
270 | 271 | code needs the database to assign an "auto-increment" primary key ID to |
|
271 | 272 | a freshly created model object (before flushing, the ID attribute will |
|
272 | 273 | be ``None``). |
|
273 | 274 | |
|
274 | 275 | TurboGears2 DebugBar |
|
275 | 276 | ^^^^^^^^^^^^^^^^^^^^ |
|
276 | 277 | |
|
277 | 278 | It is possible to enable the TurboGears2-provided DebugBar_, a toolbar overlayed |
|
278 | 279 | over the Kallithea web interface, allowing you to see: |
|
279 | 280 | |
|
280 | 281 | * timing information of the current request, including profiling information |
|
281 | 282 | * request data, including GET data, POST data, cookies, headers and environment |
|
282 | 283 | variables |
|
283 | 284 | * a list of executed database queries, including timing and result values |
|
284 | 285 | |
|
285 | 286 | DebugBar is only activated when ``debug = true`` is set in the configuration |
|
286 | 287 | file. This is important, because the DebugBar toolbar will be visible for all |
|
287 | 288 | users, and allow them to see information they should not be allowed to see. Like |
|
288 | 289 | is anyway the case for ``debug = true``, do not use this in production! |
|
289 | 290 | |
|
290 | 291 | To enable DebugBar, install ``tgext.debugbar`` and ``kajiki`` (typically via |
|
291 | 292 | ``pip``) and restart Kallithea (in debug mode). |
|
292 | 293 | |
|
293 | 294 | |
|
294 | 295 | "Roadmap" |
|
295 | 296 | --------- |
|
296 | 297 | |
|
297 | 298 | We do not have a road map but are waiting for your contributions. Refer to the |
|
298 | 299 | wiki_ for some ideas of places we might want to go -- contributions in these |
|
299 | 300 | areas are very welcome. |
|
300 | 301 | |
|
301 | 302 | |
|
302 | 303 | Thank you for your contribution! |
|
303 | 304 | -------------------------------- |
|
304 | 305 | |
|
305 | 306 | |
|
306 | 307 | .. _Weblate: http://weblate.org/ |
|
307 | 308 | .. _issue tracking: https://bitbucket.org/conservancy/kallithea/issues?status=new&status=open |
|
308 | 309 | .. _pull requests: https://bitbucket.org/conservancy/kallithea/pull-requests |
|
309 | 310 | .. _bitbucket: http://bitbucket.org/ |
|
310 | 311 | .. _mailing list: http://lists.sfconservancy.org/mailman/listinfo/kallithea-general |
|
311 | 312 | .. _kallithea-general: http://lists.sfconservancy.org/mailman/listinfo/kallithea-general |
|
312 | 313 | .. _Hosted Weblate: https://hosted.weblate.org/projects/kallithea/kallithea/ |
|
313 | 314 | .. _wiki: https://bitbucket.org/conservancy/kallithea/wiki/Home |
|
314 | 315 | .. _DebugBar: https://github.com/TurboGears/tgext.debugbar |
|
315 | 316 | .. _Quick Start: https://www.mercurial-scm.org/wiki/QuickStart |
|
316 | 317 | .. _Beginners Guide: https://www.mercurial-scm.org/wiki/BeginnersGuides |
@@ -1,100 +1,97 b'' | |||
|
1 | 1 | ============ |
|
2 | 2 | Translations |
|
3 | 3 | ============ |
|
4 | 4 | |
|
5 | 5 | Translations are available on Hosted Weblate at the following URL: |
|
6 | 6 | |
|
7 | 7 | https://hosted.weblate.org/projects/kallithea/kallithea/ |
|
8 | 8 | |
|
9 | 9 | Registered users may contribute to the existing languages, or request a new |
|
10 | 10 | language translations. |
|
11 | 11 | |
|
12 | 12 | |
|
13 | 13 | Translating using Weblate |
|
14 | 14 | ------------------------- |
|
15 | 15 | |
|
16 | 16 | Weblate_ offers a simple and easy to use interface featuring glossary, machine |
|
17 | 17 | translation, suggestions based on similar translations in other projects, |
|
18 | 18 | automatic checks etc. Weblate imports the source code tree directly from |
|
19 | 19 | the version control system, and commits edits back from time to time. |
|
20 | 20 | |
|
21 | 21 | When registering at Weblate, make sure you name and email address you prefer to |
|
22 | 22 | be used when your changes are committed. We can and probably will amend changesets |
|
23 | 23 | coming from Weblate, but having things right from the beginning makes things easier. |
|
24 | 24 | |
|
25 | 25 | Weblate performs sanity checks all the time and tries to prevent you from ignoring |
|
26 | 26 | them. Most common mistakes are inconsistent punctuation, whitespaces, missing or extra |
|
27 | 27 | format parameters, untranslated strings copied into the translation. Please perform |
|
28 | 28 | necessary corrections when they're needed, or override the false positives. |
|
29 | 29 | |
|
30 | 30 | |
|
31 | 31 | Merging translations from Weblate |
|
32 | 32 | --------------------------------- |
|
33 | 33 | |
|
34 | 34 | Weblate rebases its changes every time it pulls from our repository. Pulls are triggered |
|
35 | 35 | by a web hook from Our Own Kallithea every time it receives new commits. Usually merging |
|
36 | 36 | the new translations is a straightforward process consisting of a pull from Weblate-hosted |
|
37 | 37 | repository which is available under Data Exports tab in Weblate interface. |
|
38 | 38 | |
|
39 | 39 | Weblate tries to minimise the number of commits, but that's not always work, especially |
|
40 | 40 | when two translators work with different languages at more or less the same time. |
|
41 | 41 | It makes sense sometimes to re-order or fold commits by the same author when they touch |
|
42 | 42 | just the same language translation. That, however, may confuse Weblate sometimes, in |
|
43 | 43 | which case it should be manually convinced it has to discard the commits it created by |
|
44 | 44 | using its administrative interface. |
|
45 | 45 | |
|
46 | 46 | |
|
47 | 47 | Manual creation of a new language translation |
|
48 | 48 | --------------------------------------------- |
|
49 | 49 | |
|
50 | 50 | In the prepared development environment, run the following to ensure |
|
51 | 51 | all translation strings are extracted and up-to-date:: |
|
52 | 52 | |
|
53 | 53 | python2 setup.py extract_messages |
|
54 | 54 | |
|
55 | 55 | Create new language by executing following command:: |
|
56 | 56 | |
|
57 | 57 | python2 setup.py init_catalog -l <new_language_code> |
|
58 | 58 | |
|
59 | 59 | This creates a new translation under directory `kallithea/i18n/<new_language_code>` |
|
60 | 60 | based on the translation template file, `kallithea/i18n/kallithea.pot`. |
|
61 | 61 | |
|
62 | 62 | Edit the new PO file located in `LC_MESSAGES` directory with poedit or your |
|
63 | 63 | favorite PO files editor. After you finished with the translations, check the |
|
64 | 64 | translation file for errors by executing:: |
|
65 | 65 | |
|
66 | 66 | msgfmt -f -c kallithea/i18n/<new_language_code>/LC_MESSAGES/<updated_file.po> |
|
67 | 67 | |
|
68 | 68 | Finally, compile the translations:: |
|
69 | 69 | |
|
70 | 70 | python2 setup.py compile_catalog -l <new_language_code> |
|
71 | 71 | |
|
72 | 72 | |
|
73 | 73 | Updating translations |
|
74 | 74 | --------------------- |
|
75 | 75 | |
|
76 | 76 | Extract the latest versions of strings for translation by running:: |
|
77 | 77 | |
|
78 | 78 | python2 setup.py extract_messages |
|
79 | 79 | |
|
80 | 80 | Update the PO file by doing:: |
|
81 | 81 | |
|
82 | 82 | python2 setup.py update_catalog -l <new_language_code> |
|
83 | 83 | |
|
84 | 84 | Edit the new updated translation file. Repeat all steps after `init_catalog` step from |
|
85 | 85 | new translation instructions |
|
86 | 86 | |
|
87 | 87 | |
|
88 | 88 | Testing translations |
|
89 | 89 | -------------------- |
|
90 | 90 | |
|
91 | Edit kallithea/tests/test.ini file and set lang attribute to:: | |
|
92 | ||
|
93 | lang=<new_language_code> | |
|
94 | ||
|
95 | Run Kallithea tests by executing:: | |
|
91 | Edit `kallithea/tests/conftest.py` and set `i18n.lang` to `<new_language_code>` | |
|
92 | and run Kallithea tests by executing:: | |
|
96 | 93 | |
|
97 | 94 | py.test |
|
98 | 95 | |
|
99 | 96 | |
|
100 | 97 | .. _Weblate: http://weblate.org/ |
@@ -1,210 +1,211 b'' | |||
|
1 | 1 | import os |
|
2 | 2 | import re |
|
3 | 3 | import sys |
|
4 | 4 | import logging |
|
5 | 5 | import pkg_resources |
|
6 | 6 | import time |
|
7 | 7 | |
|
8 | 8 | import formencode |
|
9 | 9 | from paste.deploy import loadwsgi |
|
10 | 10 | from routes.util import URLGenerator |
|
11 | 11 | import pytest |
|
12 | 12 | from pytest_localserver.http import WSGIServer |
|
13 | 13 | |
|
14 | 14 | from kallithea.controllers.root import RootController |
|
15 | 15 | from kallithea.lib import inifile |
|
16 | 16 | from kallithea.lib.utils import repo2db_mapper |
|
17 | 17 | from kallithea.model.user import UserModel |
|
18 | 18 | from kallithea.model.meta import Session |
|
19 | 19 | from kallithea.model.db import Setting, User, UserIpMap |
|
20 | 20 | from kallithea.model.scm import ScmModel |
|
21 | 21 | from kallithea.tests.base import invalidate_all_caches, TEST_USER_REGULAR_LOGIN, TESTS_TMP_PATH, \ |
|
22 | 22 | TEST_USER_ADMIN_LOGIN, TEST_USER_ADMIN_PASS |
|
23 | 23 | import kallithea.tests.base # FIXME: needed for setting testapp instance!!! |
|
24 | 24 | |
|
25 | 25 | from tg.util.webtest import test_context |
|
26 | 26 | |
|
27 | 27 | |
|
28 | 28 | def pytest_configure(): |
|
29 | 29 | os.environ['TZ'] = 'UTC' |
|
30 | 30 | if not kallithea.is_windows: |
|
31 | 31 | time.tzset() # only available on Unix |
|
32 | 32 | |
|
33 | 33 | path = os.getcwd() |
|
34 | 34 | sys.path.insert(0, path) |
|
35 | 35 | pkg_resources.working_set.add_entry(path) |
|
36 | 36 | |
|
37 | 37 | # Disable INFO logging of test database creation, restore with NOTSET |
|
38 | 38 | logging.disable(logging.INFO) |
|
39 | 39 | |
|
40 | 40 | ini_settings = { |
|
41 | 41 | '[server:main]': { |
|
42 | 42 | 'port': '4999', |
|
43 | 43 | }, |
|
44 | 44 | '[app:main]': { |
|
45 | 45 | 'app_instance_uuid': 'test', |
|
46 | 46 | 'show_revision_number': 'true', |
|
47 | 47 | 'beaker.cache.sql_cache_short.expire': '1', |
|
48 | 48 | 'beaker.session.secret': '{74e0cd75-b339-478b-b129-07dd221def1f}', |
|
49 | #'i18n.lang': '', | |
|
49 | 50 | }, |
|
50 | 51 | '[handler_console]': { |
|
51 | 52 | 'formatter': 'color_formatter', |
|
52 | 53 | }, |
|
53 | 54 | # The 'handler_console_sql' block is very similar to the one in |
|
54 | 55 | # development.ini, but without the explicit 'level=DEBUG' setting: |
|
55 | 56 | # it causes duplicate sqlalchemy debug logs, one through |
|
56 | 57 | # handler_console_sql and another through another path. |
|
57 | 58 | '[handler_console_sql]': { |
|
58 | 59 | 'formatter': 'color_formatter_sql', |
|
59 | 60 | }, |
|
60 | 61 | } |
|
61 | 62 | if os.environ.get('TEST_DB'): |
|
62 | 63 | ini_settings['[app:main]']['sqlalchemy.url'] = os.environ.get('TEST_DB') |
|
63 | 64 | |
|
64 | 65 | test_ini_file = os.path.join(TESTS_TMP_PATH, 'test.ini') |
|
65 | 66 | inifile.create(test_ini_file, None, ini_settings) |
|
66 | 67 | |
|
67 | 68 | context = loadwsgi.loadcontext(loadwsgi.APP, 'config:%s' % test_ini_file) |
|
68 | 69 | from kallithea.tests.fixture import create_test_env, create_test_index |
|
69 | 70 | |
|
70 | 71 | # set KALLITHEA_NO_TMP_PATH=1 to disable re-creating the database and test repos |
|
71 | 72 | if not int(os.environ.get('KALLITHEA_NO_TMP_PATH', 0)): |
|
72 | 73 | create_test_env(TESTS_TMP_PATH, context.config()) |
|
73 | 74 | |
|
74 | 75 | # set KALLITHEA_WHOOSH_TEST_DISABLE=1 to disable whoosh index during tests |
|
75 | 76 | if not int(os.environ.get('KALLITHEA_WHOOSH_TEST_DISABLE', 0)): |
|
76 | 77 | create_test_index(TESTS_TMP_PATH, context.config(), True) |
|
77 | 78 | |
|
78 | 79 | kallithea.tests.base.testapp = context.create() |
|
79 | 80 | # do initial repo scan |
|
80 | 81 | repo2db_mapper(ScmModel().repo_scan(TESTS_TMP_PATH)) |
|
81 | 82 | |
|
82 | 83 | logging.disable(logging.NOTSET) |
|
83 | 84 | |
|
84 | 85 | kallithea.tests.base.url = URLGenerator(RootController().mapper, {'HTTP_HOST': 'example.com'}) |
|
85 | 86 | |
|
86 | 87 | # set fixed language for form messages, regardless of environment settings |
|
87 | 88 | formencode.api.set_stdtranslation(languages=[]) |
|
88 | 89 | |
|
89 | 90 | |
|
90 | 91 | @pytest.fixture |
|
91 | 92 | def create_test_user(): |
|
92 | 93 | """Provide users that automatically disappear after test is over.""" |
|
93 | 94 | test_user_ids = [] |
|
94 | 95 | |
|
95 | 96 | def _create_test_user(user_form): |
|
96 | 97 | user = UserModel().create(user_form) |
|
97 | 98 | test_user_ids.append(user.user_id) |
|
98 | 99 | return user |
|
99 | 100 | yield _create_test_user |
|
100 | 101 | for user_id in test_user_ids: |
|
101 | 102 | UserModel().delete(user_id) |
|
102 | 103 | Session().commit() |
|
103 | 104 | |
|
104 | 105 | |
|
105 | 106 | def _set_settings(*kvtseq): |
|
106 | 107 | session = Session() |
|
107 | 108 | for kvt in kvtseq: |
|
108 | 109 | assert len(kvt) in (2, 3) |
|
109 | 110 | k = kvt[0] |
|
110 | 111 | v = kvt[1] |
|
111 | 112 | t = kvt[2] if len(kvt) == 3 else 'unicode' |
|
112 | 113 | Setting.create_or_update(k, v, t) |
|
113 | 114 | session.commit() |
|
114 | 115 | |
|
115 | 116 | |
|
116 | 117 | @pytest.fixture |
|
117 | 118 | def set_test_settings(): |
|
118 | 119 | """Restore settings after test is over.""" |
|
119 | 120 | # Save settings. |
|
120 | 121 | settings_snapshot = [ |
|
121 | 122 | (s.app_settings_name, s.app_settings_value, s.app_settings_type) |
|
122 | 123 | for s in Setting.query().all()] |
|
123 | 124 | yield _set_settings |
|
124 | 125 | # Restore settings. |
|
125 | 126 | session = Session() |
|
126 | 127 | keys = frozenset(k for (k, v, t) in settings_snapshot) |
|
127 | 128 | for s in Setting.query().all(): |
|
128 | 129 | if s.app_settings_name not in keys: |
|
129 | 130 | session.delete(s) |
|
130 | 131 | for k, v, t in settings_snapshot: |
|
131 | 132 | if t == 'list' and hasattr(v, '__iter__'): |
|
132 | 133 | v = ','.join(v) # Quirk: must format list value manually. |
|
133 | 134 | Setting.create_or_update(k, v, t) |
|
134 | 135 | session.commit() |
|
135 | 136 | |
|
136 | 137 | |
|
137 | 138 | @pytest.fixture |
|
138 | 139 | def auto_clear_ip_permissions(): |
|
139 | 140 | """Fixture that provides nothing but clearing IP permissions upon test |
|
140 | 141 | exit. This clearing is needed to avoid other test failing to make fake http |
|
141 | 142 | accesses.""" |
|
142 | 143 | yield |
|
143 | 144 | # cleanup |
|
144 | 145 | user_model = UserModel() |
|
145 | 146 | |
|
146 | 147 | user_ids = [] |
|
147 | 148 | user_ids.append(User.get_default_user().user_id) |
|
148 | 149 | user_ids.append(User.get_by_username(TEST_USER_REGULAR_LOGIN).user_id) |
|
149 | 150 | |
|
150 | 151 | for user_id in user_ids: |
|
151 | 152 | for ip in UserIpMap.query().filter(UserIpMap.user_id == user_id): |
|
152 | 153 | user_model.delete_extra_ip(user_id, ip.ip_id) |
|
153 | 154 | |
|
154 | 155 | # IP permissions are cached, need to invalidate this cache explicitly |
|
155 | 156 | invalidate_all_caches() |
|
156 | 157 | |
|
157 | 158 | |
|
158 | 159 | @pytest.fixture |
|
159 | 160 | def test_context_fixture(app_fixture): |
|
160 | 161 | """ |
|
161 | 162 | Encompass the entire test using this fixture in a test_context, |
|
162 | 163 | making sure that certain functionality still works even if no call to |
|
163 | 164 | self.app.get/post has been made. |
|
164 | 165 | The typical error message indicating you need a test_context is: |
|
165 | 166 | TypeError: No object (name: context) has been registered for this thread |
|
166 | 167 | |
|
167 | 168 | The standard way to fix this is simply using the test_context context |
|
168 | 169 | manager directly inside your test: |
|
169 | 170 | with test_context(self.app): |
|
170 | 171 | <actions> |
|
171 | 172 | but if test setup code (xUnit-style or pytest fixtures) also needs to be |
|
172 | 173 | executed inside the test context, that method is not possible. |
|
173 | 174 | Even if there is no such setup code, the fixture may reduce code complexity |
|
174 | 175 | if the entire test needs to run inside a test context. |
|
175 | 176 | |
|
176 | 177 | To apply this fixture (like any other fixture) to all test methods of a |
|
177 | 178 | class, use the following class decorator: |
|
178 | 179 | @pytest.mark.usefixtures("test_context_fixture") |
|
179 | 180 | class TestFoo(TestController): |
|
180 | 181 | ... |
|
181 | 182 | """ |
|
182 | 183 | with test_context(app_fixture): |
|
183 | 184 | yield |
|
184 | 185 | |
|
185 | 186 | |
|
186 | 187 | class MyWSGIServer(WSGIServer): |
|
187 | 188 | def repo_url(self, repo_name, username=TEST_USER_ADMIN_LOGIN, password=TEST_USER_ADMIN_PASS): |
|
188 | 189 | """Return URL to repo on this web server.""" |
|
189 | 190 | host, port = self.server_address |
|
190 | 191 | proto = 'http' if self._server.ssl_context is None else 'https' |
|
191 | 192 | auth = '' |
|
192 | 193 | if username is not None: |
|
193 | 194 | auth = username |
|
194 | 195 | if password is not None: |
|
195 | 196 | auth += ':' + password |
|
196 | 197 | if auth: |
|
197 | 198 | auth += '@' |
|
198 | 199 | return '%s://%s%s:%s/%s' % (proto, auth, host, port, repo_name) |
|
199 | 200 | |
|
200 | 201 | |
|
201 | 202 | @pytest.yield_fixture(scope="session") |
|
202 | 203 | def webserver(): |
|
203 | 204 | """Start web server while tests are running. |
|
204 | 205 | Useful for debugging and necessary for vcs operation tests.""" |
|
205 | 206 | server = MyWSGIServer(application=kallithea.tests.base.testapp) |
|
206 | 207 | server.start() |
|
207 | 208 | |
|
208 | 209 | yield server |
|
209 | 210 | |
|
210 | 211 | server.stop() |
@@ -1,74 +1,72 b'' | |||
|
1 | 1 | #!/usr/bin/env python2 |
|
2 | 2 | """ |
|
3 | Based on kallithea/lib/paster_commands/template.ini.mako, generate | |
|
4 | development.ini | |
|
5 | kallithea/tests/test.ini | |
|
3 | Based on kallithea/lib/paster_commands/template.ini.mako, generate development.ini | |
|
6 | 4 | """ |
|
7 | 5 | |
|
8 | 6 | import re |
|
9 | 7 | |
|
10 | 8 | from kallithea.lib import inifile |
|
11 | 9 | |
|
12 | 10 | # files to be generated from the mako template |
|
13 | 11 | ini_files = [ |
|
14 | 12 | ('development.ini', |
|
15 | 13 | { |
|
16 | 14 | '[server:main]': { |
|
17 | 15 | 'host': '0.0.0.0', |
|
18 | 16 | }, |
|
19 | 17 | '[app:main]': { |
|
20 | 18 | 'debug': 'true', |
|
21 | 19 | 'app_instance_uuid': 'development-not-secret', |
|
22 | 20 | 'beaker.session.secret': 'development-not-secret', |
|
23 | 21 | }, |
|
24 | 22 | '[handler_console]': { |
|
25 | 23 | 'formatter': 'color_formatter', |
|
26 | 24 | }, |
|
27 | 25 | '[handler_console_sql]': { |
|
28 | 26 | 'formatter': 'color_formatter_sql', |
|
29 | 27 | }, |
|
30 | 28 | '[logger_routes]': { |
|
31 | 29 | 'level': 'DEBUG', |
|
32 | 30 | }, |
|
33 | 31 | '[logger_beaker]': { |
|
34 | 32 | 'level': 'DEBUG', |
|
35 | 33 | }, |
|
36 | 34 | '[logger_templates]': { |
|
37 | 35 | 'level': 'INFO', |
|
38 | 36 | }, |
|
39 | 37 | '[logger_kallithea]': { |
|
40 | 38 | 'level': 'DEBUG', |
|
41 | 39 | }, |
|
42 | 40 | '[logger_tg]': { |
|
43 | 41 | 'level': 'DEBUG', |
|
44 | 42 | }, |
|
45 | 43 | '[logger_gearbox]': { |
|
46 | 44 | 'level': 'DEBUG', |
|
47 | 45 | }, |
|
48 | 46 | '[logger_whoosh_indexer]': { |
|
49 | 47 | 'level': 'DEBUG', |
|
50 | 48 | }, |
|
51 | 49 | }, |
|
52 | 50 | ), |
|
53 | 51 | ] |
|
54 | 52 | |
|
55 | 53 | |
|
56 | 54 | def main(): |
|
57 | 55 | # make sure all mako lines starting with '#' (the '##' comments) are marked up as <text> |
|
58 | 56 | makofile = inifile.template_file |
|
59 | 57 | print 'reading:', makofile |
|
60 | 58 | mako_org = open(makofile).read() |
|
61 | 59 | mako_no_text_markup = re.sub(r'</?%text>', '', mako_org) |
|
62 | 60 | mako_marked_up = re.sub(r'\n(##.*)', r'\n<%text>\1</%text>', mako_no_text_markup, flags=re.MULTILINE) |
|
63 | 61 | if mako_marked_up != mako_org: |
|
64 | 62 | print 'writing:', makofile |
|
65 | 63 | open(makofile, 'w').write(mako_marked_up) |
|
66 | 64 | |
|
67 | 65 | # create ini files |
|
68 | 66 | for fn, settings in ini_files: |
|
69 | 67 | print 'updating:', fn |
|
70 | 68 | inifile.create(fn, None, settings) |
|
71 | 69 | |
|
72 | 70 | |
|
73 | 71 | if __name__ == '__main__': |
|
74 | 72 | main() |
General Comments 0
You need to be logged in to leave comments.
Login now