##// END OF EJS Templates
test: add warning about not mounting /tmp noexec
domruf -
r6570:3af2dea7 default
parent child Browse files
Show More
@@ -1,247 +1,252 b''
1 .. _contributing:
1 .. _contributing:
2
2
3 =========================
3 =========================
4 Contributing to Kallithea
4 Contributing to Kallithea
5 =========================
5 =========================
6
6
7 Kallithea is developed and maintained by its users. Please join us and scratch
7 Kallithea is developed and maintained by its users. Please join us and scratch
8 your own itch.
8 your own itch.
9
9
10
10
11 Infrastructure
11 Infrastructure
12 --------------
12 --------------
13
13
14 The main repository is hosted on Our Own Kallithea (aka OOK) at
14 The main repository is hosted on Our Own Kallithea (aka OOK) at
15 https://kallithea-scm.org/repos/kallithea/, our self-hosted instance
15 https://kallithea-scm.org/repos/kallithea/, our self-hosted instance
16 of Kallithea.
16 of Kallithea.
17
17
18 For now, we use Bitbucket_ for `pull requests`_ and `issue tracking`_. The
18 For now, we use Bitbucket_ for `pull requests`_ and `issue tracking`_. The
19 issue tracker is for tracking bugs, not for support, discussion, or ideas --
19 issue tracker is for tracking bugs, not for support, discussion, or ideas --
20 please use the `mailing list`_ or :ref:`IRC <readme>` to reach the community.
20 please use the `mailing list`_ or :ref:`IRC <readme>` to reach the community.
21
21
22 We use Weblate_ to translate the user interface messages into languages other
22 We use Weblate_ to translate the user interface messages into languages other
23 than English. Join our project on `Hosted Weblate`_ to help us.
23 than English. Join our project on `Hosted Weblate`_ to help us.
24 To register, you can use your Bitbucket or GitHub account. See :ref:`translations`
24 To register, you can use your Bitbucket or GitHub account. See :ref:`translations`
25 for more details.
25 for more details.
26
26
27
27
28 Getting started
28 Getting started
29 ---------------
29 ---------------
30
30
31 To get started with development::
31 To get started with development::
32
32
33 hg clone https://kallithea-scm.org/repos/kallithea
33 hg clone https://kallithea-scm.org/repos/kallithea
34 cd kallithea
34 cd kallithea
35 virtualenv ../kallithea-venv
35 virtualenv ../kallithea-venv
36 source ../kallithea-venv/bin/activate
36 source ../kallithea-venv/bin/activate
37 pip install --upgrade pip setuptools
37 pip install --upgrade pip setuptools
38 pip install -e .
38 pip install -e .
39 gearbox make-config my.ini
39 gearbox make-config my.ini
40 gearbox setup-db -c my.ini --user=user --email=user@example.com --password=password --repos=/tmp
40 gearbox setup-db -c my.ini --user=user --email=user@example.com --password=password --repos=/tmp
41 gearbox serve -c my.ini --reload &
41 gearbox serve -c my.ini --reload &
42 firefox http://127.0.0.1:5000/
42 firefox http://127.0.0.1:5000/
43
43
44 You can also start out by forking https://bitbucket.org/conservancy/kallithea
44 You can also start out by forking https://bitbucket.org/conservancy/kallithea
45 on Bitbucket_ and create a local clone of your own fork.
45 on Bitbucket_ and create a local clone of your own fork.
46
46
47
47
48 Running tests
48 Running tests
49 -------------
49 -------------
50
50
51 After finishing your changes make sure all tests pass cleanly. Install the test
51 After finishing your changes make sure all tests pass cleanly. Install the test
52 dependencies, then run the testsuite by invoking ``py.test`` from the
52 dependencies, then run the testsuite by invoking ``py.test`` from the
53 project root::
53 project root::
54
54
55 pip install -r dev_requirements.txt
55 pip install -r dev_requirements.txt
56 py.test
56 py.test
57
57
58 Note that testing on Python 2.6 also requires ``unittest2``.
58 Note that testing on Python 2.6 also requires ``unittest2``.
59
59
60 Note that on unix systems, the temporary directory (``/tmp`` or where
61 ``$TMPDIR`` points) must allow executable files; Git hooks must be executable,
62 and the test suite creates repositories in the temporary directory. Linux
63 systems with /tmp mounted noexec will thus fail.
64
60 You can also use ``tox`` to run the tests with all supported Python versions
65 You can also use ``tox`` to run the tests with all supported Python versions
61 (currently Python 2.6--2.7).
66 (currently Python 2.6--2.7).
62
67
63 When running tests, Kallithea uses `kallithea/tests/test.ini` and populates the
68 When running tests, Kallithea uses `kallithea/tests/test.ini` and populates the
64 SQLite database specified there.
69 SQLite database specified there.
65
70
66 It is possible to avoid recreating the full test database on each invocation of
71 It is possible to avoid recreating the full test database on each invocation of
67 the tests, thus eliminating the initial delay. To achieve this, run the tests as::
72 the tests, thus eliminating the initial delay. To achieve this, run the tests as::
68
73
69 gearbox serve -c kallithea/tests/test.ini --pid-file=test.pid --daemon
74 gearbox serve -c kallithea/tests/test.ini --pid-file=test.pid --daemon
70 KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test
75 KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test
71 kill -9 $(cat test.pid)
76 kill -9 $(cat test.pid)
72
77
73 In these commands, the following variables are used::
78 In these commands, the following variables are used::
74
79
75 KALLITHEA_WHOOSH_TEST_DISABLE=1 - skip whoosh index building and tests
80 KALLITHEA_WHOOSH_TEST_DISABLE=1 - skip whoosh index building and tests
76 KALLITHEA_NO_TMP_PATH=1 - disable new temp path for tests, used mostly for testing_vcs_operations
81 KALLITHEA_NO_TMP_PATH=1 - disable new temp path for tests, used mostly for testing_vcs_operations
77
82
78 You can run individual tests by specifying their path as argument to py.test.
83 You can run individual tests by specifying their path as argument to py.test.
79 py.test also has many more options, see `py.test -h`. Some useful options
84 py.test also has many more options, see `py.test -h`. Some useful options
80 are::
85 are::
81
86
82 -k EXPRESSION only run tests which match the given substring
87 -k EXPRESSION only run tests which match the given substring
83 expression. An expression is a python evaluable
88 expression. An expression is a python evaluable
84 expression where all names are substring-matched
89 expression where all names are substring-matched
85 against test names and their parent classes. Example:
90 against test names and their parent classes. Example:
86 -x, --exitfirst exit instantly on first error or failed test.
91 -x, --exitfirst exit instantly on first error or failed test.
87 --lf rerun only the tests that failed at the last run (or
92 --lf rerun only the tests that failed at the last run (or
88 all if none failed)
93 all if none failed)
89 --ff run all tests but run the last failures first. This
94 --ff run all tests but run the last failures first. This
90 may re-order tests and thus lead to repeated fixture
95 may re-order tests and thus lead to repeated fixture
91 setup/teardown
96 setup/teardown
92 --pdb start the interactive Python debugger on errors.
97 --pdb start the interactive Python debugger on errors.
93 -s, --capture=no don't capture stdout (any stdout output will be
98 -s, --capture=no don't capture stdout (any stdout output will be
94 printed immediately)
99 printed immediately)
95
100
96
101
97 Contribution guidelines
102 Contribution guidelines
98 -----------------------
103 -----------------------
99
104
100 Kallithea is GPLv3 and we assume all contributions are made by the
105 Kallithea is GPLv3 and we assume all contributions are made by the
101 committer/contributor and under GPLv3 unless explicitly stated. We do care a
106 committer/contributor and under GPLv3 unless explicitly stated. We do care a
102 lot about preservation of copyright and license information for existing code
107 lot about preservation of copyright and license information for existing code
103 that is brought into the project.
108 that is brought into the project.
104
109
105 Contributions will be accepted in most formats -- such as pull requests on
110 Contributions will be accepted in most formats -- such as pull requests on
106 Bitbucket, something hosted on your own Kallithea instance, or patches sent by
111 Bitbucket, something hosted on your own Kallithea instance, or patches sent by
107 email to the `kallithea-general`_ mailing list.
112 email to the `kallithea-general`_ mailing list.
108
113
109 When contributing via Bitbucket, please make your fork of
114 When contributing via Bitbucket, please make your fork of
110 https://bitbucket.org/conservancy/kallithea/ `non-publishing`_ -- it is one of
115 https://bitbucket.org/conservancy/kallithea/ `non-publishing`_ -- it is one of
111 the settings on "Repository details" page. This ensures your commits are in
116 the settings on "Repository details" page. This ensures your commits are in
112 "draft" phase and makes it easier for you to address feedback and for project
117 "draft" phase and makes it easier for you to address feedback and for project
113 maintainers to integrate your changes.
118 maintainers to integrate your changes.
114
119
115 .. _non-publishing: https://www.mercurial-scm.org/wiki/Phases#Publishing_Repository
120 .. _non-publishing: https://www.mercurial-scm.org/wiki/Phases#Publishing_Repository
116
121
117 Make sure to test your changes both manually and with the automatic tests
122 Make sure to test your changes both manually and with the automatic tests
118 before posting.
123 before posting.
119
124
120 We care about quality and review and keeping a clean repository history. We
125 We care about quality and review and keeping a clean repository history. We
121 might give feedback that requests polishing contributions until they are
126 might give feedback that requests polishing contributions until they are
122 "perfect". We might also rebase and collapse and make minor adjustments to your
127 "perfect". We might also rebase and collapse and make minor adjustments to your
123 changes when we apply them.
128 changes when we apply them.
124
129
125 We try to make sure we have consensus on the direction the project is taking.
130 We try to make sure we have consensus on the direction the project is taking.
126 Everything non-sensitive should be discussed in public -- preferably on the
131 Everything non-sensitive should be discussed in public -- preferably on the
127 mailing list. We aim at having all non-trivial changes reviewed by at least
132 mailing list. We aim at having all non-trivial changes reviewed by at least
128 one other core developer before pushing. Obvious non-controversial changes will
133 one other core developer before pushing. Obvious non-controversial changes will
129 be handled more casually.
134 be handled more casually.
130
135
131 For now we just have one official branch ("default") and will keep it so stable
136 For now we just have one official branch ("default") and will keep it so stable
132 that it can be (and is) used in production. Experimental changes should live
137 that it can be (and is) used in production. Experimental changes should live
133 elsewhere (for example in a pull request) until they are ready.
138 elsewhere (for example in a pull request) until they are ready.
134
139
135
140
136 Coding guidelines
141 Coding guidelines
137 -----------------
142 -----------------
138
143
139 We don't have a formal coding/formatting standard. We are currently using a mix
144 We don't have a formal coding/formatting standard. We are currently using a mix
140 of Mercurial's (https://www.mercurial-scm.org/wiki/CodingStyle), pep8, and
145 of Mercurial's (https://www.mercurial-scm.org/wiki/CodingStyle), pep8, and
141 consistency with existing code. Run ``scripts/run-all-cleanup`` before
146 consistency with existing code. Run ``scripts/run-all-cleanup`` before
142 committing to ensure some basic code formatting consistency.
147 committing to ensure some basic code formatting consistency.
143
148
144 We support both Python 2.6.x and 2.7.x and nothing else. For now we don't care
149 We support both Python 2.6.x and 2.7.x and nothing else. For now we don't care
145 about Python 3 compatibility.
150 about Python 3 compatibility.
146
151
147 We try to support the most common modern web browsers. IE9 is still supported
152 We try to support the most common modern web browsers. IE9 is still supported
148 to the extent it is feasible, IE8 is not.
153 to the extent it is feasible, IE8 is not.
149
154
150 We primarily support Linux and OS X on the server side but Windows should also work.
155 We primarily support Linux and OS X on the server side but Windows should also work.
151
156
152 HTML templates should use 2 spaces for indentation ... but be pragmatic. We
157 HTML templates should use 2 spaces for indentation ... but be pragmatic. We
153 should use templates cleverly and avoid duplication. We should use reasonable
158 should use templates cleverly and avoid duplication. We should use reasonable
154 semantic markup with element classes and IDs that can be used for styling and testing.
159 semantic markup with element classes and IDs that can be used for styling and testing.
155 We should only use inline styles in places where it really is semantic (such as
160 We should only use inline styles in places where it really is semantic (such as
156 ``display: none``).
161 ``display: none``).
157
162
158 JavaScript must use ``;`` between/after statements. Indentation 4 spaces. Inline
163 JavaScript must use ``;`` between/after statements. Indentation 4 spaces. Inline
159 multiline functions should be indented two levels -- one for the ``()`` and one for
164 multiline functions should be indented two levels -- one for the ``()`` and one for
160 ``{}``.
165 ``{}``.
161 Variables holding jQuery objects should be named with a leading ``$``.
166 Variables holding jQuery objects should be named with a leading ``$``.
162
167
163 Commit messages should have a leading short line summarizing the changes. For
168 Commit messages should have a leading short line summarizing the changes. For
164 bug fixes, put ``(Issue #123)`` at the end of this line.
169 bug fixes, put ``(Issue #123)`` at the end of this line.
165
170
166 Use American English grammar and spelling overall. Use `English title case`_ for
171 Use American English grammar and spelling overall. Use `English title case`_ for
167 page titles, button labels, headers, and 'labels' for fields in forms.
172 page titles, button labels, headers, and 'labels' for fields in forms.
168
173
169 .. _English title case: https://en.wikipedia.org/wiki/Capitalization#Title_case
174 .. _English title case: https://en.wikipedia.org/wiki/Capitalization#Title_case
170
175
171 Template helpers (that is, everything in ``kallithea.lib.helpers``)
176 Template helpers (that is, everything in ``kallithea.lib.helpers``)
172 should only be referenced from templates. If you need to call a
177 should only be referenced from templates. If you need to call a
173 helper from the Python code, consider moving the function somewhere
178 helper from the Python code, consider moving the function somewhere
174 else (e.g. to the model).
179 else (e.g. to the model).
175
180
176 Notes on the SQLAlchemy session
181 Notes on the SQLAlchemy session
177 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
182 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
178
183
179 Each HTTP request runs inside an independent SQLAlchemy session (as well
184 Each HTTP request runs inside an independent SQLAlchemy session (as well
180 as in an independent database transaction). Database model objects
185 as in an independent database transaction). Database model objects
181 (almost) always belong to a particular SQLAlchemy session, which means
186 (almost) always belong to a particular SQLAlchemy session, which means
182 that SQLAlchemy will ensure that they're kept in sync with the database
187 that SQLAlchemy will ensure that they're kept in sync with the database
183 (but also means that they cannot be shared across requests).
188 (but also means that they cannot be shared across requests).
184
189
185 Objects can be added to the session using ``Session().add``, but this is
190 Objects can be added to the session using ``Session().add``, but this is
186 rarely needed:
191 rarely needed:
187
192
188 * When creating a database object by calling the constructor directly,
193 * When creating a database object by calling the constructor directly,
189 it must explicitly be added to the session.
194 it must explicitly be added to the session.
190
195
191 * When creating an object using a factory function (like
196 * When creating an object using a factory function (like
192 ``create_repo``), the returned object has already (by convention)
197 ``create_repo``), the returned object has already (by convention)
193 been added to the session, and should not be added again.
198 been added to the session, and should not be added again.
194
199
195 * When getting an object from the session (via ``Session().query`` or
200 * When getting an object from the session (via ``Session().query`` or
196 any of the utility functions that look up objects in the database),
201 any of the utility functions that look up objects in the database),
197 it's already part of the session, and should not be added again.
202 it's already part of the session, and should not be added again.
198 SQLAlchemy monitors attribute modifications automatically for all
203 SQLAlchemy monitors attribute modifications automatically for all
199 objects it knows about and syncs them to the database.
204 objects it knows about and syncs them to the database.
200
205
201 SQLAlchemy also flushes changes to the database automatically; manually
206 SQLAlchemy also flushes changes to the database automatically; manually
202 calling ``Session().flush`` is usually only necessary when the Python
207 calling ``Session().flush`` is usually only necessary when the Python
203 code needs the database to assign an "auto-increment" primary key ID to
208 code needs the database to assign an "auto-increment" primary key ID to
204 a freshly created model object (before flushing, the ID attribute will
209 a freshly created model object (before flushing, the ID attribute will
205 be ``None``).
210 be ``None``).
206
211
207 TurboGears2 DebugBar
212 TurboGears2 DebugBar
208 ^^^^^^^^^^^^^^^^^^^^
213 ^^^^^^^^^^^^^^^^^^^^
209
214
210 It is possible to enable the TurboGears2-provided DebugBar_, a toolbar overlayed
215 It is possible to enable the TurboGears2-provided DebugBar_, a toolbar overlayed
211 over the Kallithea web interface, allowing you to see:
216 over the Kallithea web interface, allowing you to see:
212
217
213 * timing information of the current request, including profiling information
218 * timing information of the current request, including profiling information
214 * request data, including GET data, POST data, cookies, headers and environment
219 * request data, including GET data, POST data, cookies, headers and environment
215 variables
220 variables
216 * a list of executed database queries, including timing and result values
221 * a list of executed database queries, including timing and result values
217
222
218 DebugBar is only activated when ``debug = true`` is set in the configuration
223 DebugBar is only activated when ``debug = true`` is set in the configuration
219 file. This is important, because the DebugBar toolbar will be visible for all
224 file. This is important, because the DebugBar toolbar will be visible for all
220 users, and allow them to see information they should not be allowed to see. Like
225 users, and allow them to see information they should not be allowed to see. Like
221 is anyway the case for ``debug = true``, do not use this in production!
226 is anyway the case for ``debug = true``, do not use this in production!
222
227
223 To enable DebugBar, install ``tgext.debugbar`` and ``kajiki`` (typically via
228 To enable DebugBar, install ``tgext.debugbar`` and ``kajiki`` (typically via
224 ``pip``) and restart Kallithea (in debug mode).
229 ``pip``) and restart Kallithea (in debug mode).
225
230
226
231
227 "Roadmap"
232 "Roadmap"
228 ---------
233 ---------
229
234
230 We do not have a road map but are waiting for your contributions. Refer to the
235 We do not have a road map but are waiting for your contributions. Refer to the
231 wiki_ for some ideas of places we might want to go -- contributions in these
236 wiki_ for some ideas of places we might want to go -- contributions in these
232 areas are very welcome.
237 areas are very welcome.
233
238
234
239
235 Thank you for your contribution!
240 Thank you for your contribution!
236 --------------------------------
241 --------------------------------
237
242
238
243
239 .. _Weblate: http://weblate.org/
244 .. _Weblate: http://weblate.org/
240 .. _issue tracking: https://bitbucket.org/conservancy/kallithea/issues?status=new&status=open
245 .. _issue tracking: https://bitbucket.org/conservancy/kallithea/issues?status=new&status=open
241 .. _pull requests: https://bitbucket.org/conservancy/kallithea/pull-requests
246 .. _pull requests: https://bitbucket.org/conservancy/kallithea/pull-requests
242 .. _bitbucket: http://bitbucket.org/
247 .. _bitbucket: http://bitbucket.org/
243 .. _mailing list: http://lists.sfconservancy.org/mailman/listinfo/kallithea-general
248 .. _mailing list: http://lists.sfconservancy.org/mailman/listinfo/kallithea-general
244 .. _kallithea-general: http://lists.sfconservancy.org/mailman/listinfo/kallithea-general
249 .. _kallithea-general: http://lists.sfconservancy.org/mailman/listinfo/kallithea-general
245 .. _Hosted Weblate: https://hosted.weblate.org/projects/kallithea/kallithea/
250 .. _Hosted Weblate: https://hosted.weblate.org/projects/kallithea/kallithea/
246 .. _wiki: https://bitbucket.org/conservancy/kallithea/wiki/Home
251 .. _wiki: https://bitbucket.org/conservancy/kallithea/wiki/Home
247 .. _DebugBar: https://github.com/TurboGears/tgext.debugbar
252 .. _DebugBar: https://github.com/TurboGears/tgext.debugbar
@@ -1,568 +1,571 b''
1 # -*- coding: utf-8 -*-
1 # -*- coding: utf-8 -*-
2 # This program is free software: you can redistribute it and/or modify
2 # This program is free software: you can redistribute it and/or modify
3 # it under the terms of the GNU General Public License as published by
3 # it under the terms of the GNU General Public License as published by
4 # the Free Software Foundation, either version 3 of the License, or
4 # the Free Software Foundation, either version 3 of the License, or
5 # (at your option) any later version.
5 # (at your option) any later version.
6 #
6 #
7 # This program is distributed in the hope that it will be useful,
7 # This program is distributed in the hope that it will be useful,
8 # but WITHOUT ANY WARRANTY; without even the implied warranty of
8 # but WITHOUT ANY WARRANTY; without even the implied warranty of
9 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
9 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
10 # GNU General Public License for more details.
10 # GNU General Public License for more details.
11 #
11 #
12 # You should have received a copy of the GNU General Public License
12 # You should have received a copy of the GNU General Public License
13 # along with this program. If not, see <http://www.gnu.org/licenses/>.
13 # along with this program. If not, see <http://www.gnu.org/licenses/>.
14 """
14 """
15 kallithea.tests.other.manual_test_vcs_operations
15 kallithea.tests.other.manual_test_vcs_operations
16 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
16 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
17
17
18 Test suite for making push/pull operations.
18 Test suite for making push/pull operations.
19
19
20 Run it in two terminals::
20 Run it in two terminals::
21 gearbox serve -c kallithea/tests/test.ini
21 gearbox serve -c kallithea/tests/test.ini
22 KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test kallithea/tests/other/manual_test_vcs_operations.py
22 KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test kallithea/tests/other/manual_test_vcs_operations.py
23
23
24 You must have git > 1.8.1 for tests to work fine
24 You must have git > 1.8.1 for tests to work fine
25
25
26 This file was forked by the Kallithea project in July 2014.
26 This file was forked by the Kallithea project in July 2014.
27 Original author and date, and relevant copyright and licensing information is below:
27 Original author and date, and relevant copyright and licensing information is below:
28 :created_on: Dec 30, 2010
28 :created_on: Dec 30, 2010
29 :author: marcink
29 :author: marcink
30 :copyright: (c) 2013 RhodeCode GmbH, and others.
30 :copyright: (c) 2013 RhodeCode GmbH, and others.
31 :license: GPLv3, see LICENSE.md for more details.
31 :license: GPLv3, see LICENSE.md for more details.
32
32
33 """
33 """
34
34
35 import os
35 import os
36 import re
36 import re
37 import tempfile
37 import tempfile
38 import time
38 import time
39 import pytest
39 import pytest
40
40
41 from tempfile import _RandomNameSequence
41 from tempfile import _RandomNameSequence
42 from subprocess import Popen, PIPE
42 from subprocess import Popen, PIPE
43
43
44 from kallithea.tests.base import *
44 from kallithea.tests.base import *
45 from kallithea.tests.fixture import Fixture
45 from kallithea.tests.fixture import Fixture
46 from kallithea.model.db import User, Repository, UserIpMap, CacheInvalidation
46 from kallithea.model.db import User, Repository, UserIpMap, CacheInvalidation
47 from kallithea.model.meta import Session
47 from kallithea.model.meta import Session
48 from kallithea.model.repo import RepoModel
48 from kallithea.model.repo import RepoModel
49 from kallithea.model.user import UserModel
49 from kallithea.model.user import UserModel
50
50
51 DEBUG = True
51 DEBUG = True
52 HOST = '127.0.0.1:4999' # test host
52 HOST = '127.0.0.1:4999' # test host
53
53
54 fixture = Fixture()
54 fixture = Fixture()
55
55
56
56
57 class Command(object):
57 class Command(object):
58
58
59 def __init__(self, cwd):
59 def __init__(self, cwd):
60 self.cwd = cwd
60 self.cwd = cwd
61
61
62 def execute(self, cmd, *args, **environ):
62 def execute(self, cmd, *args, **environ):
63 """
63 """
64 Runs command on the system with given ``args``.
64 Runs command on the system with given ``args``.
65 """
65 """
66
66
67 command = cmd + ' ' + ' '.join(args)
67 command = cmd + ' ' + ' '.join(args)
68 ignoreReturnCode = environ.pop('ignoreReturnCode', False)
68 ignoreReturnCode = environ.pop('ignoreReturnCode', False)
69 if DEBUG:
69 if DEBUG:
70 print '*** CMD %s ***' % command
70 print '*** CMD %s ***' % command
71 testenv = dict(os.environ)
71 testenv = dict(os.environ)
72 testenv['LANG'] = 'en_US.UTF-8'
72 testenv['LANG'] = 'en_US.UTF-8'
73 testenv['LANGUAGE'] = 'en_US:en'
73 testenv['LANGUAGE'] = 'en_US:en'
74 testenv.update(environ)
74 testenv.update(environ)
75 p = Popen(command, shell=True, stdout=PIPE, stderr=PIPE, cwd=self.cwd, env=testenv)
75 p = Popen(command, shell=True, stdout=PIPE, stderr=PIPE, cwd=self.cwd, env=testenv)
76 stdout, stderr = p.communicate()
76 stdout, stderr = p.communicate()
77 if DEBUG:
77 if DEBUG:
78 if stdout:
78 if stdout:
79 print 'stdout:', repr(stdout)
79 print 'stdout:', repr(stdout)
80 if stderr:
80 if stderr:
81 print 'stderr:', repr(stderr)
81 print 'stderr:', repr(stderr)
82 if not ignoreReturnCode:
82 if not ignoreReturnCode:
83 assert p.returncode == 0
83 assert p.returncode == 0
84 return stdout, stderr
84 return stdout, stderr
85
85
86
86
87 def _get_tmp_dir():
87 def _get_tmp_dir():
88 return tempfile.mkdtemp(prefix='rc_integration_test')
88 return tempfile.mkdtemp(prefix='rc_integration_test')
89
89
90
90
91 def _construct_url(repo, **kwargs):
91 def _construct_url(repo, **kwargs):
92 """Return a clone url for the provided repo path.
92 """Return a clone url for the provided repo path.
93 Optional named parameters: user, passwd and host."""
93 Optional named parameters: user, passwd and host."""
94 params = {
94 params = {
95 'user': TEST_USER_ADMIN_LOGIN,
95 'user': TEST_USER_ADMIN_LOGIN,
96 'passwd': TEST_USER_ADMIN_PASS,
96 'passwd': TEST_USER_ADMIN_PASS,
97 'host': HOST,
97 'host': HOST,
98 'cloned_repo': repo,
98 'cloned_repo': repo,
99 }
99 }
100 params.update(**kwargs)
100 params.update(**kwargs)
101 if params['user'] and params['passwd']:
101 if params['user'] and params['passwd']:
102 _url = 'http://%(user)s:%(passwd)s@%(host)s/%(cloned_repo)s' % params
102 _url = 'http://%(user)s:%(passwd)s@%(host)s/%(cloned_repo)s' % params
103 else:
103 else:
104 _url = 'http://(host)s/%(cloned_repo)s' % params
104 _url = 'http://(host)s/%(cloned_repo)s' % params
105 return _url
105 return _url
106
106
107
107
108 def _add_files_and_push(vcs, DEST, ignoreReturnCode=False, **kwargs):
108 def _add_files_and_push(vcs, DEST, ignoreReturnCode=False, **kwargs):
109 """
109 """
110 Generate some files, add it to DEST repo and push back
110 Generate some files, add it to DEST repo and push back
111 vcs is git or hg and defines what VCS we want to make those files for
111 vcs is git or hg and defines what VCS we want to make those files for
112
112
113 :param vcs:
113 :param vcs:
114 :param DEST:
114 :param DEST:
115 """
115 """
116 # commit some stuff into this repo
116 # commit some stuff into this repo
117 cwd = os.path.join(DEST)
117 cwd = os.path.join(DEST)
118 #added_file = '%ssetupΔ…ΕΌΕΊΔ‡.py' % _RandomNameSequence().next()
118 #added_file = '%ssetupΔ…ΕΌΕΊΔ‡.py' % _RandomNameSequence().next()
119 added_file = '%ssetup.py' % _RandomNameSequence().next()
119 added_file = '%ssetup.py' % _RandomNameSequence().next()
120 Command(cwd).execute('touch %s' % added_file)
120 Command(cwd).execute('touch %s' % added_file)
121 Command(cwd).execute('%s add %s' % (vcs, added_file))
121 Command(cwd).execute('%s add %s' % (vcs, added_file))
122
122
123 email = 'me@example.com'
123 email = 'me@example.com'
124 if os.name == 'nt':
124 if os.name == 'nt':
125 author_str = 'User <%s>' % email
125 author_str = 'User <%s>' % email
126 else:
126 else:
127 author_str = 'User ǝɯɐᴎ <%s>' % email
127 author_str = 'User ǝɯɐᴎ <%s>' % email
128 for i in xrange(kwargs.get('files_no', 3)):
128 for i in xrange(kwargs.get('files_no', 3)):
129 cmd = """echo "added_line%s" >> %s""" % (i, added_file)
129 cmd = """echo "added_line%s" >> %s""" % (i, added_file)
130 Command(cwd).execute(cmd)
130 Command(cwd).execute(cmd)
131 if vcs == 'hg':
131 if vcs == 'hg':
132 cmd = """hg commit -m "committed new %s" -u "%s" "%s" """ % (
132 cmd = """hg commit -m "committed new %s" -u "%s" "%s" """ % (
133 i, author_str, added_file
133 i, author_str, added_file
134 )
134 )
135 elif vcs == 'git':
135 elif vcs == 'git':
136 cmd = """git commit -m "committed new %s" --author "%s" "%s" """ % (
136 cmd = """git commit -m "committed new %s" --author "%s" "%s" """ % (
137 i, author_str, added_file
137 i, author_str, added_file
138 )
138 )
139 # git commit needs EMAIL on some machines
139 # git commit needs EMAIL on some machines
140 Command(cwd).execute(cmd, EMAIL=email)
140 Command(cwd).execute(cmd, EMAIL=email)
141
141
142 # PUSH it back
142 # PUSH it back
143 _REPO = None
143 _REPO = None
144 if vcs == 'hg':
144 if vcs == 'hg':
145 _REPO = HG_REPO
145 _REPO = HG_REPO
146 elif vcs == 'git':
146 elif vcs == 'git':
147 _REPO = GIT_REPO
147 _REPO = GIT_REPO
148
148
149 clone_url = _construct_url(_REPO, **kwargs)
149 clone_url = _construct_url(_REPO, **kwargs)
150 if 'clone_url' in kwargs:
150 if 'clone_url' in kwargs:
151 clone_url = kwargs['clone_url']
151 clone_url = kwargs['clone_url']
152 stdout = stderr = None
152 stdout = stderr = None
153 if vcs == 'hg':
153 if vcs == 'hg':
154 stdout, stderr = Command(cwd).execute('hg push --verbose', clone_url, ignoreReturnCode=ignoreReturnCode)
154 stdout, stderr = Command(cwd).execute('hg push --verbose', clone_url, ignoreReturnCode=ignoreReturnCode)
155 elif vcs == 'git':
155 elif vcs == 'git':
156 stdout, stderr = Command(cwd).execute('git push --verbose', clone_url, "master", ignoreReturnCode=ignoreReturnCode)
156 stdout, stderr = Command(cwd).execute('git push --verbose', clone_url, "master", ignoreReturnCode=ignoreReturnCode)
157
157
158 return stdout, stderr
158 return stdout, stderr
159
159
160
160
161 def set_anonymous_access(enable=True):
161 def set_anonymous_access(enable=True):
162 user = User.get_default_user()
162 user = User.get_default_user()
163 user.active = enable
163 user.active = enable
164 Session().commit()
164 Session().commit()
165 print '\tanonymous access is now:', enable
165 print '\tanonymous access is now:', enable
166 if enable != User.get_default_user().active:
166 if enable != User.get_default_user().active:
167 raise Exception('Cannot set anonymous access')
167 raise Exception('Cannot set anonymous access')
168
168
169
169
170 #==============================================================================
170 #==============================================================================
171 # TESTS
171 # TESTS
172 #==============================================================================
172 #==============================================================================
173
173
174
174
175 def _check_proper_git_push(stdout, stderr):
175 def _check_proper_git_push(stdout, stderr):
176 #WTF Git stderr is output ?!
176 #WTF Git stderr is output ?!
177 assert 'fatal' not in stderr
177 assert 'fatal' not in stderr
178 assert 'rejected' not in stderr
178 assert 'rejected' not in stderr
179 assert 'Pushing to' in stderr
179 assert 'Pushing to' in stderr
180 assert 'master -> master' in stderr
180 assert 'master -> master' in stderr
181
181
182
182
183 @pytest.mark.usefixtures("test_context_fixture")
183 @pytest.mark.usefixtures("test_context_fixture")
184 class TestVCSOperations(TestController):
184 class TestVCSOperations(TestController):
185
185
186 @classmethod
186 @classmethod
187 def setup_class(cls):
187 def setup_class(cls):
188 #DISABLE ANONYMOUS ACCESS
188 #DISABLE ANONYMOUS ACCESS
189 set_anonymous_access(False)
189 set_anonymous_access(False)
190
190
191 def setup_method(self, method):
191 def setup_method(self, method):
192 r = Repository.get_by_repo_name(GIT_REPO)
192 r = Repository.get_by_repo_name(GIT_REPO)
193 Repository.unlock(r)
193 Repository.unlock(r)
194 r.enable_locking = False
194 r.enable_locking = False
195 Session().commit()
195 Session().commit()
196
196
197 r = Repository.get_by_repo_name(HG_REPO)
197 r = Repository.get_by_repo_name(HG_REPO)
198 Repository.unlock(r)
198 Repository.unlock(r)
199 r.enable_locking = False
199 r.enable_locking = False
200 Session().commit()
200 Session().commit()
201
201
202 def test_clone_hg_repo_by_admin(self):
202 def test_clone_hg_repo_by_admin(self):
203 clone_url = _construct_url(HG_REPO)
203 clone_url = _construct_url(HG_REPO)
204 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir())
204 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir())
205
205
206 assert 'requesting all changes' in stdout
206 assert 'requesting all changes' in stdout
207 assert 'adding changesets' in stdout
207 assert 'adding changesets' in stdout
208 assert 'adding manifests' in stdout
208 assert 'adding manifests' in stdout
209 assert 'adding file changes' in stdout
209 assert 'adding file changes' in stdout
210
210
211 assert stderr == ''
211 assert stderr == ''
212
212
213 def test_clone_git_repo_by_admin(self):
213 def test_clone_git_repo_by_admin(self):
214 clone_url = _construct_url(GIT_REPO)
214 clone_url = _construct_url(GIT_REPO)
215 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir())
215 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir())
216
216
217 assert 'Cloning into' in stdout + stderr
217 assert 'Cloning into' in stdout + stderr
218 assert stderr == '' or stdout == ''
218 assert stderr == '' or stdout == ''
219
219
220 def test_clone_wrong_credentials_hg(self):
220 def test_clone_wrong_credentials_hg(self):
221 clone_url = _construct_url(HG_REPO, passwd='bad!')
221 clone_url = _construct_url(HG_REPO, passwd='bad!')
222 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
222 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
223 assert 'abort: authorization failed' in stderr
223 assert 'abort: authorization failed' in stderr
224
224
225 def test_clone_wrong_credentials_git(self):
225 def test_clone_wrong_credentials_git(self):
226 clone_url = _construct_url(GIT_REPO, passwd='bad!')
226 clone_url = _construct_url(GIT_REPO, passwd='bad!')
227 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
227 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
228 assert 'fatal: Authentication failed' in stderr
228 assert 'fatal: Authentication failed' in stderr
229
229
230 def test_clone_git_dir_as_hg(self):
230 def test_clone_git_dir_as_hg(self):
231 clone_url = _construct_url(GIT_REPO)
231 clone_url = _construct_url(GIT_REPO)
232 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
232 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
233 assert 'HTTP Error 404: Not Found' in stderr
233 assert 'HTTP Error 404: Not Found' in stderr
234
234
235 def test_clone_hg_repo_as_git(self):
235 def test_clone_hg_repo_as_git(self):
236 clone_url = _construct_url(HG_REPO)
236 clone_url = _construct_url(HG_REPO)
237 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
237 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
238 assert 'not found' in stderr
238 assert 'not found' in stderr
239
239
240 def test_clone_non_existing_path_hg(self):
240 def test_clone_non_existing_path_hg(self):
241 clone_url = _construct_url('trololo')
241 clone_url = _construct_url('trololo')
242 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
242 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
243 assert 'HTTP Error 404: Not Found' in stderr
243 assert 'HTTP Error 404: Not Found' in stderr
244
244
245 def test_clone_non_existing_path_git(self):
245 def test_clone_non_existing_path_git(self):
246 clone_url = _construct_url('trololo')
246 clone_url = _construct_url('trololo')
247 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
247 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
248 assert 'not found' in stderr
248 assert 'not found' in stderr
249
249
250 def test_push_new_file_hg(self):
250 def test_push_new_file_hg(self):
251 DEST = _get_tmp_dir()
251 DEST = _get_tmp_dir()
252 clone_url = _construct_url(HG_REPO)
252 clone_url = _construct_url(HG_REPO)
253 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)
253 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)
254
254
255 fork_name = '%s_fork%s' % (HG_REPO, _RandomNameSequence().next())
255 fork_name = '%s_fork%s' % (HG_REPO, _RandomNameSequence().next())
256 fixture.create_fork(HG_REPO, fork_name)
256 fixture.create_fork(HG_REPO, fork_name)
257 clone_url = _construct_url(fork_name).split()[0]
257 clone_url = _construct_url(fork_name).split()[0]
258 stdout, stderr = _add_files_and_push('hg', DEST, clone_url=clone_url)
258 stdout, stderr = _add_files_and_push('hg', DEST, clone_url=clone_url)
259
259
260 assert 'pushing to' in stdout
260 assert 'pushing to' in stdout
261 assert 'Repository size' in stdout
261 assert 'Repository size' in stdout
262 assert 'Last revision is now' in stdout
262 assert 'Last revision is now' in stdout
263
263
264 def test_push_new_file_git(self):
264 def test_push_new_file_git(self):
265 DEST = _get_tmp_dir()
265 DEST = _get_tmp_dir()
266 clone_url = _construct_url(GIT_REPO)
266 clone_url = _construct_url(GIT_REPO)
267 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)
267 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)
268
268
269 # commit some stuff into this repo
269 # commit some stuff into this repo
270 fork_name = '%s_fork%s' % (GIT_REPO, _RandomNameSequence().next())
270 fork_name = '%s_fork%s' % (GIT_REPO, _RandomNameSequence().next())
271 fixture.create_fork(GIT_REPO, fork_name)
271 fixture.create_fork(GIT_REPO, fork_name)
272 clone_url = _construct_url(fork_name).split()[0]
272 clone_url = _construct_url(fork_name).split()[0]
273 stdout, stderr = _add_files_and_push('git', DEST, clone_url=clone_url)
273 stdout, stderr = _add_files_and_push('git', DEST, clone_url=clone_url)
274 print [(x.repo_full_path,x.repo_path) for x in Repository.query()] # TODO: what is this for
274 print [(x.repo_full_path,x.repo_path) for x in Repository.query()] # TODO: what is this for
275 _check_proper_git_push(stdout, stderr)
275 _check_proper_git_push(stdout, stderr)
276
276
277 def test_push_invalidates_cache_hg(self):
277 def test_push_invalidates_cache_hg(self):
278 key = CacheInvalidation.query().filter(CacheInvalidation.cache_key
278 key = CacheInvalidation.query().filter(CacheInvalidation.cache_key
279 ==HG_REPO).scalar()
279 ==HG_REPO).scalar()
280 if not key:
280 if not key:
281 key = CacheInvalidation(HG_REPO, HG_REPO)
281 key = CacheInvalidation(HG_REPO, HG_REPO)
282 Session().add(key)
282 Session().add(key)
283
283
284 key.cache_active = True
284 key.cache_active = True
285 Session().commit()
285 Session().commit()
286
286
287 DEST = _get_tmp_dir()
287 DEST = _get_tmp_dir()
288 clone_url = _construct_url(HG_REPO)
288 clone_url = _construct_url(HG_REPO)
289 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)
289 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)
290
290
291 fork_name = '%s_fork%s' % (HG_REPO, _RandomNameSequence().next())
291 fork_name = '%s_fork%s' % (HG_REPO, _RandomNameSequence().next())
292 fixture.create_fork(HG_REPO, fork_name)
292 fixture.create_fork(HG_REPO, fork_name)
293 clone_url = _construct_url(fork_name).split()[0]
293 clone_url = _construct_url(fork_name).split()[0]
294 stdout, stderr = _add_files_and_push('hg', DEST, files_no=1, clone_url=clone_url)
294 stdout, stderr = _add_files_and_push('hg', DEST, files_no=1, clone_url=clone_url)
295
295
296 key = CacheInvalidation.query().filter(CacheInvalidation.cache_key
296 key = CacheInvalidation.query().filter(CacheInvalidation.cache_key
297 ==fork_name).all()
297 ==fork_name).all()
298 assert key == []
298 assert key == []
299
299
300 def test_push_invalidates_cache_git(self):
300 def test_push_invalidates_cache_git(self):
301 key = CacheInvalidation.query().filter(CacheInvalidation.cache_key
301 key = CacheInvalidation.query().filter(CacheInvalidation.cache_key
302 ==GIT_REPO).scalar()
302 ==GIT_REPO).scalar()
303 if not key:
303 if not key:
304 key = CacheInvalidation(GIT_REPO, GIT_REPO)
304 key = CacheInvalidation(GIT_REPO, GIT_REPO)
305 Session().add(key)
305 Session().add(key)
306
306
307 key.cache_active = True
307 key.cache_active = True
308 Session().commit()
308 Session().commit()
309
309
310 DEST = _get_tmp_dir()
310 DEST = _get_tmp_dir()
311 clone_url = _construct_url(GIT_REPO)
311 clone_url = _construct_url(GIT_REPO)
312 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)
312 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)
313
313
314 # commit some stuff into this repo
314 # commit some stuff into this repo
315 fork_name = '%s_fork%s' % (GIT_REPO, _RandomNameSequence().next())
315 fork_name = '%s_fork%s' % (GIT_REPO, _RandomNameSequence().next())
316 fixture.create_fork(GIT_REPO, fork_name)
316 fixture.create_fork(GIT_REPO, fork_name)
317 clone_url = _construct_url(fork_name).split()[0]
317 clone_url = _construct_url(fork_name).split()[0]
318 stdout, stderr = _add_files_and_push('git', DEST, files_no=1, clone_url=clone_url)
318 stdout, stderr = _add_files_and_push('git', DEST, files_no=1, clone_url=clone_url)
319 _check_proper_git_push(stdout, stderr)
319 _check_proper_git_push(stdout, stderr)
320
320
321 key = CacheInvalidation.query().filter(CacheInvalidation.cache_key
321 key = CacheInvalidation.query().filter(CacheInvalidation.cache_key
322 ==fork_name).all()
322 ==fork_name).all()
323 assert key == []
323 assert key == []
324
324
325 def test_push_wrong_credentials_hg(self):
325 def test_push_wrong_credentials_hg(self):
326 DEST = _get_tmp_dir()
326 DEST = _get_tmp_dir()
327 clone_url = _construct_url(HG_REPO)
327 clone_url = _construct_url(HG_REPO)
328 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)
328 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)
329
329
330 stdout, stderr = _add_files_and_push('hg', DEST, user='bad',
330 stdout, stderr = _add_files_and_push('hg', DEST, user='bad',
331 passwd='name', ignoreReturnCode=True)
331 passwd='name', ignoreReturnCode=True)
332
332
333 assert 'abort: authorization failed' in stderr
333 assert 'abort: authorization failed' in stderr
334
334
335 def test_push_wrong_credentials_git(self):
335 def test_push_wrong_credentials_git(self):
336 DEST = _get_tmp_dir()
336 DEST = _get_tmp_dir()
337 clone_url = _construct_url(GIT_REPO)
337 clone_url = _construct_url(GIT_REPO)
338 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)
338 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)
339
339
340 stdout, stderr = _add_files_and_push('git', DEST, user='bad',
340 stdout, stderr = _add_files_and_push('git', DEST, user='bad',
341 passwd='name', ignoreReturnCode=True)
341 passwd='name', ignoreReturnCode=True)
342
342
343 assert 'fatal: Authentication failed' in stderr
343 assert 'fatal: Authentication failed' in stderr
344
344
345 def test_push_back_to_wrong_url_hg(self):
345 def test_push_back_to_wrong_url_hg(self):
346 DEST = _get_tmp_dir()
346 DEST = _get_tmp_dir()
347 clone_url = _construct_url(HG_REPO)
347 clone_url = _construct_url(HG_REPO)
348 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)
348 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)
349
349
350 stdout, stderr = _add_files_and_push('hg', DEST,
350 stdout, stderr = _add_files_and_push('hg', DEST,
351 clone_url='http://%s/tmp' % HOST,
351 clone_url='http://%s/tmp' % HOST,
352 ignoreReturnCode = True)
352 ignoreReturnCode = True)
353
353
354 assert 'HTTP Error 404: Not Found' in stderr
354 assert 'HTTP Error 404: Not Found' in stderr
355
355
356 def test_push_back_to_wrong_url_git(self):
356 def test_push_back_to_wrong_url_git(self):
357 DEST = _get_tmp_dir()
357 DEST = _get_tmp_dir()
358 clone_url = _construct_url(GIT_REPO)
358 clone_url = _construct_url(GIT_REPO)
359 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)
359 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)
360
360
361 stdout, stderr = _add_files_and_push('git', DEST,
361 stdout, stderr = _add_files_and_push('git', DEST,
362 clone_url='http://%s/tmp' % HOST,
362 clone_url='http://%s/tmp' % HOST,
363 ignoreReturnCode = True)
363 ignoreReturnCode = True)
364
364
365 assert 'not found' in stderr
365 assert 'not found' in stderr
366
366
367 def test_clone_and_create_lock_hg(self):
367 def test_clone_and_create_lock_hg(self):
368 # enable locking
368 # enable locking
369 r = Repository.get_by_repo_name(HG_REPO)
369 r = Repository.get_by_repo_name(HG_REPO)
370 r.enable_locking = True
370 r.enable_locking = True
371 Session().commit()
371 Session().commit()
372 # clone
372 # clone
373 clone_url = _construct_url(HG_REPO)
373 clone_url = _construct_url(HG_REPO)
374 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir())
374 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir())
375
375
376 #check if lock was made
376 #check if lock was made
377 r = Repository.get_by_repo_name(HG_REPO)
377 r = Repository.get_by_repo_name(HG_REPO)
378 assert r.locked[0] == User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id
378 assert r.locked[0] == User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id
379
379
380 def test_clone_and_create_lock_git(self):
380 def test_clone_and_create_lock_git(self):
381 # enable locking
381 # enable locking
382 r = Repository.get_by_repo_name(GIT_REPO)
382 r = Repository.get_by_repo_name(GIT_REPO)
383 r.enable_locking = True
383 r.enable_locking = True
384 Session().commit()
384 Session().commit()
385 # clone
385 # clone
386 clone_url = _construct_url(GIT_REPO)
386 clone_url = _construct_url(GIT_REPO)
387 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir())
387 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir())
388
388
389 #check if lock was made
389 #check if lock was made
390 r = Repository.get_by_repo_name(GIT_REPO)
390 r = Repository.get_by_repo_name(GIT_REPO)
391 assert r.locked[0] == User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id
391 assert r.locked[0] == User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id
392
392
393 def test_clone_after_repo_was_locked_hg(self):
393 def test_clone_after_repo_was_locked_hg(self):
394 #lock repo
394 #lock repo
395 r = Repository.get_by_repo_name(HG_REPO)
395 r = Repository.get_by_repo_name(HG_REPO)
396 Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)
396 Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)
397 #pull fails since repo is locked
397 #pull fails since repo is locked
398 clone_url = _construct_url(HG_REPO)
398 clone_url = _construct_url(HG_REPO)
399 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
399 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
400 msg = ("""abort: HTTP Error 423: Repository `%s` locked by user `%s`"""
400 msg = ("""abort: HTTP Error 423: Repository `%s` locked by user `%s`"""
401 % (HG_REPO, TEST_USER_ADMIN_LOGIN))
401 % (HG_REPO, TEST_USER_ADMIN_LOGIN))
402 assert msg in stderr
402 assert msg in stderr
403
403
404 def test_clone_after_repo_was_locked_git(self):
404 def test_clone_after_repo_was_locked_git(self):
405 #lock repo
405 #lock repo
406 r = Repository.get_by_repo_name(GIT_REPO)
406 r = Repository.get_by_repo_name(GIT_REPO)
407 Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)
407 Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)
408 #pull fails since repo is locked
408 #pull fails since repo is locked
409 clone_url = _construct_url(GIT_REPO)
409 clone_url = _construct_url(GIT_REPO)
410 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
410 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
411 msg = ("""The requested URL returned error: 423""")
411 msg = ("""The requested URL returned error: 423""")
412 assert msg in stderr
412 assert msg in stderr
413
413
414 def test_push_on_locked_repo_by_other_user_hg(self):
414 def test_push_on_locked_repo_by_other_user_hg(self):
415 #clone some temp
415 #clone some temp
416 DEST = _get_tmp_dir()
416 DEST = _get_tmp_dir()
417 clone_url = _construct_url(HG_REPO)
417 clone_url = _construct_url(HG_REPO)
418 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)
418 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)
419
419
420 #lock repo
420 #lock repo
421 r = Repository.get_by_repo_name(HG_REPO)
421 r = Repository.get_by_repo_name(HG_REPO)
422 # let this user actually push !
422 # let this user actually push !
423 RepoModel().grant_user_permission(repo=r, user=TEST_USER_REGULAR_LOGIN,
423 RepoModel().grant_user_permission(repo=r, user=TEST_USER_REGULAR_LOGIN,
424 perm='repository.write')
424 perm='repository.write')
425 Session().commit()
425 Session().commit()
426 Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)
426 Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)
427
427
428 #push fails repo is locked by other user !
428 #push fails repo is locked by other user !
429 stdout, stderr = _add_files_and_push('hg', DEST,
429 stdout, stderr = _add_files_and_push('hg', DEST,
430 user=TEST_USER_REGULAR_LOGIN,
430 user=TEST_USER_REGULAR_LOGIN,
431 passwd=TEST_USER_REGULAR_PASS,
431 passwd=TEST_USER_REGULAR_PASS,
432 ignoreReturnCode=True)
432 ignoreReturnCode=True)
433 msg = ("""abort: HTTP Error 423: Repository `%s` locked by user `%s`"""
433 msg = ("""abort: HTTP Error 423: Repository `%s` locked by user `%s`"""
434 % (HG_REPO, TEST_USER_ADMIN_LOGIN))
434 % (HG_REPO, TEST_USER_ADMIN_LOGIN))
435 assert msg in stderr
435 assert msg in stderr
436
436
437 def test_push_on_locked_repo_by_other_user_git(self):
437 def test_push_on_locked_repo_by_other_user_git(self):
438 # Note: Git hooks must be executable on unix. This test will thus fail
439 # for example on Linux if /tmp is mounted noexec.
440
438 #clone some temp
441 #clone some temp
439 DEST = _get_tmp_dir()
442 DEST = _get_tmp_dir()
440 clone_url = _construct_url(GIT_REPO)
443 clone_url = _construct_url(GIT_REPO)
441 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)
444 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)
442
445
443 #lock repo
446 #lock repo
444 r = Repository.get_by_repo_name(GIT_REPO)
447 r = Repository.get_by_repo_name(GIT_REPO)
445 # let this user actually push !
448 # let this user actually push !
446 RepoModel().grant_user_permission(repo=r, user=TEST_USER_REGULAR_LOGIN,
449 RepoModel().grant_user_permission(repo=r, user=TEST_USER_REGULAR_LOGIN,
447 perm='repository.write')
450 perm='repository.write')
448 Session().commit()
451 Session().commit()
449 Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)
452 Repository.lock(r, User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id)
450
453
451 #push fails repo is locked by other user !
454 #push fails repo is locked by other user !
452 stdout, stderr = _add_files_and_push('git', DEST,
455 stdout, stderr = _add_files_and_push('git', DEST,
453 user=TEST_USER_REGULAR_LOGIN,
456 user=TEST_USER_REGULAR_LOGIN,
454 passwd=TEST_USER_REGULAR_PASS,
457 passwd=TEST_USER_REGULAR_PASS,
455 ignoreReturnCode=True)
458 ignoreReturnCode=True)
456 err = 'Repository `%s` locked by user `%s`' % (GIT_REPO, TEST_USER_ADMIN_LOGIN)
459 err = 'Repository `%s` locked by user `%s`' % (GIT_REPO, TEST_USER_ADMIN_LOGIN)
457 assert err in stderr
460 assert err in stderr
458
461
459 #TODO: fix this somehow later on Git, Git is stupid and even if we throw
462 #TODO: fix this somehow later on Git, Git is stupid and even if we throw
460 #back 423 to it, it makes ANOTHER request and we fail there with 405 :/
463 #back 423 to it, it makes ANOTHER request and we fail there with 405 :/
461
464
462 msg = ("""abort: HTTP Error 423: Repository `%s` locked by user `%s`"""
465 msg = ("""abort: HTTP Error 423: Repository `%s` locked by user `%s`"""
463 % (GIT_REPO, TEST_USER_ADMIN_LOGIN))
466 % (GIT_REPO, TEST_USER_ADMIN_LOGIN))
464 #msg = "405 Method Not Allowed"
467 #msg = "405 Method Not Allowed"
465 #assert msg in stderr
468 #assert msg in stderr
466
469
467 def test_push_unlocks_repository_hg(self):
470 def test_push_unlocks_repository_hg(self):
468 # enable locking
471 # enable locking
469 fork_name = '%s_fork%s' % (HG_REPO, _RandomNameSequence().next())
472 fork_name = '%s_fork%s' % (HG_REPO, _RandomNameSequence().next())
470 fixture.create_fork(HG_REPO, fork_name)
473 fixture.create_fork(HG_REPO, fork_name)
471 r = Repository.get_by_repo_name(fork_name)
474 r = Repository.get_by_repo_name(fork_name)
472 r.enable_locking = True
475 r.enable_locking = True
473 Session().commit()
476 Session().commit()
474 #clone some temp
477 #clone some temp
475 DEST = _get_tmp_dir()
478 DEST = _get_tmp_dir()
476 clone_url = _construct_url(fork_name)
479 clone_url = _construct_url(fork_name)
477 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)
480 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, DEST)
478
481
479 #check for lock repo after clone
482 #check for lock repo after clone
480 r = Repository.get_by_repo_name(fork_name)
483 r = Repository.get_by_repo_name(fork_name)
481 uid = User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id
484 uid = User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id
482 assert r.locked[0] == uid
485 assert r.locked[0] == uid
483
486
484 #push is ok and repo is now unlocked
487 #push is ok and repo is now unlocked
485 stdout, stderr = _add_files_and_push('hg', DEST, clone_url=clone_url.split()[0])
488 stdout, stderr = _add_files_and_push('hg', DEST, clone_url=clone_url.split()[0])
486 assert str('remote: Released lock on repo `%s`' % fork_name) in stdout
489 assert str('remote: Released lock on repo `%s`' % fork_name) in stdout
487 #we need to cleanup the Session Here !
490 #we need to cleanup the Session Here !
488 Session.remove()
491 Session.remove()
489 r = Repository.get_by_repo_name(fork_name)
492 r = Repository.get_by_repo_name(fork_name)
490 assert r.locked == [None, None]
493 assert r.locked == [None, None]
491
494
492 #TODO: fix me ! somehow during tests hooks don't get called on Git
495 #TODO: fix me ! somehow during tests hooks don't get called on Git
493 def test_push_unlocks_repository_git(self):
496 def test_push_unlocks_repository_git(self):
494 # enable locking
497 # enable locking
495 fork_name = '%s_fork%s' % (GIT_REPO, _RandomNameSequence().next())
498 fork_name = '%s_fork%s' % (GIT_REPO, _RandomNameSequence().next())
496 fixture.create_fork(GIT_REPO, fork_name)
499 fixture.create_fork(GIT_REPO, fork_name)
497 r = Repository.get_by_repo_name(fork_name)
500 r = Repository.get_by_repo_name(fork_name)
498 r.enable_locking = True
501 r.enable_locking = True
499 Session().commit()
502 Session().commit()
500 #clone some temp
503 #clone some temp
501 DEST = _get_tmp_dir()
504 DEST = _get_tmp_dir()
502 clone_url = _construct_url(fork_name)
505 clone_url = _construct_url(fork_name)
503 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)
506 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, DEST)
504
507
505 #check for lock repo after clone
508 #check for lock repo after clone
506 r = Repository.get_by_repo_name(fork_name)
509 r = Repository.get_by_repo_name(fork_name)
507 assert r.locked[0] == User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id
510 assert r.locked[0] == User.get_by_username(TEST_USER_ADMIN_LOGIN).user_id
508
511
509 #push is ok and repo is now unlocked
512 #push is ok and repo is now unlocked
510 stdout, stderr = _add_files_and_push('git', DEST, clone_url=clone_url.split()[0])
513 stdout, stderr = _add_files_and_push('git', DEST, clone_url=clone_url.split()[0])
511 _check_proper_git_push(stdout, stderr)
514 _check_proper_git_push(stdout, stderr)
512
515
513 assert ('remote: Released lock on repo `%s`' % fork_name) in stderr
516 assert ('remote: Released lock on repo `%s`' % fork_name) in stderr
514 #we need to cleanup the Session Here !
517 #we need to cleanup the Session Here !
515 Session.remove()
518 Session.remove()
516 r = Repository.get_by_repo_name(fork_name)
519 r = Repository.get_by_repo_name(fork_name)
517 assert r.locked == [None, None]
520 assert r.locked == [None, None]
518
521
519 def test_ip_restriction_hg(self):
522 def test_ip_restriction_hg(self):
520 user_model = UserModel()
523 user_model = UserModel()
521 try:
524 try:
522 user_model.add_extra_ip(TEST_USER_ADMIN_LOGIN, '10.10.10.10/32')
525 user_model.add_extra_ip(TEST_USER_ADMIN_LOGIN, '10.10.10.10/32')
523 Session().commit()
526 Session().commit()
524 clone_url = _construct_url(HG_REPO)
527 clone_url = _construct_url(HG_REPO)
525 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
528 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
526 assert 'abort: HTTP Error 403: Forbidden' in stderr
529 assert 'abort: HTTP Error 403: Forbidden' in stderr
527 finally:
530 finally:
528 #release IP restrictions
531 #release IP restrictions
529 for ip in UserIpMap.query():
532 for ip in UserIpMap.query():
530 UserIpMap.delete(ip.ip_id)
533 UserIpMap.delete(ip.ip_id)
531 Session().commit()
534 Session().commit()
532
535
533 # IP permissions are cached, need to wait for the cache in the server process to expire
536 # IP permissions are cached, need to wait for the cache in the server process to expire
534 time.sleep(1.5)
537 time.sleep(1.5)
535
538
536 clone_url = _construct_url(HG_REPO)
539 clone_url = _construct_url(HG_REPO)
537 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir())
540 stdout, stderr = Command(tempfile.gettempdir()).execute('hg clone', clone_url, _get_tmp_dir())
538
541
539 assert 'requesting all changes' in stdout
542 assert 'requesting all changes' in stdout
540 assert 'adding changesets' in stdout
543 assert 'adding changesets' in stdout
541 assert 'adding manifests' in stdout
544 assert 'adding manifests' in stdout
542 assert 'adding file changes' in stdout
545 assert 'adding file changes' in stdout
543
546
544 assert stderr == ''
547 assert stderr == ''
545
548
546 def test_ip_restriction_git(self):
549 def test_ip_restriction_git(self):
547 user_model = UserModel()
550 user_model = UserModel()
548 try:
551 try:
549 user_model.add_extra_ip(TEST_USER_ADMIN_LOGIN, '10.10.10.10/32')
552 user_model.add_extra_ip(TEST_USER_ADMIN_LOGIN, '10.10.10.10/32')
550 Session().commit()
553 Session().commit()
551 clone_url = _construct_url(GIT_REPO)
554 clone_url = _construct_url(GIT_REPO)
552 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
555 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir(), ignoreReturnCode=True)
553 # The message apparently changed in Git 1.8.3, so match it loosely.
556 # The message apparently changed in Git 1.8.3, so match it loosely.
554 assert re.search(r'\b403\b', stderr)
557 assert re.search(r'\b403\b', stderr)
555 finally:
558 finally:
556 #release IP restrictions
559 #release IP restrictions
557 for ip in UserIpMap.query():
560 for ip in UserIpMap.query():
558 UserIpMap.delete(ip.ip_id)
561 UserIpMap.delete(ip.ip_id)
559 Session().commit()
562 Session().commit()
560
563
561 # IP permissions are cached, need to wait for the cache in the server process to expire
564 # IP permissions are cached, need to wait for the cache in the server process to expire
562 time.sleep(1.5)
565 time.sleep(1.5)
563
566
564 clone_url = _construct_url(GIT_REPO)
567 clone_url = _construct_url(GIT_REPO)
565 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir())
568 stdout, stderr = Command(tempfile.gettempdir()).execute('git clone', clone_url, _get_tmp_dir())
566
569
567 assert 'Cloning into' in stdout + stderr
570 assert 'Cloning into' in stdout + stderr
568 assert stderr == '' or stdout == ''
571 assert stderr == '' or stdout == ''
General Comments 0
You need to be logged in to leave comments. Login now