Show More
@@ -1,74 +1,79 b'' | |||||
1 | .. _email: |
|
1 | .. _email: | |
2 |
|
2 | |||
3 | ============== |
|
3 | ============== | |
4 | Email settings |
|
4 | Email settings | |
5 | ============== |
|
5 | ============== | |
6 |
|
6 | |||
7 | The Kallithea configuration file has several email related settings. When |
|
7 | The Kallithea configuration file has several email related settings. When | |
8 | these contain correct values, Kallithea will send email in the situations |
|
8 | these contain correct values, Kallithea will send email in the situations | |
9 | described below. If the email configuration is not correct so that emails |
|
9 | described below. If the email configuration is not correct so that emails | |
10 | cannot be sent, all mails will show up in the log output. |
|
10 | cannot be sent, all mails will show up in the log output. | |
11 |
|
11 | |||
12 | Before any email can be sent, an SMTP server has to be configured using the |
|
12 | Before any email can be sent, an SMTP server has to be configured using the | |
13 | configuration file setting ``smtp_server``. If required for that server, specify |
|
13 | configuration file setting ``smtp_server``. If required for that server, specify | |
14 | a username (``smtp_username``) and password (``smtp_password``), a non-standard |
|
14 | a username (``smtp_username``) and password (``smtp_password``), a non-standard | |
15 | port (``smtp_port``), encryption settings (``smtp_use_tls`` or ``smtp_use_ssl``) |
|
15 | port (``smtp_port``), encryption settings (``smtp_use_tls`` or ``smtp_use_ssl``) | |
16 | and/or specific authentication parameters (``smtp_auth``). |
|
16 | and/or specific authentication parameters (``smtp_auth``). | |
17 |
|
17 | |||
18 |
|
18 | |||
19 | Application emails |
|
19 | Application emails | |
20 | ------------------ |
|
20 | ------------------ | |
21 |
|
21 | |||
22 | Kallithea sends an email to `users` on several occasions: |
|
22 | Kallithea sends an email to `users` on several occasions: | |
23 |
|
23 | |||
24 | - when comments are given on one of their changesets |
|
24 | - when comments are given on one of their changesets | |
25 | - when comments are given on changesets they are reviewer on or on which they |
|
25 | - when comments are given on changesets they are reviewer on or on which they | |
26 | commented regardless |
|
26 | commented regardless | |
27 | - when they are invited as reviewer in pull requests |
|
27 | - when they are invited as reviewer in pull requests | |
28 | - when they request a password reset |
|
28 | - when they request a password reset | |
29 |
|
29 | |||
30 | Kallithea sends an email to all `administrators` upon new account registration. |
|
30 | Kallithea sends an email to all `administrators` upon new account registration. | |
31 | Administrators are users with the ``Admin`` flag set on the *Admin > Users* |
|
31 | Administrators are users with the ``Admin`` flag set on the *Admin > Users* | |
32 | page. |
|
32 | page. | |
33 |
|
33 | |||
34 | When Kallithea wants to send an email but due to an error cannot correctly |
|
34 | When Kallithea wants to send an email but due to an error cannot correctly | |
35 | determine the intended recipients, the administrators and the addresses |
|
35 | determine the intended recipients, the administrators and the addresses | |
36 | specified in ``email_to`` in the configuration file are used as fallback. |
|
36 | specified in ``email_to`` in the configuration file are used as fallback. | |
37 |
|
37 | |||
38 | Recipients will see these emails originating from the sender specified in the |
|
38 | Recipients will see these emails originating from the sender specified in the | |
39 | ``app_email_from`` setting in the configuration file. This setting can either |
|
39 | ``app_email_from`` setting in the configuration file. This setting can either | |
40 | contain only an email address, like `kallithea-noreply@example.com`, or both |
|
40 | contain only an email address, like `kallithea-noreply@example.com`, or both | |
41 | a name and an address in the following format: `Kallithea |
|
41 | a name and an address in the following format: `Kallithea | |
42 |
<kallithea-noreply@example.com>`. |
|
42 | <kallithea-noreply@example.com>`. However, if the email is sent due to an | |
43 | optionally be prefixed with the value of ``email_prefix`` in the configuration |
|
43 | action of a particular user, for example when a comment is given or a pull | |
44 | file. |
|
44 | request created, the name of that user will be combined with the email address | |
|
45 | specified in ``app_email_from`` to form the sender (and any name part in that | |||
|
46 | configuration setting disregarded). | |||
|
47 | ||||
|
48 | The subject of these emails can optionally be prefixed with the value of | |||
|
49 | ``email_prefix`` in the configuration file. | |||
45 |
|
50 | |||
46 |
|
51 | |||
47 | Error emails |
|
52 | Error emails | |
48 | ------------ |
|
53 | ------------ | |
49 |
|
54 | |||
50 | When an exception occurs in Kallithea -- and unless interactive debugging is |
|
55 | When an exception occurs in Kallithea -- and unless interactive debugging is | |
51 | enabled using ``set debug = true`` in the ``[app:main]`` section of the |
|
56 | enabled using ``set debug = true`` in the ``[app:main]`` section of the | |
52 | configuration file -- an email with exception details is sent by WebError_'s |
|
57 | configuration file -- an email with exception details is sent by WebError_'s | |
53 | ``ErrorMiddleware`` to the addresses specified in ``email_to`` in the |
|
58 | ``ErrorMiddleware`` to the addresses specified in ``email_to`` in the | |
54 | configuration file. |
|
59 | configuration file. | |
55 |
|
60 | |||
56 | Recipients will see these emails originating from the sender specified in the |
|
61 | Recipients will see these emails originating from the sender specified in the | |
57 | ``error_email_from`` setting in the configuration file. This setting can either |
|
62 | ``error_email_from`` setting in the configuration file. This setting can either | |
58 | contain only an email address, like `kallithea-noreply@example.com`, or both |
|
63 | contain only an email address, like `kallithea-noreply@example.com`, or both | |
59 | a name and an address in the following format: `Kallithea Errors |
|
64 | a name and an address in the following format: `Kallithea Errors | |
60 | <kallithea-noreply@example.com>`. |
|
65 | <kallithea-noreply@example.com>`. | |
61 |
|
66 | |||
62 | *Note:* The WebError_ package does not respect ``smtp_port`` and assumes the |
|
67 | *Note:* The WebError_ package does not respect ``smtp_port`` and assumes the | |
63 | standard SMTP port (25). If you have a remote SMTP server with a different port, |
|
68 | standard SMTP port (25). If you have a remote SMTP server with a different port, | |
64 | you could set up a local forwarding SMTP server on port 25. |
|
69 | you could set up a local forwarding SMTP server on port 25. | |
65 |
|
70 | |||
66 |
|
71 | |||
67 | References |
|
72 | References | |
68 | ---------- |
|
73 | ---------- | |
69 |
|
74 | |||
70 | - `Error Middleware (Pylons documentation) <http://pylons-webframework.readthedocs.org/en/latest/debugging.html#error-middleware>`_ |
|
75 | - `Error Middleware (Pylons documentation) <http://pylons-webframework.readthedocs.org/en/latest/debugging.html#error-middleware>`_ | |
71 | - `ErrorHandler (Pylons modules documentation) <http://pylons-webframework.readthedocs.org/en/latest/modules/middleware.html#pylons.middleware.ErrorHandler>`_ |
|
76 | - `ErrorHandler (Pylons modules documentation) <http://pylons-webframework.readthedocs.org/en/latest/modules/middleware.html#pylons.middleware.ErrorHandler>`_ | |
72 |
|
77 | |||
73 |
|
78 | |||
74 | .. _WebError: https://pypi.python.org/pypi/WebError |
|
79 | .. _WebError: https://pypi.python.org/pypi/WebError |
@@ -1,510 +1,530 b'' | |||||
1 | # -*- coding: utf-8 -*- |
|
1 | # -*- coding: utf-8 -*- | |
2 | # This program is free software: you can redistribute it and/or modify |
|
2 | # This program is free software: you can redistribute it and/or modify | |
3 | # it under the terms of the GNU General Public License as published by |
|
3 | # it under the terms of the GNU General Public License as published by | |
4 | # the Free Software Foundation, either version 3 of the License, or |
|
4 | # the Free Software Foundation, either version 3 of the License, or | |
5 | # (at your option) any later version. |
|
5 | # (at your option) any later version. | |
6 | # |
|
6 | # | |
7 | # This program is distributed in the hope that it will be useful, |
|
7 | # This program is distributed in the hope that it will be useful, | |
8 | # but WITHOUT ANY WARRANTY; without even the implied warranty of |
|
8 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | |
9 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
|
9 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | |
10 | # GNU General Public License for more details. |
|
10 | # GNU General Public License for more details. | |
11 | # |
|
11 | # | |
12 | # You should have received a copy of the GNU General Public License |
|
12 | # You should have received a copy of the GNU General Public License | |
13 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
|
13 | # along with this program. If not, see <http://www.gnu.org/licenses/>. | |
14 | """ |
|
14 | """ | |
15 | kallithea.lib.celerylib.tasks |
|
15 | kallithea.lib.celerylib.tasks | |
16 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
16 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
17 |
|
17 | |||
18 | Kallithea task modules, containing all task that suppose to be run |
|
18 | Kallithea task modules, containing all task that suppose to be run | |
19 | by celery daemon |
|
19 | by celery daemon | |
20 |
|
20 | |||
21 | This file was forked by the Kallithea project in July 2014. |
|
21 | This file was forked by the Kallithea project in July 2014. | |
22 | Original author and date, and relevant copyright and licensing information is below: |
|
22 | Original author and date, and relevant copyright and licensing information is below: | |
23 | :created_on: Oct 6, 2010 |
|
23 | :created_on: Oct 6, 2010 | |
24 | :author: marcink |
|
24 | :author: marcink | |
25 | :copyright: (c) 2013 RhodeCode GmbH, and others. |
|
25 | :copyright: (c) 2013 RhodeCode GmbH, and others. | |
26 | :license: GPLv3, see LICENSE.md for more details. |
|
26 | :license: GPLv3, see LICENSE.md for more details. | |
27 | """ |
|
27 | """ | |
28 |
|
28 | |||
29 | from celery.decorators import task |
|
29 | from celery.decorators import task | |
30 |
|
30 | |||
31 | import os |
|
31 | import os | |
32 | import traceback |
|
32 | import traceback | |
33 | import logging |
|
33 | import logging | |
|
34 | import rfc822 | |||
34 | from os.path import join as jn |
|
35 | from os.path import join as jn | |
35 |
|
36 | |||
36 | from time import mktime |
|
37 | from time import mktime | |
37 | from operator import itemgetter |
|
38 | from operator import itemgetter | |
38 | from string import lower |
|
39 | from string import lower | |
39 |
|
40 | |||
40 | from pylons import config |
|
41 | from pylons import config | |
41 |
|
42 | |||
42 | from kallithea import CELERY_ON |
|
43 | from kallithea import CELERY_ON | |
43 | from kallithea.lib.celerylib import run_task, locked_task, dbsession, \ |
|
44 | from kallithea.lib.celerylib import run_task, locked_task, dbsession, \ | |
44 | str2bool, __get_lockkey, LockHeld, DaemonLock, get_session |
|
45 | str2bool, __get_lockkey, LockHeld, DaemonLock, get_session | |
45 | from kallithea.lib.helpers import person |
|
46 | from kallithea.lib.helpers import person | |
46 | from kallithea.lib.rcmail.smtp_mailer import SmtpMailer |
|
47 | from kallithea.lib.rcmail.smtp_mailer import SmtpMailer | |
47 | from kallithea.lib.utils import add_cache, action_logger |
|
48 | from kallithea.lib.utils import add_cache, action_logger | |
|
49 | from kallithea.lib.vcs.utils import author_email | |||
48 | from kallithea.lib.compat import json, OrderedDict |
|
50 | from kallithea.lib.compat import json, OrderedDict | |
49 | from kallithea.lib.hooks import log_create_repository |
|
51 | from kallithea.lib.hooks import log_create_repository | |
50 |
|
52 | |||
51 | from kallithea.model.db import Statistics, Repository, User |
|
53 | from kallithea.model.db import Statistics, Repository, User | |
52 |
|
54 | |||
53 |
|
55 | |||
54 | add_cache(config) # pragma: no cover |
|
56 | add_cache(config) # pragma: no cover | |
55 |
|
57 | |||
56 | __all__ = ['whoosh_index', 'get_commits_stats', 'send_email'] |
|
58 | __all__ = ['whoosh_index', 'get_commits_stats', 'send_email'] | |
57 |
|
59 | |||
58 |
|
60 | |||
59 | def get_logger(cls): |
|
61 | def get_logger(cls): | |
60 | if CELERY_ON: |
|
62 | if CELERY_ON: | |
61 | try: |
|
63 | try: | |
62 | return cls.get_logger() |
|
64 | return cls.get_logger() | |
63 | except AttributeError: |
|
65 | except AttributeError: | |
64 | pass |
|
66 | pass | |
65 | return logging.getLogger(__name__) |
|
67 | return logging.getLogger(__name__) | |
66 |
|
68 | |||
67 |
|
69 | |||
68 | @task(ignore_result=True) |
|
70 | @task(ignore_result=True) | |
69 | @locked_task |
|
71 | @locked_task | |
70 | @dbsession |
|
72 | @dbsession | |
71 | def whoosh_index(repo_location, full_index): |
|
73 | def whoosh_index(repo_location, full_index): | |
72 | from kallithea.lib.indexers.daemon import WhooshIndexingDaemon |
|
74 | from kallithea.lib.indexers.daemon import WhooshIndexingDaemon | |
73 | DBS = get_session() |
|
75 | DBS = get_session() | |
74 |
|
76 | |||
75 | index_location = config['index_dir'] |
|
77 | index_location = config['index_dir'] | |
76 | WhooshIndexingDaemon(index_location=index_location, |
|
78 | WhooshIndexingDaemon(index_location=index_location, | |
77 | repo_location=repo_location, sa=DBS)\ |
|
79 | repo_location=repo_location, sa=DBS)\ | |
78 | .run(full_index=full_index) |
|
80 | .run(full_index=full_index) | |
79 |
|
81 | |||
80 |
|
82 | |||
81 | @task(ignore_result=True) |
|
83 | @task(ignore_result=True) | |
82 | @dbsession |
|
84 | @dbsession | |
83 | def get_commits_stats(repo_name, ts_min_y, ts_max_y, recurse_limit=100): |
|
85 | def get_commits_stats(repo_name, ts_min_y, ts_max_y, recurse_limit=100): | |
84 | log = get_logger(get_commits_stats) |
|
86 | log = get_logger(get_commits_stats) | |
85 | DBS = get_session() |
|
87 | DBS = get_session() | |
86 | lockkey = __get_lockkey('get_commits_stats', repo_name, ts_min_y, |
|
88 | lockkey = __get_lockkey('get_commits_stats', repo_name, ts_min_y, | |
87 | ts_max_y) |
|
89 | ts_max_y) | |
88 | lockkey_path = config['app_conf']['cache_dir'] |
|
90 | lockkey_path = config['app_conf']['cache_dir'] | |
89 |
|
91 | |||
90 | log.info('running task with lockkey %s', lockkey) |
|
92 | log.info('running task with lockkey %s', lockkey) | |
91 |
|
93 | |||
92 | try: |
|
94 | try: | |
93 | lock = l = DaemonLock(file_=jn(lockkey_path, lockkey)) |
|
95 | lock = l = DaemonLock(file_=jn(lockkey_path, lockkey)) | |
94 |
|
96 | |||
95 | # for js data compatibility cleans the key for person from ' |
|
97 | # for js data compatibility cleans the key for person from ' | |
96 | akc = lambda k: person(k).replace('"', "") |
|
98 | akc = lambda k: person(k).replace('"', "") | |
97 |
|
99 | |||
98 | co_day_auth_aggr = {} |
|
100 | co_day_auth_aggr = {} | |
99 | commits_by_day_aggregate = {} |
|
101 | commits_by_day_aggregate = {} | |
100 | repo = Repository.get_by_repo_name(repo_name) |
|
102 | repo = Repository.get_by_repo_name(repo_name) | |
101 | if repo is None: |
|
103 | if repo is None: | |
102 | return True |
|
104 | return True | |
103 |
|
105 | |||
104 | repo = repo.scm_instance |
|
106 | repo = repo.scm_instance | |
105 | repo_size = repo.count() |
|
107 | repo_size = repo.count() | |
106 | # return if repo have no revisions |
|
108 | # return if repo have no revisions | |
107 | if repo_size < 1: |
|
109 | if repo_size < 1: | |
108 | lock.release() |
|
110 | lock.release() | |
109 | return True |
|
111 | return True | |
110 |
|
112 | |||
111 | skip_date_limit = True |
|
113 | skip_date_limit = True | |
112 | parse_limit = int(config['app_conf'].get('commit_parse_limit')) |
|
114 | parse_limit = int(config['app_conf'].get('commit_parse_limit')) | |
113 | last_rev = None |
|
115 | last_rev = None | |
114 | last_cs = None |
|
116 | last_cs = None | |
115 | timegetter = itemgetter('time') |
|
117 | timegetter = itemgetter('time') | |
116 |
|
118 | |||
117 | dbrepo = DBS.query(Repository)\ |
|
119 | dbrepo = DBS.query(Repository)\ | |
118 | .filter(Repository.repo_name == repo_name).scalar() |
|
120 | .filter(Repository.repo_name == repo_name).scalar() | |
119 | cur_stats = DBS.query(Statistics)\ |
|
121 | cur_stats = DBS.query(Statistics)\ | |
120 | .filter(Statistics.repository == dbrepo).scalar() |
|
122 | .filter(Statistics.repository == dbrepo).scalar() | |
121 |
|
123 | |||
122 | if cur_stats is not None: |
|
124 | if cur_stats is not None: | |
123 | last_rev = cur_stats.stat_on_revision |
|
125 | last_rev = cur_stats.stat_on_revision | |
124 |
|
126 | |||
125 | if last_rev == repo.get_changeset().revision and repo_size > 1: |
|
127 | if last_rev == repo.get_changeset().revision and repo_size > 1: | |
126 | # pass silently without any work if we're not on first revision or |
|
128 | # pass silently without any work if we're not on first revision or | |
127 | # current state of parsing revision(from db marker) is the |
|
129 | # current state of parsing revision(from db marker) is the | |
128 | # last revision |
|
130 | # last revision | |
129 | lock.release() |
|
131 | lock.release() | |
130 | return True |
|
132 | return True | |
131 |
|
133 | |||
132 | if cur_stats: |
|
134 | if cur_stats: | |
133 | commits_by_day_aggregate = OrderedDict(json.loads( |
|
135 | commits_by_day_aggregate = OrderedDict(json.loads( | |
134 | cur_stats.commit_activity_combined)) |
|
136 | cur_stats.commit_activity_combined)) | |
135 | co_day_auth_aggr = json.loads(cur_stats.commit_activity) |
|
137 | co_day_auth_aggr = json.loads(cur_stats.commit_activity) | |
136 |
|
138 | |||
137 | log.debug('starting parsing %s', parse_limit) |
|
139 | log.debug('starting parsing %s', parse_limit) | |
138 | lmktime = mktime |
|
140 | lmktime = mktime | |
139 |
|
141 | |||
140 | last_rev = last_rev + 1 if last_rev >= 0 else 0 |
|
142 | last_rev = last_rev + 1 if last_rev >= 0 else 0 | |
141 | log.debug('Getting revisions from %s to %s', |
|
143 | log.debug('Getting revisions from %s to %s', | |
142 | last_rev, last_rev + parse_limit |
|
144 | last_rev, last_rev + parse_limit | |
143 | ) |
|
145 | ) | |
144 | for cs in repo[last_rev:last_rev + parse_limit]: |
|
146 | for cs in repo[last_rev:last_rev + parse_limit]: | |
145 | log.debug('parsing %s', cs) |
|
147 | log.debug('parsing %s', cs) | |
146 | last_cs = cs # remember last parsed changeset |
|
148 | last_cs = cs # remember last parsed changeset | |
147 | k = lmktime([cs.date.timetuple()[0], cs.date.timetuple()[1], |
|
149 | k = lmktime([cs.date.timetuple()[0], cs.date.timetuple()[1], | |
148 | cs.date.timetuple()[2], 0, 0, 0, 0, 0, 0]) |
|
150 | cs.date.timetuple()[2], 0, 0, 0, 0, 0, 0]) | |
149 |
|
151 | |||
150 | if akc(cs.author) in co_day_auth_aggr: |
|
152 | if akc(cs.author) in co_day_auth_aggr: | |
151 | try: |
|
153 | try: | |
152 | l = [timegetter(x) for x in |
|
154 | l = [timegetter(x) for x in | |
153 | co_day_auth_aggr[akc(cs.author)]['data']] |
|
155 | co_day_auth_aggr[akc(cs.author)]['data']] | |
154 | time_pos = l.index(k) |
|
156 | time_pos = l.index(k) | |
155 | except ValueError: |
|
157 | except ValueError: | |
156 | time_pos = None |
|
158 | time_pos = None | |
157 |
|
159 | |||
158 | if time_pos >= 0 and time_pos is not None: |
|
160 | if time_pos >= 0 and time_pos is not None: | |
159 |
|
161 | |||
160 | datadict = \ |
|
162 | datadict = \ | |
161 | co_day_auth_aggr[akc(cs.author)]['data'][time_pos] |
|
163 | co_day_auth_aggr[akc(cs.author)]['data'][time_pos] | |
162 |
|
164 | |||
163 | datadict["commits"] += 1 |
|
165 | datadict["commits"] += 1 | |
164 | datadict["added"] += len(cs.added) |
|
166 | datadict["added"] += len(cs.added) | |
165 | datadict["changed"] += len(cs.changed) |
|
167 | datadict["changed"] += len(cs.changed) | |
166 | datadict["removed"] += len(cs.removed) |
|
168 | datadict["removed"] += len(cs.removed) | |
167 |
|
169 | |||
168 | else: |
|
170 | else: | |
169 | if k >= ts_min_y and k <= ts_max_y or skip_date_limit: |
|
171 | if k >= ts_min_y and k <= ts_max_y or skip_date_limit: | |
170 |
|
172 | |||
171 | datadict = {"time": k, |
|
173 | datadict = {"time": k, | |
172 | "commits": 1, |
|
174 | "commits": 1, | |
173 | "added": len(cs.added), |
|
175 | "added": len(cs.added), | |
174 | "changed": len(cs.changed), |
|
176 | "changed": len(cs.changed), | |
175 | "removed": len(cs.removed), |
|
177 | "removed": len(cs.removed), | |
176 | } |
|
178 | } | |
177 | co_day_auth_aggr[akc(cs.author)]['data']\ |
|
179 | co_day_auth_aggr[akc(cs.author)]['data']\ | |
178 | .append(datadict) |
|
180 | .append(datadict) | |
179 |
|
181 | |||
180 | else: |
|
182 | else: | |
181 | if k >= ts_min_y and k <= ts_max_y or skip_date_limit: |
|
183 | if k >= ts_min_y and k <= ts_max_y or skip_date_limit: | |
182 | co_day_auth_aggr[akc(cs.author)] = { |
|
184 | co_day_auth_aggr[akc(cs.author)] = { | |
183 | "label": akc(cs.author), |
|
185 | "label": akc(cs.author), | |
184 | "data": [{"time":k, |
|
186 | "data": [{"time":k, | |
185 | "commits":1, |
|
187 | "commits":1, | |
186 | "added":len(cs.added), |
|
188 | "added":len(cs.added), | |
187 | "changed":len(cs.changed), |
|
189 | "changed":len(cs.changed), | |
188 | "removed":len(cs.removed), |
|
190 | "removed":len(cs.removed), | |
189 | }], |
|
191 | }], | |
190 | "schema": ["commits"], |
|
192 | "schema": ["commits"], | |
191 | } |
|
193 | } | |
192 |
|
194 | |||
193 | #gather all data by day |
|
195 | #gather all data by day | |
194 | if k in commits_by_day_aggregate: |
|
196 | if k in commits_by_day_aggregate: | |
195 | commits_by_day_aggregate[k] += 1 |
|
197 | commits_by_day_aggregate[k] += 1 | |
196 | else: |
|
198 | else: | |
197 | commits_by_day_aggregate[k] = 1 |
|
199 | commits_by_day_aggregate[k] = 1 | |
198 |
|
200 | |||
199 | overview_data = sorted(commits_by_day_aggregate.items(), |
|
201 | overview_data = sorted(commits_by_day_aggregate.items(), | |
200 | key=itemgetter(0)) |
|
202 | key=itemgetter(0)) | |
201 |
|
203 | |||
202 | if not co_day_auth_aggr: |
|
204 | if not co_day_auth_aggr: | |
203 | co_day_auth_aggr[akc(repo.contact)] = { |
|
205 | co_day_auth_aggr[akc(repo.contact)] = { | |
204 | "label": akc(repo.contact), |
|
206 | "label": akc(repo.contact), | |
205 | "data": [0, 1], |
|
207 | "data": [0, 1], | |
206 | "schema": ["commits"], |
|
208 | "schema": ["commits"], | |
207 | } |
|
209 | } | |
208 |
|
210 | |||
209 | stats = cur_stats if cur_stats else Statistics() |
|
211 | stats = cur_stats if cur_stats else Statistics() | |
210 | stats.commit_activity = json.dumps(co_day_auth_aggr) |
|
212 | stats.commit_activity = json.dumps(co_day_auth_aggr) | |
211 | stats.commit_activity_combined = json.dumps(overview_data) |
|
213 | stats.commit_activity_combined = json.dumps(overview_data) | |
212 |
|
214 | |||
213 | log.debug('last revision %s', last_rev) |
|
215 | log.debug('last revision %s', last_rev) | |
214 | leftovers = len(repo.revisions[last_rev:]) |
|
216 | leftovers = len(repo.revisions[last_rev:]) | |
215 | log.debug('revisions to parse %s', leftovers) |
|
217 | log.debug('revisions to parse %s', leftovers) | |
216 |
|
218 | |||
217 | if last_rev == 0 or leftovers < parse_limit: |
|
219 | if last_rev == 0 or leftovers < parse_limit: | |
218 | log.debug('getting code trending stats') |
|
220 | log.debug('getting code trending stats') | |
219 | stats.languages = json.dumps(__get_codes_stats(repo_name)) |
|
221 | stats.languages = json.dumps(__get_codes_stats(repo_name)) | |
220 |
|
222 | |||
221 | try: |
|
223 | try: | |
222 | stats.repository = dbrepo |
|
224 | stats.repository = dbrepo | |
223 | stats.stat_on_revision = last_cs.revision if last_cs else 0 |
|
225 | stats.stat_on_revision = last_cs.revision if last_cs else 0 | |
224 | DBS.add(stats) |
|
226 | DBS.add(stats) | |
225 | DBS.commit() |
|
227 | DBS.commit() | |
226 | except: |
|
228 | except: | |
227 | log.error(traceback.format_exc()) |
|
229 | log.error(traceback.format_exc()) | |
228 | DBS.rollback() |
|
230 | DBS.rollback() | |
229 | lock.release() |
|
231 | lock.release() | |
230 | return False |
|
232 | return False | |
231 |
|
233 | |||
232 | # final release |
|
234 | # final release | |
233 | lock.release() |
|
235 | lock.release() | |
234 |
|
236 | |||
235 | # execute another task if celery is enabled |
|
237 | # execute another task if celery is enabled | |
236 | if len(repo.revisions) > 1 and CELERY_ON and recurse_limit > 0: |
|
238 | if len(repo.revisions) > 1 and CELERY_ON and recurse_limit > 0: | |
237 | recurse_limit -= 1 |
|
239 | recurse_limit -= 1 | |
238 | run_task(get_commits_stats, repo_name, ts_min_y, ts_max_y, |
|
240 | run_task(get_commits_stats, repo_name, ts_min_y, ts_max_y, | |
239 | recurse_limit) |
|
241 | recurse_limit) | |
240 | if recurse_limit <= 0: |
|
242 | if recurse_limit <= 0: | |
241 | log.debug('Breaking recursive mode due to reach of recurse limit') |
|
243 | log.debug('Breaking recursive mode due to reach of recurse limit') | |
242 | return True |
|
244 | return True | |
243 | except LockHeld: |
|
245 | except LockHeld: | |
244 | log.info('LockHeld') |
|
246 | log.info('LockHeld') | |
245 | return 'Task with key %s already running' % lockkey |
|
247 | return 'Task with key %s already running' % lockkey | |
246 |
|
248 | |||
247 |
|
249 | |||
248 | @task(ignore_result=True) |
|
250 | @task(ignore_result=True) | |
249 | @dbsession |
|
251 | @dbsession | |
250 | def send_email(recipients, subject, body='', html_body='', headers=None): |
|
252 | def send_email(recipients, subject, body='', html_body='', headers=None, author=None): | |
251 | """ |
|
253 | """ | |
252 | Sends an email with defined parameters from the .ini files. |
|
254 | Sends an email with defined parameters from the .ini files. | |
253 |
|
255 | |||
254 | :param recipients: list of recipients, if this is None, the defined email |
|
256 | :param recipients: list of recipients, if this is None, the defined email | |
255 | address from field 'email_to' and all admins is used instead |
|
257 | address from field 'email_to' and all admins is used instead | |
256 | :param subject: subject of the mail |
|
258 | :param subject: subject of the mail | |
257 | :param body: body of the mail |
|
259 | :param body: body of the mail | |
258 | :param html_body: html version of body |
|
260 | :param html_body: html version of body | |
|
261 | :param headers: dictionary of prepopulated e-mail headers | |||
|
262 | :param author: User object of the author of this mail, if known and relevant | |||
259 | """ |
|
263 | """ | |
260 | log = get_logger(send_email) |
|
264 | log = get_logger(send_email) | |
261 | assert isinstance(recipients, list), recipients |
|
265 | assert isinstance(recipients, list), recipients | |
|
266 | if headers is None: | |||
|
267 | headers = {} | |||
|
268 | else: | |||
|
269 | # do not modify the original headers object passed by the caller | |||
|
270 | headers = headers.copy() | |||
262 |
|
271 | |||
263 | email_config = config |
|
272 | email_config = config | |
264 | email_prefix = email_config.get('email_prefix', '') |
|
273 | email_prefix = email_config.get('email_prefix', '') | |
265 | if email_prefix: |
|
274 | if email_prefix: | |
266 | subject = "%s %s" % (email_prefix, subject) |
|
275 | subject = "%s %s" % (email_prefix, subject) | |
267 |
|
276 | |||
268 | if not recipients: |
|
277 | if not recipients: | |
269 | # if recipients are not defined we send to email_config + all admins |
|
278 | # if recipients are not defined we send to email_config + all admins | |
270 | recipients = [u.email for u in User.query() |
|
279 | recipients = [u.email for u in User.query() | |
271 | .filter(User.admin == True).all()] |
|
280 | .filter(User.admin == True).all()] | |
272 | if email_config.get('email_to') is not None: |
|
281 | if email_config.get('email_to') is not None: | |
273 | recipients += [email_config.get('email_to')] |
|
282 | recipients += [email_config.get('email_to')] | |
274 |
|
283 | |||
275 | # If there are still no recipients, there are no admins and no address |
|
284 | # If there are still no recipients, there are no admins and no address | |
276 | # configured in email_to, so return. |
|
285 | # configured in email_to, so return. | |
277 | if not recipients: |
|
286 | if not recipients: | |
278 | log.error("No recipients specified and no fallback available.") |
|
287 | log.error("No recipients specified and no fallback available.") | |
279 | return False |
|
288 | return False | |
280 |
|
289 | |||
281 | log.warning("No recipients specified for '%s' - sending to admins %s", subject, ' '.join(recipients)) |
|
290 | log.warning("No recipients specified for '%s' - sending to admins %s", subject, ' '.join(recipients)) | |
282 |
|
291 | |||
283 | mail_from = email_config.get('app_email_from', 'Kallithea') |
|
292 | # SMTP sender | |
|
293 | envelope_from = email_config.get('app_email_from', 'Kallithea') | |||
|
294 | # 'From' header | |||
|
295 | if author is not None: | |||
|
296 | # set From header based on author but with a generic e-mail address | |||
|
297 | # In case app_email_from is in "Some Name <e-mail>" format, we first | |||
|
298 | # extract the e-mail address. | |||
|
299 | envelope_addr = author_email(envelope_from) | |||
|
300 | headers['From'] = '"%s" <%s>' % ( | |||
|
301 | rfc822.quote('%s (no-reply)' % author.full_name_or_username), | |||
|
302 | envelope_addr) | |||
|
303 | ||||
284 | user = email_config.get('smtp_username') |
|
304 | user = email_config.get('smtp_username') | |
285 | passwd = email_config.get('smtp_password') |
|
305 | passwd = email_config.get('smtp_password') | |
286 | mail_server = email_config.get('smtp_server') |
|
306 | mail_server = email_config.get('smtp_server') | |
287 | mail_port = email_config.get('smtp_port') |
|
307 | mail_port = email_config.get('smtp_port') | |
288 | tls = str2bool(email_config.get('smtp_use_tls')) |
|
308 | tls = str2bool(email_config.get('smtp_use_tls')) | |
289 | ssl = str2bool(email_config.get('smtp_use_ssl')) |
|
309 | ssl = str2bool(email_config.get('smtp_use_ssl')) | |
290 | debug = str2bool(email_config.get('debug')) |
|
310 | debug = str2bool(email_config.get('debug')) | |
291 | smtp_auth = email_config.get('smtp_auth') |
|
311 | smtp_auth = email_config.get('smtp_auth') | |
292 |
|
312 | |||
293 | logmsg = ("Mail details:\n" |
|
313 | logmsg = ("Mail details:\n" | |
294 | "recipients: %s\n" |
|
314 | "recipients: %s\n" | |
295 | "headers: %s\n" |
|
315 | "headers: %s\n" | |
296 | "subject: %s\n" |
|
316 | "subject: %s\n" | |
297 | "body:\n%s\n" |
|
317 | "body:\n%s\n" | |
298 | "html:\n%s\n" |
|
318 | "html:\n%s\n" | |
299 | % (' '.join(recipients), headers, subject, body, html_body)) |
|
319 | % (' '.join(recipients), headers, subject, body, html_body)) | |
300 |
|
320 | |||
301 | if mail_server: |
|
321 | if mail_server: | |
302 | log.debug("Sending e-mail. " + logmsg) |
|
322 | log.debug("Sending e-mail. " + logmsg) | |
303 | else: |
|
323 | else: | |
304 | log.error("SMTP mail server not configured - cannot send e-mail.") |
|
324 | log.error("SMTP mail server not configured - cannot send e-mail.") | |
305 | log.warning(logmsg) |
|
325 | log.warning(logmsg) | |
306 | return False |
|
326 | return False | |
307 |
|
327 | |||
308 | try: |
|
328 | try: | |
309 |
m = SmtpMailer( |
|
329 | m = SmtpMailer(envelope_from, user, passwd, mail_server, smtp_auth, | |
310 | mail_port, ssl, tls, debug=debug) |
|
330 | mail_port, ssl, tls, debug=debug) | |
311 | m.send(recipients, subject, body, html_body, headers=headers) |
|
331 | m.send(recipients, subject, body, html_body, headers=headers) | |
312 | except: |
|
332 | except: | |
313 | log.error('Mail sending failed') |
|
333 | log.error('Mail sending failed') | |
314 | log.error(traceback.format_exc()) |
|
334 | log.error(traceback.format_exc()) | |
315 | return False |
|
335 | return False | |
316 | return True |
|
336 | return True | |
317 |
|
337 | |||
318 | @task(ignore_result=False) |
|
338 | @task(ignore_result=False) | |
319 | @dbsession |
|
339 | @dbsession | |
320 | def create_repo(form_data, cur_user): |
|
340 | def create_repo(form_data, cur_user): | |
321 | from kallithea.model.repo import RepoModel |
|
341 | from kallithea.model.repo import RepoModel | |
322 | from kallithea.model.user import UserModel |
|
342 | from kallithea.model.user import UserModel | |
323 | from kallithea.model.db import Setting |
|
343 | from kallithea.model.db import Setting | |
324 |
|
344 | |||
325 | log = get_logger(create_repo) |
|
345 | log = get_logger(create_repo) | |
326 | DBS = get_session() |
|
346 | DBS = get_session() | |
327 |
|
347 | |||
328 | cur_user = UserModel(DBS)._get_user(cur_user) |
|
348 | cur_user = UserModel(DBS)._get_user(cur_user) | |
329 |
|
349 | |||
330 | owner = cur_user |
|
350 | owner = cur_user | |
331 | repo_name = form_data['repo_name'] |
|
351 | repo_name = form_data['repo_name'] | |
332 | repo_name_full = form_data['repo_name_full'] |
|
352 | repo_name_full = form_data['repo_name_full'] | |
333 | repo_type = form_data['repo_type'] |
|
353 | repo_type = form_data['repo_type'] | |
334 | description = form_data['repo_description'] |
|
354 | description = form_data['repo_description'] | |
335 | private = form_data['repo_private'] |
|
355 | private = form_data['repo_private'] | |
336 | clone_uri = form_data.get('clone_uri') |
|
356 | clone_uri = form_data.get('clone_uri') | |
337 | repo_group = form_data['repo_group'] |
|
357 | repo_group = form_data['repo_group'] | |
338 | landing_rev = form_data['repo_landing_rev'] |
|
358 | landing_rev = form_data['repo_landing_rev'] | |
339 | copy_fork_permissions = form_data.get('copy_permissions') |
|
359 | copy_fork_permissions = form_data.get('copy_permissions') | |
340 | copy_group_permissions = form_data.get('repo_copy_permissions') |
|
360 | copy_group_permissions = form_data.get('repo_copy_permissions') | |
341 | fork_of = form_data.get('fork_parent_id') |
|
361 | fork_of = form_data.get('fork_parent_id') | |
342 | state = form_data.get('repo_state', Repository.STATE_PENDING) |
|
362 | state = form_data.get('repo_state', Repository.STATE_PENDING) | |
343 |
|
363 | |||
344 | # repo creation defaults, private and repo_type are filled in form |
|
364 | # repo creation defaults, private and repo_type are filled in form | |
345 | defs = Setting.get_default_repo_settings(strip_prefix=True) |
|
365 | defs = Setting.get_default_repo_settings(strip_prefix=True) | |
346 | enable_statistics = defs.get('repo_enable_statistics') |
|
366 | enable_statistics = defs.get('repo_enable_statistics') | |
347 | enable_locking = defs.get('repo_enable_locking') |
|
367 | enable_locking = defs.get('repo_enable_locking') | |
348 | enable_downloads = defs.get('repo_enable_downloads') |
|
368 | enable_downloads = defs.get('repo_enable_downloads') | |
349 |
|
369 | |||
350 | try: |
|
370 | try: | |
351 | repo = RepoModel(DBS)._create_repo( |
|
371 | repo = RepoModel(DBS)._create_repo( | |
352 | repo_name=repo_name_full, |
|
372 | repo_name=repo_name_full, | |
353 | repo_type=repo_type, |
|
373 | repo_type=repo_type, | |
354 | description=description, |
|
374 | description=description, | |
355 | owner=owner, |
|
375 | owner=owner, | |
356 | private=private, |
|
376 | private=private, | |
357 | clone_uri=clone_uri, |
|
377 | clone_uri=clone_uri, | |
358 | repo_group=repo_group, |
|
378 | repo_group=repo_group, | |
359 | landing_rev=landing_rev, |
|
379 | landing_rev=landing_rev, | |
360 | fork_of=fork_of, |
|
380 | fork_of=fork_of, | |
361 | copy_fork_permissions=copy_fork_permissions, |
|
381 | copy_fork_permissions=copy_fork_permissions, | |
362 | copy_group_permissions=copy_group_permissions, |
|
382 | copy_group_permissions=copy_group_permissions, | |
363 | enable_statistics=enable_statistics, |
|
383 | enable_statistics=enable_statistics, | |
364 | enable_locking=enable_locking, |
|
384 | enable_locking=enable_locking, | |
365 | enable_downloads=enable_downloads, |
|
385 | enable_downloads=enable_downloads, | |
366 | state=state |
|
386 | state=state | |
367 | ) |
|
387 | ) | |
368 |
|
388 | |||
369 | action_logger(cur_user, 'user_created_repo', |
|
389 | action_logger(cur_user, 'user_created_repo', | |
370 | form_data['repo_name_full'], '', DBS) |
|
390 | form_data['repo_name_full'], '', DBS) | |
371 |
|
391 | |||
372 | DBS.commit() |
|
392 | DBS.commit() | |
373 | # now create this repo on Filesystem |
|
393 | # now create this repo on Filesystem | |
374 | RepoModel(DBS)._create_filesystem_repo( |
|
394 | RepoModel(DBS)._create_filesystem_repo( | |
375 | repo_name=repo_name, |
|
395 | repo_name=repo_name, | |
376 | repo_type=repo_type, |
|
396 | repo_type=repo_type, | |
377 | repo_group=RepoModel(DBS)._get_repo_group(repo_group), |
|
397 | repo_group=RepoModel(DBS)._get_repo_group(repo_group), | |
378 | clone_uri=clone_uri, |
|
398 | clone_uri=clone_uri, | |
379 | ) |
|
399 | ) | |
380 | repo = Repository.get_by_repo_name(repo_name_full) |
|
400 | repo = Repository.get_by_repo_name(repo_name_full) | |
381 | log_create_repository(repo.get_dict(), created_by=owner.username) |
|
401 | log_create_repository(repo.get_dict(), created_by=owner.username) | |
382 |
|
402 | |||
383 | # update repo changeset caches initially |
|
403 | # update repo changeset caches initially | |
384 | repo.update_changeset_cache() |
|
404 | repo.update_changeset_cache() | |
385 |
|
405 | |||
386 | # set new created state |
|
406 | # set new created state | |
387 | repo.set_state(Repository.STATE_CREATED) |
|
407 | repo.set_state(Repository.STATE_CREATED) | |
388 | DBS.commit() |
|
408 | DBS.commit() | |
389 | except Exception as e: |
|
409 | except Exception as e: | |
390 | log.warning('Exception %s occurred when forking repository, ' |
|
410 | log.warning('Exception %s occurred when forking repository, ' | |
391 | 'doing cleanup...' % e) |
|
411 | 'doing cleanup...' % e) | |
392 | # rollback things manually ! |
|
412 | # rollback things manually ! | |
393 | repo = Repository.get_by_repo_name(repo_name_full) |
|
413 | repo = Repository.get_by_repo_name(repo_name_full) | |
394 | if repo: |
|
414 | if repo: | |
395 | Repository.delete(repo.repo_id) |
|
415 | Repository.delete(repo.repo_id) | |
396 | DBS.commit() |
|
416 | DBS.commit() | |
397 | RepoModel(DBS)._delete_filesystem_repo(repo) |
|
417 | RepoModel(DBS)._delete_filesystem_repo(repo) | |
398 | raise |
|
418 | raise | |
399 |
|
419 | |||
400 | # it's an odd fix to make celery fail task when exception occurs |
|
420 | # it's an odd fix to make celery fail task when exception occurs | |
401 | def on_failure(self, *args, **kwargs): |
|
421 | def on_failure(self, *args, **kwargs): | |
402 | pass |
|
422 | pass | |
403 |
|
423 | |||
404 | return True |
|
424 | return True | |
405 |
|
425 | |||
406 |
|
426 | |||
407 | @task(ignore_result=False) |
|
427 | @task(ignore_result=False) | |
408 | @dbsession |
|
428 | @dbsession | |
409 | def create_repo_fork(form_data, cur_user): |
|
429 | def create_repo_fork(form_data, cur_user): | |
410 | """ |
|
430 | """ | |
411 | Creates a fork of repository using interval VCS methods |
|
431 | Creates a fork of repository using interval VCS methods | |
412 |
|
432 | |||
413 | :param form_data: |
|
433 | :param form_data: | |
414 | :param cur_user: |
|
434 | :param cur_user: | |
415 | """ |
|
435 | """ | |
416 | from kallithea.model.repo import RepoModel |
|
436 | from kallithea.model.repo import RepoModel | |
417 | from kallithea.model.user import UserModel |
|
437 | from kallithea.model.user import UserModel | |
418 |
|
438 | |||
419 | log = get_logger(create_repo_fork) |
|
439 | log = get_logger(create_repo_fork) | |
420 | DBS = get_session() |
|
440 | DBS = get_session() | |
421 |
|
441 | |||
422 | base_path = Repository.base_path() |
|
442 | base_path = Repository.base_path() | |
423 | cur_user = UserModel(DBS)._get_user(cur_user) |
|
443 | cur_user = UserModel(DBS)._get_user(cur_user) | |
424 |
|
444 | |||
425 | repo_name = form_data['repo_name'] # fork in this case |
|
445 | repo_name = form_data['repo_name'] # fork in this case | |
426 | repo_name_full = form_data['repo_name_full'] |
|
446 | repo_name_full = form_data['repo_name_full'] | |
427 |
|
447 | |||
428 | repo_type = form_data['repo_type'] |
|
448 | repo_type = form_data['repo_type'] | |
429 | owner = cur_user |
|
449 | owner = cur_user | |
430 | private = form_data['private'] |
|
450 | private = form_data['private'] | |
431 | clone_uri = form_data.get('clone_uri') |
|
451 | clone_uri = form_data.get('clone_uri') | |
432 | repo_group = form_data['repo_group'] |
|
452 | repo_group = form_data['repo_group'] | |
433 | landing_rev = form_data['landing_rev'] |
|
453 | landing_rev = form_data['landing_rev'] | |
434 | copy_fork_permissions = form_data.get('copy_permissions') |
|
454 | copy_fork_permissions = form_data.get('copy_permissions') | |
435 |
|
455 | |||
436 | try: |
|
456 | try: | |
437 | fork_of = RepoModel(DBS)._get_repo(form_data.get('fork_parent_id')) |
|
457 | fork_of = RepoModel(DBS)._get_repo(form_data.get('fork_parent_id')) | |
438 |
|
458 | |||
439 | RepoModel(DBS)._create_repo( |
|
459 | RepoModel(DBS)._create_repo( | |
440 | repo_name=repo_name_full, |
|
460 | repo_name=repo_name_full, | |
441 | repo_type=repo_type, |
|
461 | repo_type=repo_type, | |
442 | description=form_data['description'], |
|
462 | description=form_data['description'], | |
443 | owner=owner, |
|
463 | owner=owner, | |
444 | private=private, |
|
464 | private=private, | |
445 | clone_uri=clone_uri, |
|
465 | clone_uri=clone_uri, | |
446 | repo_group=repo_group, |
|
466 | repo_group=repo_group, | |
447 | landing_rev=landing_rev, |
|
467 | landing_rev=landing_rev, | |
448 | fork_of=fork_of, |
|
468 | fork_of=fork_of, | |
449 | copy_fork_permissions=copy_fork_permissions |
|
469 | copy_fork_permissions=copy_fork_permissions | |
450 | ) |
|
470 | ) | |
451 | action_logger(cur_user, 'user_forked_repo:%s' % repo_name_full, |
|
471 | action_logger(cur_user, 'user_forked_repo:%s' % repo_name_full, | |
452 | fork_of.repo_name, '', DBS) |
|
472 | fork_of.repo_name, '', DBS) | |
453 | DBS.commit() |
|
473 | DBS.commit() | |
454 |
|
474 | |||
455 | update_after_clone = form_data['update_after_clone'] # FIXME - unused! |
|
475 | update_after_clone = form_data['update_after_clone'] # FIXME - unused! | |
456 | source_repo_path = os.path.join(base_path, fork_of.repo_name) |
|
476 | source_repo_path = os.path.join(base_path, fork_of.repo_name) | |
457 |
|
477 | |||
458 | # now create this repo on Filesystem |
|
478 | # now create this repo on Filesystem | |
459 | RepoModel(DBS)._create_filesystem_repo( |
|
479 | RepoModel(DBS)._create_filesystem_repo( | |
460 | repo_name=repo_name, |
|
480 | repo_name=repo_name, | |
461 | repo_type=repo_type, |
|
481 | repo_type=repo_type, | |
462 | repo_group=RepoModel(DBS)._get_repo_group(repo_group), |
|
482 | repo_group=RepoModel(DBS)._get_repo_group(repo_group), | |
463 | clone_uri=source_repo_path, |
|
483 | clone_uri=source_repo_path, | |
464 | ) |
|
484 | ) | |
465 | repo = Repository.get_by_repo_name(repo_name_full) |
|
485 | repo = Repository.get_by_repo_name(repo_name_full) | |
466 | log_create_repository(repo.get_dict(), created_by=owner.username) |
|
486 | log_create_repository(repo.get_dict(), created_by=owner.username) | |
467 |
|
487 | |||
468 | # update repo changeset caches initially |
|
488 | # update repo changeset caches initially | |
469 | repo.update_changeset_cache() |
|
489 | repo.update_changeset_cache() | |
470 |
|
490 | |||
471 | # set new created state |
|
491 | # set new created state | |
472 | repo.set_state(Repository.STATE_CREATED) |
|
492 | repo.set_state(Repository.STATE_CREATED) | |
473 | DBS.commit() |
|
493 | DBS.commit() | |
474 | except Exception as e: |
|
494 | except Exception as e: | |
475 | log.warning('Exception %s occurred when forking repository, ' |
|
495 | log.warning('Exception %s occurred when forking repository, ' | |
476 | 'doing cleanup...' % e) |
|
496 | 'doing cleanup...' % e) | |
477 | #rollback things manually ! |
|
497 | #rollback things manually ! | |
478 | repo = Repository.get_by_repo_name(repo_name_full) |
|
498 | repo = Repository.get_by_repo_name(repo_name_full) | |
479 | if repo: |
|
499 | if repo: | |
480 | Repository.delete(repo.repo_id) |
|
500 | Repository.delete(repo.repo_id) | |
481 | DBS.commit() |
|
501 | DBS.commit() | |
482 | RepoModel(DBS)._delete_filesystem_repo(repo) |
|
502 | RepoModel(DBS)._delete_filesystem_repo(repo) | |
483 | raise |
|
503 | raise | |
484 |
|
504 | |||
485 | # it's an odd fix to make celery fail task when exception occurs |
|
505 | # it's an odd fix to make celery fail task when exception occurs | |
486 | def on_failure(self, *args, **kwargs): |
|
506 | def on_failure(self, *args, **kwargs): | |
487 | pass |
|
507 | pass | |
488 |
|
508 | |||
489 | return True |
|
509 | return True | |
490 |
|
510 | |||
491 |
|
511 | |||
492 | def __get_codes_stats(repo_name): |
|
512 | def __get_codes_stats(repo_name): | |
493 | from kallithea.config.conf import LANGUAGES_EXTENSIONS_MAP |
|
513 | from kallithea.config.conf import LANGUAGES_EXTENSIONS_MAP | |
494 | repo = Repository.get_by_repo_name(repo_name).scm_instance |
|
514 | repo = Repository.get_by_repo_name(repo_name).scm_instance | |
495 |
|
515 | |||
496 | tip = repo.get_changeset() |
|
516 | tip = repo.get_changeset() | |
497 | code_stats = {} |
|
517 | code_stats = {} | |
498 |
|
518 | |||
499 | def aggregate(cs): |
|
519 | def aggregate(cs): | |
500 | for f in cs[2]: |
|
520 | for f in cs[2]: | |
501 | ext = lower(f.extension) |
|
521 | ext = lower(f.extension) | |
502 | if ext in LANGUAGES_EXTENSIONS_MAP.keys() and not f.is_binary: |
|
522 | if ext in LANGUAGES_EXTENSIONS_MAP.keys() and not f.is_binary: | |
503 | if ext in code_stats: |
|
523 | if ext in code_stats: | |
504 | code_stats[ext] += 1 |
|
524 | code_stats[ext] += 1 | |
505 | else: |
|
525 | else: | |
506 | code_stats[ext] = 1 |
|
526 | code_stats[ext] = 1 | |
507 |
|
527 | |||
508 | map(aggregate, tip.walk('/')) |
|
528 | map(aggregate, tip.walk('/')) | |
509 |
|
529 | |||
510 | return code_stats or {} |
|
530 | return code_stats or {} |
@@ -1,342 +1,342 b'' | |||||
1 | # -*- coding: utf-8 -*- |
|
1 | # -*- coding: utf-8 -*- | |
2 | # This program is free software: you can redistribute it and/or modify |
|
2 | # This program is free software: you can redistribute it and/or modify | |
3 | # it under the terms of the GNU General Public License as published by |
|
3 | # it under the terms of the GNU General Public License as published by | |
4 | # the Free Software Foundation, either version 3 of the License, or |
|
4 | # the Free Software Foundation, either version 3 of the License, or | |
5 | # (at your option) any later version. |
|
5 | # (at your option) any later version. | |
6 | # |
|
6 | # | |
7 | # This program is distributed in the hope that it will be useful, |
|
7 | # This program is distributed in the hope that it will be useful, | |
8 | # but WITHOUT ANY WARRANTY; without even the implied warranty of |
|
8 | # but WITHOUT ANY WARRANTY; without even the implied warranty of | |
9 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
|
9 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | |
10 | # GNU General Public License for more details. |
|
10 | # GNU General Public License for more details. | |
11 | # |
|
11 | # | |
12 | # You should have received a copy of the GNU General Public License |
|
12 | # You should have received a copy of the GNU General Public License | |
13 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
|
13 | # along with this program. If not, see <http://www.gnu.org/licenses/>. | |
14 | """ |
|
14 | """ | |
15 | kallithea.model.notification |
|
15 | kallithea.model.notification | |
16 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
16 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
17 |
|
17 | |||
18 | Model for notifications |
|
18 | Model for notifications | |
19 |
|
19 | |||
20 |
|
20 | |||
21 | This file was forked by the Kallithea project in July 2014. |
|
21 | This file was forked by the Kallithea project in July 2014. | |
22 | Original author and date, and relevant copyright and licensing information is below: |
|
22 | Original author and date, and relevant copyright and licensing information is below: | |
23 | :created_on: Nov 20, 2011 |
|
23 | :created_on: Nov 20, 2011 | |
24 | :author: marcink |
|
24 | :author: marcink | |
25 | :copyright: (c) 2013 RhodeCode GmbH, and others. |
|
25 | :copyright: (c) 2013 RhodeCode GmbH, and others. | |
26 | :license: GPLv3, see LICENSE.md for more details. |
|
26 | :license: GPLv3, see LICENSE.md for more details. | |
27 | """ |
|
27 | """ | |
28 |
|
28 | |||
29 | import logging |
|
29 | import logging | |
30 | import traceback |
|
30 | import traceback | |
31 |
|
31 | |||
32 | from pylons import tmpl_context as c |
|
32 | from pylons import tmpl_context as c | |
33 | from pylons.i18n.translation import _ |
|
33 | from pylons.i18n.translation import _ | |
34 | from sqlalchemy.orm import joinedload, subqueryload |
|
34 | from sqlalchemy.orm import joinedload, subqueryload | |
35 |
|
35 | |||
36 | import kallithea |
|
36 | import kallithea | |
37 | from kallithea.lib import helpers as h |
|
37 | from kallithea.lib import helpers as h | |
38 | from kallithea.lib.utils2 import safe_unicode |
|
38 | from kallithea.lib.utils2 import safe_unicode | |
39 | from kallithea.model import BaseModel |
|
39 | from kallithea.model import BaseModel | |
40 | from kallithea.model.db import Notification, User, UserNotification |
|
40 | from kallithea.model.db import Notification, User, UserNotification | |
41 | from kallithea.model.meta import Session |
|
41 | from kallithea.model.meta import Session | |
42 |
|
42 | |||
43 | log = logging.getLogger(__name__) |
|
43 | log = logging.getLogger(__name__) | |
44 |
|
44 | |||
45 |
|
45 | |||
46 | class NotificationModel(BaseModel): |
|
46 | class NotificationModel(BaseModel): | |
47 |
|
47 | |||
48 | cls = Notification |
|
48 | cls = Notification | |
49 |
|
49 | |||
50 | def __get_notification(self, notification): |
|
50 | def __get_notification(self, notification): | |
51 | if isinstance(notification, Notification): |
|
51 | if isinstance(notification, Notification): | |
52 | return notification |
|
52 | return notification | |
53 | elif isinstance(notification, (int, long)): |
|
53 | elif isinstance(notification, (int, long)): | |
54 | return Notification.get(notification) |
|
54 | return Notification.get(notification) | |
55 | else: |
|
55 | else: | |
56 | if notification is not None: |
|
56 | if notification is not None: | |
57 | raise Exception('notification must be int, long or Instance' |
|
57 | raise Exception('notification must be int, long or Instance' | |
58 | ' of Notification got %s' % type(notification)) |
|
58 | ' of Notification got %s' % type(notification)) | |
59 |
|
59 | |||
60 | def create(self, created_by, subject, body, recipients=None, |
|
60 | def create(self, created_by, subject, body, recipients=None, | |
61 | type_=Notification.TYPE_MESSAGE, with_email=True, |
|
61 | type_=Notification.TYPE_MESSAGE, with_email=True, | |
62 | email_kwargs={}): |
|
62 | email_kwargs={}): | |
63 | """ |
|
63 | """ | |
64 |
|
64 | |||
65 | Creates notification of given type |
|
65 | Creates notification of given type | |
66 |
|
66 | |||
67 | :param created_by: int, str or User instance. User who created this |
|
67 | :param created_by: int, str or User instance. User who created this | |
68 | notification |
|
68 | notification | |
69 | :param subject: |
|
69 | :param subject: | |
70 | :param body: |
|
70 | :param body: | |
71 | :param recipients: list of int, str or User objects, when None |
|
71 | :param recipients: list of int, str or User objects, when None | |
72 | is given send to all admins |
|
72 | is given send to all admins | |
73 | :param type_: type of notification |
|
73 | :param type_: type of notification | |
74 | :param with_email: send email with this notification |
|
74 | :param with_email: send email with this notification | |
75 | :param email_kwargs: additional dict to pass as args to email template |
|
75 | :param email_kwargs: additional dict to pass as args to email template | |
76 | """ |
|
76 | """ | |
77 | from kallithea.lib.celerylib import tasks, run_task |
|
77 | from kallithea.lib.celerylib import tasks, run_task | |
78 |
|
78 | |||
79 | if recipients and not getattr(recipients, '__iter__', False): |
|
79 | if recipients and not getattr(recipients, '__iter__', False): | |
80 | raise Exception('recipients must be a list or iterable') |
|
80 | raise Exception('recipients must be a list or iterable') | |
81 |
|
81 | |||
82 | created_by_obj = self._get_user(created_by) |
|
82 | created_by_obj = self._get_user(created_by) | |
83 |
|
83 | |||
84 | recipients_objs = [] |
|
84 | recipients_objs = [] | |
85 | if recipients: |
|
85 | if recipients: | |
86 | for u in recipients: |
|
86 | for u in recipients: | |
87 | obj = self._get_user(u) |
|
87 | obj = self._get_user(u) | |
88 | if obj is not None: |
|
88 | if obj is not None: | |
89 | recipients_objs.append(obj) |
|
89 | recipients_objs.append(obj) | |
90 | else: |
|
90 | else: | |
91 | # TODO: inform user that requested operation couldn't be completed |
|
91 | # TODO: inform user that requested operation couldn't be completed | |
92 | log.error('cannot email unknown user %r', u) |
|
92 | log.error('cannot email unknown user %r', u) | |
93 | recipients_objs = set(recipients_objs) |
|
93 | recipients_objs = set(recipients_objs) | |
94 | log.debug('sending notifications %s to %s', |
|
94 | log.debug('sending notifications %s to %s', | |
95 | type_, recipients_objs |
|
95 | type_, recipients_objs | |
96 | ) |
|
96 | ) | |
97 | elif recipients is None: |
|
97 | elif recipients is None: | |
98 | # empty recipients means to all admins |
|
98 | # empty recipients means to all admins | |
99 | recipients_objs = User.query().filter(User.admin == True).all() |
|
99 | recipients_objs = User.query().filter(User.admin == True).all() | |
100 | log.debug('sending notifications %s to admins: %s', |
|
100 | log.debug('sending notifications %s to admins: %s', | |
101 | type_, recipients_objs |
|
101 | type_, recipients_objs | |
102 | ) |
|
102 | ) | |
103 | #else: silently skip notification mails? |
|
103 | #else: silently skip notification mails? | |
104 |
|
104 | |||
105 | # TODO: inform user who are notified |
|
105 | # TODO: inform user who are notified | |
106 | notif = Notification.create( |
|
106 | notif = Notification.create( | |
107 | created_by=created_by_obj, subject=subject, |
|
107 | created_by=created_by_obj, subject=subject, | |
108 | body=body, recipients=recipients_objs, type_=type_ |
|
108 | body=body, recipients=recipients_objs, type_=type_ | |
109 | ) |
|
109 | ) | |
110 |
|
110 | |||
111 | if not with_email: |
|
111 | if not with_email: | |
112 | return notif |
|
112 | return notif | |
113 |
|
113 | |||
114 | #don't send email to person who created this comment |
|
114 | #don't send email to person who created this comment | |
115 | rec_objs = set(recipients_objs).difference(set([created_by_obj])) |
|
115 | rec_objs = set(recipients_objs).difference(set([created_by_obj])) | |
116 |
|
116 | |||
117 | headers = None |
|
117 | headers = None | |
118 | if 'threading' in email_kwargs: |
|
118 | if 'threading' in email_kwargs: | |
119 | headers = {'References': ' '.join('<%s>' % x for x in email_kwargs['threading'])} |
|
119 | headers = {'References': ' '.join('<%s>' % x for x in email_kwargs['threading'])} | |
120 |
|
120 | |||
121 | # send email with notification to all other participants |
|
121 | # send email with notification to all other participants | |
122 | for rec in rec_objs: |
|
122 | for rec in rec_objs: | |
123 | ## this is passed into template |
|
123 | ## this is passed into template | |
124 | html_kwargs = { |
|
124 | html_kwargs = { | |
125 | 'subject': subject, |
|
125 | 'subject': subject, | |
126 | 'body': h.rst_w_mentions(body), |
|
126 | 'body': h.rst_w_mentions(body), | |
127 | 'when': h.fmt_date(notif.created_on), |
|
127 | 'when': h.fmt_date(notif.created_on), | |
128 | 'user': notif.created_by_user.username, |
|
128 | 'user': notif.created_by_user.username, | |
129 | } |
|
129 | } | |
130 |
|
130 | |||
131 | txt_kwargs = { |
|
131 | txt_kwargs = { | |
132 | 'subject': subject, |
|
132 | 'subject': subject, | |
133 | 'body': body, |
|
133 | 'body': body, | |
134 | 'when': h.fmt_date(notif.created_on), |
|
134 | 'when': h.fmt_date(notif.created_on), | |
135 | 'user': notif.created_by_user.username, |
|
135 | 'user': notif.created_by_user.username, | |
136 | } |
|
136 | } | |
137 |
|
137 | |||
138 | html_kwargs.update(email_kwargs) |
|
138 | html_kwargs.update(email_kwargs) | |
139 | txt_kwargs.update(email_kwargs) |
|
139 | txt_kwargs.update(email_kwargs) | |
140 | email_subject = EmailNotificationModel()\ |
|
140 | email_subject = EmailNotificationModel()\ | |
141 | .get_email_description(type_, **txt_kwargs) |
|
141 | .get_email_description(type_, **txt_kwargs) | |
142 | email_txt_body = EmailNotificationModel()\ |
|
142 | email_txt_body = EmailNotificationModel()\ | |
143 | .get_email_tmpl(type_, 'txt', **txt_kwargs) |
|
143 | .get_email_tmpl(type_, 'txt', **txt_kwargs) | |
144 | email_html_body = EmailNotificationModel()\ |
|
144 | email_html_body = EmailNotificationModel()\ | |
145 | .get_email_tmpl(type_, 'html', **html_kwargs) |
|
145 | .get_email_tmpl(type_, 'html', **html_kwargs) | |
146 |
|
146 | |||
147 | run_task(tasks.send_email, [rec.email], email_subject, email_txt_body, |
|
147 | run_task(tasks.send_email, [rec.email], email_subject, email_txt_body, | |
148 | email_html_body, headers) |
|
148 | email_html_body, headers, author=created_by_obj) | |
149 |
|
149 | |||
150 | return notif |
|
150 | return notif | |
151 |
|
151 | |||
152 | def delete(self, user, notification): |
|
152 | def delete(self, user, notification): | |
153 | # we don't want to remove actual notification just the assignment |
|
153 | # we don't want to remove actual notification just the assignment | |
154 | try: |
|
154 | try: | |
155 | notification = self.__get_notification(notification) |
|
155 | notification = self.__get_notification(notification) | |
156 | user = self._get_user(user) |
|
156 | user = self._get_user(user) | |
157 | if notification and user: |
|
157 | if notification and user: | |
158 | obj = UserNotification.query()\ |
|
158 | obj = UserNotification.query()\ | |
159 | .filter(UserNotification.user == user)\ |
|
159 | .filter(UserNotification.user == user)\ | |
160 | .filter(UserNotification.notification |
|
160 | .filter(UserNotification.notification | |
161 | == notification)\ |
|
161 | == notification)\ | |
162 | .one() |
|
162 | .one() | |
163 | Session().delete(obj) |
|
163 | Session().delete(obj) | |
164 | return True |
|
164 | return True | |
165 | except Exception: |
|
165 | except Exception: | |
166 | log.error(traceback.format_exc()) |
|
166 | log.error(traceback.format_exc()) | |
167 | raise |
|
167 | raise | |
168 |
|
168 | |||
169 | def get_for_user(self, user, filter_=None): |
|
169 | def get_for_user(self, user, filter_=None): | |
170 | """ |
|
170 | """ | |
171 | Get notifications for given user, filter them if filter dict is given |
|
171 | Get notifications for given user, filter them if filter dict is given | |
172 |
|
172 | |||
173 | :param user: |
|
173 | :param user: | |
174 | :param filter: |
|
174 | :param filter: | |
175 | """ |
|
175 | """ | |
176 | user = self._get_user(user) |
|
176 | user = self._get_user(user) | |
177 |
|
177 | |||
178 | q = UserNotification.query()\ |
|
178 | q = UserNotification.query()\ | |
179 | .filter(UserNotification.user == user)\ |
|
179 | .filter(UserNotification.user == user)\ | |
180 | .join((Notification, UserNotification.notification_id == |
|
180 | .join((Notification, UserNotification.notification_id == | |
181 | Notification.notification_id))\ |
|
181 | Notification.notification_id))\ | |
182 | .options(joinedload('notification'))\ |
|
182 | .options(joinedload('notification'))\ | |
183 | .options(subqueryload('notification.created_by_user'))\ |
|
183 | .options(subqueryload('notification.created_by_user'))\ | |
184 | .order_by(Notification.created_on.desc()) |
|
184 | .order_by(Notification.created_on.desc()) | |
185 |
|
185 | |||
186 | if filter_: |
|
186 | if filter_: | |
187 | q = q.filter(Notification.type_.in_(filter_)) |
|
187 | q = q.filter(Notification.type_.in_(filter_)) | |
188 |
|
188 | |||
189 | return q.all() |
|
189 | return q.all() | |
190 |
|
190 | |||
191 | def mark_read(self, user, notification): |
|
191 | def mark_read(self, user, notification): | |
192 | try: |
|
192 | try: | |
193 | notification = self.__get_notification(notification) |
|
193 | notification = self.__get_notification(notification) | |
194 | user = self._get_user(user) |
|
194 | user = self._get_user(user) | |
195 | if notification and user: |
|
195 | if notification and user: | |
196 | obj = UserNotification.query()\ |
|
196 | obj = UserNotification.query()\ | |
197 | .filter(UserNotification.user == user)\ |
|
197 | .filter(UserNotification.user == user)\ | |
198 | .filter(UserNotification.notification |
|
198 | .filter(UserNotification.notification | |
199 | == notification)\ |
|
199 | == notification)\ | |
200 | .one() |
|
200 | .one() | |
201 | obj.read = True |
|
201 | obj.read = True | |
202 | Session().add(obj) |
|
202 | Session().add(obj) | |
203 | return True |
|
203 | return True | |
204 | except Exception: |
|
204 | except Exception: | |
205 | log.error(traceback.format_exc()) |
|
205 | log.error(traceback.format_exc()) | |
206 | raise |
|
206 | raise | |
207 |
|
207 | |||
208 | def mark_all_read_for_user(self, user, filter_=None): |
|
208 | def mark_all_read_for_user(self, user, filter_=None): | |
209 | user = self._get_user(user) |
|
209 | user = self._get_user(user) | |
210 | q = UserNotification.query()\ |
|
210 | q = UserNotification.query()\ | |
211 | .filter(UserNotification.user == user)\ |
|
211 | .filter(UserNotification.user == user)\ | |
212 | .filter(UserNotification.read == False)\ |
|
212 | .filter(UserNotification.read == False)\ | |
213 | .join((Notification, UserNotification.notification_id == |
|
213 | .join((Notification, UserNotification.notification_id == | |
214 | Notification.notification_id)) |
|
214 | Notification.notification_id)) | |
215 | if filter_: |
|
215 | if filter_: | |
216 | q = q.filter(Notification.type_.in_(filter_)) |
|
216 | q = q.filter(Notification.type_.in_(filter_)) | |
217 |
|
217 | |||
218 | # this is a little inefficient but sqlalchemy doesn't support |
|
218 | # this is a little inefficient but sqlalchemy doesn't support | |
219 | # update on joined tables :( |
|
219 | # update on joined tables :( | |
220 | for obj in q.all(): |
|
220 | for obj in q.all(): | |
221 | obj.read = True |
|
221 | obj.read = True | |
222 | Session().add(obj) |
|
222 | Session().add(obj) | |
223 |
|
223 | |||
224 | def get_unread_cnt_for_user(self, user): |
|
224 | def get_unread_cnt_for_user(self, user): | |
225 | user = self._get_user(user) |
|
225 | user = self._get_user(user) | |
226 | return UserNotification.query()\ |
|
226 | return UserNotification.query()\ | |
227 | .filter(UserNotification.read == False)\ |
|
227 | .filter(UserNotification.read == False)\ | |
228 | .filter(UserNotification.user == user).count() |
|
228 | .filter(UserNotification.user == user).count() | |
229 |
|
229 | |||
230 | def get_unread_for_user(self, user): |
|
230 | def get_unread_for_user(self, user): | |
231 | user = self._get_user(user) |
|
231 | user = self._get_user(user) | |
232 | return [x.notification for x in UserNotification.query()\ |
|
232 | return [x.notification for x in UserNotification.query()\ | |
233 | .filter(UserNotification.read == False)\ |
|
233 | .filter(UserNotification.read == False)\ | |
234 | .filter(UserNotification.user == user).all()] |
|
234 | .filter(UserNotification.user == user).all()] | |
235 |
|
235 | |||
236 | def get_user_notification(self, user, notification): |
|
236 | def get_user_notification(self, user, notification): | |
237 | user = self._get_user(user) |
|
237 | user = self._get_user(user) | |
238 | notification = self.__get_notification(notification) |
|
238 | notification = self.__get_notification(notification) | |
239 |
|
239 | |||
240 | return UserNotification.query()\ |
|
240 | return UserNotification.query()\ | |
241 | .filter(UserNotification.notification == notification)\ |
|
241 | .filter(UserNotification.notification == notification)\ | |
242 | .filter(UserNotification.user == user).scalar() |
|
242 | .filter(UserNotification.user == user).scalar() | |
243 |
|
243 | |||
244 | def make_description(self, notification, show_age=True): |
|
244 | def make_description(self, notification, show_age=True): | |
245 | """ |
|
245 | """ | |
246 | Creates a human readable description based on properties |
|
246 | Creates a human readable description based on properties | |
247 | of notification object |
|
247 | of notification object | |
248 | """ |
|
248 | """ | |
249 | #alias |
|
249 | #alias | |
250 | _n = notification |
|
250 | _n = notification | |
251 |
|
251 | |||
252 | if show_age: |
|
252 | if show_age: | |
253 | return { |
|
253 | return { | |
254 | _n.TYPE_CHANGESET_COMMENT: _('%(user)s commented on changeset %(age)s'), |
|
254 | _n.TYPE_CHANGESET_COMMENT: _('%(user)s commented on changeset %(age)s'), | |
255 | _n.TYPE_MESSAGE: _('%(user)s sent message %(age)s'), |
|
255 | _n.TYPE_MESSAGE: _('%(user)s sent message %(age)s'), | |
256 | _n.TYPE_MENTION: _('%(user)s mentioned you %(age)s'), |
|
256 | _n.TYPE_MENTION: _('%(user)s mentioned you %(age)s'), | |
257 | _n.TYPE_REGISTRATION: _('%(user)s registered in Kallithea %(age)s'), |
|
257 | _n.TYPE_REGISTRATION: _('%(user)s registered in Kallithea %(age)s'), | |
258 | _n.TYPE_PULL_REQUEST: _('%(user)s opened new pull request %(age)s'), |
|
258 | _n.TYPE_PULL_REQUEST: _('%(user)s opened new pull request %(age)s'), | |
259 | _n.TYPE_PULL_REQUEST_COMMENT: _('%(user)s commented on pull request %(age)s'), |
|
259 | _n.TYPE_PULL_REQUEST_COMMENT: _('%(user)s commented on pull request %(age)s'), | |
260 | }[notification.type_] % dict( |
|
260 | }[notification.type_] % dict( | |
261 | user=notification.created_by_user.username, |
|
261 | user=notification.created_by_user.username, | |
262 | age=h.age(notification.created_on), |
|
262 | age=h.age(notification.created_on), | |
263 | ) |
|
263 | ) | |
264 | else: |
|
264 | else: | |
265 | return { |
|
265 | return { | |
266 | _n.TYPE_CHANGESET_COMMENT: _('%(user)s commented on changeset at %(when)s'), |
|
266 | _n.TYPE_CHANGESET_COMMENT: _('%(user)s commented on changeset at %(when)s'), | |
267 | _n.TYPE_MESSAGE: _('%(user)s sent message at %(when)s'), |
|
267 | _n.TYPE_MESSAGE: _('%(user)s sent message at %(when)s'), | |
268 | _n.TYPE_MENTION: _('%(user)s mentioned you at %(when)s'), |
|
268 | _n.TYPE_MENTION: _('%(user)s mentioned you at %(when)s'), | |
269 | _n.TYPE_REGISTRATION: _('%(user)s registered in Kallithea at %(when)s'), |
|
269 | _n.TYPE_REGISTRATION: _('%(user)s registered in Kallithea at %(when)s'), | |
270 | _n.TYPE_PULL_REQUEST: _('%(user)s opened new pull request at %(when)s'), |
|
270 | _n.TYPE_PULL_REQUEST: _('%(user)s opened new pull request at %(when)s'), | |
271 | _n.TYPE_PULL_REQUEST_COMMENT: _('%(user)s commented on pull request at %(when)s'), |
|
271 | _n.TYPE_PULL_REQUEST_COMMENT: _('%(user)s commented on pull request at %(when)s'), | |
272 | }[notification.type_] % dict( |
|
272 | }[notification.type_] % dict( | |
273 | user=notification.created_by_user.username, |
|
273 | user=notification.created_by_user.username, | |
274 | when=h.fmt_date(notification.created_on), |
|
274 | when=h.fmt_date(notification.created_on), | |
275 | ) |
|
275 | ) | |
276 |
|
276 | |||
277 |
|
277 | |||
278 | class EmailNotificationModel(BaseModel): |
|
278 | class EmailNotificationModel(BaseModel): | |
279 |
|
279 | |||
280 | TYPE_CHANGESET_COMMENT = Notification.TYPE_CHANGESET_COMMENT |
|
280 | TYPE_CHANGESET_COMMENT = Notification.TYPE_CHANGESET_COMMENT | |
281 | TYPE_MESSAGE = Notification.TYPE_MESSAGE # only used for testing |
|
281 | TYPE_MESSAGE = Notification.TYPE_MESSAGE # only used for testing | |
282 | # Notification.TYPE_MENTION is not used |
|
282 | # Notification.TYPE_MENTION is not used | |
283 | TYPE_PASSWORD_RESET = 'password_link' |
|
283 | TYPE_PASSWORD_RESET = 'password_link' | |
284 | TYPE_REGISTRATION = Notification.TYPE_REGISTRATION |
|
284 | TYPE_REGISTRATION = Notification.TYPE_REGISTRATION | |
285 | TYPE_PULL_REQUEST = Notification.TYPE_PULL_REQUEST |
|
285 | TYPE_PULL_REQUEST = Notification.TYPE_PULL_REQUEST | |
286 | TYPE_PULL_REQUEST_COMMENT = Notification.TYPE_PULL_REQUEST_COMMENT |
|
286 | TYPE_PULL_REQUEST_COMMENT = Notification.TYPE_PULL_REQUEST_COMMENT | |
287 | TYPE_DEFAULT = 'default' |
|
287 | TYPE_DEFAULT = 'default' | |
288 |
|
288 | |||
289 | def __init__(self): |
|
289 | def __init__(self): | |
290 | super(EmailNotificationModel, self).__init__() |
|
290 | super(EmailNotificationModel, self).__init__() | |
291 | self._template_root = kallithea.CONFIG['pylons.paths']['templates'][0] |
|
291 | self._template_root = kallithea.CONFIG['pylons.paths']['templates'][0] | |
292 | self._tmpl_lookup = kallithea.CONFIG['pylons.app_globals'].mako_lookup |
|
292 | self._tmpl_lookup = kallithea.CONFIG['pylons.app_globals'].mako_lookup | |
293 | self.email_types = { |
|
293 | self.email_types = { | |
294 | self.TYPE_CHANGESET_COMMENT: 'changeset_comment', |
|
294 | self.TYPE_CHANGESET_COMMENT: 'changeset_comment', | |
295 | self.TYPE_PASSWORD_RESET: 'password_reset', |
|
295 | self.TYPE_PASSWORD_RESET: 'password_reset', | |
296 | self.TYPE_REGISTRATION: 'registration', |
|
296 | self.TYPE_REGISTRATION: 'registration', | |
297 | self.TYPE_DEFAULT: 'default', |
|
297 | self.TYPE_DEFAULT: 'default', | |
298 | self.TYPE_PULL_REQUEST: 'pull_request', |
|
298 | self.TYPE_PULL_REQUEST: 'pull_request', | |
299 | self.TYPE_PULL_REQUEST_COMMENT: 'pull_request_comment', |
|
299 | self.TYPE_PULL_REQUEST_COMMENT: 'pull_request_comment', | |
300 | } |
|
300 | } | |
301 | self._subj_map = { |
|
301 | self._subj_map = { | |
302 | self.TYPE_CHANGESET_COMMENT: _('[Comment from %(comment_username)s] %(repo_name)s changeset %(short_id)s on %(branch)s'), |
|
302 | self.TYPE_CHANGESET_COMMENT: _('[Comment from %(comment_username)s] %(repo_name)s changeset %(short_id)s on %(branch)s'), | |
303 | self.TYPE_MESSAGE: 'Test Message', |
|
303 | self.TYPE_MESSAGE: 'Test Message', | |
304 | # self.TYPE_PASSWORD_RESET |
|
304 | # self.TYPE_PASSWORD_RESET | |
305 | self.TYPE_REGISTRATION: _('New user %(new_username)s registered'), |
|
305 | self.TYPE_REGISTRATION: _('New user %(new_username)s registered'), | |
306 | # self.TYPE_DEFAULT |
|
306 | # self.TYPE_DEFAULT | |
307 | self.TYPE_PULL_REQUEST: _('[Added by %(pr_username)s] %(repo_name)s pull request %(pr_nice_id)s from %(ref)s'), |
|
307 | self.TYPE_PULL_REQUEST: _('[Added by %(pr_username)s] %(repo_name)s pull request %(pr_nice_id)s from %(ref)s'), | |
308 | self.TYPE_PULL_REQUEST_COMMENT: _('[Comment from %(comment_username)s] %(repo_name)s pull request %(pr_nice_id)s from %(ref)s'), |
|
308 | self.TYPE_PULL_REQUEST_COMMENT: _('[Comment from %(comment_username)s] %(repo_name)s pull request %(pr_nice_id)s from %(ref)s'), | |
309 | } |
|
309 | } | |
310 |
|
310 | |||
311 | def get_email_description(self, type_, **kwargs): |
|
311 | def get_email_description(self, type_, **kwargs): | |
312 | """ |
|
312 | """ | |
313 | return subject for email based on given type |
|
313 | return subject for email based on given type | |
314 | """ |
|
314 | """ | |
315 | tmpl = self._subj_map[type_] |
|
315 | tmpl = self._subj_map[type_] | |
316 | try: |
|
316 | try: | |
317 | subj = tmpl % kwargs |
|
317 | subj = tmpl % kwargs | |
318 | except KeyError as e: |
|
318 | except KeyError as e: | |
319 | log.error('error generating email subject for %r from %s: %s', type_, ','.join(self._subj_map.keys()), e) |
|
319 | log.error('error generating email subject for %r from %s: %s', type_, ','.join(self._subj_map.keys()), e) | |
320 | raise |
|
320 | raise | |
321 | l = [safe_unicode(x) for x in [kwargs.get('status_change'), kwargs.get('closing_pr') and _('Closing')] if x] |
|
321 | l = [safe_unicode(x) for x in [kwargs.get('status_change'), kwargs.get('closing_pr') and _('Closing')] if x] | |
322 | if l: |
|
322 | if l: | |
323 | if subj.startswith('['): |
|
323 | if subj.startswith('['): | |
324 | subj = '[' + ', '.join(l) + ': ' + subj[1:] |
|
324 | subj = '[' + ', '.join(l) + ': ' + subj[1:] | |
325 | else: |
|
325 | else: | |
326 | subj = '[' + ', '.join(l) + '] ' + subj |
|
326 | subj = '[' + ', '.join(l) + '] ' + subj | |
327 | return subj |
|
327 | return subj | |
328 |
|
328 | |||
329 | def get_email_tmpl(self, type_, content_type, **kwargs): |
|
329 | def get_email_tmpl(self, type_, content_type, **kwargs): | |
330 | """ |
|
330 | """ | |
331 | return generated template for email based on given type |
|
331 | return generated template for email based on given type | |
332 | """ |
|
332 | """ | |
333 |
|
333 | |||
334 | base = 'email_templates/' + self.email_types.get(type_, self.email_types[self.TYPE_DEFAULT]) + '.' + content_type |
|
334 | base = 'email_templates/' + self.email_types.get(type_, self.email_types[self.TYPE_DEFAULT]) + '.' + content_type | |
335 | email_template = self._tmpl_lookup.get_template(base) |
|
335 | email_template = self._tmpl_lookup.get_template(base) | |
336 | # translator and helpers inject |
|
336 | # translator and helpers inject | |
337 | _kwargs = {'_': _, |
|
337 | _kwargs = {'_': _, | |
338 | 'h': h, |
|
338 | 'h': h, | |
339 | 'c': c} |
|
339 | 'c': c} | |
340 | _kwargs.update(kwargs) |
|
340 | _kwargs.update(kwargs) | |
341 | log.debug('rendering tmpl %s with kwargs %s', base, _kwargs) |
|
341 | log.debug('rendering tmpl %s with kwargs %s', base, _kwargs) | |
342 | return email_template.render(**_kwargs) |
|
342 | return email_template.render(**_kwargs) |
@@ -1,91 +1,167 b'' | |||||
1 | import mock |
|
1 | import mock | |
2 |
|
2 | |||
3 | import kallithea |
|
3 | import kallithea | |
4 | from kallithea.tests import * |
|
4 | from kallithea.tests import * | |
|
5 | from kallithea.model.db import User | |||
5 |
|
6 | |||
6 | class smtplib_mock(object): |
|
7 | class smtplib_mock(object): | |
7 |
|
8 | |||
8 | @classmethod |
|
9 | @classmethod | |
9 | def SMTP(cls, server, port, local_hostname): |
|
10 | def SMTP(cls, server, port, local_hostname): | |
10 | return smtplib_mock() |
|
11 | return smtplib_mock() | |
11 |
|
12 | |||
12 | def ehlo(self): |
|
13 | def ehlo(self): | |
13 | pass |
|
14 | pass | |
14 | def quit(self): |
|
15 | def quit(self): | |
15 | pass |
|
16 | pass | |
16 | def sendmail(self, sender, dest, msg): |
|
17 | def sendmail(self, sender, dest, msg): | |
17 | smtplib_mock.lastsender = sender |
|
18 | smtplib_mock.lastsender = sender | |
18 | smtplib_mock.lastdest = dest |
|
19 | smtplib_mock.lastdest = dest | |
19 | smtplib_mock.lastmsg = msg |
|
20 | smtplib_mock.lastmsg = msg | |
20 | pass |
|
21 | pass | |
21 |
|
22 | |||
22 | @mock.patch('kallithea.lib.rcmail.smtp_mailer.smtplib', smtplib_mock) |
|
23 | @mock.patch('kallithea.lib.rcmail.smtp_mailer.smtplib', smtplib_mock) | |
23 | class TestMail(BaseTestCase): |
|
24 | class TestMail(BaseTestCase): | |
24 |
|
25 | |||
25 | def test_send_mail_trivial(self): |
|
26 | def test_send_mail_trivial(self): | |
26 | mailserver = 'smtp.mailserver.org' |
|
27 | mailserver = 'smtp.mailserver.org' | |
27 | recipients = ['rcpt1', 'rcpt2'] |
|
28 | recipients = ['rcpt1', 'rcpt2'] | |
28 | envelope_from = 'noreply@mailserver.org' |
|
29 | envelope_from = 'noreply@mailserver.org' | |
29 | subject = 'subject' |
|
30 | subject = 'subject' | |
30 | body = 'body' |
|
31 | body = 'body' | |
31 | html_body = 'html_body' |
|
32 | html_body = 'html_body' | |
32 |
|
33 | |||
33 | config_mock = { |
|
34 | config_mock = { | |
34 | 'smtp_server': mailserver, |
|
35 | 'smtp_server': mailserver, | |
35 | 'app_email_from': envelope_from, |
|
36 | 'app_email_from': envelope_from, | |
36 | } |
|
37 | } | |
37 | with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock): |
|
38 | with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock): | |
38 | kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body) |
|
39 | kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body) | |
39 |
|
40 | |||
40 | self.assertSetEqual(smtplib_mock.lastdest, set(recipients)) |
|
41 | self.assertSetEqual(smtplib_mock.lastdest, set(recipients)) | |
41 | self.assertEqual(smtplib_mock.lastsender, envelope_from) |
|
42 | self.assertEqual(smtplib_mock.lastsender, envelope_from) | |
42 | self.assertIn('From: %s' % envelope_from, smtplib_mock.lastmsg) |
|
43 | self.assertIn('From: %s' % envelope_from, smtplib_mock.lastmsg) | |
43 | self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg) |
|
44 | self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg) | |
44 | self.assertIn(body, smtplib_mock.lastmsg) |
|
45 | self.assertIn(body, smtplib_mock.lastmsg) | |
45 | self.assertIn(html_body, smtplib_mock.lastmsg) |
|
46 | self.assertIn(html_body, smtplib_mock.lastmsg) | |
46 |
|
47 | |||
47 | def test_send_mail_no_recipients(self): |
|
48 | def test_send_mail_no_recipients(self): | |
48 | mailserver = 'smtp.mailserver.org' |
|
49 | mailserver = 'smtp.mailserver.org' | |
49 | recipients = [] |
|
50 | recipients = [] | |
50 | envelope_from = 'noreply@mailserver.org' |
|
51 | envelope_from = 'noreply@mailserver.org' | |
51 | email_to = 'admin@mailserver.org' |
|
52 | email_to = 'admin@mailserver.org' | |
52 | subject = 'subject' |
|
53 | subject = 'subject' | |
53 | body = 'body' |
|
54 | body = 'body' | |
54 | html_body = 'html_body' |
|
55 | html_body = 'html_body' | |
55 |
|
56 | |||
56 | config_mock = { |
|
57 | config_mock = { | |
57 | 'smtp_server': mailserver, |
|
58 | 'smtp_server': mailserver, | |
58 | 'app_email_from': envelope_from, |
|
59 | 'app_email_from': envelope_from, | |
59 | 'email_to': email_to, |
|
60 | 'email_to': email_to, | |
60 | } |
|
61 | } | |
61 | with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock): |
|
62 | with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock): | |
62 | kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body) |
|
63 | kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body) | |
63 |
|
64 | |||
64 | self.assertSetEqual(smtplib_mock.lastdest, set([TEST_USER_ADMIN_EMAIL, email_to])) |
|
65 | self.assertSetEqual(smtplib_mock.lastdest, set([TEST_USER_ADMIN_EMAIL, email_to])) | |
65 | self.assertEqual(smtplib_mock.lastsender, envelope_from) |
|
66 | self.assertEqual(smtplib_mock.lastsender, envelope_from) | |
66 | self.assertIn('From: %s' % envelope_from, smtplib_mock.lastmsg) |
|
67 | self.assertIn('From: %s' % envelope_from, smtplib_mock.lastmsg) | |
67 | self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg) |
|
68 | self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg) | |
68 | self.assertIn(body, smtplib_mock.lastmsg) |
|
69 | self.assertIn(body, smtplib_mock.lastmsg) | |
69 | self.assertIn(html_body, smtplib_mock.lastmsg) |
|
70 | self.assertIn(html_body, smtplib_mock.lastmsg) | |
70 |
|
71 | |||
71 | def test_send_mail_no_recipients_no_email_to(self): |
|
72 | def test_send_mail_no_recipients_no_email_to(self): | |
72 | mailserver = 'smtp.mailserver.org' |
|
73 | mailserver = 'smtp.mailserver.org' | |
73 | recipients = [] |
|
74 | recipients = [] | |
74 | envelope_from = 'noreply@mailserver.org' |
|
75 | envelope_from = 'noreply@mailserver.org' | |
75 | subject = 'subject' |
|
76 | subject = 'subject' | |
76 | body = 'body' |
|
77 | body = 'body' | |
77 | html_body = 'html_body' |
|
78 | html_body = 'html_body' | |
78 |
|
79 | |||
79 | config_mock = { |
|
80 | config_mock = { | |
80 | 'smtp_server': mailserver, |
|
81 | 'smtp_server': mailserver, | |
81 | 'app_email_from': envelope_from, |
|
82 | 'app_email_from': envelope_from, | |
82 | } |
|
83 | } | |
83 | with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock): |
|
84 | with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock): | |
84 | kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body) |
|
85 | kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body) | |
85 |
|
86 | |||
86 | self.assertSetEqual(smtplib_mock.lastdest, set([TEST_USER_ADMIN_EMAIL])) |
|
87 | self.assertSetEqual(smtplib_mock.lastdest, set([TEST_USER_ADMIN_EMAIL])) | |
87 | self.assertEqual(smtplib_mock.lastsender, envelope_from) |
|
88 | self.assertEqual(smtplib_mock.lastsender, envelope_from) | |
88 | self.assertIn('From: %s' % envelope_from, smtplib_mock.lastmsg) |
|
89 | self.assertIn('From: %s' % envelope_from, smtplib_mock.lastmsg) | |
89 | self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg) |
|
90 | self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg) | |
90 | self.assertIn(body, smtplib_mock.lastmsg) |
|
91 | self.assertIn(body, smtplib_mock.lastmsg) | |
91 | self.assertIn(html_body, smtplib_mock.lastmsg) |
|
92 | self.assertIn(html_body, smtplib_mock.lastmsg) | |
|
93 | ||||
|
94 | def test_send_mail_with_author(self): | |||
|
95 | mailserver = 'smtp.mailserver.org' | |||
|
96 | recipients = ['rcpt1', 'rcpt2'] | |||
|
97 | envelope_from = 'noreply@mailserver.org' | |||
|
98 | subject = 'subject' | |||
|
99 | body = 'body' | |||
|
100 | html_body = 'html_body' | |||
|
101 | author = User.get_by_username(TEST_USER_REGULAR_LOGIN) | |||
|
102 | ||||
|
103 | config_mock = { | |||
|
104 | 'smtp_server': mailserver, | |||
|
105 | 'app_email_from': envelope_from, | |||
|
106 | } | |||
|
107 | with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock): | |||
|
108 | kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body, author=author) | |||
|
109 | ||||
|
110 | self.assertSetEqual(smtplib_mock.lastdest, set(recipients)) | |||
|
111 | self.assertEqual(smtplib_mock.lastsender, envelope_from) | |||
|
112 | self.assertIn('From: "Kallithea Admin (no-reply)" <%s>' % envelope_from, smtplib_mock.lastmsg) | |||
|
113 | self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg) | |||
|
114 | self.assertIn(body, smtplib_mock.lastmsg) | |||
|
115 | self.assertIn(html_body, smtplib_mock.lastmsg) | |||
|
116 | ||||
|
117 | def test_send_mail_with_author_full_mail_from(self): | |||
|
118 | mailserver = 'smtp.mailserver.org' | |||
|
119 | recipients = ['rcpt1', 'rcpt2'] | |||
|
120 | envelope_addr = 'noreply@mailserver.org' | |||
|
121 | envelope_from = 'Some Name <%s>' % envelope_addr | |||
|
122 | subject = 'subject' | |||
|
123 | body = 'body' | |||
|
124 | html_body = 'html_body' | |||
|
125 | author = User.get_by_username(TEST_USER_REGULAR_LOGIN) | |||
|
126 | ||||
|
127 | config_mock = { | |||
|
128 | 'smtp_server': mailserver, | |||
|
129 | 'app_email_from': envelope_from, | |||
|
130 | } | |||
|
131 | with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock): | |||
|
132 | kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body, author=author) | |||
|
133 | ||||
|
134 | self.assertSetEqual(smtplib_mock.lastdest, set(recipients)) | |||
|
135 | self.assertEqual(smtplib_mock.lastsender, envelope_from) | |||
|
136 | self.assertIn('From: "Kallithea Admin (no-reply)" <%s>' % envelope_addr, smtplib_mock.lastmsg) | |||
|
137 | self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg) | |||
|
138 | self.assertIn(body, smtplib_mock.lastmsg) | |||
|
139 | self.assertIn(html_body, smtplib_mock.lastmsg) | |||
|
140 | ||||
|
141 | def test_send_mail_extra_headers(self): | |||
|
142 | mailserver = 'smtp.mailserver.org' | |||
|
143 | recipients = ['rcpt1', 'rcpt2'] | |||
|
144 | envelope_from = 'noreply@mailserver.org' | |||
|
145 | subject = 'subject' | |||
|
146 | body = 'body' | |||
|
147 | html_body = 'html_body' | |||
|
148 | author = User(name='foo', lastname='(fubar) "baz"') | |||
|
149 | headers = {'extra': 'yes'} | |||
|
150 | ||||
|
151 | config_mock = { | |||
|
152 | 'smtp_server': mailserver, | |||
|
153 | 'app_email_from': envelope_from, | |||
|
154 | } | |||
|
155 | with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock): | |||
|
156 | kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body, | |||
|
157 | author=author, headers=headers) | |||
|
158 | ||||
|
159 | self.assertSetEqual(smtplib_mock.lastdest, set(recipients)) | |||
|
160 | self.assertEqual(smtplib_mock.lastsender, envelope_from) | |||
|
161 | self.assertIn(r'From: "foo (fubar) \"baz\" (no-reply)" <%s>' % envelope_from, smtplib_mock.lastmsg) | |||
|
162 | self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg) | |||
|
163 | self.assertIn(body, smtplib_mock.lastmsg) | |||
|
164 | self.assertIn(html_body, smtplib_mock.lastmsg) | |||
|
165 | self.assertIn('Extra: yes', smtplib_mock.lastmsg) | |||
|
166 | # verify that headers dict hasn't mutated by send_email | |||
|
167 | self.assertDictEqual(headers, {'extra': 'yes'}) |
General Comments 0
You need to be logged in to leave comments.
Login now