email: send comment and pullrequest mails with the author's name in 'From'

When emails are sent for comments and pullrequest invitations, set the From
header to:
    Author's Name (no-reply) <generic email address>

Using the name of the person that causes the email, makes the emails more
useful and interpretable for the recipient of the emails.
To avoid replies directly to the author, triggering an 'offline' email
discussion that is not visible in the Kallithea interface, a generic
'no-reply' email address is used instead of the author's email
address.
This approach is assumed to be accepted by spam filters, as several other
web services are using the same approach.

The sender used for other email types, e.g. password reset mails, is
untouched and remains the value configured in app_email_from.

The sender used for the SMTP envelope is untouched as well.

Based on code by Cedric De Herdt.

docs/usage/email.rst

@@ -1,74 +1,79 b''
 .. _email:
 
 ==============
 Email settings
 ==============
 
 The Kallithea configuration file has several email related settings. When
 these contain correct values, Kallithea will send email in the situations
 described below. If the email configuration is not correct so that emails
 cannot be sent, all mails will show up in the log output.
 
 Before any email can be sent, an SMTP server has to be configured using the
 configuration file setting ``smtp_server``. If required for that server, specify
 a username (``smtp_username``) and password (``smtp_password``), a non-standard
 port (``smtp_port``), encryption settings (``smtp_use_tls`` or ``smtp_use_ssl``)
 and/or specific authentication parameters (``smtp_auth``).
 
 
 Application emails
 ------------------
 
 Kallithea sends an email to `users` on several occasions:
 
 - when comments are given on one of their changesets
 - when comments are given on changesets they are reviewer on or on which they
   commented regardless
 - when they are invited as reviewer in pull requests
 - when they request a password reset
 
 Kallithea sends an email to all `administrators` upon new account registration.
 Administrators are users with the ``Admin`` flag set on the *Admin > Users*
 page.
 
 When Kallithea wants to send an email but due to an error cannot correctly
 determine the intended recipients, the administrators and the addresses
 specified in ``email_to`` in the configuration file are used as fallback.
 
 Recipients will see these emails originating from the sender specified in the
 ``app_email_from`` setting in the configuration file. This setting can either
 contain only an email address, like `[e-mail]`, or both
 a name and an address in the following format: `Kallithea
-<[e-mail]>`. The subject of these emails can
-optionally be prefixed with the value of ``email_prefix`` in the configuration
-file.
+<[e-mail]>`. However, if the email is sent due to an
+action of a particular user, for example when a comment is given or a pull
+request created, the name of that user will be combined with the email address
+specified in ``app_email_from`` to form the sender (and any name part in that
+configuration setting disregarded).
+
+The subject of these emails can optionally be prefixed with the value of
+``email_prefix`` in the configuration file.
 
 
 Error emails
 ------------
 
 When an exception occurs in Kallithea -- and unless interactive debugging is
 enabled using ``set debug = true`` in the ``[app:main]`` section of the
 configuration file -- an email with exception details is sent by WebError_'s
 ``ErrorMiddleware`` to the addresses specified in ``email_to`` in the
 configuration file.
 
 Recipients will see these emails originating from the sender specified in the
 ``error_email_from`` setting in the configuration file. This setting can either
 contain only an email address, like `[e-mail]`, or both
 a name and an address in the following format: `Kallithea Errors
 <[e-mail]>`.
 
 *Note:* The WebError_ package does not respect ``smtp_port`` and assumes the
 standard SMTP port (25). If you have a remote SMTP server with a different port,
 you could set up a local forwarding SMTP server on port 25.
 
 
 References
 ----------
 
 - `Error Middleware (Pylons documentation) <http://pylons-webframework.readthedocs.org/en/latest/debugging.html#error-middleware>`_
 - `ErrorHandler (Pylons modules documentation) <http://pylons-webframework.readthedocs.org/en/latest/modules/middleware.html#pylons.middleware.ErrorHandler>`_
 
 
 .. _WebError: https://pypi.python.org/pypi/WebError

kallithea/lib/celerylib/tasks.py

@@ -1,510 +1,530 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 kallithea.lib.celerylib.tasks
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Kallithea task modules, containing all task that suppose to be run
 by celery daemon
 
 This file was forked by the Kallithea project in July 2014.
 Original author and date, and relevant copyright and licensing information is below:
 :created_on: Oct 6, 2010
 :author: marcink
 :copyright: (c) 2013 RhodeCode GmbH, and others.
 :license: GPLv3, see LICENSE.md for more details.
 """
 
 from celery.decorators import task
 
 import os
 import traceback
 import logging
+import rfc822
 from os.path import join as jn
 
 from time import mktime
 from operator import itemgetter
 from string import lower
 
 from pylons import config
 
 from kallithea import CELERY_ON
 from kallithea.lib.celerylib import run_task, locked_task, dbsession, \
     str2bool, __get_lockkey, LockHeld, DaemonLock, get_session
 from kallithea.lib.helpers import person
 from kallithea.lib.rcmail.smtp_mailer import SmtpMailer
 from kallithea.lib.utils import add_cache, action_logger
+from kallithea.lib.vcs.utils import author_email
 from kallithea.lib.compat import json, OrderedDict
 from kallithea.lib.hooks import log_create_repository
 
 from kallithea.model.db import Statistics, Repository, User
 
 
 add_cache(config)  # pragma: no cover
 
 __all__ = ['whoosh_index', 'get_commits_stats', 'send_email']
 
 
 def get_logger(cls):
     if CELERY_ON:
         try:
             return cls.get_logger()
         except AttributeError:
             pass
     return logging.getLogger(__name__)
 
 
 @task(ignore_result=True)
 @locked_task
 @dbsession
 def whoosh_index(repo_location, full_index):
     from kallithea.lib.indexers.daemon import WhooshIndexingDaemon
     DBS = get_session()
 
     index_location = config['index_dir']
     WhooshIndexingDaemon(index_location=index_location,
                          repo_location=repo_location, sa=DBS)\
                          .run(full_index=full_index)
 
 
 @task(ignore_result=True)
 @dbsession
 def get_commits_stats(repo_name, ts_min_y, ts_max_y, recurse_limit=100):
     log = get_logger(get_commits_stats)
     DBS = get_session()
     lockkey = __get_lockkey('get_commits_stats', repo_name, ts_min_y,
                             ts_max_y)
     lockkey_path = config['app_conf']['cache_dir']
 
     log.info('running task with lockkey %s', lockkey)
 
     try:
         lock = l = DaemonLock(file_=jn(lockkey_path, lockkey))
 
         # for js data compatibility cleans the key for person from '
         akc = lambda k: person(k).replace('"', "")
 
         co_day_auth_aggr = {}
         commits_by_day_aggregate = {}
         repo = Repository.get_by_repo_name(repo_name)
         if repo is None:
             return True
 
         repo = repo.scm_instance
         repo_size = repo.count()
         # return if repo have no revisions
         if repo_size < 1:
             lock.release()
             return True
 
         skip_date_limit = True
         parse_limit = int(config['app_conf'].get('commit_parse_limit'))
         last_rev = None
         last_cs = None
         timegetter = itemgetter('time')
 
         dbrepo = DBS.query(Repository)\
             .filter(Repository.repo_name == repo_name).scalar()
         cur_stats = DBS.query(Statistics)\
             .filter(Statistics.repository == dbrepo).scalar()
 
         if cur_stats is not None:
             last_rev = cur_stats.stat_on_revision
 
         if last_rev == repo.get_changeset().revision and repo_size > 1:
             # pass silently without any work if we're not on first revision or
             # current state of parsing revision(from db marker) is the
             # last revision
             lock.release()
             return True
 
         if cur_stats:
             commits_by_day_aggregate = OrderedDict(json.loads(
                                         cur_stats.commit_activity_combined))
             co_day_auth_aggr = json.loads(cur_stats.commit_activity)
 
         log.debug('starting parsing %s', parse_limit)
         lmktime = mktime
 
         last_rev = last_rev + 1 if last_rev >= 0 else 0
         log.debug('Getting revisions from %s to %s',
              last_rev, last_rev + parse_limit
         )
         for cs in repo[last_rev:last_rev + parse_limit]:
             log.debug('parsing %s', cs)
             last_cs = cs  # remember last parsed changeset
             k = lmktime([cs.date.timetuple()[0], cs.date.timetuple()[1],
                           cs.date.timetuple()[2], 0, 0, 0, 0, 0, 0])
 
             if akc(cs.author) in co_day_auth_aggr:
                 try:
                     l = [timegetter(x) for x in
                          co_day_auth_aggr[akc(cs.author)]['data']]
                     time_pos = l.index(k)
                 except ValueError:
                     time_pos = None
 
                 if time_pos >= 0 and time_pos is not None:
 
                     datadict = \
                         co_day_auth_aggr[akc(cs.author)]['data'][time_pos]
 
                     datadict["commits"] += 1
                     datadict["added"] += len(cs.added)
                     datadict["changed"] += len(cs.changed)
                     datadict["removed"] += len(cs.removed)
 
                 else:
                     if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
 
                         datadict = {"time": k,
                                     "commits": 1,
                                     "added": len(cs.added),
                                     "changed": len(cs.changed),
                                     "removed": len(cs.removed),
                                    }
                         co_day_auth_aggr[akc(cs.author)]['data']\
                             .append(datadict)
 
             else:
                 if k >= ts_min_y and k <= ts_max_y or skip_date_limit:
                     co_day_auth_aggr[akc(cs.author)] = {
                                         "label": akc(cs.author),
                                         "data": [{"time":k,
                                                  "commits":1,
                                                  "added":len(cs.added),
                                                  "changed":len(cs.changed),
                                                  "removed":len(cs.removed),
                                                  }],
                                         "schema": ["commits"],
                                         }
 
             #gather all data by day
             if k in commits_by_day_aggregate:
                 commits_by_day_aggregate[k] += 1
             else:
                 commits_by_day_aggregate[k] = 1
 
         overview_data = sorted(commits_by_day_aggregate.items(),
                                key=itemgetter(0))
 
         if not co_day_auth_aggr:
             co_day_auth_aggr[akc(repo.contact)] = {
                 "label": akc(repo.contact),
                 "data": [0, 1],
                 "schema": ["commits"],
             }
 
         stats = cur_stats if cur_stats else Statistics()
         stats.commit_activity = json.dumps(co_day_auth_aggr)
         stats.commit_activity_combined = json.dumps(overview_data)
 
         log.debug('last revision %s', last_rev)
         leftovers = len(repo.revisions[last_rev:])
         log.debug('revisions to parse %s', leftovers)
 
         if last_rev == 0 or leftovers < parse_limit:
             log.debug('getting code trending stats')
             stats.languages = json.dumps(__get_codes_stats(repo_name))
 
         try:
             stats.repository = dbrepo
             stats.stat_on_revision = last_cs.revision if last_cs else 0
             DBS.add(stats)
             DBS.commit()
         except:
             log.error(traceback.format_exc())
             DBS.rollback()
             lock.release()
             return False
 
         # final release
         lock.release()
 
         # execute another task if celery is enabled
         if len(repo.revisions) > 1 and CELERY_ON and recurse_limit > 0:
             recurse_limit -= 1
             run_task(get_commits_stats, repo_name, ts_min_y, ts_max_y,
                      recurse_limit)
         if recurse_limit <= 0:
             log.debug('Breaking recursive mode due to reach of recurse limit')
         return True
     except LockHeld:
         log.info('LockHeld')
         return 'Task with key %s already running' % lockkey
 
 
 @task(ignore_result=True)
 @dbsession
-def send_email(recipients, subject, body='', html_body='', headers=None):
+def send_email(recipients, subject, body='', html_body='', headers=None, author=None):
     """
     Sends an email with defined parameters from the .ini files.
 
     :param recipients: list of recipients, if this is None, the defined email
         address from field 'email_to' and all admins is used instead
     :param subject: subject of the mail
     :param body: body of the mail
     :param html_body: html version of body
+    :param headers: dictionary of prepopulated e-mail headers
+    :param author: User object of the author of this mail, if known and relevant
     """
     log = get_logger(send_email)
     assert isinstance(recipients, list), recipients
+    if headers is None:
+        headers = {}
+    else:
+        # do not modify the original headers object passed by the caller
+        headers = headers.copy()
 
     email_config = config
     email_prefix = email_config.get('email_prefix', '')
     if email_prefix:
         subject = "%s %s" % (email_prefix, subject)
 
     if not recipients:
         # if recipients are not defined we send to email_config + all admins
         recipients = [u.email for u in User.query()
                       .filter(User.admin == True).all()]
         if email_config.get('email_to') is not None:
             recipients += [email_config.get('email_to')]
 
         # If there are still no recipients, there are no admins and no address
         # configured in email_to, so return.
         if not recipients:
             log.error("No recipients specified and no fallback available.")
             return False
 
         log.warning("No recipients specified for '%s' - sending to admins %s", subject, ' '.join(recipients))
 
-    mail_from = email_config.get('app_email_from', 'Kallithea')
+    # SMTP sender
+    envelope_from = email_config.get('app_email_from', 'Kallithea')
+    # 'From' header
+    if author is not None:
+        # set From header based on author but with a generic e-mail address
+        # In case app_email_from is in "Some Name <e-mail>" format, we first
+        # extract the e-mail address.
+        envelope_addr = author_email(envelope_from)
+        headers['From'] = '"%s" <%s>' % (
+            rfc822.quote('%s (no-reply)' % author.full_name_or_username),
+            envelope_addr)
+
     user = email_config.get('smtp_username')
     passwd = email_config.get('smtp_password')
     mail_server = email_config.get('smtp_server')
     mail_port = email_config.get('smtp_port')
     tls = str2bool(email_config.get('smtp_use_tls'))
     ssl = str2bool(email_config.get('smtp_use_ssl'))
     debug = str2bool(email_config.get('debug'))
     smtp_auth = email_config.get('smtp_auth')
 
     logmsg = ("Mail details:\n"
               "recipients: %s\n"
               "headers: %s\n"
               "subject: %s\n"
               "body:\n%s\n"
               "html:\n%s\n"
               % (' '.join(recipients), headers, subject, body, html_body))
 
     if mail_server:
         log.debug("Sending e-mail. " + logmsg)
     else:
         log.error("SMTP mail server not configured - cannot send e-mail.")
         log.warning(logmsg)
         return False
 
     try:
-        m = SmtpMailer(mail_from, user, passwd, mail_server, smtp_auth,
+        m = SmtpMailer(envelope_from, user, passwd, mail_server, smtp_auth,
                        mail_port, ssl, tls, debug=debug)
         m.send(recipients, subject, body, html_body, headers=headers)
     except:
         log.error('Mail sending failed')
         log.error(traceback.format_exc())
         return False
     return True
 
 @task(ignore_result=False)
 @dbsession
 def create_repo(form_data, cur_user):
     from kallithea.model.repo import RepoModel
     from kallithea.model.user import UserModel
     from kallithea.model.db import Setting
 
     log = get_logger(create_repo)
     DBS = get_session()
 
     cur_user = UserModel(DBS)._get_user(cur_user)
 
     owner = cur_user
     repo_name = form_data['repo_name']
     repo_name_full = form_data['repo_name_full']
     repo_type = form_data['repo_type']
     description = form_data['repo_description']
     private = form_data['repo_private']
     clone_uri = form_data.get('clone_uri')
     repo_group = form_data['repo_group']
     landing_rev = form_data['repo_landing_rev']
     copy_fork_permissions = form_data.get('copy_permissions')
     copy_group_permissions = form_data.get('repo_copy_permissions')
     fork_of = form_data.get('fork_parent_id')
     state = form_data.get('repo_state', Repository.STATE_PENDING)
 
     # repo creation defaults, private and repo_type are filled in form
     defs = Setting.get_default_repo_settings(strip_prefix=True)
     enable_statistics = defs.get('repo_enable_statistics')
     enable_locking = defs.get('repo_enable_locking')
     enable_downloads = defs.get('repo_enable_downloads')
 
     try:
         repo = RepoModel(DBS)._create_repo(
             repo_name=repo_name_full,
             repo_type=repo_type,
             description=description,
             owner=owner,
             private=private,
             clone_uri=clone_uri,
             repo_group=repo_group,
             landing_rev=landing_rev,
             fork_of=fork_of,
             copy_fork_permissions=copy_fork_permissions,
             copy_group_permissions=copy_group_permissions,
             enable_statistics=enable_statistics,
             enable_locking=enable_locking,
             enable_downloads=enable_downloads,
             state=state
         )
 
         action_logger(cur_user, 'user_created_repo',
                       form_data['repo_name_full'], '', DBS)
 
         DBS.commit()
         # now create this repo on Filesystem
         RepoModel(DBS)._create_filesystem_repo(
             repo_name=repo_name,
             repo_type=repo_type,
             repo_group=RepoModel(DBS)._get_repo_group(repo_group),
             clone_uri=clone_uri,
         )
         repo = Repository.get_by_repo_name(repo_name_full)
         log_create_repository(repo.get_dict(), created_by=owner.username)
 
         # update repo changeset caches initially
         repo.update_changeset_cache()
 
         # set new created state
         repo.set_state(Repository.STATE_CREATED)
         DBS.commit()
     except Exception as e:
         log.warning('Exception %s occurred when forking repository, '
                     'doing cleanup...' % e)
         # rollback things manually !
         repo = Repository.get_by_repo_name(repo_name_full)
         if repo:
             Repository.delete(repo.repo_id)
             DBS.commit()
             RepoModel(DBS)._delete_filesystem_repo(repo)
         raise
 
     # it's an odd fix to make celery fail task when exception occurs
     def on_failure(self, *args, **kwargs):
         pass
 
     return True
 
 
 @task(ignore_result=False)
 @dbsession
 def create_repo_fork(form_data, cur_user):
     """
     Creates a fork of repository using interval VCS methods
 
     :param form_data:
     :param cur_user:
     """
     from kallithea.model.repo import RepoModel
     from kallithea.model.user import UserModel
 
     log = get_logger(create_repo_fork)
     DBS = get_session()
 
     base_path = Repository.base_path()
     cur_user = UserModel(DBS)._get_user(cur_user)
 
     repo_name = form_data['repo_name']  # fork in this case
     repo_name_full = form_data['repo_name_full']
 
     repo_type = form_data['repo_type']
     owner = cur_user
     private = form_data['private']
     clone_uri = form_data.get('clone_uri')
     repo_group = form_data['repo_group']
     landing_rev = form_data['landing_rev']
     copy_fork_permissions = form_data.get('copy_permissions')
 
     try:
         fork_of = RepoModel(DBS)._get_repo(form_data.get('fork_parent_id'))
 
         RepoModel(DBS)._create_repo(
             repo_name=repo_name_full,
             repo_type=repo_type,
             description=form_data['description'],
             owner=owner,
             private=private,
             clone_uri=clone_uri,
             repo_group=repo_group,
             landing_rev=landing_rev,
             fork_of=fork_of,
             copy_fork_permissions=copy_fork_permissions
         )
         action_logger(cur_user, 'user_forked_repo:%s' % repo_name_full,
                       fork_of.repo_name, '', DBS)
         DBS.commit()
 
         update_after_clone = form_data['update_after_clone'] # FIXME - unused!
         source_repo_path = os.path.join(base_path, fork_of.repo_name)
 
         # now create this repo on Filesystem
         RepoModel(DBS)._create_filesystem_repo(
             repo_name=repo_name,
             repo_type=repo_type,
             repo_group=RepoModel(DBS)._get_repo_group(repo_group),
             clone_uri=source_repo_path,
         )
         repo = Repository.get_by_repo_name(repo_name_full)
         log_create_repository(repo.get_dict(), created_by=owner.username)
 
         # update repo changeset caches initially
         repo.update_changeset_cache()
 
         # set new created state
         repo.set_state(Repository.STATE_CREATED)
         DBS.commit()
     except Exception as e:
         log.warning('Exception %s occurred when forking repository, '
                     'doing cleanup...' % e)
         #rollback things manually !
         repo = Repository.get_by_repo_name(repo_name_full)
         if repo:
             Repository.delete(repo.repo_id)
             DBS.commit()
             RepoModel(DBS)._delete_filesystem_repo(repo)
         raise
 
     # it's an odd fix to make celery fail task when exception occurs
     def on_failure(self, *args, **kwargs):
         pass
 
     return True
 
 
 def __get_codes_stats(repo_name):
     from kallithea.config.conf import LANGUAGES_EXTENSIONS_MAP
     repo = Repository.get_by_repo_name(repo_name).scm_instance
 
     tip = repo.get_changeset()
     code_stats = {}
 
     def aggregate(cs):
         for f in cs[2]:
             ext = lower(f.extension)
             if ext in LANGUAGES_EXTENSIONS_MAP.keys() and not f.is_binary:
                 if ext in code_stats:
                     code_stats[ext] += 1
                 else:
                     code_stats[ext] = 1
 
     map(aggregate, tip.walk('/'))
 
     return code_stats or {}

kallithea/model/notification.py

@@ -1,342 +1,342 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 kallithea.model.notification
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Model for notifications
 
 
 This file was forked by the Kallithea project in July 2014.
 Original author and date, and relevant copyright and licensing information is below:
 :created_on: Nov 20, 2011
 :author: marcink
 :copyright: (c) 2013 RhodeCode GmbH, and others.
 :license: GPLv3, see LICENSE.md for more details.
 """
 
 import logging
 import traceback
 
 from pylons import tmpl_context as c
 from pylons.i18n.translation import _
 from sqlalchemy.orm import joinedload, subqueryload
 
 import kallithea
 from kallithea.lib import helpers as h
 from kallithea.lib.utils2 import safe_unicode
 from kallithea.model import BaseModel
 from kallithea.model.db import Notification, User, UserNotification
 from kallithea.model.meta import Session
 
 log = logging.getLogger(__name__)
 
 
 class NotificationModel(BaseModel):
 
     cls = Notification
 
     def __get_notification(self, notification):
         if isinstance(notification, Notification):
             return notification
         elif isinstance(notification, (int, long)):
             return Notification.get(notification)
         else:
             if notification is not None:
                 raise Exception('notification must be int, long or Instance'
                                 ' of Notification got %s' % type(notification))
 
     def create(self, created_by, subject, body, recipients=None,
                type_=Notification.TYPE_MESSAGE, with_email=True,
                email_kwargs={}):
         """
 
         Creates notification of given type
 
         :param created_by: int, str or User instance. User who created this
             notification
         :param subject:
         :param body:
         :param recipients: list of int, str or User objects, when None
             is given send to all admins
         :param type_: type of notification
         :param with_email: send email with this notification
         :param email_kwargs: additional dict to pass as args to email template
         """
         from kallithea.lib.celerylib import tasks, run_task
 
         if recipients and not getattr(recipients, '__iter__', False):
             raise Exception('recipients must be a list or iterable')
 
         created_by_obj = self._get_user(created_by)
 
         recipients_objs = []
         if recipients:
             for u in recipients:
                 obj = self._get_user(u)
                 if obj is not None:
                     recipients_objs.append(obj)
                 else:
                     # TODO: inform user that requested operation couldn't be completed
                     log.error('cannot email unknown user %r', u)
             recipients_objs = set(recipients_objs)
             log.debug('sending notifications %s to %s',
                 type_, recipients_objs
             )
         elif recipients is None:
             # empty recipients means to all admins
             recipients_objs = User.query().filter(User.admin == True).all()
             log.debug('sending notifications %s to admins: %s',
                 type_, recipients_objs
             )
         #else: silently skip notification mails?
 
         # TODO: inform user who are notified
         notif = Notification.create(
             created_by=created_by_obj, subject=subject,
             body=body, recipients=recipients_objs, type_=type_
         )
 
         if not with_email:
             return notif
 
         #don't send email to person who created this comment
         rec_objs = set(recipients_objs).difference(set([created_by_obj]))
 
         headers = None
         if 'threading' in email_kwargs:
             headers = {'References': ' '.join('<%s>' % x for x in email_kwargs['threading'])}
 
         # send email with notification to all other participants
         for rec in rec_objs:
             ## this is passed into template
             html_kwargs = {
                       'subject': subject,
                       'body': h.rst_w_mentions(body),
                       'when': h.fmt_date(notif.created_on),
                       'user': notif.created_by_user.username,
                       }
 
             txt_kwargs = {
                       'subject': subject,
                       'body': body,
                       'when': h.fmt_date(notif.created_on),
                       'user': notif.created_by_user.username,
                       }
 
             html_kwargs.update(email_kwargs)
             txt_kwargs.update(email_kwargs)
             email_subject = EmailNotificationModel()\
                                 .get_email_description(type_, **txt_kwargs)
             email_txt_body = EmailNotificationModel()\
                                 .get_email_tmpl(type_, 'txt', **txt_kwargs)
             email_html_body = EmailNotificationModel()\
                                 .get_email_tmpl(type_, 'html', **html_kwargs)
 
             run_task(tasks.send_email, [rec.email], email_subject, email_txt_body,
-                     email_html_body, headers)
+                     email_html_body, headers, author=created_by_obj)
 
         return notif
 
     def delete(self, user, notification):
         # we don't want to remove actual notification just the assignment
         try:
             notification = self.__get_notification(notification)
             user = self._get_user(user)
             if notification and user:
                 obj = UserNotification.query()\
                         .filter(UserNotification.user == user)\
                         .filter(UserNotification.notification
                                 == notification)\
                         .one()
                 Session().delete(obj)
                 return True
         except Exception:
             log.error(traceback.format_exc())
             raise
 
     def get_for_user(self, user, filter_=None):
         """
         Get notifications for given user, filter them if filter dict is given
 
         :param user:
         :param filter:
         """
         user = self._get_user(user)
 
         q = UserNotification.query()\
             .filter(UserNotification.user == user)\
             .join((Notification, UserNotification.notification_id ==
                                  Notification.notification_id))\
             .options(joinedload('notification'))\
             .options(subqueryload('notification.created_by_user'))\
             .order_by(Notification.created_on.desc())
 
         if filter_:
             q = q.filter(Notification.type_.in_(filter_))
 
         return q.all()
 
     def mark_read(self, user, notification):
         try:
             notification = self.__get_notification(notification)
             user = self._get_user(user)
             if notification and user:
                 obj = UserNotification.query()\
                         .filter(UserNotification.user == user)\
                         .filter(UserNotification.notification
                                 == notification)\
                         .one()
                 obj.read = True
                 Session().add(obj)
                 return True
         except Exception:
             log.error(traceback.format_exc())
             raise
 
     def mark_all_read_for_user(self, user, filter_=None):
         user = self._get_user(user)
         q = UserNotification.query()\
             .filter(UserNotification.user == user)\
             .filter(UserNotification.read == False)\
             .join((Notification, UserNotification.notification_id ==
                                  Notification.notification_id))
         if filter_:
             q = q.filter(Notification.type_.in_(filter_))
 
         # this is a little inefficient but sqlalchemy doesn't support
         # update on joined tables :(
         for obj in q.all():
             obj.read = True
             Session().add(obj)
 
     def get_unread_cnt_for_user(self, user):
         user = self._get_user(user)
         return UserNotification.query()\
                 .filter(UserNotification.read == False)\
                 .filter(UserNotification.user == user).count()
 
     def get_unread_for_user(self, user):
         user = self._get_user(user)
         return [x.notification for x in UserNotification.query()\
                 .filter(UserNotification.read == False)\
                 .filter(UserNotification.user == user).all()]
 
     def get_user_notification(self, user, notification):
         user = self._get_user(user)
         notification = self.__get_notification(notification)
 
         return UserNotification.query()\
             .filter(UserNotification.notification == notification)\
             .filter(UserNotification.user == user).scalar()
 
     def make_description(self, notification, show_age=True):
         """
         Creates a human readable description based on properties
         of notification object
         """
         #alias
         _n = notification
 
         if show_age:
             return {
                     _n.TYPE_CHANGESET_COMMENT: _('%(user)s commented on changeset %(age)s'),
                     _n.TYPE_MESSAGE: _('%(user)s sent message %(age)s'),
                     _n.TYPE_MENTION: _('%(user)s mentioned you %(age)s'),
                     _n.TYPE_REGISTRATION: _('%(user)s registered in Kallithea %(age)s'),
                     _n.TYPE_PULL_REQUEST: _('%(user)s opened new pull request %(age)s'),
                     _n.TYPE_PULL_REQUEST_COMMENT: _('%(user)s commented on pull request %(age)s'),
                 }[notification.type_] % dict(
                     user=notification.created_by_user.username,
                     age=h.age(notification.created_on),
                 )
         else:
             return {
                     _n.TYPE_CHANGESET_COMMENT: _('%(user)s commented on changeset at %(when)s'),
                     _n.TYPE_MESSAGE: _('%(user)s sent message at %(when)s'),
                     _n.TYPE_MENTION: _('%(user)s mentioned you at %(when)s'),
                     _n.TYPE_REGISTRATION: _('%(user)s registered in Kallithea at %(when)s'),
                     _n.TYPE_PULL_REQUEST: _('%(user)s opened new pull request at %(when)s'),
                     _n.TYPE_PULL_REQUEST_COMMENT: _('%(user)s commented on pull request at %(when)s'),
                 }[notification.type_] % dict(
                     user=notification.created_by_user.username,
                     when=h.fmt_date(notification.created_on),
                 )
 
 
 class EmailNotificationModel(BaseModel):
 
     TYPE_CHANGESET_COMMENT = Notification.TYPE_CHANGESET_COMMENT
     TYPE_MESSAGE = Notification.TYPE_MESSAGE # only used for testing
     # Notification.TYPE_MENTION is not used
     TYPE_PASSWORD_RESET = 'password_link'
     TYPE_REGISTRATION = Notification.TYPE_REGISTRATION
     TYPE_PULL_REQUEST = Notification.TYPE_PULL_REQUEST
     TYPE_PULL_REQUEST_COMMENT = Notification.TYPE_PULL_REQUEST_COMMENT
     TYPE_DEFAULT = 'default'
 
     def __init__(self):
         super(EmailNotificationModel, self).__init__()
         self._template_root = kallithea.CONFIG['pylons.paths']['templates'][0]
         self._tmpl_lookup = kallithea.CONFIG['pylons.app_globals'].mako_lookup
         self.email_types = {
             self.TYPE_CHANGESET_COMMENT: 'changeset_comment',
             self.TYPE_PASSWORD_RESET: 'password_reset',
             self.TYPE_REGISTRATION: 'registration',
             self.TYPE_DEFAULT: 'default',
             self.TYPE_PULL_REQUEST: 'pull_request',
             self.TYPE_PULL_REQUEST_COMMENT: 'pull_request_comment',
         }
         self._subj_map = {
             self.TYPE_CHANGESET_COMMENT: _('[Comment from %(comment_username)s] %(repo_name)s changeset %(short_id)s on %(branch)s'),
             self.TYPE_MESSAGE: 'Test Message',
             # self.TYPE_PASSWORD_RESET
             self.TYPE_REGISTRATION: _('New user %(new_username)s registered'),
             # self.TYPE_DEFAULT
             self.TYPE_PULL_REQUEST: _('[Added by %(pr_username)s] %(repo_name)s pull request %(pr_nice_id)s from %(ref)s'),
             self.TYPE_PULL_REQUEST_COMMENT: _('[Comment from %(comment_username)s] %(repo_name)s pull request %(pr_nice_id)s from %(ref)s'),
         }
 
     def get_email_description(self, type_, **kwargs):
         """
         return subject for email based on given type
         """
         tmpl = self._subj_map[type_]
         try:
             subj = tmpl % kwargs
         except KeyError as e:
             log.error('error generating email subject for %r from %s: %s', type_, ','.join(self._subj_map.keys()), e)
             raise
         l = [safe_unicode(x) for x in [kwargs.get('status_change'), kwargs.get('closing_pr') and _('Closing')] if x]
         if l:
             if subj.startswith('['):
                 subj = '[' + ', '.join(l) + ': ' + subj[1:]
             else:
                 subj = '[' + ', '.join(l) + '] ' + subj
         return subj
 
     def get_email_tmpl(self, type_, content_type, **kwargs):
         """
         return generated template for email based on given type
         """
 
         base = 'email_templates/' + self.email_types.get(type_, self.email_types[self.TYPE_DEFAULT]) + '.' + content_type
         email_template = self._tmpl_lookup.get_template(base)
         # translator and helpers inject
         _kwargs = {'_': _,
                    'h': h,
                    'c': c}
         _kwargs.update(kwargs)
         log.debug('rendering tmpl %s with kwargs %s', base, _kwargs)
         return email_template.render(**_kwargs)

kallithea/tests/other/test_mail.py

@@ -1,91 +1,167 b''
 import mock
 
 import kallithea
 from kallithea.tests import *
+from kallithea.model.db import User
 
 class smtplib_mock(object):
 
     @classmethod
     def SMTP(cls, server, port, local_hostname):
         return smtplib_mock()
 
     def ehlo(self):
         pass
     def quit(self):
         pass
     def sendmail(self, sender, dest, msg):
         smtplib_mock.lastsender = sender
         smtplib_mock.lastdest = dest
         smtplib_mock.lastmsg = msg
         pass
 
 @mock.patch('kallithea.lib.rcmail.smtp_mailer.smtplib', smtplib_mock)
 class TestMail(BaseTestCase):
 
     def test_send_mail_trivial(self):
         mailserver = 'smtp.mailserver.org'
         recipients = ['rcpt1', 'rcpt2']
         envelope_from = '[e-mail]'
         subject = 'subject'
         body = 'body'
         html_body = 'html_body'
 
         config_mock = {
             'smtp_server': mailserver,
             'app_email_from': envelope_from,
         }
         with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock):
             kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body)
 
         self.assertSetEqual(smtplib_mock.lastdest, set(recipients))
         self.assertEqual(smtplib_mock.lastsender, envelope_from)
         self.assertIn('From: %s' % envelope_from, smtplib_mock.lastmsg)
         self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg)
         self.assertIn(body, smtplib_mock.lastmsg)
         self.assertIn(html_body, smtplib_mock.lastmsg)
 
     def test_send_mail_no_recipients(self):
         mailserver = 'smtp.mailserver.org'
         recipients = []
         envelope_from = '[e-mail]'
         email_to = '[e-mail]'
         subject = 'subject'
         body = 'body'
         html_body = 'html_body'
 
         config_mock = {
             'smtp_server': mailserver,
             'app_email_from': envelope_from,
             'email_to': email_to,
         }
         with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock):
             kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body)
 
         self.assertSetEqual(smtplib_mock.lastdest, set([TEST_USER_ADMIN_EMAIL, email_to]))
         self.assertEqual(smtplib_mock.lastsender, envelope_from)
         self.assertIn('From: %s' % envelope_from, smtplib_mock.lastmsg)
         self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg)
         self.assertIn(body, smtplib_mock.lastmsg)
         self.assertIn(html_body, smtplib_mock.lastmsg)
 
     def test_send_mail_no_recipients_no_email_to(self):
         mailserver = 'smtp.mailserver.org'
         recipients = []
         envelope_from = '[e-mail]'
         subject = 'subject'
         body = 'body'
         html_body = 'html_body'
 
         config_mock = {
             'smtp_server': mailserver,
             'app_email_from': envelope_from,
         }
         with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock):
             kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body)
 
         self.assertSetEqual(smtplib_mock.lastdest, set([TEST_USER_ADMIN_EMAIL]))
         self.assertEqual(smtplib_mock.lastsender, envelope_from)
         self.assertIn('From: %s' % envelope_from, smtplib_mock.lastmsg)
         self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg)
         self.assertIn(body, smtplib_mock.lastmsg)
         self.assertIn(html_body, smtplib_mock.lastmsg)
+
+    def test_send_mail_with_author(self):
+        mailserver = 'smtp.mailserver.org'
+        recipients = ['rcpt1', 'rcpt2']
+        envelope_from = '[e-mail]'
+        subject = 'subject'
+        body = 'body'
+        html_body = 'html_body'
+        author = User.get_by_username(TEST_USER_REGULAR_LOGIN)
+
+        config_mock = {
+            'smtp_server': mailserver,
+            'app_email_from': envelope_from,
+        }
+        with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock):
+            kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body, author=author)
+
+        self.assertSetEqual(smtplib_mock.lastdest, set(recipients))
+        self.assertEqual(smtplib_mock.lastsender, envelope_from)
+        self.assertIn('From: "Kallithea Admin (no-reply)" <%s>' % envelope_from, smtplib_mock.lastmsg)
+        self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg)
+        self.assertIn(body, smtplib_mock.lastmsg)
+        self.assertIn(html_body, smtplib_mock.lastmsg)
+
+    def test_send_mail_with_author_full_mail_from(self):
+        mailserver = 'smtp.mailserver.org'
+        recipients = ['rcpt1', 'rcpt2']
+        envelope_addr = '[e-mail]'
+        envelope_from = 'Some Name <%s>' % envelope_addr
+        subject = 'subject'
+        body = 'body'
+        html_body = 'html_body'
+        author = User.get_by_username(TEST_USER_REGULAR_LOGIN)
+
+        config_mock = {
+            'smtp_server': mailserver,
+            'app_email_from': envelope_from,
+        }
+        with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock):
+            kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body, author=author)
+
+        self.assertSetEqual(smtplib_mock.lastdest, set(recipients))
+        self.assertEqual(smtplib_mock.lastsender, envelope_from)
+        self.assertIn('From: "Kallithea Admin (no-reply)" <%s>' % envelope_addr, smtplib_mock.lastmsg)
+        self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg)
+        self.assertIn(body, smtplib_mock.lastmsg)
+        self.assertIn(html_body, smtplib_mock.lastmsg)
+
+    def test_send_mail_extra_headers(self):
+        mailserver = 'smtp.mailserver.org'
+        recipients = ['rcpt1', 'rcpt2']
+        envelope_from = '[e-mail]'
+        subject = 'subject'
+        body = 'body'
+        html_body = 'html_body'
+        author = User(name='foo', lastname='(fubar) "baz"')
+        headers = {'extra': 'yes'}
+
+        config_mock = {
+            'smtp_server': mailserver,
+            'app_email_from': envelope_from,
+        }
+        with mock.patch('kallithea.lib.celerylib.tasks.config', config_mock):
+            kallithea.lib.celerylib.tasks.send_email(recipients, subject, body, html_body,
+                                                     author=author, headers=headers)
+
+        self.assertSetEqual(smtplib_mock.lastdest, set(recipients))
+        self.assertEqual(smtplib_mock.lastsender, envelope_from)
+        self.assertIn(r'From: "foo (fubar) \"baz\" (no-reply)" <%s>' % envelope_from, smtplib_mock.lastmsg)
+        self.assertIn('Subject: %s' % subject, smtplib_mock.lastmsg)
+        self.assertIn(body, smtplib_mock.lastmsg)
+        self.assertIn(html_body, smtplib_mock.lastmsg)
+        self.assertIn('Extra: yes', smtplib_mock.lastmsg)
+        # verify that headers dict hasn't mutated by send_email
+        self.assertDictEqual(headers, {'extra': 'yes'})