bugzilla: add XMLRPC interface.

Add support for access to Bugzilla via the XMLRPC interface.

This requires a single username and password used to log in to Bugzilla,
plus the URL of the Bugzilla installation. Commit messages are added to
bugs as before, but security only permits them to be added as the
username used to log in.

hgext/bugzilla.py

@@ -1,6 +1,7 b''
 # bugzilla.py - bugzilla integration for mercurial
 #
 # Copyright 2006 Vadim Gelfer <[e-mail]>
+# Copyright 2011 Jim Hague <[e-mail]>
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
@@ -8,56 +9,43 b''
 '''hooks for integrating with the Bugzilla bug tracker
 
 This hook extension adds comments on bugs in Bugzilla when changesets
-that refer to bugs by Bugzilla ID are seen. The hook does not change
-bug status.
+that refer to bugs by Bugzilla ID are seen. The comment is formatted using
+the Mercurial template mechanism.
 
-The hook updates the Bugzilla database directly. Only Bugzilla
-installations using MySQL are supported.
+The hook does not change bug status.
 
-The hook relies on a Bugzilla script to send bug change notification
-emails. That script changes between Bugzilla versions; the
-'processmail' script used prior to 2.18 is replaced in 2.18 and
-subsequent versions by 'config/sendbugmail.pl'. Note that these will
-be run by Mercurial as the user pushing the change; you will need to
-ensure the Bugzilla install file permissions are set appropriately.
+Two basic modes of access to Bugzilla are provided:
 
-The extension is configured through three different configuration
-sections. These keys are recognized in the [bugzilla] section:
-
-host
-  Hostname of the MySQL server holding the Bugzilla database.
+1. Access via the Bugzilla XMLRPC interface (requires Bugzilla 3.4 or later).
 
-db
-  Name of the Bugzilla database in MySQL. Default 'bugs'.
-
-user
-  Username to use to access MySQL server. Default 'bugs'.
+2. Writing directly to the Bugzilla database. Only Bugzilla installations
+   using MySQL are supported. Requires Python MySQLdb.
 
-password
-  Password to use to access MySQL server.
-
-timeout
-  Database connection timeout (seconds). Default 5.
-
-version
-  Bugzilla version. Specify '3.0' for Bugzilla versions 3.0 and later,
-  '2.18' for Bugzilla versions from 2.18 and '2.16' for versions prior
-  to 2.18.
+Writing directly to the database is susceptible to schema changes, and
+relies on a Bugzilla contrib script to send out bug change
+notification emails. This script runs as the user running Mercurial,
+must be run on the host with the Bugzilla install, and requires
+permission to read Bugzilla configuration details and the necessary
+MySQL user and password to have full access rights to the Bugzilla
+database. For these reasons this access mode is now considered
+deprecated, and will not be updated for new Bugzilla versions going
+forward.
 
-bzuser
-  Fallback Bugzilla user name to record comments with, if changeset
-  committer cannot be found as a Bugzilla user.
+Access via XMLRPC needs a Bugzilla username and password to be specified
+in the configuration. Comments are added under that username. Since the
+configuration must be readable by all Mercurial users, it is recommended
+that the rights of that user are restricted in Bugzilla to the minimum
+necessary to add comments.
+
+Configuration items common to both access modes:
 
-bzdir
-   Bugzilla install directory. Used by default notify. Default
-   '/var/www/html/bugzilla'.
-
-notify
-  The command to run to get Bugzilla to send bug change notification
-  emails. Substitutes from a map with 3 keys, 'bzdir', 'id' (bug id)
-  and 'user' (committer bugzilla email). Default depends on version;
-  from 2.18 it is "cd %(bzdir)s && perl -T contrib/sendbugmail.pl
-  %(id)s %(user)s".
+[bugzilla]
+version
+  This access type to use. Values recognised are:
+  xmlrpc  Bugzilla XMLRPC interface.
+  3.0     MySQL access, Bugzilla 3.0 and later.
+  2.18    MySQL access, Bugzilla 2.18 and up to but not including 3.0.
+  2.16    MySQL access, Bugzilla 2.16 and up to but not including 2.18.
 
 regexp
   Regular expression to match bug IDs in changeset commit message.
@@ -82,23 +70,72 b' template'
           'to bug {bug}.\\ndetails:\\n\\t{desc|tabindent}'
 
 strip
-  The number of slashes to strip from the front of {root} to produce
-  {webroot}. Default 0.
+  The number of path separator characters to strip from the front of the
+  Mercurial repository path ('{root}' in templates) to produce '{webroot}'.
+  For example, a repository with '{root}' '/var/local/my-project' with a
+  strip of 2 gives a value for '{webroot}' of 'my-project'. Default 0.
+
+[web]
+baseurl
+  Base URL for browsing Mercurial repositories. Referenced from
+  templates as {hgweb}.
+
+XMLRPC access mode configuration:
+
+[bugzilla]
+bzurl
+  The base URL for the Bugzilla installation.
+  Default 'http://localhost/bugzilla'.
+
+user
+  The username to use to log into Bugzilla via XMLRPC. Default 'bugs'.
+
+password
+  The password for Bugzilla login.
+
+MySQL access mode configuration:
+
+[bugzilla]
+host
+  Hostname of the MySQL server holding the Bugzilla database.
+  Default 'localhost'.
+
+db
+  Name of the Bugzilla database in MySQL. Default 'bugs'.
+
+user
+  Username to use to access MySQL server. Default 'bugs'.
+
+password
+  Password to use to access MySQL server.
+
+timeout
+  Database connection timeout (seconds). Default 5.
+
+bzuser
+  Fallback Bugzilla user name to record comments with, if changeset
+  committer cannot be found as a Bugzilla user.
+
+bzdir
+   Bugzilla install directory. Used by default notify. Default
+   '/var/www/html/bugzilla'.
+
+notify
+  The command to run to get Bugzilla to send bug change notification
+  emails. Substitutes from a map with 3 keys, 'bzdir', 'id' (bug id)
+  and 'user' (committer bugzilla email). Default depends on version;
+  from 2.18 it is "cd %(bzdir)s && perl -T contrib/sendbugmail.pl
+  %(id)s %(user)s".
 
 usermap
   Path of file containing Mercurial committer ID to Bugzilla user ID
   mappings. If specified, the file should contain one mapping per
   line, "committer"="Bugzilla user". See also the [usermap] section.
 
+[usermap]
 The [usermap] section is used to specify mappings of Mercurial
-committer ID to Bugzilla user ID. See also [bugzilla].usermap.
-"committer"="Bugzilla user"
-
-Finally, the [web] section supports one entry:
-
-baseurl
-  Base URL for browsing Mercurial repositories. Reference from
-  templates as {hgweb}.
+committer email to Bugzilla user email. See also [bugzilla].usermap.
+Contains entries of the form "committer"="Bugzilla user".
 
 Activating the extension::
 
@@ -109,11 +146,27 b' Activating the extension::'
     # run bugzilla hook on every change pulled or pushed in here
     incoming.bugzilla = python:hgext.bugzilla.hook
 
-Example configuration:
+Example configurations:
+
+XMLRPC example configuration. This uses the Bugzilla at
+'http://my-project.org/bugzilla', logging in as user '[e-mail]'
+wityh password 'plugh'. It is used with a collection of Mercurial
+repositories in '/var/local/hg/repos/'. ::
 
-This example configuration is for a collection of Mercurial
-repositories in /var/local/hg/repos/ used with a local Bugzilla 3.2
-installation in /opt/bugzilla-3.2. ::
+    [bugzilla]
+    bzurl=http://my-project.org/bugzilla
+    user=[e-mail]
+    password=plugh
+    version=xmlrpc
+
+    [web]
+    baseurl=http://my-project.org/hg
+
+MySQL example configuration. This is for a collection of Mercurial
+repositories in '/var/local/hg/repos/' used with a local Bugzilla 3.2
+installation in /opt/bugzilla-3.2. The MySQL database is on 'localhost',
+the Bugzilla database name is 'bugs' and MySQL is accessed with MySQL
+username 'bugs' password 'XYZZY'. ::
 
     [bugzilla]
     host=localhost
@@ -132,7 +185,7 b' installation in /opt/bugzilla-3.2. ::'
     [usermap]
     [e-mail]=[e-mail]
 
-Commits add a comment to the Bugzilla bug record of the form::
+Both the above add a comment to the Bugzilla bug record of the form::
 
     Changeset 3b16791d6642 in repository-name.
     http://dev.domain.com/hg/repository-name/rev/3b16791d6642
@@ -143,7 +196,7 b' Commits add a comment to the Bugzilla bu'
 from mercurial.i18n import _
 from mercurial.node import short
 from mercurial import cmdutil, templater, util
-import re, time
+import re, time, xmlrpclib
 
 class bzaccess(object):
     '''Base class for access to Bugzilla.'''
@@ -187,6 +240,9 b' class bzmysql(bzaccess):'
     '''Support for direct MySQL access to Bugzilla.
 
     The earliest Bugzilla version this is tested with is version 2.16.
+
+    If your Bugzilla is version 3.2 or above, you are strongly
+    recommended to use the XMLRPC access method instead.
     '''
 
     @staticmethod
@@ -301,7 +357,7 b' class bzmysql(bzaccess):'
             return userid
 
     def get_bugzilla_user(self, committer):
-        '''see if committer is a registered bugzilla user. Return
+        '''See if committer is a registered bugzilla user. Return
         bugzilla username and userid if so. If not, return default
         bugzilla username and userid.'''
         user = self.map_committer(committer)
@@ -356,13 +412,122 b' class bzmysql_3_0(bzmysql_2_18):'
             raise util.Abort(_('unknown database schema'))
         return ids[0][0]
 
+# Buzgilla via XMLRPC interface.
+
+class CookieSafeTransport(xmlrpclib.SafeTransport):
+    """A SafeTransport that retains cookies over its lifetime.
+
+    The regular xmlrpclib transports ignore cookies. Which causes
+    a bit of a problem when you need a cookie-based login, as with
+    the Bugzilla XMLRPC interface.
+
+    So this is a SafeTransport which looks for cookies being set
+    in responses and saves them to add to all future requests.
+    It appears a SafeTransport can do both HTTP and HTTPS sessions,
+    which saves us having to do a CookieTransport too.
+    """
+
+    # Inspiration drawn from
+    # http://blog.godson.in/2010/09/how-to-make-python-xmlrpclib-client.html
+    # http://www.itkovian.net/base/transport-class-for-pythons-xml-rpc-lib/
+
+    cookies = []
+    def send_cookies(self, connection):
+        if self.cookies:
+            for cookie in self.cookies:
+                connection.putheader("Cookie", cookie)
+
+    def request(self, host, handler, request_body, verbose=0):
+        self.verbose = verbose
+
+        # issue XML-RPC request
+        h = self.make_connection(host)
+        if verbose:
+            h.set_debuglevel(1)
+
+        self.send_request(h, handler, request_body)
+        self.send_host(h, host)
+        self.send_cookies(h)
+        self.send_user_agent(h)
+        self.send_content(h, request_body)
+
+        # Deal with differences between Python 2.4-2.6 and 2.7.
+        # In the former h is a HTTP(S). In the latter it's a
+        # HTTP(S)Connection. Luckily, the 2.4-2.6 implementation of
+        # HTTP(S) has an underlying HTTP(S)Connection, so extract
+        # that and use it.
+        try:
+            response = h.getresponse()
+        except AttributeError:
+            response = h._conn.getresponse()
+
+        # Add any cookie definitions to our list.
+        for header in response.msg.getallmatchingheaders("Set-Cookie"):
+            val = header.split(": ", 1)[1]
+            cookie = val.split(";", 1)[0]
+            self.cookies.append(cookie)
+
+        if response.status != 200:
+            raise xmlrpclib.ProtocolError(host + handler, response.status,
+                                          response.reason, response.msg.headers)
+
+        payload = response.read()
+        parser, unmarshaller = self.getparser()
+        parser.feed(payload)
+        parser.close()
+
+        return unmarshaller.close()
+
+class bzxmlrpc(bzaccess):
+    """Support for access to Bugzilla via the Bugzilla XMLRPC API.
+
+    Requires a minimum Bugzilla version 3.4.
+    """
+
+    def __init__(self, ui):
+        bzaccess.__init__(self, ui)
+
+        bzweb = self.ui.config('bugzilla', 'bzurl',
+                               'http://localhost/bugzilla/')
+        bzweb = bzweb.rstrip("/") + "/xmlrpc.cgi"
+
+        user = self.ui.config('bugzilla', 'user', 'bugs')
+        passwd = self.ui.config('bugzilla', 'password')
+
+        self.bzproxy = xmlrpclib.ServerProxy(bzweb, CookieSafeTransport())
+        self.bzproxy.User.login(dict(login=user, password=passwd))
+
+    def get_bug_comments(self, id):
+        """Return a string with all comment text for a bug."""
+        c = self.bzproxy.Bug.comments(dict(ids=[id]))
+        return ''.join([t['text'] for t in c['bugs'][str(id)]['comments']])
+
+    def filter_real_bug_ids(self, ids):
+        res = set()
+        bugs = self.bzproxy.Bug.get(dict(ids=sorted(ids), permissive=True))
+        for bug in bugs['bugs']:
+            res.add(bug['id'])
+        return res
+
+    def filter_cset_known_bug_ids(self, node, ids):
+        for id in sorted(ids):
+            if self.get_bug_comments(id).find(short(node)) != -1:
+                self.ui.status(_('bug %d already knows about changeset %s\n') %
+                               (id, short(node)))
+                ids.discard(id)
+        return ids
+
+    def add_comment(self, bugid, text, committer):
+        self.bzproxy.Bug.add_comment(dict(id=bugid, comment=text))
+
 class bugzilla(object):
     # supported versions of bugzilla. different versions have
     # different schemas.
     _versions = {
         '2.16': bzmysql,
         '2.18': bzmysql_2_18,
-        '3.0':  bzmysql_3_0
+        '3.0':  bzmysql_3_0,
+        'xmlrpc': bzxmlrpc
         }
 
     _default_bug_re = (r'bugs?\s*,?\s*(?:#|nos?\.?|num(?:ber)?s?)?\s*'