py3: conditionalize the urlparse import

The urlparse library is renamed to urllib.parse in python 3

hgext/bugzilla.py

@@ -1,927 +1,928 b''
 # bugzilla.py - bugzilla integration for mercurial
 #
 # Copyright 2006 Vadim Gelfer <[e-mail]>
 # Copyright 2011-4 Jim Hague <[e-mail]>
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 '''hooks for integrating with the Bugzilla bug tracker
 
 This hook extension adds comments on bugs in Bugzilla when changesets
 that refer to bugs by Bugzilla ID are seen. The comment is formatted using
 the Mercurial template mechanism.
 
 The bug references can optionally include an update for Bugzilla of the
 hours spent working on the bug. Bugs can also be marked fixed.
 
 Three basic modes of access to Bugzilla are provided:
 
 1. Access via the Bugzilla XMLRPC interface. Requires Bugzilla 3.4 or later.
 
 2. Check data via the Bugzilla XMLRPC interface and submit bug change
    via email to Bugzilla email interface. Requires Bugzilla 3.4 or later.
 
 3. Writing directly to the Bugzilla database. Only Bugzilla installations
    using MySQL are supported. Requires Python MySQLdb.
 
 Writing directly to the database is susceptible to schema changes, and
 relies on a Bugzilla contrib script to send out bug change
 notification emails. This script runs as the user running Mercurial,
 must be run on the host with the Bugzilla install, and requires
 permission to read Bugzilla configuration details and the necessary
 MySQL user and password to have full access rights to the Bugzilla
 database. For these reasons this access mode is now considered
 deprecated, and will not be updated for new Bugzilla versions going
 forward. Only adding comments is supported in this access mode.
 
 Access via XMLRPC needs a Bugzilla username and password to be specified
 in the configuration. Comments are added under that username. Since the
 configuration must be readable by all Mercurial users, it is recommended
 that the rights of that user are restricted in Bugzilla to the minimum
 necessary to add comments. Marking bugs fixed requires Bugzilla 4.0 and later.
 
 Access via XMLRPC/email uses XMLRPC to query Bugzilla, but sends
 email to the Bugzilla email interface to submit comments to bugs.
 The From: address in the email is set to the email address of the Mercurial
 user, so the comment appears to come from the Mercurial user. In the event
 that the Mercurial user email is not recognized by Bugzilla as a Bugzilla
 user, the email associated with the Bugzilla username used to log into
 Bugzilla is used instead as the source of the comment. Marking bugs fixed
 works on all supported Bugzilla versions.
 
 Configuration items common to all access modes:
 
 bugzilla.version
   The access type to use. Values recognized are:
 
   :``xmlrpc``:       Bugzilla XMLRPC interface.
   :``xmlrpc+email``: Bugzilla XMLRPC and email interfaces.
   :``3.0``:          MySQL access, Bugzilla 3.0 and later.
   :``2.18``:         MySQL access, Bugzilla 2.18 and up to but not
                      including 3.0.
   :``2.16``:         MySQL access, Bugzilla 2.16 and up to but not
                      including 2.18.
 
 bugzilla.regexp
   Regular expression to match bug IDs for update in changeset commit message.
   It must contain one "()" named group ``<ids>`` containing the bug
   IDs separated by non-digit characters. It may also contain
   a named group ``<hours>`` with a floating-point number giving the
   hours worked on the bug. If no named groups are present, the first
   "()" group is assumed to contain the bug IDs, and work time is not
   updated. The default expression matches ``Bug 1234``, ``Bug no. 1234``,
   ``Bug number 1234``, ``Bugs 1234,5678``, ``Bug 1234 and 5678`` and
   variations thereof, followed by an hours number prefixed by ``h`` or
   ``hours``, e.g. ``hours 1.5``. Matching is case insensitive.
 
 bugzilla.fixregexp
   Regular expression to match bug IDs for marking fixed in changeset
   commit message. This must contain a "()" named group ``<ids>` containing
   the bug IDs separated by non-digit characters. It may also contain
   a named group ``<hours>`` with a floating-point number giving the
   hours worked on the bug. If no named groups are present, the first
   "()" group is assumed to contain the bug IDs, and work time is not
   updated. The default expression matches ``Fixes 1234``, ``Fixes bug 1234``,
   ``Fixes bugs 1234,5678``, ``Fixes 1234 and 5678`` and
   variations thereof, followed by an hours number prefixed by ``h`` or
   ``hours``, e.g. ``hours 1.5``. Matching is case insensitive.
 
 bugzilla.fixstatus
   The status to set a bug to when marking fixed. Default ``RESOLVED``.
 
 bugzilla.fixresolution
   The resolution to set a bug to when marking fixed. Default ``FIXED``.
 
 bugzilla.style
   The style file to use when formatting comments.
 
 bugzilla.template
   Template to use when formatting comments. Overrides style if
   specified. In addition to the usual Mercurial keywords, the
   extension specifies:
 
   :``{bug}``:     The Bugzilla bug ID.
   :``{root}``:    The full pathname of the Mercurial repository.
   :``{webroot}``: Stripped pathname of the Mercurial repository.
   :``{hgweb}``:   Base URL for browsing Mercurial repositories.
 
   Default ``changeset {node|short} in repo {root} refers to bug
   {bug}.\\ndetails:\\n\\t{desc|tabindent}``
 
 bugzilla.strip
   The number of path separator characters to strip from the front of
   the Mercurial repository path (``{root}`` in templates) to produce
   ``{webroot}``. For example, a repository with ``{root}``
   ``/var/local/my-project`` with a strip of 2 gives a value for
   ``{webroot}`` of ``my-project``. Default 0.
 
 web.baseurl
   Base URL for browsing Mercurial repositories. Referenced from
   templates as ``{hgweb}``.
 
 Configuration items common to XMLRPC+email and MySQL access modes:
 
 bugzilla.usermap
   Path of file containing Mercurial committer email to Bugzilla user email
   mappings. If specified, the file should contain one mapping per
   line::
 
     committer = Bugzilla user
 
   See also the ``[usermap]`` section.
 
 The ``[usermap]`` section is used to specify mappings of Mercurial
 committer email to Bugzilla user email. See also ``bugzilla.usermap``.
 Contains entries of the form ``committer = Bugzilla user``.
 
 XMLRPC access mode configuration:
 
 bugzilla.bzurl
   The base URL for the Bugzilla installation.
   Default ``http://localhost/bugzilla``.
 
 bugzilla.user
   The username to use to log into Bugzilla via XMLRPC. Default
   ``bugs``.
 
 bugzilla.password
   The password for Bugzilla login.
 
 XMLRPC+email access mode uses the XMLRPC access mode configuration items,
 and also:
 
 bugzilla.bzemail
   The Bugzilla email address.
 
 In addition, the Mercurial email settings must be configured. See the
 documentation in hgrc(5), sections ``[email]`` and ``[smtp]``.
 
 MySQL access mode configuration:
 
 bugzilla.host
   Hostname of the MySQL server holding the Bugzilla database.
   Default ``localhost``.
 
 bugzilla.db
   Name of the Bugzilla database in MySQL. Default ``bugs``.
 
 bugzilla.user
   Username to use to access MySQL server. Default ``bugs``.
 
 bugzilla.password
   Password to use to access MySQL server.
 
 bugzilla.timeout
   Database connection timeout (seconds). Default 5.
 
 bugzilla.bzuser
   Fallback Bugzilla user name to record comments with, if changeset
   committer cannot be found as a Bugzilla user.
 
 bugzilla.bzdir
    Bugzilla install directory. Used by default notify. Default
    ``/var/www/html/bugzilla``.
 
 bugzilla.notify
   The command to run to get Bugzilla to send bug change notification
   emails. Substitutes from a map with 3 keys, ``bzdir``, ``id`` (bug
   id) and ``user`` (committer bugzilla email). Default depends on
   version; from 2.18 it is "cd %(bzdir)s && perl -T
   contrib/sendbugmail.pl %(id)s %(user)s".
 
 Activating the extension::
 
     [extensions]
     bugzilla =
 
     [hooks]
     # run bugzilla hook on every change pulled or pushed in here
     incoming.bugzilla = python:hgext.bugzilla.hook
 
 Example configurations:
 
 XMLRPC example configuration. This uses the Bugzilla at
 ``http://my-project.org/bugzilla``, logging in as user
 ``[e-mail]`` with password ``plugh``. It is used with a
 collection of Mercurial repositories in ``/var/local/hg/repos/``,
 with a web interface at ``http://my-project.org/hg``. ::
 
     [bugzilla]
     bzurl=http://my-project.org/bugzilla
     user=[e-mail]
     password=plugh
     version=xmlrpc
     template=Changeset {node|short} in {root|basename}.
              {hgweb}/{webroot}/rev/{node|short}\\n
              {desc}\\n
     strip=5
 
     [web]
     baseurl=http://my-project.org/hg
 
 XMLRPC+email example configuration. This uses the Bugzilla at
 ``http://my-project.org/bugzilla``, logging in as user
 ``[e-mail]`` with password ``plugh``. It is used with a
 collection of Mercurial repositories in ``/var/local/hg/repos/``,
 with a web interface at ``http://my-project.org/hg``. Bug comments
 are sent to the Bugzilla email address
 ``[e-mail]``. ::
 
     [bugzilla]
     bzurl=http://my-project.org/bugzilla
     user=[e-mail]
     password=plugh
     version=xmlrpc+email
     bzemail=[e-mail]
     template=Changeset {node|short} in {root|basename}.
              {hgweb}/{webroot}/rev/{node|short}\\n
              {desc}\\n
     strip=5
 
     [web]
     baseurl=http://my-project.org/hg
 
     [usermap]
     [e-mail]=[e-mail]
 
 MySQL example configuration. This has a local Bugzilla 3.2 installation
 in ``/opt/bugzilla-3.2``. The MySQL database is on ``localhost``,
 the Bugzilla database name is ``bugs`` and MySQL is
 accessed with MySQL username ``bugs`` password ``XYZZY``. It is used
 with a collection of Mercurial repositories in ``/var/local/hg/repos/``,
 with a web interface at ``http://my-project.org/hg``. ::
 
     [bugzilla]
     host=localhost
     password=XYZZY
     version=3.0
     bzuser=[e-mail]
     bzdir=/opt/bugzilla-3.2
     template=Changeset {node|short} in {root|basename}.
              {hgweb}/{webroot}/rev/{node|short}\\n
              {desc}\\n
     strip=5
 
     [web]
     baseurl=http://my-project.org/hg
 
     [usermap]
     [e-mail]=[e-mail]
 
 All the above add a comment to the Bugzilla bug record of the form::
 
     Changeset 3b16791d6642 in repository-name.
     http://my-project.org/hg/repository-name/rev/3b16791d6642
 
     Changeset commit comment. Bug 1234.
 '''
 
 from __future__ import absolute_import
 
 import re
 import time
-import urlparse
 import xmlrpclib
 
 from mercurial.i18n import _
 from mercurial.node import short
 from mercurial import (
     cmdutil,
     error,
     mail,
     util,
 )
 
+urlparse = util.urlparse
+
 # Note for extension authors: ONLY specify testedwith = 'internal' for
 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
 # be specifying the version(s) of Mercurial they are tested with, or
 # leave the attribute unspecified.
 testedwith = 'internal'
 
 class bzaccess(object):
     '''Base class for access to Bugzilla.'''
 
     def __init__(self, ui):
         self.ui = ui
         usermap = self.ui.config('bugzilla', 'usermap')
         if usermap:
             self.ui.readconfig(usermap, sections=['usermap'])
 
     def map_committer(self, user):
         '''map name of committer to Bugzilla user name.'''
         for committer, bzuser in self.ui.configitems('usermap'):
             if committer.lower() == user.lower():
                 return bzuser
         return user
 
     # Methods to be implemented by access classes.
     #
     # 'bugs' is a dict keyed on bug id, where values are a dict holding
     # updates to bug state. Recognized dict keys are:
     #
     # 'hours': Value, float containing work hours to be updated.
     # 'fix':   If key present, bug is to be marked fixed. Value ignored.
 
     def filter_real_bug_ids(self, bugs):
         '''remove bug IDs that do not exist in Bugzilla from bugs.'''
         pass
 
     def filter_cset_known_bug_ids(self, node, bugs):
         '''remove bug IDs where node occurs in comment text from bugs.'''
         pass
 
     def updatebug(self, bugid, newstate, text, committer):
         '''update the specified bug. Add comment text and set new states.
 
         If possible add the comment as being from the committer of
         the changeset. Otherwise use the default Bugzilla user.
         '''
         pass
 
     def notify(self, bugs, committer):
         '''Force sending of Bugzilla notification emails.
 
         Only required if the access method does not trigger notification
         emails automatically.
         '''
         pass
 
 # Bugzilla via direct access to MySQL database.
 class bzmysql(bzaccess):
     '''Support for direct MySQL access to Bugzilla.
 
     The earliest Bugzilla version this is tested with is version 2.16.
 
     If your Bugzilla is version 3.4 or above, you are strongly
     recommended to use the XMLRPC access method instead.
     '''
 
     @staticmethod
     def sql_buglist(ids):
         '''return SQL-friendly list of bug ids'''
         return '(' + ','.join(map(str, ids)) + ')'
 
     _MySQLdb = None
 
     def __init__(self, ui):
         try:
             import MySQLdb as mysql
             bzmysql._MySQLdb = mysql
         except ImportError as err:
             raise error.Abort(_('python mysql support not available: %s') % err)
 
         bzaccess.__init__(self, ui)
 
         host = self.ui.config('bugzilla', 'host', 'localhost')
         user = self.ui.config('bugzilla', 'user', 'bugs')
         passwd = self.ui.config('bugzilla', 'password')
         db = self.ui.config('bugzilla', 'db', 'bugs')
         timeout = int(self.ui.config('bugzilla', 'timeout', 5))
         self.ui.note(_('connecting to %s:%s as %s, password %s\n') %
                      (host, db, user, '*' * len(passwd)))
         self.conn = bzmysql._MySQLdb.connect(host=host,
                                                    user=user, passwd=passwd,
                                                    db=db,
                                                    connect_timeout=timeout)
         self.cursor = self.conn.cursor()
         self.longdesc_id = self.get_longdesc_id()
         self.user_ids = {}
         self.default_notify = "cd %(bzdir)s && ./processmail %(id)s %(user)s"
 
     def run(self, *args, **kwargs):
         '''run a query.'''
         self.ui.note(_('query: %s %s\n') % (args, kwargs))
         try:
             self.cursor.execute(*args, **kwargs)
         except bzmysql._MySQLdb.MySQLError:
             self.ui.note(_('failed query: %s %s\n') % (args, kwargs))
             raise
 
     def get_longdesc_id(self):
         '''get identity of longdesc field'''
         self.run('select fieldid from fielddefs where name = "longdesc"')
         ids = self.cursor.fetchall()
         if len(ids) != 1:
             raise error.Abort(_('unknown database schema'))
         return ids[0][0]
 
     def filter_real_bug_ids(self, bugs):
         '''filter not-existing bugs from set.'''
         self.run('select bug_id from bugs where bug_id in %s' %
                  bzmysql.sql_buglist(bugs.keys()))
         existing = [id for (id,) in self.cursor.fetchall()]
         for id in bugs.keys():
             if id not in existing:
                 self.ui.status(_('bug %d does not exist\n') % id)
                 del bugs[id]
 
     def filter_cset_known_bug_ids(self, node, bugs):
         '''filter bug ids that already refer to this changeset from set.'''
         self.run('''select bug_id from longdescs where
                     bug_id in %s and thetext like "%%%s%%"''' %
                  (bzmysql.sql_buglist(bugs.keys()), short(node)))
         for (id,) in self.cursor.fetchall():
             self.ui.status(_('bug %d already knows about changeset %s\n') %
                            (id, short(node)))
             del bugs[id]
 
     def notify(self, bugs, committer):
         '''tell bugzilla to send mail.'''
         self.ui.status(_('telling bugzilla to send mail:\n'))
         (user, userid) = self.get_bugzilla_user(committer)
         for id in bugs.keys():
             self.ui.status(_('  bug %s\n') % id)
             cmdfmt = self.ui.config('bugzilla', 'notify', self.default_notify)
             bzdir = self.ui.config('bugzilla', 'bzdir',
                                    '/var/www/html/bugzilla')
             try:
                 # Backwards-compatible with old notify string, which
                 # took one string. This will throw with a new format
                 # string.
                 cmd = cmdfmt % id
             except TypeError:
                 cmd = cmdfmt % {'bzdir': bzdir, 'id': id, 'user': user}
             self.ui.note(_('running notify command %s\n') % cmd)
             fp = util.popen('(%s) 2>&1' % cmd)
             out = fp.read()
             ret = fp.close()
             if ret:
                 self.ui.warn(out)
                 raise error.Abort(_('bugzilla notify command %s') %
                                  util.explainexit(ret)[0])
         self.ui.status(_('done\n'))
 
     def get_user_id(self, user):
         '''look up numeric bugzilla user id.'''
         try:
             return self.user_ids[user]
         except KeyError:
             try:
                 userid = int(user)
             except ValueError:
                 self.ui.note(_('looking up user %s\n') % user)
                 self.run('''select userid from profiles
                             where login_name like %s''', user)
                 all = self.cursor.fetchall()
                 if len(all) != 1:
                     raise KeyError(user)
                 userid = int(all[0][0])
             self.user_ids[user] = userid
             return userid
 
     def get_bugzilla_user(self, committer):
         '''See if committer is a registered bugzilla user. Return
         bugzilla username and userid if so. If not, return default
         bugzilla username and userid.'''
         user = self.map_committer(committer)
         try:
             userid = self.get_user_id(user)
         except KeyError:
             try:
                 defaultuser = self.ui.config('bugzilla', 'bzuser')
                 if not defaultuser:
                     raise error.Abort(_('cannot find bugzilla user id for %s') %
                                      user)
                 userid = self.get_user_id(defaultuser)
                 user = defaultuser
             except KeyError:
                 raise error.Abort(_('cannot find bugzilla user id for %s or %s')
                                  % (user, defaultuser))
         return (user, userid)
 
     def updatebug(self, bugid, newstate, text, committer):
         '''update bug state with comment text.
 
         Try adding comment as committer of changeset, otherwise as
         default bugzilla user.'''
         if len(newstate) > 0:
             self.ui.warn(_("Bugzilla/MySQL cannot update bug state\n"))
 
         (user, userid) = self.get_bugzilla_user(committer)
         now = time.strftime('%Y-%m-%d %H:%M:%S')
         self.run('''insert into longdescs
                     (bug_id, who, bug_when, thetext)
                     values (%s, %s, %s, %s)''',
                  (bugid, userid, now, text))
         self.run('''insert into bugs_activity (bug_id, who, bug_when, fieldid)
                     values (%s, %s, %s, %s)''',
                  (bugid, userid, now, self.longdesc_id))
         self.conn.commit()
 
 class bzmysql_2_18(bzmysql):
     '''support for bugzilla 2.18 series.'''
 
     def __init__(self, ui):
         bzmysql.__init__(self, ui)
         self.default_notify = \
             "cd %(bzdir)s && perl -T contrib/sendbugmail.pl %(id)s %(user)s"
 
 class bzmysql_3_0(bzmysql_2_18):
     '''support for bugzilla 3.0 series.'''
 
     def __init__(self, ui):
         bzmysql_2_18.__init__(self, ui)
 
     def get_longdesc_id(self):
         '''get identity of longdesc field'''
         self.run('select id from fielddefs where name = "longdesc"')
         ids = self.cursor.fetchall()
         if len(ids) != 1:
             raise error.Abort(_('unknown database schema'))
         return ids[0][0]
 
 # Bugzilla via XMLRPC interface.
 
 class cookietransportrequest(object):
     """A Transport request method that retains cookies over its lifetime.
 
     The regular xmlrpclib transports ignore cookies. Which causes
     a bit of a problem when you need a cookie-based login, as with
     the Bugzilla XMLRPC interface prior to 4.4.3.
 
     So this is a helper for defining a Transport which looks for
     cookies being set in responses and saves them to add to all future
     requests.
     """
 
     # Inspiration drawn from
     # http://blog.godson.in/2010/09/how-to-make-python-xmlrpclib-client.html
     # http://www.itkovian.net/base/transport-class-for-pythons-xml-rpc-lib/
 
     cookies = []
     def send_cookies(self, connection):
         if self.cookies:
             for cookie in self.cookies:
                 connection.putheader("Cookie", cookie)
 
     def request(self, host, handler, request_body, verbose=0):
         self.verbose = verbose
         self.accept_gzip_encoding = False
 
         # issue XML-RPC request
         h = self.make_connection(host)
         if verbose:
             h.set_debuglevel(1)
 
         self.send_request(h, handler, request_body)
         self.send_host(h, host)
         self.send_cookies(h)
         self.send_user_agent(h)
         self.send_content(h, request_body)
 
         # Deal with differences between Python 2.4-2.6 and 2.7.
         # In the former h is a HTTP(S). In the latter it's a
         # HTTP(S)Connection. Luckily, the 2.4-2.6 implementation of
         # HTTP(S) has an underlying HTTP(S)Connection, so extract
         # that and use it.
         try:
             response = h.getresponse()
         except AttributeError:
             response = h._conn.getresponse()
 
         # Add any cookie definitions to our list.
         for header in response.msg.getallmatchingheaders("Set-Cookie"):
             val = header.split(": ", 1)[1]
             cookie = val.split(";", 1)[0]
             self.cookies.append(cookie)
 
         if response.status != 200:
             raise xmlrpclib.ProtocolError(host + handler, response.status,
                                           response.reason, response.msg.headers)
 
         payload = response.read()
         parser, unmarshaller = self.getparser()
         parser.feed(payload)
         parser.close()
 
         return unmarshaller.close()
 
 # The explicit calls to the underlying xmlrpclib __init__() methods are
 # necessary. The xmlrpclib.Transport classes are old-style classes, and
 # it turns out their __init__() doesn't get called when doing multiple
 # inheritance with a new-style class.
 class cookietransport(cookietransportrequest, xmlrpclib.Transport):
     def __init__(self, use_datetime=0):
         if util.safehasattr(xmlrpclib.Transport, "__init__"):
             xmlrpclib.Transport.__init__(self, use_datetime)
 
 class cookiesafetransport(cookietransportrequest, xmlrpclib.SafeTransport):
     def __init__(self, use_datetime=0):
         if util.safehasattr(xmlrpclib.Transport, "__init__"):
             xmlrpclib.SafeTransport.__init__(self, use_datetime)
 
 class bzxmlrpc(bzaccess):
     """Support for access to Bugzilla via the Bugzilla XMLRPC API.
 
     Requires a minimum Bugzilla version 3.4.
     """
 
     def __init__(self, ui):
         bzaccess.__init__(self, ui)
 
         bzweb = self.ui.config('bugzilla', 'bzurl',
                                'http://localhost/bugzilla/')
         bzweb = bzweb.rstrip("/") + "/xmlrpc.cgi"
 
         user = self.ui.config('bugzilla', 'user', 'bugs')
         passwd = self.ui.config('bugzilla', 'password')
 
         self.fixstatus = self.ui.config('bugzilla', 'fixstatus', 'RESOLVED')
         self.fixresolution = self.ui.config('bugzilla', 'fixresolution',
                                             'FIXED')
 
         self.bzproxy = xmlrpclib.ServerProxy(bzweb, self.transport(bzweb))
         ver = self.bzproxy.Bugzilla.version()['version'].split('.')
         self.bzvermajor = int(ver[0])
         self.bzverminor = int(ver[1])
         login = self.bzproxy.User.login({'login': user, 'password': passwd,
                                          'restrict_login': True})
         self.bztoken = login.get('token', '')
 
     def transport(self, uri):
         if urlparse.urlparse(uri, "http")[0] == "https":
             return cookiesafetransport()
         else:
             return cookietransport()
 
     def get_bug_comments(self, id):
         """Return a string with all comment text for a bug."""
         c = self.bzproxy.Bug.comments({'ids': [id],
                                        'include_fields': ['text'],
                                        'token': self.bztoken})
         return ''.join([t['text'] for t in c['bugs'][str(id)]['comments']])
 
     def filter_real_bug_ids(self, bugs):
         probe = self.bzproxy.Bug.get({'ids': sorted(bugs.keys()),
                                       'include_fields': [],
                                       'permissive': True,
                                       'token': self.bztoken,
                                       })
         for badbug in probe['faults']:
             id = badbug['id']
             self.ui.status(_('bug %d does not exist\n') % id)
             del bugs[id]
 
     def filter_cset_known_bug_ids(self, node, bugs):
         for id in sorted(bugs.keys()):
             if self.get_bug_comments(id).find(short(node)) != -1:
                 self.ui.status(_('bug %d already knows about changeset %s\n') %
                                (id, short(node)))
                 del bugs[id]
 
     def updatebug(self, bugid, newstate, text, committer):
         args = {}
         if 'hours' in newstate:
             args['work_time'] = newstate['hours']
 
         if self.bzvermajor >= 4:
             args['ids'] = [bugid]
             args['comment'] = {'body' : text}
             if 'fix' in newstate:
                 args['status'] = self.fixstatus
                 args['resolution'] = self.fixresolution
             args['token'] = self.bztoken
             self.bzproxy.Bug.update(args)
         else:
             if 'fix' in newstate:
                 self.ui.warn(_("Bugzilla/XMLRPC needs Bugzilla 4.0 or later "
                                "to mark bugs fixed\n"))
             args['id'] = bugid
             args['comment'] = text
             self.bzproxy.Bug.add_comment(args)
 
 class bzxmlrpcemail(bzxmlrpc):
     """Read data from Bugzilla via XMLRPC, send updates via email.
 
     Advantages of sending updates via email:
       1. Comments can be added as any user, not just logged in user.
       2. Bug statuses or other fields not accessible via XMLRPC can
          potentially be updated.
 
     There is no XMLRPC function to change bug status before Bugzilla
     4.0, so bugs cannot be marked fixed via XMLRPC before Bugzilla 4.0.
     But bugs can be marked fixed via email from 3.4 onwards.
     """
 
     # The email interface changes subtly between 3.4 and 3.6. In 3.4,
     # in-email fields are specified as '@<fieldname> = <value>'. In
     # 3.6 this becomes '@<fieldname> <value>'. And fieldname @bug_id
     # in 3.4 becomes @id in 3.6. 3.6 and 4.0 both maintain backwards
     # compatibility, but rather than rely on this use the new format for
     # 4.0 onwards.
 
     def __init__(self, ui):
         bzxmlrpc.__init__(self, ui)
 
         self.bzemail = self.ui.config('bugzilla', 'bzemail')
         if not self.bzemail:
             raise error.Abort(_("configuration 'bzemail' missing"))
         mail.validateconfig(self.ui)
 
     def makecommandline(self, fieldname, value):
         if self.bzvermajor >= 4:
             return "@%s %s" % (fieldname, str(value))
         else:
             if fieldname == "id":
                 fieldname = "bug_id"
             return "@%s = %s" % (fieldname, str(value))
 
     def send_bug_modify_email(self, bugid, commands, comment, committer):
         '''send modification message to Bugzilla bug via email.
 
         The message format is documented in the Bugzilla email_in.pl
         specification. commands is a list of command lines, comment is the
         comment text.
 
         To stop users from crafting commit comments with
         Bugzilla commands, specify the bug ID via the message body, rather
         than the subject line, and leave a blank line after it.
         '''
         user = self.map_committer(committer)
         matches = self.bzproxy.User.get({'match': [user],
                                          'token': self.bztoken})
         if not matches['users']:
             user = self.ui.config('bugzilla', 'user', 'bugs')
             matches = self.bzproxy.User.get({'match': [user],
                                              'token': self.bztoken})
             if not matches['users']:
                 raise error.Abort(_("default bugzilla user %s email not found")
                                   % user)
         user = matches['users'][0]['email']
         commands.append(self.makecommandline("id", bugid))
 
         text = "\n".join(commands) + "\n\n" + comment
 
         _charsets = mail._charsets(self.ui)
         user = mail.addressencode(self.ui, user, _charsets)
         bzemail = mail.addressencode(self.ui, self.bzemail, _charsets)
         msg = mail.mimeencode(self.ui, text, _charsets)
         msg['From'] = user
         msg['To'] = bzemail
         msg['Subject'] = mail.headencode(self.ui, "Bug modification", _charsets)
         sendmail = mail.connect(self.ui)
         sendmail(user, bzemail, msg.as_string())
 
     def updatebug(self, bugid, newstate, text, committer):
         cmds = []
         if 'hours' in newstate:
             cmds.append(self.makecommandline("work_time", newstate['hours']))
         if 'fix' in newstate:
             cmds.append(self.makecommandline("bug_status", self.fixstatus))
             cmds.append(self.makecommandline("resolution", self.fixresolution))
         self.send_bug_modify_email(bugid, cmds, text, committer)
 
 class bugzilla(object):
     # supported versions of bugzilla. different versions have
     # different schemas.
     _versions = {
         '2.16': bzmysql,
         '2.18': bzmysql_2_18,
         '3.0':  bzmysql_3_0,
         'xmlrpc': bzxmlrpc,
         'xmlrpc+email': bzxmlrpcemail
         }
 
     _default_bug_re = (r'bugs?\s*,?\s*(?:#|nos?\.?|num(?:ber)?s?)?\s*'
                        r'(?P<ids>(?:\d+\s*(?:,?\s*(?:and)?)?\s*)+)'
                        r'\.?\s*(?:h(?:ours?)?\s*(?P<hours>\d*(?:\.\d+)?))?')
 
     _default_fix_re = (r'fix(?:es)?\s*(?:bugs?\s*)?,?\s*'
                        r'(?:nos?\.?|num(?:ber)?s?)?\s*'
                        r'(?P<ids>(?:#?\d+\s*(?:,?\s*(?:and)?)?\s*)+)'
                        r'\.?\s*(?:h(?:ours?)?\s*(?P<hours>\d*(?:\.\d+)?))?')
 
     def __init__(self, ui, repo):
         self.ui = ui
         self.repo = repo
 
         bzversion = self.ui.config('bugzilla', 'version')
         try:
             bzclass = bugzilla._versions[bzversion]
         except KeyError:
             raise error.Abort(_('bugzilla version %s not supported') %
                              bzversion)
         self.bzdriver = bzclass(self.ui)
 
         self.bug_re = re.compile(
             self.ui.config('bugzilla', 'regexp',
                            bugzilla._default_bug_re), re.IGNORECASE)
         self.fix_re = re.compile(
             self.ui.config('bugzilla', 'fixregexp',
                            bugzilla._default_fix_re), re.IGNORECASE)
         self.split_re = re.compile(r'\D+')
 
     def find_bugs(self, ctx):
         '''return bugs dictionary created from commit comment.
 
         Extract bug info from changeset comments. Filter out any that are
         not known to Bugzilla, and any that already have a reference to
         the given changeset in their comments.
         '''
         start = 0
         hours = 0.0
         bugs = {}
         bugmatch = self.bug_re.search(ctx.description(), start)
         fixmatch = self.fix_re.search(ctx.description(), start)
         while True:
             bugattribs = {}
             if not bugmatch and not fixmatch:
                 break
             if not bugmatch:
                 m = fixmatch
             elif not fixmatch:
                 m = bugmatch
             else:
                 if bugmatch.start() < fixmatch.start():
                     m = bugmatch
                 else:
                     m = fixmatch
             start = m.end()
             if m is bugmatch:
                 bugmatch = self.bug_re.search(ctx.description(), start)
                 if 'fix' in bugattribs:
                     del bugattribs['fix']
             else:
                 fixmatch = self.fix_re.search(ctx.description(), start)
                 bugattribs['fix'] = None
 
             try:
                 ids = m.group('ids')
             except IndexError:
                 ids = m.group(1)
             try:
                 hours = float(m.group('hours'))
                 bugattribs['hours'] = hours
             except IndexError:
                 pass
             except TypeError:
                 pass
             except ValueError:
                 self.ui.status(_("%s: invalid hours\n") % m.group('hours'))
 
             for id in self.split_re.split(ids):
                 if not id:
                     continue
                 bugs[int(id)] = bugattribs
         if bugs:
             self.bzdriver.filter_real_bug_ids(bugs)
         if bugs:
             self.bzdriver.filter_cset_known_bug_ids(ctx.node(), bugs)
         return bugs
 
     def update(self, bugid, newstate, ctx):
         '''update bugzilla bug with reference to changeset.'''
 
         def webroot(root):
             '''strip leading prefix of repo root and turn into
             url-safe path.'''
             count = int(self.ui.config('bugzilla', 'strip', 0))
             root = util.pconvert(root)
             while count > 0:
                 c = root.find('/')
                 if c == -1:
                     break
                 root = root[c + 1:]
                 count -= 1
             return root
 
         mapfile = None
         tmpl = self.ui.config('bugzilla', 'template')
         if not tmpl:
             mapfile = self.ui.config('bugzilla', 'style')
         if not mapfile and not tmpl:
             tmpl = _('changeset {node|short} in repo {root} refers '
                      'to bug {bug}.\ndetails:\n\t{desc|tabindent}')
         t = cmdutil.changeset_templater(self.ui, self.repo,
                                         False, None, tmpl, mapfile, False)
         self.ui.pushbuffer()
         t.show(ctx, changes=ctx.changeset(),
                bug=str(bugid),
                hgweb=self.ui.config('web', 'baseurl'),
                root=self.repo.root,
                webroot=webroot(self.repo.root))
         data = self.ui.popbuffer()
         self.bzdriver.updatebug(bugid, newstate, data, util.email(ctx.user()))
 
     def notify(self, bugs, committer):
         '''ensure Bugzilla users are notified of bug change.'''
         self.bzdriver.notify(bugs, committer)
 
 def hook(ui, repo, hooktype, node=None, **kwargs):
     '''add comment to bugzilla for each changeset that refers to a
     bugzilla bug id. only add a comment once per bug, so same change
     seen multiple times does not fill bug with duplicate data.'''
     if node is None:
         raise error.Abort(_('hook type %s does not pass a changeset id') %
                          hooktype)
     try:
         bz = bugzilla(ui, repo)
         ctx = repo[node]
         bugs = bz.find_bugs(ctx)
         if bugs:
             for bug in bugs:
                 bz.update(bug, bugs[bug], ctx)
             bz.notify(bugs, util.email(ctx.user()))
     except Exception as e:
         raise error.Abort(_('Bugzilla error: %s') % e)

mercurial/pycompat.py

@@ -1,131 +1,138 b''
 # pycompat.py - portability shim for python 3
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 """Mercurial portability shim for python 3.
 
 This contains aliases to hide python version-specific details from the core.
 """
 
 from __future__ import absolute_import
 
 try:
     import cPickle as pickle
     pickle.dumps
 except ImportError:
     import pickle
     pickle.dumps # silence pyflakes
 
 try:
+    import urlparse
+    urlparse.urlparse
+except ImportError:
+    import urllib.parse as urlparse
+    urlparse.urlparse
+
+try:
     import cStringIO as io
     stringio = io.StringIO
 except ImportError:
     import io
     stringio = io.StringIO
 
 try:
     import Queue as _queue
     _queue.Queue
 except ImportError:
     import queue as _queue
 empty = _queue.Empty
 queue = _queue.Queue
 
 class _pycompatstub(object):
     pass
 
 def _alias(alias, origin, items):
     """ populate a _pycompatstub
 
     copies items from origin to alias
     """
     def hgcase(item):
         return item.replace('_', '').lower()
     for item in items:
         try:
             setattr(alias, hgcase(item), getattr(origin, item))
         except AttributeError:
             pass
 
 urlreq = _pycompatstub()
 urlerr = _pycompatstub()
 try:
     import urllib2
     import urllib
     _alias(urlreq, urllib, (
         "addclosehook",
         "addinfourl",
         "ftpwrapper",
         "pathname2url",
         "quote",
         "splitattr",
         "splitpasswd",
         "splitport",
         "splituser",
         "unquote",
         "url2pathname",
         "urlencode",
         "urlencode",
     ))
     _alias(urlreq, urllib2, (
         "AbstractHTTPHandler",
         "BaseHandler",
         "build_opener",
         "FileHandler",
         "FTPHandler",
         "HTTPBasicAuthHandler",
         "HTTPDigestAuthHandler",
         "HTTPHandler",
         "HTTPPasswordMgrWithDefaultRealm",
         "HTTPSHandler",
         "install_opener",
         "ProxyHandler",
         "Request",
         "urlopen",
     ))
     _alias(urlerr, urllib2, (
         "HTTPError",
         "URLError",
     ))
 
 except ImportError:
     import urllib.request
     _alias(urlreq, urllib.request, (
         "AbstractHTTPHandler",
         "addclosehook",
         "addinfourl",
         "BaseHandler",
         "build_opener",
         "FileHandler",
         "FTPHandler",
         "ftpwrapper",
         "HTTPHandler",
         "HTTPSHandler",
         "install_opener",
         "pathname2url",
         "HTTPBasicAuthHandler",
         "HTTPDigestAuthHandler",
         "HTTPPasswordMgrWithDefaultRealm",
         "ProxyHandler",
         "quote",
         "Request",
         "splitattr",
         "splitpasswd",
         "splitport",
         "splituser",
         "unquote",
         "url2pathname",
         "urlopen",
     ))
     import urllib.error
     _alias(urlerr, urllib.error, (
         "HTTPError",
         "URLError",
     ))
 
 try:
     xrange
 except NameError:
     import builtins
     builtins.xrange = range

mercurial/util.py

@@ -1,2853 +1,2854 b''
 # util.py - Mercurial utility functions and platform specific implementations
 #
 #  Copyright 2005 K. Thananchayan <[e-mail]>
 #  Copyright 2005-2007 Matt Mackall <[e-mail]>
 #  Copyright 2006 Vadim Gelfer <[e-mail]>
 #
 # This software may be used and distributed according to the terms of the
 # GNU General Public License version 2 or any later version.
 
 """Mercurial utility functions and platform specific implementations.
 
 This contains helper routines that are independent of the SCM core and
 hide platform-specific details from the core.
 """
 
 from __future__ import absolute_import
 
 import bz2
 import calendar
 import collections
 import datetime
 import errno
 import gc
 import hashlib
 import imp
 import os
 import re as remod
 import shutil
 import signal
 import socket
 import subprocess
 import sys
 import tempfile
 import textwrap
 import time
 import traceback
 import zlib
 
 from . import (
     encoding,
     error,
     i18n,
     osutil,
     parsers,
     pycompat,
 )
 
 for attr in (
     'empty',
     'pickle',
     'queue',
     'urlerr',
+    'urlparse',
     # we do import urlreq, but we do it outside the loop
     #'urlreq',
     'stringio',
 ):
     globals()[attr] = getattr(pycompat, attr)
 
 # This line is to make pyflakes happy:
 urlreq = pycompat.urlreq
 
 if os.name == 'nt':
     from . import windows as platform
 else:
     from . import posix as platform
 
 _ = i18n._
 
 cachestat = platform.cachestat
 checkexec = platform.checkexec
 checklink = platform.checklink
 copymode = platform.copymode
 executablepath = platform.executablepath
 expandglobs = platform.expandglobs
 explainexit = platform.explainexit
 findexe = platform.findexe
 gethgcmd = platform.gethgcmd
 getuser = platform.getuser
 getpid = os.getpid
 groupmembers = platform.groupmembers
 groupname = platform.groupname
 hidewindow = platform.hidewindow
 isexec = platform.isexec
 isowner = platform.isowner
 localpath = platform.localpath
 lookupreg = platform.lookupreg
 makedir = platform.makedir
 nlinks = platform.nlinks
 normpath = platform.normpath
 normcase = platform.normcase
 normcasespec = platform.normcasespec
 normcasefallback = platform.normcasefallback
 openhardlinks = platform.openhardlinks
 oslink = platform.oslink
 parsepatchoutput = platform.parsepatchoutput
 pconvert = platform.pconvert
 poll = platform.poll
 popen = platform.popen
 posixfile = platform.posixfile
 quotecommand = platform.quotecommand
 readpipe = platform.readpipe
 rename = platform.rename
 removedirs = platform.removedirs
 samedevice = platform.samedevice
 samefile = platform.samefile
 samestat = platform.samestat
 setbinary = platform.setbinary
 setflags = platform.setflags
 setsignalhandler = platform.setsignalhandler
 shellquote = platform.shellquote
 spawndetached = platform.spawndetached
 split = platform.split
 sshargs = platform.sshargs
 statfiles = getattr(osutil, 'statfiles', platform.statfiles)
 statisexec = platform.statisexec
 statislink = platform.statislink
 termwidth = platform.termwidth
 testpid = platform.testpid
 umask = platform.umask
 unlink = platform.unlink
 unlinkpath = platform.unlinkpath
 username = platform.username
 
 # Python compatibility
 
 _notset = object()
 
 # disable Python's problematic floating point timestamps (issue4836)
 # (Python hypocritically says you shouldn't change this behavior in
 # libraries, and sure enough Mercurial is not a library.)
 os.stat_float_times(False)
 
 def safehasattr(thing, attr):
     return getattr(thing, attr, _notset) is not _notset
 
 DIGESTS = {
     'md5': hashlib.md5,
     'sha1': hashlib.sha1,
     'sha512': hashlib.sha512,
 }
 # List of digest types from strongest to weakest
 DIGESTS_BY_STRENGTH = ['sha512', 'sha1', 'md5']
 
 for k in DIGESTS_BY_STRENGTH:
     assert k in DIGESTS
 
 class digester(object):
     """helper to compute digests.
 
     This helper can be used to compute one or more digests given their name.
 
     >>> d = digester(['md5', 'sha1'])
     >>> d.update('foo')
     >>> [k for k in sorted(d)]
     ['md5', 'sha1']
     >>> d['md5']
     'acbd18db4cc2f85cedef654fccc4a4d8'
     >>> d['sha1']
     '0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33'
     >>> digester.preferred(['md5', 'sha1'])
     'sha1'
     """
 
     def __init__(self, digests, s=''):
         self._hashes = {}
         for k in digests:
             if k not in DIGESTS:
                 raise Abort(_('unknown digest type: %s') % k)
             self._hashes[k] = DIGESTS[k]()
         if s:
             self.update(s)
 
     def update(self, data):
         for h in self._hashes.values():
             h.update(data)
 
     def __getitem__(self, key):
         if key not in DIGESTS:
             raise Abort(_('unknown digest type: %s') % k)
         return self._hashes[key].hexdigest()
 
     def __iter__(self):
         return iter(self._hashes)
 
     @staticmethod
     def preferred(supported):
         """returns the strongest digest type in both supported and DIGESTS."""
 
         for k in DIGESTS_BY_STRENGTH:
             if k in supported:
                 return k
         return None
 
 class digestchecker(object):
     """file handle wrapper that additionally checks content against a given
     size and digests.
 
         d = digestchecker(fh, size, {'md5': '...'})
 
     When multiple digests are given, all of them are validated.
     """
 
     def __init__(self, fh, size, digests):
         self._fh = fh
         self._size = size
         self._got = 0
         self._digests = dict(digests)
         self._digester = digester(self._digests.keys())
 
     def read(self, length=-1):
         content = self._fh.read(length)
         self._digester.update(content)
         self._got += len(content)
         return content
 
     def validate(self):
         if self._size != self._got:
             raise Abort(_('size mismatch: expected %d, got %d') %
                 (self._size, self._got))
         for k, v in self._digests.items():
             if v != self._digester[k]:
                 # i18n: first parameter is a digest name
                 raise Abort(_('%s mismatch: expected %s, got %s') %
                     (k, v, self._digester[k]))
 
 try:
     buffer = buffer
 except NameError:
     if sys.version_info[0] < 3:
         def buffer(sliceable, offset=0):
             return sliceable[offset:]
     else:
         def buffer(sliceable, offset=0):
             return memoryview(sliceable)[offset:]
 
 closefds = os.name == 'posix'
 
 _chunksize = 4096
 
 class bufferedinputpipe(object):
     """a manually buffered input pipe
 
     Python will not let us use buffered IO and lazy reading with 'polling' at
     the same time. We cannot probe the buffer state and select will not detect
     that data are ready to read if they are already buffered.
 
     This class let us work around that by implementing its own buffering
     (allowing efficient readline) while offering a way to know if the buffer is
     empty from the output (allowing collaboration of the buffer with polling).
 
     This class lives in the 'util' module because it makes use of the 'os'
     module from the python stdlib.
     """
 
     def __init__(self, input):
         self._input = input
         self._buffer = []
         self._eof = False
         self._lenbuf = 0
 
     @property
     def hasbuffer(self):
         """True is any data is currently buffered
 
         This will be used externally a pre-step for polling IO. If there is
         already data then no polling should be set in place."""
         return bool(self._buffer)
 
     @property
     def closed(self):
         return self._input.closed
 
     def fileno(self):
         return self._input.fileno()
 
     def close(self):
         return self._input.close()
 
     def read(self, size):
         while (not self._eof) and (self._lenbuf < size):
             self._fillbuffer()
         return self._frombuffer(size)
 
     def readline(self, *args, **kwargs):
         if 1 < len(self._buffer):
             # this should not happen because both read and readline end with a
             # _frombuffer call that collapse it.
             self._buffer = [''.join(self._buffer)]
             self._lenbuf = len(self._buffer[0])
         lfi = -1
         if self._buffer:
             lfi = self._buffer[-1].find('\n')
         while (not self._eof) and lfi < 0:
             self._fillbuffer()
             if self._buffer:
                 lfi = self._buffer[-1].find('\n')
         size = lfi + 1
         if lfi < 0: # end of file
             size = self._lenbuf
         elif 1 < len(self._buffer):
             # we need to take previous chunks into account
             size += self._lenbuf - len(self._buffer[-1])
         return self._frombuffer(size)
 
     def _frombuffer(self, size):
         """return at most 'size' data from the buffer
 
         The data are removed from the buffer."""
         if size == 0 or not self._buffer:
             return ''
         buf = self._buffer[0]
         if 1 < len(self._buffer):
             buf = ''.join(self._buffer)
 
         data = buf[:size]
         buf = buf[len(data):]
         if buf:
             self._buffer = [buf]
             self._lenbuf = len(buf)
         else:
             self._buffer = []
             self._lenbuf = 0
         return data
 
     def _fillbuffer(self):
         """read data to the buffer"""
         data = os.read(self._input.fileno(), _chunksize)
         if not data:
             self._eof = True
         else:
             self._lenbuf += len(data)
             self._buffer.append(data)
 
 def popen2(cmd, env=None, newlines=False):
     # Setting bufsize to -1 lets the system decide the buffer size.
     # The default for bufsize is 0, meaning unbuffered. This leads to
     # poor performance on Mac OS X: http://bugs.python.org/issue4194
     p = subprocess.Popen(cmd, shell=True, bufsize=-1,
                          close_fds=closefds,
                          stdin=subprocess.PIPE, stdout=subprocess.PIPE,
                          universal_newlines=newlines,
                          env=env)
     return p.stdin, p.stdout
 
 def popen3(cmd, env=None, newlines=False):
     stdin, stdout, stderr, p = popen4(cmd, env, newlines)
     return stdin, stdout, stderr
 
 def popen4(cmd, env=None, newlines=False, bufsize=-1):
     p = subprocess.Popen(cmd, shell=True, bufsize=bufsize,
                          close_fds=closefds,
                          stdin=subprocess.PIPE, stdout=subprocess.PIPE,
                          stderr=subprocess.PIPE,
                          universal_newlines=newlines,
                          env=env)
     return p.stdin, p.stdout, p.stderr, p
 
 def version():
     """Return version information if available."""
     try:
         from . import __version__
         return __version__.version
     except ImportError:
         return 'unknown'
 
 def versiontuple(v=None, n=4):
     """Parses a Mercurial version string into an N-tuple.
 
     The version string to be parsed is specified with the ``v`` argument.
     If it isn't defined, the current Mercurial version string will be parsed.
 
     ``n`` can be 2, 3, or 4. Here is how some version strings map to
     returned values:
 
     >>> v = '3.6.1+190-df9b73d2d444'
     >>> versiontuple(v, 2)
     (3, 6)
     >>> versiontuple(v, 3)
     (3, 6, 1)
     >>> versiontuple(v, 4)
     (3, 6, 1, '190-df9b73d2d444')
 
     >>> versiontuple('3.6.1+190-df9b73d2d444+20151118')
     (3, 6, 1, '190-df9b73d2d444+20151118')
 
     >>> v = '3.6'
     >>> versiontuple(v, 2)
     (3, 6)
     >>> versiontuple(v, 3)
     (3, 6, None)
     >>> versiontuple(v, 4)
     (3, 6, None, None)
     """
     if not v:
         v = version()
     parts = v.split('+', 1)
     if len(parts) == 1:
         vparts, extra = parts[0], None
     else:
         vparts, extra = parts
 
     vints = []
     for i in vparts.split('.'):
         try:
             vints.append(int(i))
         except ValueError:
             break
     # (3, 6) -> (3, 6, None)
     while len(vints) < 3:
         vints.append(None)
 
     if n == 2:
         return (vints[0], vints[1])
     if n == 3:
         return (vints[0], vints[1], vints[2])
     if n == 4:
         return (vints[0], vints[1], vints[2], extra)
 
 # used by parsedate
 defaultdateformats = (
     '%Y-%m-%d %H:%M:%S',
     '%Y-%m-%d %I:%M:%S%p',
     '%Y-%m-%d %H:%M',
     '%Y-%m-%d %I:%M%p',
     '%Y-%m-%d',
     '%m-%d',
     '%m/%d',
     '%m/%d/%y',
     '%m/%d/%Y',
     '%a %b %d %H:%M:%S %Y',
     '%a %b %d %I:%M:%S%p %Y',
     '%a, %d %b %Y %H:%M:%S',        #  GNU coreutils "/bin/date --rfc-2822"
     '%b %d %H:%M:%S %Y',
     '%b %d %I:%M:%S%p %Y',
     '%b %d %H:%M:%S',
     '%b %d %I:%M:%S%p',
     '%b %d %H:%M',
     '%b %d %I:%M%p',
     '%b %d %Y',
     '%b %d',
     '%H:%M:%S',
     '%I:%M:%S%p',
     '%H:%M',
     '%I:%M%p',
 )
 
 extendeddateformats = defaultdateformats + (
     "%Y",
     "%Y-%m",
     "%b",
     "%b %Y",
     )
 
 def cachefunc(func):
     '''cache the result of function calls'''
     # XXX doesn't handle keywords args
     if func.__code__.co_argcount == 0:
         cache = []
         def f():
             if len(cache) == 0:
                 cache.append(func())
             return cache[0]
         return f
     cache = {}
     if func.__code__.co_argcount == 1:
         # we gain a small amount of time because
         # we don't need to pack/unpack the list
         def f(arg):
             if arg not in cache:
                 cache[arg] = func(arg)
             return cache[arg]
     else:
         def f(*args):
             if args not in cache:
                 cache[args] = func(*args)
             return cache[args]
 
     return f
 
 class sortdict(dict):
     '''a simple sorted dictionary'''
     def __init__(self, data=None):
         self._list = []
         if data:
             self.update(data)
     def copy(self):
         return sortdict(self)
     def __setitem__(self, key, val):
         if key in self:
             self._list.remove(key)
         self._list.append(key)
         dict.__setitem__(self, key, val)
     def __iter__(self):
         return self._list.__iter__()
     def update(self, src):
         if isinstance(src, dict):
             src = src.iteritems()
         for k, v in src:
             self[k] = v
     def clear(self):
         dict.clear(self)
         self._list = []
     def items(self):
         return [(k, self[k]) for k in self._list]
     def __delitem__(self, key):
         dict.__delitem__(self, key)
         self._list.remove(key)
     def pop(self, key, *args, **kwargs):
         dict.pop(self, key, *args, **kwargs)
         try:
             self._list.remove(key)
         except ValueError:
             pass
     def keys(self):
         return self._list
     def iterkeys(self):
         return self._list.__iter__()
     def iteritems(self):
         for k in self._list:
             yield k, self[k]
     def insert(self, index, key, val):
         self._list.insert(index, key)
         dict.__setitem__(self, key, val)
 
 class _lrucachenode(object):
     """A node in a doubly linked list.
 
     Holds a reference to nodes on either side as well as a key-value
     pair for the dictionary entry.
     """
     __slots__ = ('next', 'prev', 'key', 'value')
 
     def __init__(self):
         self.next = None
         self.prev = None
 
         self.key = _notset
         self.value = None
 
     def markempty(self):
         """Mark the node as emptied."""
         self.key = _notset
 
 class lrucachedict(object):
     """Dict that caches most recent accesses and sets.
 
     The dict consists of an actual backing dict - indexed by original
     key - and a doubly linked circular list defining the order of entries in
     the cache.
 
     The head node is the newest entry in the cache. If the cache is full,
     we recycle head.prev and make it the new head. Cache accesses result in
     the node being moved to before the existing head and being marked as the
     new head node.
     """
     def __init__(self, max):
         self._cache = {}
 
         self._head = head = _lrucachenode()
         head.prev = head
         head.next = head
         self._size = 1
         self._capacity = max
 
     def __len__(self):
         return len(self._cache)
 
     def __contains__(self, k):
         return k in self._cache
 
     def __iter__(self):
         # We don't have to iterate in cache order, but why not.
         n = self._head
         for i in range(len(self._cache)):
             yield n.key
             n = n.next
 
     def __getitem__(self, k):
         node = self._cache[k]
         self._movetohead(node)
         return node.value
 
     def __setitem__(self, k, v):
         node = self._cache.get(k)
         # Replace existing value and mark as newest.
         if node is not None:
             node.value = v
             self._movetohead(node)
             return
 
         if self._size < self._capacity:
             node = self._addcapacity()
         else:
             # Grab the last/oldest item.
             node = self._head.prev
 
         # At capacity. Kill the old entry.
         if node.key is not _notset:
             del self._cache[node.key]
 
         node.key = k
         node.value = v
         self._cache[k] = node
         # And mark it as newest entry. No need to adjust order since it
         # is already self._head.prev.
         self._head = node
 
     def __delitem__(self, k):
         node = self._cache.pop(k)
         node.markempty()
 
         # Temporarily mark as newest item before re-adjusting head to make
         # this node the oldest item.
         self._movetohead(node)
         self._head = node.next
 
     # Additional dict methods.
 
     def get(self, k, default=None):
         try:
             return self._cache[k]
         except KeyError:
             return default
 
     def clear(self):
         n = self._head
         while n.key is not _notset:
             n.markempty()
             n = n.next
 
         self._cache.clear()
 
     def copy(self):
         result = lrucachedict(self._capacity)
         n = self._head.prev
         # Iterate in oldest-to-newest order, so the copy has the right ordering
         for i in range(len(self._cache)):
             result[n.key] = n.value
             n = n.prev
         return result
 
     def _movetohead(self, node):
         """Mark a node as the newest, making it the new head.
 
         When a node is accessed, it becomes the freshest entry in the LRU
         list, which is denoted by self._head.
 
         Visually, let's make ``N`` the new head node (* denotes head):
 
             previous/oldest <-> head <-> next/next newest
 
             ----<->--- A* ---<->-----
             |                       |
             E <-> D <-> N <-> C <-> B
 
         To:
 
             ----<->--- N* ---<->-----
             |                       |
             E <-> D <-> C <-> B <-> A
 
         This requires the following moves:
 
            C.next = D  (node.prev.next = node.next)
            D.prev = C  (node.next.prev = node.prev)
            E.next = N  (head.prev.next = node)
            N.prev = E  (node.prev = head.prev)
            N.next = A  (node.next = head)
            A.prev = N  (head.prev = node)
         """
         head = self._head
         # C.next = D
         node.prev.next = node.next
         # D.prev = C
         node.next.prev = node.prev
         # N.prev = E
         node.prev = head.prev
         # N.next = A
         # It is tempting to do just "head" here, however if node is
         # adjacent to head, this will do bad things.
         node.next = head.prev.next
         # E.next = N
         node.next.prev = node
         # A.prev = N
         node.prev.next = node
 
         self._head = node
 
     def _addcapacity(self):
         """Add a node to the circular linked list.
 
         The new node is inserted before the head node.
         """
         head = self._head
         node = _lrucachenode()
         head.prev.next = node
         node.prev = head.prev
         node.next = head
         head.prev = node
         self._size += 1
         return node
 
 def lrucachefunc(func):
     '''cache most recent results of function calls'''
     cache = {}
     order = collections.deque()
     if func.__code__.co_argcount == 1:
         def f(arg):
             if arg not in cache:
                 if len(cache) > 20:
                     del cache[order.popleft()]
                 cache[arg] = func(arg)
             else:
                 order.remove(arg)
             order.append(arg)
             return cache[arg]
     else:
         def f(*args):
             if args not in cache:
                 if len(cache) > 20:
                     del cache[order.popleft()]
                 cache[args] = func(*args)
             else:
                 order.remove(args)
             order.append(args)
             return cache[args]
 
     return f
 
 class propertycache(object):
     def __init__(self, func):
         self.func = func
         self.name = func.__name__
     def __get__(self, obj, type=None):
         result = self.func(obj)
         self.cachevalue(obj, result)
         return result
 
     def cachevalue(self, obj, value):
         # __dict__ assignment required to bypass __setattr__ (eg: repoview)
         obj.__dict__[self.name] = value
 
 def pipefilter(s, cmd):
     '''filter string S through command CMD, returning its output'''
     p = subprocess.Popen(cmd, shell=True, close_fds=closefds,
                          stdin=subprocess.PIPE, stdout=subprocess.PIPE)
     pout, perr = p.communicate(s)
     return pout
 
 def tempfilter(s, cmd):
     '''filter string S through a pair of temporary files with CMD.
     CMD is used as a template to create the real command to be run,
     with the strings INFILE and OUTFILE replaced by the real names of
     the temporary files generated.'''
     inname, outname = None, None
     try:
         infd, inname = tempfile.mkstemp(prefix='hg-filter-in-')
         fp = os.fdopen(infd, 'wb')
         fp.write(s)
         fp.close()
         outfd, outname = tempfile.mkstemp(prefix='hg-filter-out-')
         os.close(outfd)
         cmd = cmd.replace('INFILE', inname)
         cmd = cmd.replace('OUTFILE', outname)
         code = os.system(cmd)
         if sys.platform == 'OpenVMS' and code & 1:
             code = 0
         if code:
             raise Abort(_("command '%s' failed: %s") %
                         (cmd, explainexit(code)))
         return readfile(outname)
     finally:
         try:
             if inname:
                 os.unlink(inname)
         except OSError:
             pass
         try:
             if outname:
                 os.unlink(outname)
         except OSError:
             pass
 
 filtertable = {
     'tempfile:': tempfilter,
     'pipe:': pipefilter,
     }
 
 def filter(s, cmd):
     "filter a string through a command that transforms its input to its output"
     for name, fn in filtertable.iteritems():
         if cmd.startswith(name):
             return fn(s, cmd[len(name):].lstrip())
     return pipefilter(s, cmd)
 
 def binary(s):
     """return true if a string is binary data"""
     return bool(s and '\0' in s)
 
 def increasingchunks(source, min=1024, max=65536):
     '''return no less than min bytes per chunk while data remains,
     doubling min after each chunk until it reaches max'''
     def log2(x):
         if not x:
             return 0
         i = 0
         while x:
             x >>= 1
             i += 1
         return i - 1
 
     buf = []
     blen = 0
     for chunk in source:
         buf.append(chunk)
         blen += len(chunk)
         if blen >= min:
             if min < max:
                 min = min << 1
                 nmin = 1 << log2(blen)
                 if nmin > min:
                     min = nmin
                 if min > max:
                     min = max
             yield ''.join(buf)
             blen = 0
             buf = []
     if buf:
         yield ''.join(buf)
 
 Abort = error.Abort
 
 def always(fn):
     return True
 
 def never(fn):
     return False
 
 def nogc(func):
     """disable garbage collector
 
     Python's garbage collector triggers a GC each time a certain number of
     container objects (the number being defined by gc.get_threshold()) are
     allocated even when marked not to be tracked by the collector. Tracking has
     no effect on when GCs are triggered, only on what objects the GC looks
     into. As a workaround, disable GC while building complex (huge)
     containers.
 
     This garbage collector issue have been fixed in 2.7.
     """
     def wrapper(*args, **kwargs):
         gcenabled = gc.isenabled()
         gc.disable()
         try:
             return func(*args, **kwargs)
         finally:
             if gcenabled:
                 gc.enable()
     return wrapper
 
 def pathto(root, n1, n2):
     '''return the relative path from one place to another.
     root should use os.sep to separate directories
     n1 should use os.sep to separate directories
     n2 should use "/" to separate directories
     returns an os.sep-separated path.
 
     If n1 is a relative path, it's assumed it's
     relative to root.
     n2 should always be relative to root.
     '''
     if not n1:
         return localpath(n2)
     if os.path.isabs(n1):
         if os.path.splitdrive(root)[0] != os.path.splitdrive(n1)[0]:
             return os.path.join(root, localpath(n2))
         n2 = '/'.join((pconvert(root), n2))
     a, b = splitpath(n1), n2.split('/')
     a.reverse()
     b.reverse()
     while a and b and a[-1] == b[-1]:
         a.pop()
         b.pop()
     b.reverse()
     return os.sep.join((['..'] * len(a)) + b) or '.'
 
 def mainfrozen():
     """return True if we are a frozen executable.
 
     The code supports py2exe (most common, Windows only) and tools/freeze
     (portable, not much used).
     """
     return (safehasattr(sys, "frozen") or # new py2exe
             safehasattr(sys, "importers") or # old py2exe
             imp.is_frozen("__main__")) # tools/freeze
 
 # the location of data files matching the source code
 if mainfrozen() and getattr(sys, 'frozen', None) != 'macosx_app':
     # executable version (py2exe) doesn't support __file__
     datapath = os.path.dirname(sys.executable)
 else:
     datapath = os.path.dirname(__file__)
 
 i18n.setdatapath(datapath)
 
 _hgexecutable = None
 
 def hgexecutable():
     """return location of the 'hg' executable.
 
     Defaults to $HG or 'hg' in the search path.
     """
     if _hgexecutable is None:
         hg = os.environ.get('HG')
         mainmod = sys.modules['__main__']
         if hg:
             _sethgexecutable(hg)
         elif mainfrozen():
             if getattr(sys, 'frozen', None) == 'macosx_app':
                 # Env variable set by py2app
                 _sethgexecutable(os.environ['EXECUTABLEPATH'])
             else:
                 _sethgexecutable(sys.executable)
         elif os.path.basename(getattr(mainmod, '__file__', '')) == 'hg':
             _sethgexecutable(mainmod.__file__)
         else:
             exe = findexe('hg') or os.path.basename(sys.argv[0])
             _sethgexecutable(exe)
     return _hgexecutable
 
 def _sethgexecutable(path):
     """set location of the 'hg' executable"""
     global _hgexecutable
     _hgexecutable = path
 
 def _isstdout(f):
     fileno = getattr(f, 'fileno', None)
     return fileno and fileno() == sys.__stdout__.fileno()
 
 def system(cmd, environ=None, cwd=None, onerr=None, errprefix=None, out=None):
     '''enhanced shell command execution.
     run with environment maybe modified, maybe in different dir.
 
     if command fails and onerr is None, return status, else raise onerr
     object as exception.
 
     if out is specified, it is assumed to be a file-like object that has a
     write() method. stdout and stderr will be redirected to out.'''
     if environ is None:
         environ = {}
     try:
         sys.stdout.flush()
     except Exception:
         pass
     def py2shell(val):
         'convert python object into string that is useful to shell'
         if val is None or val is False:
             return '0'
         if val is True:
             return '1'
         return str(val)
     origcmd = cmd
     cmd = quotecommand(cmd)
     if sys.platform == 'plan9' and (sys.version_info[0] == 2
                                     and sys.version_info[1] < 7):
         # subprocess kludge to work around issues in half-baked Python
         # ports, notably bichued/python:
         if not cwd is None:
             os.chdir(cwd)
         rc = os.system(cmd)
     else:
         env = dict(os.environ)
         env.update((k, py2shell(v)) for k, v in environ.iteritems())
         env['HG'] = hgexecutable()
         if out is None or _isstdout(out):
             rc = subprocess.call(cmd, shell=True, close_fds=closefds,
                                  env=env, cwd=cwd)
         else:
             proc = subprocess.Popen(cmd, shell=True, close_fds=closefds,
                                     env=env, cwd=cwd, stdout=subprocess.PIPE,
                                     stderr=subprocess.STDOUT)
             while True:
                 line = proc.stdout.readline()
                 if not line:
                     break
                 out.write(line)
             proc.wait()
             rc = proc.returncode
         if sys.platform == 'OpenVMS' and rc & 1:
             rc = 0
     if rc and onerr:
         errmsg = '%s %s' % (os.path.basename(origcmd.split(None, 1)[0]),
                             explainexit(rc)[0])
         if errprefix:
             errmsg = '%s: %s' % (errprefix, errmsg)
         raise onerr(errmsg)
     return rc
 
 def checksignature(func):
     '''wrap a function with code to check for calling errors'''
     def check(*args, **kwargs):
         try:
             return func(*args, **kwargs)
         except TypeError:
             if len(traceback.extract_tb(sys.exc_info()[2])) == 1:
                 raise error.SignatureError
             raise
 
     return check
 
 def copyfile(src, dest, hardlink=False, copystat=False, checkambig=False):
     '''copy a file, preserving mode and optionally other stat info like
     atime/mtime
 
     checkambig argument is used with filestat, and is useful only if
     destination file is guarded by any lock (e.g. repo.lock or
     repo.wlock).
 
     copystat and checkambig should be exclusive.
     '''
     assert not (copystat and checkambig)
     oldstat = None
     if os.path.lexists(dest):
         if checkambig:
             oldstat = checkambig and filestat(dest)
         unlink(dest)
     # hardlinks are problematic on CIFS, quietly ignore this flag
     # until we find a way to work around it cleanly (issue4546)
     if False and hardlink:
         try:
             oslink(src, dest)
             return
         except (IOError, OSError):
             pass # fall back to normal copy
     if os.path.islink(src):
         os.symlink(os.readlink(src), dest)
         # copytime is ignored for symlinks, but in general copytime isn't needed
         # for them anyway
     else:
         try:
             shutil.copyfile(src, dest)
             if copystat:
                 # copystat also copies mode
                 shutil.copystat(src, dest)
             else:
                 shutil.copymode(src, dest)
                 if oldstat and oldstat.stat:
                     newstat = filestat(dest)
                     if newstat.isambig(oldstat):
                         # stat of copied file is ambiguous to original one
                         advanced = (oldstat.stat.st_mtime + 1) & 0x7fffffff
                         os.utime(dest, (advanced, advanced))
         except shutil.Error as inst:
             raise Abort(str(inst))
 
 def copyfiles(src, dst, hardlink=None, progress=lambda t, pos: None):
     """Copy a directory tree using hardlinks if possible."""
     num = 0
 
     if hardlink is None:
         hardlink = (os.stat(src).st_dev ==
                     os.stat(os.path.dirname(dst)).st_dev)
     if hardlink:
         topic = _('linking')
     else:
         topic = _('copying')
 
     if os.path.isdir(src):
         os.mkdir(dst)
         for name, kind in osutil.listdir(src):
             srcname = os.path.join(src, name)
             dstname = os.path.join(dst, name)
             def nprog(t, pos):
                 if pos is not None:
                     return progress(t, pos + num)
             hardlink, n = copyfiles(srcname, dstname, hardlink, progress=nprog)
             num += n
     else:
         if hardlink:
             try:
                 oslink(src, dst)
             except (IOError, OSError):
                 hardlink = False
                 shutil.copy(src, dst)
         else:
             shutil.copy(src, dst)
         num += 1
         progress(topic, num)
     progress(topic, None)
 
     return hardlink, num
 
 _winreservednames = '''con prn aux nul
     com1 com2 com3 com4 com5 com6 com7 com8 com9
     lpt1 lpt2 lpt3 lpt4 lpt5 lpt6 lpt7 lpt8 lpt9'''.split()
 _winreservedchars = ':*?"<>|'
 def checkwinfilename(path):
     r'''Check that the base-relative path is a valid filename on Windows.
     Returns None if the path is ok, or a UI string describing the problem.
 
     >>> checkwinfilename("just/a/normal/path")
     >>> checkwinfilename("foo/bar/con.xml")
     "filename contains 'con', which is reserved on Windows"
     >>> checkwinfilename("foo/con.xml/bar")
     "filename contains 'con', which is reserved on Windows"
     >>> checkwinfilename("foo/bar/xml.con")
     >>> checkwinfilename("foo/bar/AUX/bla.txt")
     "filename contains 'AUX', which is reserved on Windows"
     >>> checkwinfilename("foo/bar/bla:.txt")
     "filename contains ':', which is reserved on Windows"
     >>> checkwinfilename("foo/bar/b\07la.txt")
     "filename contains '\\x07', which is invalid on Windows"
     >>> checkwinfilename("foo/bar/bla ")
     "filename ends with ' ', which is not allowed on Windows"
     >>> checkwinfilename("../bar")
     >>> checkwinfilename("foo\\")
     "filename ends with '\\', which is invalid on Windows"
     >>> checkwinfilename("foo\\/bar")
     "directory name ends with '\\', which is invalid on Windows"
     '''
     if path.endswith('\\'):
         return _("filename ends with '\\', which is invalid on Windows")
     if '\\/' in path:
         return _("directory name ends with '\\', which is invalid on Windows")
     for n in path.replace('\\', '/').split('/'):
         if not n:
             continue
         for c in n:
             if c in _winreservedchars:
                 return _("filename contains '%s', which is reserved "
                          "on Windows") % c
             if ord(c) <= 31:
                 return _("filename contains %r, which is invalid "
                          "on Windows") % c
         base = n.split('.')[0]
         if base and base.lower() in _winreservednames:
             return _("filename contains '%s', which is reserved "
                      "on Windows") % base
         t = n[-1]
         if t in '. ' and n not in '..':
             return _("filename ends with '%s', which is not allowed "
                      "on Windows") % t
 
 if os.name == 'nt':
     checkosfilename = checkwinfilename
 else:
     checkosfilename = platform.checkosfilename
 
 def makelock(info, pathname):
     try:
         return os.symlink(info, pathname)
     except OSError as why:
         if why.errno == errno.EEXIST:
             raise
     except AttributeError: # no symlink in os
         pass
 
     ld = os.open(pathname, os.O_CREAT | os.O_WRONLY | os.O_EXCL)
     os.write(ld, info)
     os.close(ld)
 
 def readlock(pathname):
     try:
         return os.readlink(pathname)
     except OSError as why:
         if why.errno not in (errno.EINVAL, errno.ENOSYS):
             raise
     except AttributeError: # no symlink in os
         pass
     fp = posixfile(pathname)
     r = fp.read()
     fp.close()
     return r
 
 def fstat(fp):
     '''stat file object that may not have fileno method.'''
     try:
         return os.fstat(fp.fileno())
     except AttributeError:
         return os.stat(fp.name)
 
 # File system features
 
 def checkcase(path):
     """
     Return true if the given path is on a case-sensitive filesystem
 
     Requires a path (like /foo/.hg) ending with a foldable final
     directory component.
     """
     s1 = os.lstat(path)
     d, b = os.path.split(path)
     b2 = b.upper()
     if b == b2:
         b2 = b.lower()
         if b == b2:
             return True # no evidence against case sensitivity
     p2 = os.path.join(d, b2)
     try:
         s2 = os.lstat(p2)
         if s2 == s1:
             return False
         return True
     except OSError:
         return True
 
 try:
     import re2
     _re2 = None
 except ImportError:
     _re2 = False
 
 class _re(object):
     def _checkre2(self):
         global _re2
         try:
             # check if match works, see issue3964
             _re2 = bool(re2.match(r'\[([^\[]+)\]', '[ui]'))
         except ImportError:
             _re2 = False
 
     def compile(self, pat, flags=0):
         '''Compile a regular expression, using re2 if possible
 
         For best performance, use only re2-compatible regexp features. The
         only flags from the re module that are re2-compatible are
         IGNORECASE and MULTILINE.'''
         if _re2 is None:
             self._checkre2()
         if _re2 and (flags & ~(remod.IGNORECASE | remod.MULTILINE)) == 0:
             if flags & remod.IGNORECASE:
                 pat = '(?i)' + pat
             if flags & remod.MULTILINE:
                 pat = '(?m)' + pat
             try:
                 return re2.compile(pat)
             except re2.error:
                 pass
         return remod.compile(pat, flags)
 
     @propertycache
     def escape(self):
         '''Return the version of escape corresponding to self.compile.
 
         This is imperfect because whether re2 or re is used for a particular
         function depends on the flags, etc, but it's the best we can do.
         '''
         global _re2
         if _re2 is None:
             self._checkre2()
         if _re2:
             return re2.escape
         else:
             return remod.escape
 
 re = _re()
 
 _fspathcache = {}
 def fspath(name, root):
     '''Get name in the case stored in the filesystem
 
     The name should be relative to root, and be normcase-ed for efficiency.
 
     Note that this function is unnecessary, and should not be
     called, for case-sensitive filesystems (simply because it's expensive).
 
     The root should be normcase-ed, too.
     '''
     def _makefspathcacheentry(dir):
         return dict((normcase(n), n) for n in os.listdir(dir))
 
     seps = os.sep
     if os.altsep:
         seps = seps + os.altsep
     # Protect backslashes. This gets silly very quickly.
     seps.replace('\\','\\\\')
     pattern = remod.compile(r'([^%s]+)|([%s]+)' % (seps, seps))
     dir = os.path.normpath(root)
     result = []
     for part, sep in pattern.findall(name):
         if sep:
             result.append(sep)
             continue
 
         if dir not in _fspathcache:
             _fspathcache[dir] = _makefspathcacheentry(dir)
         contents = _fspathcache[dir]
 
         found = contents.get(part)
         if not found:
             # retry "once per directory" per "dirstate.walk" which
             # may take place for each patches of "hg qpush", for example
             _fspathcache[dir] = contents = _makefspathcacheentry(dir)
             found = contents.get(part)
 
         result.append(found or part)
         dir = os.path.join(dir, part)
 
     return ''.join(result)
 
 def checknlink(testfile):
     '''check whether hardlink count reporting works properly'''
 
     # testfile may be open, so we need a separate file for checking to
     # work around issue2543 (or testfile may get lost on Samba shares)
     f1 = testfile + ".hgtmp1"
     if os.path.lexists(f1):
         return False
     try:
         posixfile(f1, 'w').close()
     except IOError:
         return False
 
     f2 = testfile + ".hgtmp2"
     fd = None
     try:
         oslink(f1, f2)
         # nlinks() may behave differently for files on Windows shares if
         # the file is open.
         fd = posixfile(f2)
         return nlinks(f2) > 1
     except OSError:
         return False
     finally:
         if fd is not None:
             fd.close()
         for f in (f1, f2):
             try:
                 os.unlink(f)
             except OSError:
                 pass
 
 def endswithsep(path):
     '''Check path ends with os.sep or os.altsep.'''
     return path.endswith(os.sep) or os.altsep and path.endswith(os.altsep)
 
 def splitpath(path):
     '''Split path by os.sep.
     Note that this function does not use os.altsep because this is
     an alternative of simple "xxx.split(os.sep)".
     It is recommended to use os.path.normpath() before using this
     function if need.'''
     return path.split(os.sep)
 
 def gui():
     '''Are we running in a GUI?'''
     if sys.platform == 'darwin':
         if 'SSH_CONNECTION' in os.environ:
             # handle SSH access to a box where the user is logged in
             return False
         elif getattr(osutil, 'isgui', None):
             # check if a CoreGraphics session is available
             return osutil.isgui()
         else:
             # pure build; use a safe default
             return True
     else:
         return os.name == "nt" or os.environ.get("DISPLAY")
 
 def mktempcopy(name, emptyok=False, createmode=None):
     """Create a temporary file with the same contents from name
 
     The permission bits are copied from the original file.
 
     If the temporary file is going to be truncated immediately, you
     can use emptyok=True as an optimization.
 
     Returns the name of the temporary file.
     """
     d, fn = os.path.split(name)
     fd, temp = tempfile.mkstemp(prefix='.%s-' % fn, dir=d)
     os.close(fd)
     # Temporary files are created with mode 0600, which is usually not
     # what we want.  If the original file already exists, just copy
     # its mode.  Otherwise, manually obey umask.
     copymode(name, temp, createmode)
     if emptyok:
         return temp
     try:
         try:
             ifp = posixfile(name, "rb")
         except IOError as inst:
             if inst.errno == errno.ENOENT:
                 return temp
             if not getattr(inst, 'filename', None):
                 inst.filename = name
             raise
         ofp = posixfile(temp, "wb")
         for chunk in filechunkiter(ifp):
             ofp.write(chunk)
         ifp.close()
         ofp.close()
     except: # re-raises
         try: os.unlink(temp)
         except OSError: pass
         raise
     return temp
 
 class filestat(object):
     """help to exactly detect change of a file
 
     'stat' attribute is result of 'os.stat()' if specified 'path'
     exists. Otherwise, it is None. This can avoid preparative
     'exists()' examination on client side of this class.
     """
     def __init__(self, path):
         try:
             self.stat = os.stat(path)
         except OSError as err:
             if err.errno != errno.ENOENT:
                 raise
             self.stat = None
 
     __hash__ = object.__hash__
 
     def __eq__(self, old):
         try:
             # if ambiguity between stat of new and old file is
             # avoided, comparision of size, ctime and mtime is enough
             # to exactly detect change of a file regardless of platform
             return (self.stat.st_size == old.stat.st_size and
                     self.stat.st_ctime == old.stat.st_ctime and
                     self.stat.st_mtime == old.stat.st_mtime)
         except AttributeError:
             return False
 
     def isambig(self, old):
         """Examine whether new (= self) stat is ambiguous against old one
 
         "S[N]" below means stat of a file at N-th change:
 
         - S[n-1].ctime  < S[n].ctime: can detect change of a file
         - S[n-1].ctime == S[n].ctime
           - S[n-1].ctime  < S[n].mtime: means natural advancing (*1)
           - S[n-1].ctime == S[n].mtime: is ambiguous (*2)
           - S[n-1].ctime  > S[n].mtime: never occurs naturally (don't care)
         - S[n-1].ctime  > S[n].ctime: never occurs naturally (don't care)
 
         Case (*2) above means that a file was changed twice or more at
         same time in sec (= S[n-1].ctime), and comparison of timestamp
         is ambiguous.
 
         Base idea to avoid such ambiguity is "advance mtime 1 sec, if
         timestamp is ambiguous".
 
         But advancing mtime only in case (*2) doesn't work as
         expected, because naturally advanced S[n].mtime in case (*1)
         might be equal to manually advanced S[n-1 or earlier].mtime.
 
         Therefore, all "S[n-1].ctime == S[n].ctime" cases should be
         treated as ambiguous regardless of mtime, to avoid overlooking
         by confliction between such mtime.
 
         Advancing mtime "if isambig(oldstat)" ensures "S[n-1].mtime !=
         S[n].mtime", even if size of a file isn't changed.
         """
         try:
             return (self.stat.st_ctime == old.stat.st_ctime)
         except AttributeError:
             return False
 
     def __ne__(self, other):
         return not self == other
 
 class atomictempfile(object):
     '''writable file object that atomically updates a file
 
     All writes will go to a temporary copy of the original file. Call
     close() when you are done writing, and atomictempfile will rename
     the temporary copy to the original name, making the changes
     visible. If the object is destroyed without being closed, all your
     writes are discarded.
 
     checkambig argument of constructor is used with filestat, and is
     useful only if target file is guarded by any lock (e.g. repo.lock
     or repo.wlock).
     '''
     def __init__(self, name, mode='w+b', createmode=None, checkambig=False):
         self.__name = name      # permanent name
         self._tempname = mktempcopy(name, emptyok=('w' in mode),
                                     createmode=createmode)
         self._fp = posixfile(self._tempname, mode)
         self._checkambig = checkambig
 
         # delegated methods
         self.read = self._fp.read
         self.write = self._fp.write
         self.seek = self._fp.seek
         self.tell = self._fp.tell
         self.fileno = self._fp.fileno
 
     def close(self):
         if not self._fp.closed:
             self._fp.close()
             filename = localpath(self.__name)
             oldstat = self._checkambig and filestat(filename)
             if oldstat and oldstat.stat:
                 rename(self._tempname, filename)
                 newstat = filestat(filename)
                 if newstat.isambig(oldstat):
                     # stat of changed file is ambiguous to original one
                     advanced = (oldstat.stat.st_mtime + 1) & 0x7fffffff
                     os.utime(filename, (advanced, advanced))
             else:
                 rename(self._tempname, filename)
 
     def discard(self):
         if not self._fp.closed:
             try:
                 os.unlink(self._tempname)
             except OSError:
                 pass
             self._fp.close()
 
     def __del__(self):
         if safehasattr(self, '_fp'): # constructor actually did something
             self.discard()
 
     def __enter__(self):
         return self
 
     def __exit__(self, exctype, excvalue, traceback):
         if exctype is not None:
             self.discard()
         else:
             self.close()
 
 def makedirs(name, mode=None, notindexed=False):
     """recursive directory creation with parent mode inheritance
 
     Newly created directories are marked as "not to be indexed by
     the content indexing service", if ``notindexed`` is specified
     for "write" mode access.
     """
     try:
         makedir(name, notindexed)
     except OSError as err:
         if err.errno == errno.EEXIST:
             return
         if err.errno != errno.ENOENT or not name:
             raise
         parent = os.path.dirname(os.path.abspath(name))
         if parent == name:
             raise
         makedirs(parent, mode, notindexed)
         try:
             makedir(name, notindexed)
         except OSError as err:
             # Catch EEXIST to handle races
             if err.errno == errno.EEXIST:
                 return
             raise
     if mode is not None:
         os.chmod(name, mode)
 
 def readfile(path):
     with open(path, 'rb') as fp:
         return fp.read()
 
 def writefile(path, text):
     with open(path, 'wb') as fp:
         fp.write(text)
 
 def appendfile(path, text):
     with open(path, 'ab') as fp:
         fp.write(text)
 
 class chunkbuffer(object):
     """Allow arbitrary sized chunks of data to be efficiently read from an
     iterator over chunks of arbitrary size."""
 
     def __init__(self, in_iter):
         """in_iter is the iterator that's iterating over the input chunks.
         targetsize is how big a buffer to try to maintain."""
         def splitbig(chunks):
             for chunk in chunks:
                 if len(chunk) > 2**20:
                     pos = 0
                     while pos < len(chunk):
                         end = pos + 2 ** 18
                         yield chunk[pos:end]
                         pos = end
                 else:
                     yield chunk
         self.iter = splitbig(in_iter)
         self._queue = collections.deque()
         self._chunkoffset = 0
 
     def read(self, l=None):
         """Read L bytes of data from the iterator of chunks of data.
         Returns less than L bytes if the iterator runs dry.
 
         If size parameter is omitted, read everything"""
         if l is None:
             return ''.join(self.iter)
 
         left = l
         buf = []
         queue = self._queue
         while left > 0:
             # refill the queue
             if not queue:
                 target = 2**18
                 for chunk in self.iter:
                     queue.append(chunk)
                     target -= len(chunk)
                     if target <= 0:
                         break
                 if not queue:
                     break
 
             # The easy way to do this would be to queue.popleft(), modify the
             # chunk (if necessary), then queue.appendleft(). However, for cases
             # where we read partial chunk content, this incurs 2 dequeue
             # mutations and creates a new str for the remaining chunk in the
             # queue. Our code below avoids this overhead.
 
             chunk = queue[0]
             chunkl = len(chunk)
             offset = self._chunkoffset
 
             # Use full chunk.
             if offset == 0 and left >= chunkl:
                 left -= chunkl
                 queue.popleft()
                 buf.append(chunk)
                 # self._chunkoffset remains at 0.
                 continue
 
             chunkremaining = chunkl - offset
 
             # Use all of unconsumed part of chunk.
             if left >= chunkremaining:
                 left -= chunkremaining
                 queue.popleft()
                 # offset == 0 is enabled by block above, so this won't merely
                 # copy via ``chunk[0:]``.
                 buf.append(chunk[offset:])
                 self._chunkoffset = 0
 
             # Partial chunk needed.
             else:
                 buf.append(chunk[offset:offset + left])
                 self._chunkoffset += left
                 left -= chunkremaining
 
         return ''.join(buf)
 
 def filechunkiter(f, size=65536, limit=None):
     """Create a generator that produces the data in the file size
     (default 65536) bytes at a time, up to optional limit (default is
     to read all data).  Chunks may be less than size bytes if the
     chunk is the last chunk in the file, or the file is a socket or
     some other type of file that sometimes reads less data than is
     requested."""
     assert size >= 0
     assert limit is None or limit >= 0
     while True:
         if limit is None:
             nbytes = size
         else:
             nbytes = min(limit, size)
         s = nbytes and f.read(nbytes)
         if not s:
             break
         if limit:
             limit -= len(s)
         yield s
 
 def makedate(timestamp=None):
     '''Return a unix timestamp (or the current time) as a (unixtime,
     offset) tuple based off the local timezone.'''
     if timestamp is None:
         timestamp = time.time()
     if timestamp < 0:
         hint = _("check your clock")
         raise Abort(_("negative timestamp: %d") % timestamp, hint=hint)
     delta = (datetime.datetime.utcfromtimestamp(timestamp) -
              datetime.datetime.fromtimestamp(timestamp))
     tz = delta.days * 86400 + delta.seconds
     return timestamp, tz
 
 def datestr(date=None, format='%a %b %d %H:%M:%S %Y %1%2'):
     """represent a (unixtime, offset) tuple as a localized time.
     unixtime is seconds since the epoch, and offset is the time zone's
     number of seconds away from UTC.
 
     >>> datestr((0, 0))
     'Thu Jan 01 00:00:00 1970 +0000'
     >>> datestr((42, 0))
     'Thu Jan 01 00:00:42 1970 +0000'
     >>> datestr((-42, 0))
     'Wed Dec 31 23:59:18 1969 +0000'
     >>> datestr((0x7fffffff, 0))
     'Tue Jan 19 03:14:07 2038 +0000'
     >>> datestr((-0x80000000, 0))
     'Fri Dec 13 20:45:52 1901 +0000'
     """
     t, tz = date or makedate()
     if "%1" in format or "%2" in format or "%z" in format:
         sign = (tz > 0) and "-" or "+"
         minutes = abs(tz) // 60
         q, r = divmod(minutes, 60)
         format = format.replace("%z", "%1%2")
         format = format.replace("%1", "%c%02d" % (sign, q))
         format = format.replace("%2", "%02d" % r)
     d = t - tz
     if d > 0x7fffffff:
         d = 0x7fffffff
     elif d < -0x80000000:
         d = -0x80000000
     # Never use time.gmtime() and datetime.datetime.fromtimestamp()
     # because they use the gmtime() system call which is buggy on Windows
     # for negative values.
     t = datetime.datetime(1970, 1, 1) + datetime.timedelta(seconds=d)
     s = t.strftime(format)
     return s
 
 def shortdate(date=None):
     """turn (timestamp, tzoff) tuple into iso 8631 date."""
     return datestr(date, format='%Y-%m-%d')
 
 def parsetimezone(tz):
     """parse a timezone string and return an offset integer"""
     if tz[0] in "+-" and len(tz) == 5 and tz[1:].isdigit():
         sign = (tz[0] == "+") and 1 or -1
         hours = int(tz[1:3])
         minutes = int(tz[3:5])
         return -sign * (hours * 60 + minutes) * 60
     if tz == "GMT" or tz == "UTC":
         return 0
     return None
 
 def strdate(string, format, defaults=[]):
     """parse a localized time string and return a (unixtime, offset) tuple.
     if the string cannot be parsed, ValueError is raised."""
     # NOTE: unixtime = localunixtime + offset
     offset, date = parsetimezone(string.split()[-1]), string
     if offset is not None:
         date = " ".join(string.split()[:-1])
 
     # add missing elements from defaults
     usenow = False # default to using biased defaults
     for part in ("S", "M", "HI", "d", "mb", "yY"): # decreasing specificity
         found = [True for p in part if ("%"+p) in format]
         if not found:
             date += "@" + defaults[part][usenow]
             format += "@%" + part[0]
         else:
             # We've found a specific time element, less specific time
             # elements are relative to today
             usenow = True
 
     timetuple = time.strptime(date, format)
     localunixtime = int(calendar.timegm(timetuple))
     if offset is None:
         # local timezone
         unixtime = int(time.mktime(timetuple))
         offset = unixtime - localunixtime
     else:
         unixtime = localunixtime + offset
     return unixtime, offset
 
 def parsedate(date, formats=None, bias=None):
     """parse a localized date/time and return a (unixtime, offset) tuple.
 
     The date may be a "unixtime offset" string or in one of the specified
     formats. If the date already is a (unixtime, offset) tuple, it is returned.
 
     >>> parsedate(' today ') == parsedate(\
                                   datetime.date.today().strftime('%b %d'))
     True
     >>> parsedate( 'yesterday ') == parsedate((datetime.date.today() -\
                                                datetime.timedelta(days=1)\
                                               ).strftime('%b %d'))
     True
     >>> now, tz = makedate()
     >>> strnow, strtz = parsedate('now')
     >>> (strnow - now) < 1
     True
     >>> tz == strtz
     True
     """
     if bias is None:
         bias = {}
     if not date:
         return 0, 0
     if isinstance(date, tuple) and len(date) == 2:
         return date
     if not formats:
         formats = defaultdateformats
     date = date.strip()
 
     if date == 'now' or date == _('now'):
         return makedate()
     if date == 'today' or date == _('today'):
         date = datetime.date.today().strftime('%b %d')
     elif date == 'yesterday' or date == _('yesterday'):
         date = (datetime.date.today() -
                 datetime.timedelta(days=1)).strftime('%b %d')
 
     try:
         when, offset = map(int, date.split(' '))
     except ValueError:
         # fill out defaults
         now = makedate()
         defaults = {}
         for part in ("d", "mb", "yY", "HI", "M", "S"):
             # this piece is for rounding the specific end of unknowns
             b = bias.get(part)
             if b is None:
                 if part[0] in "HMS":
                     b = "00"
                 else:
                     b = "0"
 
             # this piece is for matching the generic end to today's date
             n = datestr(now, "%" + part[0])
 
             defaults[part] = (b, n)
 
         for format in formats:
             try:
                 when, offset = strdate(date, format, defaults)
             except (ValueError, OverflowError):
                 pass
             else:
                 break
         else:
             raise Abort(_('invalid date: %r') % date)
     # validate explicit (probably user-specified) date and
     # time zone offset. values must fit in signed 32 bits for
     # current 32-bit linux runtimes. timezones go from UTC-12
     # to UTC+14
     if when < -0x80000000 or when > 0x7fffffff:
         raise Abort(_('date exceeds 32 bits: %d') % when)
     if offset < -50400 or offset > 43200:
         raise Abort(_('impossible time zone offset: %d') % offset)
     return when, offset
 
 def matchdate(date):
     """Return a function that matches a given date match specifier
 
     Formats include:
 
     '{date}' match a given date to the accuracy provided
 
     '<{date}' on or before a given date
 
     '>{date}' on or after a given date
 
     >>> p1 = parsedate("10:29:59")
     >>> p2 = parsedate("10:30:00")
     >>> p3 = parsedate("10:30:59")
     >>> p4 = parsedate("10:31:00")
     >>> p5 = parsedate("Sep 15 10:30:00 1999")
     >>> f = matchdate("10:30")
     >>> f(p1[0])
     False
     >>> f(p2[0])
     True
     >>> f(p3[0])
     True
     >>> f(p4[0])
     False
     >>> f(p5[0])
     False
     """
 
     def lower(date):
         d = {'mb': "1", 'd': "1"}
         return parsedate(date, extendeddateformats, d)[0]
 
     def upper(date):
         d = {'mb': "12", 'HI': "23", 'M': "59", 'S': "59"}
         for days in ("31", "30", "29"):
             try:
                 d["d"] = days
                 return parsedate(date, extendeddateformats, d)[0]
             except Abort:
                 pass
         d["d"] = "28"
         return parsedate(date, extendeddateformats, d)[0]
 
     date = date.strip()
 
     if not date:
         raise Abort(_("dates cannot consist entirely of whitespace"))
     elif date[0] == "<":
         if not date[1:]:
             raise Abort(_("invalid day spec, use '<DATE'"))
         when = upper(date[1:])
         return lambda x: x <= when
     elif date[0] == ">":
         if not date[1:]:
             raise Abort(_("invalid day spec, use '>DATE'"))
         when = lower(date[1:])
         return lambda x: x >= when
     elif date[0] == "-":
         try:
             days = int(date[1:])
         except ValueError:
             raise Abort(_("invalid day spec: %s") % date[1:])
         if days < 0:
             raise Abort(_('%s must be nonnegative (see "hg help dates")')
                 % date[1:])
         when = makedate()[0] - days * 3600 * 24
         return lambda x: x >= when
     elif " to " in date:
         a, b = date.split(" to ")
         start, stop = lower(a), upper(b)
         return lambda x: x >= start and x <= stop
     else:
         start, stop = lower(date), upper(date)
         return lambda x: x >= start and x <= stop
 
 def stringmatcher(pattern):
     """
     accepts a string, possibly starting with 're:' or 'literal:' prefix.
     returns the matcher name, pattern, and matcher function.
     missing or unknown prefixes are treated as literal matches.
 
     helper for tests:
     >>> def test(pattern, *tests):
     ...     kind, pattern, matcher = stringmatcher(pattern)
     ...     return (kind, pattern, [bool(matcher(t)) for t in tests])
 
     exact matching (no prefix):
     >>> test('abcdefg', 'abc', 'def', 'abcdefg')
     ('literal', 'abcdefg', [False, False, True])
 
     regex matching ('re:' prefix)
     >>> test('re:a.+b', 'nomatch', 'fooadef', 'fooadefbar')
     ('re', 'a.+b', [False, False, True])
 
     force exact matches ('literal:' prefix)
     >>> test('literal:re:foobar', 'foobar', 're:foobar')
     ('literal', 're:foobar', [False, True])
 
     unknown prefixes are ignored and treated as literals
     >>> test('foo:bar', 'foo', 'bar', 'foo:bar')
     ('literal', 'foo:bar', [False, False, True])
     """
     if pattern.startswith('re:'):
         pattern = pattern[3:]
         try:
             regex = remod.compile(pattern)
         except remod.error as e:
             raise error.ParseError(_('invalid regular expression: %s')
                                    % e)
         return 're', pattern, regex.search
     elif pattern.startswith('literal:'):
         pattern = pattern[8:]
     return 'literal', pattern, pattern.__eq__
 
 def shortuser(user):
     """Return a short representation of a user name or email address."""
     f = user.find('@')
     if f >= 0:
         user = user[:f]
     f = user.find('<')
     if f >= 0:
         user = user[f + 1:]
     f = user.find(' ')
     if f >= 0:
         user = user[:f]
     f = user.find('.')
     if f >= 0:
         user = user[:f]
     return user
 
 def emailuser(user):
     """Return the user portion of an email address."""
     f = user.find('@')
     if f >= 0:
         user = user[:f]
     f = user.find('<')
     if f >= 0:
         user = user[f + 1:]
     return user
 
 def email(author):
     '''get email of author.'''
     r = author.find('>')
     if r == -1:
         r = None
     return author[author.find('<') + 1:r]
 
 def ellipsis(text, maxlength=400):
     """Trim string to at most maxlength (default: 400) columns in display."""
     return encoding.trim(text, maxlength, ellipsis='...')
 
 def unitcountfn(*unittable):
     '''return a function that renders a readable count of some quantity'''
 
     def go(count):
         for multiplier, divisor, format in unittable:
             if count >= divisor * multiplier:
                 return format % (count / float(divisor))
         return unittable[-1][2] % count
 
     return go
 
 bytecount = unitcountfn(
     (100, 1 << 30, _('%.0f GB')),
     (10, 1 << 30, _('%.1f GB')),
     (1, 1 << 30, _('%.2f GB')),
     (100, 1 << 20, _('%.0f MB')),
     (10, 1 << 20, _('%.1f MB')),
     (1, 1 << 20, _('%.2f MB')),
     (100, 1 << 10, _('%.0f KB')),
     (10, 1 << 10, _('%.1f KB')),
     (1, 1 << 10, _('%.2f KB')),
     (1, 1, _('%.0f bytes')),
     )
 
 def uirepr(s):
     # Avoid double backslash in Windows path repr()
     return repr(s).replace('\\\\', '\\')
 
 # delay import of textwrap
 def MBTextWrapper(**kwargs):
     class tw(textwrap.TextWrapper):
         """
         Extend TextWrapper for width-awareness.
 
         Neither number of 'bytes' in any encoding nor 'characters' is
         appropriate to calculate terminal columns for specified string.
 
         Original TextWrapper implementation uses built-in 'len()' directly,
         so overriding is needed to use width information of each characters.
 
         In addition, characters classified into 'ambiguous' width are
         treated as wide in East Asian area, but as narrow in other.
 
         This requires use decision to determine width of such characters.
         """
         def _cutdown(self, ucstr, space_left):
             l = 0
             colwidth = encoding.ucolwidth
             for i in xrange(len(ucstr)):
                 l += colwidth(ucstr[i])
                 if space_left < l:
                     return (ucstr[:i], ucstr[i:])
             return ucstr, ''
 
         # overriding of base class
         def _handle_long_word(self, reversed_chunks, cur_line, cur_len, width):
             space_left = max(width - cur_len, 1)
 
             if self.break_long_words:
                 cut, res = self._cutdown(reversed_chunks[-1], space_left)
                 cur_line.append(cut)
                 reversed_chunks[-1] = res
             elif not cur_line:
                 cur_line.append(reversed_chunks.pop())
 
         # this overriding code is imported from TextWrapper of Python 2.6
         # to calculate columns of string by 'encoding.ucolwidth()'
         def _wrap_chunks(self, chunks):
             colwidth = encoding.ucolwidth
 
             lines = []
             if self.width <= 0:
                 raise ValueError("invalid width %r (must be > 0)" % self.width)
 
             # Arrange in reverse order so items can be efficiently popped
             # from a stack of chucks.
             chunks.reverse()
 
             while chunks:
 
                 # Start the list of chunks that will make up the current line.
                 # cur_len is just the length of all the chunks in cur_line.
                 cur_line = []
                 cur_len = 0
 
                 # Figure out which static string will prefix this line.
                 if lines:
                     indent = self.subsequent_indent
                 else:
                     indent = self.initial_indent
 
                 # Maximum width for this line.
                 width = self.width - len(indent)
 
                 # First chunk on line is whitespace -- drop it, unless this
                 # is the very beginning of the text (i.e. no lines started yet).
                 if self.drop_whitespace and chunks[-1].strip() == '' and lines:
                     del chunks[-1]
 
                 while chunks:
                     l = colwidth(chunks[-1])
 
                     # Can at least squeeze this chunk onto the current line.
                     if cur_len + l <= width:
                         cur_line.append(chunks.pop())
                         cur_len += l
 
                     # Nope, this line is full.
                     else:
                         break
 
                 # The current line is full, and the next chunk is too big to
                 # fit on *any* line (not just this one).
                 if chunks and colwidth(chunks[-1]) > width:
                     self._handle_long_word(chunks, cur_line, cur_len, width)
 
                 # If the last chunk on this line is all whitespace, drop it.
                 if (self.drop_whitespace and
                     cur_line and cur_line[-1].strip() == ''):
                     del cur_line[-1]
 
                 # Convert current line back to a string and store it in list
                 # of all lines (return value).
                 if cur_line:
                     lines.append(indent + ''.join(cur_line))
 
             return lines
 
     global MBTextWrapper
     MBTextWrapper = tw
     return tw(**kwargs)
 
 def wrap(line, width, initindent='', hangindent=''):
     maxindent = max(len(hangindent), len(initindent))
     if width <= maxindent:
         # adjust for weird terminal size
         width = max(78, maxindent + 1)
     line = line.decode(encoding.encoding, encoding.encodingmode)
     initindent = initindent.decode(encoding.encoding, encoding.encodingmode)
     hangindent = hangindent.decode(encoding.encoding, encoding.encodingmode)
     wrapper = MBTextWrapper(width=width,
                             initial_indent=initindent,
                             subsequent_indent=hangindent)
     return wrapper.fill(line).encode(encoding.encoding)
 
 def iterlines(iterator):
     for chunk in iterator:
         for line in chunk.splitlines():
             yield line
 
 def expandpath(path):
     return os.path.expanduser(os.path.expandvars(path))
 
 def hgcmd():
     """Return the command used to execute current hg
 
     This is different from hgexecutable() because on Windows we want
     to avoid things opening new shell windows like batch files, so we
     get either the python call or current executable.
     """
     if mainfrozen():
         if getattr(sys, 'frozen', None) == 'macosx_app':
             # Env variable set by py2app
             return [os.environ['EXECUTABLEPATH']]
         else:
             return [sys.executable]
     return gethgcmd()
 
 def rundetached(args, condfn):
     """Execute the argument list in a detached process.
 
     condfn is a callable which is called repeatedly and should return
     True once the child process is known to have started successfully.
     At this point, the child process PID is returned. If the child
     process fails to start or finishes before condfn() evaluates to
     True, return -1.
     """
     # Windows case is easier because the child process is either
     # successfully starting and validating the condition or exiting
     # on failure. We just poll on its PID. On Unix, if the child
     # process fails to start, it will be left in a zombie state until
     # the parent wait on it, which we cannot do since we expect a long
     # running process on success. Instead we listen for SIGCHLD telling
     # us our child process terminated.
     terminated = set()
     def handler(signum, frame):
         terminated.add(os.wait())
     prevhandler = None
     SIGCHLD = getattr(signal, 'SIGCHLD', None)
     if SIGCHLD is not None:
         prevhandler = signal.signal(SIGCHLD, handler)
     try:
         pid = spawndetached(args)
         while not condfn():
             if ((pid in terminated or not testpid(pid))
                 and not condfn()):
                 return -1
             time.sleep(0.1)
         return pid
     finally:
         if prevhandler is not None:
             signal.signal(signal.SIGCHLD, prevhandler)
 
 def interpolate(prefix, mapping, s, fn=None, escape_prefix=False):
     """Return the result of interpolating items in the mapping into string s.
 
     prefix is a single character string, or a two character string with
     a backslash as the first character if the prefix needs to be escaped in
     a regular expression.
 
     fn is an optional function that will be applied to the replacement text
     just before replacement.
 
     escape_prefix is an optional flag that allows using doubled prefix for
     its escaping.
     """
     fn = fn or (lambda s: s)
     patterns = '|'.join(mapping.keys())
     if escape_prefix:
         patterns += '|' + prefix
         if len(prefix) > 1:
             prefix_char = prefix[1:]
         else:
             prefix_char = prefix
         mapping[prefix_char] = prefix_char
     r = remod.compile(r'%s(%s)' % (prefix, patterns))
     return r.sub(lambda x: fn(mapping[x.group()[1:]]), s)
 
 def getport(port):
     """Return the port for a given network service.
 
     If port is an integer, it's returned as is. If it's a string, it's
     looked up using socket.getservbyname(). If there's no matching
     service, error.Abort is raised.
     """
     try:
         return int(port)
     except ValueError:
         pass
 
     try:
         return socket.getservbyname(port)
     except socket.error:
         raise Abort(_("no port number associated with service '%s'") % port)
 
 _booleans = {'1': True, 'yes': True, 'true': True, 'on': True, 'always': True,
              '0': False, 'no': False, 'false': False, 'off': False,
              'never': False}
 
 def parsebool(s):
     """Parse s into a boolean.
 
     If s is not a valid boolean, returns None.
     """
     return _booleans.get(s.lower(), None)
 
 _hexdig = '0123456789ABCDEFabcdef'
 _hextochr = dict((a + b, chr(int(a + b, 16)))
                  for a in _hexdig for b in _hexdig)
 
 def _urlunquote(s):
     """Decode HTTP/HTML % encoding.
 
     >>> _urlunquote('abc%20def')
     'abc def'
     """
     res = s.split('%')
     # fastpath
     if len(res) == 1:
         return s
     s = res[0]
     for item in res[1:]:
         try:
             s += _hextochr[item[:2]] + item[2:]
         except KeyError:
             s += '%' + item
         except UnicodeDecodeError:
             s += unichr(int(item[:2], 16)) + item[2:]
     return s
 
 class url(object):
     r"""Reliable URL parser.
 
     This parses URLs and provides attributes for the following
     components:
 
     <scheme>://<user>:<passwd>@<host>:<port>/<path>?<query>#<fragment>
 
     Missing components are set to None. The only exception is
     fragment, which is set to '' if present but empty.
 
     If parsefragment is False, fragment is included in query. If
     parsequery is False, query is included in path. If both are
     False, both fragment and query are included in path.
 
     See http://www.ietf.org/rfc/rfc2396.txt for more information.
 
     Note that for backward compatibility reasons, bundle URLs do not
     take host names. That means 'bundle://../' has a path of '../'.
 
     Examples:
 
     >>> url('http://www.ietf.org/rfc/rfc2396.txt')
     <url scheme: 'http', host: 'www.ietf.org', path: 'rfc/rfc2396.txt'>
     >>> url('ssh://[::1]:2200//home/joe/repo')
     <url scheme: 'ssh', host: '[::1]', port: '2200', path: '/home/joe/repo'>
     >>> url('file:///home/joe/repo')
     <url scheme: 'file', path: '/home/joe/repo'>
     >>> url('file:///c:/temp/foo/')
     <url scheme: 'file', path: 'c:/temp/foo/'>
     >>> url('bundle:foo')
     <url scheme: 'bundle', path: 'foo'>
     >>> url('bundle://../foo')
     <url scheme: 'bundle', path: '../foo'>
     >>> url(r'c:\foo\bar')
     <url path: 'c:\\foo\\bar'>
     >>> url(r'\\blah\blah\blah')
     <url path: '\\\\blah\\blah\\blah'>
     >>> url(r'\\blah\blah\blah#baz')
     <url path: '\\\\blah\\blah\\blah', fragment: 'baz'>
     >>> url(r'file:///C:\users\me')
     <url scheme: 'file', path: 'C:\\users\\me'>
 
     Authentication credentials:
 
     >>> url('ssh://joe:xyz@x/repo')
     <url scheme: 'ssh', user: 'joe', passwd: 'xyz', host: 'x', path: 'repo'>
     >>> url('ssh://joe@x/repo')
     <url scheme: 'ssh', user: 'joe', host: 'x', path: 'repo'>
 
     Query strings and fragments:
 
     >>> url('http://host/a?b#c')
     <url scheme: 'http', host: 'host', path: 'a', query: 'b', fragment: 'c'>
     >>> url('http://host/a?b#c', parsequery=False, parsefragment=False)
     <url scheme: 'http', host: 'host', path: 'a?b#c'>
     """
 
     _safechars = "!~*'()+"
     _safepchars = "/!~*'()+:\\"
     _matchscheme = remod.compile(r'^[a-zA-Z0-9+.\-]+:').match
 
     def __init__(self, path, parsequery=True, parsefragment=True):
         # We slowly chomp away at path until we have only the path left
         self.scheme = self.user = self.passwd = self.host = None
         self.port = self.path = self.query = self.fragment = None
         self._localpath = True
         self._hostport = ''
         self._origpath = path
 
         if parsefragment and '#' in path:
             path, self.fragment = path.split('#', 1)
             if not path:
                 path = None
 
         # special case for Windows drive letters and UNC paths
         if hasdriveletter(path) or path.startswith(r'\\'):
             self.path = path
             return
 
         # For compatibility reasons, we can't handle bundle paths as
         # normal URLS
         if path.startswith('bundle:'):
             self.scheme = 'bundle'
             path = path[7:]
             if path.startswith('//'):
                 path = path[2:]
             self.path = path
             return
 
         if self._matchscheme(path):
             parts = path.split(':', 1)
             if parts[0]:
                 self.scheme, path = parts
                 self._localpath = False
 
         if not path:
             path = None
             if self._localpath:
                 self.path = ''
                 return
         else:
             if self._localpath:
                 self.path = path
                 return
 
             if parsequery and '?' in path:
                 path, self.query = path.split('?', 1)
                 if not path:
                     path = None
                 if not self.query:
                     self.query = None
 
             # // is required to specify a host/authority
             if path and path.startswith('//'):
                 parts = path[2:].split('/', 1)
                 if len(parts) > 1:
                     self.host, path = parts
                 else:
                     self.host = parts[0]
                     path = None
                 if not self.host:
                     self.host = None
                     # path of file:///d is /d
                     # path of file:///d:/ is d:/, not /d:/
                     if path and not hasdriveletter(path):
                         path = '/' + path
 
             if self.host and '@' in self.host:
                 self.user, self.host = self.host.rsplit('@', 1)
                 if ':' in self.user:
                     self.user, self.passwd = self.user.split(':', 1)
                 if not self.host:
                     self.host = None
 
             # Don't split on colons in IPv6 addresses without ports
             if (self.host and ':' in self.host and
                 not (self.host.startswith('[') and self.host.endswith(']'))):
                 self._hostport = self.host
                 self.host, self.port = self.host.rsplit(':', 1)
                 if not self.host:
                     self.host = None
 
             if (self.host and self.scheme == 'file' and
                 self.host not in ('localhost', '127.0.0.1', '[::1]')):
                 raise Abort(_('file:// URLs can only refer to localhost'))
 
         self.path = path
 
         # leave the query string escaped
         for a in ('user', 'passwd', 'host', 'port',
                   'path', 'fragment'):
             v = getattr(self, a)
             if v is not None:
                 setattr(self, a, _urlunquote(v))
 
     def __repr__(self):
         attrs = []
         for a in ('scheme', 'user', 'passwd', 'host', 'port', 'path',
                   'query', 'fragment'):
             v = getattr(self, a)
             if v is not None:
                 attrs.append('%s: %r' % (a, v))
         return '<url %s>' % ', '.join(attrs)
 
     def __str__(self):
         r"""Join the URL's components back into a URL string.
 
         Examples:
 
         >>> str(url('http://user:pw@host:80/c:/bob?fo:oo#ba:ar'))
         'http://user:pw@host:80/c:/bob?fo:oo#ba:ar'
         >>> str(url('http://user:pw@host:80/?foo=bar&baz=42'))
         'http://user:pw@host:80/?foo=bar&baz=42'
         >>> str(url('http://user:pw@host:80/?foo=bar%3dbaz'))
         'http://user:pw@host:80/?foo=bar%3dbaz'
         >>> str(url('ssh://user:pw@[::1]:2200//home/joe#'))
         'ssh://user:pw@[::1]:2200//home/joe#'
         >>> str(url('http://localhost:80//'))
         'http://localhost:80//'
         >>> str(url('http://localhost:80/'))
         'http://localhost:80/'
         >>> str(url('http://localhost:80'))
         'http://localhost:80/'
         >>> str(url('bundle:foo'))
         'bundle:foo'
         >>> str(url('bundle://../foo'))
         'bundle:../foo'
         >>> str(url('path'))
         'path'
         >>> str(url('file:///tmp/foo/bar'))
         'file:///tmp/foo/bar'
         >>> str(url('file:///c:/tmp/foo/bar'))
         'file:///c:/tmp/foo/bar'
         >>> print url(r'bundle:foo\bar')
         bundle:foo\bar
         >>> print url(r'file:///D:\data\hg')
         file:///D:\data\hg
         """
         if self._localpath:
             s = self.path
             if self.scheme == 'bundle':
                 s = 'bundle:' + s
             if self.fragment:
                 s += '#' + self.fragment
             return s
 
         s = self.scheme + ':'
         if self.user or self.passwd or self.host:
             s += '//'
         elif self.scheme and (not self.path or self.path.startswith('/')
                               or hasdriveletter(self.path)):
             s += '//'
             if hasdriveletter(self.path):
                 s += '/'
         if self.user:
             s += urlreq.quote(self.user, safe=self._safechars)
         if self.passwd:
             s += ':' + urlreq.quote(self.passwd, safe=self._safechars)
         if self.user or self.passwd:
             s += '@'
         if self.host:
             if not (self.host.startswith('[') and self.host.endswith(']')):
                 s += urlreq.quote(self.host)
             else:
                 s += self.host
         if self.port:
             s += ':' + urlreq.quote(self.port)
         if self.host:
             s += '/'
         if self.path:
             # TODO: similar to the query string, we should not unescape the
             # path when we store it, the path might contain '%2f' = '/',
             # which we should *not* escape.
             s += urlreq.quote(self.path, safe=self._safepchars)
         if self.query:
             # we store the query in escaped form.
             s += '?' + self.query
         if self.fragment is not None:
             s += '#' + urlreq.quote(self.fragment, safe=self._safepchars)
         return s
 
     def authinfo(self):
         user, passwd = self.user, self.passwd
         try:
             self.user, self.passwd = None, None
             s = str(self)
         finally:
             self.user, self.passwd = user, passwd
         if not self.user:
             return (s, None)
         # authinfo[1] is passed to urllib2 password manager, and its
         # URIs must not contain credentials. The host is passed in the
         # URIs list because Python < 2.4.3 uses only that to search for
         # a password.
         return (s, (None, (s, self.host),
                     self.user, self.passwd or ''))
 
     def isabs(self):
         if self.scheme and self.scheme != 'file':
             return True # remote URL
         if hasdriveletter(self.path):
             return True # absolute for our purposes - can't be joined()
         if self.path.startswith(r'\\'):
             return True # Windows UNC path
         if self.path.startswith('/'):
             return True # POSIX-style
         return False
 
     def localpath(self):
         if self.scheme == 'file' or self.scheme == 'bundle':
             path = self.path or '/'
             # For Windows, we need to promote hosts containing drive
             # letters to paths with drive letters.
             if hasdriveletter(self._hostport):
                 path = self._hostport + '/' + self.path
             elif (self.host is not None and self.path
                   and not hasdriveletter(path)):
                 path = '/' + path
             return path
         return self._origpath
 
     def islocal(self):
         '''whether localpath will return something that posixfile can open'''
         return (not self.scheme or self.scheme == 'file'
                 or self.scheme == 'bundle')
 
 def hasscheme(path):
     return bool(url(path).scheme)
 
 def hasdriveletter(path):
     return path and path[1:2] == ':' and path[0:1].isalpha()
 
 def urllocalpath(path):
     return url(path, parsequery=False, parsefragment=False).localpath()
 
 def hidepassword(u):
     '''hide user credential in a url string'''
     u = url(u)
     if u.passwd:
         u.passwd = '***'
     return str(u)
 
 def removeauth(u):
     '''remove all authentication information from a url string'''
     u = url(u)
     u.user = u.passwd = None
     return str(u)
 
 def isatty(fp):
     try:
         return fp.isatty()
     except AttributeError:
         return False
 
 timecount = unitcountfn(
     (1, 1e3, _('%.0f s')),
     (100, 1, _('%.1f s')),
     (10, 1, _('%.2f s')),
     (1, 1, _('%.3f s')),
     (100, 0.001, _('%.1f ms')),
     (10, 0.001, _('%.2f ms')),
     (1, 0.001, _('%.3f ms')),
     (100, 0.000001, _('%.1f us')),
     (10, 0.000001, _('%.2f us')),
     (1, 0.000001, _('%.3f us')),
     (100, 0.000000001, _('%.1f ns')),
     (10, 0.000000001, _('%.2f ns')),
     (1, 0.000000001, _('%.3f ns')),
     )
 
 _timenesting = [0]
 
 def timed(func):
     '''Report the execution time of a function call to stderr.
 
     During development, use as a decorator when you need to measure
     the cost of a function, e.g. as follows:
 
     @util.timed
     def foo(a, b, c):
         pass
     '''
 
     def wrapper(*args, **kwargs):
         start = time.time()
         indent = 2
         _timenesting[0] += indent
         try:
             return func(*args, **kwargs)
         finally:
             elapsed = time.time() - start
             _timenesting[0] -= indent
             sys.stderr.write('%s%s: %s\n' %
                              (' ' * _timenesting[0], func.__name__,
                               timecount(elapsed)))
     return wrapper
 
 _sizeunits = (('m', 2**20), ('k', 2**10), ('g', 2**30),
               ('kb', 2**10), ('mb', 2**20), ('gb', 2**30), ('b', 1))
 
 def sizetoint(s):
     '''Convert a space specifier to a byte count.
 
     >>> sizetoint('30')
     30
     >>> sizetoint('2.2kb')
     2252
     >>> sizetoint('6M')
     6291456
     '''
     t = s.strip().lower()
     try:
         for k, u in _sizeunits:
             if t.endswith(k):
                 return int(float(t[:-len(k)]) * u)
         return int(t)
     except ValueError:
         raise error.ParseError(_("couldn't parse size: %s") % s)
 
 class hooks(object):
     '''A collection of hook functions that can be used to extend a
     function's behavior. Hooks are called in lexicographic order,
     based on the names of their sources.'''
 
     def __init__(self):
         self._hooks = []
 
     def add(self, source, hook):
         self._hooks.append((source, hook))
 
     def __call__(self, *args):
         self._hooks.sort(key=lambda x: x[0])
         results = []
         for source, hook in self._hooks:
             results.append(hook(*args))
         return results
 
 def getstackframes(skip=0, line=' %-*s in %s\n', fileline='%s:%s'):
     '''Yields lines for a nicely formatted stacktrace.
     Skips the 'skip' last entries.
     Each file+linenumber is formatted according to fileline.
     Each line is formatted according to line.
     If line is None, it yields:
       length of longest filepath+line number,
       filepath+linenumber,
       function
 
     Not be used in production code but very convenient while developing.
     '''
     entries = [(fileline % (fn, ln), func)
         for fn, ln, func, _text in traceback.extract_stack()[:-skip - 1]]
     if entries:
         fnmax = max(len(entry[0]) for entry in entries)
         for fnln, func in entries:
             if line is None:
                 yield (fnmax, fnln, func)
             else:
                 yield line % (fnmax, fnln, func)
 
 def debugstacktrace(msg='stacktrace', skip=0, f=sys.stderr, otherf=sys.stdout):
     '''Writes a message to f (stderr) with a nicely formatted stacktrace.
     Skips the 'skip' last entries. By default it will flush stdout first.
     It can be used everywhere and intentionally does not require an ui object.
     Not be used in production code but very convenient while developing.
     '''
     if otherf:
         otherf.flush()
     f.write('%s at:\n' % msg)
     for line in getstackframes(skip + 1):
         f.write(line)
     f.flush()
 
 class dirs(object):
     '''a multiset of directory names from a dirstate or manifest'''
 
     def __init__(self, map, skip=None):
         self._dirs = {}
         addpath = self.addpath
         if safehasattr(map, 'iteritems') and skip is not None:
             for f, s in map.iteritems():
                 if s[0] != skip:
                     addpath(f)
         else:
             for f in map:
                 addpath(f)
 
     def addpath(self, path):
         dirs = self._dirs
         for base in finddirs(path):
             if base in dirs:
                 dirs[base] += 1
                 return
             dirs[base] = 1
 
     def delpath(self, path):
         dirs = self._dirs
         for base in finddirs(path):
             if dirs[base] > 1:
                 dirs[base] -= 1
                 return
             del dirs[base]
 
     def __iter__(self):
         return self._dirs.iterkeys()
 
     def __contains__(self, d):
         return d in self._dirs
 
 if safehasattr(parsers, 'dirs'):
     dirs = parsers.dirs
 
 def finddirs(path):
     pos = path.rfind('/')
     while pos != -1:
         yield path[:pos]
         pos = path.rfind('/', 0, pos)
 
 # compression utility
 
 class nocompress(object):
     def compress(self, x):
         return x
     def flush(self):
         return ""
 
 compressors = {
     None: nocompress,
     # lambda to prevent early import
     'BZ': lambda: bz2.BZ2Compressor(),
     'GZ': lambda: zlib.compressobj(),
     }
 # also support the old form by courtesies
 compressors['UN'] = compressors[None]
 
 def _makedecompressor(decompcls):
     def generator(f):
         d = decompcls()
         for chunk in filechunkiter(f):
             yield d.decompress(chunk)
     def func(fh):
         return chunkbuffer(generator(fh))
     return func
 
 class ctxmanager(object):
     '''A context manager for use in 'with' blocks to allow multiple
     contexts to be entered at once.  This is both safer and more
     flexible than contextlib.nested.
 
     Once Mercurial supports Python 2.7+, this will become mostly
     unnecessary.
     '''
 
     def __init__(self, *args):
         '''Accepts a list of no-argument functions that return context
         managers.  These will be invoked at __call__ time.'''
         self._pending = args
         self._atexit = []
 
     def __enter__(self):
         return self
 
     def enter(self):
         '''Create and enter context managers in the order in which they were
         passed to the constructor.'''
         values = []
         for func in self._pending:
             obj = func()
             values.append(obj.__enter__())
             self._atexit.append(obj.__exit__)
         del self._pending
         return values
 
     def atexit(self, func, *args, **kwargs):
         '''Add a function to call when this context manager exits.  The
         ordering of multiple atexit calls is unspecified, save that
         they will happen before any __exit__ functions.'''
         def wrapper(exc_type, exc_val, exc_tb):
             func(*args, **kwargs)
         self._atexit.append(wrapper)
         return func
 
     def __exit__(self, exc_type, exc_val, exc_tb):
         '''Context managers are exited in the reverse order from which
         they were created.'''
         received = exc_type is not None
         suppressed = False
         pending = None
         self._atexit.reverse()
         for exitfunc in self._atexit:
             try:
                 if exitfunc(exc_type, exc_val, exc_tb):
                     suppressed = True
                     exc_type = None
                     exc_val = None
                     exc_tb = None
             except BaseException:
                 pending = sys.exc_info()
                 exc_type, exc_val, exc_tb = pending = sys.exc_info()
         del self._atexit
         if pending:
             raise exc_val
         return received and suppressed
 
 def _bz2():
     d = bz2.BZ2Decompressor()
     # Bzip2 stream start with BZ, but we stripped it.
     # we put it back for good measure.
     d.decompress('BZ')
     return d
 
 decompressors = {None: lambda fh: fh,
                  '_truncatedBZ': _makedecompressor(_bz2),
                  'BZ': _makedecompressor(lambda: bz2.BZ2Decompressor()),
                  'GZ': _makedecompressor(lambda: zlib.decompressobj()),
                  }
 # also support the old form by courtesies
 decompressors['UN'] = decompressors[None]
 
 # convenient shortcut
 dst = debugstacktrace

tests/test-check-py3-compat.t

@@ -1,151 +1,151 b''
 #require test-repo
 
   $ . "$TESTDIR/helpers-testrepo.sh"
   $ cd "$TESTDIR"/..
 
   $ hg files 'set:(**.py)' | sed 's|\\|/|g' | xargs python contrib/check-py3-compat.py
   hgext/fsmonitor/pywatchman/__init__.py not using absolute_import
   hgext/fsmonitor/pywatchman/__init__.py requires print_function
   hgext/fsmonitor/pywatchman/capabilities.py not using absolute_import
   hgext/fsmonitor/pywatchman/pybser.py not using absolute_import
   hgext/highlight/__init__.py not using absolute_import
   hgext/highlight/highlight.py not using absolute_import
   hgext/share.py not using absolute_import
   hgext/win32text.py not using absolute_import
   i18n/check-translation.py not using absolute_import
   i18n/polib.py not using absolute_import
   setup.py not using absolute_import
   tests/heredoctest.py requires print_function
   tests/md5sum.py not using absolute_import
   tests/readlink.py not using absolute_import
   tests/run-tests.py not using absolute_import
   tests/test-demandimport.py not using absolute_import
 
 #if py3exe
   $ hg files 'set:(**.py)' | sed 's|\\|/|g' | xargs $PYTHON3 contrib/check-py3-compat.py
   doc/hgmanpage.py: invalid syntax: invalid syntax (<unknown>, line *) (glob)
   hgext/automv.py: error importing module: <SyntaxError> invalid syntax (commands.py, line *) (line *) (glob)
   hgext/blackbox.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
-  hgext/bugzilla.py: error importing module: <ImportError> No module named 'urlparse' (line *) (glob)
+  hgext/bugzilla.py: error importing module: <ImportError> No module named 'xmlrpclib' (line *) (glob)
   hgext/censor.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/chgserver.py: error importing module: <ImportError> No module named 'SocketServer' (line *) (glob)
   hgext/children.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/churn.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/clonebundles.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/color.py: invalid syntax: invalid syntax (<unknown>, line *) (glob)
   hgext/convert/bzr.py: error importing module: <SystemError> Parent module 'hgext.convert' not loaded, cannot perform relative import (line *) (glob)
   hgext/convert/convcmd.py: error importing: <SyntaxError> invalid syntax (bundle*.py, line *) (error at bundlerepo.py:*) (glob)
   hgext/convert/cvs.py: error importing module: <SystemError> Parent module 'hgext.convert' not loaded, cannot perform relative import (line *) (glob)
   hgext/convert/cvsps.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/convert/darcs.py: error importing module: <SystemError> Parent module 'hgext.convert' not loaded, cannot perform relative import (line *) (glob)
   hgext/convert/filemap.py: error importing module: <SystemError> Parent module 'hgext.convert' not loaded, cannot perform relative import (line *) (glob)
   hgext/convert/git.py: error importing module: <SystemError> Parent module 'hgext.convert' not loaded, cannot perform relative import (line *) (glob)
   hgext/convert/gnuarch.py: error importing module: <SystemError> Parent module 'hgext.convert' not loaded, cannot perform relative import (line *) (glob)
   hgext/convert/hg.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/convert/monotone.py: error importing module: <SystemError> Parent module 'hgext.convert' not loaded, cannot perform relative import (line *) (glob)
   hgext/convert/p*.py: error importing module: <SystemError> Parent module 'hgext.convert' not loaded, cannot perform relative import (line *) (glob)
   hgext/convert/subversion.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/convert/transport.py: error importing module: <ImportError> No module named 'svn.client' (line *) (glob)
   hgext/eol.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/extdiff.py: error importing module: <SyntaxError> invalid syntax (archival.py, line *) (line *) (glob)
   hgext/factotum.py: error importing: <ImportError> No module named 'rfc822' (error at __init__.py:*) (glob)
   hgext/fetch.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/fsmonitor/watchmanclient.py: error importing module: <SystemError> Parent module 'hgext.fsmonitor' not loaded, cannot perform relative import (line *) (glob)
   hgext/gpg.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/graphlog.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/hgk.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/histedit.py: error importing module: <SyntaxError> invalid syntax (bundle*.py, line *) (line *) (glob)
   hgext/keyword.py: error importing: <ImportError> No module named 'BaseHTTPServer' (error at common.py:*) (glob)
   hgext/largefiles/basestore.py: error importing module: <SystemError> Parent module 'hgext.largefiles' not loaded, cannot perform relative import (line *) (glob)
   hgext/largefiles/lfcommands.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/largefiles/lfutil.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/largefiles/localstore.py: error importing module: <SystemError> Parent module 'hgext.largefiles' not loaded, cannot perform relative import (line *) (glob)
   hgext/largefiles/overrides.py: error importing module: <SyntaxError> invalid syntax (archival.py, line *) (line *) (glob)
   hgext/largefiles/proto.py: error importing: <ImportError> No module named 'httplib' (error at httppeer.py:*) (glob)
   hgext/largefiles/remotestore.py: error importing: <SyntaxError> invalid syntax (bundle*.py, line *) (error at wireproto.py:*) (glob)
   hgext/largefiles/reposetup.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/largefiles/storefactory.py: error importing: <SyntaxError> invalid syntax (bundle2.py, line *) (error at bundlerepo.py:*) (glob)
   hgext/largefiles/uisetup.py: error importing: <ImportError> No module named 'BaseHTTPServer' (error at common.py:*) (glob)
   hgext/largefiles/wirestore.py: error importing module: <SystemError> Parent module 'hgext.largefiles' not loaded, cannot perform relative import (line *) (glob)
   hgext/mq.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/notify.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/pager.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/patchbomb.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/purge.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/rebase.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/record.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/relink.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/schemes.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/share.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/shelve.py: error importing module: <SyntaxError> invalid syntax (bundle*.py, line *) (line *) (glob)
   hgext/strip.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   hgext/transplant.py: error importing: <SyntaxError> invalid syntax (bundle*.py, line *) (error at bundlerepo.py:*) (glob)
   mercurial/archival.py: invalid syntax: invalid syntax (<unknown>, line *) (glob)
   mercurial/branchmap.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/bundle*.py: invalid syntax: invalid syntax (<unknown>, line *) (glob)
   mercurial/bundlerepo.py: error importing module: <SyntaxError> invalid syntax (bundle*.py, line *) (line *) (glob)
   mercurial/changegroup.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/changelog.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/cmdutil.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/commands.py: invalid syntax: invalid syntax (<unknown>, line *) (glob)
   mercurial/commandserver.py: error importing module: <ImportError> No module named 'SocketServer' (line *) (glob)
   mercurial/context.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/copies.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/crecord.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/dirstate.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/discovery.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/dispatch.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/exchange.py: error importing module: <SyntaxError> invalid syntax (bundle*.py, line *) (line *) (glob)
   mercurial/extensions.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/filelog.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/filemerge.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/fileset.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/formatter.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/graphmod.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/help.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/hg.py: error importing: <SyntaxError> invalid syntax (bundle*.py, line *) (error at bundlerepo.py:*) (glob)
   mercurial/hgweb/common.py: error importing module: <ImportError> No module named 'BaseHTTPServer' (line *) (glob)
   mercurial/hgweb/hgweb_mod.py: error importing module: <SystemError> Parent module 'mercurial.hgweb' not loaded, cannot perform relative import (line *) (glob)
   mercurial/hgweb/hgwebdir_mod.py: error importing module: <SystemError> Parent module 'mercurial.hgweb' not loaded, cannot perform relative import (line *) (glob)
   mercurial/hgweb/protocol.py: error importing module: <SystemError> Parent module 'mercurial.hgweb' not loaded, cannot perform relative import (line *) (glob)
   mercurial/hgweb/request.py: error importing module: <SystemError> Parent module 'mercurial.hgweb' not loaded, cannot perform relative import (line *) (glob)
   mercurial/hgweb/server.py: error importing module: <ImportError> No module named 'BaseHTTPServer' (line *) (glob)
   mercurial/hgweb/webcommands.py: error importing module: <SystemError> Parent module 'mercurial.hgweb' not loaded, cannot perform relative import (line *) (glob)
   mercurial/hgweb/webutil.py: error importing module: <SystemError> Parent module 'mercurial.hgweb' not loaded, cannot perform relative import (line *) (glob)
   mercurial/hgweb/wsgicgi.py: error importing module: <SystemError> Parent module 'mercurial.hgweb' not loaded, cannot perform relative import (line *) (glob)
   mercurial/hook.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/httpconnection.py: error importing: <ImportError> No module named 'rfc822' (error at __init__.py:*) (glob)
   mercurial/httppeer.py: error importing module: <ImportError> No module named 'httplib' (line *) (glob)
   mercurial/keepalive.py: error importing module: <ImportError> No module named 'httplib' (line *) (glob)
   mercurial/localrepo.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/mail.py: error importing module: <AttributeError> module 'email' has no attribute 'Header' (line *) (glob)
   mercurial/manifest.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/merge.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/namespaces.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/patch.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/pure/mpatch.py: error importing module: <ImportError> cannot import name 'pycompat' (line *) (glob)
   mercurial/pure/parsers.py: error importing module: <ImportError> No module named 'mercurial.pure.node' (line *) (glob)
   mercurial/repair.py: error importing module: <SyntaxError> invalid syntax (bundle*.py, line *) (line *) (glob)
   mercurial/revlog.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/revset.py: error importing module: <AttributeError> 'dict' object has no attribute 'iteritems' (line *) (glob)
   mercurial/scmutil.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/scmwindows.py: error importing module: <ImportError> No module named '_winreg' (line *) (glob)
   mercurial/simplemerge.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/sshpeer.py: error importing: <SyntaxError> invalid syntax (bundle*.py, line *) (error at wireproto.py:*) (glob)
   mercurial/sshserver.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/statichttprepo.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/store.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/streamclone.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/subrepo.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/templatefilters.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/templatekw.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/templater.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/ui.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/unionrepo.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/url.py: error importing module: <ImportError> No module named 'httplib' (line *) (glob)
   mercurial/verify.py: error importing: <AttributeError> 'dict' object has no attribute 'iteritems' (error at revset.py:*) (glob)
   mercurial/win*.py: error importing module: <ImportError> No module named 'msvcrt' (line *) (glob)
   mercurial/windows.py: error importing module: <ImportError> No module named '_winreg' (line *) (glob)
   mercurial/wireproto.py: error importing module: <SyntaxError> invalid syntax (bundle*.py, line *) (line *) (glob)
 
 #endif

tests/tinyproxy.py

@@ -1,158 +1,161 b''
 #!/usr/bin/env python
 
 from __future__ import absolute_import, print_function
 
 __doc__ = """Tiny HTTP Proxy.
 
 This module implements GET, HEAD, POST, PUT and DELETE methods
 on BaseHTTPServer, and behaves as an HTTP proxy.  The CONNECT
 method is also implemented experimentally, but has not been
 tested yet.
 
 Any help will be greatly appreciated.           SUZUKI Hisao
 """
 
 __version__ = "0.2.1"
 
 import BaseHTTPServer
 import SocketServer
 import os
 import select
 import socket
 import sys
-import urlparse
+
+from mercurial import util
+
+urlparse = util.urlparse
 
 class ProxyHandler (BaseHTTPServer.BaseHTTPRequestHandler):
     __base = BaseHTTPServer.BaseHTTPRequestHandler
     __base_handle = __base.handle
 
     server_version = "TinyHTTPProxy/" + __version__
     rbufsize = 0                        # self.rfile Be unbuffered
 
     def handle(self):
         (ip, port) = self.client_address
         allowed = getattr(self, 'allowed_clients', None)
         if allowed is not None and ip not in allowed:
             self.raw_requestline = self.rfile.readline()
             if self.parse_request():
                 self.send_error(403)
         else:
             self.__base_handle()
 
     def log_request(self, code='-', size='-'):
         xheaders = [h for h in self.headers.items() if h[0].startswith('x-')]
         self.log_message('"%s" %s %s%s',
                          self.requestline, str(code), str(size),
                          ''.join([' %s:%s' % h for h in sorted(xheaders)]))
 
     def _connect_to(self, netloc, soc):
         i = netloc.find(':')
         if i >= 0:
             host_port = netloc[:i], int(netloc[i + 1:])
         else:
             host_port = netloc, 80
         print("\t" "connect to %s:%d" % host_port)
         try: soc.connect(host_port)
         except socket.error as arg:
             try: msg = arg[1]
             except (IndexError, TypeError): msg = arg
             self.send_error(404, msg)
             return 0
         return 1
 
     def do_CONNECT(self):
         soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         try:
             if self._connect_to(self.path, soc):
                 self.log_request(200)
                 self.wfile.write(self.protocol_version +
                                  " 200 Connection established\r\n")
                 self.wfile.write("Proxy-agent: %s\r\n" % self.version_string())
                 self.wfile.write("\r\n")
                 self._read_write(soc, 300)
         finally:
             print("\t" "bye")
             soc.close()
             self.connection.close()
 
     def do_GET(self):
         (scm, netloc, path, params, query, fragment) = urlparse.urlparse(
             self.path, 'http')
         if scm != 'http' or fragment or not netloc:
             self.send_error(400, "bad url %s" % self.path)
             return
         soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         try:
             if self._connect_to(netloc, soc):
                 self.log_request()
                 soc.send("%s %s %s\r\n" % (
                     self.command,
                     urlparse.urlunparse(('', '', path, params, query, '')),
                     self.request_version))
                 self.headers['Connection'] = 'close'
                 del self.headers['Proxy-Connection']
                 for key_val in self.headers.items():
                     soc.send("%s: %s\r\n" % key_val)
                 soc.send("\r\n")
                 self._read_write(soc)
         finally:
             print("\t" "bye")
             soc.close()
             self.connection.close()
 
     def _read_write(self, soc, max_idling=20):
         iw = [self.connection, soc]
         ow = []
         count = 0
         while True:
             count += 1
             (ins, _, exs) = select.select(iw, ow, iw, 3)
             if exs:
                 break
             if ins:
                 for i in ins:
                     if i is soc:
                         out = self.connection
                     else:
                         out = soc
                     try:
                         data = i.recv(8192)
                     except socket.error:
                         break
                     if data:
                         out.send(data)
                         count = 0
             else:
                 print("\t" "idle", count)
             if count == max_idling:
                 break
 
     do_HEAD = do_GET
     do_POST = do_GET
     do_PUT  = do_GET
     do_DELETE = do_GET
 
 class ThreadingHTTPServer (SocketServer.ThreadingMixIn,
                            BaseHTTPServer.HTTPServer):
     def __init__(self, *args, **kwargs):
         BaseHTTPServer.HTTPServer.__init__(self, *args, **kwargs)
         a = open("proxy.pid", "w")
         a.write(str(os.getpid()) + "\n")
         a.close()
 
 if __name__ == '__main__':
     argv = sys.argv
     if argv[1:] and argv[1] in ('-h', '--help'):
         print(argv[0], "[port [allowed_client_name ...]]")
     else:
         if argv[2:]:
             allowed = []
             for name in argv[2:]:
                 client = socket.gethostbyname(name)
                 allowed.append(client)
                 print("Accept: %s (%s)" % (client, name))
             ProxyHandler.allowed_clients = allowed
             del argv[2:]
         else:
             print("Any clients will be served...")
         BaseHTTPServer.test(ProxyHandler, ThreadingHTTPServer)