clone: disable stream support on server side by default.

enable in hgrc like this:
[server]
stream=True

doc/hgrc.5.txt

@@ -1,452 +1,464 b''
 HGRC(5)
 =======
 Bryan O'Sullivan <[e-mail]>
 
 NAME
 ----
 hgrc - configuration files for Mercurial
 
 SYNOPSIS
 --------
 
 The Mercurial system uses a set of configuration files to control
 aspects of its behaviour.
 
 FILES
 -----
 
 Mercurial reads configuration data from several files, if they exist.
 The names of these files depend on the system on which Mercurial is
 installed.
 
 (Unix)    <install-root>/etc/mercurial/hgrc.d/*.rc::
 (Unix)    <install-root>/etc/mercurial/hgrc::
     Per-installation configuration files, searched for in the
     directory where Mercurial is installed.  For example, if installed
     in /shared/tools, Mercurial will look in
     /shared/tools/etc/mercurial/hgrc.  Options in these files apply to
     all Mercurial commands executed by any user in any directory.
 
 (Unix)    /etc/mercurial/hgrc.d/*.rc::
 (Unix)    /etc/mercurial/hgrc::
 (Windows) C:\Mercurial\Mercurial.ini::
     Per-system configuration files, for the system on which Mercurial
     is running.  Options in these files apply to all Mercurial
     commands executed by any user in any directory.  Options in these
     files override per-installation options.
 
 (Unix)    $HOME/.hgrc::
 (Windows) C:\Documents and Settings\USERNAME\Mercurial.ini::
 (Windows) $HOME\Mercurial.ini::
     Per-user configuration file, for the user running Mercurial.
     Options in this file apply to all Mercurial commands executed by
     any user in any directory.  Options in this file override
     per-installation and per-system options.
     On Windows system, one of these is chosen exclusively according
     to definition of HOME environment variable.
 
 (Unix, Windows) <repo>/.hg/hgrc::
     Per-repository configuration options that only apply in a
     particular repository.  This file is not version-controlled, and
     will not get transferred during a "clone" operation.  Options in
     this file override options in all other configuration files.
 
 SYNTAX
 ------
 
 A configuration file consists of sections, led by a "[section]" header
 and followed by "name: value" entries; "name=value" is also accepted.
 
     [spam]
     eggs=ham
     green=
        eggs
 
 Each line contains one entry.  If the lines that follow are indented,
 they are treated as continuations of that entry.
 
 Leading whitespace is removed from values.  Empty lines are skipped.
 
 The optional values can contain format strings which refer to other
 values in the same section, or values in a special DEFAULT section.
 
 Lines beginning with "#" or ";" are ignored and may be used to provide
 comments.
 
 SECTIONS
 --------
 
 This section describes the different sections that may appear in a
 Mercurial "hgrc" file, the purpose of each section, its possible
 keys, and their possible values.
 
 decode/encode::
   Filters for transforming files on checkout/checkin. This would
   typically be used for newline processing or other
   localization/canonicalization of files.
 
   Filters consist of a filter pattern followed by a filter command.
   Filter patterns are globs by default, rooted at the repository
   root.  For example, to match any file ending in ".txt" in the root
   directory only, use the pattern "*.txt".  To match any file ending
   in ".c" anywhere in the repository, use the pattern "**.c".
 
   The filter command can start with a specifier, either "pipe:" or
   "tempfile:".  If no specifier is given, "pipe:" is used by default.
 
   A "pipe:" command must accept data on stdin and return the
   transformed data on stdout.
 
   Pipe example:
 
     [encode]
     # uncompress gzip files on checkin to improve delta compression
     # note: not necessarily a good idea, just an example
     *.gz = pipe: gunzip
 
     [decode]
     # recompress gzip files when writing them to the working dir (we
     # can safely omit "pipe:", because it's the default)
     *.gz = gzip
 
   A "tempfile:" command is a template.  The string INFILE is replaced
   with the name of a temporary file that contains the data to be
   filtered by the command.  The string OUTFILE is replaced with the
   name of an empty temporary file, where the filtered data must be
   written by the command.
 
   NOTE: the tempfile mechanism is recommended for Windows systems,
   where the standard shell I/O redirection operators often have
   strange effects.  In particular, if you are doing line ending
   conversion on Windows using the popular dos2unix and unix2dos
   programs, you *must* use the tempfile mechanism, as using pipes will
   corrupt the contents of your files.
 
   Tempfile example:
 
     [encode]
     # convert files to unix line ending conventions on checkin
     **.txt = tempfile: dos2unix -n INFILE OUTFILE
 
     [decode]
     # convert files to windows line ending conventions when writing
     # them to the working dir
     **.txt = tempfile: unix2dos -n INFILE OUTFILE
 
 email::
   Settings for extensions that send email messages.
   from;;
     Optional.  Email address to use in "From" header and SMTP envelope
     of outgoing messages.
   method;;
     Optional.  Method to use to send email messages.  If value is
     "smtp" (default), use SMTP (see section "[mail]" for
     configuration).  Otherwise, use as name of program to run that
     acts like sendmail (takes "-f" option for sender, list of
     recipients on command line, message on stdin).  Normally, setting
     this to "sendmail" or "/usr/sbin/sendmail" is enough to use
     sendmail to send messages.
 
   Email example:
 
     [email]
     from = Joseph User <[e-mail]>
     method = /usr/sbin/sendmail
 
 extensions::
   Mercurial has an extension mechanism for adding new features. To
   enable an extension, create an entry for it in this section.
 
   If you know that the extension is already in Python's search path,
   you can give the name of the module, followed by "=", with nothing
   after the "=".
 
   Otherwise, give a name that you choose, followed by "=", followed by
   the path to the ".py" file (including the file name extension) that
   defines the extension.
 
   Example for ~/.hgrc:
 
     [extensions]
     # (the mq extension will get loaded from mercurial's path)
     hgext.mq =
     # (this extension will get loaded from the file specified)
     myfeature = ~/.hgext/myfeature.py
 
 hooks::
   Commands or Python functions that get automatically executed by
   various actions such as starting or finishing a commit. Multiple
   hooks can be run for the same action by appending a suffix to the
   action. Overriding a site-wide hook can be done by changing its
   value or setting it to an empty string.
 
   Example .hg/hgrc:
 
     [hooks]
     # do not use the site-wide hook
     incoming =
     incoming.email = /my/email/hook
     incoming.autobuild = /my/build/hook
 
   Most hooks are run with environment variables set that give added
   useful information.  For each hook below, the environment variables
   it is passed are listed with names of the form "$HG_foo".
 
   changegroup;;
     Run after a changegroup has been added via push, pull or
     unbundle. ID of the first new changeset is in $HG_NODE.
   commit;;
     Run after a changeset has been created in the local repository.
     ID of the newly created changeset is in $HG_NODE.  Parent
     changeset IDs are in $HG_PARENT1 and $HG_PARENT2.
   incoming;;
     Run after a changeset has been pulled, pushed, or unbundled into
     the local repository.  The ID of the newly arrived changeset is in
     $HG_NODE.
   outgoing;;
     Run after sending changes from local repository to another.  ID of
     first changeset sent is in $HG_NODE.  Source of operation is in
     $HG_SOURCE; see "preoutgoing" hook for description.
   prechangegroup;;
     Run before a changegroup is added via push, pull or unbundle.
     Exit status 0 allows the changegroup to proceed.  Non-zero status
     will cause the push, pull or unbundle to fail.
   precommit;;
     Run before starting a local commit.  Exit status 0 allows the
     commit to proceed.  Non-zero status will cause the commit to fail.
     Parent changeset IDs are in $HG_PARENT1 and $HG_PARENT2.
   preoutgoing;;
     Run before computing changes to send from the local repository to
     another.  Non-zero status will cause failure.  This lets you
     prevent pull over http or ssh.  Also prevents against local pull,
     push (outbound) or bundle commands, but not effective, since you
     can just copy files instead then.  Source of operation is in
     $HG_SOURCE.  If "serve", operation is happening on behalf of
     remote ssh or http repository.  If "push", "pull" or "bundle",
     operation is happening on behalf of repository on same system.
   pretag;;
     Run before creating a tag.  Exit status 0 allows the tag to be
     created.  Non-zero status will cause the tag to fail.  ID of
     changeset to tag is in $HG_NODE.  Name of tag is in $HG_TAG.  Tag
     is local if $HG_LOCAL=1, in repo if $HG_LOCAL=0.
   pretxnchangegroup;;
     Run after a changegroup has been added via push, pull or unbundle,
     but before the transaction has been committed.  Changegroup is
     visible to hook program.  This lets you validate incoming changes
     before accepting them.  Passed the ID of the first new changeset
     in $HG_NODE.  Exit status 0 allows the transaction to commit.
     Non-zero status will cause the transaction to be rolled back and
     the push, pull or unbundle will fail.
   pretxncommit;;
     Run after a changeset has been created but the transaction not yet
     committed.  Changeset is visible to hook program.  This lets you
     validate commit message and changes.  Exit status 0 allows the
     commit to proceed.  Non-zero status will cause the transaction to
     be rolled back.  ID of changeset is in $HG_NODE.  Parent changeset
     IDs are in $HG_PARENT1 and $HG_PARENT2.
   preupdate;;
     Run before updating the working directory.  Exit status 0 allows
     the update to proceed.  Non-zero status will prevent the update.
     Changeset ID of first new parent is in $HG_PARENT1.  If merge, ID
     of second new parent is in $HG_PARENT2.
   tag;;
     Run after a tag is created.  ID of tagged changeset is in
     $HG_NODE.  Name of tag is in $HG_TAG.  Tag is local if
     $HG_LOCAL=1, in repo if $HG_LOCAL=0.
   update;;
     Run after updating the working directory.  Changeset ID of first
     new parent is in $HG_PARENT1.  If merge, ID of second new parent
     is in $HG_PARENT2.  If update succeeded, $HG_ERROR=0.  If update
     failed (e.g. because conflicts not resolved), $HG_ERROR=1.
 
   Note: In earlier releases, the names of hook environment variables
   did not have a "HG_" prefix.  The old unprefixed names are no longer
   provided in the environment.
 
   The syntax for Python hooks is as follows:
 
     hookname = python:modulename.submodule.callable
 
   Python hooks are run within the Mercurial process.  Each hook is
   called with at least three keyword arguments: a ui object (keyword
   "ui"), a repository object (keyword "repo"), and a "hooktype"
   keyword that tells what kind of hook is used.  Arguments listed as
   environment variables above are passed as keyword arguments, with no
   "HG_" prefix, and names in lower case.
 
   A Python hook must return a "true" value to succeed.  Returning a
   "false" value or raising an exception is treated as failure of the
   hook.
 
 http_proxy::
   Used to access web-based Mercurial repositories through a HTTP
   proxy.
   host;;
     Host name and (optional) port of the proxy server, for example
     "myproxy:8000".
   no;;
     Optional.  Comma-separated list of host names that should bypass
     the proxy.
   passwd;;
     Optional.  Password to authenticate with at the proxy server.
   user;;
     Optional.  User name to authenticate with at the proxy server.
 
 smtp::
   Configuration for extensions that need to send email messages.
   host;;
     Optional.  Host name of mail server.  Default: "mail".
   port;;
     Optional.  Port to connect to on mail server.  Default: 25.
   tls;;
     Optional.  Whether to connect to mail server using TLS.  True or
     False.  Default: False.
   username;;
     Optional.  User name to authenticate to SMTP server with.
     If username is specified, password must also be specified.
     Default: none.
   password;;
     Optional.  Password to authenticate to SMTP server with.
     If username is specified, password must also be specified.
     Default: none.
   local_hostname;;
     Optional.  It's the hostname that the sender can use to identify itself
     to the MTA.
 
 paths::
   Assigns symbolic names to repositories.  The left side is the
   symbolic name, and the right gives the directory or URL that is the
   location of the repository.  Default paths can be declared by
- setting the following entries.
+  setting the following entries.
   default;;
     Directory or URL to use when pulling if no source is specified.
     Default is set to repository from which the current repository
     was cloned.
   default-push;;
     Optional.  Directory or URL to use when pushing if no destination
     is specified.
 
+server::
+  Controls generic server settings.
+  stream;;
+    Whether to allow clients to clone a repo using the uncompressed
+    streaming protocol.  This transfers about 40% more data than a
+    regular clone, but uses less memory and CPU on both server and
+    client.  Over a LAN (100Mbps or better) or a very fast WAN, an
+    uncompressed streaming clone is a lot faster (~10x) than a regular
+    clone.  Over most WAN connections (anything slower than about
+    6Mbps), uncompressed streaming is slower, because of the extra
+    data transfer overhead.  Default is False.
+
 ui::
   User interface controls.
   debug;;
     Print debugging information.  True or False.  Default is False.
   editor;;
     The editor to use during a commit.  Default is $EDITOR or "vi".
   ignore;;
     A file to read per-user ignore patterns from. This file should be in
     the same format as a repository-wide .hgignore file. This option
     supports hook syntax, so if you want to specify multiple ignore
     files, you can do so by setting something like
     "ignore.other = ~/.hgignore2". For details of the ignore file
     format, see the hgignore(5) man page.
   interactive;;
     Allow to prompt the user.  True or False.  Default is True.
   logtemplate;;
     Template string for commands that print changesets.
   style;;
     Name of style to use for command output.
   merge;;
     The conflict resolution program to use during a manual merge.
     Default is "hgmerge".
   quiet;;
     Reduce the amount of output printed.  True or False.  Default is False.
   remotecmd;;
     remote command to use for clone/push/pull operations. Default is 'hg'.
   ssh;;
     command to use for SSH connections. Default is 'ssh'.
   timeout;;
     The timeout used when a lock is held (in seconds), a negative value
     means no timeout. Default is 600.
   username;;
     The committer of a changeset created when running "commit".
     Typically a person's name and email address, e.g. "Fred Widget
     <[e-mail]>".  Default is $EMAIL or username@hostname, unless
     username is set to an empty string, which enforces specifying the
     username manually.
   verbose;;
     Increase the amount of output printed.  True or False.  Default is False.
 
 
 web::
   Web interface configuration.
   accesslog;;
     Where to output the access log. Default is stdout.
   address;;
     Interface address to bind to. Default is all.
   allow_archive;;
     List of archive format (bz2, gz, zip) allowed for downloading.
     Default is empty.
   allowbz2;;
     (DEPRECATED) Whether to allow .tar.bz2 downloading of repo revisions.
     Default is false.
   allowgz;;
     (DEPRECATED) Whether to allow .tar.gz downloading of repo revisions.
     Default is false.
   allowpull;;
     Whether to allow pulling from the repository. Default is true.
   allow_push;;
     Whether to allow pushing to the repository.  If empty or not set,
     push is not allowed.  If the special value "*", any remote user
     can push, including unauthenticated users.  Otherwise, the remote
     user must have been authenticated, and the authenticated user name
     must be present in this list (separated by whitespace or ",").
     The contents of the allow_push list are examined after the
     deny_push list.
   allowzip;;
     (DEPRECATED) Whether to allow .zip downloading of repo revisions.
     Default is false. This feature creates temporary files.
   baseurl;;
     Base URL to use when publishing URLs in other locations, so
     third-party tools like email notification hooks can construct URLs.
     Example: "http://hgserver/repos/"
   contact;;
     Name or email address of the person in charge of the repository.
     Default is "unknown".
   deny_push;;
     Whether to deny pushing to the repository.  If empty or not set,
     push is not denied.  If the special value "*", all remote users
     are denied push.  Otherwise, unauthenticated users are all denied,
     and any authenticated user name present in this list (separated by
     whitespace or ",") is also denied.  The contents of the deny_push
     list are examined before the allow_push list.
   description;;
     Textual description of the repository's purpose or contents.
     Default is "unknown".
   errorlog;;
     Where to output the error log. Default is stderr.
   ipv6;;
     Whether to use IPv6. Default is false.
   name;;
     Repository name to use in the web interface. Default is current
     working directory.
   maxchanges;;
     Maximum number of changes to list on the changelog. Default is 10.
   maxfiles;;
     Maximum number of files to list per changeset. Default is 10.
   port;;
     Port to listen on. Default is 8000.
   push_ssl;;
     Whether to require that inbound pushes be transported over SSL to
     prevent password sniffing.  Default is true.
   style;;
     Which template map style to use.
   templates;;
     Where to find the HTML templates. Default is install path.
 
 
 AUTHOR
 ------
 Bryan O'Sullivan <[e-mail]>.
 
 Mercurial was written by Matt Mackall <[e-mail]>.
 
 SEE ALSO
 --------
 hg(1), hgignore(5)
 
 COPYING
 -------
 This manual page is copyright 2005 Bryan O'Sullivan.
 Mercurial is copyright 2005, 2006 Matt Mackall.
 Free use of this software is granted under the terms of the GNU General
 Public License (GPL).

mercurial/hg.py

@@ -1,208 +1,209 b''
 # hg.py - repository classes for mercurial
 #
 # Copyright 2005 Matt Mackall <[e-mail]>
 #
 # This software may be used and distributed according to the terms
 # of the GNU General Public License, incorporated herein by reference.
 
 from node import *
 from repo import *
 from demandload import *
 from i18n import gettext as _
 demandload(globals(), "localrepo bundlerepo httprepo sshrepo statichttprepo")
 demandload(globals(), "errno lock os shutil util")
 
 def bundle(ui, path):
     if path.startswith('bundle://'):
         path = path[9:]
     else:
         path = path[7:]
     s = path.split("+", 1)
     if len(s) == 1:
         repopath, bundlename = "", s[0]
     else:
         repopath, bundlename = s
     return bundlerepo.bundlerepository(ui, repopath, bundlename)
 
 def hg(ui, path):
     ui.warn(_("hg:// syntax is deprecated, please use http:// instead\n"))
     return httprepo.httprepository(ui, path.replace("hg://", "http://"))
 
 def local_(ui, path, create=0):
     if path.startswith('file:'):
         path = path[5:]
     return localrepo.localrepository(ui, path, create)
 
 def ssh_(ui, path, create=0):
     return sshrepo.sshrepository(ui, path, create)
 
 def old_http(ui, path):
     ui.warn(_("old-http:// syntax is deprecated, "
               "please use static-http:// instead\n"))
     return statichttprepo.statichttprepository(
         ui, path.replace("old-http://", "http://"))
 
 def static_http(ui, path):
     return statichttprepo.statichttprepository(
         ui, path.replace("static-http://", "http://"))
 
 schemes = {
     'bundle': bundle,
     'file': local_,
     'hg': hg,
     'http': lambda ui, path: httprepo.httprepository(ui, path),
     'https': lambda ui, path: httprepo.httpsrepository(ui, path),
     'old-http': old_http,
     'ssh': ssh_,
     'static-http': static_http,
     }
 
 def repository(ui, path=None, create=0):
     scheme = None
     if path:
         c = path.find(':')
         if c > 0:
             scheme = schemes.get(path[:c])
     else:
         path = ''
     ctor = scheme or schemes['file']
     if create:
         try:
             return ctor(ui, path, create)
         except TypeError:
             raise util.Abort(_('cannot create new repository over "%s" protocol') %
                              scheme)
     return ctor(ui, path)
 
 def clone(ui, source, dest=None, pull=False, rev=None, update=True,
           stream=False):
     """Make a copy of an existing repository.
 
     Create a copy of an existing repository in a new directory.  The
     source and destination are URLs, as passed to the repository
     function.  Returns a pair of repository objects, the source and
     newly created destination.
 
     The location of the source is added to the new repository's
     .hg/hgrc file, as the default to be used for future pulls and
     pushes.
 
     If an exception is raised, the partly cloned/updated destination
     repository will be deleted.
 
     Keyword arguments:
 
     dest: URL of destination repository to create (defaults to base
     name of source repository)
 
     pull: always pull from source repository, even in local case
 
-    stream: stream from repository (fast over LAN, slow over WAN)
+    stream: stream raw data uncompressed from repository (fast over
+    LAN, slow over WAN)
 
     rev: revision to clone up to (implies pull=True)
 
     update: update working directory after clone completes, if
     destination is local repository
     """
     if dest is None:
         dest = os.path.basename(os.path.normpath(source))
 
     if os.path.exists(dest):
         raise util.Abort(_("destination '%s' already exists"), dest)
 
     class DirCleanup(object):
         def __init__(self, dir_):
             self.rmtree = shutil.rmtree
             self.dir_ = dir_
         def close(self):
             self.dir_ = None
         def __del__(self):
             if self.dir_:
                 self.rmtree(self.dir_, True)
 
     src_repo = repository(ui, source)
 
     dest_repo = None
     try:
         dest_repo = repository(ui, dest)
         raise util.Abort(_("destination '%s' already exists." % dest))
     except RepoError:
         dest_repo = repository(ui, dest, create=True)
 
     dest_path = None
     dir_cleanup = None
     if dest_repo.local():
         dest_path = os.path.realpath(dest)
         dir_cleanup = DirCleanup(dest_path)
 
     abspath = source
     copy = False
     if src_repo.local() and dest_repo.local():
         abspath = os.path.abspath(source)
         copy = not pull and not rev
 
     src_lock, dest_lock = None, None
     if copy:
         try:
             # we use a lock here because if we race with commit, we
             # can end up with extra data in the cloned revlogs that's
             # not pointed to by changesets, thus causing verify to
             # fail
             src_lock = src_repo.lock()
         except lock.LockException:
             copy = False
 
     if copy:
         # we lock here to avoid premature writing to the target
         dest_lock = lock.lock(os.path.join(dest_path, ".hg", "lock"))
 
 	# we need to remove the (empty) data dir in dest so copyfiles
 	# can do its work
 	os.rmdir(os.path.join(dest_path, ".hg", "data"))
         files = "data 00manifest.d 00manifest.i 00changelog.d 00changelog.i"
         for f in files.split():
             src = os.path.join(source, ".hg", f)
             dst = os.path.join(dest_path, ".hg", f)
             try:
                 util.copyfiles(src, dst)
             except OSError, inst:
                 if inst.errno != errno.ENOENT:
                     raise
 
 	# we need to re-init the repo after manually copying the data
 	# into it
         dest_repo = repository(ui, dest)
 
     else:
         revs = None
         if rev:
             if not src_repo.local():
                 raise util.Abort(_("clone by revision not supported yet "
                                    "for remote repositories"))
             revs = [src_repo.lookup(r) for r in rev]
 
         if dest_repo.local():
             dest_repo.clone(src_repo, heads=revs, stream=stream)
         elif src_repo.local():
             src_repo.push(dest_repo, revs=revs)
         else:
             raise util.Abort(_("clone from remote to remote not supported"))
 
     if src_lock:
         src_lock.release()
 
     if dest_repo.local():
         fp = dest_repo.opener("hgrc", "w", text=True)
         fp.write("[paths]\n")
         fp.write("default = %s\n" % abspath)
         fp.close()
 
         if dest_lock:
             dest_lock.release()
 
         if update:
             dest_repo.update(dest_repo.changelog.tip())
     if dir_cleanup:
         dir_cleanup.close()
 
     return src_repo, dest_repo

mercurial/hgweb/hgweb_mod.py

@@ -1,957 +1,960 b''
 # hgweb/hgweb_mod.py - Web interface for a repository.
 #
 # Copyright 21 May 2005 - (c) 2005 Jake Edge <[e-mail]>
 # Copyright 2005 Matt Mackall <[e-mail]>
 #
 # This software may be used and distributed according to the terms
 # of the GNU General Public License, incorporated herein by reference.
 
 import os
 import os.path
 import mimetypes
 from mercurial.demandload import demandload
 demandload(globals(), "re zlib ConfigParser mimetools cStringIO sys tempfile")
 demandload(globals(), "mercurial:mdiff,ui,hg,util,archival,streamclone")
 demandload(globals(), "mercurial:templater")
 demandload(globals(), "mercurial.hgweb.common:get_mtime,staticfile")
 from mercurial.node import *
 from mercurial.i18n import gettext as _
 
 def _up(p):
     if p[0] != "/":
         p = "/" + p
     if p[-1] == "/":
         p = p[:-1]
     up = os.path.dirname(p)
     if up == "/":
         return "/"
     return up + "/"
 
 class hgweb(object):
     def __init__(self, repo, name=None):
         if type(repo) == type(""):
             self.repo = hg.repository(ui.ui(), repo)
         else:
             self.repo = repo
 
         self.mtime = -1
         self.reponame = name
         self.archives = 'zip', 'gz', 'bz2'
         self.templatepath = self.repo.ui.config("web", "templates",
                                                 templater.templatepath())
 
     def refresh(self):
         mtime = get_mtime(self.repo.root)
         if mtime != self.mtime:
             self.mtime = mtime
             self.repo = hg.repository(self.repo.ui, self.repo.root)
             self.maxchanges = int(self.repo.ui.config("web", "maxchanges", 10))
             self.maxfiles = int(self.repo.ui.config("web", "maxfiles", 10))
             self.allowpull = self.repo.ui.configbool("web", "allowpull", True)
 
     def archivelist(self, nodeid):
         allowed = self.repo.ui.configlist("web", "allow_archive")
         for i in self.archives:
             if i in allowed or self.repo.ui.configbool("web", "allow" + i):
                 yield {"type" : i, "node" : nodeid, "url": ""}
 
     def listfiles(self, files, mf):
         for f in files[:self.maxfiles]:
             yield self.t("filenodelink", node=hex(mf[f]), file=f)
         if len(files) > self.maxfiles:
             yield self.t("fileellipses")
 
     def listfilediffs(self, files, changeset):
         for f in files[:self.maxfiles]:
             yield self.t("filedifflink", node=hex(changeset), file=f)
         if len(files) > self.maxfiles:
             yield self.t("fileellipses")
 
     def siblings(self, siblings=[], rev=None, hiderev=None, **args):
         if not rev:
             rev = lambda x: ""
         siblings = [s for s in siblings if s != nullid]
         if len(siblings) == 1 and rev(siblings[0]) == hiderev:
             return
         for s in siblings:
             yield dict(node=hex(s), rev=rev(s), **args)
 
     def renamelink(self, fl, node):
         r = fl.renamed(node)
         if r:
             return [dict(file=r[0], node=hex(r[1]))]
         return []
 
     def showtag(self, t1, node=nullid, **args):
         for t in self.repo.nodetags(node):
              yield self.t(t1, tag=t, **args)
 
     def diff(self, node1, node2, files):
         def filterfiles(filters, files):
             l = [x for x in files if x in filters]
 
             for t in filters:
                 if t and t[-1] != os.sep:
                     t += os.sep
                 l += [x for x in files if x.startswith(t)]
             return l
 
         parity = [0]
         def diffblock(diff, f, fn):
             yield self.t("diffblock",
                          lines=prettyprintlines(diff),
                          parity=parity[0],
                          file=f,
                          filenode=hex(fn or nullid))
             parity[0] = 1 - parity[0]
 
         def prettyprintlines(diff):
             for l in diff.splitlines(1):
                 if l.startswith('+'):
                     yield self.t("difflineplus", line=l)
                 elif l.startswith('-'):
                     yield self.t("difflineminus", line=l)
                 elif l.startswith('@'):
                     yield self.t("difflineat", line=l)
                 else:
                     yield self.t("diffline", line=l)
 
         r = self.repo
         cl = r.changelog
         mf = r.manifest
         change1 = cl.read(node1)
         change2 = cl.read(node2)
         mmap1 = mf.read(change1[0])
         mmap2 = mf.read(change2[0])
         date1 = util.datestr(change1[2])
         date2 = util.datestr(change2[2])
 
         modified, added, removed, deleted, unknown = r.changes(node1, node2)
         if files:
             modified, added, removed = map(lambda x: filterfiles(files, x),
                                            (modified, added, removed))
 
         diffopts = self.repo.ui.diffopts()
         showfunc = diffopts['showfunc']
         ignorews = diffopts['ignorews']
         ignorewsamount = diffopts['ignorewsamount']
         ignoreblanklines = diffopts['ignoreblanklines']
         for f in modified:
             to = r.file(f).read(mmap1[f])
             tn = r.file(f).read(mmap2[f])
             yield diffblock(mdiff.unidiff(to, date1, tn, date2, f,
                             showfunc=showfunc, ignorews=ignorews,
                             ignorewsamount=ignorewsamount,
                             ignoreblanklines=ignoreblanklines), f, tn)
         for f in added:
             to = None
             tn = r.file(f).read(mmap2[f])
             yield diffblock(mdiff.unidiff(to, date1, tn, date2, f,
                             showfunc=showfunc, ignorews=ignorews,
                             ignorewsamount=ignorewsamount,
                             ignoreblanklines=ignoreblanklines), f, tn)
         for f in removed:
             to = r.file(f).read(mmap1[f])
             tn = None
             yield diffblock(mdiff.unidiff(to, date1, tn, date2, f,
                             showfunc=showfunc, ignorews=ignorews,
                             ignorewsamount=ignorewsamount,
                             ignoreblanklines=ignoreblanklines), f, tn)
 
     def changelog(self, pos):
         def changenav(**map):
             def seq(factor, maxchanges=None):
                 if maxchanges:
                     yield maxchanges
                     if maxchanges >= 20 and maxchanges <= 40:
                         yield 50
                 else:
                     yield 1 * factor
                     yield 3 * factor
                 for f in seq(factor * 10):
                     yield f
 
             l = []
             last = 0
             for f in seq(1, self.maxchanges):
                 if f < self.maxchanges or f <= last:
                     continue
                 if f > count:
                     break
                 last = f
                 r = "%d" % f
                 if pos + f < count:
                     l.append(("+" + r, pos + f))
                 if pos - f >= 0:
                     l.insert(0, ("-" + r, pos - f))
 
             yield {"rev": 0, "label": "(0)"}
 
             for label, rev in l:
                 yield {"label": label, "rev": rev}
 
             yield {"label": "tip", "rev": "tip"}
 
         def changelist(**map):
             parity = (start - end) & 1
             cl = self.repo.changelog
             l = [] # build a list in forward order for efficiency
             for i in range(start, end):
                 n = cl.node(i)
                 changes = cl.read(n)
                 hn = hex(n)
 
                 l.insert(0, {"parity": parity,
                              "author": changes[1],
                              "parent": self.siblings(cl.parents(n), cl.rev,
                                                      cl.rev(n) - 1),
                              "child": self.siblings(cl.children(n), cl.rev,
                                                     cl.rev(n) + 1),
                              "changelogtag": self.showtag("changelogtag",n),
                              "manifest": hex(changes[0]),
                              "desc": changes[4],
                              "date": changes[2],
                              "files": self.listfilediffs(changes[3], n),
                              "rev": i,
                              "node": hn})
                 parity = 1 - parity
 
             for e in l:
                 yield e
 
         cl = self.repo.changelog
         mf = cl.read(cl.tip())[0]
         count = cl.count()
         start = max(0, pos - self.maxchanges + 1)
         end = min(count, start + self.maxchanges)
         pos = end - 1
 
         yield self.t('changelog',
                      changenav=changenav,
                      manifest=hex(mf),
                      rev=pos, changesets=count, entries=changelist,
                      archives=self.archivelist("tip"))
 
     def search(self, query):
 
         def changelist(**map):
             cl = self.repo.changelog
             count = 0
             qw = query.lower().split()
 
             def revgen():
                 for i in range(cl.count() - 1, 0, -100):
                     l = []
                     for j in range(max(0, i - 100), i):
                         n = cl.node(j)
                         changes = cl.read(n)
                         l.append((n, j, changes))
                     l.reverse()
                     for e in l:
                         yield e
 
             for n, i, changes in revgen():
                 miss = 0
                 for q in qw:
                     if not (q in changes[1].lower() or
                             q in changes[4].lower() or
                             q in " ".join(changes[3][:20]).lower()):
                         miss = 1
                         break
                 if miss:
                     continue
 
                 count += 1
                 hn = hex(n)
 
                 yield self.t('searchentry',
                              parity=count & 1,
                              author=changes[1],
                              parent=self.siblings(cl.parents(n), cl.rev),
                              child=self.siblings(cl.children(n), cl.rev),
                              changelogtag=self.showtag("changelogtag",n),
                              manifest=hex(changes[0]),
                              desc=changes[4],
                              date=changes[2],
                              files=self.listfilediffs(changes[3], n),
                              rev=i,
                              node=hn)
 
                 if count >= self.maxchanges:
                     break
 
         cl = self.repo.changelog
         mf = cl.read(cl.tip())[0]
 
         yield self.t('search',
                      query=query,
                      manifest=hex(mf),
                      entries=changelist)
 
     def changeset(self, nodeid):
         cl = self.repo.changelog
         n = self.repo.lookup(nodeid)
         nodeid = hex(n)
         changes = cl.read(n)
         p1 = cl.parents(n)[0]
 
         files = []
         mf = self.repo.manifest.read(changes[0])
         for f in changes[3]:
             files.append(self.t("filenodelink",
                                 filenode=hex(mf.get(f, nullid)), file=f))
 
         def diff(**map):
             yield self.diff(p1, n, None)
 
         yield self.t('changeset',
                      diff=diff,
                      rev=cl.rev(n),
                      node=nodeid,
                      parent=self.siblings(cl.parents(n), cl.rev),
                      child=self.siblings(cl.children(n), cl.rev),
                      changesettag=self.showtag("changesettag",n),
                      manifest=hex(changes[0]),
                      author=changes[1],
                      desc=changes[4],
                      date=changes[2],
                      files=files,
                      archives=self.archivelist(nodeid))
 
     def filelog(self, f, filenode):
         cl = self.repo.changelog
         fl = self.repo.file(f)
         filenode = hex(fl.lookup(filenode))
         count = fl.count()
 
         def entries(**map):
             l = []
             parity = (count - 1) & 1
 
             for i in range(count):
                 n = fl.node(i)
                 lr = fl.linkrev(n)
                 cn = cl.node(lr)
                 cs = cl.read(cl.node(lr))
 
                 l.insert(0, {"parity": parity,
                              "filenode": hex(n),
                              "filerev": i,
                              "file": f,
                              "node": hex(cn),
                              "author": cs[1],
                              "date": cs[2],
                              "rename": self.renamelink(fl, n),
                              "parent": self.siblings(fl.parents(n),
                                                      fl.rev, file=f),
                              "child": self.siblings(fl.children(n),
                                                     fl.rev, file=f),
                              "desc": cs[4]})
                 parity = 1 - parity
 
             for e in l:
                 yield e
 
         yield self.t("filelog", file=f, filenode=filenode, entries=entries)
 
     def filerevision(self, f, node):
         fl = self.repo.file(f)
         n = fl.lookup(node)
         node = hex(n)
         text = fl.read(n)
         changerev = fl.linkrev(n)
         cl = self.repo.changelog
         cn = cl.node(changerev)
         cs = cl.read(cn)
         mfn = cs[0]
 
         mt = mimetypes.guess_type(f)[0]
         rawtext = text
         if util.binary(text):
             mt = mt or 'application/octet-stream'
             text = "(binary:%s)" % mt
         mt = mt or 'text/plain'
 
         def lines():
             for l, t in enumerate(text.splitlines(1)):
                 yield {"line": t,
                        "linenumber": "% 6d" % (l + 1),
                        "parity": l & 1}
 
         yield self.t("filerevision",
                      file=f,
                      filenode=node,
                      path=_up(f),
                      text=lines(),
                      raw=rawtext,
                      mimetype=mt,
                      rev=changerev,
                      node=hex(cn),
                      manifest=hex(mfn),
                      author=cs[1],
                      date=cs[2],
                      parent=self.siblings(fl.parents(n), fl.rev, file=f),
                      child=self.siblings(fl.children(n), fl.rev, file=f),
                      rename=self.renamelink(fl, n),
                      permissions=self.repo.manifest.readflags(mfn)[f])
 
     def fileannotate(self, f, node):
         bcache = {}
         ncache = {}
         fl = self.repo.file(f)
         n = fl.lookup(node)
         node = hex(n)
         changerev = fl.linkrev(n)
 
         cl = self.repo.changelog
         cn = cl.node(changerev)
         cs = cl.read(cn)
         mfn = cs[0]
 
         def annotate(**map):
             parity = 1
             last = None
             for r, l in fl.annotate(n):
                 try:
                     cnode = ncache[r]
                 except KeyError:
                     cnode = ncache[r] = self.repo.changelog.node(r)
 
                 try:
                     name = bcache[r]
                 except KeyError:
                     cl = self.repo.changelog.read(cnode)
                     bcache[r] = name = self.repo.ui.shortuser(cl[1])
 
                 if last != cnode:
                     parity = 1 - parity
                     last = cnode
 
                 yield {"parity": parity,
                        "node": hex(cnode),
                        "rev": r,
                        "author": name,
                        "file": f,
                        "line": l}
 
         yield self.t("fileannotate",
                      file=f,
                      filenode=node,
                      annotate=annotate,
                      path=_up(f),
                      rev=changerev,
                      node=hex(cn),
                      manifest=hex(mfn),
                      author=cs[1],
                      date=cs[2],
                      rename=self.renamelink(fl, n),
                      parent=self.siblings(fl.parents(n), fl.rev, file=f),
                      child=self.siblings(fl.children(n), fl.rev, file=f),
                      permissions=self.repo.manifest.readflags(mfn)[f])
 
     def manifest(self, mnode, path):
         man = self.repo.manifest
         mn = man.lookup(mnode)
         mnode = hex(mn)
         mf = man.read(mn)
         rev = man.rev(mn)
         changerev = man.linkrev(mn)
         node = self.repo.changelog.node(changerev)
         mff = man.readflags(mn)
 
         files = {}
 
         p = path[1:]
         if p and p[-1] != "/":
             p += "/"
         l = len(p)
 
         for f,n in mf.items():
             if f[:l] != p:
                 continue
             remain = f[l:]
             if "/" in remain:
                 short = remain[:remain.index("/") + 1] # bleah
                 files[short] = (f, None)
             else:
                 short = os.path.basename(remain)
                 files[short] = (f, n)
 
         def filelist(**map):
             parity = 0
             fl = files.keys()
             fl.sort()
             for f in fl:
                 full, fnode = files[f]
                 if not fnode:
                     continue
 
                 yield {"file": full,
                        "manifest": mnode,
                        "filenode": hex(fnode),
                        "parity": parity,
                        "basename": f,
                        "permissions": mff[full]}
                 parity = 1 - parity
 
         def dirlist(**map):
             parity = 0
             fl = files.keys()
             fl.sort()
             for f in fl:
                 full, fnode = files[f]
                 if fnode:
                     continue
 
                 yield {"parity": parity,
                        "path": os.path.join(path, f),
                        "manifest": mnode,
                        "basename": f[:-1]}
                 parity = 1 - parity
 
         yield self.t("manifest",
                      manifest=mnode,
                      rev=rev,
                      node=hex(node),
                      path=path,
                      up=_up(path),
                      fentries=filelist,
                      dentries=dirlist,
                      archives=self.archivelist(hex(node)))
 
     def tags(self):
         cl = self.repo.changelog
         mf = cl.read(cl.tip())[0]
 
         i = self.repo.tagslist()
         i.reverse()
 
         def entries(notip=False, **map):
             parity = 0
             for k,n in i:
                 if notip and k == "tip": continue
                 yield {"parity": parity,
                        "tag": k,
                        "tagmanifest": hex(cl.read(n)[0]),
                        "date": cl.read(n)[2],
                        "node": hex(n)}
                 parity = 1 - parity
 
         yield self.t("tags",
                      manifest=hex(mf),
                      entries=lambda **x: entries(False, **x),
                      entriesnotip=lambda **x: entries(True, **x))
 
     def summary(self):
         cl = self.repo.changelog
         mf = cl.read(cl.tip())[0]
 
         i = self.repo.tagslist()
         i.reverse()
 
         def tagentries(**map):
             parity = 0
             count = 0
             for k,n in i:
                 if k == "tip": # skip tip
                     continue;
 
                 count += 1
                 if count > 10: # limit to 10 tags
                     break;
 
                 c = cl.read(n)
                 m = c[0]
                 t = c[2]
 
                 yield self.t("tagentry",
                              parity = parity,
                              tag = k,
                              node = hex(n),
                              date = t,
                              tagmanifest = hex(m))
                 parity = 1 - parity
 
         def changelist(**map):
             parity = 0
             cl = self.repo.changelog
             l = [] # build a list in forward order for efficiency
             for i in range(start, end):
                 n = cl.node(i)
                 changes = cl.read(n)
                 hn = hex(n)
                 t = changes[2]
 
                 l.insert(0, self.t(
                     'shortlogentry',
                     parity = parity,
                     author = changes[1],
                     manifest = hex(changes[0]),
                     desc = changes[4],
                     date = t,
                     rev = i,
                     node = hn))
                 parity = 1 - parity
 
             yield l
 
         cl = self.repo.changelog
         mf = cl.read(cl.tip())[0]
         count = cl.count()
         start = max(0, count - self.maxchanges)
         end = min(count, start + self.maxchanges)
 
         yield self.t("summary",
                  desc = self.repo.ui.config("web", "description", "unknown"),
                  owner = (self.repo.ui.config("ui", "username") or # preferred
                           self.repo.ui.config("web", "contact") or # deprecated
                           self.repo.ui.config("web", "author", "unknown")), # also
                  lastchange = (0, 0), # FIXME
                  manifest = hex(mf),
                  tags = tagentries,
                  shortlog = changelist)
 
     def filediff(self, file, changeset):
         cl = self.repo.changelog
         n = self.repo.lookup(changeset)
         changeset = hex(n)
         p1 = cl.parents(n)[0]
         cs = cl.read(n)
         mf = self.repo.manifest.read(cs[0])
 
         def diff(**map):
             yield self.diff(p1, n, [file])
 
         yield self.t("filediff",
                      file=file,
                      filenode=hex(mf.get(file, nullid)),
                      node=changeset,
                      rev=self.repo.changelog.rev(n),
                      parent=self.siblings(cl.parents(n), cl.rev),
                      child=self.siblings(cl.children(n), cl.rev),
                      diff=diff)
 
     archive_specs = {
         'bz2': ('application/x-tar', 'tbz2', '.tar.bz2', None),
         'gz': ('application/x-tar', 'tgz', '.tar.gz', None),
         'zip': ('application/zip', 'zip', '.zip', None),
         }
 
     def archive(self, req, cnode, type_):
         reponame = re.sub(r"\W+", "-", os.path.basename(self.reponame))
         name = "%s-%s" % (reponame, short(cnode))
         mimetype, artype, extension, encoding = self.archive_specs[type_]
         headers = [('Content-type', mimetype),
                    ('Content-disposition', 'attachment; filename=%s%s' %
                     (name, extension))]
         if encoding:
             headers.append(('Content-encoding', encoding))
         req.header(headers)
         archival.archive(self.repo, req.out, cnode, artype, prefix=name)
 
     # add tags to things
     # tags -> list of changesets corresponding to tags
     # find tag, changeset, file
 
     def cleanpath(self, path):
         p = util.normpath(path)
         if p[:2] == "..":
             raise Exception("suspicious path")
         return p
 
     def run(self):
         if not os.environ.get('GATEWAY_INTERFACE', '').startswith("CGI/1."):
             raise RuntimeError("This function is only intended to be called while running as a CGI script.")
         import mercurial.hgweb.wsgicgi as wsgicgi
         from request import wsgiapplication
         def make_web_app():
             return self
         wsgicgi.launch(wsgiapplication(make_web_app))
 
     def run_wsgi(self, req):
         def header(**map):
             header_file = cStringIO.StringIO(''.join(self.t("header", **map)))
             msg = mimetools.Message(header_file, 0)
             req.header(msg.items())
             yield header_file.read()
 
         def rawfileheader(**map):
             req.header([('Content-type', map['mimetype']),
                         ('Content-disposition', 'filename=%s' % map['file']),
                         ('Content-length', str(len(map['raw'])))])
             yield ''
 
         def footer(**map):
             yield self.t("footer",
                          motd=self.repo.ui.config("web", "motd", ""),
                          **map)
 
         def expand_form(form):
             shortcuts = {
                 'cl': [('cmd', ['changelog']), ('rev', None)],
                 'cs': [('cmd', ['changeset']), ('node', None)],
                 'f': [('cmd', ['file']), ('filenode', None)],
                 'fl': [('cmd', ['filelog']), ('filenode', None)],
                 'fd': [('cmd', ['filediff']), ('node', None)],
                 'fa': [('cmd', ['annotate']), ('filenode', None)],
                 'mf': [('cmd', ['manifest']), ('manifest', None)],
                 'ca': [('cmd', ['archive']), ('node', None)],
                 'tags': [('cmd', ['tags'])],
                 'tip': [('cmd', ['changeset']), ('node', ['tip'])],
                 'static': [('cmd', ['static']), ('file', None)]
             }
 
             for k in shortcuts.iterkeys():
                 if form.has_key(k):
                     for name, value in shortcuts[k]:
                         if value is None:
                             value = form[k]
                         form[name] = value
                     del form[k]
 
         self.refresh()
 
         expand_form(req.form)
 
         m = os.path.join(self.templatepath, "map")
         style = self.repo.ui.config("web", "style", "")
         if req.form.has_key('style'):
             style = req.form['style'][0]
         if style:
             b = os.path.basename("map-" + style)
             p = os.path.join(self.templatepath, b)
             if os.path.isfile(p):
                 m = p
 
         port = req.env["SERVER_PORT"]
         port = port != "80" and (":" + port) or ""
         uri = req.env["REQUEST_URI"]
         if "?" in uri:
             uri = uri.split("?")[0]
         url = "http://%s%s%s" % (req.env["SERVER_NAME"], port, uri)
         if not self.reponame:
             self.reponame = (self.repo.ui.config("web", "name")
                              or uri.strip('/') or self.repo.root)
 
         self.t = templater.templater(m, templater.common_filters,
                                      defaults={"url": url,
                                                "repo": self.reponame,
                                                "header": header,
                                                "footer": footer,
                                                "rawfileheader": rawfileheader,
                                                })
 
         if not req.form.has_key('cmd'):
             req.form['cmd'] = [self.t.cache['default'],]
 
         cmd = req.form['cmd'][0]
 
         method = getattr(self, 'do_' + cmd, None)
         if method:
             method(req)
         else:
             req.write(self.t("error"))
 
     def do_changelog(self, req):
         hi = self.repo.changelog.count() - 1
         if req.form.has_key('rev'):
             hi = req.form['rev'][0]
             try:
                 hi = self.repo.changelog.rev(self.repo.lookup(hi))
             except hg.RepoError:
                 req.write(self.search(hi)) # XXX redirect to 404 page?
                 return
 
         req.write(self.changelog(hi))
 
     def do_changeset(self, req):
         req.write(self.changeset(req.form['node'][0]))
 
     def do_manifest(self, req):
         req.write(self.manifest(req.form['manifest'][0],
                                 self.cleanpath(req.form['path'][0])))
 
     def do_tags(self, req):
         req.write(self.tags())
 
     def do_summary(self, req):
         req.write(self.summary())
 
     def do_filediff(self, req):
         req.write(self.filediff(self.cleanpath(req.form['file'][0]),
                                 req.form['node'][0]))
 
     def do_file(self, req):
         req.write(self.filerevision(self.cleanpath(req.form['file'][0]),
                                     req.form['filenode'][0]))
 
     def do_annotate(self, req):
         req.write(self.fileannotate(self.cleanpath(req.form['file'][0]),
                                     req.form['filenode'][0]))
 
     def do_filelog(self, req):
         req.write(self.filelog(self.cleanpath(req.form['file'][0]),
                                req.form['filenode'][0]))
 
     def do_heads(self, req):
         resp = " ".join(map(hex, self.repo.heads())) + "\n"
         req.httphdr("application/mercurial-0.1", length=len(resp))
         req.write(resp)
 
     def do_branches(self, req):
         nodes = []
         if req.form.has_key('nodes'):
             nodes = map(bin, req.form['nodes'][0].split(" "))
         resp = cStringIO.StringIO()
         for b in self.repo.branches(nodes):
             resp.write(" ".join(map(hex, b)) + "\n")
         resp = resp.getvalue()
         req.httphdr("application/mercurial-0.1", length=len(resp))
         req.write(resp)
 
     def do_between(self, req):
         nodes = []
         if req.form.has_key('pairs'):
             pairs = [map(bin, p.split("-"))
                      for p in req.form['pairs'][0].split(" ")]
         resp = cStringIO.StringIO()
         for b in self.repo.between(pairs):
             resp.write(" ".join(map(hex, b)) + "\n")
         resp = resp.getvalue()
         req.httphdr("application/mercurial-0.1", length=len(resp))
         req.write(resp)
 
     def do_changegroup(self, req):
         req.httphdr("application/mercurial-0.1")
         nodes = []
         if not self.allowpull:
             return
 
         if req.form.has_key('roots'):
             nodes = map(bin, req.form['roots'][0].split(" "))
 
         z = zlib.compressobj()
         f = self.repo.changegroup(nodes, 'serve')
         while 1:
             chunk = f.read(4096)
             if not chunk:
                 break
             req.write(z.compress(chunk))
 
         req.write(z.flush())
 
     def do_archive(self, req):
         changeset = self.repo.lookup(req.form['node'][0])
         type_ = req.form['type'][0]
         allowed = self.repo.ui.configlist("web", "allow_archive")
         if (type_ in self.archives and (type_ in allowed or
             self.repo.ui.configbool("web", "allow" + type_, False))):
             self.archive(req, changeset, type_)
             return
 
         req.write(self.t("error"))
 
     def do_static(self, req):
         fname = req.form['file'][0]
         static = self.repo.ui.config("web", "static",
                                      os.path.join(self.templatepath,
                                                   "static"))
         req.write(staticfile(static, fname, req)
                   or self.t("error", error="%r not found" % fname))
 
     def do_capabilities(self, req):
-        resp = 'unbundle stream=%d' % (self.repo.revlogversion,)
+        caps = ['unbundle']
+        if self.repo.ui.configbool('server', 'stream'):
+            caps.append('stream=%d' % self.repo.revlogversion)
+        resp = ' '.join(caps)
         req.httphdr("application/mercurial-0.1", length=len(resp))
         req.write(resp)
 
     def check_perm(self, req, op, default):
         '''check permission for operation based on user auth.
         return true if op allowed, else false.
         default is policy to use if no config given.'''
 
         user = req.env.get('REMOTE_USER')
 
         deny = self.repo.ui.configlist('web', 'deny_' + op)
         if deny and (not user or deny == ['*'] or user in deny):
             return False
 
         allow = self.repo.ui.configlist('web', 'allow_' + op)
         return (allow and (allow == ['*'] or user in allow)) or default
 
     def do_unbundle(self, req):
         def bail(response, headers={}):
             length = int(req.env['CONTENT_LENGTH'])
             for s in util.filechunkiter(req, limit=length):
                 # drain incoming bundle, else client will not see
                 # response when run outside cgi script
                 pass
             req.httphdr("application/mercurial-0.1", headers=headers)
             req.write('0\n')
             req.write(response)
 
         # require ssl by default, auth info cannot be sniffed and
         # replayed
         ssl_req = self.repo.ui.configbool('web', 'push_ssl', True)
         if ssl_req and not req.env.get('HTTPS'):
             bail(_('ssl required\n'))
             return
 
         # do not allow push unless explicitly allowed
         if not self.check_perm(req, 'push', False):
             bail(_('push not authorized\n'),
                  headers={'status': '401 Unauthorized'})
             return
 
         req.httphdr("application/mercurial-0.1")
 
         their_heads = req.form['heads'][0].split(' ')
 
         def check_heads():
             heads = map(hex, self.repo.heads())
             return their_heads == [hex('force')] or their_heads == heads
 
         # fail early if possible
         if not check_heads():
             bail(_('unsynced changes\n'))
             return
 
         # do not lock repo until all changegroup data is
         # streamed. save to temporary file.
 
         fd, tempname = tempfile.mkstemp(prefix='hg-unbundle-')
         fp = os.fdopen(fd, 'wb+')
         try:
             length = int(req.env['CONTENT_LENGTH'])
             for s in util.filechunkiter(req, limit=length):
                 fp.write(s)
 
             lock = self.repo.lock()
             try:
                 if not check_heads():
                     req.write('0\n')
                     req.write(_('unsynced changes\n'))
                     return
 
                 fp.seek(0)
 
                 # send addchangegroup output to client
 
                 old_stdout = sys.stdout
                 sys.stdout = cStringIO.StringIO()
 
                 try:
                     ret = self.repo.addchangegroup(fp, 'serve')
                 finally:
                     val = sys.stdout.getvalue()
                     sys.stdout = old_stdout
                 req.write('%d\n' % ret)
                 req.write(val)
             finally:
                 lock.release()
         finally:
             fp.close()
             os.unlink(tempname)
 
     def do_stream_out(self, req):
         req.httphdr("application/mercurial-0.1")
         streamclone.stream_out(self.repo, req)

mercurial/localrepo.py

@@ -1,2254 +1,2258 b''
 # localrepo.py - read/write repository class for mercurial
 #
 # Copyright 2005 Matt Mackall <[e-mail]>
 #
 # This software may be used and distributed according to the terms
 # of the GNU General Public License, incorporated herein by reference.
 
 from node import *
 from i18n import gettext as _
 from demandload import *
 import repo
 demandload(globals(), "appendfile changegroup")
 demandload(globals(), "changelog dirstate filelog manifest context")
 demandload(globals(), "re lock transaction tempfile stat mdiff errno ui")
 demandload(globals(), "os revlog time util")
 
 class localrepository(repo.repository):
     capabilities = ()
 
     def __del__(self):
         self.transhandle = None
     def __init__(self, parentui, path=None, create=0):
         repo.repository.__init__(self)
         if not path:
             p = os.getcwd()
             while not os.path.isdir(os.path.join(p, ".hg")):
                 oldp = p
                 p = os.path.dirname(p)
                 if p == oldp:
                     raise repo.RepoError(_("no repo found"))
             path = p
         self.path = os.path.join(path, ".hg")
 
         if not create and not os.path.isdir(self.path):
             raise repo.RepoError(_("repository %s not found") % path)
 
         self.root = os.path.abspath(path)
         self.origroot = path
         self.ui = ui.ui(parentui=parentui)
         self.opener = util.opener(self.path)
         self.wopener = util.opener(self.root)
 
         try:
             self.ui.readconfig(self.join("hgrc"), self.root)
         except IOError:
             pass
 
         v = self.ui.revlogopts
         self.revlogversion = int(v.get('format', revlog.REVLOG_DEFAULT_FORMAT))
         self.revlogv1 = self.revlogversion != revlog.REVLOGV0
         fl = v.get('flags', None)
         flags = 0
         if fl != None:
             for x in fl.split():
                 flags |= revlog.flagstr(x)
         elif self.revlogv1:
             flags = revlog.REVLOG_DEFAULT_FLAGS
 
         v = self.revlogversion | flags
         self.manifest = manifest.manifest(self.opener, v)
         self.changelog = changelog.changelog(self.opener, v)
 
         # the changelog might not have the inline index flag
         # on.  If the format of the changelog is the same as found in
         # .hgrc, apply any flags found in the .hgrc as well.
         # Otherwise, just version from the changelog
         v = self.changelog.version
         if v == self.revlogversion:
             v |= flags
         self.revlogversion = v
 
         self.tagscache = None
         self.nodetagscache = None
         self.encodepats = None
         self.decodepats = None
         self.transhandle = None
 
         if create:
             if not os.path.exists(path):
                 os.mkdir(path)
             os.mkdir(self.path)
             os.mkdir(self.join("data"))
 
         self.dirstate = dirstate.dirstate(self.opener, self.ui, self.root)
 
     def hook(self, name, throw=False, **args):
         def callhook(hname, funcname):
             '''call python hook. hook is callable object, looked up as
             name in python module. if callable returns "true", hook
             fails, else passes. if hook raises exception, treated as
             hook failure. exception propagates if throw is "true".
 
             reason for "true" meaning "hook failed" is so that
             unmodified commands (e.g. mercurial.commands.update) can
             be run as hooks without wrappers to convert return values.'''
 
             self.ui.note(_("calling hook %s: %s\n") % (hname, funcname))
             d = funcname.rfind('.')
             if d == -1:
                 raise util.Abort(_('%s hook is invalid ("%s" not in a module)')
                                  % (hname, funcname))
             modname = funcname[:d]
             try:
                 obj = __import__(modname)
             except ImportError:
                 try:
                     # extensions are loaded with hgext_ prefix
                     obj = __import__("hgext_%s" % modname)
                 except ImportError:
                     raise util.Abort(_('%s hook is invalid '
                                        '(import of "%s" failed)') %
                                      (hname, modname))
             try:
                 for p in funcname.split('.')[1:]:
                     obj = getattr(obj, p)
             except AttributeError, err:
                 raise util.Abort(_('%s hook is invalid '
                                    '("%s" is not defined)') %
                                  (hname, funcname))
             if not callable(obj):
                 raise util.Abort(_('%s hook is invalid '
                                    '("%s" is not callable)') %
                                  (hname, funcname))
             try:
                 r = obj(ui=self.ui, repo=self, hooktype=name, **args)
             except (KeyboardInterrupt, util.SignalInterrupt):
                 raise
             except Exception, exc:
                 if isinstance(exc, util.Abort):
                     self.ui.warn(_('error: %s hook failed: %s\n') %
                                  (hname, exc.args[0] % exc.args[1:]))
                 else:
                     self.ui.warn(_('error: %s hook raised an exception: '
                                    '%s\n') % (hname, exc))
                 if throw:
                     raise
                 self.ui.print_exc()
                 return True
             if r:
                 if throw:
                     raise util.Abort(_('%s hook failed') % hname)
                 self.ui.warn(_('warning: %s hook failed\n') % hname)
             return r
 
         def runhook(name, cmd):
             self.ui.note(_("running hook %s: %s\n") % (name, cmd))
             env = dict([('HG_' + k.upper(), v) for k, v in args.iteritems()])
             r = util.system(cmd, environ=env, cwd=self.root)
             if r:
                 desc, r = util.explain_exit(r)
                 if throw:
                     raise util.Abort(_('%s hook %s') % (name, desc))
                 self.ui.warn(_('warning: %s hook %s\n') % (name, desc))
             return r
 
         r = False
         hooks = [(hname, cmd) for hname, cmd in self.ui.configitems("hooks")
                  if hname.split(".", 1)[0] == name and cmd]
         hooks.sort()
         for hname, cmd in hooks:
             if cmd.startswith('python:'):
                 r = callhook(hname, cmd[7:].strip()) or r
             else:
                 r = runhook(hname, cmd) or r
         return r
 
     tag_disallowed = ':\r\n'
 
     def tag(self, name, node, local=False, message=None, user=None, date=None):
         '''tag a revision with a symbolic name.
 
         if local is True, the tag is stored in a per-repository file.
         otherwise, it is stored in the .hgtags file, and a new
         changeset is committed with the change.
 
         keyword arguments:
 
         local: whether to store tag in non-version-controlled file
         (default False)
 
         message: commit message to use if committing
 
         user: name of user to use if committing
 
         date: date tuple to use if committing'''
 
         for c in self.tag_disallowed:
             if c in name:
                 raise util.Abort(_('%r cannot be used in a tag name') % c)
 
         self.hook('pretag', throw=True, node=node, tag=name, local=local)
 
         if local:
             self.opener('localtags', 'a').write('%s %s\n' % (node, name))
             self.hook('tag', node=node, tag=name, local=local)
             return
 
         for x in self.changes():
             if '.hgtags' in x:
                 raise util.Abort(_('working copy of .hgtags is changed '
                                    '(please commit .hgtags manually)'))
 
         self.wfile('.hgtags', 'ab').write('%s %s\n' % (node, name))
         if self.dirstate.state('.hgtags') == '?':
             self.add(['.hgtags'])
 
         if not message:
             message = _('Added tag %s for changeset %s') % (name, node)
 
         self.commit(['.hgtags'], message, user, date)
         self.hook('tag', node=node, tag=name, local=local)
 
     def tags(self):
         '''return a mapping of tag to node'''
         if not self.tagscache:
             self.tagscache = {}
 
             def parsetag(line, context):
                 if not line:
                     return
                 s = l.split(" ", 1)
                 if len(s) != 2:
                     self.ui.warn(_("%s: cannot parse entry\n") % context)
                     return
                 node, key = s
                 key = key.strip()
                 try:
                     bin_n = bin(node)
                 except TypeError:
                     self.ui.warn(_("%s: node '%s' is not well formed\n") %
                                  (context, node))
                     return
                 if bin_n not in self.changelog.nodemap:
                     self.ui.warn(_("%s: tag '%s' refers to unknown node\n") %
                                  (context, key))
                     return
                 self.tagscache[key] = bin_n
 
             # read the tags file from each head, ending with the tip,
             # and add each tag found to the map, with "newer" ones
             # taking precedence
             heads = self.heads()
             heads.reverse()
             fl = self.file(".hgtags")
             for node in heads:
                 change = self.changelog.read(node)
                 rev = self.changelog.rev(node)
                 fn, ff = self.manifest.find(change[0], '.hgtags')
                 if fn is None: continue
                 count = 0
                 for l in fl.read(fn).splitlines():
                     count += 1
                     parsetag(l, _(".hgtags (rev %d:%s), line %d") %
                              (rev, short(node), count))
             try:
                 f = self.opener("localtags")
                 count = 0
                 for l in f:
                     count += 1
                     parsetag(l, _("localtags, line %d") % count)
             except IOError:
                 pass
 
             self.tagscache['tip'] = self.changelog.tip()
 
         return self.tagscache
 
     def tagslist(self):
         '''return a list of tags ordered by revision'''
         l = []
         for t, n in self.tags().items():
             try:
                 r = self.changelog.rev(n)
             except:
                 r = -2 # sort to the beginning of the list if unknown
             l.append((r, t, n))
         l.sort()
         return [(t, n) for r, t, n in l]
 
     def nodetags(self, node):
         '''return the tags associated with a node'''
         if not self.nodetagscache:
             self.nodetagscache = {}
             for t, n in self.tags().items():
                 self.nodetagscache.setdefault(n, []).append(t)
         return self.nodetagscache.get(node, [])
 
     def lookup(self, key):
         try:
             return self.tags()[key]
         except KeyError:
             try:
                 return self.changelog.lookup(key)
             except:
                 raise repo.RepoError(_("unknown revision '%s'") % key)
 
     def dev(self):
         return os.lstat(self.path).st_dev
 
     def local(self):
         return True
 
     def join(self, f):
         return os.path.join(self.path, f)
 
     def wjoin(self, f):
         return os.path.join(self.root, f)
 
     def file(self, f):
         if f[0] == '/':
             f = f[1:]
         return filelog.filelog(self.opener, f, self.revlogversion)
 
     def changectx(self, changeid):
         return context.changectx(self, changeid)
 
     def filectx(self, path, changeid=None, fileid=None):
         """changeid can be a changeset revision, node, or tag.
            fileid can be a file revision or node."""
         return context.filectx(self, path, changeid, fileid)
 
     def getcwd(self):
         return self.dirstate.getcwd()
 
     def wfile(self, f, mode='r'):
         return self.wopener(f, mode)
 
     def wread(self, filename):
         if self.encodepats == None:
             l = []
             for pat, cmd in self.ui.configitems("encode"):
                 mf = util.matcher(self.root, "", [pat], [], [])[1]
                 l.append((mf, cmd))
             self.encodepats = l
 
         data = self.wopener(filename, 'r').read()
 
         for mf, cmd in self.encodepats:
             if mf(filename):
                 self.ui.debug(_("filtering %s through %s\n") % (filename, cmd))
                 data = util.filter(data, cmd)
                 break
 
         return data
 
     def wwrite(self, filename, data, fd=None):
         if self.decodepats == None:
             l = []
             for pat, cmd in self.ui.configitems("decode"):
                 mf = util.matcher(self.root, "", [pat], [], [])[1]
                 l.append((mf, cmd))
             self.decodepats = l
 
         for mf, cmd in self.decodepats:
             if mf(filename):
                 self.ui.debug(_("filtering %s through %s\n") % (filename, cmd))
                 data = util.filter(data, cmd)
                 break
 
         if fd:
             return fd.write(data)
         return self.wopener(filename, 'w').write(data)
 
     def transaction(self):
         tr = self.transhandle
         if tr != None and tr.running():
             return tr.nest()
 
         # save dirstate for rollback
         try:
             ds = self.opener("dirstate").read()
         except IOError:
             ds = ""
         self.opener("journal.dirstate", "w").write(ds)
 
         tr = transaction.transaction(self.ui.warn, self.opener,
                                        self.join("journal"),
                                        aftertrans(self.path))
         self.transhandle = tr
         return tr
 
     def recover(self):
         l = self.lock()
         if os.path.exists(self.join("journal")):
             self.ui.status(_("rolling back interrupted transaction\n"))
             transaction.rollback(self.opener, self.join("journal"))
             self.reload()
             return True
         else:
             self.ui.warn(_("no interrupted transaction available\n"))
             return False
 
     def rollback(self, wlock=None):
         if not wlock:
             wlock = self.wlock()
         l = self.lock()
         if os.path.exists(self.join("undo")):
             self.ui.status(_("rolling back last transaction\n"))
             transaction.rollback(self.opener, self.join("undo"))
             util.rename(self.join("undo.dirstate"), self.join("dirstate"))
             self.reload()
             self.wreload()
         else:
             self.ui.warn(_("no rollback information available\n"))
 
     def wreload(self):
         self.dirstate.read()
 
     def reload(self):
         self.changelog.load()
         self.manifest.load()
         self.tagscache = None
         self.nodetagscache = None
 
     def do_lock(self, lockname, wait, releasefn=None, acquirefn=None,
                 desc=None):
         try:
             l = lock.lock(self.join(lockname), 0, releasefn, desc=desc)
         except lock.LockHeld, inst:
             if not wait:
                 raise
             self.ui.warn(_("waiting for lock on %s held by %s\n") %
                          (desc, inst.args[0]))
             # default to 600 seconds timeout
             l = lock.lock(self.join(lockname),
                           int(self.ui.config("ui", "timeout") or 600),
                           releasefn, desc=desc)
         if acquirefn:
             acquirefn()
         return l
 
     def lock(self, wait=1):
         return self.do_lock("lock", wait, acquirefn=self.reload,
                             desc=_('repository %s') % self.origroot)
 
     def wlock(self, wait=1):
         return self.do_lock("wlock", wait, self.dirstate.write,
                             self.wreload,
                             desc=_('working directory of %s') % self.origroot)
 
     def checkfilemerge(self, filename, text, filelog, manifest1, manifest2):
         "determine whether a new filenode is needed"
         fp1 = manifest1.get(filename, nullid)
         fp2 = manifest2.get(filename, nullid)
 
         if fp2 != nullid:
             # is one parent an ancestor of the other?
             fpa = filelog.ancestor(fp1, fp2)
             if fpa == fp1:
                 fp1, fp2 = fp2, nullid
             elif fpa == fp2:
                 fp2 = nullid
 
             # is the file unmodified from the parent? report existing entry
             if fp2 == nullid and text == filelog.read(fp1):
                 return (fp1, None, None)
 
         return (None, fp1, fp2)
 
     def rawcommit(self, files, text, user, date, p1=None, p2=None, wlock=None):
         orig_parent = self.dirstate.parents()[0] or nullid
         p1 = p1 or self.dirstate.parents()[0] or nullid
         p2 = p2 or self.dirstate.parents()[1] or nullid
         c1 = self.changelog.read(p1)
         c2 = self.changelog.read(p2)
         m1 = self.manifest.read(c1[0])
         mf1 = self.manifest.readflags(c1[0])
         m2 = self.manifest.read(c2[0])
         changed = []
 
         if orig_parent == p1:
             update_dirstate = 1
         else:
             update_dirstate = 0
 
         if not wlock:
             wlock = self.wlock()
         l = self.lock()
         tr = self.transaction()
         mm = m1.copy()
         mfm = mf1.copy()
         linkrev = self.changelog.count()
         for f in files:
             try:
                 t = self.wread(f)
                 tm = util.is_exec(self.wjoin(f), mfm.get(f, False))
                 r = self.file(f)
                 mfm[f] = tm
 
                 (entry, fp1, fp2) = self.checkfilemerge(f, t, r, m1, m2)
                 if entry:
                     mm[f] = entry
                     continue
 
                 mm[f] = r.add(t, {}, tr, linkrev, fp1, fp2)
                 changed.append(f)
                 if update_dirstate:
                     self.dirstate.update([f], "n")
             except IOError:
                 try:
                     del mm[f]
                     del mfm[f]
                     if update_dirstate:
                         self.dirstate.forget([f])
                 except:
                     # deleted from p2?
                     pass
 
         mnode = self.manifest.add(mm, mfm, tr, linkrev, c1[0], c2[0])
         user = user or self.ui.username()
         n = self.changelog.add(mnode, changed, text, tr, p1, p2, user, date)
         tr.close()
         if update_dirstate:
             self.dirstate.setparents(n, nullid)
 
     def commit(self, files=None, text="", user=None, date=None,
                match=util.always, force=False, lock=None, wlock=None,
                force_editor=False):
         commit = []
         remove = []
         changed = []
 
         if files:
             for f in files:
                 s = self.dirstate.state(f)
                 if s in 'nmai':
                     commit.append(f)
                 elif s == 'r':
                     remove.append(f)
                 else:
                     self.ui.warn(_("%s not tracked!\n") % f)
         else:
             modified, added, removed, deleted, unknown = self.changes(match=match)
             commit = modified + added
             remove = removed
 
         p1, p2 = self.dirstate.parents()
         c1 = self.changelog.read(p1)
         c2 = self.changelog.read(p2)
         m1 = self.manifest.read(c1[0])
         mf1 = self.manifest.readflags(c1[0])
         m2 = self.manifest.read(c2[0])
 
         if not commit and not remove and not force and p2 == nullid:
             self.ui.status(_("nothing changed\n"))
             return None
 
         xp1 = hex(p1)
         if p2 == nullid: xp2 = ''
         else: xp2 = hex(p2)
 
         self.hook("precommit", throw=True, parent1=xp1, parent2=xp2)
 
         if not wlock:
             wlock = self.wlock()
         if not lock:
             lock = self.lock()
         tr = self.transaction()
 
         # check in files
         new = {}
         linkrev = self.changelog.count()
         commit.sort()
         for f in commit:
             self.ui.note(f + "\n")
             try:
                 mf1[f] = util.is_exec(self.wjoin(f), mf1.get(f, False))
                 t = self.wread(f)
             except IOError:
                 self.ui.warn(_("trouble committing %s!\n") % f)
                 raise
 
             r = self.file(f)
 
             meta = {}
             cp = self.dirstate.copied(f)
             if cp:
                 meta["copy"] = cp
                 meta["copyrev"] = hex(m1.get(cp, m2.get(cp, nullid)))
                 self.ui.debug(_(" %s: copy %s:%s\n") % (f, cp, meta["copyrev"]))
                 fp1, fp2 = nullid, nullid
             else:
                 entry, fp1, fp2 = self.checkfilemerge(f, t, r, m1, m2)
                 if entry:
                     new[f] = entry
                     continue
 
             new[f] = r.add(t, meta, tr, linkrev, fp1, fp2)
             # remember what we've added so that we can later calculate
             # the files to pull from a set of changesets
             changed.append(f)
 
         # update manifest
         m1 = m1.copy()
         m1.update(new)
         for f in remove:
             if f in m1:
                 del m1[f]
         mn = self.manifest.add(m1, mf1, tr, linkrev, c1[0], c2[0],
                                (new, remove))
 
         # add changeset
         new = new.keys()
         new.sort()
 
         user = user or self.ui.username()
         if not text or force_editor:
             edittext = []
             if text:
                 edittext.append(text)
             edittext.append("")
             if p2 != nullid:
                 edittext.append("HG: branch merge")
             edittext.extend(["HG: changed %s" % f for f in changed])
             edittext.extend(["HG: removed %s" % f for f in remove])
             if not changed and not remove:
                 edittext.append("HG: no files changed")
             edittext.append("")
             # run editor in the repository root
             olddir = os.getcwd()
             os.chdir(self.root)
             text = self.ui.edit("\n".join(edittext), user)
             os.chdir(olddir)
 
         lines = [line.rstrip() for line in text.rstrip().splitlines()]
         while lines and not lines[0]:
             del lines[0]
         if not lines:
             return None
         text = '\n'.join(lines)
         n = self.changelog.add(mn, changed + remove, text, tr, p1, p2, user, date)
         self.hook('pretxncommit', throw=True, node=hex(n), parent1=xp1,
                   parent2=xp2)
         tr.close()
 
         self.dirstate.setparents(n)
         self.dirstate.update(new, "n")
         self.dirstate.forget(remove)
 
         self.hook("commit", node=hex(n), parent1=xp1, parent2=xp2)
         return n
 
     def walk(self, node=None, files=[], match=util.always, badmatch=None):
         if node:
             fdict = dict.fromkeys(files)
             for fn in self.manifest.read(self.changelog.read(node)[0]):
                 fdict.pop(fn, None)
                 if match(fn):
                     yield 'm', fn
             for fn in fdict:
                 if badmatch and badmatch(fn):
                     if match(fn):
                         yield 'b', fn
                 else:
                     self.ui.warn(_('%s: No such file in rev %s\n') % (
                         util.pathto(self.getcwd(), fn), short(node)))
         else:
             for src, fn in self.dirstate.walk(files, match, badmatch=badmatch):
                 yield src, fn
 
     def changes(self, node1=None, node2=None, files=[], match=util.always,
                 wlock=None, show_ignored=None):
         """return changes between two nodes or node and working directory
 
         If node1 is None, use the first dirstate parent instead.
         If node2 is None, compare node1 with working directory.
         """
 
         def fcmp(fn, mf):
             t1 = self.wread(fn)
             t2 = self.file(fn).read(mf.get(fn, nullid))
             return cmp(t1, t2)
 
         def mfmatches(node):
             change = self.changelog.read(node)
             mf = dict(self.manifest.read(change[0]))
             for fn in mf.keys():
                 if not match(fn):
                     del mf[fn]
             return mf
 
         modified, added, removed, deleted, unknown, ignored = [],[],[],[],[],[]
         compareworking = False
         if not node1 or (not node2 and node1 == self.dirstate.parents()[0]):
             compareworking = True
 
         if not compareworking:
             # read the manifest from node1 before the manifest from node2,
             # so that we'll hit the manifest cache if we're going through
             # all the revisions in parent->child order.
             mf1 = mfmatches(node1)
 
         # are we comparing the working directory?
         if not node2:
             if not wlock:
                 try:
                     wlock = self.wlock(wait=0)
                 except lock.LockException:
                     wlock = None
             lookup, modified, added, removed, deleted, unknown, ignored = (
                 self.dirstate.changes(files, match, show_ignored))
 
             # are we comparing working dir against its parent?
             if compareworking:
                 if lookup:
                     # do a full compare of any files that might have changed
                     mf2 = mfmatches(self.dirstate.parents()[0])
                     for f in lookup:
                         if fcmp(f, mf2):
                             modified.append(f)
                         elif wlock is not None:
                             self.dirstate.update([f], "n")
             else:
                 # we are comparing working dir against non-parent
                 # generate a pseudo-manifest for the working dir
                 mf2 = mfmatches(self.dirstate.parents()[0])
                 for f in lookup + modified + added:
                     mf2[f] = ""
                 for f in removed:
                     if f in mf2:
                         del mf2[f]
         else:
             # we are comparing two revisions
             deleted, unknown, ignored = [], [], []
             mf2 = mfmatches(node2)
 
         if not compareworking:
             # flush lists from dirstate before comparing manifests
             modified, added = [], []
 
             # make sure to sort the files so we talk to the disk in a
             # reasonable order
             mf2keys = mf2.keys()
             mf2keys.sort()
             for fn in mf2keys:
                 if mf1.has_key(fn):
                     if mf1[fn] != mf2[fn] and (mf2[fn] != "" or fcmp(fn, mf1)):
                         modified.append(fn)
                     del mf1[fn]
                 else:
                     added.append(fn)
 
             removed = mf1.keys()
 
         # sort and return results:
         for l in modified, added, removed, deleted, unknown, ignored:
             l.sort()
         if show_ignored is None:
             return (modified, added, removed, deleted, unknown)
         else:
             return (modified, added, removed, deleted, unknown, ignored)
 
     def add(self, list, wlock=None):
         if not wlock:
             wlock = self.wlock()
         for f in list:
             p = self.wjoin(f)
             if not os.path.exists(p):
                 self.ui.warn(_("%s does not exist!\n") % f)
             elif not os.path.isfile(p):
                 self.ui.warn(_("%s not added: only files supported currently\n")
                              % f)
             elif self.dirstate.state(f) in 'an':
                 self.ui.warn(_("%s already tracked!\n") % f)
             else:
                 self.dirstate.update([f], "a")
 
     def forget(self, list, wlock=None):
         if not wlock:
             wlock = self.wlock()
         for f in list:
             if self.dirstate.state(f) not in 'ai':
                 self.ui.warn(_("%s not added!\n") % f)
             else:
                 self.dirstate.forget([f])
 
     def remove(self, list, unlink=False, wlock=None):
         if unlink:
             for f in list:
                 try:
                     util.unlink(self.wjoin(f))
                 except OSError, inst:
                     if inst.errno != errno.ENOENT:
                         raise
         if not wlock:
             wlock = self.wlock()
         for f in list:
             p = self.wjoin(f)
             if os.path.exists(p):
                 self.ui.warn(_("%s still exists!\n") % f)
             elif self.dirstate.state(f) == 'a':
                 self.dirstate.forget([f])
             elif f not in self.dirstate:
                 self.ui.warn(_("%s not tracked!\n") % f)
             else:
                 self.dirstate.update([f], "r")
 
     def undelete(self, list, wlock=None):
         p = self.dirstate.parents()[0]
         mn = self.changelog.read(p)[0]
         mf = self.manifest.readflags(mn)
         m = self.manifest.read(mn)
         if not wlock:
             wlock = self.wlock()
         for f in list:
             if self.dirstate.state(f) not in  "r":
                 self.ui.warn("%s not removed!\n" % f)
             else:
                 t = self.file(f).read(m[f])
                 self.wwrite(f, t)
                 util.set_exec(self.wjoin(f), mf[f])
                 self.dirstate.update([f], "n")
 
     def copy(self, source, dest, wlock=None):
         p = self.wjoin(dest)
         if not os.path.exists(p):
             self.ui.warn(_("%s does not exist!\n") % dest)
         elif not os.path.isfile(p):
             self.ui.warn(_("copy failed: %s is not a file\n") % dest)
         else:
             if not wlock:
                 wlock = self.wlock()
             if self.dirstate.state(dest) == '?':
                 self.dirstate.update([dest], "a")
             self.dirstate.copy(source, dest)
 
     def heads(self, start=None):
         heads = self.changelog.heads(start)
         # sort the output in rev descending order
         heads = [(-self.changelog.rev(h), h) for h in heads]
         heads.sort()
         return [n for (r, n) in heads]
 
     # branchlookup returns a dict giving a list of branches for
     # each head.  A branch is defined as the tag of a node or
     # the branch of the node's parents.  If a node has multiple
     # branch tags, tags are eliminated if they are visible from other
     # branch tags.
     #
     # So, for this graph:  a->b->c->d->e
     #                       \         /
     #                        aa -----/
     # a has tag 2.6.12
     # d has tag 2.6.13
     # e would have branch tags for 2.6.12 and 2.6.13.  Because the node
     # for 2.6.12 can be reached from the node 2.6.13, that is eliminated
     # from the list.
     #
     # It is possible that more than one head will have the same branch tag.
     # callers need to check the result for multiple heads under the same
     # branch tag if that is a problem for them (ie checkout of a specific
     # branch).
     #
     # passing in a specific branch will limit the depth of the search
     # through the parents.  It won't limit the branches returned in the
     # result though.
     def branchlookup(self, heads=None, branch=None):
         if not heads:
             heads = self.heads()
         headt = [ h for h in heads ]
         chlog = self.changelog
         branches = {}
         merges = []
         seenmerge = {}
 
         # traverse the tree once for each head, recording in the branches
         # dict which tags are visible from this head.  The branches
         # dict also records which tags are visible from each tag
         # while we traverse.
         while headt or merges:
             if merges:
                 n, found = merges.pop()
                 visit = [n]
             else:
                 h = headt.pop()
                 visit = [h]
                 found = [h]
                 seen = {}
             while visit:
                 n = visit.pop()
                 if n in seen:
                     continue
                 pp = chlog.parents(n)
                 tags = self.nodetags(n)
                 if tags:
                     for x in tags:
                         if x == 'tip':
                             continue
                         for f in found:
                             branches.setdefault(f, {})[n] = 1
                         branches.setdefault(n, {})[n] = 1
                         break
                     if n not in found:
                         found.append(n)
                     if branch in tags:
                         continue
                 seen[n] = 1
                 if pp[1] != nullid and n not in seenmerge:
                     merges.append((pp[1], [x for x in found]))
                     seenmerge[n] = 1
                 if pp[0] != nullid:
                     visit.append(pp[0])
         # traverse the branches dict, eliminating branch tags from each
         # head that are visible from another branch tag for that head.
         out = {}
         viscache = {}
         for h in heads:
             def visible(node):
                 if node in viscache:
                     return viscache[node]
                 ret = {}
                 visit = [node]
                 while visit:
                     x = visit.pop()
                     if x in viscache:
                         ret.update(viscache[x])
                     elif x not in ret:
                         ret[x] = 1
                         if x in branches:
                             visit[len(visit):] = branches[x].keys()
                 viscache[node] = ret
                 return ret
             if h not in branches:
                 continue
             # O(n^2), but somewhat limited.  This only searches the
             # tags visible from a specific head, not all the tags in the
             # whole repo.
             for b in branches[h]:
                 vis = False
                 for bb in branches[h].keys():
                     if b != bb:
                         if b in visible(bb):
                             vis = True
                             break
                 if not vis:
                     l = out.setdefault(h, [])
                     l[len(l):] = self.nodetags(b)
         return out
 
     def branches(self, nodes):
         if not nodes:
             nodes = [self.changelog.tip()]
         b = []
         for n in nodes:
             t = n
             while 1:
                 p = self.changelog.parents(n)
                 if p[1] != nullid or p[0] == nullid:
                     b.append((t, n, p[0], p[1]))
                     break
                 n = p[0]
         return b
 
     def between(self, pairs):
         r = []
 
         for top, bottom in pairs:
             n, l, i = top, [], 0
             f = 1
 
             while n != bottom:
                 p = self.changelog.parents(n)[0]
                 if i == f:
                     l.append(n)
                     f = f * 2
                 n = p
                 i += 1
 
             r.append(l)
 
         return r
 
     def findincoming(self, remote, base=None, heads=None, force=False):
         """Return list of roots of the subsets of missing nodes from remote
 
         If base dict is specified, assume that these nodes and their parents
         exist on the remote side and that no child of a node of base exists
         in both remote and self.
         Furthermore base will be updated to include the nodes that exists
         in self and remote but no children exists in self and remote.
         If a list of heads is specified, return only nodes which are heads
         or ancestors of these heads.
 
         All the ancestors of base are in self and in remote.
         All the descendants of the list returned are missing in self.
         (and so we know that the rest of the nodes are missing in remote, see
         outgoing)
         """
         m = self.changelog.nodemap
         search = []
         fetch = {}
         seen = {}
         seenbranch = {}
         if base == None:
             base = {}
 
         if not heads:
             heads = remote.heads()
 
         if self.changelog.tip() == nullid:
             base[nullid] = 1
             if heads != [nullid]:
                 return [nullid]
             return []
 
         # assume we're closer to the tip than the root
         # and start by examining the heads
         self.ui.status(_("searching for changes\n"))
 
         unknown = []
         for h in heads:
             if h not in m:
                 unknown.append(h)
             else:
                 base[h] = 1
 
         if not unknown:
             return []
 
         req = dict.fromkeys(unknown)
         reqcnt = 0
 
         # search through remote branches
         # a 'branch' here is a linear segment of history, with four parts:
         # head, root, first parent, second parent
         # (a branch always has two parents (or none) by definition)
         unknown = remote.branches(unknown)
         while unknown:
             r = []
             while unknown:
                 n = unknown.pop(0)
                 if n[0] in seen:
                     continue
 
                 self.ui.debug(_("examining %s:%s\n")
                               % (short(n[0]), short(n[1])))
                 if n[0] == nullid: # found the end of the branch
                     pass
                 elif n in seenbranch:
                     self.ui.debug(_("branch already found\n"))
                     continue
                 elif n[1] and n[1] in m: # do we know the base?
                     self.ui.debug(_("found incomplete branch %s:%s\n")
                                   % (short(n[0]), short(n[1])))
                     search.append(n) # schedule branch range for scanning
                     seenbranch[n] = 1
                 else:
                     if n[1] not in seen and n[1] not in fetch:
                         if n[2] in m and n[3] in m:
                             self.ui.debug(_("found new changeset %s\n") %
                                           short(n[1]))
                             fetch[n[1]] = 1 # earliest unknown
                         for p in n[2:4]:
                             if p in m:
                                 base[p] = 1 # latest known
 
                     for p in n[2:4]:
                         if p not in req and p not in m:
                             r.append(p)
                             req[p] = 1
                 seen[n[0]] = 1
 
             if r:
                 reqcnt += 1
                 self.ui.debug(_("request %d: %s\n") %
                             (reqcnt, " ".join(map(short, r))))
                 for p in range(0, len(r), 10):
                     for b in remote.branches(r[p:p+10]):
                         self.ui.debug(_("received %s:%s\n") %
                                       (short(b[0]), short(b[1])))
                         unknown.append(b)
 
         # do binary search on the branches we found
         while search:
             n = search.pop(0)
             reqcnt += 1
             l = remote.between([(n[0], n[1])])[0]
             l.append(n[1])
             p = n[0]
             f = 1
             for i in l:
                 self.ui.debug(_("narrowing %d:%d %s\n") % (f, len(l), short(i)))
                 if i in m:
                     if f <= 2:
                         self.ui.debug(_("found new branch changeset %s\n") %
                                           short(p))
                         fetch[p] = 1
                         base[i] = 1
                     else:
                         self.ui.debug(_("narrowed branch search to %s:%s\n")
                                       % (short(p), short(i)))
                         search.append((p, i))
                     break
                 p, f = i, f * 2
 
         # sanity check our fetch list
         for f in fetch.keys():
             if f in m:
                 raise repo.RepoError(_("already have changeset ") + short(f[:4]))
 
         if base.keys() == [nullid]:
             if force:
                 self.ui.warn(_("warning: repository is unrelated\n"))
             else:
                 raise util.Abort(_("repository is unrelated"))
 
         self.ui.note(_("found new changesets starting at ") +
                      " ".join([short(f) for f in fetch]) + "\n")
 
         self.ui.debug(_("%d total queries\n") % reqcnt)
 
         return fetch.keys()
 
     def findoutgoing(self, remote, base=None, heads=None, force=False):
         """Return list of nodes that are roots of subsets not in remote
 
         If base dict is specified, assume that these nodes and their parents
         exist on the remote side.
         If a list of heads is specified, return only nodes which are heads
         or ancestors of these heads, and return a second element which
         contains all remote heads which get new children.
         """
         if base == None:
             base = {}
             self.findincoming(remote, base, heads, force=force)
 
         self.ui.debug(_("common changesets up to ")
                       + " ".join(map(short, base.keys())) + "\n")
 
         remain = dict.fromkeys(self.changelog.nodemap)
 
         # prune everything remote has from the tree
         del remain[nullid]
         remove = base.keys()
         while remove:
             n = remove.pop(0)
             if n in remain:
                 del remain[n]
                 for p in self.changelog.parents(n):
                     remove.append(p)
 
         # find every node whose parents have been pruned
         subset = []
         # find every remote head that will get new children
         updated_heads = {}
         for n in remain:
             p1, p2 = self.changelog.parents(n)
             if p1 not in remain and p2 not in remain:
                 subset.append(n)
             if heads:
                 if p1 in heads:
                     updated_heads[p1] = True
                 if p2 in heads:
                     updated_heads[p2] = True
 
         # this is the set of all roots we have to push
         if heads:
             return subset, updated_heads.keys()
         else:
             return subset
 
     def pull(self, remote, heads=None, force=False):
         l = self.lock()
 
         fetch = self.findincoming(remote, force=force)
         if fetch == [nullid]:
             self.ui.status(_("requesting all changes\n"))
 
         if not fetch:
             self.ui.status(_("no changes found\n"))
             return 0
 
         if heads is None:
             cg = remote.changegroup(fetch, 'pull')
         else:
             cg = remote.changegroupsubset(fetch, heads, 'pull')
         return self.addchangegroup(cg, 'pull')
 
     def push(self, remote, force=False, revs=None):
         # there are two ways to push to remote repo:
         #
         # addchangegroup assumes local user can lock remote
         # repo (local filesystem, old ssh servers).
         #
         # unbundle assumes local user cannot lock remote repo (new ssh
         # servers, http servers).
 
         if remote.capable('unbundle'):
             return self.push_unbundle(remote, force, revs)
         return self.push_addchangegroup(remote, force, revs)
 
     def prepush(self, remote, force, revs):
         base = {}
         remote_heads = remote.heads()
         inc = self.findincoming(remote, base, remote_heads, force=force)
         if not force and inc:
             self.ui.warn(_("abort: unsynced remote changes!\n"))
             self.ui.status(_("(did you forget to sync?"
                              " use push -f to force)\n"))
             return None, 1
 
         update, updated_heads = self.findoutgoing(remote, base, remote_heads)
         if revs is not None:
             msng_cl, bases, heads = self.changelog.nodesbetween(update, revs)
         else:
             bases, heads = update, self.changelog.heads()
 
         if not bases:
             self.ui.status(_("no changes found\n"))
             return None, 1
         elif not force:
             # FIXME we don't properly detect creation of new heads
             # in the push -r case, assume the user knows what he's doing
             if not revs and len(remote_heads) < len(heads) \
                    and remote_heads != [nullid]:
                 self.ui.warn(_("abort: push creates new remote branches!\n"))
                 self.ui.status(_("(did you forget to merge?"
                                  " use push -f to force)\n"))
                 return None, 1
 
         if revs is None:
             cg = self.changegroup(update, 'push')
         else:
             cg = self.changegroupsubset(update, revs, 'push')
         return cg, remote_heads
 
     def push_addchangegroup(self, remote, force, revs):
         lock = remote.lock()
 
         ret = self.prepush(remote, force, revs)
         if ret[0] is not None:
             cg, remote_heads = ret
             return remote.addchangegroup(cg, 'push')
         return ret[1]
 
     def push_unbundle(self, remote, force, revs):
         # local repo finds heads on server, finds out what revs it
         # must push.  once revs transferred, if server finds it has
         # different heads (someone else won commit/push race), server
         # aborts.
 
         ret = self.prepush(remote, force, revs)
         if ret[0] is not None:
             cg, remote_heads = ret
             if force: remote_heads = ['force']
             return remote.unbundle(cg, remote_heads, 'push')
         return ret[1]
 
     def changegroupsubset(self, bases, heads, source):
         """This function generates a changegroup consisting of all the nodes
         that are descendents of any of the bases, and ancestors of any of
         the heads.
 
         It is fairly complex as determining which filenodes and which
         manifest nodes need to be included for the changeset to be complete
         is non-trivial.
 
         Another wrinkle is doing the reverse, figuring out which changeset in
         the changegroup a particular filenode or manifestnode belongs to."""
 
         self.hook('preoutgoing', throw=True, source=source)
 
         # Set up some initial variables
         # Make it easy to refer to self.changelog
         cl = self.changelog
         # msng is short for missing - compute the list of changesets in this
         # changegroup.
         msng_cl_lst, bases, heads = cl.nodesbetween(bases, heads)
         # Some bases may turn out to be superfluous, and some heads may be
         # too.  nodesbetween will return the minimal set of bases and heads
         # necessary to re-create the changegroup.
 
         # Known heads are the list of heads that it is assumed the recipient
         # of this changegroup will know about.
         knownheads = {}
         # We assume that all parents of bases are known heads.
         for n in bases:
             for p in cl.parents(n):
                 if p != nullid:
                     knownheads[p] = 1
         knownheads = knownheads.keys()
         if knownheads:
             # Now that we know what heads are known, we can compute which
             # changesets are known.  The recipient must know about all
             # changesets required to reach the known heads from the null
             # changeset.
             has_cl_set, junk, junk = cl.nodesbetween(None, knownheads)
             junk = None
             # Transform the list into an ersatz set.
             has_cl_set = dict.fromkeys(has_cl_set)
         else:
             # If there were no known heads, the recipient cannot be assumed to
             # know about any changesets.
             has_cl_set = {}
 
         # Make it easy to refer to self.manifest
         mnfst = self.manifest
         # We don't know which manifests are missing yet
         msng_mnfst_set = {}
         # Nor do we know which filenodes are missing.
         msng_filenode_set = {}
 
         junk = mnfst.index[mnfst.count() - 1] # Get around a bug in lazyindex
         junk = None
 
         # A changeset always belongs to itself, so the changenode lookup
         # function for a changenode is identity.
         def identity(x):
             return x
 
         # A function generating function.  Sets up an environment for the
         # inner function.
         def cmp_by_rev_func(revlog):
             # Compare two nodes by their revision number in the environment's
             # revision history.  Since the revision number both represents the
             # most efficient order to read the nodes in, and represents a
             # topological sorting of the nodes, this function is often useful.
             def cmp_by_rev(a, b):
                 return cmp(revlog.rev(a), revlog.rev(b))
             return cmp_by_rev
 
         # If we determine that a particular file or manifest node must be a
         # node that the recipient of the changegroup will already have, we can
         # also assume the recipient will have all the parents.  This function
         # prunes them from the set of missing nodes.
         def prune_parents(revlog, hasset, msngset):
             haslst = hasset.keys()
             haslst.sort(cmp_by_rev_func(revlog))
             for node in haslst:
                 parentlst = [p for p in revlog.parents(node) if p != nullid]
                 while parentlst:
                     n = parentlst.pop()
                     if n not in hasset:
                         hasset[n] = 1
                         p = [p for p in revlog.parents(n) if p != nullid]
                         parentlst.extend(p)
             for n in hasset:
                 msngset.pop(n, None)
 
         # This is a function generating function used to set up an environment
         # for the inner function to execute in.
         def manifest_and_file_collector(changedfileset):
             # This is an information gathering function that gathers
             # information from each changeset node that goes out as part of
             # the changegroup.  The information gathered is a list of which
             # manifest nodes are potentially required (the recipient may
             # already have them) and total list of all files which were
             # changed in any changeset in the changegroup.
             #
             # We also remember the first changenode we saw any manifest
             # referenced by so we can later determine which changenode 'owns'
             # the manifest.
             def collect_manifests_and_files(clnode):
                 c = cl.read(clnode)
                 for f in c[3]:
                     # This is to make sure we only have one instance of each
                     # filename string for each filename.
                     changedfileset.setdefault(f, f)
                 msng_mnfst_set.setdefault(c[0], clnode)
             return collect_manifests_and_files
 
         # Figure out which manifest nodes (of the ones we think might be part
         # of the changegroup) the recipient must know about and remove them
         # from the changegroup.
         def prune_manifests():
             has_mnfst_set = {}
             for n in msng_mnfst_set:
                 # If a 'missing' manifest thinks it belongs to a changenode
                 # the recipient is assumed to have, obviously the recipient
                 # must have that manifest.
                 linknode = cl.node(mnfst.linkrev(n))
                 if linknode in has_cl_set:
                     has_mnfst_set[n] = 1
             prune_parents(mnfst, has_mnfst_set, msng_mnfst_set)
 
         # Use the information collected in collect_manifests_and_files to say
         # which changenode any manifestnode belongs to.
         def lookup_manifest_link(mnfstnode):
             return msng_mnfst_set[mnfstnode]
 
         # A function generating function that sets up the initial environment
         # the inner function.
         def filenode_collector(changedfiles):
             next_rev = [0]
             # This gathers information from each manifestnode included in the
             # changegroup about which filenodes the manifest node references
             # so we can include those in the changegroup too.
             #
             # It also remembers which changenode each filenode belongs to.  It
             # does this by assuming the a filenode belongs to the changenode
             # the first manifest that references it belongs to.
             def collect_msng_filenodes(mnfstnode):
                 r = mnfst.rev(mnfstnode)
                 if r == next_rev[0]:
                     # If the last rev we looked at was the one just previous,
                     # we only need to see a diff.
                     delta = mdiff.patchtext(mnfst.delta(mnfstnode))
                     # For each line in the delta
                     for dline in delta.splitlines():
                         # get the filename and filenode for that line
                         f, fnode = dline.split('\0')
                         fnode = bin(fnode[:40])
                         f = changedfiles.get(f, None)
                         # And if the file is in the list of files we care
                         # about.
                         if f is not None:
                             # Get the changenode this manifest belongs to
                             clnode = msng_mnfst_set[mnfstnode]
                             # Create the set of filenodes for the file if
                             # there isn't one already.
                             ndset = msng_filenode_set.setdefault(f, {})
                             # And set the filenode's changelog node to the
                             # manifest's if it hasn't been set already.
                             ndset.setdefault(fnode, clnode)
                 else:
                     # Otherwise we need a full manifest.
                     m = mnfst.read(mnfstnode)
                     # For every file in we care about.
                     for f in changedfiles:
                         fnode = m.get(f, None)
                         # If it's in the manifest
                         if fnode is not None:
                             # See comments above.
                             clnode = msng_mnfst_set[mnfstnode]
                             ndset = msng_filenode_set.setdefault(f, {})
                             ndset.setdefault(fnode, clnode)
                 # Remember the revision we hope to see next.
                 next_rev[0] = r + 1
             return collect_msng_filenodes
 
         # We have a list of filenodes we think we need for a file, lets remove
         # all those we now the recipient must have.
         def prune_filenodes(f, filerevlog):
             msngset = msng_filenode_set[f]
             hasset = {}
             # If a 'missing' filenode thinks it belongs to a changenode we
             # assume the recipient must have, then the recipient must have
             # that filenode.
             for n in msngset:
                 clnode = cl.node(filerevlog.linkrev(n))
                 if clnode in has_cl_set:
                     hasset[n] = 1
             prune_parents(filerevlog, hasset, msngset)
 
         # A function generator function that sets up the a context for the
         # inner function.
         def lookup_filenode_link_func(fname):
             msngset = msng_filenode_set[fname]
             # Lookup the changenode the filenode belongs to.
             def lookup_filenode_link(fnode):
                 return msngset[fnode]
             return lookup_filenode_link
 
         # Now that we have all theses utility functions to help out and
         # logically divide up the task, generate the group.
         def gengroup():
             # The set of changed files starts empty.
             changedfiles = {}
             # Create a changenode group generator that will call our functions
             # back to lookup the owning changenode and collect information.
             group = cl.group(msng_cl_lst, identity,
                              manifest_and_file_collector(changedfiles))
             for chnk in group:
                 yield chnk
 
             # The list of manifests has been collected by the generator
             # calling our functions back.
             prune_manifests()
             msng_mnfst_lst = msng_mnfst_set.keys()
             # Sort the manifestnodes by revision number.
             msng_mnfst_lst.sort(cmp_by_rev_func(mnfst))
             # Create a generator for the manifestnodes that calls our lookup
             # and data collection functions back.
             group = mnfst.group(msng_mnfst_lst, lookup_manifest_link,
                                 filenode_collector(changedfiles))
             for chnk in group:
                 yield chnk
 
             # These are no longer needed, dereference and toss the memory for
             # them.
             msng_mnfst_lst = None
             msng_mnfst_set.clear()
 
             changedfiles = changedfiles.keys()
             changedfiles.sort()
             # Go through all our files in order sorted by name.
             for fname in changedfiles:
                 filerevlog = self.file(fname)
                 # Toss out the filenodes that the recipient isn't really
                 # missing.
                 if msng_filenode_set.has_key(fname):
                     prune_filenodes(fname, filerevlog)
                     msng_filenode_lst = msng_filenode_set[fname].keys()
                 else:
                     msng_filenode_lst = []
                 # If any filenodes are left, generate the group for them,
                 # otherwise don't bother.
                 if len(msng_filenode_lst) > 0:
                     yield changegroup.genchunk(fname)
                     # Sort the filenodes by their revision #
                     msng_filenode_lst.sort(cmp_by_rev_func(filerevlog))
                     # Create a group generator and only pass in a changenode
                     # lookup function as we need to collect no information
                     # from filenodes.
                     group = filerevlog.group(msng_filenode_lst,
                                              lookup_filenode_link_func(fname))
                     for chnk in group:
                         yield chnk
                 if msng_filenode_set.has_key(fname):
                     # Don't need this anymore, toss it to free memory.
                     del msng_filenode_set[fname]
             # Signal that no more groups are left.
             yield changegroup.closechunk()
 
             if msng_cl_lst:
                 self.hook('outgoing', node=hex(msng_cl_lst[0]), source=source)
 
         return util.chunkbuffer(gengroup())
 
     def changegroup(self, basenodes, source):
         """Generate a changegroup of all nodes that we have that a recipient
         doesn't.
 
         This is much easier than the previous function as we can assume that
         the recipient has any changenode we aren't sending them."""
 
         self.hook('preoutgoing', throw=True, source=source)
 
         cl = self.changelog
         nodes = cl.nodesbetween(basenodes, None)[0]
         revset = dict.fromkeys([cl.rev(n) for n in nodes])
 
         def identity(x):
             return x
 
         def gennodelst(revlog):
             for r in xrange(0, revlog.count()):
                 n = revlog.node(r)
                 if revlog.linkrev(n) in revset:
                     yield n
 
         def changed_file_collector(changedfileset):
             def collect_changed_files(clnode):
                 c = cl.read(clnode)
                 for fname in c[3]:
                     changedfileset[fname] = 1
             return collect_changed_files
 
         def lookuprevlink_func(revlog):
             def lookuprevlink(n):
                 return cl.node(revlog.linkrev(n))
             return lookuprevlink
 
         def gengroup():
             # construct a list of all changed files
             changedfiles = {}
 
             for chnk in cl.group(nodes, identity,
                                  changed_file_collector(changedfiles)):
                 yield chnk
             changedfiles = changedfiles.keys()
             changedfiles.sort()
 
             mnfst = self.manifest
             nodeiter = gennodelst(mnfst)
             for chnk in mnfst.group(nodeiter, lookuprevlink_func(mnfst)):
                 yield chnk
 
             for fname in changedfiles:
                 filerevlog = self.file(fname)
                 nodeiter = gennodelst(filerevlog)
                 nodeiter = list(nodeiter)
                 if nodeiter:
                     yield changegroup.genchunk(fname)
                     lookup = lookuprevlink_func(filerevlog)
                     for chnk in filerevlog.group(nodeiter, lookup):
                         yield chnk
 
             yield changegroup.closechunk()
 
             if nodes:
                 self.hook('outgoing', node=hex(nodes[0]), source=source)
 
         return util.chunkbuffer(gengroup())
 
     def addchangegroup(self, source, srctype):
         """add changegroup to repo.
         returns number of heads modified or added + 1."""
 
         def csmap(x):
             self.ui.debug(_("add changeset %s\n") % short(x))
             return cl.count()
 
         def revmap(x):
             return cl.rev(x)
 
         if not source:
             return 0
 
         self.hook('prechangegroup', throw=True, source=srctype)
 
         changesets = files = revisions = 0
 
         tr = self.transaction()
 
         # write changelog data to temp files so concurrent readers will not see
         # inconsistent view
         cl = None
         try:
             cl = appendfile.appendchangelog(self.opener, self.changelog.version)
 
             oldheads = len(cl.heads())
 
             # pull off the changeset group
             self.ui.status(_("adding changesets\n"))
             cor = cl.count() - 1
             chunkiter = changegroup.chunkiter(source)
             if cl.addgroup(chunkiter, csmap, tr, 1) is None:
                 raise util.Abort(_("received changelog group is empty"))
             cnr = cl.count() - 1
             changesets = cnr - cor
 
             # pull off the manifest group
             self.ui.status(_("adding manifests\n"))
             chunkiter = changegroup.chunkiter(source)
             # no need to check for empty manifest group here:
             # if the result of the merge of 1 and 2 is the same in 3 and 4,
             # no new manifest will be created and the manifest group will
             # be empty during the pull
             self.manifest.addgroup(chunkiter, revmap, tr)
 
             # process the files
             self.ui.status(_("adding file changes\n"))
             while 1:
                 f = changegroup.getchunk(source)
                 if not f:
                     break
                 self.ui.debug(_("adding %s revisions\n") % f)
                 fl = self.file(f)
                 o = fl.count()
                 chunkiter = changegroup.chunkiter(source)
                 if fl.addgroup(chunkiter, revmap, tr) is None:
                     raise util.Abort(_("received file revlog group is empty"))
                 revisions += fl.count() - o
                 files += 1
 
             cl.writedata()
         finally:
             if cl:
                 cl.cleanup()
 
         # make changelog see real files again
         self.changelog = changelog.changelog(self.opener, self.changelog.version)
         self.changelog.checkinlinesize(tr)
 
         newheads = len(self.changelog.heads())
         heads = ""
         if oldheads and newheads != oldheads:
             heads = _(" (%+d heads)") % (newheads - oldheads)
 
         self.ui.status(_("added %d changesets"
                          " with %d changes to %d files%s\n")
                          % (changesets, revisions, files, heads))
 
         if changesets > 0:
             self.hook('pretxnchangegroup', throw=True,
                       node=hex(self.changelog.node(cor+1)), source=srctype)
 
         tr.close()
 
         if changesets > 0:
             self.hook("changegroup", node=hex(self.changelog.node(cor+1)),
                       source=srctype)
 
             for i in range(cor + 1, cnr + 1):
                 self.hook("incoming", node=hex(self.changelog.node(i)),
                           source=srctype)
 
         return newheads - oldheads + 1
 
     def update(self, node, allow=False, force=False, choose=None,
                moddirstate=True, forcemerge=False, wlock=None, show_stats=True):
         pl = self.dirstate.parents()
         if not force and pl[1] != nullid:
             raise util.Abort(_("outstanding uncommitted merges"))
 
         err = False
 
         p1, p2 = pl[0], node
         pa = self.changelog.ancestor(p1, p2)
         m1n = self.changelog.read(p1)[0]
         m2n = self.changelog.read(p2)[0]
         man = self.manifest.ancestor(m1n, m2n)
         m1 = self.manifest.read(m1n)
         mf1 = self.manifest.readflags(m1n)
         m2 = self.manifest.read(m2n).copy()
         mf2 = self.manifest.readflags(m2n)
         ma = self.manifest.read(man)
         mfa = self.manifest.readflags(man)
 
         modified, added, removed, deleted, unknown = self.changes()
 
         # is this a jump, or a merge?  i.e. is there a linear path
         # from p1 to p2?
         linear_path = (pa == p1 or pa == p2)
 
         if allow and linear_path:
             raise util.Abort(_("there is nothing to merge, just use "
                                "'hg update' or look at 'hg heads'"))
         if allow and not forcemerge:
             if modified or added or removed:
                 raise util.Abort(_("outstanding uncommitted changes"))
 
         if not forcemerge and not force:
             for f in unknown:
                 if f in m2:
                     t1 = self.wread(f)
                     t2 = self.file(f).read(m2[f])
                     if cmp(t1, t2) != 0:
                         raise util.Abort(_("'%s' already exists in the working"
                                            " dir and differs from remote") % f)
 
         # resolve the manifest to determine which files
         # we care about merging
         self.ui.note(_("resolving manifests\n"))
         self.ui.debug(_(" force %s allow %s moddirstate %s linear %s\n") %
                       (force, allow, moddirstate, linear_path))
         self.ui.debug(_(" ancestor %s local %s remote %s\n") %
                       (short(man), short(m1n), short(m2n)))
 
         merge = {}
         get = {}
         remove = []
 
         # construct a working dir manifest
         mw = m1.copy()
         mfw = mf1.copy()
         umap = dict.fromkeys(unknown)
 
         for f in added + modified + unknown:
             mw[f] = ""
             mfw[f] = util.is_exec(self.wjoin(f), mfw.get(f, False))
 
         if moddirstate and not wlock:
             wlock = self.wlock()
 
         for f in deleted + removed:
             if f in mw:
                 del mw[f]
 
             # If we're jumping between revisions (as opposed to merging),
             # and if neither the working directory nor the target rev has
             # the file, then we need to remove it from the dirstate, to
             # prevent the dirstate from listing the file when it is no
             # longer in the manifest.
             if moddirstate and linear_path and f not in m2:
                 self.dirstate.forget((f,))
 
         # Compare manifests
         for f, n in mw.iteritems():
             if choose and not choose(f):
                 continue
             if f in m2:
                 s = 0
 
                 # is the wfile new since m1, and match m2?
                 if f not in m1:
                     t1 = self.wread(f)
                     t2 = self.file(f).read(m2[f])
                     if cmp(t1, t2) == 0:
                         n = m2[f]
                     del t1, t2
 
                 # are files different?
                 if n != m2[f]:
                     a = ma.get(f, nullid)
                     # are both different from the ancestor?
                     if n != a and m2[f] != a:
                         self.ui.debug(_(" %s versions differ, resolve\n") % f)
                         # merge executable bits
                         # "if we changed or they changed, change in merge"
                         a, b, c = mfa.get(f, 0), mfw[f], mf2[f]
                         mode = ((a^b) | (a^c)) ^ a
                         merge[f] = (m1.get(f, nullid), m2[f], mode)
                         s = 1
                     # are we clobbering?
                     # is remote's version newer?
                     # or are we going back in time?
                     elif force or m2[f] != a or (p2 == pa and mw[f] == m1[f]):
                         self.ui.debug(_(" remote %s is newer, get\n") % f)
                         get[f] = m2[f]
                         s = 1
                 elif f in umap or f in added:
                     # this unknown file is the same as the checkout
                     # we need to reset the dirstate if the file was added
                     get[f] = m2[f]
 
                 if not s and mfw[f] != mf2[f]:
                     if force:
                         self.ui.debug(_(" updating permissions for %s\n") % f)
                         util.set_exec(self.wjoin(f), mf2[f])
                     else:
                         a, b, c = mfa.get(f, 0), mfw[f], mf2[f]
                         mode = ((a^b) | (a^c)) ^ a
                         if mode != b:
                             self.ui.debug(_(" updating permissions for %s\n")
                                           % f)
                             util.set_exec(self.wjoin(f), mode)
                 del m2[f]
             elif f in ma:
                 if n != ma[f]:
                     r = _("d")
                     if not force and (linear_path or allow):
                         r = self.ui.prompt(
                             (_(" local changed %s which remote deleted\n") % f) +
                              _("(k)eep or (d)elete?"), _("[kd]"), _("k"))
                     if r == _("d"):
                         remove.append(f)
                 else:
                     self.ui.debug(_("other deleted %s\n") % f)
                     remove.append(f) # other deleted it
             else:
                 # file is created on branch or in working directory
                 if force and f not in umap:
                     self.ui.debug(_("remote deleted %s, clobbering\n") % f)
                     remove.append(f)
                 elif n == m1.get(f, nullid): # same as parent
                     if p2 == pa: # going backwards?
                         self.ui.debug(_("remote deleted %s\n") % f)
                         remove.append(f)
                     else:
                         self.ui.debug(_("local modified %s, keeping\n") % f)
                 else:
                     self.ui.debug(_("working dir created %s, keeping\n") % f)
 
         for f, n in m2.iteritems():
             if choose and not choose(f):
                 continue
             if f[0] == "/":
                 continue
             if f in ma and n != ma[f]:
                 r = _("k")
                 if not force and (linear_path or allow):
                     r = self.ui.prompt(
                         (_("remote changed %s which local deleted\n") % f) +
                          _("(k)eep or (d)elete?"), _("[kd]"), _("k"))
                 if r == _("k"):
                     get[f] = n
             elif f not in ma:
                 self.ui.debug(_("remote created %s\n") % f)
                 get[f] = n
             else:
                 if force or p2 == pa: # going backwards?
                     self.ui.debug(_("local deleted %s, recreating\n") % f)
                     get[f] = n
                 else:
                     self.ui.debug(_("local deleted %s\n") % f)
 
         del mw, m1, m2, ma
 
         if force:
             for f in merge:
                 get[f] = merge[f][1]
             merge = {}
 
         if linear_path or force:
             # we don't need to do any magic, just jump to the new rev
             branch_merge = False
             p1, p2 = p2, nullid
         else:
             if not allow:
                 self.ui.status(_("this update spans a branch"
                                  " affecting the following files:\n"))
                 fl = merge.keys() + get.keys()
                 fl.sort()
                 for f in fl:
                     cf = ""
                     if f in merge:
                         cf = _(" (resolve)")
                     self.ui.status(" %s%s\n" % (f, cf))
                 self.ui.warn(_("aborting update spanning branches!\n"))
                 self.ui.status(_("(use 'hg merge' to merge across branches"
                                  " or 'hg update -C' to lose changes)\n"))
                 return 1
             branch_merge = True
 
         xp1 = hex(p1)
         xp2 = hex(p2)
         if p2 == nullid: xxp2 = ''
         else: xxp2 = xp2
 
         self.hook('preupdate', throw=True, parent1=xp1, parent2=xxp2)
 
         # get the files we don't need to change
         files = get.keys()
         files.sort()
         for f in files:
             if f[0] == "/":
                 continue
             self.ui.note(_("getting %s\n") % f)
             t = self.file(f).read(get[f])
             self.wwrite(f, t)
             util.set_exec(self.wjoin(f), mf2[f])
             if moddirstate:
                 if branch_merge:
                     self.dirstate.update([f], 'n', st_mtime=-1)
                 else:
                     self.dirstate.update([f], 'n')
 
         # merge the tricky bits
         failedmerge = []
         files = merge.keys()
         files.sort()
         for f in files:
             self.ui.status(_("merging %s\n") % f)
             my, other, flag = merge[f]
             ret = self.merge3(f, my, other, xp1, xp2)
             if ret:
                 err = True
                 failedmerge.append(f)
             util.set_exec(self.wjoin(f), flag)
             if moddirstate:
                 if branch_merge:
                     # We've done a branch merge, mark this file as merged
                     # so that we properly record the merger later
                     self.dirstate.update([f], 'm')
                 else:
                     # We've update-merged a locally modified file, so
                     # we set the dirstate to emulate a normal checkout
                     # of that file some time in the past. Thus our
                     # merge will appear as a normal local file
                     # modification.
                     f_len = len(self.file(f).read(other))
                     self.dirstate.update([f], 'n', st_size=f_len, st_mtime=-1)
 
         remove.sort()
         for f in remove:
             self.ui.note(_("removing %s\n") % f)
             util.audit_path(f)
             try:
                 util.unlink(self.wjoin(f))
             except OSError, inst:
                 if inst.errno != errno.ENOENT:
                     self.ui.warn(_("update failed to remove %s: %s!\n") %
                                  (f, inst.strerror))
         if moddirstate:
             if branch_merge:
                 self.dirstate.update(remove, 'r')
             else:
                 self.dirstate.forget(remove)
 
         if moddirstate:
             self.dirstate.setparents(p1, p2)
 
         if show_stats:
             stats = ((len(get), _("updated")),
                      (len(merge) - len(failedmerge), _("merged")),
                      (len(remove), _("removed")),
                      (len(failedmerge), _("unresolved")))
             note = ", ".join([_("%d files %s") % s for s in stats])
             self.ui.status("%s\n" % note)
         if moddirstate:
             if branch_merge:
                 if failedmerge:
                     self.ui.status(_("There are unresolved merges,"
                                     " you can redo the full merge using:\n"
                                     "  hg update -C %s\n"
                                     "  hg merge %s\n"
                                     % (self.changelog.rev(p1),
                                         self.changelog.rev(p2))))
                 else:
                     self.ui.status(_("(branch merge, don't forget to commit)\n"))
             elif failedmerge:
                 self.ui.status(_("There are unresolved merges with"
                                  " locally modified files.\n"))
 
         self.hook('update', parent1=xp1, parent2=xxp2, error=int(err))
         return err
 
     def merge3(self, fn, my, other, p1, p2):
         """perform a 3-way merge in the working directory"""
 
         def temp(prefix, node):
             pre = "%s~%s." % (os.path.basename(fn), prefix)
             (fd, name) = tempfile.mkstemp(prefix=pre)
             f = os.fdopen(fd, "wb")
             self.wwrite(fn, fl.read(node), f)
             f.close()
             return name
 
         fl = self.file(fn)
         base = fl.ancestor(my, other)
         a = self.wjoin(fn)
         b = temp("base", base)
         c = temp("other", other)
 
         self.ui.note(_("resolving %s\n") % fn)
         self.ui.debug(_("file %s: my %s other %s ancestor %s\n") %
                               (fn, short(my), short(other), short(base)))
 
         cmd = (os.environ.get("HGMERGE") or self.ui.config("ui", "merge")
                or "hgmerge")
         r = util.system('%s "%s" "%s" "%s"' % (cmd, a, b, c), cwd=self.root,
                         environ={'HG_FILE': fn,
                                  'HG_MY_NODE': p1,
                                  'HG_OTHER_NODE': p2,
                                  'HG_FILE_MY_NODE': hex(my),
                                  'HG_FILE_OTHER_NODE': hex(other),
                                  'HG_FILE_BASE_NODE': hex(base)})
         if r:
             self.ui.warn(_("merging %s failed!\n") % fn)
 
         os.unlink(b)
         os.unlink(c)
         return r
 
     def verify(self):
         filelinkrevs = {}
         filenodes = {}
         changesets = revisions = files = 0
         errors = [0]
         warnings = [0]
         neededmanifests = {}
 
         def err(msg):
             self.ui.warn(msg + "\n")
             errors[0] += 1
 
         def warn(msg):
             self.ui.warn(msg + "\n")
             warnings[0] += 1
 
         def checksize(obj, name):
             d = obj.checksize()
             if d[0]:
                 err(_("%s data length off by %d bytes") % (name, d[0]))
             if d[1]:
                 err(_("%s index contains %d extra bytes") % (name, d[1]))
 
         def checkversion(obj, name):
             if obj.version != revlog.REVLOGV0:
                 if not revlogv1:
                     warn(_("warning: `%s' uses revlog format 1") % name)
             elif revlogv1:
                 warn(_("warning: `%s' uses revlog format 0") % name)
 
         revlogv1 = self.revlogversion != revlog.REVLOGV0
         if self.ui.verbose or revlogv1 != self.revlogv1:
             self.ui.status(_("repository uses revlog format %d\n") %
                            (revlogv1 and 1 or 0))
 
         seen = {}
         self.ui.status(_("checking changesets\n"))
         checksize(self.changelog, "changelog")
 
         for i in range(self.changelog.count()):
             changesets += 1
             n = self.changelog.node(i)
             l = self.changelog.linkrev(n)
             if l != i:
                 err(_("incorrect link (%d) for changeset revision %d") %(l, i))
             if n in seen:
                 err(_("duplicate changeset at revision %d") % i)
             seen[n] = 1
 
             for p in self.changelog.parents(n):
                 if p not in self.changelog.nodemap:
                     err(_("changeset %s has unknown parent %s") %
                                  (short(n), short(p)))
             try:
                 changes = self.changelog.read(n)
             except KeyboardInterrupt:
                 self.ui.warn(_("interrupted"))
                 raise
             except Exception, inst:
                 err(_("unpacking changeset %s: %s") % (short(n), inst))
                 continue
 
             neededmanifests[changes[0]] = n
 
             for f in changes[3]:
                 filelinkrevs.setdefault(f, []).append(i)
 
         seen = {}
         self.ui.status(_("checking manifests\n"))
         checkversion(self.manifest, "manifest")
         checksize(self.manifest, "manifest")
 
         for i in range(self.manifest.count()):
             n = self.manifest.node(i)
             l = self.manifest.linkrev(n)
 
             if l < 0 or l >= self.changelog.count():
                 err(_("bad manifest link (%d) at revision %d") % (l, i))
 
             if n in neededmanifests:
                 del neededmanifests[n]
 
             if n in seen:
                 err(_("duplicate manifest at revision %d") % i)
 
             seen[n] = 1
 
             for p in self.manifest.parents(n):
                 if p not in self.manifest.nodemap:
                     err(_("manifest %s has unknown parent %s") %
                         (short(n), short(p)))
 
             try:
                 delta = mdiff.patchtext(self.manifest.delta(n))
             except KeyboardInterrupt:
                 self.ui.warn(_("interrupted"))
                 raise
             except Exception, inst:
                 err(_("unpacking manifest %s: %s") % (short(n), inst))
                 continue
 
             try:
                 ff = [ l.split('\0') for l in delta.splitlines() ]
                 for f, fn in ff:
                     filenodes.setdefault(f, {})[bin(fn[:40])] = 1
             except (ValueError, TypeError), inst:
                 err(_("broken delta in manifest %s: %s") % (short(n), inst))
 
         self.ui.status(_("crosschecking files in changesets and manifests\n"))
 
         for m, c in neededmanifests.items():
             err(_("Changeset %s refers to unknown manifest %s") %
                 (short(m), short(c)))
         del neededmanifests
 
         for f in filenodes:
             if f not in filelinkrevs:
                 err(_("file %s in manifest but not in changesets") % f)
 
         for f in filelinkrevs:
             if f not in filenodes:
                 err(_("file %s in changeset but not in manifest") % f)
 
         self.ui.status(_("checking files\n"))
         ff = filenodes.keys()
         ff.sort()
         for f in ff:
             if f == "/dev/null":
                 continue
             files += 1
             if not f:
                 err(_("file without name in manifest %s") % short(n))
                 continue
             fl = self.file(f)
             checkversion(fl, f)
             checksize(fl, f)
 
             nodes = {nullid: 1}
             seen = {}
             for i in range(fl.count()):
                 revisions += 1
                 n = fl.node(i)
 
                 if n in seen:
                     err(_("%s: duplicate revision %d") % (f, i))
                 if n not in filenodes[f]:
                     err(_("%s: %d:%s not in manifests") % (f, i, short(n)))
                 else:
                     del filenodes[f][n]
 
                 flr = fl.linkrev(n)
                 if flr not in filelinkrevs.get(f, []):
                     err(_("%s:%s points to unexpected changeset %d")
                             % (f, short(n), flr))
                 else:
                     filelinkrevs[f].remove(flr)
 
                 # verify contents
                 try:
                     t = fl.read(n)
                 except KeyboardInterrupt:
                     self.ui.warn(_("interrupted"))
                     raise
                 except Exception, inst:
                     err(_("unpacking file %s %s: %s") % (f, short(n), inst))
 
                 # verify parents
                 (p1, p2) = fl.parents(n)
                 if p1 not in nodes:
                     err(_("file %s:%s unknown parent 1 %s") %
                         (f, short(n), short(p1)))
                 if p2 not in nodes:
                     err(_("file %s:%s unknown parent 2 %s") %
                             (f, short(n), short(p1)))
                 nodes[n] = 1
 
             # cross-check
             for node in filenodes[f]:
                 err(_("node %s in manifests not in %s") % (hex(node), f))
 
         self.ui.status(_("%d files, %d changesets, %d total revisions\n") %
                        (files, changesets, revisions))
 
         if warnings[0]:
             self.ui.warn(_("%d warnings encountered!\n") % warnings[0])
         if errors[0]:
             self.ui.warn(_("%d integrity errors encountered!\n") % errors[0])
             return 1
 
     def stream_in(self, remote):
+        fp = remote.stream_out()
+        resp = int(fp.readline())
+        if resp != 0:
+            raise util.Abort(_('operation forbidden by server'))
         self.ui.status(_('streaming all changes\n'))
-        fp = remote.stream_out()
         total_files, total_bytes = map(int, fp.readline().split(' ', 1))
         self.ui.status(_('%d files to transfer, %s of data\n') %
                        (total_files, util.bytecount(total_bytes)))
         start = time.time()
         for i in xrange(total_files):
             name, size = fp.readline().split('\0', 1)
             size = int(size)
             self.ui.debug('adding %s (%s)\n' % (name, util.bytecount(size)))
             ofp = self.opener(name, 'w')
             for chunk in util.filechunkiter(fp, limit=size):
                 ofp.write(chunk)
             ofp.close()
         elapsed = time.time() - start
         self.ui.status(_('transferred %s in %.1f seconds (%s/sec)\n') %
                        (util.bytecount(total_bytes), elapsed,
                         util.bytecount(total_bytes / elapsed)))
         self.reload()
         return len(self.heads()) + 1
         
     def clone(self, remote, heads=[], stream=False):
         '''clone remote repository.
 
         keyword arguments:
         heads: list of revs to clone (forces use of pull)
-        pull: force use of pull, even if remote can stream'''
+        stream: use streaming clone if possible'''
 
-        # now, all clients that can stream can read repo formats
-        # supported by all servers that can stream.
+        # now, all clients that can request uncompressed clones can
+        # read repo formats supported by all servers that can serve
+        # them.
 
         # if revlog format changes, client will have to check version
-        # and format flags on "stream" capability, and stream only if
-        # compatible.
+        # and format flags on "stream" capability, and use
+        # uncompressed only if compatible.
 
         if stream and not heads and remote.capable('stream'):
             return self.stream_in(remote)
         return self.pull(remote, heads)
 
 # used to avoid circular references so destructors work
 def aftertrans(base):
     p = base
     def a():
         util.rename(os.path.join(p, "journal"), os.path.join(p, "undo"))
         util.rename(os.path.join(p, "journal.dirstate"),
                     os.path.join(p, "undo.dirstate"))
     return a
 

mercurial/sshserver.py

@@ -1,171 +1,173 b''
 # sshserver.py - ssh protocol server support for mercurial
 #
 # Copyright 2005 Matt Mackall <[e-mail]>
 #
 # This software may be used and distributed according to the terms
 # of the GNU General Public License, incorporated herein by reference.
 
 from demandload import demandload
 from i18n import gettext as _
 from node import *
 demandload(globals(), "os streamclone sys tempfile util")
 
 class sshserver(object):
     def __init__(self, ui, repo):
         self.ui = ui
         self.repo = repo
         self.lock = None
         self.fin = sys.stdin
         self.fout = sys.stdout
 
         sys.stdout = sys.stderr
 
         # Prevent insertion/deletion of CRs
         util.set_binary(self.fin)
         util.set_binary(self.fout)
 
     def getarg(self):
         argline = self.fin.readline()[:-1]
         arg, l = argline.split()
         val = self.fin.read(int(l))
         return arg, val
 
     def respond(self, v):
         self.fout.write("%d\n" % len(v))
         self.fout.write(v)
         self.fout.flush()
 
     def serve_forever(self):
         while self.serve_one(): pass
         sys.exit(0)
 
     def serve_one(self):
         cmd = self.fin.readline()[:-1]
         if cmd:
             impl = getattr(self, 'do_' + cmd, None)
             if impl: impl()
             else: self.respond("")
         return cmd != ''
 
     def do_heads(self):
         h = self.repo.heads()
         self.respond(" ".join(map(hex, h)) + "\n")
 
     def do_hello(self):
         '''the hello command returns a set of lines describing various
         interesting things about the server, in an RFC822-like format.
         Currently the only one defined is "capabilities", which
         consists of a line in the form:
 
         capabilities: space separated list of tokens
         '''
 
-        r = "capabilities: unbundle stream=%d\n" % (self.repo.revlogversion,)
-        self.respond(r)
+        caps = ['unbundle']
+        if self.ui.configbool('server', 'stream'):
+            caps.append('stream=%d' % self.repo.revlogversion)
+        self.respond("capabilities: %s\n" % (' '.join(caps),))
 
     def do_lock(self):
         '''DEPRECATED - allowing remote client to lock repo is not safe'''
 
         self.lock = self.repo.lock()
         self.respond("")
 
     def do_unlock(self):
         '''DEPRECATED'''
 
         if self.lock:
             self.lock.release()
         self.lock = None
         self.respond("")
 
     def do_branches(self):
         arg, nodes = self.getarg()
         nodes = map(bin, nodes.split(" "))
         r = []
         for b in self.repo.branches(nodes):
             r.append(" ".join(map(hex, b)) + "\n")
         self.respond("".join(r))
 
     def do_between(self):
         arg, pairs = self.getarg()
         pairs = [map(bin, p.split("-")) for p in pairs.split(" ")]
         r = []
         for b in self.repo.between(pairs):
             r.append(" ".join(map(hex, b)) + "\n")
         self.respond("".join(r))
 
     def do_changegroup(self):
         nodes = []
         arg, roots = self.getarg()
         nodes = map(bin, roots.split(" "))
 
         cg = self.repo.changegroup(nodes, 'serve')
         while True:
             d = cg.read(4096)
             if not d:
                 break
             self.fout.write(d)
 
         self.fout.flush()
 
     def do_addchangegroup(self):
         '''DEPRECATED'''
 
         if not self.lock:
             self.respond("not locked")
             return
 
         self.respond("")
         r = self.repo.addchangegroup(self.fin, 'serve')
         self.respond(str(r))
 
     def do_unbundle(self):
         their_heads = self.getarg()[1].split()
 
         def check_heads():
             heads = map(hex, self.repo.heads())
             return their_heads == [hex('force')] or their_heads == heads
 
         # fail early if possible
         if not check_heads():
             self.respond(_('unsynced changes'))
             return
 
         self.respond('')
 
         # write bundle data to temporary file because it can be big
 
         try:
             fd, tempname = tempfile.mkstemp(prefix='hg-unbundle-')
             fp = os.fdopen(fd, 'wb+')
 
             count = int(self.fin.readline())
             while count:
                 fp.write(self.fin.read(count))
                 count = int(self.fin.readline())
 
             was_locked = self.lock is not None
             if not was_locked:
                 self.lock = self.repo.lock()
             try:
                 if not check_heads():
                     # someone else committed/pushed/unbundled while we
                     # were transferring data
                     self.respond(_('unsynced changes'))
                     return
                 self.respond('')
 
                 # push can proceed
 
                 fp.seek(0)
                 r = self.repo.addchangegroup(fp, 'serve')
                 self.respond(str(r))
             finally:
                 if not was_locked:
                     self.lock.release()
                     self.lock = None
         finally:
             fp.close()
             os.unlink(tempname)
 
     def do_stream_out(self):
         streamclone.stream_out(self.repo, self.fout)

mercurial/streamclone.py

@@ -1,82 +1,89 b''
 # streamclone.py - streaming clone server support for mercurial
 #
 # Copyright 2006 Vadim Gelfer <[e-mail]>
 #
 # This software may be used and distributed according to the terms
 # of the GNU General Public License, incorporated herein by reference.
 
 from demandload import demandload
 from i18n import gettext as _
 demandload(globals(), "os stat util")
 
 # if server supports streaming clone, it advertises "stream"
 # capability with value that is version+flags of repo it is serving.
 # client only streams if it can read that repo format.
 
 def walkrepo(root):
     '''iterate over metadata files in repository.
     walk in natural (sorted) order.
     yields 2-tuples: name of .d or .i file, size of file.'''
 
     strip_count = len(root) + len(os.sep)
     def walk(path, recurse):
         ents = os.listdir(path)
         ents.sort()
         for e in ents:
             pe = os.path.join(path, e)
             st = os.lstat(pe)
             if stat.S_ISDIR(st.st_mode):
                 if recurse:
                     for x in walk(pe, True):
                         yield x
             else:
                 if not stat.S_ISREG(st.st_mode) or len(e) < 2:
                     continue
                 sfx = e[-2:]
                 if sfx in ('.d', '.i'):
                     yield pe[strip_count:], st.st_size
     # write file data first
     for x in walk(os.path.join(root, 'data'), True):
         yield x
     # write manifest before changelog
     meta = list(walk(root, False))
     meta.sort(reverse=True)
     for x in meta:
         yield x
 
 # stream file format is simple.
 #
 # server writes out line that says how many files, how many total
 # bytes.  separator is ascii space, byte counts are strings.
 #
 # then for each file:
 #
 #   server writes out line that says file name, how many bytes in
 #   file.  separator is ascii nul, byte count is string.
 #
 #   server writes out raw file data.
 
 def stream_out(repo, fileobj):
     '''stream out all metadata files in repository.
     writes to file-like object, must support write() and optional flush().'''
+
+    if not repo.ui.configbool('server', 'stream'):
+        fileobj.write('1\n')
+        return
+
+    fileobj.write('0\n')
+
     # get consistent snapshot of repo. lock during scan so lock not
     # needed while we stream, and commits can happen.
     lock = repo.lock()
     repo.ui.debug('scanning\n')
     entries = []
     total_bytes = 0
     for name, size in walkrepo(repo.path):
         entries.append((name, size))
         total_bytes += size
     lock.release()
 
     repo.ui.debug('%d files, %d bytes to transfer\n' %
                   (len(entries), total_bytes))
     fileobj.write('%d %d\n' % (len(entries), total_bytes))
     for name, size in entries:
         repo.ui.debug('sending %s (%d bytes)\n' % (name, size))
         fileobj.write('%s\0%d\n' % (name, size))
         for chunk in util.filechunkiter(repo.opener(name), limit=size):
             fileobj.write(chunk)
     flush = getattr(fileobj, 'flush', None)
     if flush: flush()

tests/test-http

@@ -1,25 +1,25 b''
 #!/bin/sh
 
-mkdir test
+hg init test
 cd test
 echo foo>foo
-hg init
-hg addremove
-hg commit -m 1
-hg verify
-hg serve -p 20059 -d --pid-file=hg.pid
-cat hg.pid >> $DAEMON_PIDS
+hg commit -A -d '0 0' -m 1
+hg --config server.stream=True serve -p 20059 -d --pid-file=hg1.pid
+cat hg1.pid >> $DAEMON_PIDS
+hg serve -p 20060 -d --pid-file=hg2.pid
+cat hg2.pid >> $DAEMON_PIDS
 cd ..
 
 echo % clone via stream
-http_proxy= hg clone --stream http://localhost:20059/ copy 2>&1 | \
+http_proxy= hg clone --uncompressed http://localhost:20059/ copy 2>&1 | \
   sed -e 's/[0-9][0-9.]*/XXX/g'
 cd copy
 hg verify
 
-cd ..
+echo % try to clone via stream, should use pull instead
+http_proxy= hg clone --uncompressed http://localhost:20060/ copy2
 
 echo % clone via pull
 http_proxy= hg clone http://localhost:20059/ copy-pull
 cd copy-pull
 hg verify

tests/test-http-proxy

@@ -1,41 +1,41 b''
 #!/bin/sh
 
 hg init a
 cd a
 echo a > a
 hg ci -Ama -d '1123456789 0'
-hg serve -p 20059 -d --pid-file=hg.pid
+hg --config server.stream=True serve -p 20059 -d --pid-file=hg.pid
 cat hg.pid >> $DAEMON_PIDS
 
 cd ..
 ("$TESTDIR/tinyproxy.py" 20060 localhost >proxy.log 2>&1 </dev/null &
 echo $! > proxy.pid)
 cat proxy.pid >> $DAEMON_PIDS
 sleep 2
 
 echo %% url for proxy, stream
-http_proxy=http://localhost:20060/ hg --config http_proxy.always=True clone --stream http://localhost:20059/ b | \
+http_proxy=http://localhost:20060/ hg --config http_proxy.always=True clone --uncompressed http://localhost:20059/ b | \
   sed -e 's/[0-9][0-9.]*/XXX/g'
 cd b
 hg verify
 cd ..
 
 echo %% url for proxy, pull
 http_proxy=http://localhost:20060/ hg --config http_proxy.always=True clone http://localhost:20059/ b-pull
 cd b-pull
 hg verify
 cd ..
 
 echo %% host:port for proxy
 http_proxy=localhost:20060 hg clone --config http_proxy.always=True http://localhost:20059/ c
 
 echo %% proxy url with user name and password
 http_proxy=http://user:passwd@localhost:20060 hg clone --config http_proxy.always=True http://localhost:20059/ d
 
 echo %% url with user name and password
 http_proxy=http://user:passwd@localhost:20060 hg clone --config http_proxy.always=True http://user:passwd@localhost:20059/ e
 
 echo %% bad host:port for proxy
 http_proxy=localhost:20061 hg clone --config http_proxy.always=True http://localhost:20059/ f
 
 exit 0

tests/test-http.out

@@ -1,29 +1,30 b''
-(the addremove command is deprecated; use add and remove --after instead)
 adding foo
-checking changesets
-checking manifests
-crosschecking files in changesets and manifests
-checking files
-1 files, 1 changesets, 1 total revisions
 % clone via stream
 streaming all changes
 XXX files to transfer, XXX bytes of data
 transferred XXX bytes in XXX seconds (XXX KB/sec)
 XXX files updated, XXX files merged, XXX files removed, XXX files unresolved
 checking changesets
 checking manifests
 crosschecking files in changesets and manifests
 checking files
 1 files, 1 changesets, 1 total revisions
+% try to clone via stream, should use pull instead
+requesting all changes
+adding changesets
+adding manifests
+adding file changes
+added 1 changesets with 1 changes to 1 files
+1 files updated, 0 files merged, 0 files removed, 0 files unresolved
 % clone via pull
 requesting all changes
 adding changesets
 adding manifests
 adding file changes
 added 1 changesets with 1 changes to 1 files
 1 files updated, 0 files merged, 0 files removed, 0 files unresolved
 checking changesets
 checking manifests
 crosschecking files in changesets and manifests
 checking files
 1 files, 1 changesets, 1 total revisions

tests/test-ssh

@@ -1,90 +1,92 b''
 #!/bin/sh
 
 # This test tries to exercise the ssh functionality with a dummy script
 
 cat <<'EOF' > dummyssh
 #!/bin/sh
 # this attempts to deal with relative pathnames
 cd `dirname $0`
 
 # check for proper args
 if [ $1 != "user@dummy" ] ; then
 	exit -1
 fi
 
 # check that we're in the right directory
 if [ ! -x dummyssh ] ; then
 	exit -1
 fi
 
 echo Got arguments 1:$1 2:$2 3:$3 4:$4 5:$5 >> dummylog
 $2
 EOF
 chmod +x dummyssh
 
 echo "# creating 'remote'"
 hg init remote
 cd remote
 echo this > foo
 hg ci -A -m "init" -d "1000000 0" foo
+echo '[server]' > .hg/hgrc
+echo 'stream = True' >> .hg/hgrc
 
 cd ..
 
 echo "# clone remote via stream"
-hg clone -e ./dummyssh --stream ssh://user@dummy/remote local-stream 2>&1 | \
+hg clone -e ./dummyssh --uncompressed ssh://user@dummy/remote local-stream 2>&1 | \
   sed -e 's/[0-9][0-9.]*/XXX/g'
 cd local-stream
 hg verify
 cd ..
 
 echo "# clone remote via pull"
 hg clone -e ./dummyssh ssh://user@dummy/remote local
 
 echo "# verify"
 cd local
 hg verify
 
 echo "# empty default pull"
 hg paths
 hg pull -e ../dummyssh
 
 echo "# local change"
 echo bleah > foo
 hg ci -m "add" -d "1000000 0"
 
 echo "# updating rc"
 echo "default-push = ssh://user@dummy/remote" >> .hg/hgrc
 echo "[ui]" >> .hg/hgrc
 echo "ssh = ../dummyssh" >> .hg/hgrc
 
 echo "# find outgoing"
 hg out ssh://user@dummy/remote
 
 echo "# find incoming on the remote side"
 hg incoming -R ../remote -e ../dummyssh ssh://user@dummy/local
 
 echo "# push"
 hg push
 
 cd ../remote
 
 echo "# check remote tip"
 hg tip
 hg verify
 hg cat foo
 
 echo z > z
 hg ci -A -m z -d '1000001 0' z
 
 cd ../local
 echo r > r
 hg ci -A -m z -d '1000002 0' r
 
 echo "# push should fail"
 hg push
 
 echo "# push should succeed"
 hg push -f
 
 cd ..
 cat dummylog