##// END OF EJS Templates
findrenames: Optimise "addremove -s100" by matching files by their SHA1 hashes....
findrenames: Optimise "addremove -s100" by matching files by their SHA1 hashes. We speed up 'findrenames' for the usecase when a user specifies they want a similarity of 100% by matching files by their exact SHA1 hash value. This reduces the number of comparisons required to find exact matches from O(n^2) to O(n). While it would be nice if we could just use mercurial's pre-calculated SHA1 hash for existing files, this hash includes the file's ancestor information making it unsuitable for our purposes. Instead, we calculate the hash of old content from scratch. The following benchmarks were taken on the current head of crew: addremove 100% similarity: rm -rf *; hg up -C; mv tests tests.new hg --time addremove -s100 --dry-run before: real 176.350 secs (user 128.890+0.000 sys 47.430+0.000) after: real 2.130 secs (user 1.890+0.000 sys 0.240+0.000) addremove 75% similarity: rm -rf *; hg up -C; mv tests tests.new; \ for i in tests.new/*; do echo x >> $i; done hg --time addremove -s75 --dry-run before: real 264.560 secs (user 215.130+0.000 sys 49.410+0.000) after: real 218.710 secs (user 172.790+0.000 sys 45.870+0.000)

File last commit:

r10282:08a0f04b default
r11060:e6df0177 default
Show More
sshrepo.py
269 lines | 8.0 KiB | text/x-python | PythonLexer
mpm@selenic.com
sshrepo: adjust file comment
r1096 # sshrepo.py - ssh repository proxy class for mercurial
mpm@selenic.com
Break apart hg.py...
r1089 #
Vadim Gelfer
update copyrights.
r2859 # Copyright 2005, 2006 Matt Mackall <mpm@selenic.com>
mpm@selenic.com
Break apart hg.py...
r1089 #
Martin Geisler
updated license to be explicit about GPL version 2
r8225 # This software may be used and distributed according to the terms of the
Matt Mackall
Update license to GPLv2+
r10263 # GNU General Public License version 2 or any later version.
mpm@selenic.com
Break apart hg.py...
r1089
Joel Rosdahl
Expand import * to allow Pyflakes to find problems
r6211 from node import bin, hex
Matt Mackall
Simplify i18n imports
r3891 from i18n import _
Henrik Stuart
support encoding fallback in branchmap to maintain compatibility with 1.3.x
r9861 import repo, util, error, encoding
Henrik Stuart
named branches: client branchmap wire protocol support (issue736)...
r8563 import re, urllib
mpm@selenic.com
Break apart hg.py...
r1089
Matt Mackall
remoterepo: no longer needed...
r6313 class remotelock(object):
def __init__(self, repo):
self.repo = repo
def release(self):
self.repo.unlock()
self.repo = None
def __del__(self):
if self.repo:
self.release()
class sshrepository(repo.repository):
Sean Meiners
Added ability to clone from a local repository to a (new) remote one....
r2549 def __init__(self, ui, path, create=0):
Vadim Gelfer
hooks: add url to changegroup, incoming, prechangegroup, pretxnchangegroup hooks...
r2673 self._url = path
mpm@selenic.com
Break apart hg.py...
r1089 self.ui = ui
Benoit Boissinot
sshrepo: fix the parsing of the ssh url
r3599 m = re.match(r'^ssh://(([^@]+)@)?([^:/]+)(:(\d+))?(/(.*))?$', path)
mpm@selenic.com
Break apart hg.py...
r1089 if not m:
Matt Mackall
sshrepo: change raise_ to abort
r7642 self.abort(error.RepoError(_("couldn't parse location %s") % path))
mpm@selenic.com
Break apart hg.py...
r1089
self.user = m.group(2)
self.host = m.group(3)
self.port = m.group(5)
self.path = m.group(7) or "."
sshcmd = self.ui.config("ui", "ssh", "ssh")
remotecmd = self.ui.config("ui", "remotecmd", "hg")
Sean Meiners
Added ability to clone from a local repository to a (new) remote one....
r2549
Steve Borho
win32: fix ssh://host:port when using Plink...
r5644 args = util.sshargs(sshcmd, self.host, self.user, self.port)
Sean Meiners
Added ability to clone from a local repository to a (new) remote one....
r2549 if create:
cmd = '%s %s "%s init %s"'
cmd = cmd % (sshcmd, args, remotecmd, self.path)
Martin Geisler
i18n: mark strings for translation in Mercurial
r6953 ui.note(_('running %s\n') % cmd)
Alexis S. L. Carvalho
sshrepo: fix Windows command quoting
r5292 res = util.system(cmd)
Sean Meiners
Added ability to clone from a local repository to a (new) remote one....
r2549 if res != 0:
Matt Mackall
sshrepo: change raise_ to abort
r7642 self.abort(error.RepoError(_("could not create remote repo")))
Sean Meiners
Added ability to clone from a local repository to a (new) remote one....
r2549
self.validate_repo(ui, sshcmd, args, remotecmd)
Vadim Gelfer
hooks: add url to changegroup, incoming, prechangegroup, pretxnchangegroup hooks...
r2673 def url(self):
return self._url
Sean Meiners
Added ability to clone from a local repository to a (new) remote one....
r2549 def validate_repo(self, ui, sshcmd, args, remotecmd):
Benoit Boissinot
sshrepo: flush stderr before connecting to the hg server
r3034 # cleanup up previous run
self.cleanup()
Bryan O'Sullivan
Give ssh a better chance of working on Windows....
r1330 cmd = '%s %s "%s -R %s serve --stdio"'
mpm@selenic.com
Break apart hg.py...
r1089 cmd = cmd % (sshcmd, args, remotecmd, self.path)
Alexis S. L. Carvalho
sshrepo: fix Windows command quoting
r5292 cmd = util.quotecommand(cmd)
Martin Geisler
i18n: mark strings for translation in Mercurial
r6953 ui.note(_('running %s\n') % cmd)
Martin Geisler
util: remove ignored mode argument in popen[23]
r8339 self.pipeo, self.pipei, self.pipee = util.popen3(cmd)
mpm@selenic.com
Break apart hg.py...
r1089
Matt Mackall
ssh: skip noise generated by remote shell...
r2028 # skip any noise generated by remote shell
Matt Mackall
ssh: add capability detection at startup...
r2421 self.do_cmd("hello")
Matt Mackall
ssh: skip noise generated by remote shell...
r2028 r = self.do_cmd("between", pairs=("%s-%s" % ("0"*40, "0"*40)))
Matt Mackall
ssh: gather initial output so we can do capability detection
r2420 lines = ["", "dummy"]
Thomas Arendsen Hein
Show remote ssh noise only with --debug and increase the limit to 500 lines....
r2046 max_noise = 500
Matt Mackall
ssh: gather initial output so we can do capability detection
r2420 while lines[-1] and max_noise:
l = r.readline()
Matt Mackall
ssh: skip noise generated by remote shell...
r2028 self.readerr()
Matt Mackall
ssh: gather initial output so we can do capability detection
r2420 if lines[-1] == "1\n" and l == "\n":
Matt Mackall
ssh: skip noise generated by remote shell...
r2028 break
Matt Mackall
ssh: gather initial output so we can do capability detection
r2420 if l:
Martin Geisler
do not attempt to translate ui.debug output
r9467 ui.debug("remote: ", l)
Matt Mackall
ssh: gather initial output so we can do capability detection
r2420 lines.append(l)
Thomas Arendsen Hein
Don't enter an endless loop if remote hg doesn't answer, show remote noise....
r2040 max_noise -= 1
else:
Matt Mackall
sshrepo: change raise_ to abort
r7642 self.abort(error.RepoError(_("no suitable response from remote hg")))
Matt Mackall
ssh: skip noise generated by remote shell...
r2028
Martin Geisler
util: use built-in set and frozenset...
r8150 self.capabilities = set()
Matt Mackall
replace various uses of list.reverse()
r8210 for l in reversed(lines):
Matt Mackall
ssh: add capability detection at startup...
r2421 if l.startswith("capabilities:"):
Bryan O'Sullivan
Turn capabilities into a mutable set, instead of a fixed tuple.
r5258 self.capabilities.update(l[:-1].split(":")[1].split())
Matt Mackall
ssh: add capability detection at startup...
r2421 break
mpm@selenic.com
Break apart hg.py...
r1089 def readerr(self):
while 1:
Vadim Gelfer
fix file handling bugs on windows....
r2176 size = util.fstat(self.pipee).st_size
Matt Mackall
many, many trivial check-code fixups
r10282 if size == 0:
break
mpm@selenic.com
Break apart hg.py...
r1089 l = self.pipee.readline()
Matt Mackall
many, many trivial check-code fixups
r10282 if not l:
break
Benoit Boissinot
i18n part2: use '_' for all strings who are part of the user interface
r1402 self.ui.status(_("remote: "), l)
mpm@selenic.com
Break apart hg.py...
r1089
Matt Mackall
sshrepo: change raise_ to abort
r7642 def abort(self, exception):
Alexis S. L. Carvalho
portability fix for test-ssh...
r3380 self.cleanup()
Thomas Arendsen Hein
Change sshrepo.repoerror() into a more flexible sshrepo.raise_()....
r3765 raise exception
Alexis S. L. Carvalho
portability fix for test-ssh...
r3380
Benoit Boissinot
sshrepo: flush stderr before connecting to the hg server
r3034 def cleanup(self):
mpm@selenic.com
Break apart hg.py...
r1089 try:
self.pipeo.close()
self.pipei.close()
Matt Mackall
Partially revert ssh change so we read all of remote ssh stream
r1358 # read the error descriptor until EOF
for l in self.pipee:
Benoit Boissinot
i18n part2: use '_' for all strings who are part of the user interface
r1402 self.ui.status(_("remote: "), l)
mpm@selenic.com
Break apart hg.py...
r1089 self.pipee.close()
except:
pass
Benoit Boissinot
sshrepo: flush stderr before connecting to the hg server
r3034 __del__ = cleanup
mpm@selenic.com
Break apart hg.py...
r1089 def do_cmd(self, cmd, **args):
Martin Geisler
do not attempt to translate ui.debug output
r9467 self.ui.debug("sending %s command\n" % cmd)
mpm@selenic.com
Break apart hg.py...
r1089 self.pipeo.write("%s\n" % cmd)
Dirkjan Ochtman
use dict.iteritems() rather than dict.items()...
r7622 for k, v in args.iteritems():
mpm@selenic.com
Break apart hg.py...
r1089 self.pipeo.write("%s %d\n" % (k, len(v)))
self.pipeo.write(v)
self.pipeo.flush()
return self.pipei
def call(self, cmd, **args):
Alexis S. L. Carvalho
sshrepo: be more careful while reading data...
r5978 self.do_cmd(cmd, **args)
return self._recv()
def _recv(self):
l = self.pipei.readline()
mpm@selenic.com
Break apart hg.py...
r1089 self.readerr()
try:
l = int(l)
except:
Matt Mackall
sshrepo: change raise_ to abort
r7642 self.abort(error.ResponseError(_("unexpected response:"), l))
Alexis S. L. Carvalho
sshrepo: be more careful while reading data...
r5978 return self.pipei.read(l)
def _send(self, data, flush=False):
self.pipeo.write("%d\n" % len(data))
if data:
self.pipeo.write(data)
if flush:
self.pipeo.flush()
self.readerr()
mpm@selenic.com
Break apart hg.py...
r1089
def lock(self):
self.call("lock")
return remotelock(self)
def unlock(self):
self.call("unlock")
Eric Hopper
Adding changegroupsubset and lookup to ssh protocol so pull -r and...
r3446 def lookup(self, key):
Bryan O'Sullivan
Push capability checking into protocol-level code.
r5259 self.requirecap('lookup', _('look up remote revision'))
Eric Hopper
Adding changegroupsubset and lookup to ssh protocol so pull -r and...
r3446 d = self.call("lookup", key=key)
Eric Hopper
sshrepo: add passing of lookup exceptions
r3447 success, data = d[:-1].split(" ", 1)
Thomas Arendsen Hein
Don't show traceback on 'hg clone -r unknown ssh://hg.example.com/'.
r3764 if int(success):
return bin(data)
else:
Matt Mackall
sshrepo: change raise_ to abort
r7642 self.abort(error.RepoError(data))
Eric Hopper
Adding changegroupsubset and lookup to ssh protocol so pull -r and...
r3446
mpm@selenic.com
Break apart hg.py...
r1089 def heads(self):
d = self.call("heads")
try:
return map(bin, d[:-1].split(" "))
except:
Matt Mackall
sshrepo: change raise_ to abort
r7642 self.abort(error.ResponseError(_("unexpected response:"), d))
mpm@selenic.com
Break apart hg.py...
r1089
Henrik Stuart
named branches: client branchmap wire protocol support (issue736)...
r8563 def branchmap(self):
d = self.call("branchmap")
try:
branchmap = {}
for branchpart in d.splitlines():
branchheads = branchpart.split(' ')
branchname = urllib.unquote(branchheads[0])
Sune Foldager
branchmap: fix defective fallback fix 0262bb59016f...
r9878 # Earlier servers (1.3.x) send branch names in (their) local
# charset. The best we can do is assume it's identical to our
# own local charset, in case it's not utf-8.
Henrik Stuart
support encoding fallback in branchmap to maintain compatibility with 1.3.x
r9861 try:
Sune Foldager
branchmap: fix defective fallback fix 0262bb59016f...
r9878 branchname.decode('utf-8')
Henrik Stuart
support encoding fallback in branchmap to maintain compatibility with 1.3.x
r9861 except UnicodeDecodeError:
Sune Foldager
branchmap: fix defective fallback fix 0262bb59016f...
r9878 branchname = encoding.fromlocal(branchname)
Henrik Stuart
named branches: client branchmap wire protocol support (issue736)...
r8563 branchheads = [bin(x) for x in branchheads[1:]]
branchmap[branchname] = branchheads
return branchmap
except:
raise error.ResponseError(_("unexpected response:"), d)
mpm@selenic.com
Break apart hg.py...
r1089 def branches(self, nodes):
n = " ".join(map(hex, nodes))
d = self.call("branches", nodes=n)
try:
Matt Mackall
many, many trivial check-code fixups
r10282 br = [tuple(map(bin, b.split(" "))) for b in d.splitlines()]
mpm@selenic.com
Break apart hg.py...
r1089 return br
except:
Matt Mackall
sshrepo: change raise_ to abort
r7642 self.abort(error.ResponseError(_("unexpected response:"), d))
mpm@selenic.com
Break apart hg.py...
r1089
def between(self, pairs):
Benoit Boissinot
protocol/between: the protocol expects to have ' '-separated tuples
r7207 n = " ".join(["-".join(map(hex, p)) for p in pairs])
mpm@selenic.com
Break apart hg.py...
r1089 d = self.call("between", pairs=n)
try:
Matt Mackall
many, many trivial check-code fixups
r10282 p = [l and map(bin, l.split(" ")) or [] for l in d.splitlines()]
mpm@selenic.com
Break apart hg.py...
r1089 return p
except:
Matt Mackall
sshrepo: change raise_ to abort
r7642 self.abort(error.ResponseError(_("unexpected response:"), d))
mpm@selenic.com
Break apart hg.py...
r1089
Vadim Gelfer
add preoutgoing and outgoing hooks....
r1736 def changegroup(self, nodes, kind):
mpm@selenic.com
Break apart hg.py...
r1089 n = " ".join(map(hex, nodes))
Benoit Boissinot
fix unused variable warning from pychecker
r2449 return self.do_cmd("changegroup", roots=n)
mpm@selenic.com
Break apart hg.py...
r1089
Eric Hopper
Adding changegroupsubset and lookup to ssh protocol so pull -r and...
r3446 def changegroupsubset(self, bases, heads, kind):
Bryan O'Sullivan
Push capability checking into protocol-level code.
r5259 self.requirecap('changegroupsubset', _('look up remote changes'))
Eric Hopper
Adding changegroupsubset and lookup to ssh protocol so pull -r and...
r3446 bases = " ".join(map(hex, bases))
heads = " ".join(map(hex, heads))
return self.do_cmd("changegroupsubset", bases=bases, heads=heads)
Vadim Gelfer
extend network protocol to stop clients from locking servers...
r2439 def unbundle(self, cg, heads, source):
d = self.call("unbundle", heads=' '.join(map(hex, heads)))
if d:
Alexis S. L. Carvalho
Fix sshrepo.unbundle...
r5190 # remote may send "unsynced changes"
Matt Mackall
sshrepo: change raise_ to abort
r7642 self.abort(error.RepoError(_("push refused: %s") % d))
Vadim Gelfer
extend network protocol to stop clients from locking servers...
r2439
while 1:
d = cg.read(4096)
Alexis S. L. Carvalho
sshrepo: be more careful while reading data...
r5978 if not d:
break
self._send(d)
Vadim Gelfer
extend network protocol to stop clients from locking servers...
r2439
Alexis S. L. Carvalho
sshrepo: be more careful while reading data...
r5978 self._send("", flush=True)
Vadim Gelfer
extend network protocol to stop clients from locking servers...
r2439
Alexis S. L. Carvalho
sshrepo: be more careful while reading data...
r5978 r = self._recv()
Alexis S. L. Carvalho
Fix sshrepo.unbundle...
r5190 if r:
# remote may send "unsynced changes"
Matt Mackall
sshrepo: change raise_ to abort
r7642 self.abort(error.RepoError(_("push failed: %s") % r))
Alexis S. L. Carvalho
Fix sshrepo.unbundle...
r5190
Alexis S. L. Carvalho
sshrepo: be more careful while reading data...
r5978 r = self._recv()
try:
return int(r)
except:
Matt Mackall
sshrepo: change raise_ to abort
r7642 self.abort(error.ResponseError(_("unexpected response:"), r))
Vadim Gelfer
extend network protocol to stop clients from locking servers...
r2439
Vadim Gelfer
hooks: add url to changegroup, incoming, prechangegroup, pretxnchangegroup hooks...
r2673 def addchangegroup(self, cg, source, url):
mpm@selenic.com
Break apart hg.py...
r1089 d = self.call("addchangegroup")
if d:
Matt Mackall
sshrepo: change raise_ to abort
r7642 self.abort(error.RepoError(_("push refused: %s") % d))
mpm@selenic.com
Break apart hg.py...
r1089 while 1:
d = cg.read(4096)
Alexis S. L. Carvalho
sshrepo: be more careful while reading data...
r5978 if not d:
break
mpm@selenic.com
Break apart hg.py...
r1089 self.pipeo.write(d)
self.readerr()
self.pipeo.flush()
self.readerr()
Alexis S. L. Carvalho
sshrepo: be more careful while reading data...
r5978 r = self._recv()
Vadim Gelfer
add merge command. means same thing as "update -m"....
r2019 if not r:
return 1
Alexis S. L. Carvalho
sshrepo: be more careful while reading data...
r5978 try:
return int(r)
except:
Matt Mackall
sshrepo: change raise_ to abort
r7642 self.abort(error.ResponseError(_("unexpected response:"), r))
Vadim Gelfer
add support for streaming clone....
r2612
def stream_out(self):
return self.do_cmd('stream_out')
Vadim Gelfer
clean up hg.py: move repo constructor code into each repo module
r2740
instance = sshrepository