##// END OF EJS Templates
performance: speedup computation of obsolete revisions...
performance: speedup computation of obsolete revisions In their current state, revset calls can be very costly as we test predicates on the entire repository. As obsolete computation is used by the "hidden" filter, it needs to be very fast. This changet drops the revset call in favor of direct testing of the phase of a changeset. Performance test on my Mercurial checkout - 19857 total changesets, - 1584 obsolete changesets, - 13310 obsolescence markers. Before: ! obsolete ! wall 0.047041 After: ! obsolete ! wall 0.004590 Performance test on a Mozilla central checkout: - 117293 total changesets, - 1 obsolete changeset, - 1 obsolescence marker. Before: ! obsolete ! wall 0.001539 After: ! obsolete ! wall 0.000017

File last commit:

r18121:f8a13f06 default
r18271:67872e93 default
Show More
repair.py
203 lines | 7.0 KiB | text/x-python | PythonLexer
Matt Mackall
strip: move strip code to a new repair module
r4702 # repair.py - functions for repository repair for mercurial
#
# Copyright 2005, 2006 Chris Mason <mason@suse.com>
# Copyright 2007 Matt Mackall
#
Martin Geisler
updated license to be explicit about GPL version 2
r8225 # This software may be used and distributed according to the terms of the
Matt Mackall
Update license to GPLv2+
r10263 # GNU General Public License version 2 or any later version.
Matt Mackall
strip: move strip code to a new repair module
r4702
Pierre-Yves David
branchmap: extract updatebranchcache from repo
r18121 from mercurial import changegroup, branchmap
Alexander Solovyov
remove unused imports and variables
r14064 from mercurial.node import short
from mercurial.i18n import _
Simon Heimberg
separate import lines from mercurial and general python modules
r8312 import os
Alain Leufroy
repair: fix missing import...
r16440 import errno
Matt Mackall
strip: move strip code to a new repair module
r4702
Matt Mackall
strip: backout 73307643a09f (issue3077)
r15386 def _bundle(repo, bases, heads, node, suffix, compress=True):
Alexis S. L. Carvalho
repair.py: don't use nested functions.
r5905 """create a bundle with the specified revisions as a backup"""
Matt Mackall
strip: backout 73307643a09f (issue3077)
r15386 cg = repo.changegroupsubset(bases, heads, 'strip')
Alexis S. L. Carvalho
repair.py: don't use nested functions.
r5905 backupdir = repo.join("strip-backup")
if not os.path.isdir(backupdir):
os.mkdir(backupdir)
Matt Mackall
strip: backup bundles should use the .hg extension
r11333 name = os.path.join(backupdir, "%s-%s.hg" % (short(node), suffix))
Nicolas Dumazet
repair: do not compress partial bundle if we do not keep it on disk...
r11791 if compress:
bundletype = "HG10BZ"
else:
bundletype = "HG10UN"
return changegroup.writebundle(cg, name, bundletype)
Matt Mackall
strip: move strip code to a new repair module
r4702
Alexis S. L. Carvalho
simplify revlog.strip interface and callers; add docstring...
r5910 def _collectfiles(repo, striprev):
"""find out the filelogs affected by the strip"""
Benoit Boissinot
repair: use set instead of dict
r8462 files = set()
Matt Mackall
strip: move strip code to a new repair module
r4702
Matt Mackall
add __len__ and __iter__ methods to repo and revlog
r6750 for x in xrange(striprev, len(repo)):
Martin Geisler
repair: bulk update sets...
r8479 files.update(repo[x].files())
Alexis S. L. Carvalho
repair.py: split stripall into two functions; clean it up a bit
r5902
Benoit Boissinot
repair: use set instead of dict
r8462 return sorted(files)
Alexis S. L. Carvalho
repair.py: split stripall into two functions; clean it up a bit
r5902
Benoit Boissinot
strip: remove usage of extranodes...
r13702 def _collectbrokencsets(repo, files, striprev):
"""return the changesets which will be broken by the truncation"""
Matt Mackall
strip: simplify collectone
r13705 s = set()
Benoit Boissinot
strip: remove usage of extranodes...
r13702 def collectone(revlog):
redstone
localrepo: cleanup var names and comments...
r16628 linkgen = (revlog.linkrev(i) for i in revlog)
Alexis S. L. Carvalho
strip: calculate list of extra nodes to save and pass it to changegroupsubset...
r5909 # find the truncation point of the revlog
redstone
localrepo: cleanup var names and comments...
r16628 for lrev in linkgen:
Benoit Boissinot
strip: remove usage of extranodes...
r13702 if lrev >= striprev:
Alexis S. L. Carvalho
strip: calculate list of extra nodes to save and pass it to changegroupsubset...
r5909 break
Matt Mackall
strip: simplify collectone
r13705 # see if any revision after this point has a linkrev
# less than striprev (those will be broken by strip)
redstone
localrepo: cleanup var names and comments...
r16628 for lrev in linkgen:
Matt Mackall
strip: simplify collectone
r13705 if lrev < striprev:
Patrick Mezard
code indentation fixes
r13747 s.add(lrev)
Alexis S. L. Carvalho
strip: calculate list of extra nodes to save and pass it to changegroupsubset...
r5909
Matt Mackall
strip: simplify collectone
r13705 collectone(repo.manifest)
for fname in files:
collectone(repo.file(fname))
Alexis S. L. Carvalho
strip: calculate list of extra nodes to save and pass it to changegroupsubset...
r5909
Matt Mackall
strip: simplify collectone
r13705 return s
Alexis S. L. Carvalho
strip: calculate list of extra nodes to save and pass it to changegroupsubset...
r5909
Idan Kamara
repair: allow giving strip backup a different name...
r16388 def strip(ui, repo, nodelist, backup="all", topic='backup'):
Pierre-Yves David
clfilter: strip logic should be unfiltered...
r18004 repo = repo.unfiltered()
Joshua Redstone
strip: incrementally update the branchheads cache after a strip...
r17013 # It simplifies the logic around updating the branchheads cache if we only
# have to consider the effect of the stripped revisions and not revisions
# missing because the cache is out-of-date.
Pierre-Yves David
branchmap: extract updatebranchcache from repo
r18121 branchmap.updatecache(repo)
Joshua Redstone
strip: incrementally update the branchheads cache after a strip...
r17013
Alexis S. L. Carvalho
repair.py: rename chlog to cl
r5901 cl = repo.changelog
Idan Kamara
repair: remove undo files after strip
r16237 # TODO handle undo of merge sets
Wagner Bruna
strip: enhance repair.strip to receive a list of nodes (issue3299)...
r16252 if isinstance(nodelist, str):
nodelist = [nodelist]
striplist = [cl.rev(node) for node in nodelist]
striprev = min(striplist)
Matt Mackall
strip: move strip code to a new repair module
r4702
Joshua Redstone
strip: incrementally update the branchheads cache after a strip...
r17013 # Generate set of branches who will have nodes stripped.
striprevs = repo.revs("%ld::", striplist)
stripbranches = set([repo[rev].branch() for rev in striprevs])
# Set of potential new heads resulting from the strip. The parents of any
# node removed could be a new head because the node to be removed could have
# been the only child of the parent.
newheadrevs = repo.revs("parents(%ld::) - %ld::", striprevs, striprevs)
newheadnodes = set([cl.node(rev) for rev in newheadrevs])
newheadbranches = set([repo[rev].branch() for rev in newheadrevs])
Nicolas Dumazet
repair: do not compress partial bundle if we do not keep it on disk...
r11791 keeppartialbundle = backup == 'strip'
Alexis S. L. Carvalho
repair.py: rewrite a loop, making it cleaner and faster
r6147 # Some revisions with rev > striprev may not be descendants of striprev.
# We have to find these revisions and put them in a bundle, so that
# we can restore them after the truncations.
# To create the bundle we use repo.changegroupsubset which requires
# the list of heads and bases of the set of interesting revisions.
# (head = revision in the set that has no descendant in the set;
# base = revision in the set that has no ancestor in the set)
Wagner Bruna
strip: enhance repair.strip to receive a list of nodes (issue3299)...
r16252 tostrip = set(striplist)
for rev in striplist:
Bryan O'Sullivan
revlog: descendants(*revs) becomes descendants(revs) (API)...
r16867 for desc in cl.descendants([rev]):
Wagner Bruna
strip: enhance repair.strip to receive a list of nodes (issue3299)...
r16252 tostrip.add(desc)
Benoit Boissinot
strip: remove usage of extranodes...
r13702
files = _collectfiles(repo, striprev)
Matt Mackall
strip: simplify collectone
r13705 saverevs = _collectbrokencsets(repo, files, striprev)
Benoit Boissinot
strip: remove usage of extranodes...
r13702
# compute heads
saveheads = set(saverevs)
Matt Mackall
add __len__ and __iter__ methods to repo and revlog
r6750 for r in xrange(striprev + 1, len(cl)):
Benoit Boissinot
strip: remove usage of extranodes...
r13702 if r not in tostrip:
saverevs.add(r)
saveheads.difference_update(cl.parentrevs(r))
saveheads.add(r)
saveheads = [cl.node(r) for r in saveheads]
Matt Mackall
strip: move strip code to a new repair module
r4702
Matt Mackall
strip: backout 73307643a09f (issue3077)
r15386 # compute base nodes
if saverevs:
Bryan O'Sullivan
revlog: descendants(*revs) becomes descendants(revs) (API)...
r16867 descendants = set(cl.descendants(saverevs))
Matt Mackall
strip: backout 73307643a09f (issue3077)
r15386 saverevs.difference_update(descendants)
savebases = [cl.node(r) for r in saverevs]
Wagner Bruna
strip: enhance repair.strip to receive a list of nodes (issue3299)...
r16252 stripbases = [cl.node(r) for r in tostrip]
Siddharth Agarwal
strip: make query to get new bookmark target cheaper...
r18040
# For a set s, max(parents(s) - s) is the same as max(heads(::s - s)), but
# is much faster
newbmtarget = repo.revs('max(parents(%ld) - (%ld))', tostrip, tostrip)
Augie Fackler
strip: move bookmarks to nearest ancestor rather than '.'...
r17264 if newbmtarget:
Matt Mackall
repair: use node to track post-strip bookmark target...
r17796 newbmtarget = repo[newbmtarget[0]].node()
Augie Fackler
strip: move bookmarks to nearest ancestor rather than '.'...
r17264 else:
newbmtarget = '.'
Matt Mackall
strip: move strip code to a new repair module
r4702
Matt Mackall
bookmarks: move strip support to repair
r13362 bm = repo._bookmarks
updatebm = []
for m in bm:
rev = repo[bm[m]].rev()
if rev in tostrip:
updatebm.append(m)
Matt Mackall
strip: move strip code to a new repair module
r4702 # create a changegroup for all the branches we need to keep
Matt Mackall
strip: be quiet about temporary internal bundle
r11197 backupfile = None
Matt Mackall
strip: move strip code to a new repair module
r4702 if backup == "all":
Idan Kamara
repair: allow giving strip backup a different name...
r16388 backupfile = _bundle(repo, stripbases, cl.heads(), node, topic)
Matt Mackall
strip: improve full backup message
r11200 repo.ui.status(_("saved backup bundle to %s\n") % backupfile)
Matt Mackall
strip: backout 73307643a09f (issue3077)
r15386 if saveheads or savebases:
Nicolas Dumazet
repair: do not compress partial bundle if we do not keep it on disk...
r11791 # do not compress partial bundle if we remove it from disk later
Matt Mackall
strip: backout 73307643a09f (issue3077)
r15386 chgrpfile = _bundle(repo, savebases, saveheads, node, 'temp',
compress=keeppartialbundle)
Matt Mackall
strip: move strip code to a new repair module
r4702
Henrik Stuart
strip: make repair.strip transactional to avoid repository corruption...
r8073 mfst = repo.manifest
Steve Borho
localrepo: add desc parameter to transaction...
r10881 tr = repo.transaction("strip")
Henrik Stuart
strip: make repair.strip transactional to avoid repository corruption...
r8073 offset = len(tr.entries)
Matt Mackall
strip: be quiet about temporary internal bundle
r11197 try:
tr.startgroup()
cl.strip(striprev, tr)
mfst.strip(striprev, tr)
for fn in files:
repo.file(fn).strip(striprev, tr)
tr.endgroup()
Henrik Stuart
strip: make repair.strip transactional to avoid repository corruption...
r8073
Matt Mackall
strip: be quiet about temporary internal bundle
r11197 try:
for i in xrange(offset, len(tr.entries)):
file, troffset, ignore = tr.entries[i]
repo.sopener(file, 'a').truncate(troffset)
tr.close()
Brodie Rao
check-code: ignore naked excepts with a "re-raise" comment...
r16705 except: # re-raises
Matt Mackall
strip: be quiet about temporary internal bundle
r11197 tr.abort()
raise
Matt Mackall
strip: backout 73307643a09f (issue3077)
r15386 if saveheads or savebases:
Matt Mackall
strip: hide unbundle messages by default...
r11202 ui.note(_("adding branch\n"))
Matt Mackall
strip: be quiet about temporary internal bundle
r11197 f = open(chgrpfile, "rb")
gen = changegroup.readbundle(f, chgrpfile)
Matt Mackall
strip: hide unbundle messages by default...
r11202 if not repo.ui.verbose:
# silence internal shuffling chatter
repo.ui.pushbuffer()
Matt Mackall
strip: be quiet about temporary internal bundle
r11197 repo.addchangegroup(gen, 'strip', 'bundle:' + chgrpfile, True)
Matt Mackall
strip: hide unbundle messages by default...
r11202 if not repo.ui.verbose:
repo.ui.popbuffer()
Matt Mackall
strip: be quiet about temporary internal bundle
r11197 f.close()
Nicolas Dumazet
repair: do not compress partial bundle if we do not keep it on disk...
r11791 if not keeppartialbundle:
Matt Mackall
strip: be quiet about temporary internal bundle
r11197 os.unlink(chgrpfile)
Matt Mackall
bookmarks: move strip support to repair
r13362
Idan Kamara
repair: remove undo files after strip
r16237 # remove undo files
for undofile in repo.undofiles():
try:
os.unlink(undofile)
except OSError, e:
if e.errno != errno.ENOENT:
ui.warn(_('error removing %s: %s\n') % (undofile, str(e)))
Matt Mackall
bookmarks: move strip support to repair
r13362 for m in updatebm:
Augie Fackler
strip: move bookmarks to nearest ancestor rather than '.'...
r17264 bm[m] = repo[newbmtarget].node()
Augie Fackler
bookmarks: introduce a bmstore to manage bookmark persistence...
r17922 bm.write()
Brodie Rao
check-code: ignore naked excepts with a "re-raise" comment...
r16705 except: # re-raises
Matt Mackall
strip: be quiet about temporary internal bundle
r11197 if backupfile:
Martin Geisler
mark ui.warn strings for translation
r11600 ui.warn(_("strip failed, full bundle stored in '%s'\n")
% backupfile)
Matt Mackall
strip: be quiet about temporary internal bundle
r11197 elif saveheads:
Martin Geisler
mark ui.warn strings for translation
r11600 ui.warn(_("strip failed, partial bundle stored in '%s'\n")
Matt Mackall
strip: be quiet about temporary internal bundle
r11197 % chgrpfile)
Henrik Stuart
strip: make repair.strip transactional to avoid repository corruption...
r8073 raise
Matt Mackall
strip: move strip code to a new repair module
r4702
Joshua Redstone
strip: incrementally update the branchheads cache after a strip...
r17013 if len(stripbranches) == 1 and len(newheadbranches) == 1 \
and stripbranches == newheadbranches:
repo.destroyed(newheadnodes)
else:
# Multiple branches involved in strip. Will allow branchcache to become
# invalid and later on rebuilt from scratch
repo.destroyed()