##// END OF EJS Templates
merge: cache unknown dir checks (issue5716)...
merge: cache unknown dir checks (issue5716) As mentioned in D1222, the recent pathconflicts change regresses update performance in large repositories when many files are being updated. To mitigate this, we introduce two caches of directories that have already found to be either: - unknown directories, but which are not aliased by files and so don't need to be checked if they are files again; and - missing directores, which cannot cause path conflicts, and cannot contain a file that causes a path conflict. When checking the paths of a file, testing against this caches means we can skip tests that involve touching the filesystem. Differential Revision: https://phab.mercurial-scm.org/D1224

File last commit:

r34972:38637dd3 default
r35181:b8596235 stable
Show More
automv.py
108 lines | 3.5 KiB | text/x-python | PythonLexer
Martijn Pieters
automv: new experimental extension...
r28129 # automv.py
#
# Copyright 2013-2016 Facebook, Inc.
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
Jun Wu
automv: use lowercase for docstring title...
r31599 """check for unrecorded moves at commit time (EXPERIMENTAL)
Martijn Pieters
automv: new experimental extension...
r28129
This extension checks at commit/amend time if any of the committed files
comes from an unrecorded mv.
The threshold at which a file is considered a move can be set with the
Martijn Pieters
automv: switch to specifying the similarity as an integer (0-100)...
r28152 ``automv.similarity`` config option. This option takes a percentage between 0
Martijn Pieters
automv: use 95 as the default similarity threshold...
r28183 (disabled) and 100 (files must be identical), the default is 95.
Martijn Pieters
automv: new experimental extension...
r28129
"""
Martijn Pieters
automv: use 95 as the default similarity threshold...
r28183
# Using 95 as a default similarity is based on an analysis of the mercurial
# repositories of the cpython, mozilla-central & mercurial repositories, as
# well as 2 very large facebook repositories. At 95 50% of all potential
# missed moves would be caught, as well as correspond with 87% of all
# explicitly marked moves. Together, 80% of moved files are 95% similar or
# more.
#
# See http://markmail.org/thread/5pxnljesvufvom57 for context.
Martijn Pieters
automv: new experimental extension...
r28129 from __future__ import absolute_import
Yuya Nishihara
py3: move up symbol imports to enforce import-checker rules...
r29205 from mercurial.i18n import _
Martijn Pieters
automv: new experimental extension...
r28129 from mercurial import (
commands,
copies,
Martijn Pieters
automv: use 95 as the default similarity threshold...
r28183 error,
Martijn Pieters
automv: new experimental extension...
r28129 extensions,
configitems: register the 'automv.similarity' config...
r33185 registrar,
Martijn Pieters
automv: new experimental extension...
r28129 scmutil,
similar
)
configitems: register the 'automv.similarity' config...
r33185 configtable = {}
configitem = registrar.configitem(configtable)
configitem('automv', 'similarity',
default=95,
)
Martijn Pieters
automv: new experimental extension...
r28129 def extsetup(ui):
entry = extensions.wrapcommand(
commands.table, 'commit', mvcheck)
entry[1].append(
('', 'no-automv', None,
_('disable automatic file move detection')))
def mvcheck(orig, ui, repo, *pats, **opts):
Martijn Pieters
automv: improve function docstrings
r28149 """Hook to check for moves at commit time"""
Martijn Pieters
automv: do not release lock between marking files and the actual commit
r28151 renames = None
Martijn Pieters
automv: new experimental extension...
r28129 disabled = opts.pop('no_automv', False)
if not disabled:
configitems: register the 'automv.similarity' config...
r33185 threshold = ui.configint('automv', 'similarity')
Martijn Pieters
automv: use 95 as the default similarity threshold...
r28183 if not 0 <= threshold <= 100:
raise error.Abort(_('automv.similarity must be between 0 and 100'))
Martijn Pieters
automv: new experimental extension...
r28129 if threshold > 0:
match = scmutil.match(repo[None], pats, opts)
added, removed = _interestingfiles(repo, match)
Martijn Pieters
automv: switch to specifying the similarity as an integer (0-100)...
r28152 renames = _findrenames(repo, match, added, removed,
threshold / 100.0)
Martijn Pieters
automv: do not release lock between marking files and the actual commit
r28151
with repo.wlock():
if renames is not None:
Martijn Pieters
automv: reuse existing scutil._markchanges() function
r28150 scmutil._markchanges(repo, (), (), renames)
Martijn Pieters
automv: do not release lock between marking files and the actual commit
r28151 return orig(ui, repo, *pats, **opts)
Martijn Pieters
automv: new experimental extension...
r28129
def _interestingfiles(repo, matcher):
Martijn Pieters
automv: improve function docstrings
r28149 """Find what files were added or removed in this commit.
Returns a tuple of two lists: (added, removed). Only files not *already*
marked as moved are included in the added list.
"""
Martijn Pieters
automv: simplify retrieving the status...
r28146 stat = repo.status(match=matcher)
Martijn Pieters
automv: new experimental extension...
r28129 added = stat[1]
removed = stat[2]
copy = copies._forwardcopies(repo['.'], repo[None], matcher)
# remove the copy files for which we already have copy info
added = [f for f in added if f not in copy]
return added, removed
def _findrenames(repo, matcher, added, removed, similarity):
Martijn Pieters
automv: improve function docstrings
r28149 """Find what files in added are really moved files.
Any file named in removed that is at least similarity% similar to a file
in added is seen as a rename.
"""
Martijn Pieters
automv: new experimental extension...
r28129 renames = {}
if similarity > 0:
for src, dst, score in similar.findrenames(
repo, added, removed, similarity):
if repo.ui.verbose:
repo.ui.status(
_('detected move of %s as %s (%d%% similar)\n') % (
matcher.rel(src), matcher.rel(dst), score * 100))
renames[dst] = src
if renames:
repo.ui.status(_('detected move of %d files\n') % len(renames))
return renames