##// END OF EJS Templates
merge: mark file gets as not thread safe (issue5933)...
merge: mark file gets as not thread safe (issue5933) In default installs, this has the effect of disabling the thread-based worker on Windows when manifesting files in the working directory. My measurements have shown that with revlog-based repositories, Mercurial spends a lot of CPU time in revlog code resolving file data. This ends up incurring a lot of context switching across threads and slows down `hg update` operations when going from an empty working directory to the tip of the repo. On mozilla-unified (246,351 files) on an i7-6700K (4+4 CPUs): before: 487s wall after: 360s wall (equivalent to worker.enabled=false) cpus=2: 379s wall Even with only 2 threads, the thread pool is still slower. The introduction of the thread-based worker (02b36e860e0b) states that it resulted in a "~50%" speedup for `hg sparse --enable-profile` and `hg sparse --disable-profile`. This disagrees with my measurement above. I theorize a few reasons for this: 1) Removal of files from the working directory is I/O - not CPU - bound and should benefit from a thread pool (unless I/O is insanely fast and the GIL release is near instantaneous). So tests like `hg sparse --enable-profile` may exercise deletion throughput and aren't good benchmarks for worker tasks that are CPU heavy. 2) The patch was authored by someone at Facebook. The results were likely measured against a repository using remotefilelog. And I believe that revision retrieval during working directory updates with remotefilelog will often use a remote store, thus being I/O and not CPU bound. This probably resulted in an overstated performance gain. Since there appears to be a need to enable the thread-based worker with some stores, I've made the flagging of file gets as thread safe configurable. I've made it experimental because I don't want to formalize a boolean flag for this option and because this attribute is best captured against the store implementation. But we don't have a proper store API for this yet. I'd rather cross this bridge later. It is possible there are revlog-based repositories that do benefit from a thread-based worker. I didn't do very comprehensive testing. If there are, we may want to devise a more proper algorithm for whether to use the thread-based worker, including possibly config options to limit the number of threads to use. But until I see evidence that justifies complexity, simplicity wins. Differential Revision: https://phab.mercurial-scm.org/D3963

File last commit:

r37138:a8a902d7 default
r38755:be498426 default
Show More
simplemerge
83 lines | 2.5 KiB | text/plain | TextLexer
#!/usr/bin/env python
from __future__ import absolute_import
import getopt
import sys
import hgdemandimport
hgdemandimport.enable()
from mercurial.i18n import _
from mercurial import (
context,
error,
fancyopts,
simplemerge,
ui as uimod,
)
from mercurial.utils import (
procutil,
)
options = [('L', 'label', [], _('labels to use on conflict markers')),
('a', 'text', None, _('treat all files as text')),
('p', 'print', None,
_('print results instead of overwriting LOCAL')),
('', 'no-minimal', None, _('no effect (DEPRECATED)')),
('h', 'help', None, _('display help and exit')),
('q', 'quiet', None, _('suppress output'))]
usage = _('''simplemerge [OPTS] LOCAL BASE OTHER
Simple three-way file merge utility with a minimal feature set.
Apply to LOCAL the changes necessary to go from BASE to OTHER.
By default, LOCAL is overwritten with the results of this operation.
''')
class ParseError(Exception):
"""Exception raised on errors in parsing the command line."""
def showhelp():
sys.stdout.write(usage)
sys.stdout.write('\noptions:\n')
out_opts = []
for shortopt, longopt, default, desc in options:
out_opts.append(('%2s%s' % (shortopt and '-%s' % shortopt,
longopt and ' --%s' % longopt),
'%s' % desc))
opts_len = max([len(opt[0]) for opt in out_opts])
for first, second in out_opts:
sys.stdout.write(' %-*s %s\n' % (opts_len, first, second))
try:
for fp in (sys.stdin, sys.stdout, sys.stderr):
procutil.setbinary(fp)
opts = {}
try:
args = fancyopts.fancyopts(sys.argv[1:], options, opts)
except getopt.GetoptError as e:
raise ParseError(e)
if opts['help']:
showhelp()
sys.exit(0)
if len(args) != 3:
raise ParseError(_('wrong number of arguments'))
local, base, other = args
sys.exit(simplemerge.simplemerge(uimod.ui.load(),
context.arbitraryfilectx(local),
context.arbitraryfilectx(base),
context.arbitraryfilectx(other),
**opts))
except ParseError as e:
sys.stdout.write("%s: %s\n" % (sys.argv[0], e))
showhelp()
sys.exit(1)
except error.Abort as e:
sys.stderr.write("abort: %s\n" % e)
sys.exit(255)
except KeyboardInterrupt:
sys.exit(255)