##// END OF EJS Templates
merge: mark file gets as not thread safe (issue5933)...
merge: mark file gets as not thread safe (issue5933) In default installs, this has the effect of disabling the thread-based worker on Windows when manifesting files in the working directory. My measurements have shown that with revlog-based repositories, Mercurial spends a lot of CPU time in revlog code resolving file data. This ends up incurring a lot of context switching across threads and slows down `hg update` operations when going from an empty working directory to the tip of the repo. On mozilla-unified (246,351 files) on an i7-6700K (4+4 CPUs): before: 487s wall after: 360s wall (equivalent to worker.enabled=false) cpus=2: 379s wall Even with only 2 threads, the thread pool is still slower. The introduction of the thread-based worker (02b36e860e0b) states that it resulted in a "~50%" speedup for `hg sparse --enable-profile` and `hg sparse --disable-profile`. This disagrees with my measurement above. I theorize a few reasons for this: 1) Removal of files from the working directory is I/O - not CPU - bound and should benefit from a thread pool (unless I/O is insanely fast and the GIL release is near instantaneous). So tests like `hg sparse --enable-profile` may exercise deletion throughput and aren't good benchmarks for worker tasks that are CPU heavy. 2) The patch was authored by someone at Facebook. The results were likely measured against a repository using remotefilelog. And I believe that revision retrieval during working directory updates with remotefilelog will often use a remote store, thus being I/O and not CPU bound. This probably resulted in an overstated performance gain. Since there appears to be a need to enable the thread-based worker with some stores, I've made the flagging of file gets as thread safe configurable. I've made it experimental because I don't want to formalize a boolean flag for this option and because this attribute is best captured against the store implementation. But we don't have a proper store API for this yet. I'd rather cross this bridge later. It is possible there are revlog-based repositories that do benefit from a thread-based worker. I didn't do very comprehensive testing. If there are, we may want to devise a more proper algorithm for whether to use the thread-based worker, including possibly config options to limit the number of threads to use. But until I see evidence that justifies complexity, simplicity wins. Differential Revision: https://phab.mercurial-scm.org/D3963

File last commit:

r38196:aea29e81 default
r38755:be498426 default
Show More
__init__.py
107 lines | 3.0 KiB | text/x-python | PythonLexer
# hgweb/__init__.py - web interface to a mercurial repository
#
# Copyright 21 May 2005 - (c) 2005 Jake Edge <jake@edge2.net>
# Copyright 2005 Matt Mackall <mpm@selenic.com>
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
from __future__ import absolute_import
import os
from ..i18n import _
from .. import (
error,
pycompat,
)
from ..utils import (
procutil,
)
from . import (
hgweb_mod,
hgwebdir_mod,
server,
)
def hgweb(config, name=None, baseui=None):
'''create an hgweb wsgi object
config can be one of:
- repo object (single repo view)
- path to repo (single repo view)
- path to config file (multi-repo view)
- dict of virtual:real pairs (multi-repo view)
- list of virtual:real tuples (multi-repo view)
'''
if ((isinstance(config, bytes) and not os.path.isdir(config)) or
isinstance(config, dict) or isinstance(config, list)):
# create a multi-dir interface
return hgwebdir_mod.hgwebdir(config, baseui=baseui)
return hgweb_mod.hgweb(config, name=name, baseui=baseui)
def hgwebdir(config, baseui=None):
return hgwebdir_mod.hgwebdir(config, baseui=baseui)
class httpservice(object):
def __init__(self, ui, app, opts):
self.ui = ui
self.app = app
self.opts = opts
def init(self):
procutil.setsignalhandler()
self.httpd = server.create_server(self.ui, self.app)
if (self.opts['port'] and
not self.ui.verbose and
not self.opts['print_url']):
return
if self.httpd.prefix:
prefix = self.httpd.prefix.strip('/') + '/'
else:
prefix = ''
port = r':%d' % self.httpd.port
if port == r':80':
port = r''
bindaddr = self.httpd.addr
if bindaddr == r'0.0.0.0':
bindaddr = r'*'
elif r':' in bindaddr: # IPv6
bindaddr = r'[%s]' % bindaddr
fqaddr = self.httpd.fqaddr
if r':' in fqaddr:
fqaddr = r'[%s]' % fqaddr
url = 'http://%s%s/%s' % (
pycompat.sysbytes(fqaddr), pycompat.sysbytes(port), prefix)
if self.opts['print_url']:
self.ui.write('%s\n' % url)
else:
if self.opts['port']:
write = self.ui.status
else:
write = self.ui.write
write(_('listening at %s (bound to %s:%d)\n') %
(url, pycompat.sysbytes(bindaddr), self.httpd.port))
self.ui.flush() # avoid buffering of status message
def run(self):
self.httpd.serve_forever()
def createapp(baseui, repo, webconf):
if webconf:
return hgwebdir_mod.hgwebdir(webconf, baseui=baseui)
else:
if not repo:
raise error.RepoError(_("there is no Mercurial repository"
" here (.hg not found)"))
return hgweb_mod.hgweb(repo, baseui=baseui)