##// END OF EJS Templates
parsers: inline fields of dirstate values in C version...
parsers: inline fields of dirstate values in C version Previously, while unpacking the dirstate we'd create 3-4 new CPython objects for most dirstate values: - the state is a single character string, which is pooled by CPython - the mode is a new object if it isn't 0 due to being in the lookup set - the size is a new object if it is greater than 255 - the mtime is a new object if it isn't -1 due to being in the lookup set - the tuple to contain them all In some cases such as regular hg status, we actually look at all the objects. In other cases like hg add, hg status for a subdirectory, or hg status with the third-party hgwatchman enabled, we look at almost none of the objects. This patch eliminates most object creation in these cases by defining a custom C struct that is exposed to Python with an interface similar to a tuple. Only when tuple elements are actually requested are the respective objects created. The gains, where they're expected, are significant. The following tests are run against a working copy with over 270,000 files. parse_dirstate becomes significantly faster: $ hg perfdirstate before: wall 0.186437 comb 0.180000 user 0.160000 sys 0.020000 (best of 35) after: wall 0.093158 comb 0.100000 user 0.090000 sys 0.010000 (best of 95) and as a result, several commands benefit: $ time hg status # with hgwatchman enabled before: 0.42s user 0.14s system 99% cpu 0.563 total after: 0.34s user 0.12s system 99% cpu 0.471 total $ time hg add new-file before: 0.85s user 0.18s system 99% cpu 1.033 total after: 0.76s user 0.17s system 99% cpu 0.931 total There is a slight regression in regular status performance, but this is fixed in an upcoming patch.

File last commit:

r21797:b009dd13 default
r21809:e250b830 default
Show More
hook.py
211 lines | 7.7 KiB | text/x-python | PythonLexer
Matt Mackall
hooks: separate hook code into a separate module
r4622 # hook.py - hook support for mercurial
#
# Copyright 2007 Matt Mackall <mpm@selenic.com>
#
Martin Geisler
updated license to be explicit about GPL version 2
r8225 # This software may be used and distributed according to the terms of the
Matt Mackall
Update license to GPLv2+
r10263 # GNU General Public License version 2 or any later version.
Matt Mackall
hooks: separate hook code into a separate module
r4622
from i18n import _
Mads Kiilerich
hooks: for python hooks, consistently use __name__ etc as name, not the repr...
r20548 import os, sys, time
Dirkjan Ochtman
hook: disable demandimport before importing hooks...
r18111 import extensions, util, demandimport
Matt Mackall
hooks: separate hook code into a separate module
r4622
def _pythonhook(ui, repo, name, hname, funcname, args, throw):
'''call python hook. hook is callable object, looked up as
name in python module. if callable returns "true", hook
fails, else passes. if hook raises exception, treated as
hook failure. exception propagates if throw is "true".
reason for "true" meaning "hook failed" is so that
unmodified commands (e.g. mercurial.commands.update) can
be run as hooks without wrappers to convert return values.'''
Augie Fackler
hook: restore use of callable() since it was readded in Python 3.2
r21797 if callable(funcname):
Mads Kiilerich
hooks: for python hooks, consistently use __name__ etc as name, not the repr...
r20548 obj = funcname
funcname = obj.__module__ + "." + obj.__name__
else:
Matt Mackall
hooks: separate hook code into a separate module
r4622 d = funcname.rfind('.')
if d == -1:
raise util.Abort(_('%s hook is invalid ("%s" not in '
'a module)') % (hname, funcname))
modname = funcname[:d]
Sune Foldager
hook: fix bug (reuse of variable) introduced in 872d49dd577a...
r10103 oldpaths = sys.path
Augie Fackler
windows: check util.mainfrozen() instead of ad-hoc checks everywhere
r14941 if util.mainfrozen():
Steve Borho
hook: fix full path imports on Windows (issue1779)...
r9332 # binary installs require sys.path manipulation
Sune Foldager
hook: fix bug (reuse of variable) introduced in 872d49dd577a...
r10103 modpath, modfile = os.path.split(modname)
if modpath and modfile:
sys.path = sys.path[:] + [modpath]
modname = modfile
Brodie Rao
hooks: only disable/re-enable demandimport when it's already enabled...
r20422 demandimportenabled = demandimport.isenabled()
if demandimportenabled:
Dirkjan Ochtman
hook: disable demandimport before importing hooks...
r18111 demandimport.disable()
Brodie Rao
hooks: only disable/re-enable demandimport when it's already enabled...
r20422 try:
Matt Mackall
hooks: separate hook code into a separate module
r4622 try:
Brodie Rao
hooks: only disable/re-enable demandimport when it's already enabled...
r20422 obj = __import__(modname)
Matt Mackall
hooks: separate hook code into a separate module
r4622 except ImportError:
Brodie Rao
hooks: only disable/re-enable demandimport when it's already enabled...
r20422 e1 = sys.exc_type, sys.exc_value, sys.exc_traceback
try:
# extensions are loaded with hgext_ prefix
obj = __import__("hgext_%s" % modname)
except ImportError:
e2 = sys.exc_type, sys.exc_value, sys.exc_traceback
if ui.tracebackflag:
ui.warn(_('exception from first failed import '
'attempt:\n'))
ui.traceback(e1)
if ui.tracebackflag:
ui.warn(_('exception from second failed import '
'attempt:\n'))
ui.traceback(e2)
raise util.Abort(_('%s hook is invalid '
'(import of "%s" failed)') %
(hname, modname))
finally:
if demandimportenabled:
Dirkjan Ochtman
hook: disable demandimport before importing hooks...
r18111 demandimport.enable()
Steve Borho
hook: fix full path imports on Windows (issue1779)...
r9332 sys.path = oldpaths
Matt Mackall
hooks: separate hook code into a separate module
r4622 try:
for p in funcname.split('.')[1:]:
obj = getattr(obj, p)
Benoit Boissinot
remove unused variables
r7280 except AttributeError:
Matt Mackall
hooks: separate hook code into a separate module
r4622 raise util.Abort(_('%s hook is invalid '
'("%s" is not defined)') %
(hname, funcname))
Augie Fackler
hook: restore use of callable() since it was readded in Python 3.2
r21797 if not callable(obj):
Matt Mackall
hooks: separate hook code into a separate module
r4622 raise util.Abort(_('%s hook is invalid '
'("%s" is not callable)') %
(hname, funcname))
Mads Kiilerich
hooks: move logging of hook name to after we have found the hook...
r20547
ui.note(_("calling hook %s: %s\n") % (hname, funcname))
starttime = time.time()
Matt Mackall
hooks: separate hook code into a separate module
r4622 try:
Lee Cantey
hooks: use python 2.4 compatible exception handling
r14916 try:
Mads Kiilerich
help: fix some instances of 'the the'
r17251 # redirect IO descriptors to the ui descriptors so hooks
Lee Cantey
hooks: use python 2.4 compatible exception handling
r14916 # that write directly to these don't mess up the command
# protocol when running through the command server
old = sys.stdout, sys.stderr, sys.stdin
sys.stdout, sys.stderr, sys.stdin = ui.fout, ui.ferr, ui.fin
Idan Kamara
hooks: redirect stdout/err/in to the ui descriptors when calling python hooks...
r14889
Lee Cantey
hooks: use python 2.4 compatible exception handling
r14916 r = obj(ui=ui, repo=repo, hooktype=name, **args)
except KeyboardInterrupt:
Matt Mackall
hooks: separate hook code into a separate module
r4622 raise
Lee Cantey
hooks: use python 2.4 compatible exception handling
r14916 except Exception, exc:
if isinstance(exc, util.Abort):
ui.warn(_('error: %s hook failed: %s\n') %
(hname, exc.args[0]))
else:
ui.warn(_('error: %s hook raised an exception: '
'%s\n') % (hname, exc))
if throw:
raise
ui.traceback()
return True
Idan Kamara
hooks: redirect stdout/err/in to the ui descriptors when calling python hooks...
r14889 finally:
sys.stdout, sys.stderr, sys.stdin = old
Durham Goode
blackbox: logs python and extension hooks via ui.log()...
r18671 duration = time.time() - starttime
Durham Goode
blackbox: do not translate the log messages...
r18691 ui.log('pythonhook', 'pythonhook-%s: %s finished in %0.2f seconds\n',
Mads Kiilerich
hooks: for python hooks, consistently use __name__ etc as name, not the repr...
r20548 name, funcname, duration)
Matt Mackall
hooks: separate hook code into a separate module
r4622 if r:
if throw:
raise util.Abort(_('%s hook failed') % hname)
ui.warn(_('warning: %s hook failed\n') % hname)
return r
def _exthook(ui, repo, name, cmd, args, throw):
ui.note(_("running hook %s: %s\n") % (name, cmd))
Matt Mackall
Introduce HG_PREPEND to solve pretxn races...
r7787
Durham Goode
blackbox: logs python and extension hooks via ui.log()...
r18671 starttime = time.time()
Matt Mackall
Introduce HG_PREPEND to solve pretxn races...
r7787 env = {}
for k, v in args.iteritems():
Augie Fackler
hook: restore use of callable() since it was readded in Python 3.2
r21797 if callable(v):
Matt Mackall
Introduce HG_PREPEND to solve pretxn races...
r7787 v = v()
Dan Villiom Podlaski Christiansen
hooks: sort any dictionaries set in the environment...
r13207 if isinstance(v, dict):
# make the dictionary element order stable across Python
# implementations
v = ('{' +
', '.join('%r: %r' % i for i in sorted(v.iteritems())) +
'}')
Matt Mackall
Introduce HG_PREPEND to solve pretxn races...
r7787 env['HG_' + k.upper()] = v
Matt Mackall
hooks: fix pre- and post- hooks specified in .hg/hgrc...
r5869 if repo:
cwd = repo.root
else:
cwd = os.getcwd()
Maxim Khitrov
http: deliver hook output to client
r11469 if 'HG_URL' in env and env['HG_URL'].startswith('remote:http'):
r = util.system(cmd, environ=env, cwd=cwd, out=ui)
else:
Idan Kamara
hook: write hook output to ui fout descriptor...
r14711 r = util.system(cmd, environ=env, cwd=cwd, out=ui.fout)
Durham Goode
blackbox: logs python and extension hooks via ui.log()...
r18671
duration = time.time() - starttime
Durham Goode
blackbox: do not translate the log messages...
r18691 ui.log('exthook', 'exthook-%s: %s finished in %0.2f seconds\n',
Durham Goode
blackbox: logs python and extension hooks via ui.log()...
r18671 name, cmd, duration)
Matt Mackall
hooks: separate hook code into a separate module
r4622 if r:
Adrian Buehlmann
rename explain_exit to explainexit
r14234 desc, r = util.explainexit(r)
Matt Mackall
hooks: separate hook code into a separate module
r4622 if throw:
raise util.Abort(_('%s hook %s') % (name, desc))
ui.warn(_('warning: %s hook %s\n') % (name, desc))
return r
Matt Zuba
hooks: prioritize run order of hooks...
r15896 def _allhooks(ui):
hooks = []
for name, cmd in ui.configitems('hooks'):
if not name.startswith('priority'):
priority = ui.configint('hooks', 'priority.%s' % name, 0)
hooks.append((-priority, len(hooks), name, cmd))
return [(k, v) for p, o, k, v in sorted(hooks)]
Matt Mackall
hook: redirect stdout to stderr for ssh and http servers
r5833 _redirect = False
def redirect(state):
Alexis S. L. Carvalho
hook.py: fix redirections introduced by 323b9c55b328...
r6266 global _redirect
Matt Mackall
hook: redirect stdout to stderr for ssh and http servers
r5833 _redirect = state
Matt Mackall
hooks: separate hook code into a separate module
r4622 def hook(ui, repo, name, throw=False, **args):
Idan Kamara
ui: add a variable to control whether hooks should be called...
r17048 if not ui.callhooks:
return False
Matt Mackall
hooks: separate hook code into a separate module
r4622 r = False
Sune Foldager
hook: only redirect stdout if it and stderr are valid files...
r9658 oldstdout = -1
Matt Mackall
hook: redirect stdout to stderr for ssh and http servers
r5833
Jesse Long
hooks: restore io correctly on exception
r7416 try:
Matt Zuba
hooks: prioritize run order of hooks...
r15896 for hname, cmd in _allhooks(ui):
Jesse Long
hooks: restore io correctly on exception
r7416 if hname.split('.')[0] != name or not cmd:
continue
Matt Mackall
hooks: delay I/O redirection until we actually run a hook (issue3711)...
r17963
if oldstdout == -1 and _redirect:
try:
stdoutno = sys.__stdout__.fileno()
stderrno = sys.__stderr__.fileno()
# temporarily redirect stdout to stderr, if possible
if stdoutno >= 0 and stderrno >= 0:
sys.__stdout__.flush()
oldstdout = os.dup(stdoutno)
os.dup2(stderrno, stdoutno)
Matt Mackall
hooks: be even more forgiven of non-fd descriptors (issue3711)...
r17964 except (OSError, AttributeError):
# files seem to be bogus, give up on redirecting (WSGI, etc)
Matt Mackall
hooks: delay I/O redirection until we actually run a hook (issue3711)...
r17963 pass
Augie Fackler
hook: restore use of callable() since it was readded in Python 3.2
r21797 if callable(cmd):
Jesse Long
hooks: restore io correctly on exception
r7416 r = _pythonhook(ui, repo, name, hname, cmd, args, throw) or r
elif cmd.startswith('python:'):
Steve Borho
hook: fix full path imports on Windows (issue1779)...
r9332 if cmd.count(':') >= 2:
path, cmd = cmd[7:].rsplit(':', 1)
Alexander Solovyov
hook: assume relative path to hook is given from repo root
r13118 path = util.expandpath(path)
Matt Mackall
hook: fix import path handling for repo=None
r13119 if repo:
path = os.path.join(repo.root, path)
Simon Heimberg
hooks: print out more information when loading a python hook fails...
r17217 try:
mod = extensions.loadpath(path, 'hghook.%s' % hname)
except Exception:
ui.write(_("loading %s hook failed:\n") % hname)
raise
Alexander Solovyov
ability to load hooks from arbitrary python module
r7916 hookfn = getattr(mod, cmd)
else:
hookfn = cmd[7:].strip()
r = _pythonhook(ui, repo, name, hname, hookfn, args, throw) or r
Jesse Long
hooks: restore io correctly on exception
r7416 else:
r = _exthook(ui, repo, hname, cmd, args, throw) or r
finally:
Sune Foldager
hook: only redirect stdout if it and stderr are valid files...
r9658 if _redirect and oldstdout >= 0:
os.dup2(oldstdout, stdoutno)
Jesse Long
hooks: restore io correctly on exception
r7416 os.close(oldstdout)
Alexis S. L. Carvalho
hook.py: fix redirections introduced by 323b9c55b328...
r6266
return r