##// END OF EJS Templates
parsers: inline fields of dirstate values in C version...
parsers: inline fields of dirstate values in C version Previously, while unpacking the dirstate we'd create 3-4 new CPython objects for most dirstate values: - the state is a single character string, which is pooled by CPython - the mode is a new object if it isn't 0 due to being in the lookup set - the size is a new object if it is greater than 255 - the mtime is a new object if it isn't -1 due to being in the lookup set - the tuple to contain them all In some cases such as regular hg status, we actually look at all the objects. In other cases like hg add, hg status for a subdirectory, or hg status with the third-party hgwatchman enabled, we look at almost none of the objects. This patch eliminates most object creation in these cases by defining a custom C struct that is exposed to Python with an interface similar to a tuple. Only when tuple elements are actually requested are the respective objects created. The gains, where they're expected, are significant. The following tests are run against a working copy with over 270,000 files. parse_dirstate becomes significantly faster: $ hg perfdirstate before: wall 0.186437 comb 0.180000 user 0.160000 sys 0.020000 (best of 35) after: wall 0.093158 comb 0.100000 user 0.090000 sys 0.010000 (best of 95) and as a result, several commands benefit: $ time hg status # with hgwatchman enabled before: 0.42s user 0.14s system 99% cpu 0.563 total after: 0.34s user 0.12s system 99% cpu 0.471 total $ time hg add new-file before: 0.85s user 0.18s system 99% cpu 1.033 total after: 0.76s user 0.17s system 99% cpu 0.931 total There is a slight regression in regular status performance, but this is fixed in an upcoming patch.

File last commit:

r20380:c697b70f default
r21809:e250b830 default
Show More
lock.py
154 lines | 5.0 KiB | text/x-python | PythonLexer
Greg Ward
localrepo: document the locking scheme a little better...
r9309 # lock.py - simple advisory locking scheme for mercurial
mpm@selenic.com
Simply repository locking...
r161 #
Vadim Gelfer
update copyrights.
r2859 # Copyright 2005, 2006 Matt Mackall <mpm@selenic.com>
mpm@selenic.com
Simply repository locking...
r161 #
Martin Geisler
updated license to be explicit about GPL version 2
r8225 # This software may be used and distributed according to the terms of the
Matt Mackall
Update license to GPLv2+
r10263 # GNU General Public License version 2 or any later version.
mpm@selenic.com
Simply repository locking...
r161
Simon Heimberg
separate import lines from mercurial and general python modules
r8312 import util, error
import errno, os, socket, time
Ronny Pfannschmidt
add a deprecation warning for gc based lock releasing
r8113 import warnings
mpm@selenic.com
Simply repository locking...
r161
Eric Hopper
Convert all classes to new-style classes by deriving them from object.
r1559 class lock(object):
Greg Ward
localrepo: document the locking scheme a little better...
r9309 '''An advisory lock held by one process to control access to a set
of files. Non-cooperating processes or incorrectly written scripts
can ignore Mercurial's locking scheme and stomp all over the
repository, so don't do that.
Typically used via localrepository.lock() to lock the repository
store (.hg/store/) or localrepository.wlock() to lock everything
else under .hg/.'''
Vadim Gelfer
change lock format to let us detect and break stale locks....
r1877 # lock is symlink on platforms that support it, file on others.
# symlink is used because create of directory entry and contents
# are atomic even over nfs.
# old-style lock: symlink to pid
# new-style lock: symlink to hostname:pid
Bryan O'Sullivan
lock.py: cache hostname, but not pid, in case we fork
r4947 _host = None
FUJIWARA Katsunori
lock: take both vfs and lock file path relative to vfs to access via vfs...
r20091 def __init__(self, vfs, file, timeout=-1, releasefn=None, desc=None):
self.vfs = vfs
mpm@selenic.com
Simply repository locking...
r161 self.f = file
self.held = 0
Benoit Boissinot
add a timeout when a lock is held (default 1024 sec)...
r1787 self.timeout = timeout
Benoit Boissinot
add a releasefn keyword to lock.lock...
r1530 self.releasefn = releasefn
Vadim Gelfer
fix backtrace printed when cannot get lock....
r2016 self.desc = desc
Matt Mackall
lock: change name of release chain
r15589 self.postrelease = []
Bryan O'Sullivan
lock: if we fork, ensure that only the parent releases...
r18907 self.pid = os.getpid()
Mads Kiilerich
localrepo: give a sigh of relief when getting lock after waiting for it...
r20380 self.delay = self.lock()
mpm@selenic.com
Simply repository locking...
r161
def __del__(self):
Ronny Pfannschmidt
made repo locks recursive and deprecate refcounting based lock releasing...
r8108 if self.held:
Ronny Pfannschmidt
add a deprecation warning for gc based lock releasing
r8113 warnings.warn("use lock.release instead of del lock",
category=DeprecationWarning,
stacklevel=2)
Ronny Pfannschmidt
made repo locks recursive and deprecate refcounting based lock releasing...
r8108 # ensure the lock will be removed
# even if recursive locking did occur
self.held = 1
mpm@selenic.com
Simply repository locking...
r161 self.release()
def lock(self):
Benoit Boissinot
add a timeout when a lock is held (default 1024 sec)...
r1787 timeout = self.timeout
Martin Geisler
check-code: flag 0/1 used as constant Boolean expression
r14494 while True:
mpm@selenic.com
Simply repository locking...
r161 try:
self.trylock()
Mads Kiilerich
localrepo: give a sigh of relief when getting lock after waiting for it...
r20380 return self.timeout - timeout
Matt Mackall
error: move lock errors...
r7640 except error.LockHeld, inst:
Benoit Boissinot
add a timeout when a lock is held (default 1024 sec)...
r1787 if timeout != 0:
mpm@selenic.com
Simply repository locking...
r161 time.sleep(1)
Benoit Boissinot
add a timeout when a lock is held (default 1024 sec)...
r1787 if timeout > 0:
timeout -= 1
mpm@selenic.com
Simply repository locking...
r161 continue
Matt Mackall
error: move lock errors...
r7640 raise error.LockHeld(errno.ETIMEDOUT, inst.filename, self.desc,
inst.locker)
mpm@selenic.com
Whitespace cleanups...
r515
mpm@selenic.com
Simply repository locking...
r161 def trylock(self):
Ronny Pfannschmidt
made repo locks recursive and deprecate refcounting based lock releasing...
r8108 if self.held:
self.held += 1
return
Bryan O'Sullivan
lock.py: cache hostname, but not pid, in case we fork
r4947 if lock._host is None:
lock._host = socket.gethostname()
Bryan O'Sullivan
lock: if we fork, ensure that only the parent releases...
r18907 lockname = '%s:%s' % (lock._host, self.pid)
Vadim Gelfer
change lock format to let us detect and break stale locks....
r1877 while not self.held:
try:
FUJIWARA Katsunori
lock: take both vfs and lock file path relative to vfs to access via vfs...
r20091 self.vfs.makelock(lockname, self.f)
Vadim Gelfer
change lock format to let us detect and break stale locks....
r1877 self.held = 1
except (OSError, IOError), why:
if why.errno == errno.EEXIST:
locker = self.testlock()
Thomas Arendsen Hein
Don't step into an endless loop when lock file is empty.
r3686 if locker is not None:
FUJIWARA Katsunori
lock: take both vfs and lock file path relative to vfs to access via vfs...
r20091 raise error.LockHeld(errno.EAGAIN,
self.vfs.join(self.f), self.desc,
Matt Mackall
error: move lock errors...
r7640 locker)
Vadim Gelfer
change lock format to let us detect and break stale locks....
r1877 else:
Matt Mackall
error: move lock errors...
r7640 raise error.LockUnavailable(why.errno, why.strerror,
why.filename, self.desc)
Vadim Gelfer
change lock format to let us detect and break stale locks....
r1877
def testlock(self):
Thomas Arendsen Hein
Don't step into an endless loop when lock file is empty.
r3686 """return id of locker if lock is valid, else None.
If old-style lock, we cannot tell what machine locker is on.
with new-style lock, if locker is on this machine, we can
see if locker is alive. If locker is on this machine but
not alive, we can safely break lock.
The lock file is only deleted when None is returned.
"""
Tomasz Kleczek
lock: fixed race condition in trylock/testlock (issue3506)...
r17682 try:
FUJIWARA Katsunori
lock: take both vfs and lock file path relative to vfs to access via vfs...
r20091 locker = self.vfs.readlock(self.f)
Yuya Nishihara
lock: handle race in trylock/testlock on Windows...
r19927 except (OSError, IOError), why:
Tomasz Kleczek
lock: fixed race condition in trylock/testlock (issue3506)...
r17682 if why.errno == errno.ENOENT:
return None
raise
Benoit Boissinot
use __contains__, index or split instead of str.find...
r2579 try:
host, pid = locker.split(":", 1)
except ValueError:
Vadim Gelfer
change lock format to let us detect and break stale locks....
r1877 return locker
Bryan O'Sullivan
lock.py: cache hostname, but not pid, in case we fork
r4947 if host != lock._host:
Vadim Gelfer
change lock format to let us detect and break stale locks....
r1877 return locker
mpm@selenic.com
Simply repository locking...
r161 try:
Benoit Boissinot
use __contains__, index or split instead of str.find...
r2579 pid = int(pid)
Benoit Boissinot
lock: catch specific exceptions
r9685 except ValueError:
Vadim Gelfer
change lock format to let us detect and break stale locks....
r1877 return locker
if util.testpid(pid):
return locker
# if locker dead, break lock. must do this with another lock
# held, or can race and break valid lock.
try:
FUJIWARA Katsunori
lock: take both vfs and lock file path relative to vfs to access via vfs...
r20091 l = lock(self.vfs, self.f + '.break', timeout=0)
self.vfs.unlink(self.f)
Vadim Gelfer
change lock format to let us detect and break stale locks....
r1877 l.release()
Matt Mackall
error: move lock errors...
r7640 except error.LockError:
Vadim Gelfer
change lock format to let us detect and break stale locks....
r1877 return locker
mpm@selenic.com
Simply repository locking...
r161
def release(self):
Pierre-Yves David
lock: add mechanism to register post release callback
r15583 """release the lock and execute callback function if any
Bryan O'Sullivan
Merge spelling fixes
r17537 If the lock has been acquired multiple times, the actual release is
timeless@mozdev.org
spelling: release
r17510 delayed to the last release call."""
Ronny Pfannschmidt
made repo locks recursive and deprecate refcounting based lock releasing...
r8108 if self.held > 1:
self.held -= 1
Benoit Boissinot
lock: use '==' instead of 'is' for integer equality ('is' may not work)
r9680 elif self.held == 1:
mpm@selenic.com
Simply repository locking...
r161 self.held = 0
Bryan O'Sullivan
lock: if we fork, ensure that only the parent releases...
r18907 if os.getpid() != self.pid:
# we forked, and are not the parent
return
Benoit Boissinot
add a releasefn keyword to lock.lock...
r1530 if self.releasefn:
self.releasefn()
mpm@selenic.com
Fix troubles with clone and exception handling...
r503 try:
FUJIWARA Katsunori
lock: take both vfs and lock file path relative to vfs to access via vfs...
r20091 self.vfs.unlink(self.f)
Benoit Boissinot
lock: catch specific exceptions
r9685 except OSError:
pass
Matt Mackall
lock: change name of release chain
r15589 for callback in self.postrelease:
Pierre-Yves David
lock: add mechanism to register post release callback
r15583 callback()
mpm@selenic.com
Simply repository locking...
r161
Ronny Pfannschmidt
made repo locks recursive and deprecate refcounting based lock releasing...
r8108 def release(*locks):
for lock in locks:
if lock is not None:
lock.release()