##// END OF EJS Templates
parsers: inline fields of dirstate values in C version...
parsers: inline fields of dirstate values in C version Previously, while unpacking the dirstate we'd create 3-4 new CPython objects for most dirstate values: - the state is a single character string, which is pooled by CPython - the mode is a new object if it isn't 0 due to being in the lookup set - the size is a new object if it is greater than 255 - the mtime is a new object if it isn't -1 due to being in the lookup set - the tuple to contain them all In some cases such as regular hg status, we actually look at all the objects. In other cases like hg add, hg status for a subdirectory, or hg status with the third-party hgwatchman enabled, we look at almost none of the objects. This patch eliminates most object creation in these cases by defining a custom C struct that is exposed to Python with an interface similar to a tuple. Only when tuple elements are actually requested are the respective objects created. The gains, where they're expected, are significant. The following tests are run against a working copy with over 270,000 files. parse_dirstate becomes significantly faster: $ hg perfdirstate before: wall 0.186437 comb 0.180000 user 0.160000 sys 0.020000 (best of 35) after: wall 0.093158 comb 0.100000 user 0.090000 sys 0.010000 (best of 95) and as a result, several commands benefit: $ time hg status # with hgwatchman enabled before: 0.42s user 0.14s system 99% cpu 0.563 total after: 0.34s user 0.12s system 99% cpu 0.471 total $ time hg add new-file before: 0.85s user 0.18s system 99% cpu 1.033 total after: 0.76s user 0.17s system 99% cpu 0.931 total There is a slight regression in regular status performance, but this is fixed in an upcoming patch.

File last commit:

r20202:a6014018 default
r21809:e250b830 default
Show More
windows.py
338 lines | 10.2 KiB | text/x-python | PythonLexer
Martin Geisler
put license and copyright info into comment blocks
r8226 # windows.py - Windows utility function implementations for Mercurial
#
# Copyright 2005-2009 Matt Mackall <mpm@selenic.com> and others
#
# This software may be used and distributed according to the terms of the
Matt Mackall
Update license to GPLv2+
r10263 # GNU General Public License version 2 or any later version.
Matt Mackall
util: split out posix, windows, and win32 modules
r7890
Steve Borho
windows: add various missing import
r7914 from i18n import _
Adrian Buehlmann
win32: move lookupreg() to windows.py...
r16807 import osutil, encoding
Bryan O'Sullivan
dirstate: move file type filtering to its source...
r18017 import errno, msvcrt, os, re, stat, sys, _winreg
Sune Foldager
posixfile: remove posixfile_nt and fix import bug in windows.py...
r8421
Matt Mackall
windows: fix pyflakes warning on unused imports...
r15016 import win32
executablepath = win32.executablepath
getuser = win32.getuser
hidewindow = win32.hidewindow
makedir = win32.makedir
nlinks = win32.nlinks
oslink = win32.oslink
samedevice = win32.samedevice
samefile = win32.samefile
setsignalhandler = win32.setsignalhandler
spawndetached = win32.spawndetached
Bryan O'Sullivan
util: implement a faster os.path.split for posix systems...
r17560 split = os.path.split
Matt Mackall
windows: fix pyflakes warning on unused imports...
r15016 termwidth = win32.termwidth
testpid = win32.testpid
unlink = win32.unlink
Adrian Buehlmann
windows: eliminate win32 wildcard import
r14985
Mads Kiilerich
windows: use umask 022 in debugstate output...
r15440 umask = 0022
Matt Mackall
util: split out posix, windows, and win32 modules
r7890
Sune Foldager
posixfile: remove posixfile_nt and fix import bug in windows.py...
r8421 # wrap osutil.posixfile to provide friendlier exceptions
def posixfile(name, mode='r', buffering=-1):
try:
return osutil.posixfile(name, mode, buffering)
except WindowsError, err:
Steve Borho
windows: provide filename in IOError exceptions...
r9448 raise IOError(err.errno, '%s: %s' % (name, err.strerror))
Sune Foldager
posixfile: remove posixfile_nt and fix import bug in windows.py...
r8421 posixfile.__doc__ = osutil.posixfile.__doc__
Matt Mackall
util: split out posix, windows, and win32 modules
r7890
Benoit Boissinot
use new style classes
r8778 class winstdout(object):
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 '''stdout on windows misbehaves if sent through a pipe'''
def __init__(self, fp):
self.fp = fp
def __getattr__(self, key):
return getattr(self.fp, key)
def close(self):
try:
self.fp.close()
Idan Kamara
eliminate various naked except clauses
r14004 except IOError:
pass
Matt Mackall
util: split out posix, windows, and win32 modules
r7890
def write(self, s):
try:
# This is workaround for "Not enough space" error on
# writing large size of data to console.
limit = 16000
l = len(s)
start = 0
Benoit Boissinot
fix coding style (reported by pylint)
r10394 self.softspace = 0
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 while start < l:
end = start + limit
self.fp.write(s[start:end])
start = end
except IOError, inst:
Matt Mackall
many, many trivial check-code fixups
r10282 if inst.errno != 0:
raise
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 self.close()
raise IOError(errno.EPIPE, 'Broken pipe')
def flush(self):
try:
return self.fp.flush()
except IOError, inst:
Matt Mackall
many, many trivial check-code fixups
r10282 if inst.errno != errno.EINVAL:
raise
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 self.close()
raise IOError(errno.EPIPE, 'Broken pipe')
Idan Kamara
win32: assign winstdout to sys.__stdout__ as well (issue2888)...
r14892 sys.__stdout__ = sys.stdout = winstdout(sys.stdout)
Matt Mackall
util: split out posix, windows, and win32 modules
r7890
def _is_win_9x():
'''return true if run on windows 95, 98 or me.'''
try:
return sys.getwindowsversion()[3] == 1
except AttributeError:
return 'command' in os.environ.get('comspec', '')
def openhardlinks():
Adrian Buehlmann
port win32.py to using the Python ctypes library...
r13375 return not _is_win_9x()
Matt Mackall
util: split out posix, windows, and win32 modules
r7890
Adrian Buehlmann
rename util.parse_patch_output to parsepatchoutput
r14231 def parsepatchoutput(output_line):
timeless
Generally replace "file name" with "filename" in help and comments.
r8761 """parses the output produced by patch and returns the filename"""
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 pf = output_line[14:]
if pf[0] == '`':
pf = pf[1:-1] # Remove the quotes
return pf
def sshargs(sshcmd, host, user, port):
'''Build argument list for ssh or Plink'''
pflag = 'plink' in sshcmd.lower() and '-P' or '-p'
args = user and ("%s@%s" % (user, host)) or host
return port and ("%s %s %s" % (args, pflag, port)) or args
Adrian Buehlmann
rename util.set_flags to setflags
r14232 def setflags(f, l, x):
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 pass
Adrian Buehlmann
util: move copymode into posix.py and windows.py...
r15011 def copymode(src, dst, mode=None):
pass
Adrian Buehlmann
util: move checkexec() to posix.py and return False on Windows
r13879 def checkexec(path):
return False
Adrian Buehlmann
util: move checklink() to posix.py and return False on Windows...
r13890 def checklink(path):
return False
Adrian Buehlmann
rename util.set_binary to setbinary
r14233 def setbinary(fd):
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 # When run without console, pipes may expose invalid
# fileno(), usually set to -1.
Augie Fackler
windows: use getattr instead of hasattr
r14969 fno = getattr(fd, 'fileno', None)
if fno is not None and fno() >= 0:
msvcrt.setmode(fno(), os.O_BINARY)
Matt Mackall
util: split out posix, windows, and win32 modules
r7890
def pconvert(path):
FUJIWARA Katsunori
windows: use 'str.replace()' instead of combination of split() and join()...
r16076 return path.replace(os.sep, '/')
Matt Mackall
util: split out posix, windows, and win32 modules
r7890
def localpath(path):
return path.replace('/', '\\')
def normpath(path):
return pconvert(os.path.normpath(path))
FUJIWARA Katsunori
windows: use upper() instead of lower() or os.path.normcase()...
r15671 def normcase(path):
Adrian Buehlmann
util, posix: eliminate encodinglower and encodingupper...
r17203 return encoding.upper(path)
Matt Mackall
dirstate: fix case-folding identity for traditional Unix...
r15488
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 def samestat(s1, s2):
return False
# A sequence of backslashes is special iff it precedes a double quote:
# - if there's an even number of backslashes, the double quote is not
# quoted (i.e. it ends the quoted region)
# - if there's an odd number of backslashes, the double quote is quoted
# - in both cases, every pair of backslashes is unquoted into a single
# backslash
# (See http://msdn2.microsoft.com/en-us/library/a1y7w461.aspx )
# So, to quote a string, we must surround it in double quotes, double
timeless@mozdev.org
spelling: precede
r17505 # the number of backslashes that precede double quotes and add another
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 # backslash before every double quote (being careful with the double
# quote we've appended to the end)
_quotere = None
def shellquote(s):
global _quotere
if _quotere is None:
_quotere = re.compile(r'(\\*)("|\\$)')
return '"%s"' % _quotere.sub(r'\1\1\\\2', s)
def quotecommand(cmd):
"""Build a command string suitable for os.popen* calls."""
Steve Borho
util: concentrate quoting knowledge to windows.py quotecommand()...
r13188 if sys.version_info < (2, 7, 1):
# Python versions since 2.7.1 do this extra quoting themselves
return '"' + cmd + '"'
return cmd
Matt Mackall
util: split out posix, windows, and win32 modules
r7890
def popen(command, mode='r'):
# Work around "popen spawned process may not write to stdout
# under windows"
# http://bugs.python.org/issue1366
Ross Lagerwall
util: replace util.nulldev with os.devnull...
r17391 command += " 2> %s" % os.devnull
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 return os.popen(quotecommand(command), mode)
Adrian Buehlmann
rename explain_exit to explainexit
r14234 def explainexit(code):
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 return _("exited with status %d") % code, code
# if you change this stub into a real check, please try to implement the
# username and groupname functions above, too.
Martin Geisler
posix: do not use fstat in isowner...
r8657 def isowner(st):
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 return True
Adrian Buehlmann
rename util.find_exe to findexe
r14271 def findexe(command):
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 '''Find executable for command searching like cmd.exe does.
If command is a basename then PATH is searched for command.
PATH isn't searched if command is an absolute or relative path.
An extension from PATHEXT is found and added if not present.
If command isn't found None is returned.'''
pathext = os.environ.get('PATHEXT', '.COM;.EXE;.BAT;.CMD')
pathexts = [ext for ext in pathext.lower().split(os.pathsep)]
if os.path.splitext(command)[1].lower() in pathexts:
pathexts = ['']
def findexisting(pathcommand):
'Will append extension (if needed) and return existing file'
for ext in pathexts:
executable = pathcommand + ext
if os.path.exists(executable):
return executable
return None
if os.sep in command:
return findexisting(command)
for path in os.environ.get('PATH', '').split(os.pathsep):
executable = findexisting(os.path.join(path, command))
if executable is not None:
return executable
Steve Borho
windows: expand environment vars in find_exe...
r10156 return findexisting(os.path.expanduser(os.path.expandvars(command)))
Matt Mackall
util: split out posix, windows, and win32 modules
r7890
Bryan O'Sullivan
dirstate: move file type filtering to its source...
r18017 _wantedkinds = set([stat.S_IFREG, stat.S_IFLNK])
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 def statfiles(files):
Bryan O'Sullivan
dirstate: move file type filtering to its source...
r18017 '''Stat each file in files. Yield each stat, or None if a file
does not exist or has a type we don't care about.
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 Cluster and cache stat per directory to minimize number of OS stat calls.'''
dircache = {} # dirname -> filename -> status | None if file does not exist
Bryan O'Sullivan
dirstate: move file type filtering to its source...
r18017 getkind = stat.S_IFMT
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 for nf in files:
FUJIWARA Katsunori
icasefs: use util.normcase() instead of str.lower() or os.path.normpath()
r15637 nf = normcase(nf)
Shun-ichi GOTO
Use os.path.split() for MBCS with win32mbcs extension.
r9099 dir, base = os.path.split(nf)
if not dir:
dir = '.'
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 cache = dircache.get(dir, None)
if cache is None:
try:
FUJIWARA Katsunori
icasefs: use util.normcase() instead of str.lower() or os.path.normpath()
r15637 dmap = dict([(normcase(n), s)
Bryan O'Sullivan
dirstate: move file type filtering to its source...
r18017 for n, k, s in osutil.listdir(dir, True)
Matt Mackall
windows: correctly pass a mode to S_IFMT in statfiles
r18041 if getkind(s.st_mode) in _wantedkinds])
Matt Mackall
util: split out posix, windows, and win32 modules
r7890 except OSError, err:
# handle directory not found in Python version prior to 2.5
# Python <= 2.4 returns native Windows code 3 in errno
# Python >= 2.5 returns ENOENT and adds winerror field
# EINVAL is raised if dir is not a directory.
if err.errno not in (3, errno.ENOENT, errno.EINVAL,
errno.ENOTDIR):
raise
dmap = {}
cache = dircache.setdefault(dir, dmap)
yield cache.get(base, None)
def username(uid=None):
"""Return the name of the user with the given uid.
If uid is None, return the name of the current user."""
return None
def groupname(gid=None):
"""Return the name of the group with the given gid.
If gid is None, return the name of the current group."""
return None
Henrik Stuart
windows: avoid deleting non-empty reparse points...
r8364 def _removedirs(name):
"""special version of os.removedirs that does not remove symlinked
directories or junction points if they actually contain files"""
if osutil.listdir(name):
return
os.rmdir(name)
head, tail = os.path.split(name)
if not tail:
head, tail = os.path.split(head)
while head and tail:
try:
Patrick Mezard
windows: fix unlink() not dropping empty tree (issue1861)
r9572 if osutil.listdir(head):
Henrik Stuart
windows: avoid deleting non-empty reparse points...
r8364 return
os.rmdir(head)
Idan Kamara
eliminate various naked except clauses
r14004 except (ValueError, OSError):
Henrik Stuart
windows: avoid deleting non-empty reparse points...
r8364 break
head, tail = os.path.split(head)
Mads Kiilerich
util: fold ENOENT check into unlinkpath, controlled by new ignoremissing flag...
r18143 def unlinkpath(f, ignoremissing=False):
Henrik Stuart
windows: avoid deleting non-empty reparse points...
r8364 """unlink and remove the directory if it is empty"""
Mads Kiilerich
util: fold ENOENT check into unlinkpath, controlled by new ignoremissing flag...
r18143 try:
unlink(f)
except OSError, e:
if not (ignoremissing and e.errno == errno.ENOENT):
raise
Henrik Stuart
windows: avoid deleting non-empty reparse points...
r8364 # try removing directories that might now be empty
try:
_removedirs(os.path.dirname(f))
except OSError:
pass
Adrian Buehlmann
util: move rename into posix.py and windows.py
r9549 def rename(src, dst):
'''atomically rename file src to dst, replacing dst if it exists'''
try:
os.rename(src, dst)
Adrian Buehlmann
windows.rename: check OSError for EEXIST...
r13278 except OSError, e:
if e.errno != errno.EEXIST:
raise
Adrian Buehlmann
reintroduces util.unlink, for POSIX and Windows....
r13280 unlink(dst)
Adrian Buehlmann
util: move rename into posix.py and windows.py
r9549 os.rename(src, dst)
Patrick Mezard
Find right hg command for detached process...
r10239 def gethgcmd():
return [sys.executable] + sys.argv[:1]
Patrick Mezard
acl: grp module is not available on windows
r11138 def groupmembers(name):
# Don't support groups on Windows for now
Brodie Rao
cleanup: "raise SomeException()" -> "raise SomeException"
r16687 raise KeyError
Patrick Mezard
acl: grp module is not available on windows
r11138
Adrian Buehlmann
util: eliminate wildcard imports
r14926 def isexec(f):
return False
Idan Kamara
posix, windows: introduce cachestat...
r14927 class cachestat(object):
def __init__(self, path):
pass
def cacheable(self):
return False
Adrian Buehlmann
win32: move lookupreg() to windows.py...
r16807 def lookupreg(key, valname=None, scope=None):
''' Look up a key/value name in the Windows registry.
valname: value name. If unspecified, the default value for the key
is used.
scope: optionally specify scope for registry lookup, this can be
a sequence of scopes to look up in order. Default (CURRENT_USER,
LOCAL_MACHINE).
'''
if scope is None:
scope = (_winreg.HKEY_CURRENT_USER, _winreg.HKEY_LOCAL_MACHINE)
elif not isinstance(scope, (list, tuple)):
scope = (scope,)
for s in scope:
try:
val = _winreg.QueryValueEx(_winreg.OpenKey(s, key), valname)[0]
# never let a Unicode string escape into the wild
return encoding.tolocal(val.encode('UTF-8'))
except EnvironmentError:
pass
Matt Mackall
cmdutils: Take over glob expansion duties from util
r8614 expandglobs = True
Bryan O'Sullivan
util: add functions to check symlink/exec bits...
r18868
def statislink(st):
'''check whether a stat result is a symlink'''
return False
def statisexec(st):
'''check whether a stat result is an executable file'''
return False