##// END OF EJS Templates
merge: perform background file closing in batchget...
merge: perform background file closing in batchget As 2fdbf22a1b63 demonstrated with stream clones, closing files on background threads on Windows can yield a significant speedup because closing files that have been created/appended to is slow on Windows/NTFS. Working directory updates can write thousands of files. Therefore it is susceptible to excessive slowness on Windows due to slow file closes. This patch enables background file closing when performing working directory file writes. The impact when performing an `hg up tip` on mozilla-central (136,357 files) from an empty working directory is significant: Before: 535s (8:55) After: 133s (2:13) Delta: -402s (6:42) That's a 4x speedup! By comparison, that same machine can perform the same operation in ~15s on Linux. So Windows went from ~35x to ~9x slower. Not bad but there's still work to do. As a reminder, background file closing is only activated on Windows because it is only beneficial on that platform. So this patch shouldn't change non-Windows behavior at all. It's worth noting that non-Windows systems perform working directory updates with multiple processes. Unfortunately, worker.py doesn't yet support Windows. So, there is still plenty of room for making working directory updates faster on Windows. Even if multiple processes are used on Windows, I believe background file closing will still provide a benefit, as individual processes will still be slowed down by the file close bottleneck (assuming the I/O system isn't saturated).

File last commit:

r27721:e4b512bb default
r28200:588695cc default
Show More
debugshell.py
58 lines | 1.3 KiB | text/x-python | PythonLexer
Vishakh H
contrib: add debugshell extension
r11633 # debugshell extension
"""a python shell with repo, changelog & manifest objects"""
Sean Farley
debugshell: check ui.debugger for which debugger to use
r19773 import sys
Vishakh H
contrib: add debugshell extension
r11633 import mercurial
import code
Gregory Szorc
debugshell: disable demand importer when importing debugger...
r27721 from mercurial import (
cmdutil,
demandimport,
)
Gregory Szorc
debugshell: declare command using decorator
r21243
cmdtable = {}
command = cmdutil.command(cmdtable)
Vishakh H
contrib: add debugshell extension
r11633
Sean Farley
debugshell: abstract out pdb code.interact
r19771 def pdb(ui, repo, msg, **opts):
Vishakh H
contrib: add debugshell extension
r11633 objects = {
'mercurial': mercurial,
'repo': repo,
'cl': repo.changelog,
'mf': repo.manifest,
}
Sean Farley
debugshell: abstract out pdb code.interact
r19771
code.interact(msg, local=objects)
Sean Farley
debugshell: add function to embed ipython
r19772 def ipdb(ui, repo, msg, **opts):
import IPython
cl = repo.changelog
mf = repo.manifest
Matt Mackall
debugshell: appease pyflakes
r19794 cl, mf # use variables to appease pyflakes
Sean Farley
debugshell: add function to embed ipython
r19772
IPython.embed()
Gregory Szorc
debugshell: declare command using decorator
r21243 @command('debugshell|dbsh', [])
Sean Farley
debugshell: abstract out pdb code.interact
r19771 def debugshell(ui, repo, **opts):
Vishakh H
contrib: add debugshell extension
r11633 bannermsg = "loaded repo : %s\n" \
"using source: %s" % (repo.root,
mercurial.__path__[0])
Sean Farley
debugshell: abstract out pdb code.interact
r19771
Sean Farley
debugshell: check ui.debugger for which debugger to use
r19773 pdbmap = {
'pdb' : 'code',
'ipdb' : 'IPython'
}
debugger = ui.config("ui", "debugger")
if not debugger:
debugger = 'pdb'
# if IPython doesn't exist, fallback to code.interact
try:
Gregory Szorc
debugshell: disable demand importer when importing debugger...
r27721 with demandimport.deactivated():
__import__(pdbmap[debugger])
Sean Farley
debugshell: check ui.debugger for which debugger to use
r19773 except ImportError:
ui.warn("%s debugger specified but %s module was not found\n"
% (debugger, pdbmap[debugger]))
debugger = 'pdb'
getattr(sys.modules[__name__], debugger)(ui, repo, bannermsg, **opts)