##// END OF EJS Templates
merge: perform background file closing in batchget...
merge: perform background file closing in batchget As 2fdbf22a1b63 demonstrated with stream clones, closing files on background threads on Windows can yield a significant speedup because closing files that have been created/appended to is slow on Windows/NTFS. Working directory updates can write thousands of files. Therefore it is susceptible to excessive slowness on Windows due to slow file closes. This patch enables background file closing when performing working directory file writes. The impact when performing an `hg up tip` on mozilla-central (136,357 files) from an empty working directory is significant: Before: 535s (8:55) After: 133s (2:13) Delta: -402s (6:42) That's a 4x speedup! By comparison, that same machine can perform the same operation in ~15s on Linux. So Windows went from ~35x to ~9x slower. Not bad but there's still work to do. As a reminder, background file closing is only activated on Windows because it is only beneficial on that platform. So this patch shouldn't change non-Windows behavior at all. It's worth noting that non-Windows systems perform working directory updates with multiple processes. Unfortunately, worker.py doesn't yet support Windows. So, there is still plenty of room for making working directory updates faster on Windows. Even if multiple processes are used on Windows, I believe background file closing will still provide a benefit, as individual processes will still be slowed down by the file close bottleneck (assuming the I/O system isn't saturated).

File last commit:

r27245:cea1473b stable
r28200:588695cc default
Show More
dockerlib.sh
42 lines | 1.6 KiB | application/x-sh | BashLexer
#!/bin/sh -eu
# This function exists to set up the DOCKER variable and verify that
# it's the binary we expect. It also verifies that the docker service
# is running on the system and we can talk to it.
function checkdocker() {
if which docker.io >> /dev/null 2>&1 ; then
DOCKER=docker.io
elif which docker >> /dev/null 2>&1 ; then
DOCKER=docker
else
echo "Error: docker must be installed"
exit 1
fi
$DOCKER -h 2> /dev/null | grep -q Jansens && { echo "Error: $DOCKER is the Docking System Tray - install docker.io instead"; exit 1; }
$DOCKER version | grep -Eq "^Client( version)?:" || { echo "Error: unexpected output from \"$DOCKER version\""; exit 1; }
$DOCKER version | grep -Eq "^Server( version)?:" || { echo "Error: could not get docker server version - check it is running and your permissions"; exit 1; }
}
# Construct a container and leave its name in $CONTAINER for future use.
function initcontainer() {
[ "$1" ] || { echo "Error: platform name must be specified"; exit 1; }
DFILE="$ROOTDIR/contrib/docker/$1"
[ -f "$DFILE" ] || { echo "Error: docker file $DFILE not found"; exit 1; }
CONTAINER="hg-dockerrpm-$1"
DBUILDUSER=build
(
cat $DFILE
if [ $(uname) = "Darwin" ] ; then
# The builder is using boot2docker on OS X, so we're going to
# *guess* the uid of the user inside the VM that is actually
# running docker. This is *very likely* to fail at some point.
echo RUN useradd $DBUILDUSER -u 1000
else
echo RUN groupadd $DBUILDUSER -g `id -g` -o
echo RUN useradd $DBUILDUSER -u `id -u` -g $DBUILDUSER -o
fi
) | $DOCKER build --tag $CONTAINER -
}