##// END OF EJS Templates
merge: mark file gets as not thread safe (issue5933)...
merge: mark file gets as not thread safe (issue5933) In default installs, this has the effect of disabling the thread-based worker on Windows when manifesting files in the working directory. My measurements have shown that with revlog-based repositories, Mercurial spends a lot of CPU time in revlog code resolving file data. This ends up incurring a lot of context switching across threads and slows down `hg update` operations when going from an empty working directory to the tip of the repo. On mozilla-unified (246,351 files) on an i7-6700K (4+4 CPUs): before: 487s wall after: 360s wall (equivalent to worker.enabled=false) cpus=2: 379s wall Even with only 2 threads, the thread pool is still slower. The introduction of the thread-based worker (02b36e860e0b) states that it resulted in a "~50%" speedup for `hg sparse --enable-profile` and `hg sparse --disable-profile`. This disagrees with my measurement above. I theorize a few reasons for this: 1) Removal of files from the working directory is I/O - not CPU - bound and should benefit from a thread pool (unless I/O is insanely fast and the GIL release is near instantaneous). So tests like `hg sparse --enable-profile` may exercise deletion throughput and aren't good benchmarks for worker tasks that are CPU heavy. 2) The patch was authored by someone at Facebook. The results were likely measured against a repository using remotefilelog. And I believe that revision retrieval during working directory updates with remotefilelog will often use a remote store, thus being I/O and not CPU bound. This probably resulted in an overstated performance gain. Since there appears to be a need to enable the thread-based worker with some stores, I've made the flagging of file gets as thread safe configurable. I've made it experimental because I don't want to formalize a boolean flag for this option and because this attribute is best captured against the store implementation. But we don't have a proper store API for this yet. I'd rather cross this bridge later. It is possible there are revlog-based repositories that do benefit from a thread-based worker. I didn't do very comprehensive testing. If there are, we may want to devise a more proper algorithm for whether to use the thread-based worker, including possibly config options to limit the number of threads to use. But until I see evidence that justifies complexity, simplicity wins. Differential Revision: https://phab.mercurial-scm.org/D3963

File last commit:

r38027:c1a028d1 default
r38755:be498426 default
Show More
mercurial.spec
164 lines | 4.7 KiB | text/x-rpm-spec | RPMSpecLexer
%global emacs_lispdir %{_datadir}/emacs/site-lisp
%define withpython %{nil}
%if "%{?withpython}"
%global pythonver %{withpython}
%global pythonname Python-%{withpython}
%global docutilsname docutils-0.14
%global docutilsmd5 c53768d63db3873b7d452833553469de
%global pythonhg python-hg
%global hgpyprefix /opt/%{pythonhg}
# byte compilation will fail on some some Python /test/ files
%global _python_bytecompile_errors_terminate_build 0
%else
%global pythonver %(python -c 'import sys;print ".".join(map(str, sys.version_info[:2]))')
%endif
Summary: A fast, lightweight Source Control Management system
Name: mercurial
Version: snapshot
Release: 0
License: GPLv2+
Group: Development/Tools
URL: https://mercurial-scm.org/
Source0: %{name}-%{version}-%{release}.tar.gz
%if "%{?withpython}"
Source1: %{pythonname}.tgz
Source2: %{docutilsname}.tar.gz
%endif
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root
BuildRequires: make, gcc, gettext
%if "%{?withpython}"
BuildRequires: readline-devel, openssl-devel, ncurses-devel, zlib-devel, bzip2-devel
%else
BuildRequires: python >= 2.7, python-devel, python-docutils >= 0.5
Requires: python >= 2.7
%endif
# The hgk extension uses the wish tcl interpreter, but we don't enforce it
#Requires: tk
%description
Mercurial is a fast, lightweight source control management system designed
for efficient handling of very large distributed projects.
%prep
%if "%{?withpython}"
%setup -q -n mercurial-%{version}-%{release} -a1 -a2
# despite the comments in cgi.py, we do this to prevent rpmdeps from picking /usr/local/bin/python up
sed -i '1c#! /usr/bin/env python' %{pythonname}/Lib/cgi.py
%else
%setup -q -n mercurial-%{version}-%{release}
%endif
%build
%if "%{?withpython}"
PYPATH=$PWD/%{pythonname}
cd $PYPATH
./configure --prefix=%{hgpyprefix}
make all %{?_smp_mflags}
cd -
cd %{docutilsname}
LD_LIBRARY_PATH=$PYPATH $PYPATH/python setup.py build
cd -
# verify Python environment
LD_LIBRARY_PATH=$PYPATH PYTHONPATH=$PWD/%{docutilsname} $PYPATH/python -c 'import sys, zlib, bz2, ssl, curses, readline'
# set environment for make
export PATH=$PYPATH:$PATH
export LD_LIBRARY_PATH=$PYPATH
export CFLAGS="-L $PYPATH"
export PYTHONPATH=$PWD/%{docutilsname}
%endif
make all
make -C contrib/chg
%install
rm -rf $RPM_BUILD_ROOT
%if "%{?withpython}"
PYPATH=$PWD/%{pythonname}
cd $PYPATH
make install DESTDIR=$RPM_BUILD_ROOT
# these .a are not necessary and they are readonly and strip fails - kill them!
rm -f %{buildroot}%{hgpyprefix}/lib/{,python2.*/config}/libpython2.*.a
cd -
cd %{docutilsname}
LD_LIBRARY_PATH=$PYPATH $PYPATH/python setup.py install --root="$RPM_BUILD_ROOT"
cd -
PATH=$PYPATH:$PATH LD_LIBRARY_PATH=$PYPATH make install DESTDIR=$RPM_BUILD_ROOT PREFIX=%{hgpyprefix} MANDIR=%{_mandir}
mkdir -p $RPM_BUILD_ROOT%{_bindir}
( cd $RPM_BUILD_ROOT%{_bindir}/ && ln -s ../..%{hgpyprefix}/bin/hg . )
( cd $RPM_BUILD_ROOT%{_bindir}/ && ln -s ../..%{hgpyprefix}/bin/python2.? %{pythonhg} )
%else
make install DESTDIR=$RPM_BUILD_ROOT PREFIX=%{_prefix} MANDIR=%{_mandir}
%endif
install -m 755 contrib/chg/chg $RPM_BUILD_ROOT%{_bindir}/
install -m 755 contrib/hgk $RPM_BUILD_ROOT%{_bindir}/
install -m 755 contrib/hg-ssh $RPM_BUILD_ROOT%{_bindir}/
bash_completion_dir=$RPM_BUILD_ROOT%{_sysconfdir}/bash_completion.d
mkdir -p $bash_completion_dir
install -m 644 contrib/bash_completion $bash_completion_dir/mercurial.sh
zsh_completion_dir=$RPM_BUILD_ROOT%{_datadir}/zsh/site-functions
mkdir -p $zsh_completion_dir
install -m 644 contrib/zsh_completion $zsh_completion_dir/_mercurial
mkdir -p $RPM_BUILD_ROOT%{emacs_lispdir}
install -m 644 contrib/mercurial.el $RPM_BUILD_ROOT%{emacs_lispdir}/
install -m 644 contrib/mq.el $RPM_BUILD_ROOT%{emacs_lispdir}/
mkdir -p $RPM_BUILD_ROOT/%{_sysconfdir}/mercurial/hgrc.d
%clean
rm -rf $RPM_BUILD_ROOT
%files
%defattr(-,root,root,-)
%doc CONTRIBUTORS COPYING doc/README doc/hg*.txt doc/hg*.html *.cgi contrib/*.fcgi
%doc %attr(644,root,root) %{_mandir}/man?/hg*
%doc %attr(644,root,root) contrib/*.svg
%dir %{_datadir}/zsh/
%dir %{_datadir}/zsh/site-functions/
%{_datadir}/zsh/site-functions/_mercurial
%dir %{_datadir}/emacs/site-lisp/
%{_datadir}/emacs/site-lisp/mercurial.el
%{_datadir}/emacs/site-lisp/mq.el
%{_bindir}/hg
%{_bindir}/chg
%{_bindir}/hgk
%{_bindir}/hg-ssh
%dir %{_sysconfdir}/bash_completion.d/
%config(noreplace) %{_sysconfdir}/bash_completion.d/mercurial.sh
%dir %{_sysconfdir}/mercurial
%dir %{_sysconfdir}/mercurial/hgrc.d
%if "%{?withpython}"
%{_bindir}/%{pythonhg}
%{hgpyprefix}
%else
%{_libdir}/python%{pythonver}/site-packages/%{name}-*-py%{pythonver}.egg-info
%{_libdir}/python%{pythonver}/site-packages/%{name}
%{_libdir}/python%{pythonver}/site-packages/hgext
%{_libdir}/python%{pythonver}/site-packages/hgext3rd
%{_libdir}/python%{pythonver}/site-packages/hgdemandimport
%endif