upstream/mercurial-mirror Files · rust/README.rst

discovery: slowly increase sampling size...

discovery: slowly increase sampling size Some pathological discovery runs can requires many roundtrip. When this happens things can get very slow. To make the algorithm more resilience again such pathological case. We slowly increase the sample size with each roundtrip (+5%). This will have a negligible impact on "normal" discovery with few roundtrips, but a large positive impact of case with many roundtrips. Asking more question per roundtrip helps to reduce the undecided set faster. Instead of reducing the undecided set a linear speed (in the worst case), we reduce it as a guaranteed (small) exponential rate. The data below show this slow ramp up in sample size: round trip | 1 | 5 | 10 | 20 | 50 | 100 | 130 | sample size | 200 | 254 | 321 | 517 | 2 199 | 25 123 | 108 549 | covered nodes | 200 | 1 357 | 2 821 | 7 031 | 42 658 | 524 530 | 2 276 755 | To be a bit more concrete, lets take a very pathological case as an example. We are doing discovery from a copy of Mozilla-try to a more recent version of mozilla-unified. Mozilla-unified heads are unknown to the mozilla-try repo and there are over 1 million "missing" changesets. (the discovery is "local" to avoid network interference) Without this change, the discovery: - last 1858 seconds (31 minutes), - does 1700 round trip, - asking about 340 000 nodes. With this change, the discovery: - last 218 seconds (3 minutes, 38 seconds a -88% improvement), - does 94 round trip (-94%), - asking about 344 211 nodes (+1%). Of course, this is an extreme case (and 3 minutes is still slow). However this give a good example of how this sample size increase act as a safety net catching any bad situations. We could image a steeper increase than 5%. For example 10% would give the following number: round trip | 1 | 5 | 10 | 20 | 50 | 75 | 100 | sample size | 200 | 321 | 514 | 1 326 | 23 060 | 249 812 | 2 706 594 | covered nodes | 200 | 1 541 | 3 690 | 12 671 | 251 871 | 2 746 254 | 29 770 966 | In parallel, it is useful to understand these pathological cases and improve them. However the current change provides a general purpose safety net to smooth the impact of pathological cases. To avoid issue with older http server, the increase in sample size only occurs if the protocol has not limit on command argument size.

Gregory Szorc - - Load All Authors

File last commit:

r35587:96421278 default


                r42546:dbd0fcca

default

Download file

             README.rst
        
                    78 lines
            
             | 2.2 KiB
            
                | text/x-rst
            
             |
                RstLexer
            
             / rust / README.rst
          
                    History
                
                 |
                  Source
                 | Raw
                 |Copy content
                 |Copy permalink

        Gregory Szorc
    
rust: implementation of `hg`...

              r35587
            
      ===================

      Mercurial Rust Code

      ===================

      This directory contains various Rust code for the Mercurial project.

      The top-level ``Cargo.toml`` file defines a workspace containing

      all primary Mercurial crates.

      Building

      ========

      To build the Rust components::

         $ cargo build

      If you prefer a non-debug / release configuration::

         $ cargo build --release

      Features

      --------

      The following Cargo features are available:

      localdev (default)

         Produce files that work with an in-source-tree build.

         In this mode, the build finds and uses a ``python2.7`` binary from

         ``PATH``. The ``hg`` binary assumes it runs from ``rust/target/<target>hg``

         and it finds Mercurial files at ``dirname($0)/../../../``.

      Build Mechanism

      ---------------

      The produced ``hg`` binary is *bound* to a CPython installation. The

      binary links against and loads a CPython library that is discovered

      at build time (by a ``build.rs`` Cargo build script). The Python

      standard library defined by this CPython installation is also used.

      Finding the appropriate CPython installation to use is done by

      the ``python27-sys`` crate's ``build.rs``. Its search order is::

      1. ``PYTHON_SYS_EXECUTABLE`` environment variable.

      2. ``python`` executable on ``PATH``

      3. ``python2`` executable on ``PATH``

      4. ``python2.7`` executable on ``PATH``

      Additional verification of the found Python will be performed by our

      ``build.rs`` to ensure it meets Mercurial's requirements.

      Details about the build-time configured Python are built into the

      produced ``hg`` binary. This means that a built ``hg`` binary is only

      suitable for a specific, well-defined role. These roles are controlled

      by Cargo features (see above).

      Running

      =======

      The ``hgcli`` crate produces an ``hg`` binary. You can run this binary

      via ``cargo run``::

         $ cargo run --manifest-path hgcli/Cargo.toml

      Or directly::

         $ target/debug/hg

         $ target/release/hg

      You can also run the test harness with this binary::

         $ ./run-tests.py --with-hg ../rust/target/debug/hg

      .. note::

         Integration with the test harness is still preliminary. Remember to

         ``cargo build`` after changes because the test harness doesn't yet

         automatically build Rust code.

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

Gregory Szorc rust: implementation of `hg`...	r35587	===================
		Mercurial Rust Code
		===================

		This directory contains various Rust code for the Mercurial project.

		The top-level ``Cargo.toml`` file defines a workspace containing
		all primary Mercurial crates.

		Building
		========

		To build the Rust components::

		$ cargo build

		If you prefer a non-debug / release configuration::

		$ cargo build --release

		Features
		--------

		The following Cargo features are available:

		localdev (default)
		Produce files that work with an in-source-tree build.

		In this mode, the build finds and uses a ``python2.7`` binary from
		``PATH``. The ``hg`` binary assumes it runs from ``rust/target/<target>hg``
		and it finds Mercurial files at ``dirname($0)/../../../``.

		Build Mechanism
		---------------

		The produced ``hg`` binary is bound to a CPython installation. The
		binary links against and loads a CPython library that is discovered
		at build time (by a ``build.rs`` Cargo build script). The Python
		standard library defined by this CPython installation is also used.

		Finding the appropriate CPython installation to use is done by
		the ``python27-sys`` crate's ``build.rs``. Its search order is::

		1. ``PYTHON_SYS_EXECUTABLE`` environment variable.
		2. ``python`` executable on ``PATH``
		3. ``python2`` executable on ``PATH``
		4. ``python2.7`` executable on ``PATH``

		Additional verification of the found Python will be performed by our
		``build.rs`` to ensure it meets Mercurial's requirements.

		Details about the build-time configured Python are built into the
		produced ``hg`` binary. This means that a built ``hg`` binary is only
		suitable for a specific, well-defined role. These roles are controlled
		by Cargo features (see above).

		Running
		=======

		The ``hgcli`` crate produces an ``hg`` binary. You can run this binary
		via ``cargo run``::

		$ cargo run --manifest-path hgcli/Cargo.toml

		Or directly::

		$ target/debug/hg
		$ target/release/hg

		You can also run the test harness with this binary::

		$ ./run-tests.py --with-hg ../rust/target/debug/hg

		.. note::

		Integration with the test harness is still preliminary. Remember to
		``cargo build`` after changes because the test harness doesn't yet
		automatically build Rust code.