upstream/mercurial-mirror Files · rust/README.rst

changegroup: introduce requests to define delta generation...

changegroup: introduce requests to define delta generation Currently, we iterate through each revision we will be producing a delta for then call into 1 of 2 functions for generating that delta. Deltas are emitted as we iterate. A problem with this model is that revision generation is tightly coupled to the changegroup code. And the storage layer needs to expose APIs like deltaparent() so changegroup delta generation can produce a delta with that knowledge. Another problem is that in this model, deltas can only be produced sequentially after the previous delta was produced and emitted. Some storage backends might be capable of producing deltas in parallel (e.g. if the changegroup deltas are cached somewhere). This commit aims to solve these problems by turning delta generation into a 2 phase implementation where the first phase determines info about all the deltas that need to be generated and the 2nd phase resolves those deltas. We introduce a "revisiondeltarequest" object that holds data about a to-be-generated delta. We perform a full pass over all revisions whose delta is to be generated and generate a "revisiondeltarequest" for each. Then we iterate over the "revisiondeltarequest" instances and derive a "revisiondelta" for each. This patch was quite large. In order to avoid even more churn, aspects of the implementation are less than ideal. e.g. we're recording revision numbers instead of nodes in a few places and we don't yet have a formal API for resolving an iterable of revisiondeltarequest instances. Things will be improved in subsequent commits. Unfortunately, this commit reduces performance substantially. For `hg perfchangegroupchangelog` on my hg repo: ! wall 1.512607 comb 1.510000 user 1.490000 sys 0.020000 (best of 7) ! wall 2.150863 comb 2.150000 user 2.150000 sys 0.000000 (best of 5) And for `hg bundle -t none-v2 -a` for the mozilla-unified repo: 178.32user 4.22system 3:02.59elapsed 190.97user 4.17system 3:15.19elapsed Some of this was attributed to changelog slowdown. `hg perfchangegroupchangelog` on mozilla-unified: ! wall 21.688715 comb 21.690000 user 21.570000 sys 0.120000 (best of 3) ! wall 25.683659 comb 25.680000 user 25.540000 sys 0.140000 (best of 3) Profiling seems to reveal that the changelog slowdown is due to reading changelog revisions multiple times. First in the linknode callback to resolve the set of files changed. Second in the delta generation. Before, we likely had hit the last revision cache in the revlog when doing delta generation since we performed that immediately after performing the linknode callback. I'm not exactly sure where the other ~8s are being spent. It might be from overhead of constructing a few million revisiondeltarequest objects. I'm OK with the regression for now because it is in service of a larger cause (storage abstraction). I'll try to profile later and claw back the performance. Differential Revision: https://phab.mercurial-scm.org/D4215

Gregory Szorc - - Load All Authors

File last commit:

r35587:96421278 default


                r39054:e793e11e

default

Download file

             README.rst
        
                    78 lines
            
             | 2.2 KiB
            
                | text/x-rst
            
             |
                RstLexer
            
             / rust / README.rst
          
                    History
                
                 |
                  Source
                 | Raw
                 |Copy content
                 |Copy permalink

        Gregory Szorc
    
rust: implementation of `hg`...

              r35587
            
      ===================

      Mercurial Rust Code

      ===================

      This directory contains various Rust code for the Mercurial project.

      The top-level ``Cargo.toml`` file defines a workspace containing

      all primary Mercurial crates.

      Building

      ========

      To build the Rust components::

         $ cargo build

      If you prefer a non-debug / release configuration::

         $ cargo build --release

      Features

      --------

      The following Cargo features are available:

      localdev (default)

         Produce files that work with an in-source-tree build.

         In this mode, the build finds and uses a ``python2.7`` binary from

         ``PATH``. The ``hg`` binary assumes it runs from ``rust/target/<target>hg``

         and it finds Mercurial files at ``dirname($0)/../../../``.

      Build Mechanism

      ---------------

      The produced ``hg`` binary is *bound* to a CPython installation. The

      binary links against and loads a CPython library that is discovered

      at build time (by a ``build.rs`` Cargo build script). The Python

      standard library defined by this CPython installation is also used.

      Finding the appropriate CPython installation to use is done by

      the ``python27-sys`` crate's ``build.rs``. Its search order is::

      1. ``PYTHON_SYS_EXECUTABLE`` environment variable.

      2. ``python`` executable on ``PATH``

      3. ``python2`` executable on ``PATH``

      4. ``python2.7`` executable on ``PATH``

      Additional verification of the found Python will be performed by our

      ``build.rs`` to ensure it meets Mercurial's requirements.

      Details about the build-time configured Python are built into the

      produced ``hg`` binary. This means that a built ``hg`` binary is only

      suitable for a specific, well-defined role. These roles are controlled

      by Cargo features (see above).

      Running

      =======

      The ``hgcli`` crate produces an ``hg`` binary. You can run this binary

      via ``cargo run``::

         $ cargo run --manifest-path hgcli/Cargo.toml

      Or directly::

         $ target/debug/hg

         $ target/release/hg

      You can also run the test harness with this binary::

         $ ./run-tests.py --with-hg ../rust/target/debug/hg

      .. note::

         Integration with the test harness is still preliminary. Remember to

         ``cargo build`` after changes because the test harness doesn't yet

         automatically build Rust code.

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

Gregory Szorc rust: implementation of `hg`...	r35587	===================
		Mercurial Rust Code
		===================

		This directory contains various Rust code for the Mercurial project.

		The top-level ``Cargo.toml`` file defines a workspace containing
		all primary Mercurial crates.

		Building
		========

		To build the Rust components::

		$ cargo build

		If you prefer a non-debug / release configuration::

		$ cargo build --release

		Features
		--------

		The following Cargo features are available:

		localdev (default)
		Produce files that work with an in-source-tree build.

		In this mode, the build finds and uses a ``python2.7`` binary from
		``PATH``. The ``hg`` binary assumes it runs from ``rust/target/<target>hg``
		and it finds Mercurial files at ``dirname($0)/../../../``.

		Build Mechanism
		---------------

		The produced ``hg`` binary is bound to a CPython installation. The
		binary links against and loads a CPython library that is discovered
		at build time (by a ``build.rs`` Cargo build script). The Python
		standard library defined by this CPython installation is also used.

		Finding the appropriate CPython installation to use is done by
		the ``python27-sys`` crate's ``build.rs``. Its search order is::

		1. ``PYTHON_SYS_EXECUTABLE`` environment variable.
		2. ``python`` executable on ``PATH``
		3. ``python2`` executable on ``PATH``
		4. ``python2.7`` executable on ``PATH``

		Additional verification of the found Python will be performed by our
		``build.rs`` to ensure it meets Mercurial's requirements.

		Details about the build-time configured Python are built into the
		produced ``hg`` binary. This means that a built ``hg`` binary is only
		suitable for a specific, well-defined role. These roles are controlled
		by Cargo features (see above).

		Running
		=======

		The ``hgcli`` crate produces an ``hg`` binary. You can run this binary
		via ``cargo run``::

		$ cargo run --manifest-path hgcli/Cargo.toml

		Or directly::

		$ target/debug/hg
		$ target/release/hg

		You can also run the test harness with this binary::

		$ ./run-tests.py --with-hg ../rust/target/debug/hg

		.. note::

		Integration with the test harness is still preliminary. Remember to
		``cargo build`` after changes because the test harness doesn't yet
		automatically build Rust code.