upstream/mercurial-mirror Files · mercurial/testing/revlog.py

branchmap-v3: filter topo heads using node for performance reason...

branchmap-v3: filter topo heads using node for performance reason The branchmap currently contains heads as nodeid. If we build a set of revnum with the topological heads, we need to turn the nodeid in the branchmap to revnum to be able to check if they are topo-heads. That nodeid → revnum lookup is "expensive" and adds up to something noticeable if you do it hundreds of thousand of time. Instead we turn all the topo-heads revnums into nodes and build a set. So we can directly test membership of the nodeids stored in the branchmap. That is much faster. Ideally we would have revnum in the branchmap and could directly test revnum against a revnum set and that would be even faster. However that's an adventure for another time. Without this change, the branchmap format "v3" was significantly slower than the "v2" format. With this changes, some of that gap is recovered With rust + persistent nodemap, this overhead was smaller because the extra lookup did not had to to build the nodemap from scratch. In addition the mozilla-unified repository is able to use the "pure_top" mode of branchmap v3, so it was not really affected by this. Future changeset will work of the remaining of the performance gap. ### benchmark.name = hg.command.unbundle # bin-env-vars.hg.py-re2-module = default # benchmark.variants.issue6528 = disabled # benchmark.variants.resource-usage = default # benchmark.variants.reuse-external-delta-parent = yes # benchmark.variants.revs = any-1-extra-rev # benchmark.variants.source = unbundle # benchmark.variants.validate = default # benchmark.variants.verbosity = quiet ## data-env-vars.name = netbeans-2018-08-01-zstd-sparse-revlog # bin-env-vars.hg.flavor = default branch-v2: 0.233711 ~~~~~ branch-v3 before: 0.380994 (+63.02%, +0.15) branch-v3 after: 0.368769 (+57.79%, +0.14) # bin-env-vars.hg.flavor = rust branch-v2: 0.235230 ~~~~~ branch-v3 before: 0.385060 (+63.70%, +0.15) branch-v3 after: 0.372460 (+58.34%, +0.14) ## data-env-vars.name = netbeans-2018-08-01-ds2-pnm # bin-env-vars.hg.flavor = rust branch-v2: 0.255586 ~~~~~ branch-v3 before: 0.317524 (+24.23%, +0.06) branch-v3 after: 0.318907 (+24.78%, +0.06) ## data-env-vars.name = mozilla-central-2024-03-22-zstd-sparse-revlog # bin-env-vars.hg.flavor = default branch-v2: 0.339010 ~~~~~ branch-v3 before: 0.410007 (+20.94%, +0.07) branch-v3 after: 0.349752 (+3.17%, +0.01) # bin-env-vars.hg.flavor = rust branch-v2: 0.346525 ~~~~~ branch-v3 before: 0.410428 (+18.44%, +0.06) branch-v3 after: 0.354300 (+2.24%, +0.01) ## data-env-vars.name = mozilla-central-2024-03-22-ds2-pnm # bin-env-vars.hg.flavor = rust branch-v2: 0.380202 ~~~~~ branch-v3 before: 0.393871 (+3.60%, +0.01) branch-v3 after: 0.396293 (+4.23%, +0.02) ## data-env-vars.name = mozilla-unified-2024-03-22-zstd-sparse-revlog # bin-env-vars.hg.flavor = default branch-v2: 0.412165 ~~~~~ branch-v3 before: 0.438105 (+6.29%, +0.03) branch-v3 after: 0.424769 (+3.06%, +0.01) # bin-env-vars.hg.flavor = rust branch-v2: 0.412397 ~~~~~ branch-v3 before: 0.438405 (+6.31%, +0.03) branch-v3 after: 0.421796 (+2.28%, +0.01) ## data-env-vars.name = mozilla-unified-2024-03-22-ds2-pnm # bin-env-vars.hg.flavor = rust branch-v2: 0.429501 ~~~~~ branch-v3 before: 0.452692 (+5.40%, +0.02) branch-v3 after: 0.443849 (+3.34%, +0.01) ## data-env-vars.name = mozilla-try-2024-03-26-zstd-sparse-revlog # bin-env-vars.hg.flavor = default branch-v2: 3.403171 ~~~~~ branch-v3 before: 6.562345 (+92.83%, +3.16) branch-v3 after: 6.234055 (+83.18%, +2.83) # bin-env-vars.hg.flavor = rust branch-v2: 3.454876 ~~~~~ branch-v3 before: 6.160248 (+78.31%, +2.71) branch-v3 after: 6.307813 (+82.58%, +2.85) ## data-env-vars.name = mozilla-try-2024-03-26-ds2-pnm # bin-env-vars.hg.flavor = rust branch-v2: 3.465435 ~~~~~ branch-v3 before: 5.381648 (+55.30%, +1.92) branch-v3 after: 5.176076 (+49.36%, +1.71)

Matt Harbison - - Load All Authors

File last commit:

r52756:f4733654 default


                r52869:41b8892a

default

Download file

             revlog.py
        
                    64 lines
            
             | 2.4 KiB
            
                | text/x-python
            
             |
                PythonLexer
            
             / mercurial / testing / revlog.py
          
                    History
                
                 |
                  Source
                 | Raw
                 |Copy content
                 |Copy permalink

        Matt Harbison
    
typing: add `from __future__ import annotations` to most files...

              r52756
            
      from __future__ import annotations

        marmoute
    
test: extract some generic data and utility from test-rust-ancestor.py...

              r44397
            
      import unittest

      # picked from test-parse-index2, copied rather than imported

      # so that it stays stable even if test-parse-index2 changes or disappears.

      data_non_inlined = (

          b'\x00\x00\x00\x01\x00\x00\x00\x00\x00\x01D\x19'

          b'\x00\x07e\x12\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff'

          b'\xff\xff\xff\xff\xd1\xf4\xbb\xb0\xbe\xfc\x13\xbd\x8c\xd3\x9d'

          b'\x0f\xcd\xd9;\x8c\x07\x8cJ/\x00\x00\x00\x00\x00\x00\x00\x00\x00'

          b'\x00\x00\x00\x00\x00\x00\x01D\x19\x00\x00\x00\x00\x00\xdf\x00'

          b'\x00\x01q\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\xff'

          b'\xff\xff\xff\xc1\x12\xb9\x04\x96\xa4Z1t\x91\xdfsJ\x90\xf0\x9bh'

          b'\x07l&\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

          b'\x00\x01D\xf8\x00\x00\x00\x00\x01\x1b\x00\x00\x01\xb8\x00\x00'

          b'\x00\x01\x00\x00\x00\x02\x00\x00\x00\x01\xff\xff\xff\xff\x02\n'

          b'\x0e\xc6&\xa1\x92\xae6\x0b\x02i\xfe-\xe5\xbao\x05\xd1\xe7\x00'

          b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01F'

          b'\x13\x00\x00\x00\x00\x01\xec\x00\x00\x03\x06\x00\x00\x00\x01'

          b'\x00\x00\x00\x03\x00\x00\x00\x02\xff\xff\xff\xff\x12\xcb\xeby1'

          b'\xb6\r\x98B\xcb\x07\xbd`\x8f\x92\xd9\xc4\x84\xbdK\x00\x00\x00'

          b'\x00\x00\x00\x00\x00\x00\x00\x00\x00'

      )

        Georges Racinet
    
rust-index: using the `hg::index::Index` in ancestors iterator and lazy set...

              r52132
            
      from ..revlogutils.constants import REVLOGV1

        marmoute
    
test: extract some generic data and utility from test-rust-ancestor.py...

              r44397
            
      try:

        Matt Harbison
    
typing: disable import error warnings that are already handled...

              r47543
            
          from ..cext import parsers as cparsers  # pytype: disable=import-error

        marmoute
    
test: extract some generic data and utility from test-rust-ancestor.py...

              r44397
            
      except ImportError:

          cparsers = None

        Georges Racinet
    
rust-index: using the `hg::index::Index` in ancestors iterator and lazy set...

              r52132
            
      try:

        Georges Racinet on incendie.racinet.fr
    
rust-index: renamed `MixedIndex` as `Index`...

              r52147
            
          from ..rustext.revlog import (  # pytype: disable=import-error

              Index as RustIndex,

          )

        Georges Racinet
    
rust-index: using the `hg::index::Index` in ancestors iterator and lazy set...

              r52132
            
      except ImportError:

        Georges Racinet on incendie.racinet.fr
    
rust-index: renamed `MixedIndex` as `Index`...

              r52147
            
          RustIndex = None

        Georges Racinet
    
rust-index: using the `hg::index::Index` in ancestors iterator and lazy set...

              r52132
            
        marmoute
    
test: extract some generic data and utility from test-rust-ancestor.py...

              r44397
            
      @unittest.skipIf(

          cparsers is None,

          'The C version of the "parsers" module is not available. It is needed for this test.',

      )

      class RevlogBasedTestBase(unittest.TestCase):

        Georges Racinet
    
rust-index: using the `hg::index::Index` in ancestors iterator and lazy set...

              r52132
            
          def parseindex(self, data=None):

              if data is None:

                  data = data_non_inlined

              return cparsers.parse_index2(data, False)[0]

        Georges Racinet on incendie.racinet.fr
    
rust-python-testing: separated base test classes...

              r52137
            
      @unittest.skipIf(

        Georges Racinet on incendie.racinet.fr
    
rust-index: renamed `MixedIndex` as `Index`...

              r52147
            
          RustIndex is None,

        Georges Racinet on incendie.racinet.fr
    
rust-python-testing: separated base test classes...

              r52137
            
          'The Rust index is not available. It is needed for this test.',

      )

      class RustRevlogBasedTestBase(unittest.TestCase):

        Georges Racinet
    
rust-index: using the `hg::index::Index` in ancestors iterator and lazy set...

              r52132
            
          def parserustindex(self, data=None):

              if data is None:

                  data = data_non_inlined

        Georges Racinet on incendie.racinet.fr
    
rust-python-testing: separated base test classes...

              r52137
            
              # not inheriting RevlogBasedTestCase to avoid having a

              # `parseindex` method that would be shadowed by future subclasses

              # this duplication will soon be removed

        Georges Racinet on incendie.racinet.fr
    
rust-index: renamed `MixedIndex` as `Index`...

              r52147
            
              return RustIndex(data, REVLOGV1)

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

Matt Harbison typing: add `from __future__ import annotations` to most files...	r52756	from __future__ import annotations

marmoute test: extract some generic data and utility from test-rust-ancestor.py...	r44397	import unittest

		# picked from test-parse-index2, copied rather than imported
		# so that it stays stable even if test-parse-index2 changes or disappears.
		data_non_inlined = (
		b'\x00\x00\x00\x01\x00\x00\x00\x00\x00\x01D\x19'
		b'\x00\x07e\x12\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff'
		b'\xff\xff\xff\xff\xd1\xf4\xbb\xb0\xbe\xfc\x13\xbd\x8c\xd3\x9d'
		b'\x0f\xcd\xd9;\x8c\x07\x8cJ/\x00\x00\x00\x00\x00\x00\x00\x00\x00'
		b'\x00\x00\x00\x00\x00\x00\x01D\x19\x00\x00\x00\x00\x00\xdf\x00'
		b'\x00\x01q\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\xff'
		b'\xff\xff\xff\xc1\x12\xb9\x04\x96\xa4Z1t\x91\xdfsJ\x90\xf0\x9bh'
		b'\x07l&\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
		b'\x00\x01D\xf8\x00\x00\x00\x00\x01\x1b\x00\x00\x01\xb8\x00\x00'
		b'\x00\x01\x00\x00\x00\x02\x00\x00\x00\x01\xff\xff\xff\xff\x02\n'
		b'\x0e\xc6&\xa1\x92\xae6\x0b\x02i\xfe-\xe5\xbao\x05\xd1\xe7\x00'
		b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01F'
		b'\x13\x00\x00\x00\x00\x01\xec\x00\x00\x03\x06\x00\x00\x00\x01'
		b'\x00\x00\x00\x03\x00\x00\x00\x02\xff\xff\xff\xff\x12\xcb\xeby1'
		b'\xb6\r\x98B\xcb\x07\xbd`\x8f\x92\xd9\xc4\x84\xbdK\x00\x00\x00'
		b'\x00\x00\x00\x00\x00\x00\x00\x00\x00'
		)

Georges Racinet rust-index: using the `hg::index::Index` in ancestors iterator and lazy set...	r52132	from ..revlogutils.constants import REVLOGV1

marmoute test: extract some generic data and utility from test-rust-ancestor.py...	r44397
		try:
Matt Harbison typing: disable import error warnings that are already handled...	r47543	from ..cext import parsers as cparsers # pytype: disable=import-error
marmoute test: extract some generic data and utility from test-rust-ancestor.py...	r44397	except ImportError:
		cparsers = None

Georges Racinet rust-index: using the `hg::index::Index` in ancestors iterator and lazy set...	r52132	try:
Georges Racinet on incendie.racinet.fr rust-index: renamed `MixedIndex` as `Index`...	r52147	from ..rustext.revlog import ( # pytype: disable=import-error
		Index as RustIndex,
		)
Georges Racinet rust-index: using the `hg::index::Index` in ancestors iterator and lazy set...	r52132	except ImportError:
Georges Racinet on incendie.racinet.fr rust-index: renamed `MixedIndex` as `Index`...	r52147	RustIndex = None
Georges Racinet rust-index: using the `hg::index::Index` in ancestors iterator and lazy set...	r52132
marmoute test: extract some generic data and utility from test-rust-ancestor.py...	r44397
		@unittest.skipIf(
		cparsers is None,
		'The C version of the "parsers" module is not available. It is needed for this test.',
		)
		class RevlogBasedTestBase(unittest.TestCase):
Georges Racinet rust-index: using the `hg::index::Index` in ancestors iterator and lazy set...	r52132	def parseindex(self, data=None):
		if data is None:
		data = data_non_inlined
		return cparsers.parse_index2(data, False)[0]

Georges Racinet on incendie.racinet.fr rust-python-testing: separated base test classes...	r52137
		@unittest.skipIf(
Georges Racinet on incendie.racinet.fr rust-index: renamed `MixedIndex` as `Index`...	r52147	RustIndex is None,
Georges Racinet on incendie.racinet.fr rust-python-testing: separated base test classes...	r52137	'The Rust index is not available. It is needed for this test.',
		)
		class RustRevlogBasedTestBase(unittest.TestCase):
Georges Racinet rust-index: using the `hg::index::Index` in ancestors iterator and lazy set...	r52132	def parserustindex(self, data=None):
		if data is None:
		data = data_non_inlined
Georges Racinet on incendie.racinet.fr rust-python-testing: separated base test classes...	r52137	# not inheriting RevlogBasedTestCase to avoid having a
		# `parseindex` method that would be shadowed by future subclasses
		# this duplication will soon be removed
Georges Racinet on incendie.racinet.fr rust-index: renamed `MixedIndex` as `Index`...	r52147	return RustIndex(data, REVLOGV1)