upstream/mercurial-mirror Files · mercurial/revlogutils/concurrency_checker.py

revlog: add a small cache of unfiltered chunk...

revlog: add a small cache of unfiltered chunk This can provides a massive boost to the reading of multiple revision and the computation of a valid delta chain. This greatly help operation like `hg log --patch`, delta computation (helping pull/unbundle), linkrev adjustment (helping copy tracing). A first round of benchmark for `hg log --patch --limit 1000` shows improvement in the 10-20% range on "small" repository like pypy or mercurial and large improvements (about 33%) for more complex ones like netbeans and mozilla's. These speeds up are consistent with the improvement to `hg pull` (from a server sending poor deltas) I saw benchmarking this last year. Further benchmark will be run during the freeze. I added some configuration in the experimental space to be able to further test the effect of various tuning for now. This feature should fit well in the "usage/resource profile" configuration that we should land next cycle. When it does not provides a benefit the overhead of the cache seem to be around 2%, a small price for the big improvement. In addition I believe we could shave most of this overhead with a more efficent lru implementation.

Kyle Lippincott - - Load All Authors

File last commit:

r47349:e9901d01 default


                r52001:0250e450

default

Download file

             concurrency_checker.py
        
                    38 lines
            
             | 1.5 KiB
            
                | text/x-python
            
             |
                PythonLexer
            
             / mercurial / revlogutils / concurrency_checker.py
          
                    History
                
                 |
                  Annotation
                 | Raw
                 |Copy content
                 |Copy permalink

      from ..i18n import _

      from .. import error

      def get_checker(ui, revlog_name=b'changelog'):

          """Get a function that checks file handle position is as expected.

          This is used to ensure that files haven't been modified outside of our

          knowledge (such as on a networked filesystem, if `hg debuglocks` was used,

          or writes to .hg that ignored locks happened).

          Due to revlogs supporting a concept of buffered, delayed, or diverted

          writes, we're allowing the files to be shorter than expected (the data may

          not have been written yet), but they can't be longer.

          Please note that this check is not perfect; it can't detect all cases (there

          may be false-negatives/false-OKs), but it should never claim there's an

          issue when there isn't (false-positives/false-failures).

          """

          vpos = ui.config(b'debug', b'revlog.verifyposition.' + revlog_name)

          # Avoid any `fh.tell` cost if this isn't enabled.

          if not vpos or vpos not in [b'log', b'warn', b'fail']:

              return None

          def _checker(fh, fn, expected):

              if fh.tell() <= expected:

                  return

              msg = _(b'%s: file cursor at position %d, expected %d')

              # Always log if we're going to warn or fail.

              ui.log(b'debug', msg + b'\n', fn, fh.tell(), expected)

              if vpos == b'warn':

                  ui.warn((msg + b'\n') % (fn, fh.tell(), expected))

              elif vpos == b'fail':

                  raise error.RevlogError(msg % (fn, fh.tell(), expected))

          return _checker

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

				from ..i18n import _
				from .. import error


				def get_checker(ui, revlog_name=b'changelog'):
				"""Get a function that checks file handle position is as expected.

				This is used to ensure that files haven't been modified outside of our
				knowledge (such as on a networked filesystem, if `hg debuglocks` was used,
				or writes to .hg that ignored locks happened).

				Due to revlogs supporting a concept of buffered, delayed, or diverted
				writes, we're allowing the files to be shorter than expected (the data may
				not have been written yet), but they can't be longer.

				Please note that this check is not perfect; it can't detect all cases (there
				may be false-negatives/false-OKs), but it should never claim there's an
				issue when there isn't (false-positives/false-failures).
				"""

				vpos = ui.config(b'debug', b'revlog.verifyposition.' + revlog_name)
				# Avoid any `fh.tell` cost if this isn't enabled.
				if not vpos or vpos not in [b'log', b'warn', b'fail']:
				return None

				def _checker(fh, fn, expected):
				if fh.tell() <= expected:
				return

				msg = _(b'%s: file cursor at position %d, expected %d')
				# Always log if we're going to warn or fail.
				ui.log(b'debug', msg + b'\n', fn, fh.tell(), expected)
				if vpos == b'warn':
				ui.warn((msg + b'\n') % (fn, fh.tell(), expected))
				elif vpos == b'fail':
				raise error.RevlogError(msg % (fn, fh.tell(), expected))

				return _checker