upstream/mercurial-mirror Files · mercurial/revlogutils/concurrency_checker.py

revlog: improve the robustness of the splitting process...

revlog: improve the robustness of the splitting process The previous "in-place" splitting, preserving the splitting on transaction failure had a couple of issue in case of transaction rollback: - a race windows that could still lead to a crash and data loss - it corrupted the `fncache`. So instead, we use a new approach that we summarized as "we do a backup of the inline revlog pre-split, and we restore this in case of failure". To make readers live easier, we don't overwrite the inline index file until transaction finalization. (once the transaction get into its finalization phase, it is not expected to rollback, unless some crash happens). To do so, we write the index of the split index in a temporary file that we use until transaction finalization. We also keep a backup of the initial inline file to be able to rollback the split if needed. As a result, transaction rollback cancel the split and no longer corrupt fncache. We also no longer have a small inconsistency windows where the transaction could be unrecoverable.

Kyle Lippincott - - Load All Authors

File last commit:

r47349:e9901d01 default


                r51242:87f0155d

stable

Download file

             concurrency_checker.py
        
                    38 lines
            
             | 1.5 KiB
            
                | text/x-python
            
             |
                PythonLexer
            
             / mercurial / revlogutils / concurrency_checker.py
          
                    History
                
                 |
                  Annotation
                 | Raw
                 |Copy content
                 |Copy permalink

      from ..i18n import _

      from .. import error

      def get_checker(ui, revlog_name=b'changelog'):

          """Get a function that checks file handle position is as expected.

          This is used to ensure that files haven't been modified outside of our

          knowledge (such as on a networked filesystem, if `hg debuglocks` was used,

          or writes to .hg that ignored locks happened).

          Due to revlogs supporting a concept of buffered, delayed, or diverted

          writes, we're allowing the files to be shorter than expected (the data may

          not have been written yet), but they can't be longer.

          Please note that this check is not perfect; it can't detect all cases (there

          may be false-negatives/false-OKs), but it should never claim there's an

          issue when there isn't (false-positives/false-failures).

          """

          vpos = ui.config(b'debug', b'revlog.verifyposition.' + revlog_name)

          # Avoid any `fh.tell` cost if this isn't enabled.

          if not vpos or vpos not in [b'log', b'warn', b'fail']:

              return None

          def _checker(fh, fn, expected):

              if fh.tell() <= expected:

                  return

              msg = _(b'%s: file cursor at position %d, expected %d')

              # Always log if we're going to warn or fail.

              ui.log(b'debug', msg + b'\n', fn, fh.tell(), expected)

              if vpos == b'warn':

                  ui.warn((msg + b'\n') % (fn, fh.tell(), expected))

              elif vpos == b'fail':

                  raise error.RevlogError(msg % (fn, fh.tell(), expected))

          return _checker

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

				from ..i18n import _
				from .. import error


				def get_checker(ui, revlog_name=b'changelog'):
				"""Get a function that checks file handle position is as expected.

				This is used to ensure that files haven't been modified outside of our
				knowledge (such as on a networked filesystem, if `hg debuglocks` was used,
				or writes to .hg that ignored locks happened).

				Due to revlogs supporting a concept of buffered, delayed, or diverted
				writes, we're allowing the files to be shorter than expected (the data may
				not have been written yet), but they can't be longer.

				Please note that this check is not perfect; it can't detect all cases (there
				may be false-negatives/false-OKs), but it should never claim there's an
				issue when there isn't (false-positives/false-failures).
				"""

				vpos = ui.config(b'debug', b'revlog.verifyposition.' + revlog_name)
				# Avoid any `fh.tell` cost if this isn't enabled.
				if not vpos or vpos not in [b'log', b'warn', b'fail']:
				return None

				def _checker(fh, fn, expected):
				if fh.tell() <= expected:
				return

				msg = _(b'%s: file cursor at position %d, expected %d')
				# Always log if we're going to warn or fail.
				ui.log(b'debug', msg + b'\n', fn, fh.tell(), expected)
				if vpos == b'warn':
				ui.warn((msg + b'\n') % (fn, fh.tell(), expected))
				elif vpos == b'fail':
				raise error.RevlogError(msg % (fn, fh.tell(), expected))

				return _checker