upstream/mercurial-mirror Files · mercurial/pure/charencode.py

phases: large rewrite on retract boundary...

phases: large rewrite on retract boundary The new code is still pure Python, so we still have room to going significantly faster. However its complexity of the complex part is `O(|[min_new_draft, tip]|)` instead of `O(|[min_draft, tip]|` which should help tremendously one repository with old draft (like mercurial-devel or mozilla-try). This is especially useful as the most common "retract boundary" operation happens when we commit/rewrite new drafts or when we push new draft to a non-publishing server. In this case, the smallest new_revs is very close to the tip and there is very few work to do. A few smaller optimisation could be done for these cases and will be introduced in later changesets. We still have iterate over large sets of roots, but this is already a great improvement for a very small amount of work. We gather information on the affected changeset as we go as we can put it to use in the next changesets. This extra data collection might slowdown the `register_new` case a bit, however for register_new, it should not really matters. The set of new nodes is either small, so the impact is negligible, or the set of new nodes is large, and the amount of work to do to had them will dominate the overhead the collecting information in `changed_revs`. As this new code compute the changes on the fly, it unlock other interesting improvement to be done in later changeset.

Gregory Szorc - - Load All Authors

File last commit:

r49762:b677bccf default


                r52302:2f39c7ae

default

Download file

             charencode.py
        
                    86 lines
            
             | 2.2 KiB
            
                | text/x-python
            
             |
                PythonLexer
            
             / mercurial / pure / charencode.py
          
                    History
                
                 |
                  Annotation
                 | Raw
                 |Copy content
                 |Copy permalink

      # charencode.py - miscellaneous character encoding

      #

      #  Copyright 2005-2009 Olivia Mackall <olivia@selenic.com> and others

      #

      # This software may be used and distributed according to the terms of the

      # GNU General Public License version 2 or any later version.

      import array

      from .. import pycompat

      def isasciistr(s):

          try:

              s.decode('ascii')

              return True

          except UnicodeDecodeError:

              return False

      def asciilower(s):

          """convert a string to lowercase if ASCII

          Raises UnicodeDecodeError if non-ASCII characters are found."""

          s.decode('ascii')

          return s.lower()

      def asciiupper(s):

          """convert a string to uppercase if ASCII

          Raises UnicodeDecodeError if non-ASCII characters are found."""

          s.decode('ascii')

          return s.upper()

      _jsonmap = []

      _jsonmap.extend(b"\\u%04x" % x for x in range(32))

      _jsonmap.extend(pycompat.bytechr(x) for x in range(32, 127))

      _jsonmap.append(b'\\u007f')

      _jsonmap[0x09] = b'\\t'

      _jsonmap[0x0A] = b'\\n'

      _jsonmap[0x22] = b'\\"'

      _jsonmap[0x5C] = b'\\\\'

      _jsonmap[0x08] = b'\\b'

      _jsonmap[0x0C] = b'\\f'

      _jsonmap[0x0D] = b'\\r'

      _paranoidjsonmap = _jsonmap[:]

      _paranoidjsonmap[0x3C] = b'\\u003c'  # '<' (e.g. escape "</script>")

      _paranoidjsonmap[0x3E] = b'\\u003e'  # '>'

      _jsonmap.extend(pycompat.bytechr(x) for x in range(128, 256))

      def jsonescapeu8fast(u8chars, paranoid):

          """Convert a UTF-8 byte string to JSON-escaped form (fast path)

          Raises ValueError if non-ASCII characters have to be escaped.

          """

          if paranoid:

              jm = _paranoidjsonmap

          else:

              jm = _jsonmap

          try:

              return b''.join(jm[x] for x in bytearray(u8chars))

          except IndexError:

              raise ValueError

      _utf8strict = r'surrogatepass'

      def jsonescapeu8fallback(u8chars, paranoid):

          """Convert a UTF-8 byte string to JSON-escaped form (slow path)

          Escapes all non-ASCII characters no matter if paranoid is False.

          """

          if paranoid:

              jm = _paranoidjsonmap

          else:

              jm = _jsonmap

          # non-BMP char is represented as UTF-16 surrogate pair

          u16b = u8chars.decode('utf-8', _utf8strict).encode('utf-16', _utf8strict)

          u16codes = array.array('H', u16b)

          u16codes.pop(0)  # drop BOM

          return b''.join(jm[x] if x < 128 else b'\\u%04x' % x for x in u16codes)

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

				# charencode.py - miscellaneous character encoding
				#
				# Copyright 2005-2009 Olivia Mackall <olivia@selenic.com> and others
				#
				# This software may be used and distributed according to the terms of the
				# GNU General Public License version 2 or any later version.


				import array

				from .. import pycompat


				def isasciistr(s):
				try:
				s.decode('ascii')
				return True
				except UnicodeDecodeError:
				return False


				def asciilower(s):
				"""convert a string to lowercase if ASCII

				Raises UnicodeDecodeError if non-ASCII characters are found."""
				s.decode('ascii')
				return s.lower()


				def asciiupper(s):
				"""convert a string to uppercase if ASCII

				Raises UnicodeDecodeError if non-ASCII characters are found."""
				s.decode('ascii')
				return s.upper()


				_jsonmap = []
				_jsonmap.extend(b"\\u%04x" % x for x in range(32))
				_jsonmap.extend(pycompat.bytechr(x) for x in range(32, 127))
				_jsonmap.append(b'\\u007f')
				_jsonmap[0x09] = b'\\t'
				_jsonmap[0x0A] = b'\\n'
				_jsonmap[0x22] = b'\\"'
				_jsonmap[0x5C] = b'\\\\'
				_jsonmap[0x08] = b'\\b'
				_jsonmap[0x0C] = b'\\f'
				_jsonmap[0x0D] = b'\\r'
				_paranoidjsonmap = _jsonmap[:]
				_paranoidjsonmap[0x3C] = b'\\u003c' # '<' (e.g. escape "</script>")
				_paranoidjsonmap[0x3E] = b'\\u003e' # '>'
				_jsonmap.extend(pycompat.bytechr(x) for x in range(128, 256))


				def jsonescapeu8fast(u8chars, paranoid):
				"""Convert a UTF-8 byte string to JSON-escaped form (fast path)

				Raises ValueError if non-ASCII characters have to be escaped.
				"""
				if paranoid:
				jm = _paranoidjsonmap
				else:
				jm = _jsonmap
				try:
				return b''.join(jm[x] for x in bytearray(u8chars))
				except IndexError:
				raise ValueError


				_utf8strict = r'surrogatepass'


				def jsonescapeu8fallback(u8chars, paranoid):
				"""Convert a UTF-8 byte string to JSON-escaped form (slow path)

				Escapes all non-ASCII characters no matter if paranoid is False.
				"""
				if paranoid:
				jm = _paranoidjsonmap
				else:
				jm = _jsonmap
				# non-BMP char is represented as UTF-16 surrogate pair
				u16b = u8chars.decode('utf-8', _utf8strict).encode('utf-16', _utf8strict)
				u16codes = array.array('H', u16b)
				u16codes.pop(0) # drop BOM
				return b''.join(jm[x] if x < 128 else b'\\u%04x' % x for x in u16codes)