upstream/mercurial-mirror Files · mercurial/interfaces/util.py

match: sort patterns before compiling them into a regex...

match: sort patterns before compiling them into a regex While investigating cripping performance for `hg cat` in some context, I discovered that, for large inputs, building a regex from out of order patterns result may result in a *much* slower regex and a much slower associated matcher's performance. So we are now sorting the patterns to help the regex engine. There is more to the story as we rely on regexp more than we should. See the next changeset for details. Benchmarks ========== In the following benchmark we are comparing the `hg cat` and `hg files` run time when matching against the full list of files in the repository. They are run: - without the rust extensions - with the standard python enfine (so without re2) sort vs non-sorted - Before this changeset (3f5137543773) --------------------------------------------------------- ###### hg files ############################################################### ### mercurial-2018-08-01-zstd-sparse-revlog sorted: 0.230092 seconds shuffled: 0.234235 seconds (+1.80%) ### pypy-2018-08-01-zstd-sparse-revlog sorted: 0.613567 seconds shuffled: 0.801880 seconds (+30.69%) ### mozilla-central-2018-08-01-zstd-sparse-revlog sorted: 62.474221 seconds shuffled: 1364.180218 seconds (+2083.59%) ### netbeans-2018-08-01-zstd-sparse-revlog sorted: 21.541828 seconds shuffled: 172.759857 seconds (+701.97%) ###### hg cat ################################################################# ### mercurial-2018-08-01-zstd-sparse-revlog sorted: 0.764407 seconds shuffled: 0.768924 seconds ### pypy-2018-08-01-zstd-sparse-revlog sorted: 2.065220 seconds shuffled: 2.276388 seconds (+10.22%) ### netbeans-2018-08-01-zstd-sparse-revlog sorted: 40.967983 seconds shuffled: 216.388709 seconds (+428.19%) ### mozilla-central-2018-08-01-zstd-sparse-revlog sorted: 105.228510 seconds shuffled: 1448.722784 seconds (+1276.74%) sort vs non-sorted - With this changeset ---------------------------------------- ###### hg files ############################################################### ### mercurial-2018-08-01-zstd-sparse-revlog all-list-pattern-sorted: 0.230069 all-list-pattern-shuffled: 0.231165 ### pypy-2018-08-01-zstd-sparse-revlog all-list-pattern-sorted: 0.616799 all-list-pattern-shuffled: 0.616393 ### netbeans-2018-08-01-zstd-sparse-revlog all-list-pattern-sorted: 21.586773 all-list-pattern-shuffled: 21.908197 ### mozilla-central-2018-08-01-zstd-sparse-revlog all-list-pattern-sorted: 61.279490 all-list-pattern-shuffled: 62.473549 ###### hg cat ################################################################# ### mercurial-2018-08-01-zstd-sparse-revlog sorted: 0.763883 seconds shuffled: 0.765848 seconds ### pypy-2018-08-01-zstd-sparse-revlog sorted: 2.070498 seconds shuffled: 2.069197 seconds ### netbeans-2018-08-01-zstd-sparse-revlog sorted: 41.392423 seconds shuffled: 41.648689 seconds ### mozilla-central-2018-08-01-zstd-sparse-revlog sorted: 103.315670 seconds shuffled: 104.369358 seconds

Gregory Szorc - - Load All Authors

File last commit:

r49801:642e31cb default


                r51285:47686726

stable

Download file

             util.py
        
                    37 lines
            
             | 1009 B
            
                | text/x-python
            
             |
                PythonLexer
            
             / mercurial / interfaces / util.py
          
                    History
                
                 |
                  Annotation
                 | Raw
                 |Copy content
                 |Copy permalink

      # util.py - Utilities for declaring interfaces.

      #

      # Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>

      #

      # This software may be used and distributed according to the terms of the

      # GNU General Public License version 2 or any later version.

      # zope.interface imposes a run-time cost due to module import overhead and

      # bookkeeping for declaring interfaces. So, we use stubs for various

      # zope.interface primitives unless instructed otherwise.

      from .. import encoding

      if encoding.environ.get(b'HGREALINTERFACES'):

          from ..thirdparty.zope import interface as zi

          Attribute = zi.Attribute

          Interface = zi.Interface

          implementer = zi.implementer

      else:

          class Attribute:

              def __init__(self, __name__, __doc__=b''):

                  pass

          class Interface:

              def __init__(

                  self, name, bases=(), attrs=None, __doc__=None, __module__=None

              ):

                  pass

          def implementer(*ifaces):

              def wrapper(cls):

                  return cls

              return wrapper

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

				# util.py - Utilities for declaring interfaces.
				#
				# Copyright 2018 Gregory Szorc <gregory.szorc@gmail.com>
				#
				# This software may be used and distributed according to the terms of the
				# GNU General Public License version 2 or any later version.

				# zope.interface imposes a run-time cost due to module import overhead and
				# bookkeeping for declaring interfaces. So, we use stubs for various
				# zope.interface primitives unless instructed otherwise.


				from .. import encoding

				if encoding.environ.get(b'HGREALINTERFACES'):
				from ..thirdparty.zope import interface as zi

				Attribute = zi.Attribute
				Interface = zi.Interface
				implementer = zi.implementer
				else:

				class Attribute:
				def __init__(self, __name__, __doc__=b''):
				pass

				class Interface:
				def __init__(
				self, name, bases=(), attrs=None, __doc__=None, __module__=None
				):
				pass

				def implementer(*ifaces):
				def wrapper(cls):
				return cls

				return wrapper