upstream/mercurial-mirror Files · mercurial/thirdparty/zope/interface/_flatten.py

store: don't read the whole fncache in memory...

store: don't read the whole fncache in memory In large repositories with lot of files, the fncache grows more than 100 MB and reading that whole thing into memory slows things down. Let's not read the whole thing into memory. This patch changes fncache loading code to read 1 MB at once. Loading 1 MB at once saves ~1 sec on perffncacheload for our internal repository. I tried various values such as 0.5 MB, 5 MB, 10 MB but best results were produced using 1 MB as the chunksize. On a narrow clone with fncache around 40 MB, this patch saves ~0.04 seconds on average on perffncacheload. To test the code, I have coded an extension in test-fncache.t which set chunksize to 1 byte, and the test passes with that. Differential Revision: https://phab.mercurial-scm.org/D5296

Gregory Szorc - - Load All Authors

File last commit:

r37195:68ee6182 default


                r42144:a5648708

default

Download file

             _flatten.py
        
                    38 lines
            
             | 1.1 KiB
            
                | text/x-python
            
             |
                PythonLexer
            
             / mercurial / thirdparty / zope / interface / _flatten.py
          
                    History
                
                 |
                  Annotation
                 | Raw
                 |Copy content
                 |Copy permalink

      ##############################################################################

      #

      # Copyright (c) 2002 Zope Foundation and Contributors.

      # All Rights Reserved.

      #

      # This software is subject to the provisions of the Zope Public License,

      # Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.

      # THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED

      # WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED

      # WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS

      # FOR A PARTICULAR PURPOSE.

      #

      ##############################################################################

      """Adapter-style interface registry

      See Adapter class.

      """

      from __future__ import absolute_import

      from .interface import Declaration

      def _flatten(implements, include_None=0):

          try:

              r = implements.flattened()

          except AttributeError:

              if implements is None:

                  r=()

              else:

                  r = Declaration(implements).flattened()

          if not include_None:

              return r

          r = list(r)

          r.append(None)

          return r

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

				##############################################################################
				#
				# Copyright (c) 2002 Zope Foundation and Contributors.
				# All Rights Reserved.
				#
				# This software is subject to the provisions of the Zope Public License,
				# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
				# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
				# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
				# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
				# FOR A PARTICULAR PURPOSE.
				#
				##############################################################################
				"""Adapter-style interface registry

				See Adapter class.
				"""

				from __future__ import absolute_import

				from .interface import Declaration

				def _flatten(implements, include_None=0):

				try:
				r = implements.flattened()
				except AttributeError:
				if implements is None:
				r=()
				else:
				r = Declaration(implements).flattened()

				if not include_None:
				return r

				r = list(r)
				r.append(None)
				return r