upstream/mercurial-mirror Files · mercurial/thirdparty/zope/interface/_flatten.py

memory-usage: fix `hg log --follow --rev R F` space complexity...

memory-usage: fix `hg log --follow --rev R F` space complexity When running `hg log --follow --rev REVS FILES`, the log code will walk the history of all FILES starting from the file revisions that exists in each REVS. Before doing so, it looks if the files actually exists in the target revisions. To do so, it opens the manifest of each revision in REVS to look up if we find the associated items in FILES. Before this changeset this was done in a way that created a changectx for each target revision, keeping them in memory while we look into each file. If the set of REVS is large, this means keeping the manifest for each entry in REVS in memory. That can be large… if REV is in the form `::X`, this can quickly become huge and saturate the memory. We have seen usage allocating 2GB per second until memory runs out. So this changeset invert the two loop so that only one revision is kept in memory during the operation. This solve the memory explosion issue.

Gregory Szorc - - Load All Authors

File last commit:

r37195:68ee6182 default


                r50517:dcb2581e

stable

Download file

             _flatten.py
        
                    38 lines
            
             | 1.1 KiB
            
                | text/x-python
            
             |
                PythonLexer
            
             / mercurial / thirdparty / zope / interface / _flatten.py
          
                    History
                
                 |
                  Annotation
                 | Raw
                 |Copy content
                 |Copy permalink

      ##############################################################################

      #

      # Copyright (c) 2002 Zope Foundation and Contributors.

      # All Rights Reserved.

      #

      # This software is subject to the provisions of the Zope Public License,

      # Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.

      # THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED

      # WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED

      # WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS

      # FOR A PARTICULAR PURPOSE.

      #

      ##############################################################################

      """Adapter-style interface registry

      See Adapter class.

      """

      from __future__ import absolute_import

      from .interface import Declaration

      def _flatten(implements, include_None=0):

          try:

              r = implements.flattened()

          except AttributeError:

              if implements is None:

                  r=()

              else:

                  r = Declaration(implements).flattened()

          if not include_None:

              return r

          r = list(r)

          r.append(None)

          return r

	Site-wide shortcuts
/	Use quick search box
g h	Goto home page
g g	Goto my private gists page
g G	Goto my public gists page
g 0-9	Goto bookmarked items from 0-9
n r	New repository page
n g	New gist page

	Repositories
g s	Goto summary page
g c	Goto changelog page
g f	Goto files page
g F	Goto files page with file search activated
g p	Goto pull requests page
g o	Goto repository settings
g O	Goto repository access permissions settings
t s	Toggle sidebar on some pages

				##############################################################################
				#
				# Copyright (c) 2002 Zope Foundation and Contributors.
				# All Rights Reserved.
				#
				# This software is subject to the provisions of the Zope Public License,
				# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
				# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
				# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
				# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
				# FOR A PARTICULAR PURPOSE.
				#
				##############################################################################
				"""Adapter-style interface registry

				See Adapter class.
				"""

				from __future__ import absolute_import

				from .interface import Declaration

				def _flatten(implements, include_None=0):

				try:
				r = implements.flattened()
				except AttributeError:
				if implements is None:
				r=()
				else:
				r = Declaration(implements).flattened()

				if not include_None:
				return r

				r = list(r)
				r.append(None)
				return r