##// END OF EJS Templates
memory-usage: fix `hg log --follow --rev R F` space complexity...
memory-usage: fix `hg log --follow --rev R F` space complexity When running `hg log --follow --rev REVS FILES`, the log code will walk the history of all FILES starting from the file revisions that exists in each REVS. Before doing so, it looks if the files actually exists in the target revisions. To do so, it opens the manifest of each revision in REVS to look up if we find the associated items in FILES. Before this changeset this was done in a way that created a changectx for each target revision, keeping them in memory while we look into each file. If the set of REVS is large, this means keeping the manifest for each entry in REVS in memory. That can be largeā€¦ if REV is in the form `::X`, this can quickly become huge and saturate the memory. We have seen usage allocating 2GB per second until memory runs out. So this changeset invert the two loop so that only one revision is kept in memory during the operation. This solve the memory explosion issue.

File last commit:

r37195:68ee6182 default
r50517:dcb2581e stable
Show More
ro.py
67 lines | 2.0 KiB | text/x-python | PythonLexer
##############################################################################
#
# Copyright (c) 2003 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""Compute a resolution order for an object and its bases
"""
from __future__ import absolute_import
__docformat__ = 'restructuredtext'
def _mergeOrderings(orderings):
"""Merge multiple orderings so that within-ordering order is preserved
Orderings are constrained in such a way that if an object appears
in two or more orderings, then the suffix that begins with the
object must be in both orderings.
For example:
>>> _mergeOrderings([
... ['x', 'y', 'z'],
... ['q', 'z'],
... [1, 3, 5],
... ['z']
... ])
['x', 'y', 'q', 1, 3, 5, 'z']
"""
seen = {}
result = []
for ordering in reversed(orderings):
for o in reversed(ordering):
if o not in seen:
seen[o] = 1
result.insert(0, o)
return result
def _flatten(ob):
result = [ob]
i = 0
for ob in iter(result):
i += 1
# The recursive calls can be avoided by inserting the base classes
# into the dynamically growing list directly after the currently
# considered object; the iterator makes sure this will keep working
# in the future, since it cannot rely on the length of the list
# by definition.
result[i:i] = ob.__bases__
return result
def ro(object):
"""Compute a "resolution order" for an object
"""
return _mergeOrderings([_flatten(object)])