##// END OF EJS Templates
revlog: optionally cache the full text when adding revisions...
revlog: optionally cache the full text when adding revisions revlog instances can cache the full text of a single revision. Typically the most recently read revision is cached. When adding a delta group via addgroup() and _addrevision(), the full text isn't always computed: sometimes only the passed in delta is sufficient for adding a new revision to the revlog. When writing the changelog from a delta group, the just-added full text revision is always read immediately after it is written because the changegroup code needs to extract the set of files from the entry. In other words, revision() is *always* being called and caching the full text of the just-added revision is guaranteed to result in a cache hit, making the cache worthwhile. This patch adds support to _addrevision() for always building and caching the full text. This option is currently only active when processing changelog entries from a changegroup. While the total number of revision() calls is the same, the location matters: buildtext() calls into revision() on the base revision when building the full text of the just-added revision. Since the previous revision's _addrevision() built the full text and the the previous revision is likely the base revision, this means that the base revision's full text is likely cached and can be used to compute the current full text from just a delta. No extra I/O required. The end result is the changelog isn't opened and read after adding every revision from a changegroup. On my 2013 MacBook Pro running OS X 10.10.5 from an SSD and Python 2.7, this patch impacted the time taken to apply ~262,000 changesets from a mozilla-central gzip bundle: before: ~43s after: ~32s ~25% reduction in changelog processing times. Not bad.

File last commit:

r21292:a7a9d84f default
r26243:83629142 default
Show More
py3kcompat.py
65 lines | 2.1 KiB | text/x-python | PythonLexer
# py3kcompat.py - compatibility definitions for running hg in py3k
#
# Copyright 2010 Renato Cunha <renatoc@gmail.com>
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
import builtins
from numbers import Number
def bytesformatter(format, args):
'''Custom implementation of a formatter for bytestrings.
This function currently relies on the string formatter to do the
formatting and always returns bytes objects.
>>> bytesformatter(20, 10)
0
>>> bytesformatter('unicode %s, %s!', ('string', 'foo'))
b'unicode string, foo!'
>>> bytesformatter(b'test %s', 'me')
b'test me'
>>> bytesformatter('test %s', 'me')
b'test me'
>>> bytesformatter(b'test %s', b'me')
b'test me'
>>> bytesformatter('test %s', b'me')
b'test me'
>>> bytesformatter('test %d: %s', (1, b'result'))
b'test 1: result'
'''
# The current implementation just converts from bytes to unicode, do
# what's needed and then convert the results back to bytes.
# Another alternative is to use the Python C API implementation.
if isinstance(format, Number):
# If the fixer erroneously passes a number remainder operation to
# bytesformatter, we just return the correct operation
return format % args
if isinstance(format, bytes):
format = format.decode('utf-8', 'surrogateescape')
if isinstance(args, bytes):
args = args.decode('utf-8', 'surrogateescape')
if isinstance(args, tuple):
newargs = []
for arg in args:
if isinstance(arg, bytes):
arg = arg.decode('utf-8', 'surrogateescape')
newargs.append(arg)
args = tuple(newargs)
ret = format % args
return ret.encode('utf-8', 'surrogateescape')
builtins.bytesformatter = bytesformatter
origord = builtins.ord
def fakeord(char):
if isinstance(char, int):
return char
return origord(char)
builtins.ord = fakeord
if __name__ == '__main__':
import doctest
doctest.testmod()