##// END OF EJS Templates
tests: add test extension implementing custom filelog storage...
tests: add test extension implementing custom filelog storage In order to better support partial clones, we'll need alternate repository storage mechanisms that aren't based on revlogs. Today, the interface for repository storage isn't very well defined. And there are various layering violations and assumptions made throughout the code that storage is backed by revlogs. In order to support alternate storage mechanisms, we'll need to formally declare and adhere to interfaces for storage. This will be a long, arduous process. This commit creates an extension that implements non-revlog storage for files. It defines a custom type that quacks like the existing revlog/filelog API but isn't backed by a revlog. The backing storage is - for simplicity reasons - a CBOR index and per-node files representing fulltext data. The localrepository class is modified so file(f) returns instances of this class instead of filelog instances. The purpose of this extension is to tease out what the actual filelog interface is - based on running the test harness - so we can formalize that interface and then implement a *real* alternate storage backend. Using `run-tests.py --extra-config-opt` to run the test harness with this extension enabled yields 83 failures out of 634 ran tests. The most common test failures are due to: * Issues with `hg verify` * LFS and largefiles (probably flags processing related) * Narrow. * Any test touching or inspecting individual filelog paths. * help and error output that is confused by the presence of an extension. * `hg debug*` commands doing low-level, revlog-y things. An 88% pass rate is pretty good for an initial implementation if you ask me! There is a bit of duplicate code in the new extension. That's by design: a point of this code is to tease out dependencies on revlog. That being said, there is opportunity to consolidate code by moving things out of the revlog API. For example, DAG traversal operations don't necessarily need to be implemented at the storage level. (Although for performance reasons they probably do.) Once we have a more well-defined interface, we could probably define the default implementations in terms of the base interface, pull those in via class inheritance, and have implementations override with faster versions if they so choose. (Or something like that.) But for now, the duplicate code should be acceptable. Differential Revision: https://phab.mercurial-scm.org/D3029

File last commit:

r35580:cf841f2b default
r37355:d257c5f2 default
Show More
storefactory.py
86 lines | 2.5 KiB | text/x-python | PythonLexer
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
from __future__ import absolute_import
import re
from mercurial.i18n import _
from mercurial import (
error,
hg,
util,
)
from . import (
lfutil,
localstore,
wirestore,
)
# During clone this function is passed the src's ui object
# but it needs the dest's ui object so it can read out of
# the config file. Use repo.ui instead.
def openstore(repo=None, remote=None, put=False, ui=None):
if ui is None:
ui = repo.ui
if not remote:
lfpullsource = getattr(repo, 'lfpullsource', None)
if lfpullsource:
path = ui.expandpath(lfpullsource)
elif put:
path = ui.expandpath('default-push', 'default')
else:
path = ui.expandpath('default')
# ui.expandpath() leaves 'default-push' and 'default' alone if
# they cannot be expanded: fallback to the empty string,
# meaning the current directory.
if repo is None:
path = ui.expandpath('default')
path, _branches = hg.parseurl(path)
remote = hg.peer(repo or ui, {}, path)
elif path == 'default-push' or path == 'default':
path = ''
remote = repo
else:
path, _branches = hg.parseurl(path)
remote = hg.peer(repo or ui, {}, path)
# The path could be a scheme so use Mercurial's normal functionality
# to resolve the scheme to a repository and use its path
path = util.safehasattr(remote, 'url') and remote.url() or remote.path
match = _scheme_re.match(path)
if not match: # regular filesystem path
scheme = 'file'
else:
scheme = match.group(1)
try:
storeproviders = _storeprovider[scheme]
except KeyError:
raise error.Abort(_('unsupported URL scheme %r') % scheme)
for classobj in storeproviders:
try:
return classobj(ui, repo, remote)
except lfutil.storeprotonotcapable:
pass
raise error.Abort(_('%s does not appear to be a largefile store') %
util.hidepassword(path))
_storeprovider = {
'file': [localstore.localstore],
'http': [wirestore.wirestore],
'https': [wirestore.wirestore],
'ssh': [wirestore.wirestore],
}
_scheme_re = re.compile(br'^([a-zA-Z0-9+-.]+)://')
def getlfile(ui, hash):
return util.chunkbuffer(openstore(ui=ui)._get(hash))