##// END OF EJS Templates
localrepo: iteratively derive local repository type...
localrepo: iteratively derive local repository type This commit implements the dynamic local repository type derivation that was explained in the recent commit bfeab472e3c0 "localrepo: create new function for instantiating a local repo object." Instead of a static localrepository class/type which must be customized after construction, we now dynamically construct a type by building up base classes/types to represent specific repository interfaces. Conceptually, the end state is similar to what was happening when various extensions would monkeypatch the __class__ of newly-constructed repo instances. However, the approach is inverted. Instead of making the instance then customizing it, we do the customization up front by influencing the behavior of the type then we instantiate that custom type. This approach gives us much more flexibility. For example, we can use completely separate classes for implementing different aspects of the repository. For example, we could have one class representing revlog-based file storage and another representing non-revlog based file storage. When then choose which implementation to use based on the presence of repo requirements. A concern with this approach is that it creates a lot more types and complexity and that complexity adds overhead. Yes, it is true that this approach will result in more types being created. Yes, this is more complicated than traditional "instantiate a static type." However, I believe the alternatives to supporting alternate storage backends are just as complicated. (Before I arrived at this solution, I had patches storing factory functions on local repo instances for e.g. constructing a file storage instance. We ended up having a handful of these. And this was logically identical to assigning custom methods. Since we were logically changing the type of the instance, I figured it would be better to just use specialized types instead of introducing levels of abstraction at run-time.) On the performance front, I don't believe that having N base classes has any significant performance overhead compared to just a single base class. Intuition says that Python will need to iterate the base classes to find an attribute. However, CPython caches method lookups: as long as the __class__ or MRO isn't changing, method attribute lookup should be constant time after first access. And non-method attributes are stored in __dict__, of which there is only 1 per object, so the number of base classes for __dict__ is irrelevant. Anyway, this commit splits up the monolithic completelocalrepository interface into sub-interfaces: 1 for file storage and 1 representing everything else. We've taught ``makelocalrepository()`` to call a series of factory functions which will produce types implementing specific interfaces. It then calls type() to create a new type from the built-up list of base types. This commit should be considered a start and not the end state. I suspect we'll hit a number of problems as we start to implement alternate storage backends: * Passing custom arguments to __init__ and setting custom attributes on __dict__. * Customizing the set of interfaces that are needed. e.g. the "readonly" intent could translate to not requesting an interface providing methods related to writing. * More ergonomic way for extensions to insert themselves so their callbacks aren't unconditionally called. * Wanting to modify vfs instances, other arguments passed to __init__. That being said, this code is usable in its current state and I'm convinced future commits will demonstrate the value in this approach. Differential Revision: https://phab.mercurial-scm.org/D4642

File last commit:

r34464:718f7acd default
r39800:e4e88157 default
Show More
state.py
134 lines | 4.4 KiB | text/x-python | PythonLexer
Martijn Pieters
fsmonitor: new experimental extension...
r28433 # state.py - fsmonitor persistent state
#
# Copyright 2013-2016 Facebook, Inc.
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
from __future__ import absolute_import
import errno
import os
import socket
import struct
Yuya Nishihara
py3: move up symbol imports to enforce import-checker rules...
r29205 from mercurial.i18n import _
Siddharth Agarwal
fsmonitor: don't write out state if identity has changed (issue5581)...
r32816 from mercurial import (
pathutil,
util,
)
Martijn Pieters
fsmonitor: new experimental extension...
r28433
_version = 4
_versionformat = ">I"
class state(object):
def __init__(self, repo):
Durham Goode
fsmonitor: remove use of repo.opener...
r31215 self._vfs = repo.vfs
Martijn Pieters
fsmonitor: new experimental extension...
r28433 self._ui = repo.ui
self._rootdir = pathutil.normasprefix(repo.root)
self._lastclock = None
Siddharth Agarwal
fsmonitor: don't write out state if identity has changed (issue5581)...
r32816 self._identity = util.filestat(None)
Martijn Pieters
fsmonitor: new experimental extension...
r28433
Gregory Szorc
fsmonitor: use configitem...
r34464 self.mode = self._ui.config('fsmonitor', 'mode')
Martijn Pieters
fsmonitor: new experimental extension...
r28433 self.walk_on_invalidate = self._ui.configbool(
Gregory Szorc
fsmonitor: use configitem...
r34464 'fsmonitor', 'walk_on_invalidate')
self.timeout = float(self._ui.config('fsmonitor', 'timeout'))
Martijn Pieters
fsmonitor: new experimental extension...
r28433
def get(self):
try:
Durham Goode
fsmonitor: remove use of repo.opener...
r31215 file = self._vfs('fsmonitor.state', 'rb')
Martijn Pieters
fsmonitor: new experimental extension...
r28433 except IOError as inst:
Siddharth Agarwal
fsmonitor: don't write out state if identity has changed (issue5581)...
r32816 self._identity = util.filestat(None)
Martijn Pieters
fsmonitor: new experimental extension...
r28433 if inst.errno != errno.ENOENT:
raise
return None, None, None
Siddharth Agarwal
fsmonitor: don't write out state if identity has changed (issue5581)...
r32816 self._identity = util.filestat.fromfp(file)
Martijn Pieters
fsmonitor: new experimental extension...
r28433 versionbytes = file.read(4)
if len(versionbytes) < 4:
self._ui.log(
'fsmonitor', 'fsmonitor: state file only has %d bytes, '
'nuking state\n' % len(versionbytes))
self.invalidate()
return None, None, None
try:
diskversion = struct.unpack(_versionformat, versionbytes)[0]
if diskversion != _version:
# different version, nuke state and start over
self._ui.log(
'fsmonitor', 'fsmonitor: version switch from %d to '
'%d, nuking state\n' % (diskversion, _version))
self.invalidate()
return None, None, None
state = file.read().split('\0')
# state = hostname\0clock\0ignorehash\0 + list of files, each
# followed by a \0
Simon Farnsworth
fsmonitor: be robust in the face of bad state...
r30539 if len(state) < 3:
self._ui.log(
'fsmonitor', 'fsmonitor: state file truncated (expected '
'3 chunks, found %d), nuking state\n', len(state))
self.invalidate()
return None, None, None
Martijn Pieters
fsmonitor: new experimental extension...
r28433 diskhostname = state[0]
hostname = socket.gethostname()
if diskhostname != hostname:
# file got moved to a different host
self._ui.log('fsmonitor', 'fsmonitor: stored hostname "%s" '
'different from current "%s", nuking state\n' %
(diskhostname, hostname))
self.invalidate()
return None, None, None
clock = state[1]
ignorehash = state[2]
# discard the value after the last \0
notefiles = state[3:-1]
finally:
file.close()
return clock, ignorehash, notefiles
def set(self, clock, ignorehash, notefiles):
if clock is None:
self.invalidate()
return
Siddharth Agarwal
fsmonitor: don't write out state if identity has changed (issue5581)...
r32816 # Read the identity from the file on disk rather than from the open file
# pointer below, because the latter is actually a brand new file.
identity = util.filestat.frompath(self._vfs.join('fsmonitor.state'))
if identity != self._identity:
self._ui.debug('skip updating fsmonitor.state: identity mismatch\n')
return
Martijn Pieters
fsmonitor: new experimental extension...
r28433 try:
Siddharth Agarwal
fsmonitor: don't write out state if identity has changed (issue5581)...
r32816 file = self._vfs('fsmonitor.state', 'wb', atomictemp=True,
checkambig=True)
Martijn Pieters
fsmonitor: new experimental extension...
r28433 except (IOError, OSError):
self._ui.warn(_("warning: unable to write out fsmonitor state\n"))
return
Simon Farnsworth
fsmonitor: be robust in the face of bad state...
r30539 with file:
Martijn Pieters
fsmonitor: new experimental extension...
r28433 file.write(struct.pack(_versionformat, _version))
file.write(socket.gethostname() + '\0')
file.write(clock + '\0')
file.write(ignorehash + '\0')
if notefiles:
file.write('\0'.join(notefiles))
file.write('\0')
def invalidate(self):
try:
os.unlink(os.path.join(self._rootdir, '.hg', 'fsmonitor.state'))
except OSError as inst:
if inst.errno != errno.ENOENT:
raise
Siddharth Agarwal
fsmonitor: don't write out state if identity has changed (issue5581)...
r32816 self._identity = util.filestat(None)
Martijn Pieters
fsmonitor: new experimental extension...
r28433
def setlastclock(self, clock):
self._lastclock = clock
def getlastclock(self):
return self._lastclock