##// END OF EJS Templates
exchangev2: fetch manifest revisions...
exchangev2: fetch manifest revisions Now that the server has support for retrieving manifest data, we can implement the client bits to call it. We teach the changeset fetching code to capture the manifest revisions that are encountered on incoming changesets. We then feed this into a new function which filters out known manifests and then batches up manifest data requests to the server. This is different from the previous wire protocol in a few notable ways. First, the client fetches manifest data separately and explicitly. Before, we'd ask the server for data pertaining to some changesets (via a "getbundle" command) and manifests (and files) would be sent automatically. Providing an API for looking up just manifest data separately gives clients much more flexibility for manifest management. For example, a client may choose to only fetch manifest data on demand instead of prefetching it (i.e. partial clone). Second, we send N commands to the server for manifest retrieval instead of 1. This property has a few nice side-effects. One is that the deterministic nature of the requests lends itself to server-side caching. For example, say the remote has 50,000 manifests. If the server is configured to cache responses, each time a new commit arrives, you will have a cache miss and need to regenerate all outgoing data. But if you makes N requests requesting 10,000 manifests each, a new commit will still yield cache hits on the initial, unchanged manifest batches/requests. A derived benefit from these properties is that resumable clone is conceptually simpler to implement. When making a monolithic request for all of the repository data, recovering from an interrupted clone is hard because the server was in the driver's seat and was maintaining state about all the data that needed transferred. With the client driving fetching, the client can persist the set of unfetched entities and retry/resume a fetch if something goes wrong. Or we can fetch all data N changesets at a time and slowly build up a repository. This approach is drastically easier to implement when we have server APIs exposing low-level repository primitives (such as manifests and files). We don't yet support tree manifests. But it should be possible to implement that with the existing wire protocol command. Differential Revision: https://phab.mercurial-scm.org/D4489

File last commit:

r39482:5bfab940 default
r39674:d292328e default
Show More
state.py
87 lines | 2.8 KiB | text/x-python | PythonLexer
# state.py - writing and reading state files in Mercurial
#
# Copyright 2018 Pulkit Goyal <pulkitmgoyal@gmail.com>
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
"""
This file contains class to wrap the state for commands and other
related logic.
All the data related to the command state is stored as dictionary in the object.
The class has methods using which the data can be stored to disk in a file under
.hg/ directory.
We store the data on disk in cbor, for which we use the third party cbor library
to serialize and deserialize data.
"""
from __future__ import absolute_import
from . import (
error,
util,
)
from .utils import (
cborutil,
)
class cmdstate(object):
"""a wrapper class to store the state of commands like `rebase`, `graft`,
`histedit`, `shelve` etc. Extensions can also use this to write state files.
All the data for the state is stored in the form of key-value pairs in a
dictionary.
The class object can write all the data to a file in .hg/ directory and
can populate the object data reading that file.
Uses cbor to serialize and deserialize data while writing and reading from
disk.
"""
def __init__(self, repo, fname):
""" repo is the repo object
fname is the file name in which data should be stored in .hg directory
"""
self._repo = repo
self.fname = fname
def read(self):
"""read the existing state file and return a dict of data stored"""
return self._read()
def save(self, version, data):
"""write all the state data stored to .hg/<filename> file
we use third-party library cbor to serialize data to write in the file.
"""
if not isinstance(version, int):
raise error.ProgrammingError("version of state file should be"
" an integer")
with self._repo.vfs(self.fname, 'wb', atomictemp=True) as fp:
fp.write('%d\n' % version)
for chunk in cborutil.streamencode(data):
fp.write(chunk)
def _read(self):
"""reads the state file and returns a dictionary which contain
data in the same format as it was before storing"""
with self._repo.vfs(self.fname, 'rb') as fp:
try:
int(fp.readline())
except ValueError:
raise error.CorruptedState("unknown version of state file"
" found")
return cborutil.decodeall(fp.read())[0]
def delete(self):
"""drop the state file if exists"""
util.unlinkpath(self._repo.vfs.join(self.fname), ignoremissing=True)
def exists(self):
"""check whether the state file exists or not"""
return self._repo.vfs.exists(self.fname)