##// END OF EJS Templates
lfs: verify lfs object content when transferring to and from the remote store...
lfs: verify lfs object content when transferring to and from the remote store This avoids inserting corrupt files into the usercache, and local and remote stores. One down side is that the bad file won't be available locally for forensic purposes after a remote download. I'm thinking about adding an 'incoming' directory to the local lfs store to handle the download, and then move it to the 'objects' directory after it passes verification. That would have the additional benefit of not concatenating each transfer chunk in memory until the full file is transferred. Verification isn't needed when the data is passed back through the revlog interface or when the oid was just calculated, but otherwise it is on by default. The additional overhead should be well worth avoiding problems with file based remote stores, or buggy lfs servers. Having two different verify functions is a little sad, but the full data of the blob is mostly passed around in memory, because that's what the revlog interface wants. The upload function, however, chunks up the data. It would be ideal if that was how the content is always handled, but that's probably a huge project. I don't really like printing the long hash, but `hg debugdata` isn't a public interface, and is the only way to get it. The filelog and revision info is nowhere near this area, so recommending `hg verify` is the easiest thing to do.

File last commit:

r29169:c9ab5a0b default
r35492:417e8e04 default
Show More
docchecker
66 lines | 1.6 KiB | text/plain | TextLexer
timeless
docchecker: introduce a way to check for poor markup...
r27730 #!/usr/bin/env python
#
# docchecker - look for problematic markup
#
# Copyright 2016 timeless <timeless@mozdev.org> and others
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
Pulkit Goyal
py3: make doc/docchecker use absolute_import
r29168
Pulkit Goyal
py3: make doc/docchecker use print_function
r29169 from __future__ import absolute_import, print_function
Pulkit Goyal
py3: make doc/docchecker use absolute_import
r29168
import re
timeless
docchecker: introduce a way to check for poor markup...
r27730 import sys
leadingline = re.compile(r'(^\s*)(\S.*)$')
timeless
docchecker: report context line at most once
r28810
checks = [
(r""":hg:`[^`]*'[^`]*`""",
"""warning: please avoid nesting ' in :hg:`...`"""),
(r'\w:hg:`',
'warning: please have a space before :hg:'),
timeless
docchecker: try to reject single quotes
r28811 (r"""(?:[^a-z][^'.])hg ([^,;"`]*'(?!hg)){2}""",
'''warning: please use " instead of ' for hg ... "..."'''),
timeless
docchecker: report context line at most once
r28810 ]
timeless
docchecker: introduce a way to check for poor markup...
r27730
def check(line):
timeless
docchecker: report context line at most once
r28810 messages = []
for match, msg in checks:
if re.search(match, line):
messages.append(msg)
if messages:
FUJIWARA Katsunori
docchecker: use indentation of 4 spaces...
r28049 print(line)
timeless
docchecker: report context line at most once
r28810 for msg in messages:
print(msg)
timeless
docchecker: introduce a way to check for poor markup...
r27730
def work(file):
FUJIWARA Katsunori
docchecker: use indentation of 4 spaces...
r28049 (llead, lline) = ('', '')
timeless
docchecker: introduce a way to check for poor markup...
r27730
FUJIWARA Katsunori
docchecker: use indentation of 4 spaces...
r28049 for line in file:
# this section unwraps lines
match = leadingline.match(line)
if not match:
check(lline)
(llead, lline) = ('', '')
continue
timeless
docchecker: introduce a way to check for poor markup...
r27730
FUJIWARA Katsunori
docchecker: use indentation of 4 spaces...
r28049 lead, line = match.group(1), match.group(2)
if (lead == llead):
if (lline != ''):
lline += ' ' + line
else:
lline = line
else:
check(lline)
(llead, lline) = (lead, line)
check(lline)
timeless
docchecker: introduce a way to check for poor markup...
r27730
def main():
FUJIWARA Katsunori
docchecker: use indentation of 4 spaces...
r28049 for f in sys.argv[1:]:
try:
with open(f) as file:
work(file)
except BaseException as e:
print("failed to process %s: %s" % (f, e))
timeless
docchecker: introduce a way to check for poor markup...
r27730
main()