##// END OF EJS Templates
context: write dirstate out explicitly after marking files as clean...
context: write dirstate out explicitly after marking files as clean To detect change of a file without redundant comparison of file content, dirstate recognizes a file as certainly clean, if: (1) it is already known as "normal", (2) dirstate entry for it has valid (= not "-1") timestamp, and (3) mode, size and timestamp of it on the filesystem are as same as ones expected in dirstate This works as expected in many cases, but doesn't in the corner case that changing a file keeps mode, size and timestamp of it on the filesystem. The timetable below shows steps in one of typical such situations: ---- ----------------------------------- ---------------- timestamp of "f" ---------------- dirstate file- time action mem file system ---- ----------------------------------- ---- ----- ----- N -1 *** - make file "f" clean N - execute 'hg foobar' - instantiate 'dirstate' -1 -1 - 'dirstate.normal("f")' N -1 (e.g. via dirty check) - change "f", but keep size N N+1 - release wlock - 'dirstate.write()' N N - 'hg status' shows "f" as "clean" N N N ---- ----------------------------------- ---- ----- ----- The most important point is that 'dirstate.write()' is executed at N+1 or later. This causes writing dirstate timestamp N of "f" out successfully. If it is executed at N, 'parsers.pack_dirstate()' replaces timestamp N with "-1" before actual writing dirstate out. Occasional test failure for unexpected file status is typical example of this corner case. Batch execution with small working directory is finished in no time, and rarely satisfies condition (2) above. This issue can occur in cases below; - 'hg revert --rev REV' for revisions other than the parent - failure of 'merge.update()' before 'merge.recordupdates()' The root cause of this issue is that files are changed without flushing in-memory dirstate changes via 'repo.commit()' (even though omitting 'dirstate.normallookup()' on changed files also causes this issue). To detect changes of files correctly, this patch writes in-memory dirstate changes out explicitly after marking files as clean in 'workingctx._checklookup()', which is invoked via 'repo.status()'. After this change, timetable is changed as below: ---- ----------------------------------- ---------------- timestamp of "f" ---------------- dirstate file- time action mem file system ---- ----------------------------------- ---- ----- ----- N -1 *** - make file "f" clean N - execute 'hg foobar' - instantiate 'dirstate' -1 -1 - 'dirstate.normal("f")' N -1 (e.g. via dirty check) ----------------------------------- ---- ----- ----- - 'dirsttate.write()' -1 -1 ----------------------------------- ---- ----- ----- - change "f", but keep size N N+1 - release wlock - 'dirstate.write()' -1 -1 - 'hg status' -1 -1 N ---- ----------------------------------- ---- ----- ----- To reproduce this issue in tests certainly, this patch emulates some timing critical actions as below: - timestamp of "f" in '.hg/dirstate' is -1 at the beginning 'hg debugrebuildstate' before command invocation ensures it. - make file "f" clean at N - change "f" at N 'touch -t 200001010000' before and after command invocation changes mtime of "f" to "2000-01-01 00:00" (= N). - invoke 'dirstate.write()' via 'repo.status()' at N 'fakedirstatewritetime.py' forces 'pack_dirstate()' to use "2000-01-01 00:00" as "now", only if 'pack_dirstate()' is invoked via 'workingctx._checklookup()'. - invoke 'dirstate.write()' via releasing wlock at N+1 (or "not at N") 'pack_dirstate()' via releasing wlock uses actual timestamp at runtime as "now", and it should be different from the "2000-01-01 00:00" of "f". BTW, this patch also changes 'test-largefiles-misc.t', because adding 'dirstate.write()' makes recent dirstate changes visible to external process.

File last commit:

r25513:0c6f9839 default
r25753:fe03f522 default
Show More
formatter.py
204 lines | 6.5 KiB | text/x-python | PythonLexer
Matt Mackall
formatter: add basic formatters
r16134 # formatter.py - generic output formatting for mercurial
#
# Copyright 2012 Matt Mackall <mpm@selenic.com>
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
Matt Mackall
formatter: add pickle format...
r22430 import cPickle
Yuya Nishihara
formatter: add general way to switch hex/short functions...
r22701 from node import hex, short
Matt Mackall
formatter: add json formatter
r22428 from i18n import _
import encoding, util
Matt Mackall
formatter: move most of template option helper to formatter...
r25511 import templater
import os
Matt Mackall
formatter: add json formatter
r22428
Matt Mackall
formatter: add basic formatters
r16134 class baseformatter(object):
def __init__(self, ui, topic, opts):
self._ui = ui
self._topic = topic
self._style = opts.get("style")
self._template = opts.get("template")
self._item = None
Yuya Nishihara
formatter: add general way to switch hex/short functions...
r22701 # function to convert node to string suitable for this output
self.hexfunc = hex
Yuya Nishihara
formatter: correct bool testing which should be __nonzero__ in Python 2
r22447 def __nonzero__(self):
Matt Mackall
formatter: add basic formatters
r16134 '''return False if we're not doing real templating so we can
skip extra work'''
return True
def _showitem(self):
'''show a formatted item once all data is collected'''
pass
def startitem(self):
'''begin an item in the format list'''
if self._item is not None:
self._showitem()
self._item = {}
def data(self, **data):
'''insert data into item that's not shown in default output'''
David M. Carr
formatter: improve implementation of data method...
r17630 self._item.update(data)
Matt Mackall
formatter: add basic formatters
r16134 def write(self, fields, deftext, *fielddata, **opts):
'''do default text output while assigning data to item'''
for k, v in zip(fields.split(), fielddata):
self._item[k] = v
Matt Mackall
formatter: add condwrite method...
r17909 def condwrite(self, cond, fields, deftext, *fielddata, **opts):
'''do conditional write (primarily for plain formatter)'''
for k, v in zip(fields.split(), fielddata):
self._item[k] = v
Matt Mackall
formatter: add basic formatters
r16134 def plain(self, text, **opts):
'''show raw text for non-templated mode'''
pass
def end(self):
'''end output for the formatter'''
if self._item is not None:
self._showitem()
class plainformatter(baseformatter):
'''the default text output scheme'''
def __init__(self, ui, topic, opts):
baseformatter.__init__(self, ui, topic, opts)
Yuya Nishihara
formatter: add general way to switch hex/short functions...
r22701 if ui.debugflag:
self.hexfunc = hex
else:
self.hexfunc = short
Yuya Nishihara
formatter: correct bool testing which should be __nonzero__ in Python 2
r22447 def __nonzero__(self):
Matt Mackall
formatter: add basic formatters
r16134 return False
def startitem(self):
pass
def data(self, **data):
pass
def write(self, fields, deftext, *fielddata, **opts):
self._ui.write(deftext % fielddata, **opts)
Matt Mackall
formatter: add condwrite method...
r17909 def condwrite(self, cond, fields, deftext, *fielddata, **opts):
'''do conditional write'''
if cond:
self._ui.write(deftext % fielddata, **opts)
Matt Mackall
formatter: add basic formatters
r16134 def plain(self, text, **opts):
self._ui.write(text, **opts)
def end(self):
pass
class debugformatter(baseformatter):
def __init__(self, ui, topic, opts):
baseformatter.__init__(self, ui, topic, opts)
Matt Mackall
formatter: make debug style match Python syntax
r22424 self._ui.write("%s = [\n" % self._topic)
Matt Mackall
formatter: add basic formatters
r16134 def _showitem(self):
self._ui.write(" " + repr(self._item) + ",\n")
def end(self):
baseformatter.end(self)
Matt Mackall
formatter: make debug style match Python syntax
r22424 self._ui.write("]\n")
Matt Mackall
formatter: add basic formatters
r16134
Matt Mackall
formatter: add pickle format...
r22430 class pickleformatter(baseformatter):
def __init__(self, ui, topic, opts):
baseformatter.__init__(self, ui, topic, opts)
self._data = []
def _showitem(self):
self._data.append(self._item)
def end(self):
baseformatter.end(self)
self._ui.write(cPickle.dumps(self._data))
Yuya Nishihara
formatter: extract function that encode values to json string...
r22474 def _jsonifyobj(v):
Yuya Nishihara
formatter: have jsonformatter accept tuple as value...
r22475 if isinstance(v, tuple):
return '[' + ', '.join(_jsonifyobj(e) for e in v) + ']'
Yuya Nishihara
formatter: convert None to json null...
r24321 elif v is None:
return 'null'
Yuya Nishihara
formatter: convert booleans to json...
r22674 elif v is True:
return 'true'
elif v is False:
return 'false'
Yuya Nishihara
formatter: convert float value to json...
r22476 elif isinstance(v, (int, float)):
return str(v)
Yuya Nishihara
formatter: extract function that encode values to json string...
r22474 else:
return '"%s"' % encoding.jsonescape(v)
Matt Mackall
formatter: add json formatter
r22428 class jsonformatter(baseformatter):
def __init__(self, ui, topic, opts):
baseformatter.__init__(self, ui, topic, opts)
self._ui.write("[")
self._ui._first = True
def _showitem(self):
if self._ui._first:
self._ui._first = False
else:
self._ui.write(",")
self._ui.write("\n {\n")
first = True
for k, v in sorted(self._item.items()):
if first:
first = False
else:
self._ui.write(",\n")
Yuya Nishihara
formatter: extract function that encode values to json string...
r22474 self._ui.write(' "%s": %s' % (k, _jsonifyobj(v)))
Matt Mackall
formatter: add json formatter
r22428 self._ui.write("\n }")
def end(self):
baseformatter.end(self)
self._ui.write("\n]\n")
Matt Mackall
formatter: add template support...
r25513 class templateformatter(baseformatter):
def __init__(self, ui, topic, opts):
baseformatter.__init__(self, ui, topic, opts)
self._topic = topic
self._t = gettemplater(ui, topic, opts.get('template', ''))
def _showitem(self):
g = self._t(self._topic, **self._item)
self._ui.write(templater.stringify(g))
Matt Mackall
formatter: move most of template option helper to formatter...
r25511 def lookuptemplate(ui, topic, tmpl):
# looks like a literal template?
if '{' in tmpl:
return tmpl, None
# perhaps a stock style?
if not os.path.split(tmpl)[0]:
mapname = (templater.templatepath('map-cmdline.' + tmpl)
or templater.templatepath(tmpl))
if mapname and os.path.isfile(mapname):
return None, mapname
# perhaps it's a reference to [templates]
t = ui.config('templates', tmpl)
if t:
try:
tmpl = templater.unquotestring(t)
except SyntaxError:
tmpl = t
return tmpl, None
if tmpl == 'list':
ui.write(_("available styles: %s\n") % templater.stylelist())
raise util.Abort(_("specify a template"))
# perhaps it's a path to a map or a template
if ('/' in tmpl or '\\' in tmpl) and os.path.isfile(tmpl):
# is it a mapfile for a style?
if os.path.basename(tmpl).startswith("map-"):
return None, os.path.realpath(tmpl)
tmpl = open(tmpl).read()
return tmpl, None
# constant string?
return tmpl, None
Matt Mackall
formatter: add a method to build a full templater from a -T option
r25512 def gettemplater(ui, topic, spec):
tmpl, mapfile = lookuptemplate(ui, topic, spec)
t = templater.templater(mapfile, {})
if tmpl:
t.cache[topic] = tmpl
return t
Matt Mackall
formatter: add basic formatters
r16134 def formatter(ui, topic, opts):
Matt Mackall
formatter: add json formatter
r22428 template = opts.get("template", "")
if template == "json":
return jsonformatter(ui, topic, opts)
Matt Mackall
formatter: add pickle format...
r22430 elif template == "pickle":
return pickleformatter(ui, topic, opts)
Matt Mackall
formatter: add json formatter
r22428 elif template == "debug":
Matt Mackall
formatter: add basic formatters
r16134 return debugformatter(ui, topic, opts)
Matt Mackall
formatter: add json formatter
r22428 elif template != "":
Matt Mackall
formatter: add template support...
r25513 return templateformatter(ui, topic, opts)
Matt Mackall
formatter: add json formatter
r22428 elif ui.configbool('ui', 'formatdebug'):
return debugformatter(ui, topic, opts)
elif ui.configbool('ui', 'formatjson'):
return jsonformatter(ui, topic, opts)
Matt Mackall
formatter: add basic formatters
r16134 return plainformatter(ui, topic, opts)