##// END OF EJS Templates
context: write dirstate out explicitly after marking files as clean...
context: write dirstate out explicitly after marking files as clean To detect change of a file without redundant comparison of file content, dirstate recognizes a file as certainly clean, if: (1) it is already known as "normal", (2) dirstate entry for it has valid (= not "-1") timestamp, and (3) mode, size and timestamp of it on the filesystem are as same as ones expected in dirstate This works as expected in many cases, but doesn't in the corner case that changing a file keeps mode, size and timestamp of it on the filesystem. The timetable below shows steps in one of typical such situations: ---- ----------------------------------- ---------------- timestamp of "f" ---------------- dirstate file- time action mem file system ---- ----------------------------------- ---- ----- ----- N -1 *** - make file "f" clean N - execute 'hg foobar' - instantiate 'dirstate' -1 -1 - 'dirstate.normal("f")' N -1 (e.g. via dirty check) - change "f", but keep size N N+1 - release wlock - 'dirstate.write()' N N - 'hg status' shows "f" as "clean" N N N ---- ----------------------------------- ---- ----- ----- The most important point is that 'dirstate.write()' is executed at N+1 or later. This causes writing dirstate timestamp N of "f" out successfully. If it is executed at N, 'parsers.pack_dirstate()' replaces timestamp N with "-1" before actual writing dirstate out. Occasional test failure for unexpected file status is typical example of this corner case. Batch execution with small working directory is finished in no time, and rarely satisfies condition (2) above. This issue can occur in cases below; - 'hg revert --rev REV' for revisions other than the parent - failure of 'merge.update()' before 'merge.recordupdates()' The root cause of this issue is that files are changed without flushing in-memory dirstate changes via 'repo.commit()' (even though omitting 'dirstate.normallookup()' on changed files also causes this issue). To detect changes of files correctly, this patch writes in-memory dirstate changes out explicitly after marking files as clean in 'workingctx._checklookup()', which is invoked via 'repo.status()'. After this change, timetable is changed as below: ---- ----------------------------------- ---------------- timestamp of "f" ---------------- dirstate file- time action mem file system ---- ----------------------------------- ---- ----- ----- N -1 *** - make file "f" clean N - execute 'hg foobar' - instantiate 'dirstate' -1 -1 - 'dirstate.normal("f")' N -1 (e.g. via dirty check) ----------------------------------- ---- ----- ----- - 'dirsttate.write()' -1 -1 ----------------------------------- ---- ----- ----- - change "f", but keep size N N+1 - release wlock - 'dirstate.write()' -1 -1 - 'hg status' -1 -1 N ---- ----------------------------------- ---- ----- ----- To reproduce this issue in tests certainly, this patch emulates some timing critical actions as below: - timestamp of "f" in '.hg/dirstate' is -1 at the beginning 'hg debugrebuildstate' before command invocation ensures it. - make file "f" clean at N - change "f" at N 'touch -t 200001010000' before and after command invocation changes mtime of "f" to "2000-01-01 00:00" (= N). - invoke 'dirstate.write()' via 'repo.status()' at N 'fakedirstatewritetime.py' forces 'pack_dirstate()' to use "2000-01-01 00:00" as "now", only if 'pack_dirstate()' is invoked via 'workingctx._checklookup()'. - invoke 'dirstate.write()' via releasing wlock at N+1 (or "not at N") 'pack_dirstate()' via releasing wlock uses actual timestamp at runtime as "now", and it should be different from the "2000-01-01 00:00" of "f". BTW, this patch also changes 'test-largefiles-misc.t', because adding 'dirstate.write()' makes recent dirstate changes visible to external process.

File last commit:

r25186:80c5b266 default
r25753:fe03f522 default
Show More
win32text.py
176 lines | 5.8 KiB | text/x-python | PythonLexer
OHASHI Hideya
win32text: Add macencode/macdecode
r6481 # win32text.py - LF <-> CRLF/CR translation utilities for Windows/Mac users
Jesse Glick
Issue 882: add standard hook to reject text files with CRLF....
r5675 #
Martin Geisler
win32text: add copyright header
r8253 # Copyright 2005, 2007-2009 Matt Mackall <mpm@selenic.com> and others
#
# This software may be used and distributed according to the terms of the
Matt Mackall
Update license to GPLv2+
r10263 # GNU General Public License version 2 or any later version.
Dirkjan Ochtman
help: add/fix docstrings for a bunch of extensions
r8873
Matt Mackall
extensions: mark win32text deprecated...
r20624 '''perform automatic newline conversion (DEPRECATED)
Dirkjan Ochtman
help: add/fix docstrings for a bunch of extensions
r8873
Martin Geisler
win32text: mark this extension as deprecated
r11269 Deprecation: The win32text extension requires each user to configure
the extension again and again for each clone since the configuration
is not copied when cloning.
We have therefore made the ``eol`` as an alternative. The ``eol``
uses a version controlled file for its configuration and each clone
will therefore use the right settings from the start.
Martin Geisler
win32text: use reST syntax for literal blocks
r9217 To perform automatic newline conversion, use::
Dirkjan Ochtman
help: add/fix docstrings for a bunch of extensions
r8873
Martin Geisler
win32text: use reST syntax for literal blocks
r9217 [extensions]
Martin Geisler
hgext: enable extensions without "hgext." prefix in help texts
r10112 win32text =
Martin Geisler
win32text: use reST syntax for literal blocks
r9217 [encode]
** = cleverencode:
# or ** = macencode:
Dirkjan Ochtman
help: add/fix docstrings for a bunch of extensions
r8873
Martin Geisler
win32text: use reST syntax for literal blocks
r9217 [decode]
** = cleverdecode:
# or ** = macdecode:
Dirkjan Ochtman
help: add/fix docstrings for a bunch of extensions
r8873
Martin Geisler
win32text: use reST syntax for literal blocks
r9217 If not doing conversion, to make sure you do not commit CRLF/CR by accident::
Dirkjan Ochtman
help: add/fix docstrings for a bunch of extensions
r8873
Martin Geisler
win32text: use reST syntax for literal blocks
r9217 [hooks]
pretxncommit.crlf = python:hgext.win32text.forbidcrlf
# or pretxncommit.cr = python:hgext.win32text.forbidcr
Dirkjan Ochtman
help: add/fix docstrings for a bunch of extensions
r8873
To do the same check on a server to prevent CRLF/CR from being
Martin Geisler
win32text: use reST syntax for literal blocks
r9217 pushed or pulled::
Dirkjan Ochtman
help: add/fix docstrings for a bunch of extensions
r8873
Martin Geisler
win32text: use reST syntax for literal blocks
r9217 [hooks]
pretxnchangegroup.crlf = python:hgext.win32text.forbidcrlf
# or pretxnchangegroup.cr = python:hgext.win32text.forbidcr
Dirkjan Ochtman
help: add/fix docstrings for a bunch of extensions
r8873 '''
Jesse Glick
Issue 882: add standard hook to reject text files with CRLF....
r5675
Martin Geisler
i18n: import _ instead of gettext
r7225 from mercurial.i18n import _
Peter Arrenbrecht
cleanup: drop unused imports
r7873 from mercurial.node import short
Dirkjan Ochtman
win32text: actually import util so util.binary() can be used
r6510 from mercurial import util
Lee Cantey
Correct inadvertent line ending change.
r4859 import re
Augie Fackler
extensions: document that `testedwith = 'internal'` is special...
r25186 # Note for extension authors: ONLY specify testedwith = 'internal' for
# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
# be specifying the version(s) of Mercurial they are tested with, or
# leave the attribute unspecified.
Augie Fackler
hgext: mark all first-party extensions as such
r16743 testedwith = 'internal'
Lee Cantey
Correct inadvertent line ending change.
r4859 # regexp for single LF without CR preceding.
re_single_lf = re.compile('(^|[^\r])\n', re.MULTILINE)
OHASHI Hideya
win32text: Add macencode/macdecode
r6481 newlinestr = {'\r\n': 'CRLF', '\r': 'CR'}
filterstr = {'\r\n': 'clever', '\r': 'mac'}
def checknewline(s, newline, ui=None, repo=None, filename=None):
# warn if already has 'newline' in repository.
Lee Cantey
Correct inadvertent line ending change.
r4859 # it might cause unexpected eol conversion.
# see issue 302:
Dirkjan Ochtman
change wiki/bts URLs to point to new hostname
r8936 # http://mercurial.selenic.com/bts/issue302
OHASHI Hideya
win32text: Add macencode/macdecode
r6481 if newline in s and ui and filename and repo:
ui.warn(_('WARNING: %s already has %s line endings\n'
Jesse Glick
Provide better context for custom Python encode/decode filters....
r5967 'and does not need EOL conversion by the win32text plugin.\n'
'Before your next commit, please reconsider your '
'encode/decode settings in \nMercurial.ini or %s.\n') %
OHASHI Hideya
win32text: Add macencode/macdecode
r6481 (filename, newlinestr[newline], repo.join('hgrc')))
def dumbdecode(s, cmd, **kwargs):
checknewline(s, '\r\n', **kwargs)
Lee Cantey
Correct inadvertent line ending change.
r4859 # replace single LF to CRLF
return re_single_lf.sub('\\1\r\n', s)
def dumbencode(s, cmd):
return s.replace('\r\n', '\n')
OHASHI Hideya
win32text: Add macencode/macdecode
r6481 def macdumbdecode(s, cmd, **kwargs):
checknewline(s, '\r', **kwargs)
return s.replace('\n', '\r')
def macdumbencode(s, cmd):
return s.replace('\r', '\n')
Jesse Glick
Provide better context for custom Python encode/decode filters....
r5967 def cleverdecode(s, cmd, **kwargs):
Bryan O'Sullivan
Get rid of reimplementations of util.binary
r6508 if not util.binary(s):
Patrick Mezard
Backed out changeset 7f4257b5cbfc...
r6473 return dumbdecode(s, cmd, **kwargs)
return s
Lee Cantey
Correct inadvertent line ending change.
r4859
def cleverencode(s, cmd):
Bryan O'Sullivan
Get rid of reimplementations of util.binary
r6508 if not util.binary(s):
Patrick Mezard
Backed out changeset 7f4257b5cbfc...
r6473 return dumbencode(s, cmd)
return s
Lee Cantey
Correct inadvertent line ending change.
r4859
OHASHI Hideya
win32text: Add macencode/macdecode
r6481 def macdecode(s, cmd, **kwargs):
Bryan O'Sullivan
Get rid of reimplementations of util.binary
r6508 if not util.binary(s):
Patrick Mezard
Merge with macencode branch
r6484 return macdumbdecode(s, cmd, **kwargs)
return s
OHASHI Hideya
win32text: Add macencode/macdecode
r6481
def macencode(s, cmd):
Bryan O'Sullivan
Get rid of reimplementations of util.binary
r6508 if not util.binary(s):
Patrick Mezard
Merge with macencode branch
r6484 return macdumbencode(s, cmd)
return s
OHASHI Hideya
win32text: Add macencode/macdecode
r6481
Patrick Mezard
Register data filters in a localrepo instead of util...
r5966 _filters = {
Lee Cantey
Correct inadvertent line ending change.
r4859 'dumbdecode:': dumbdecode,
'dumbencode:': dumbencode,
'cleverdecode:': cleverdecode,
'cleverencode:': cleverencode,
OHASHI Hideya
win32text: Add macencode/macdecode
r6481 'macdumbdecode:': macdumbdecode,
'macdumbencode:': macdumbencode,
'macdecode:': macdecode,
'macencode:': macencode,
Patrick Mezard
Register data filters in a localrepo instead of util...
r5966 }
Jesse Glick
Issue 882: add standard hook to reject text files with CRLF....
r5675
Patrick Mezard
win32text: rename forbidcrlforcr() function
r6483 def forbidnewline(ui, repo, hooktype, node, newline, **kwargs):
Jesse Glick
Issue 882: add standard hook to reject text files with CRLF....
r5675 halt = False
Martin Geisler
util: use built-in set and frozenset...
r8150 seen = set()
Bryan O'Sullivan
win32text: be more careful about rejecting violating changesets...
r8147 # we try to walk changesets in reverse order from newest to
# oldest, so that if we see a file multiple times, we take the
# newest version as canonical. this prevents us from blocking a
# changegroup that contains an unacceptable commit followed later
# by a commit that fixes the problem.
tip = repo['tip']
Mads Kiilerich
check-code: there must also be whitespace between ')' and operator...
r18054 for rev in xrange(len(repo) - 1, repo[node].rev() - 1, -1):
Matt Mackall
use repo[changeid] to get a changectx
r6747 c = repo[rev]
Jesse Glick
Issue 882: add standard hook to reject text files with CRLF....
r5675 for f in c.files():
Bryan O'Sullivan
win32text: be more careful about rejecting violating changesets...
r8147 if f in seen or f not in tip or f not in c:
Jesse Glick
Issue 882: add standard hook to reject text files with CRLF....
r5675 continue
Bryan O'Sullivan
win32text: be more careful about rejecting violating changesets...
r8147 seen.add(f)
Jesse Glick
Issue 882: add standard hook to reject text files with CRLF....
r5675 data = c[f].data()
Bryan O'Sullivan
Get rid of reimplementations of util.binary
r6508 if not util.binary(data) and newline in data:
Jesse Glick
Issue 882: add standard hook to reject text files with CRLF....
r5675 if not halt:
Martin Geisler
win32text: lowercase warning message
r16932 ui.warn(_('attempt to commit or push text file(s) '
OHASHI Hideya
win32text: Add macencode/macdecode
r6481 'using %s line endings\n') %
newlinestr[newline])
Jesse Glick
Issue 882: add standard hook to reject text files with CRLF....
r5675 ui.warn(_('in %s: %s\n') % (short(c.node()), f))
halt = True
if halt and hooktype == 'pretxnchangegroup':
OHASHI Hideya
win32text: Add macencode/macdecode
r6481 crlf = newlinestr[newline].lower()
filter = filterstr[newline]
Jesse Glick
Issue 882: add standard hook to reject text files with CRLF....
r5675 ui.warn(_('\nTo prevent this mistake in your local repository,\n'
'add to Mercurial.ini or .hg/hgrc:\n'
'\n'
'[hooks]\n'
OHASHI Hideya
win32text: Add macencode/macdecode
r6481 'pretxncommit.%s = python:hgext.win32text.forbid%s\n'
Jesse Glick
Issue 882: add standard hook to reject text files with CRLF....
r5675 '\n'
'and also consider adding:\n'
'\n'
'[extensions]\n'
Martin Geisler
win32text: do not mention hgext in warning
r10132 'win32text =\n'
Jesse Glick
Issue 882: add standard hook to reject text files with CRLF....
r5675 '[encode]\n'
OHASHI Hideya
win32text: Add macencode/macdecode
r6481 '** = %sencode:\n'
Jesse Glick
Issue 882: add standard hook to reject text files with CRLF....
r5675 '[decode]\n'
OHASHI Hideya
win32text: Add macencode/macdecode
r6481 '** = %sdecode:\n') % (crlf, crlf, filter, filter))
Jesse Glick
Issue 882: add standard hook to reject text files with CRLF....
r5675 return halt
Patrick Mezard
Register data filters in a localrepo instead of util...
r5966
OHASHI Hideya
win32text: Add macencode/macdecode
r6481 def forbidcrlf(ui, repo, hooktype, node, **kwargs):
Patrick Mezard
win32text: rename forbidcrlforcr() function
r6483 return forbidnewline(ui, repo, hooktype, node, '\r\n', **kwargs)
OHASHI Hideya
win32text: Add macencode/macdecode
r6481
def forbidcr(ui, repo, hooktype, node, **kwargs):
Patrick Mezard
win32text: rename forbidcrlforcr() function
r6483 return forbidnewline(ui, repo, hooktype, node, '\r', **kwargs)
OHASHI Hideya
win32text: Add macencode/macdecode
r6481
Patrick Mezard
Register data filters in a localrepo instead of util...
r5966 def reposetup(ui, repo):
if not repo.local():
return
for name, fn in _filters.iteritems():
repo.adddatafilter(name, fn)
Steve Borho
win32text: give deprecation warning...
r12837 def extsetup(ui):
if ui.configbool('win32text', 'warn', True):
ui.warn(_("win32text is deprecated: "
"http://mercurial.selenic.com/wiki/Win32TextExtension\n"))