##// END OF EJS Templates
eol: fix win32text encode/decode filter names
Martin Geisler -
r12979:733345a1 default
parent child Browse files
Show More
@@ -1,264 +1,269 b''
1 """automatically manage newlines in repository files
1 """automatically manage newlines in repository files
2
2
3 This extension allows you to manage the type of line endings (CRLF or
3 This extension allows you to manage the type of line endings (CRLF or
4 LF) that are used in the repository and in the local working
4 LF) that are used in the repository and in the local working
5 directory. That way you can get CRLF line endings on Windows and LF on
5 directory. That way you can get CRLF line endings on Windows and LF on
6 Unix/Mac, thereby letting everybody use their OS native line endings.
6 Unix/Mac, thereby letting everybody use their OS native line endings.
7
7
8 The extension reads its configuration from a versioned ``.hgeol``
8 The extension reads its configuration from a versioned ``.hgeol``
9 configuration file every time you run an ``hg`` command. The
9 configuration file every time you run an ``hg`` command. The
10 ``.hgeol`` file use the same syntax as all other Mercurial
10 ``.hgeol`` file use the same syntax as all other Mercurial
11 configuration files. It uses two sections, ``[patterns]`` and
11 configuration files. It uses two sections, ``[patterns]`` and
12 ``[repository]``.
12 ``[repository]``.
13
13
14 The ``[patterns]`` section specifies the line endings used in the
14 The ``[patterns]`` section specifies the line endings used in the
15 working directory. The format is specified by a file pattern. The
15 working directory. The format is specified by a file pattern. The
16 first match is used, so put more specific patterns first. The
16 first match is used, so put more specific patterns first. The
17 available line endings are ``LF``, ``CRLF``, and ``BIN``.
17 available line endings are ``LF``, ``CRLF``, and ``BIN``.
18
18
19 Files with the declared format of ``CRLF`` or ``LF`` are always
19 Files with the declared format of ``CRLF`` or ``LF`` are always
20 checked out in that format and files declared to be binary (``BIN``)
20 checked out in that format and files declared to be binary (``BIN``)
21 are left unchanged. Additionally, ``native`` is an alias for the
21 are left unchanged. Additionally, ``native`` is an alias for the
22 platform's default line ending: ``LF`` on Unix (including Mac OS X)
22 platform's default line ending: ``LF`` on Unix (including Mac OS X)
23 and ``CRLF`` on Windows. Note that ``BIN`` (do nothing to line
23 and ``CRLF`` on Windows. Note that ``BIN`` (do nothing to line
24 endings) is Mercurial's default behaviour; it is only needed if you
24 endings) is Mercurial's default behaviour; it is only needed if you
25 need to override a later, more general pattern.
25 need to override a later, more general pattern.
26
26
27 The optional ``[repository]`` section specifies the line endings to
27 The optional ``[repository]`` section specifies the line endings to
28 use for files stored in the repository. It has a single setting,
28 use for files stored in the repository. It has a single setting,
29 ``native``, which determines the storage line endings for files
29 ``native``, which determines the storage line endings for files
30 declared as ``native`` in the ``[patterns]`` section. It can be set to
30 declared as ``native`` in the ``[patterns]`` section. It can be set to
31 ``LF`` or ``CRLF``. The default is ``LF``. For example, this means
31 ``LF`` or ``CRLF``. The default is ``LF``. For example, this means
32 that on Windows, files configured as ``native`` (``CRLF`` by default)
32 that on Windows, files configured as ``native`` (``CRLF`` by default)
33 will be converted to ``LF`` when stored in the repository. Files
33 will be converted to ``LF`` when stored in the repository. Files
34 declared as ``LF``, ``CRLF``, or ``BIN`` in the ``[patterns]`` section
34 declared as ``LF``, ``CRLF``, or ``BIN`` in the ``[patterns]`` section
35 are always stored as-is in the repository.
35 are always stored as-is in the repository.
36
36
37 Example versioned ``.hgeol`` file::
37 Example versioned ``.hgeol`` file::
38
38
39 [patterns]
39 [patterns]
40 **.py = native
40 **.py = native
41 **.vcproj = CRLF
41 **.vcproj = CRLF
42 **.txt = native
42 **.txt = native
43 Makefile = LF
43 Makefile = LF
44 **.jpg = BIN
44 **.jpg = BIN
45
45
46 [repository]
46 [repository]
47 native = LF
47 native = LF
48
48
49 The extension uses an optional ``[eol]`` section in your hgrc file
49 The extension uses an optional ``[eol]`` section in your hgrc file
50 (not the ``.hgeol`` file) for settings that control the overall
50 (not the ``.hgeol`` file) for settings that control the overall
51 behavior. There are two settings:
51 behavior. There are two settings:
52
52
53 - ``eol.native`` (default ``os.linesep``) can be set to ``LF`` or
53 - ``eol.native`` (default ``os.linesep``) can be set to ``LF`` or
54 ``CRLF`` to override the default interpretation of ``native`` for
54 ``CRLF`` to override the default interpretation of ``native`` for
55 checkout. This can be used with :hg:`archive` on Unix, say, to
55 checkout. This can be used with :hg:`archive` on Unix, say, to
56 generate an archive where files have line endings for Windows.
56 generate an archive where files have line endings for Windows.
57
57
58 - ``eol.only-consistent`` (default True) can be set to False to make
58 - ``eol.only-consistent`` (default True) can be set to False to make
59 the extension convert files with inconsistent EOLs. Inconsistent
59 the extension convert files with inconsistent EOLs. Inconsistent
60 means that there is both ``CRLF`` and ``LF`` present in the file.
60 means that there is both ``CRLF`` and ``LF`` present in the file.
61 Such files are normally not touched under the assumption that they
61 Such files are normally not touched under the assumption that they
62 have mixed EOLs on purpose.
62 have mixed EOLs on purpose.
63
63
64 The extension provides ``cleverencode:`` and ``cleverdecode:`` filters
65 like the deprecated win32text extension does. This means that you can
66 disable win32text and enable eol and your filters will still work. You
67 only need to these filters until you have prepared a ``.hgeol`` file.
68
64 See :hg:`help patterns` for more information about the glob patterns
69 See :hg:`help patterns` for more information about the glob patterns
65 used.
70 used.
66 """
71 """
67
72
68 from mercurial.i18n import _
73 from mercurial.i18n import _
69 from mercurial import util, config, extensions, match
74 from mercurial import util, config, extensions, match
70 import re, os
75 import re, os
71
76
72 # Matches a lone LF, i.e., one that is not part of CRLF.
77 # Matches a lone LF, i.e., one that is not part of CRLF.
73 singlelf = re.compile('(^|[^\r])\n')
78 singlelf = re.compile('(^|[^\r])\n')
74 # Matches a single EOL which can either be a CRLF where repeated CR
79 # Matches a single EOL which can either be a CRLF where repeated CR
75 # are removed or a LF. We do not care about old Machintosh files, so a
80 # are removed or a LF. We do not care about old Machintosh files, so a
76 # stray CR is an error.
81 # stray CR is an error.
77 eolre = re.compile('\r*\n')
82 eolre = re.compile('\r*\n')
78
83
79
84
80 def inconsistenteol(data):
85 def inconsistenteol(data):
81 return '\r\n' in data and singlelf.search(data)
86 return '\r\n' in data and singlelf.search(data)
82
87
83 def tolf(s, params, ui, **kwargs):
88 def tolf(s, params, ui, **kwargs):
84 """Filter to convert to LF EOLs."""
89 """Filter to convert to LF EOLs."""
85 if util.binary(s):
90 if util.binary(s):
86 return s
91 return s
87 if ui.configbool('eol', 'only-consistent', True) and inconsistenteol(s):
92 if ui.configbool('eol', 'only-consistent', True) and inconsistenteol(s):
88 return s
93 return s
89 return eolre.sub('\n', s)
94 return eolre.sub('\n', s)
90
95
91 def tocrlf(s, params, ui, **kwargs):
96 def tocrlf(s, params, ui, **kwargs):
92 """Filter to convert to CRLF EOLs."""
97 """Filter to convert to CRLF EOLs."""
93 if util.binary(s):
98 if util.binary(s):
94 return s
99 return s
95 if ui.configbool('eol', 'only-consistent', True) and inconsistenteol(s):
100 if ui.configbool('eol', 'only-consistent', True) and inconsistenteol(s):
96 return s
101 return s
97 return eolre.sub('\r\n', s)
102 return eolre.sub('\r\n', s)
98
103
99 def isbinary(s, params):
104 def isbinary(s, params):
100 """Filter to do nothing with the file."""
105 """Filter to do nothing with the file."""
101 return s
106 return s
102
107
103 filters = {
108 filters = {
104 'to-lf': tolf,
109 'to-lf': tolf,
105 'to-crlf': tocrlf,
110 'to-crlf': tocrlf,
106 'is-binary': isbinary,
111 'is-binary': isbinary,
107 # The following provide backwards compatibility with win32text
112 # The following provide backwards compatibility with win32text
108 'cleverencode': tolf,
113 'cleverencode:': tolf,
109 'cleverdecode': tocrlf
114 'cleverdecode:': tocrlf
110 }
115 }
111
116
112
117
113 def hook(ui, repo, node, hooktype, **kwargs):
118 def hook(ui, repo, node, hooktype, **kwargs):
114 """verify that files have expected EOLs"""
119 """verify that files have expected EOLs"""
115 files = set()
120 files = set()
116 for rev in xrange(repo[node].rev(), len(repo)):
121 for rev in xrange(repo[node].rev(), len(repo)):
117 files.update(repo[rev].files())
122 files.update(repo[rev].files())
118 tip = repo['tip']
123 tip = repo['tip']
119 for f in files:
124 for f in files:
120 if f not in tip:
125 if f not in tip:
121 continue
126 continue
122 for pattern, target in ui.configitems('encode'):
127 for pattern, target in ui.configitems('encode'):
123 if match.match(repo.root, '', [pattern])(f):
128 if match.match(repo.root, '', [pattern])(f):
124 data = tip[f].data()
129 data = tip[f].data()
125 if target == "to-lf" and "\r\n" in data:
130 if target == "to-lf" and "\r\n" in data:
126 raise util.Abort(_("%s should not have CRLF line endings")
131 raise util.Abort(_("%s should not have CRLF line endings")
127 % f)
132 % f)
128 elif target == "to-crlf" and singlelf.search(data):
133 elif target == "to-crlf" and singlelf.search(data):
129 raise util.Abort(_("%s should not have LF line endings")
134 raise util.Abort(_("%s should not have LF line endings")
130 % f)
135 % f)
131
136
132
137
133 def preupdate(ui, repo, hooktype, parent1, parent2):
138 def preupdate(ui, repo, hooktype, parent1, parent2):
134 #print "preupdate for %s: %s -> %s" % (repo.root, parent1, parent2)
139 #print "preupdate for %s: %s -> %s" % (repo.root, parent1, parent2)
135 repo.readhgeol(parent1)
140 repo.readhgeol(parent1)
136 return False
141 return False
137
142
138 def uisetup(ui):
143 def uisetup(ui):
139 ui.setconfig('hooks', 'preupdate.eol', preupdate)
144 ui.setconfig('hooks', 'preupdate.eol', preupdate)
140
145
141 def extsetup(ui):
146 def extsetup(ui):
142 try:
147 try:
143 extensions.find('win32text')
148 extensions.find('win32text')
144 raise util.Abort(_("the eol extension is incompatible with the "
149 raise util.Abort(_("the eol extension is incompatible with the "
145 "win32text extension"))
150 "win32text extension"))
146 except KeyError:
151 except KeyError:
147 pass
152 pass
148
153
149
154
150 def reposetup(ui, repo):
155 def reposetup(ui, repo):
151 uisetup(repo.ui)
156 uisetup(repo.ui)
152 #print "reposetup for", repo.root
157 #print "reposetup for", repo.root
153
158
154 if not repo.local():
159 if not repo.local():
155 return
160 return
156 for name, fn in filters.iteritems():
161 for name, fn in filters.iteritems():
157 repo.adddatafilter(name, fn)
162 repo.adddatafilter(name, fn)
158
163
159 ui.setconfig('patch', 'eol', 'auto')
164 ui.setconfig('patch', 'eol', 'auto')
160
165
161 class eolrepo(repo.__class__):
166 class eolrepo(repo.__class__):
162
167
163 _decode = {'LF': 'to-lf', 'CRLF': 'to-crlf', 'BIN': 'is-binary'}
168 _decode = {'LF': 'to-lf', 'CRLF': 'to-crlf', 'BIN': 'is-binary'}
164 _encode = {'LF': 'to-lf', 'CRLF': 'to-crlf', 'BIN': 'is-binary'}
169 _encode = {'LF': 'to-lf', 'CRLF': 'to-crlf', 'BIN': 'is-binary'}
165
170
166 def readhgeol(self, node=None, data=None):
171 def readhgeol(self, node=None, data=None):
167 if data is None:
172 if data is None:
168 try:
173 try:
169 if node is None:
174 if node is None:
170 data = self.wfile('.hgeol').read()
175 data = self.wfile('.hgeol').read()
171 else:
176 else:
172 data = self[node]['.hgeol'].data()
177 data = self[node]['.hgeol'].data()
173 except (IOError, LookupError):
178 except (IOError, LookupError):
174 return None
179 return None
175
180
176 if self.ui.config('eol', 'native', os.linesep) in ('LF', '\n'):
181 if self.ui.config('eol', 'native', os.linesep) in ('LF', '\n'):
177 self._decode['NATIVE'] = 'to-lf'
182 self._decode['NATIVE'] = 'to-lf'
178 else:
183 else:
179 self._decode['NATIVE'] = 'to-crlf'
184 self._decode['NATIVE'] = 'to-crlf'
180
185
181 eol = config.config()
186 eol = config.config()
182 # Our files should not be touched. The pattern must be
187 # Our files should not be touched. The pattern must be
183 # inserted first override a '** = native' pattern.
188 # inserted first override a '** = native' pattern.
184 eol.set('patterns', '.hg*', 'BIN')
189 eol.set('patterns', '.hg*', 'BIN')
185 # We can then parse the user's patterns.
190 # We can then parse the user's patterns.
186 eol.parse('.hgeol', data)
191 eol.parse('.hgeol', data)
187
192
188 if eol.get('repository', 'native') == 'CRLF':
193 if eol.get('repository', 'native') == 'CRLF':
189 self._encode['NATIVE'] = 'to-crlf'
194 self._encode['NATIVE'] = 'to-crlf'
190 else:
195 else:
191 self._encode['NATIVE'] = 'to-lf'
196 self._encode['NATIVE'] = 'to-lf'
192
197
193 for pattern, style in eol.items('patterns'):
198 for pattern, style in eol.items('patterns'):
194 key = style.upper()
199 key = style.upper()
195 try:
200 try:
196 self.ui.setconfig('decode', pattern, self._decode[key])
201 self.ui.setconfig('decode', pattern, self._decode[key])
197 self.ui.setconfig('encode', pattern, self._encode[key])
202 self.ui.setconfig('encode', pattern, self._encode[key])
198 except KeyError:
203 except KeyError:
199 self.ui.warn(_("ignoring unknown EOL style '%s' from %s\n")
204 self.ui.warn(_("ignoring unknown EOL style '%s' from %s\n")
200 % (style, eol.source('patterns', pattern)))
205 % (style, eol.source('patterns', pattern)))
201
206
202 include = []
207 include = []
203 exclude = []
208 exclude = []
204 for pattern, style in eol.items('patterns'):
209 for pattern, style in eol.items('patterns'):
205 key = style.upper()
210 key = style.upper()
206 if key == 'BIN':
211 if key == 'BIN':
207 exclude.append(pattern)
212 exclude.append(pattern)
208 else:
213 else:
209 include.append(pattern)
214 include.append(pattern)
210
215
211 # This will match the files for which we need to care
216 # This will match the files for which we need to care
212 # about inconsistent newlines.
217 # about inconsistent newlines.
213 return match.match(self.root, '', [], include, exclude)
218 return match.match(self.root, '', [], include, exclude)
214
219
215 def _hgcleardirstate(self):
220 def _hgcleardirstate(self):
216 self._eolfile = self.readhgeol() or self.readhgeol('tip')
221 self._eolfile = self.readhgeol() or self.readhgeol('tip')
217
222
218 if not self._eolfile:
223 if not self._eolfile:
219 self._eolfile = util.never
224 self._eolfile = util.never
220 return
225 return
221
226
222 try:
227 try:
223 cachemtime = os.path.getmtime(self.join("eol.cache"))
228 cachemtime = os.path.getmtime(self.join("eol.cache"))
224 except OSError:
229 except OSError:
225 cachemtime = 0
230 cachemtime = 0
226
231
227 try:
232 try:
228 eolmtime = os.path.getmtime(self.wjoin(".hgeol"))
233 eolmtime = os.path.getmtime(self.wjoin(".hgeol"))
229 except OSError:
234 except OSError:
230 eolmtime = 0
235 eolmtime = 0
231
236
232 if eolmtime > cachemtime:
237 if eolmtime > cachemtime:
233 ui.debug("eol: detected change in .hgeol\n")
238 ui.debug("eol: detected change in .hgeol\n")
234 # TODO: we could introduce a method for this in dirstate.
239 # TODO: we could introduce a method for this in dirstate.
235 wlock = None
240 wlock = None
236 try:
241 try:
237 wlock = self.wlock()
242 wlock = self.wlock()
238 for f, e in self.dirstate._map.iteritems():
243 for f, e in self.dirstate._map.iteritems():
239 self.dirstate._map[f] = (e[0], e[1], -1, 0)
244 self.dirstate._map[f] = (e[0], e[1], -1, 0)
240 self.dirstate._dirty = True
245 self.dirstate._dirty = True
241 # Touch the cache to update mtime. TODO: are we sure this
246 # Touch the cache to update mtime. TODO: are we sure this
242 # always enought to update the mtime, or should we write a
247 # always enought to update the mtime, or should we write a
243 # bit to the file?
248 # bit to the file?
244 self.opener("eol.cache", "w").close()
249 self.opener("eol.cache", "w").close()
245 finally:
250 finally:
246 if wlock is not None:
251 if wlock is not None:
247 wlock.release()
252 wlock.release()
248
253
249 def commitctx(self, ctx, error=False):
254 def commitctx(self, ctx, error=False):
250 for f in sorted(ctx.added() + ctx.modified()):
255 for f in sorted(ctx.added() + ctx.modified()):
251 if not self._eolfile(f):
256 if not self._eolfile(f):
252 continue
257 continue
253 data = ctx[f].data()
258 data = ctx[f].data()
254 if util.binary(data):
259 if util.binary(data):
255 # We should not abort here, since the user should
260 # We should not abort here, since the user should
256 # be able to say "** = native" to automatically
261 # be able to say "** = native" to automatically
257 # have all non-binary files taken care of.
262 # have all non-binary files taken care of.
258 continue
263 continue
259 if inconsistenteol(data):
264 if inconsistenteol(data):
260 raise util.Abort(_("inconsistent newline style "
265 raise util.Abort(_("inconsistent newline style "
261 "in %s\n" % f))
266 "in %s\n" % f))
262 return super(eolrepo, self).commitctx(ctx, error)
267 return super(eolrepo, self).commitctx(ctx, error)
263 repo.__class__ = eolrepo
268 repo.__class__ = eolrepo
264 repo._hgcleardirstate()
269 repo._hgcleardirstate()
General Comments 0
You need to be logged in to leave comments. Login now