Show More
@@ -1,264 +1,269 b'' | |||||
1 | """automatically manage newlines in repository files |
|
1 | """automatically manage newlines in repository files | |
2 |
|
2 | |||
3 | This extension allows you to manage the type of line endings (CRLF or |
|
3 | This extension allows you to manage the type of line endings (CRLF or | |
4 | LF) that are used in the repository and in the local working |
|
4 | LF) that are used in the repository and in the local working | |
5 | directory. That way you can get CRLF line endings on Windows and LF on |
|
5 | directory. That way you can get CRLF line endings on Windows and LF on | |
6 | Unix/Mac, thereby letting everybody use their OS native line endings. |
|
6 | Unix/Mac, thereby letting everybody use their OS native line endings. | |
7 |
|
7 | |||
8 | The extension reads its configuration from a versioned ``.hgeol`` |
|
8 | The extension reads its configuration from a versioned ``.hgeol`` | |
9 | configuration file every time you run an ``hg`` command. The |
|
9 | configuration file every time you run an ``hg`` command. The | |
10 | ``.hgeol`` file use the same syntax as all other Mercurial |
|
10 | ``.hgeol`` file use the same syntax as all other Mercurial | |
11 | configuration files. It uses two sections, ``[patterns]`` and |
|
11 | configuration files. It uses two sections, ``[patterns]`` and | |
12 | ``[repository]``. |
|
12 | ``[repository]``. | |
13 |
|
13 | |||
14 | The ``[patterns]`` section specifies the line endings used in the |
|
14 | The ``[patterns]`` section specifies the line endings used in the | |
15 | working directory. The format is specified by a file pattern. The |
|
15 | working directory. The format is specified by a file pattern. The | |
16 | first match is used, so put more specific patterns first. The |
|
16 | first match is used, so put more specific patterns first. The | |
17 | available line endings are ``LF``, ``CRLF``, and ``BIN``. |
|
17 | available line endings are ``LF``, ``CRLF``, and ``BIN``. | |
18 |
|
18 | |||
19 | Files with the declared format of ``CRLF`` or ``LF`` are always |
|
19 | Files with the declared format of ``CRLF`` or ``LF`` are always | |
20 | checked out in that format and files declared to be binary (``BIN``) |
|
20 | checked out in that format and files declared to be binary (``BIN``) | |
21 | are left unchanged. Additionally, ``native`` is an alias for the |
|
21 | are left unchanged. Additionally, ``native`` is an alias for the | |
22 | platform's default line ending: ``LF`` on Unix (including Mac OS X) |
|
22 | platform's default line ending: ``LF`` on Unix (including Mac OS X) | |
23 | and ``CRLF`` on Windows. Note that ``BIN`` (do nothing to line |
|
23 | and ``CRLF`` on Windows. Note that ``BIN`` (do nothing to line | |
24 | endings) is Mercurial's default behaviour; it is only needed if you |
|
24 | endings) is Mercurial's default behaviour; it is only needed if you | |
25 | need to override a later, more general pattern. |
|
25 | need to override a later, more general pattern. | |
26 |
|
26 | |||
27 | The optional ``[repository]`` section specifies the line endings to |
|
27 | The optional ``[repository]`` section specifies the line endings to | |
28 | use for files stored in the repository. It has a single setting, |
|
28 | use for files stored in the repository. It has a single setting, | |
29 | ``native``, which determines the storage line endings for files |
|
29 | ``native``, which determines the storage line endings for files | |
30 | declared as ``native`` in the ``[patterns]`` section. It can be set to |
|
30 | declared as ``native`` in the ``[patterns]`` section. It can be set to | |
31 | ``LF`` or ``CRLF``. The default is ``LF``. For example, this means |
|
31 | ``LF`` or ``CRLF``. The default is ``LF``. For example, this means | |
32 | that on Windows, files configured as ``native`` (``CRLF`` by default) |
|
32 | that on Windows, files configured as ``native`` (``CRLF`` by default) | |
33 | will be converted to ``LF`` when stored in the repository. Files |
|
33 | will be converted to ``LF`` when stored in the repository. Files | |
34 | declared as ``LF``, ``CRLF``, or ``BIN`` in the ``[patterns]`` section |
|
34 | declared as ``LF``, ``CRLF``, or ``BIN`` in the ``[patterns]`` section | |
35 | are always stored as-is in the repository. |
|
35 | are always stored as-is in the repository. | |
36 |
|
36 | |||
37 | Example versioned ``.hgeol`` file:: |
|
37 | Example versioned ``.hgeol`` file:: | |
38 |
|
38 | |||
39 | [patterns] |
|
39 | [patterns] | |
40 | **.py = native |
|
40 | **.py = native | |
41 | **.vcproj = CRLF |
|
41 | **.vcproj = CRLF | |
42 | **.txt = native |
|
42 | **.txt = native | |
43 | Makefile = LF |
|
43 | Makefile = LF | |
44 | **.jpg = BIN |
|
44 | **.jpg = BIN | |
45 |
|
45 | |||
46 | [repository] |
|
46 | [repository] | |
47 | native = LF |
|
47 | native = LF | |
48 |
|
48 | |||
49 | The extension uses an optional ``[eol]`` section in your hgrc file |
|
49 | The extension uses an optional ``[eol]`` section in your hgrc file | |
50 | (not the ``.hgeol`` file) for settings that control the overall |
|
50 | (not the ``.hgeol`` file) for settings that control the overall | |
51 | behavior. There are two settings: |
|
51 | behavior. There are two settings: | |
52 |
|
52 | |||
53 | - ``eol.native`` (default ``os.linesep``) can be set to ``LF`` or |
|
53 | - ``eol.native`` (default ``os.linesep``) can be set to ``LF`` or | |
54 | ``CRLF`` to override the default interpretation of ``native`` for |
|
54 | ``CRLF`` to override the default interpretation of ``native`` for | |
55 | checkout. This can be used with :hg:`archive` on Unix, say, to |
|
55 | checkout. This can be used with :hg:`archive` on Unix, say, to | |
56 | generate an archive where files have line endings for Windows. |
|
56 | generate an archive where files have line endings for Windows. | |
57 |
|
57 | |||
58 | - ``eol.only-consistent`` (default True) can be set to False to make |
|
58 | - ``eol.only-consistent`` (default True) can be set to False to make | |
59 | the extension convert files with inconsistent EOLs. Inconsistent |
|
59 | the extension convert files with inconsistent EOLs. Inconsistent | |
60 | means that there is both ``CRLF`` and ``LF`` present in the file. |
|
60 | means that there is both ``CRLF`` and ``LF`` present in the file. | |
61 | Such files are normally not touched under the assumption that they |
|
61 | Such files are normally not touched under the assumption that they | |
62 | have mixed EOLs on purpose. |
|
62 | have mixed EOLs on purpose. | |
63 |
|
63 | |||
|
64 | The extension provides ``cleverencode:`` and ``cleverdecode:`` filters | |||
|
65 | like the deprecated win32text extension does. This means that you can | |||
|
66 | disable win32text and enable eol and your filters will still work. You | |||
|
67 | only need to these filters until you have prepared a ``.hgeol`` file. | |||
|
68 | ||||
64 | See :hg:`help patterns` for more information about the glob patterns |
|
69 | See :hg:`help patterns` for more information about the glob patterns | |
65 | used. |
|
70 | used. | |
66 | """ |
|
71 | """ | |
67 |
|
72 | |||
68 | from mercurial.i18n import _ |
|
73 | from mercurial.i18n import _ | |
69 | from mercurial import util, config, extensions, match |
|
74 | from mercurial import util, config, extensions, match | |
70 | import re, os |
|
75 | import re, os | |
71 |
|
76 | |||
72 | # Matches a lone LF, i.e., one that is not part of CRLF. |
|
77 | # Matches a lone LF, i.e., one that is not part of CRLF. | |
73 | singlelf = re.compile('(^|[^\r])\n') |
|
78 | singlelf = re.compile('(^|[^\r])\n') | |
74 | # Matches a single EOL which can either be a CRLF where repeated CR |
|
79 | # Matches a single EOL which can either be a CRLF where repeated CR | |
75 | # are removed or a LF. We do not care about old Machintosh files, so a |
|
80 | # are removed or a LF. We do not care about old Machintosh files, so a | |
76 | # stray CR is an error. |
|
81 | # stray CR is an error. | |
77 | eolre = re.compile('\r*\n') |
|
82 | eolre = re.compile('\r*\n') | |
78 |
|
83 | |||
79 |
|
84 | |||
80 | def inconsistenteol(data): |
|
85 | def inconsistenteol(data): | |
81 | return '\r\n' in data and singlelf.search(data) |
|
86 | return '\r\n' in data and singlelf.search(data) | |
82 |
|
87 | |||
83 | def tolf(s, params, ui, **kwargs): |
|
88 | def tolf(s, params, ui, **kwargs): | |
84 | """Filter to convert to LF EOLs.""" |
|
89 | """Filter to convert to LF EOLs.""" | |
85 | if util.binary(s): |
|
90 | if util.binary(s): | |
86 | return s |
|
91 | return s | |
87 | if ui.configbool('eol', 'only-consistent', True) and inconsistenteol(s): |
|
92 | if ui.configbool('eol', 'only-consistent', True) and inconsistenteol(s): | |
88 | return s |
|
93 | return s | |
89 | return eolre.sub('\n', s) |
|
94 | return eolre.sub('\n', s) | |
90 |
|
95 | |||
91 | def tocrlf(s, params, ui, **kwargs): |
|
96 | def tocrlf(s, params, ui, **kwargs): | |
92 | """Filter to convert to CRLF EOLs.""" |
|
97 | """Filter to convert to CRLF EOLs.""" | |
93 | if util.binary(s): |
|
98 | if util.binary(s): | |
94 | return s |
|
99 | return s | |
95 | if ui.configbool('eol', 'only-consistent', True) and inconsistenteol(s): |
|
100 | if ui.configbool('eol', 'only-consistent', True) and inconsistenteol(s): | |
96 | return s |
|
101 | return s | |
97 | return eolre.sub('\r\n', s) |
|
102 | return eolre.sub('\r\n', s) | |
98 |
|
103 | |||
99 | def isbinary(s, params): |
|
104 | def isbinary(s, params): | |
100 | """Filter to do nothing with the file.""" |
|
105 | """Filter to do nothing with the file.""" | |
101 | return s |
|
106 | return s | |
102 |
|
107 | |||
103 | filters = { |
|
108 | filters = { | |
104 | 'to-lf': tolf, |
|
109 | 'to-lf': tolf, | |
105 | 'to-crlf': tocrlf, |
|
110 | 'to-crlf': tocrlf, | |
106 | 'is-binary': isbinary, |
|
111 | 'is-binary': isbinary, | |
107 | # The following provide backwards compatibility with win32text |
|
112 | # The following provide backwards compatibility with win32text | |
108 | 'cleverencode': tolf, |
|
113 | 'cleverencode:': tolf, | |
109 | 'cleverdecode': tocrlf |
|
114 | 'cleverdecode:': tocrlf | |
110 | } |
|
115 | } | |
111 |
|
116 | |||
112 |
|
117 | |||
113 | def hook(ui, repo, node, hooktype, **kwargs): |
|
118 | def hook(ui, repo, node, hooktype, **kwargs): | |
114 | """verify that files have expected EOLs""" |
|
119 | """verify that files have expected EOLs""" | |
115 | files = set() |
|
120 | files = set() | |
116 | for rev in xrange(repo[node].rev(), len(repo)): |
|
121 | for rev in xrange(repo[node].rev(), len(repo)): | |
117 | files.update(repo[rev].files()) |
|
122 | files.update(repo[rev].files()) | |
118 | tip = repo['tip'] |
|
123 | tip = repo['tip'] | |
119 | for f in files: |
|
124 | for f in files: | |
120 | if f not in tip: |
|
125 | if f not in tip: | |
121 | continue |
|
126 | continue | |
122 | for pattern, target in ui.configitems('encode'): |
|
127 | for pattern, target in ui.configitems('encode'): | |
123 | if match.match(repo.root, '', [pattern])(f): |
|
128 | if match.match(repo.root, '', [pattern])(f): | |
124 | data = tip[f].data() |
|
129 | data = tip[f].data() | |
125 | if target == "to-lf" and "\r\n" in data: |
|
130 | if target == "to-lf" and "\r\n" in data: | |
126 | raise util.Abort(_("%s should not have CRLF line endings") |
|
131 | raise util.Abort(_("%s should not have CRLF line endings") | |
127 | % f) |
|
132 | % f) | |
128 | elif target == "to-crlf" and singlelf.search(data): |
|
133 | elif target == "to-crlf" and singlelf.search(data): | |
129 | raise util.Abort(_("%s should not have LF line endings") |
|
134 | raise util.Abort(_("%s should not have LF line endings") | |
130 | % f) |
|
135 | % f) | |
131 |
|
136 | |||
132 |
|
137 | |||
133 | def preupdate(ui, repo, hooktype, parent1, parent2): |
|
138 | def preupdate(ui, repo, hooktype, parent1, parent2): | |
134 | #print "preupdate for %s: %s -> %s" % (repo.root, parent1, parent2) |
|
139 | #print "preupdate for %s: %s -> %s" % (repo.root, parent1, parent2) | |
135 | repo.readhgeol(parent1) |
|
140 | repo.readhgeol(parent1) | |
136 | return False |
|
141 | return False | |
137 |
|
142 | |||
138 | def uisetup(ui): |
|
143 | def uisetup(ui): | |
139 | ui.setconfig('hooks', 'preupdate.eol', preupdate) |
|
144 | ui.setconfig('hooks', 'preupdate.eol', preupdate) | |
140 |
|
145 | |||
141 | def extsetup(ui): |
|
146 | def extsetup(ui): | |
142 | try: |
|
147 | try: | |
143 | extensions.find('win32text') |
|
148 | extensions.find('win32text') | |
144 | raise util.Abort(_("the eol extension is incompatible with the " |
|
149 | raise util.Abort(_("the eol extension is incompatible with the " | |
145 | "win32text extension")) |
|
150 | "win32text extension")) | |
146 | except KeyError: |
|
151 | except KeyError: | |
147 | pass |
|
152 | pass | |
148 |
|
153 | |||
149 |
|
154 | |||
150 | def reposetup(ui, repo): |
|
155 | def reposetup(ui, repo): | |
151 | uisetup(repo.ui) |
|
156 | uisetup(repo.ui) | |
152 | #print "reposetup for", repo.root |
|
157 | #print "reposetup for", repo.root | |
153 |
|
158 | |||
154 | if not repo.local(): |
|
159 | if not repo.local(): | |
155 | return |
|
160 | return | |
156 | for name, fn in filters.iteritems(): |
|
161 | for name, fn in filters.iteritems(): | |
157 | repo.adddatafilter(name, fn) |
|
162 | repo.adddatafilter(name, fn) | |
158 |
|
163 | |||
159 | ui.setconfig('patch', 'eol', 'auto') |
|
164 | ui.setconfig('patch', 'eol', 'auto') | |
160 |
|
165 | |||
161 | class eolrepo(repo.__class__): |
|
166 | class eolrepo(repo.__class__): | |
162 |
|
167 | |||
163 | _decode = {'LF': 'to-lf', 'CRLF': 'to-crlf', 'BIN': 'is-binary'} |
|
168 | _decode = {'LF': 'to-lf', 'CRLF': 'to-crlf', 'BIN': 'is-binary'} | |
164 | _encode = {'LF': 'to-lf', 'CRLF': 'to-crlf', 'BIN': 'is-binary'} |
|
169 | _encode = {'LF': 'to-lf', 'CRLF': 'to-crlf', 'BIN': 'is-binary'} | |
165 |
|
170 | |||
166 | def readhgeol(self, node=None, data=None): |
|
171 | def readhgeol(self, node=None, data=None): | |
167 | if data is None: |
|
172 | if data is None: | |
168 | try: |
|
173 | try: | |
169 | if node is None: |
|
174 | if node is None: | |
170 | data = self.wfile('.hgeol').read() |
|
175 | data = self.wfile('.hgeol').read() | |
171 | else: |
|
176 | else: | |
172 | data = self[node]['.hgeol'].data() |
|
177 | data = self[node]['.hgeol'].data() | |
173 | except (IOError, LookupError): |
|
178 | except (IOError, LookupError): | |
174 | return None |
|
179 | return None | |
175 |
|
180 | |||
176 | if self.ui.config('eol', 'native', os.linesep) in ('LF', '\n'): |
|
181 | if self.ui.config('eol', 'native', os.linesep) in ('LF', '\n'): | |
177 | self._decode['NATIVE'] = 'to-lf' |
|
182 | self._decode['NATIVE'] = 'to-lf' | |
178 | else: |
|
183 | else: | |
179 | self._decode['NATIVE'] = 'to-crlf' |
|
184 | self._decode['NATIVE'] = 'to-crlf' | |
180 |
|
185 | |||
181 | eol = config.config() |
|
186 | eol = config.config() | |
182 | # Our files should not be touched. The pattern must be |
|
187 | # Our files should not be touched. The pattern must be | |
183 | # inserted first override a '** = native' pattern. |
|
188 | # inserted first override a '** = native' pattern. | |
184 | eol.set('patterns', '.hg*', 'BIN') |
|
189 | eol.set('patterns', '.hg*', 'BIN') | |
185 | # We can then parse the user's patterns. |
|
190 | # We can then parse the user's patterns. | |
186 | eol.parse('.hgeol', data) |
|
191 | eol.parse('.hgeol', data) | |
187 |
|
192 | |||
188 | if eol.get('repository', 'native') == 'CRLF': |
|
193 | if eol.get('repository', 'native') == 'CRLF': | |
189 | self._encode['NATIVE'] = 'to-crlf' |
|
194 | self._encode['NATIVE'] = 'to-crlf' | |
190 | else: |
|
195 | else: | |
191 | self._encode['NATIVE'] = 'to-lf' |
|
196 | self._encode['NATIVE'] = 'to-lf' | |
192 |
|
197 | |||
193 | for pattern, style in eol.items('patterns'): |
|
198 | for pattern, style in eol.items('patterns'): | |
194 | key = style.upper() |
|
199 | key = style.upper() | |
195 | try: |
|
200 | try: | |
196 | self.ui.setconfig('decode', pattern, self._decode[key]) |
|
201 | self.ui.setconfig('decode', pattern, self._decode[key]) | |
197 | self.ui.setconfig('encode', pattern, self._encode[key]) |
|
202 | self.ui.setconfig('encode', pattern, self._encode[key]) | |
198 | except KeyError: |
|
203 | except KeyError: | |
199 | self.ui.warn(_("ignoring unknown EOL style '%s' from %s\n") |
|
204 | self.ui.warn(_("ignoring unknown EOL style '%s' from %s\n") | |
200 | % (style, eol.source('patterns', pattern))) |
|
205 | % (style, eol.source('patterns', pattern))) | |
201 |
|
206 | |||
202 | include = [] |
|
207 | include = [] | |
203 | exclude = [] |
|
208 | exclude = [] | |
204 | for pattern, style in eol.items('patterns'): |
|
209 | for pattern, style in eol.items('patterns'): | |
205 | key = style.upper() |
|
210 | key = style.upper() | |
206 | if key == 'BIN': |
|
211 | if key == 'BIN': | |
207 | exclude.append(pattern) |
|
212 | exclude.append(pattern) | |
208 | else: |
|
213 | else: | |
209 | include.append(pattern) |
|
214 | include.append(pattern) | |
210 |
|
215 | |||
211 | # This will match the files for which we need to care |
|
216 | # This will match the files for which we need to care | |
212 | # about inconsistent newlines. |
|
217 | # about inconsistent newlines. | |
213 | return match.match(self.root, '', [], include, exclude) |
|
218 | return match.match(self.root, '', [], include, exclude) | |
214 |
|
219 | |||
215 | def _hgcleardirstate(self): |
|
220 | def _hgcleardirstate(self): | |
216 | self._eolfile = self.readhgeol() or self.readhgeol('tip') |
|
221 | self._eolfile = self.readhgeol() or self.readhgeol('tip') | |
217 |
|
222 | |||
218 | if not self._eolfile: |
|
223 | if not self._eolfile: | |
219 | self._eolfile = util.never |
|
224 | self._eolfile = util.never | |
220 | return |
|
225 | return | |
221 |
|
226 | |||
222 | try: |
|
227 | try: | |
223 | cachemtime = os.path.getmtime(self.join("eol.cache")) |
|
228 | cachemtime = os.path.getmtime(self.join("eol.cache")) | |
224 | except OSError: |
|
229 | except OSError: | |
225 | cachemtime = 0 |
|
230 | cachemtime = 0 | |
226 |
|
231 | |||
227 | try: |
|
232 | try: | |
228 | eolmtime = os.path.getmtime(self.wjoin(".hgeol")) |
|
233 | eolmtime = os.path.getmtime(self.wjoin(".hgeol")) | |
229 | except OSError: |
|
234 | except OSError: | |
230 | eolmtime = 0 |
|
235 | eolmtime = 0 | |
231 |
|
236 | |||
232 | if eolmtime > cachemtime: |
|
237 | if eolmtime > cachemtime: | |
233 | ui.debug("eol: detected change in .hgeol\n") |
|
238 | ui.debug("eol: detected change in .hgeol\n") | |
234 | # TODO: we could introduce a method for this in dirstate. |
|
239 | # TODO: we could introduce a method for this in dirstate. | |
235 | wlock = None |
|
240 | wlock = None | |
236 | try: |
|
241 | try: | |
237 | wlock = self.wlock() |
|
242 | wlock = self.wlock() | |
238 | for f, e in self.dirstate._map.iteritems(): |
|
243 | for f, e in self.dirstate._map.iteritems(): | |
239 | self.dirstate._map[f] = (e[0], e[1], -1, 0) |
|
244 | self.dirstate._map[f] = (e[0], e[1], -1, 0) | |
240 | self.dirstate._dirty = True |
|
245 | self.dirstate._dirty = True | |
241 | # Touch the cache to update mtime. TODO: are we sure this |
|
246 | # Touch the cache to update mtime. TODO: are we sure this | |
242 | # always enought to update the mtime, or should we write a |
|
247 | # always enought to update the mtime, or should we write a | |
243 | # bit to the file? |
|
248 | # bit to the file? | |
244 | self.opener("eol.cache", "w").close() |
|
249 | self.opener("eol.cache", "w").close() | |
245 | finally: |
|
250 | finally: | |
246 | if wlock is not None: |
|
251 | if wlock is not None: | |
247 | wlock.release() |
|
252 | wlock.release() | |
248 |
|
253 | |||
249 | def commitctx(self, ctx, error=False): |
|
254 | def commitctx(self, ctx, error=False): | |
250 | for f in sorted(ctx.added() + ctx.modified()): |
|
255 | for f in sorted(ctx.added() + ctx.modified()): | |
251 | if not self._eolfile(f): |
|
256 | if not self._eolfile(f): | |
252 | continue |
|
257 | continue | |
253 | data = ctx[f].data() |
|
258 | data = ctx[f].data() | |
254 | if util.binary(data): |
|
259 | if util.binary(data): | |
255 | # We should not abort here, since the user should |
|
260 | # We should not abort here, since the user should | |
256 | # be able to say "** = native" to automatically |
|
261 | # be able to say "** = native" to automatically | |
257 | # have all non-binary files taken care of. |
|
262 | # have all non-binary files taken care of. | |
258 | continue |
|
263 | continue | |
259 | if inconsistenteol(data): |
|
264 | if inconsistenteol(data): | |
260 | raise util.Abort(_("inconsistent newline style " |
|
265 | raise util.Abort(_("inconsistent newline style " | |
261 | "in %s\n" % f)) |
|
266 | "in %s\n" % f)) | |
262 | return super(eolrepo, self).commitctx(ctx, error) |
|
267 | return super(eolrepo, self).commitctx(ctx, error) | |
263 | repo.__class__ = eolrepo |
|
268 | repo.__class__ = eolrepo | |
264 | repo._hgcleardirstate() |
|
269 | repo._hgcleardirstate() |
General Comments 0
You need to be logged in to leave comments.
Login now