##// END OF EJS Templates
match: move matchers from sparse into core...
Gregory Szorc -
r33319:3c84591e default
parent child Browse files
Show More
@@ -1,943 +1,864
1 # sparse.py - allow sparse checkouts of the working directory
1 # sparse.py - allow sparse checkouts of the working directory
2 #
2 #
3 # Copyright 2014 Facebook, Inc.
3 # Copyright 2014 Facebook, Inc.
4 #
4 #
5 # This software may be used and distributed according to the terms of the
5 # This software may be used and distributed according to the terms of the
6 # GNU General Public License version 2 or any later version.
6 # GNU General Public License version 2 or any later version.
7
7
8 """allow sparse checkouts of the working directory (EXPERIMENTAL)
8 """allow sparse checkouts of the working directory (EXPERIMENTAL)
9
9
10 (This extension is not yet protected by backwards compatibility
10 (This extension is not yet protected by backwards compatibility
11 guarantees. Any aspect may break in future releases until this
11 guarantees. Any aspect may break in future releases until this
12 notice is removed.)
12 notice is removed.)
13
13
14 This extension allows the working directory to only consist of a
14 This extension allows the working directory to only consist of a
15 subset of files for the revision. This allows specific files or
15 subset of files for the revision. This allows specific files or
16 directories to be explicitly included or excluded. Many repository
16 directories to be explicitly included or excluded. Many repository
17 operations have performance proportional to the number of files in
17 operations have performance proportional to the number of files in
18 the working directory. So only realizing a subset of files in the
18 the working directory. So only realizing a subset of files in the
19 working directory can improve performance.
19 working directory can improve performance.
20
20
21 Sparse Config Files
21 Sparse Config Files
22 -------------------
22 -------------------
23
23
24 The set of files that are part of a sparse checkout are defined by
24 The set of files that are part of a sparse checkout are defined by
25 a sparse config file. The file defines 3 things: includes (files to
25 a sparse config file. The file defines 3 things: includes (files to
26 include in the sparse checkout), excludes (files to exclude from the
26 include in the sparse checkout), excludes (files to exclude from the
27 sparse checkout), and profiles (links to other config files).
27 sparse checkout), and profiles (links to other config files).
28
28
29 The file format is newline delimited. Empty lines and lines beginning
29 The file format is newline delimited. Empty lines and lines beginning
30 with ``#`` are ignored.
30 with ``#`` are ignored.
31
31
32 Lines beginning with ``%include `` denote another sparse config file
32 Lines beginning with ``%include `` denote another sparse config file
33 to include. e.g. ``%include tests.sparse``. The filename is relative
33 to include. e.g. ``%include tests.sparse``. The filename is relative
34 to the repository root.
34 to the repository root.
35
35
36 The special lines ``[include]`` and ``[exclude]`` denote the section
36 The special lines ``[include]`` and ``[exclude]`` denote the section
37 for includes and excludes that follow, respectively. It is illegal to
37 for includes and excludes that follow, respectively. It is illegal to
38 have ``[include]`` after ``[exclude]``. If no sections are defined,
38 have ``[include]`` after ``[exclude]``. If no sections are defined,
39 entries are assumed to be in the ``[include]`` section.
39 entries are assumed to be in the ``[include]`` section.
40
40
41 Non-special lines resemble file patterns to be added to either includes
41 Non-special lines resemble file patterns to be added to either includes
42 or excludes. The syntax of these lines is documented by :hg:`help patterns`.
42 or excludes. The syntax of these lines is documented by :hg:`help patterns`.
43 Patterns are interpreted as ``glob:`` by default and match against the
43 Patterns are interpreted as ``glob:`` by default and match against the
44 root of the repository.
44 root of the repository.
45
45
46 Exclusion patterns take precedence over inclusion patterns. So even
46 Exclusion patterns take precedence over inclusion patterns. So even
47 if a file is explicitly included, an ``[exclude]`` entry can remove it.
47 if a file is explicitly included, an ``[exclude]`` entry can remove it.
48
48
49 For example, say you have a repository with 3 directories, ``frontend/``,
49 For example, say you have a repository with 3 directories, ``frontend/``,
50 ``backend/``, and ``tools/``. ``frontend/`` and ``backend/`` correspond
50 ``backend/``, and ``tools/``. ``frontend/`` and ``backend/`` correspond
51 to different projects and it is uncommon for someone working on one
51 to different projects and it is uncommon for someone working on one
52 to need the files for the other. But ``tools/`` contains files shared
52 to need the files for the other. But ``tools/`` contains files shared
53 between both projects. Your sparse config files may resemble::
53 between both projects. Your sparse config files may resemble::
54
54
55 # frontend.sparse
55 # frontend.sparse
56 frontend/**
56 frontend/**
57 tools/**
57 tools/**
58
58
59 # backend.sparse
59 # backend.sparse
60 backend/**
60 backend/**
61 tools/**
61 tools/**
62
62
63 Say the backend grows in size. Or there's a directory with thousands
63 Say the backend grows in size. Or there's a directory with thousands
64 of files you wish to exclude. You can modify the profile to exclude
64 of files you wish to exclude. You can modify the profile to exclude
65 certain files::
65 certain files::
66
66
67 [include]
67 [include]
68 backend/**
68 backend/**
69 tools/**
69 tools/**
70
70
71 [exclude]
71 [exclude]
72 tools/tests/**
72 tools/tests/**
73 """
73 """
74
74
75 from __future__ import absolute_import
75 from __future__ import absolute_import
76
76
77 import collections
77 import collections
78 import os
78 import os
79
79
80 from mercurial.i18n import _
80 from mercurial.i18n import _
81 from mercurial.node import nullid
81 from mercurial.node import nullid
82 from mercurial import (
82 from mercurial import (
83 cmdutil,
83 cmdutil,
84 commands,
84 commands,
85 context,
85 context,
86 dirstate,
86 dirstate,
87 error,
87 error,
88 extensions,
88 extensions,
89 hg,
89 hg,
90 localrepo,
90 localrepo,
91 match as matchmod,
91 match as matchmod,
92 merge as mergemod,
92 merge as mergemod,
93 registrar,
93 registrar,
94 sparse,
94 sparse,
95 util,
95 util,
96 )
96 )
97
97
98 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
98 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
99 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
99 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
100 # be specifying the version(s) of Mercurial they are tested with, or
100 # be specifying the version(s) of Mercurial they are tested with, or
101 # leave the attribute unspecified.
101 # leave the attribute unspecified.
102 testedwith = 'ships-with-hg-core'
102 testedwith = 'ships-with-hg-core'
103
103
104 cmdtable = {}
104 cmdtable = {}
105 command = registrar.command(cmdtable)
105 command = registrar.command(cmdtable)
106
106
107 def uisetup(ui):
107 def uisetup(ui):
108 _setupupdates(ui)
108 _setupupdates(ui)
109 _setupcommit(ui)
109 _setupcommit(ui)
110
110
111 def extsetup(ui):
111 def extsetup(ui):
112 sparse.enabled = True
112 sparse.enabled = True
113
113
114 _setupclone(ui)
114 _setupclone(ui)
115 _setuplog(ui)
115 _setuplog(ui)
116 _setupadd(ui)
116 _setupadd(ui)
117 _setupdirstate(ui)
117 _setupdirstate(ui)
118
118
119 def reposetup(ui, repo):
119 def reposetup(ui, repo):
120 if not util.safehasattr(repo, 'dirstate'):
120 if not util.safehasattr(repo, 'dirstate'):
121 return
121 return
122
122
123 _wraprepo(ui, repo)
123 _wraprepo(ui, repo)
124
124
125 def replacefilecache(cls, propname, replacement):
125 def replacefilecache(cls, propname, replacement):
126 """Replace a filecache property with a new class. This allows changing the
126 """Replace a filecache property with a new class. This allows changing the
127 cache invalidation condition."""
127 cache invalidation condition."""
128 origcls = cls
128 origcls = cls
129 assert callable(replacement)
129 assert callable(replacement)
130 while cls is not object:
130 while cls is not object:
131 if propname in cls.__dict__:
131 if propname in cls.__dict__:
132 orig = cls.__dict__[propname]
132 orig = cls.__dict__[propname]
133 setattr(cls, propname, replacement(orig))
133 setattr(cls, propname, replacement(orig))
134 break
134 break
135 cls = cls.__bases__[0]
135 cls = cls.__bases__[0]
136
136
137 if cls is object:
137 if cls is object:
138 raise AttributeError(_("type '%s' has no property '%s'") % (origcls,
138 raise AttributeError(_("type '%s' has no property '%s'") % (origcls,
139 propname))
139 propname))
140
140
141 def _setupupdates(ui):
141 def _setupupdates(ui):
142 def _calculateupdates(orig, repo, wctx, mctx, ancestors, branchmerge, *arg,
142 def _calculateupdates(orig, repo, wctx, mctx, ancestors, branchmerge, *arg,
143 **kwargs):
143 **kwargs):
144 """Filter updates to only lay out files that match the sparse rules.
144 """Filter updates to only lay out files that match the sparse rules.
145 """
145 """
146 actions, diverge, renamedelete = orig(repo, wctx, mctx, ancestors,
146 actions, diverge, renamedelete = orig(repo, wctx, mctx, ancestors,
147 branchmerge, *arg, **kwargs)
147 branchmerge, *arg, **kwargs)
148
148
149 if not util.safehasattr(repo, 'sparsematch'):
149 if not util.safehasattr(repo, 'sparsematch'):
150 return actions, diverge, renamedelete
150 return actions, diverge, renamedelete
151
151
152 files = set()
152 files = set()
153 prunedactions = {}
153 prunedactions = {}
154 oldrevs = [pctx.rev() for pctx in wctx.parents()]
154 oldrevs = [pctx.rev() for pctx in wctx.parents()]
155 oldsparsematch = repo.sparsematch(*oldrevs)
155 oldsparsematch = repo.sparsematch(*oldrevs)
156
156
157 if branchmerge:
157 if branchmerge:
158 # If we're merging, use the wctx filter, since we're merging into
158 # If we're merging, use the wctx filter, since we're merging into
159 # the wctx.
159 # the wctx.
160 sparsematch = repo.sparsematch(wctx.parents()[0].rev())
160 sparsematch = repo.sparsematch(wctx.parents()[0].rev())
161 else:
161 else:
162 # If we're updating, use the target context's filter, since we're
162 # If we're updating, use the target context's filter, since we're
163 # moving to the target context.
163 # moving to the target context.
164 sparsematch = repo.sparsematch(mctx.rev())
164 sparsematch = repo.sparsematch(mctx.rev())
165
165
166 temporaryfiles = []
166 temporaryfiles = []
167 for file, action in actions.iteritems():
167 for file, action in actions.iteritems():
168 type, args, msg = action
168 type, args, msg = action
169 files.add(file)
169 files.add(file)
170 if sparsematch(file):
170 if sparsematch(file):
171 prunedactions[file] = action
171 prunedactions[file] = action
172 elif type == 'm':
172 elif type == 'm':
173 temporaryfiles.append(file)
173 temporaryfiles.append(file)
174 prunedactions[file] = action
174 prunedactions[file] = action
175 elif branchmerge:
175 elif branchmerge:
176 if type != 'k':
176 if type != 'k':
177 temporaryfiles.append(file)
177 temporaryfiles.append(file)
178 prunedactions[file] = action
178 prunedactions[file] = action
179 elif type == 'f':
179 elif type == 'f':
180 prunedactions[file] = action
180 prunedactions[file] = action
181 elif file in wctx:
181 elif file in wctx:
182 prunedactions[file] = ('r', args, msg)
182 prunedactions[file] = ('r', args, msg)
183
183
184 if len(temporaryfiles) > 0:
184 if len(temporaryfiles) > 0:
185 ui.status(_("temporarily included %d file(s) in the sparse checkout"
185 ui.status(_("temporarily included %d file(s) in the sparse checkout"
186 " for merging\n") % len(temporaryfiles))
186 " for merging\n") % len(temporaryfiles))
187 sparse.addtemporaryincludes(repo, temporaryfiles)
187 sparse.addtemporaryincludes(repo, temporaryfiles)
188
188
189 # Add the new files to the working copy so they can be merged, etc
189 # Add the new files to the working copy so they can be merged, etc
190 actions = []
190 actions = []
191 message = 'temporarily adding to sparse checkout'
191 message = 'temporarily adding to sparse checkout'
192 wctxmanifest = repo[None].manifest()
192 wctxmanifest = repo[None].manifest()
193 for file in temporaryfiles:
193 for file in temporaryfiles:
194 if file in wctxmanifest:
194 if file in wctxmanifest:
195 fctx = repo[None][file]
195 fctx = repo[None][file]
196 actions.append((file, (fctx.flags(), False), message))
196 actions.append((file, (fctx.flags(), False), message))
197
197
198 typeactions = collections.defaultdict(list)
198 typeactions = collections.defaultdict(list)
199 typeactions['g'] = actions
199 typeactions['g'] = actions
200 mergemod.applyupdates(repo, typeactions, repo[None], repo['.'],
200 mergemod.applyupdates(repo, typeactions, repo[None], repo['.'],
201 False)
201 False)
202
202
203 dirstate = repo.dirstate
203 dirstate = repo.dirstate
204 for file, flags, msg in actions:
204 for file, flags, msg in actions:
205 dirstate.normal(file)
205 dirstate.normal(file)
206
206
207 profiles = sparse.activeprofiles(repo)
207 profiles = sparse.activeprofiles(repo)
208 changedprofiles = profiles & files
208 changedprofiles = profiles & files
209 # If an active profile changed during the update, refresh the checkout.
209 # If an active profile changed during the update, refresh the checkout.
210 # Don't do this during a branch merge, since all incoming changes should
210 # Don't do this during a branch merge, since all incoming changes should
211 # have been handled by the temporary includes above.
211 # have been handled by the temporary includes above.
212 if changedprofiles and not branchmerge:
212 if changedprofiles and not branchmerge:
213 mf = mctx.manifest()
213 mf = mctx.manifest()
214 for file in mf:
214 for file in mf:
215 old = oldsparsematch(file)
215 old = oldsparsematch(file)
216 new = sparsematch(file)
216 new = sparsematch(file)
217 if not old and new:
217 if not old and new:
218 flags = mf.flags(file)
218 flags = mf.flags(file)
219 prunedactions[file] = ('g', (flags, False), '')
219 prunedactions[file] = ('g', (flags, False), '')
220 elif old and not new:
220 elif old and not new:
221 prunedactions[file] = ('r', [], '')
221 prunedactions[file] = ('r', [], '')
222
222
223 return prunedactions, diverge, renamedelete
223 return prunedactions, diverge, renamedelete
224
224
225 extensions.wrapfunction(mergemod, 'calculateupdates', _calculateupdates)
225 extensions.wrapfunction(mergemod, 'calculateupdates', _calculateupdates)
226
226
227 def _update(orig, repo, node, branchmerge, *args, **kwargs):
227 def _update(orig, repo, node, branchmerge, *args, **kwargs):
228 results = orig(repo, node, branchmerge, *args, **kwargs)
228 results = orig(repo, node, branchmerge, *args, **kwargs)
229
229
230 # If we're updating to a location, clean up any stale temporary includes
230 # If we're updating to a location, clean up any stale temporary includes
231 # (ex: this happens during hg rebase --abort).
231 # (ex: this happens during hg rebase --abort).
232 if not branchmerge and util.safehasattr(repo, 'sparsematch'):
232 if not branchmerge and util.safehasattr(repo, 'sparsematch'):
233 repo.prunetemporaryincludes()
233 repo.prunetemporaryincludes()
234 return results
234 return results
235
235
236 extensions.wrapfunction(mergemod, 'update', _update)
236 extensions.wrapfunction(mergemod, 'update', _update)
237
237
238 def _setupcommit(ui):
238 def _setupcommit(ui):
239 def _refreshoncommit(orig, self, node):
239 def _refreshoncommit(orig, self, node):
240 """Refresh the checkout when commits touch .hgsparse
240 """Refresh the checkout when commits touch .hgsparse
241 """
241 """
242 orig(self, node)
242 orig(self, node)
243 repo = self._repo
243 repo = self._repo
244
244
245 ctx = repo[node]
245 ctx = repo[node]
246 profiles = sparse.patternsforrev(repo, ctx.rev())[2]
246 profiles = sparse.patternsforrev(repo, ctx.rev())[2]
247
247
248 # profiles will only have data if sparse is enabled.
248 # profiles will only have data if sparse is enabled.
249 if set(profiles) & set(ctx.files()):
249 if set(profiles) & set(ctx.files()):
250 origstatus = repo.status()
250 origstatus = repo.status()
251 origsparsematch = repo.sparsematch()
251 origsparsematch = repo.sparsematch()
252 _refresh(repo.ui, repo, origstatus, origsparsematch, True)
252 _refresh(repo.ui, repo, origstatus, origsparsematch, True)
253
253
254 if util.safehasattr(repo, 'prunetemporaryincludes'):
254 if util.safehasattr(repo, 'prunetemporaryincludes'):
255 repo.prunetemporaryincludes()
255 repo.prunetemporaryincludes()
256
256
257 extensions.wrapfunction(context.committablectx, 'markcommitted',
257 extensions.wrapfunction(context.committablectx, 'markcommitted',
258 _refreshoncommit)
258 _refreshoncommit)
259
259
260 def _setuplog(ui):
260 def _setuplog(ui):
261 entry = commands.table['^log|history']
261 entry = commands.table['^log|history']
262 entry[1].append(('', 'sparse', None,
262 entry[1].append(('', 'sparse', None,
263 "limit to changesets affecting the sparse checkout"))
263 "limit to changesets affecting the sparse checkout"))
264
264
265 def _logrevs(orig, repo, opts):
265 def _logrevs(orig, repo, opts):
266 revs = orig(repo, opts)
266 revs = orig(repo, opts)
267 if opts.get('sparse'):
267 if opts.get('sparse'):
268 sparsematch = repo.sparsematch()
268 sparsematch = repo.sparsematch()
269 def ctxmatch(rev):
269 def ctxmatch(rev):
270 ctx = repo[rev]
270 ctx = repo[rev]
271 return any(f for f in ctx.files() if sparsematch(f))
271 return any(f for f in ctx.files() if sparsematch(f))
272 revs = revs.filter(ctxmatch)
272 revs = revs.filter(ctxmatch)
273 return revs
273 return revs
274 extensions.wrapfunction(cmdutil, '_logrevs', _logrevs)
274 extensions.wrapfunction(cmdutil, '_logrevs', _logrevs)
275
275
276 def _clonesparsecmd(orig, ui, repo, *args, **opts):
276 def _clonesparsecmd(orig, ui, repo, *args, **opts):
277 include_pat = opts.get('include')
277 include_pat = opts.get('include')
278 exclude_pat = opts.get('exclude')
278 exclude_pat = opts.get('exclude')
279 enableprofile_pat = opts.get('enable_profile')
279 enableprofile_pat = opts.get('enable_profile')
280 include = exclude = enableprofile = False
280 include = exclude = enableprofile = False
281 if include_pat:
281 if include_pat:
282 pat = include_pat
282 pat = include_pat
283 include = True
283 include = True
284 if exclude_pat:
284 if exclude_pat:
285 pat = exclude_pat
285 pat = exclude_pat
286 exclude = True
286 exclude = True
287 if enableprofile_pat:
287 if enableprofile_pat:
288 pat = enableprofile_pat
288 pat = enableprofile_pat
289 enableprofile = True
289 enableprofile = True
290 if sum([include, exclude, enableprofile]) > 1:
290 if sum([include, exclude, enableprofile]) > 1:
291 raise error.Abort(_("too many flags specified."))
291 raise error.Abort(_("too many flags specified."))
292 if include or exclude or enableprofile:
292 if include or exclude or enableprofile:
293 def clonesparse(orig, self, node, overwrite, *args, **kwargs):
293 def clonesparse(orig, self, node, overwrite, *args, **kwargs):
294 _config(self.ui, self.unfiltered(), pat, {}, include=include,
294 _config(self.ui, self.unfiltered(), pat, {}, include=include,
295 exclude=exclude, enableprofile=enableprofile)
295 exclude=exclude, enableprofile=enableprofile)
296 return orig(self, node, overwrite, *args, **kwargs)
296 return orig(self, node, overwrite, *args, **kwargs)
297 extensions.wrapfunction(hg, 'updaterepo', clonesparse)
297 extensions.wrapfunction(hg, 'updaterepo', clonesparse)
298 return orig(ui, repo, *args, **opts)
298 return orig(ui, repo, *args, **opts)
299
299
300 def _setupclone(ui):
300 def _setupclone(ui):
301 entry = commands.table['^clone']
301 entry = commands.table['^clone']
302 entry[1].append(('', 'enable-profile', [],
302 entry[1].append(('', 'enable-profile', [],
303 'enable a sparse profile'))
303 'enable a sparse profile'))
304 entry[1].append(('', 'include', [],
304 entry[1].append(('', 'include', [],
305 'include sparse pattern'))
305 'include sparse pattern'))
306 entry[1].append(('', 'exclude', [],
306 entry[1].append(('', 'exclude', [],
307 'exclude sparse pattern'))
307 'exclude sparse pattern'))
308 extensions.wrapcommand(commands.table, 'clone', _clonesparsecmd)
308 extensions.wrapcommand(commands.table, 'clone', _clonesparsecmd)
309
309
310 def _setupadd(ui):
310 def _setupadd(ui):
311 entry = commands.table['^add']
311 entry = commands.table['^add']
312 entry[1].append(('s', 'sparse', None,
312 entry[1].append(('s', 'sparse', None,
313 'also include directories of added files in sparse config'))
313 'also include directories of added files in sparse config'))
314
314
315 def _add(orig, ui, repo, *pats, **opts):
315 def _add(orig, ui, repo, *pats, **opts):
316 if opts.get('sparse'):
316 if opts.get('sparse'):
317 dirs = set()
317 dirs = set()
318 for pat in pats:
318 for pat in pats:
319 dirname, basename = util.split(pat)
319 dirname, basename = util.split(pat)
320 dirs.add(dirname)
320 dirs.add(dirname)
321 _config(ui, repo, list(dirs), opts, include=True)
321 _config(ui, repo, list(dirs), opts, include=True)
322 return orig(ui, repo, *pats, **opts)
322 return orig(ui, repo, *pats, **opts)
323
323
324 extensions.wrapcommand(commands.table, 'add', _add)
324 extensions.wrapcommand(commands.table, 'add', _add)
325
325
326 def _setupdirstate(ui):
326 def _setupdirstate(ui):
327 """Modify the dirstate to prevent stat'ing excluded files,
327 """Modify the dirstate to prevent stat'ing excluded files,
328 and to prevent modifications to files outside the checkout.
328 and to prevent modifications to files outside the checkout.
329 """
329 """
330
330
331 def _dirstate(orig, repo):
331 def _dirstate(orig, repo):
332 dirstate = orig(repo)
332 dirstate = orig(repo)
333 dirstate.repo = repo
333 dirstate.repo = repo
334 return dirstate
334 return dirstate
335 extensions.wrapfunction(
335 extensions.wrapfunction(
336 localrepo.localrepository.dirstate, 'func', _dirstate)
336 localrepo.localrepository.dirstate, 'func', _dirstate)
337
337
338 # The atrocity below is needed to wrap dirstate._ignore. It is a cached
338 # The atrocity below is needed to wrap dirstate._ignore. It is a cached
339 # property, which means normal function wrapping doesn't work.
339 # property, which means normal function wrapping doesn't work.
340 class ignorewrapper(object):
340 class ignorewrapper(object):
341 def __init__(self, orig):
341 def __init__(self, orig):
342 self.orig = orig
342 self.orig = orig
343 self.origignore = None
343 self.origignore = None
344 self.func = None
344 self.func = None
345 self.sparsematch = None
345 self.sparsematch = None
346
346
347 def __get__(self, obj, type=None):
347 def __get__(self, obj, type=None):
348 repo = obj.repo
348 repo = obj.repo
349 origignore = self.orig.__get__(obj)
349 origignore = self.orig.__get__(obj)
350 if not util.safehasattr(repo, 'sparsematch'):
350 if not util.safehasattr(repo, 'sparsematch'):
351 return origignore
351 return origignore
352
352
353 sparsematch = repo.sparsematch()
353 sparsematch = repo.sparsematch()
354 if self.sparsematch != sparsematch or self.origignore != origignore:
354 if self.sparsematch != sparsematch or self.origignore != origignore:
355 self.func = unionmatcher([origignore,
355 self.func = matchmod.unionmatcher([
356 negatematcher(sparsematch)])
356 origignore, matchmod.negatematcher(sparsematch)])
357 self.sparsematch = sparsematch
357 self.sparsematch = sparsematch
358 self.origignore = origignore
358 self.origignore = origignore
359 return self.func
359 return self.func
360
360
361 def __set__(self, obj, value):
361 def __set__(self, obj, value):
362 return self.orig.__set__(obj, value)
362 return self.orig.__set__(obj, value)
363
363
364 def __delete__(self, obj):
364 def __delete__(self, obj):
365 return self.orig.__delete__(obj)
365 return self.orig.__delete__(obj)
366
366
367 replacefilecache(dirstate.dirstate, '_ignore', ignorewrapper)
367 replacefilecache(dirstate.dirstate, '_ignore', ignorewrapper)
368
368
369 # dirstate.rebuild should not add non-matching files
369 # dirstate.rebuild should not add non-matching files
370 def _rebuild(orig, self, parent, allfiles, changedfiles=None):
370 def _rebuild(orig, self, parent, allfiles, changedfiles=None):
371 if util.safehasattr(self.repo, 'sparsematch'):
371 if util.safehasattr(self.repo, 'sparsematch'):
372 matcher = self.repo.sparsematch()
372 matcher = self.repo.sparsematch()
373 allfiles = allfiles.matches(matcher)
373 allfiles = allfiles.matches(matcher)
374 if changedfiles:
374 if changedfiles:
375 changedfiles = [f for f in changedfiles if matcher(f)]
375 changedfiles = [f for f in changedfiles if matcher(f)]
376
376
377 if changedfiles is not None:
377 if changedfiles is not None:
378 # In _rebuild, these files will be deleted from the dirstate
378 # In _rebuild, these files will be deleted from the dirstate
379 # when they are not found to be in allfiles
379 # when they are not found to be in allfiles
380 dirstatefilestoremove = set(f for f in self if not matcher(f))
380 dirstatefilestoremove = set(f for f in self if not matcher(f))
381 changedfiles = dirstatefilestoremove.union(changedfiles)
381 changedfiles = dirstatefilestoremove.union(changedfiles)
382
382
383 return orig(self, parent, allfiles, changedfiles)
383 return orig(self, parent, allfiles, changedfiles)
384 extensions.wrapfunction(dirstate.dirstate, 'rebuild', _rebuild)
384 extensions.wrapfunction(dirstate.dirstate, 'rebuild', _rebuild)
385
385
386 # Prevent adding files that are outside the sparse checkout
386 # Prevent adding files that are outside the sparse checkout
387 editfuncs = ['normal', 'add', 'normallookup', 'copy', 'remove', 'merge']
387 editfuncs = ['normal', 'add', 'normallookup', 'copy', 'remove', 'merge']
388 hint = _('include file with `hg debugsparse --include <pattern>` or use ' +
388 hint = _('include file with `hg debugsparse --include <pattern>` or use ' +
389 '`hg add -s <file>` to include file directory while adding')
389 '`hg add -s <file>` to include file directory while adding')
390 for func in editfuncs:
390 for func in editfuncs:
391 def _wrapper(orig, self, *args):
391 def _wrapper(orig, self, *args):
392 repo = self.repo
392 repo = self.repo
393 if util.safehasattr(repo, 'sparsematch'):
393 if util.safehasattr(repo, 'sparsematch'):
394 dirstate = repo.dirstate
394 dirstate = repo.dirstate
395 sparsematch = repo.sparsematch()
395 sparsematch = repo.sparsematch()
396 for f in args:
396 for f in args:
397 if (f is not None and not sparsematch(f) and
397 if (f is not None and not sparsematch(f) and
398 f not in dirstate):
398 f not in dirstate):
399 raise error.Abort(_("cannot add '%s' - it is outside "
399 raise error.Abort(_("cannot add '%s' - it is outside "
400 "the sparse checkout") % f,
400 "the sparse checkout") % f,
401 hint=hint)
401 hint=hint)
402 return orig(self, *args)
402 return orig(self, *args)
403 extensions.wrapfunction(dirstate.dirstate, func, _wrapper)
403 extensions.wrapfunction(dirstate.dirstate, func, _wrapper)
404
404
405 def _wraprepo(ui, repo):
405 def _wraprepo(ui, repo):
406 class SparseRepo(repo.__class__):
406 class SparseRepo(repo.__class__):
407 def sparsematch(self, *revs, **kwargs):
407 def sparsematch(self, *revs, **kwargs):
408 """Returns the sparse match function for the given revs.
408 """Returns the sparse match function for the given revs.
409
409
410 If multiple revs are specified, the match function is the union
410 If multiple revs are specified, the match function is the union
411 of all the revs.
411 of all the revs.
412
412
413 `includetemp` is used to indicate if the temporarily included file
413 `includetemp` is used to indicate if the temporarily included file
414 should be part of the matcher.
414 should be part of the matcher.
415 """
415 """
416 if not revs or revs == (None,):
416 if not revs or revs == (None,):
417 revs = [self.changelog.rev(node) for node in
417 revs = [self.changelog.rev(node) for node in
418 self.dirstate.parents() if node != nullid]
418 self.dirstate.parents() if node != nullid]
419
419
420 includetemp = kwargs.get('includetemp', True)
420 includetemp = kwargs.get('includetemp', True)
421 signature = sparse.configsignature(self, includetemp=includetemp)
421 signature = sparse.configsignature(self, includetemp=includetemp)
422
422
423 key = '%s %s' % (str(signature), ' '.join([str(r) for r in revs]))
423 key = '%s %s' % (str(signature), ' '.join([str(r) for r in revs]))
424
424
425 result = self._sparsematchercache.get(key, None)
425 result = self._sparsematchercache.get(key, None)
426 if result:
426 if result:
427 return result
427 return result
428
428
429 matchers = []
429 matchers = []
430 for rev in revs:
430 for rev in revs:
431 try:
431 try:
432 includes, excludes, profiles = sparse.patternsforrev(
432 includes, excludes, profiles = sparse.patternsforrev(
433 self, rev)
433 self, rev)
434
434
435 if includes or excludes:
435 if includes or excludes:
436 # Explicitly include subdirectories of includes so
436 # Explicitly include subdirectories of includes so
437 # status will walk them down to the actual include.
437 # status will walk them down to the actual include.
438 subdirs = set()
438 subdirs = set()
439 for include in includes:
439 for include in includes:
440 dirname = os.path.dirname(include)
440 dirname = os.path.dirname(include)
441 # basename is used to avoid issues with absolute
441 # basename is used to avoid issues with absolute
442 # paths (which on Windows can include the drive).
442 # paths (which on Windows can include the drive).
443 while os.path.basename(dirname):
443 while os.path.basename(dirname):
444 subdirs.add(dirname)
444 subdirs.add(dirname)
445 dirname = os.path.dirname(dirname)
445 dirname = os.path.dirname(dirname)
446
446
447 matcher = matchmod.match(self.root, '', [],
447 matcher = matchmod.match(self.root, '', [],
448 include=includes, exclude=excludes,
448 include=includes, exclude=excludes,
449 default='relpath')
449 default='relpath')
450 if subdirs:
450 if subdirs:
451 matcher = forceincludematcher(matcher, subdirs)
451 matcher = matchmod.forceincludematcher(matcher,
452 subdirs)
452 matchers.append(matcher)
453 matchers.append(matcher)
453 except IOError:
454 except IOError:
454 pass
455 pass
455
456
456 result = None
457 result = None
457 if not matchers:
458 if not matchers:
458 result = matchmod.always(self.root, '')
459 result = matchmod.always(self.root, '')
459 elif len(matchers) == 1:
460 elif len(matchers) == 1:
460 result = matchers[0]
461 result = matchers[0]
461 else:
462 else:
462 result = unionmatcher(matchers)
463 result = matchmod.unionmatcher(matchers)
463
464
464 if kwargs.get('includetemp', True):
465 if kwargs.get('includetemp', True):
465 tempincludes = sparse.readtemporaryincludes(self)
466 tempincludes = sparse.readtemporaryincludes(self)
466 result = forceincludematcher(result, tempincludes)
467 result = matchmod.forceincludematcher(result, tempincludes)
467
468
468 self._sparsematchercache[key] = result
469 self._sparsematchercache[key] = result
469
470
470 return result
471 return result
471
472
472 def prunetemporaryincludes(self):
473 def prunetemporaryincludes(self):
473 if repo.vfs.exists('tempsparse'):
474 if repo.vfs.exists('tempsparse'):
474 origstatus = self.status()
475 origstatus = self.status()
475 modified, added, removed, deleted, a, b, c = origstatus
476 modified, added, removed, deleted, a, b, c = origstatus
476 if modified or added or removed or deleted:
477 if modified or added or removed or deleted:
477 # Still have pending changes. Don't bother trying to prune.
478 # Still have pending changes. Don't bother trying to prune.
478 return
479 return
479
480
480 sparsematch = self.sparsematch(includetemp=False)
481 sparsematch = self.sparsematch(includetemp=False)
481 dirstate = self.dirstate
482 dirstate = self.dirstate
482 actions = []
483 actions = []
483 dropped = []
484 dropped = []
484 tempincludes = sparse.readtemporaryincludes(self)
485 tempincludes = sparse.readtemporaryincludes(self)
485 for file in tempincludes:
486 for file in tempincludes:
486 if file in dirstate and not sparsematch(file):
487 if file in dirstate and not sparsematch(file):
487 message = 'dropping temporarily included sparse files'
488 message = 'dropping temporarily included sparse files'
488 actions.append((file, None, message))
489 actions.append((file, None, message))
489 dropped.append(file)
490 dropped.append(file)
490
491
491 typeactions = collections.defaultdict(list)
492 typeactions = collections.defaultdict(list)
492 typeactions['r'] = actions
493 typeactions['r'] = actions
493 mergemod.applyupdates(self, typeactions, self[None], self['.'],
494 mergemod.applyupdates(self, typeactions, self[None], self['.'],
494 False)
495 False)
495
496
496 # Fix dirstate
497 # Fix dirstate
497 for file in dropped:
498 for file in dropped:
498 dirstate.drop(file)
499 dirstate.drop(file)
499
500
500 self.vfs.unlink('tempsparse')
501 self.vfs.unlink('tempsparse')
501 sparse.invalidatesignaturecache(self)
502 sparse.invalidatesignaturecache(self)
502 msg = _("cleaned up %d temporarily added file(s) from the "
503 msg = _("cleaned up %d temporarily added file(s) from the "
503 "sparse checkout\n")
504 "sparse checkout\n")
504 ui.status(msg % len(tempincludes))
505 ui.status(msg % len(tempincludes))
505
506
506 if 'dirstate' in repo._filecache:
507 if 'dirstate' in repo._filecache:
507 repo.dirstate.repo = repo
508 repo.dirstate.repo = repo
508
509
509 repo.__class__ = SparseRepo
510 repo.__class__ = SparseRepo
510
511
511 @command('^debugsparse', [
512 @command('^debugsparse', [
512 ('I', 'include', False, _('include files in the sparse checkout')),
513 ('I', 'include', False, _('include files in the sparse checkout')),
513 ('X', 'exclude', False, _('exclude files in the sparse checkout')),
514 ('X', 'exclude', False, _('exclude files in the sparse checkout')),
514 ('d', 'delete', False, _('delete an include/exclude rule')),
515 ('d', 'delete', False, _('delete an include/exclude rule')),
515 ('f', 'force', False, _('allow changing rules even with pending changes')),
516 ('f', 'force', False, _('allow changing rules even with pending changes')),
516 ('', 'enable-profile', False, _('enables the specified profile')),
517 ('', 'enable-profile', False, _('enables the specified profile')),
517 ('', 'disable-profile', False, _('disables the specified profile')),
518 ('', 'disable-profile', False, _('disables the specified profile')),
518 ('', 'import-rules', False, _('imports rules from a file')),
519 ('', 'import-rules', False, _('imports rules from a file')),
519 ('', 'clear-rules', False, _('clears local include/exclude rules')),
520 ('', 'clear-rules', False, _('clears local include/exclude rules')),
520 ('', 'refresh', False, _('updates the working after sparseness changes')),
521 ('', 'refresh', False, _('updates the working after sparseness changes')),
521 ('', 'reset', False, _('makes the repo full again')),
522 ('', 'reset', False, _('makes the repo full again')),
522 ] + commands.templateopts,
523 ] + commands.templateopts,
523 _('[--OPTION] PATTERN...'))
524 _('[--OPTION] PATTERN...'))
524 def debugsparse(ui, repo, *pats, **opts):
525 def debugsparse(ui, repo, *pats, **opts):
525 """make the current checkout sparse, or edit the existing checkout
526 """make the current checkout sparse, or edit the existing checkout
526
527
527 The sparse command is used to make the current checkout sparse.
528 The sparse command is used to make the current checkout sparse.
528 This means files that don't meet the sparse condition will not be
529 This means files that don't meet the sparse condition will not be
529 written to disk, or show up in any working copy operations. It does
530 written to disk, or show up in any working copy operations. It does
530 not affect files in history in any way.
531 not affect files in history in any way.
531
532
532 Passing no arguments prints the currently applied sparse rules.
533 Passing no arguments prints the currently applied sparse rules.
533
534
534 --include and --exclude are used to add and remove files from the sparse
535 --include and --exclude are used to add and remove files from the sparse
535 checkout. The effects of adding an include or exclude rule are applied
536 checkout. The effects of adding an include or exclude rule are applied
536 immediately. If applying the new rule would cause a file with pending
537 immediately. If applying the new rule would cause a file with pending
537 changes to be added or removed, the command will fail. Pass --force to
538 changes to be added or removed, the command will fail. Pass --force to
538 force a rule change even with pending changes (the changes on disk will
539 force a rule change even with pending changes (the changes on disk will
539 be preserved).
540 be preserved).
540
541
541 --delete removes an existing include/exclude rule. The effects are
542 --delete removes an existing include/exclude rule. The effects are
542 immediate.
543 immediate.
543
544
544 --refresh refreshes the files on disk based on the sparse rules. This is
545 --refresh refreshes the files on disk based on the sparse rules. This is
545 only necessary if .hg/sparse was changed by hand.
546 only necessary if .hg/sparse was changed by hand.
546
547
547 --enable-profile and --disable-profile accept a path to a .hgsparse file.
548 --enable-profile and --disable-profile accept a path to a .hgsparse file.
548 This allows defining sparse checkouts and tracking them inside the
549 This allows defining sparse checkouts and tracking them inside the
549 repository. This is useful for defining commonly used sparse checkouts for
550 repository. This is useful for defining commonly used sparse checkouts for
550 many people to use. As the profile definition changes over time, the sparse
551 many people to use. As the profile definition changes over time, the sparse
551 checkout will automatically be updated appropriately, depending on which
552 checkout will automatically be updated appropriately, depending on which
552 changeset is checked out. Changes to .hgsparse are not applied until they
553 changeset is checked out. Changes to .hgsparse are not applied until they
553 have been committed.
554 have been committed.
554
555
555 --import-rules accepts a path to a file containing rules in the .hgsparse
556 --import-rules accepts a path to a file containing rules in the .hgsparse
556 format, allowing you to add --include, --exclude and --enable-profile rules
557 format, allowing you to add --include, --exclude and --enable-profile rules
557 in bulk. Like the --include, --exclude and --enable-profile switches, the
558 in bulk. Like the --include, --exclude and --enable-profile switches, the
558 changes are applied immediately.
559 changes are applied immediately.
559
560
560 --clear-rules removes all local include and exclude rules, while leaving
561 --clear-rules removes all local include and exclude rules, while leaving
561 any enabled profiles in place.
562 any enabled profiles in place.
562
563
563 Returns 0 if editing the sparse checkout succeeds.
564 Returns 0 if editing the sparse checkout succeeds.
564 """
565 """
565 include = opts.get('include')
566 include = opts.get('include')
566 exclude = opts.get('exclude')
567 exclude = opts.get('exclude')
567 force = opts.get('force')
568 force = opts.get('force')
568 enableprofile = opts.get('enable_profile')
569 enableprofile = opts.get('enable_profile')
569 disableprofile = opts.get('disable_profile')
570 disableprofile = opts.get('disable_profile')
570 importrules = opts.get('import_rules')
571 importrules = opts.get('import_rules')
571 clearrules = opts.get('clear_rules')
572 clearrules = opts.get('clear_rules')
572 delete = opts.get('delete')
573 delete = opts.get('delete')
573 refresh = opts.get('refresh')
574 refresh = opts.get('refresh')
574 reset = opts.get('reset')
575 reset = opts.get('reset')
575 count = sum([include, exclude, enableprofile, disableprofile, delete,
576 count = sum([include, exclude, enableprofile, disableprofile, delete,
576 importrules, refresh, clearrules, reset])
577 importrules, refresh, clearrules, reset])
577 if count > 1:
578 if count > 1:
578 raise error.Abort(_("too many flags specified"))
579 raise error.Abort(_("too many flags specified"))
579
580
580 if count == 0:
581 if count == 0:
581 if repo.vfs.exists('sparse'):
582 if repo.vfs.exists('sparse'):
582 ui.status(repo.vfs.read("sparse") + "\n")
583 ui.status(repo.vfs.read("sparse") + "\n")
583 temporaryincludes = sparse.readtemporaryincludes(repo)
584 temporaryincludes = sparse.readtemporaryincludes(repo)
584 if temporaryincludes:
585 if temporaryincludes:
585 ui.status(_("Temporarily Included Files (for merge/rebase):\n"))
586 ui.status(_("Temporarily Included Files (for merge/rebase):\n"))
586 ui.status(("\n".join(temporaryincludes) + "\n"))
587 ui.status(("\n".join(temporaryincludes) + "\n"))
587 else:
588 else:
588 ui.status(_('repo is not sparse\n'))
589 ui.status(_('repo is not sparse\n'))
589 return
590 return
590
591
591 if include or exclude or delete or reset or enableprofile or disableprofile:
592 if include or exclude or delete or reset or enableprofile or disableprofile:
592 _config(ui, repo, pats, opts, include=include, exclude=exclude,
593 _config(ui, repo, pats, opts, include=include, exclude=exclude,
593 reset=reset, delete=delete, enableprofile=enableprofile,
594 reset=reset, delete=delete, enableprofile=enableprofile,
594 disableprofile=disableprofile, force=force)
595 disableprofile=disableprofile, force=force)
595
596
596 if importrules:
597 if importrules:
597 _import(ui, repo, pats, opts, force=force)
598 _import(ui, repo, pats, opts, force=force)
598
599
599 if clearrules:
600 if clearrules:
600 _clear(ui, repo, pats, force=force)
601 _clear(ui, repo, pats, force=force)
601
602
602 if refresh:
603 if refresh:
603 try:
604 try:
604 wlock = repo.wlock()
605 wlock = repo.wlock()
605 fcounts = map(
606 fcounts = map(
606 len,
607 len,
607 _refresh(ui, repo, repo.status(), repo.sparsematch(), force))
608 _refresh(ui, repo, repo.status(), repo.sparsematch(), force))
608 _verbose_output(ui, opts, 0, 0, 0, *fcounts)
609 _verbose_output(ui, opts, 0, 0, 0, *fcounts)
609 finally:
610 finally:
610 wlock.release()
611 wlock.release()
611
612
612 def _config(ui, repo, pats, opts, include=False, exclude=False, reset=False,
613 def _config(ui, repo, pats, opts, include=False, exclude=False, reset=False,
613 delete=False, enableprofile=False, disableprofile=False,
614 delete=False, enableprofile=False, disableprofile=False,
614 force=False):
615 force=False):
615 """
616 """
616 Perform a sparse config update. Only one of the kwargs may be specified.
617 Perform a sparse config update. Only one of the kwargs may be specified.
617 """
618 """
618 wlock = repo.wlock()
619 wlock = repo.wlock()
619 try:
620 try:
620 oldsparsematch = repo.sparsematch()
621 oldsparsematch = repo.sparsematch()
621
622
622 raw = repo.vfs.tryread('sparse')
623 raw = repo.vfs.tryread('sparse')
623 if raw:
624 if raw:
624 oldinclude, oldexclude, oldprofiles = map(
625 oldinclude, oldexclude, oldprofiles = map(
625 set, sparse.parseconfig(ui, raw))
626 set, sparse.parseconfig(ui, raw))
626 else:
627 else:
627 oldinclude = set()
628 oldinclude = set()
628 oldexclude = set()
629 oldexclude = set()
629 oldprofiles = set()
630 oldprofiles = set()
630
631
631 try:
632 try:
632 if reset:
633 if reset:
633 newinclude = set()
634 newinclude = set()
634 newexclude = set()
635 newexclude = set()
635 newprofiles = set()
636 newprofiles = set()
636 else:
637 else:
637 newinclude = set(oldinclude)
638 newinclude = set(oldinclude)
638 newexclude = set(oldexclude)
639 newexclude = set(oldexclude)
639 newprofiles = set(oldprofiles)
640 newprofiles = set(oldprofiles)
640
641
641 oldstatus = repo.status()
642 oldstatus = repo.status()
642
643
643 if any(pat.startswith('/') for pat in pats):
644 if any(pat.startswith('/') for pat in pats):
644 ui.warn(_('warning: paths cannot start with /, ignoring: %s\n')
645 ui.warn(_('warning: paths cannot start with /, ignoring: %s\n')
645 % ([pat for pat in pats if pat.startswith('/')]))
646 % ([pat for pat in pats if pat.startswith('/')]))
646 elif include:
647 elif include:
647 newinclude.update(pats)
648 newinclude.update(pats)
648 elif exclude:
649 elif exclude:
649 newexclude.update(pats)
650 newexclude.update(pats)
650 elif enableprofile:
651 elif enableprofile:
651 newprofiles.update(pats)
652 newprofiles.update(pats)
652 elif disableprofile:
653 elif disableprofile:
653 newprofiles.difference_update(pats)
654 newprofiles.difference_update(pats)
654 elif delete:
655 elif delete:
655 newinclude.difference_update(pats)
656 newinclude.difference_update(pats)
656 newexclude.difference_update(pats)
657 newexclude.difference_update(pats)
657
658
658 sparse.writeconfig(repo, newinclude, newexclude, newprofiles)
659 sparse.writeconfig(repo, newinclude, newexclude, newprofiles)
659
660
660 fcounts = map(
661 fcounts = map(
661 len, _refresh(ui, repo, oldstatus, oldsparsematch, force))
662 len, _refresh(ui, repo, oldstatus, oldsparsematch, force))
662
663
663 profilecount = (len(newprofiles - oldprofiles) -
664 profilecount = (len(newprofiles - oldprofiles) -
664 len(oldprofiles - newprofiles))
665 len(oldprofiles - newprofiles))
665 includecount = (len(newinclude - oldinclude) -
666 includecount = (len(newinclude - oldinclude) -
666 len(oldinclude - newinclude))
667 len(oldinclude - newinclude))
667 excludecount = (len(newexclude - oldexclude) -
668 excludecount = (len(newexclude - oldexclude) -
668 len(oldexclude - newexclude))
669 len(oldexclude - newexclude))
669 _verbose_output(
670 _verbose_output(
670 ui, opts, profilecount, includecount, excludecount, *fcounts)
671 ui, opts, profilecount, includecount, excludecount, *fcounts)
671 except Exception:
672 except Exception:
672 sparse.writeconfig(repo, oldinclude, oldexclude, oldprofiles)
673 sparse.writeconfig(repo, oldinclude, oldexclude, oldprofiles)
673 raise
674 raise
674 finally:
675 finally:
675 wlock.release()
676 wlock.release()
676
677
677 def _import(ui, repo, files, opts, force=False):
678 def _import(ui, repo, files, opts, force=False):
678 with repo.wlock():
679 with repo.wlock():
679 # load union of current active profile
680 # load union of current active profile
680 revs = [repo.changelog.rev(node) for node in
681 revs = [repo.changelog.rev(node) for node in
681 repo.dirstate.parents() if node != nullid]
682 repo.dirstate.parents() if node != nullid]
682
683
683 # read current configuration
684 # read current configuration
684 raw = repo.vfs.tryread('sparse')
685 raw = repo.vfs.tryread('sparse')
685 oincludes, oexcludes, oprofiles = sparse.parseconfig(ui, raw)
686 oincludes, oexcludes, oprofiles = sparse.parseconfig(ui, raw)
686 includes, excludes, profiles = map(
687 includes, excludes, profiles = map(
687 set, (oincludes, oexcludes, oprofiles))
688 set, (oincludes, oexcludes, oprofiles))
688
689
689 # all active rules
690 # all active rules
690 aincludes, aexcludes, aprofiles = set(), set(), set()
691 aincludes, aexcludes, aprofiles = set(), set(), set()
691 for rev in revs:
692 for rev in revs:
692 rincludes, rexcludes, rprofiles = sparse.patternsforrev(repo, rev)
693 rincludes, rexcludes, rprofiles = sparse.patternsforrev(repo, rev)
693 aincludes.update(rincludes)
694 aincludes.update(rincludes)
694 aexcludes.update(rexcludes)
695 aexcludes.update(rexcludes)
695 aprofiles.update(rprofiles)
696 aprofiles.update(rprofiles)
696
697
697 # import rules on top; only take in rules that are not yet
698 # import rules on top; only take in rules that are not yet
698 # part of the active rules.
699 # part of the active rules.
699 changed = False
700 changed = False
700 for file in files:
701 for file in files:
701 with util.posixfile(util.expandpath(file)) as importfile:
702 with util.posixfile(util.expandpath(file)) as importfile:
702 iincludes, iexcludes, iprofiles = sparse.parseconfig(
703 iincludes, iexcludes, iprofiles = sparse.parseconfig(
703 ui, importfile.read())
704 ui, importfile.read())
704 oldsize = len(includes) + len(excludes) + len(profiles)
705 oldsize = len(includes) + len(excludes) + len(profiles)
705 includes.update(iincludes - aincludes)
706 includes.update(iincludes - aincludes)
706 excludes.update(iexcludes - aexcludes)
707 excludes.update(iexcludes - aexcludes)
707 profiles.update(set(iprofiles) - aprofiles)
708 profiles.update(set(iprofiles) - aprofiles)
708 if len(includes) + len(excludes) + len(profiles) > oldsize:
709 if len(includes) + len(excludes) + len(profiles) > oldsize:
709 changed = True
710 changed = True
710
711
711 profilecount = includecount = excludecount = 0
712 profilecount = includecount = excludecount = 0
712 fcounts = (0, 0, 0)
713 fcounts = (0, 0, 0)
713
714
714 if changed:
715 if changed:
715 profilecount = len(profiles - aprofiles)
716 profilecount = len(profiles - aprofiles)
716 includecount = len(includes - aincludes)
717 includecount = len(includes - aincludes)
717 excludecount = len(excludes - aexcludes)
718 excludecount = len(excludes - aexcludes)
718
719
719 oldstatus = repo.status()
720 oldstatus = repo.status()
720 oldsparsematch = repo.sparsematch()
721 oldsparsematch = repo.sparsematch()
721 sparse.writeconfig(repo, includes, excludes, profiles)
722 sparse.writeconfig(repo, includes, excludes, profiles)
722
723
723 try:
724 try:
724 fcounts = map(
725 fcounts = map(
725 len, _refresh(ui, repo, oldstatus, oldsparsematch, force))
726 len, _refresh(ui, repo, oldstatus, oldsparsematch, force))
726 except Exception:
727 except Exception:
727 sparse.writeconfig(repo, oincludes, oexcludes, oprofiles)
728 sparse.writeconfig(repo, oincludes, oexcludes, oprofiles)
728 raise
729 raise
729
730
730 _verbose_output(ui, opts, profilecount, includecount, excludecount,
731 _verbose_output(ui, opts, profilecount, includecount, excludecount,
731 *fcounts)
732 *fcounts)
732
733
733 def _clear(ui, repo, files, force=False):
734 def _clear(ui, repo, files, force=False):
734 with repo.wlock():
735 with repo.wlock():
735 raw = repo.vfs.tryread('sparse')
736 raw = repo.vfs.tryread('sparse')
736 includes, excludes, profiles = sparse.parseconfig(ui, raw)
737 includes, excludes, profiles = sparse.parseconfig(ui, raw)
737
738
738 if includes or excludes:
739 if includes or excludes:
739 oldstatus = repo.status()
740 oldstatus = repo.status()
740 oldsparsematch = repo.sparsematch()
741 oldsparsematch = repo.sparsematch()
741 sparse.writeconfig(repo, set(), set(), profiles)
742 sparse.writeconfig(repo, set(), set(), profiles)
742 _refresh(ui, repo, oldstatus, oldsparsematch, force)
743 _refresh(ui, repo, oldstatus, oldsparsematch, force)
743
744
744 def _refresh(ui, repo, origstatus, origsparsematch, force):
745 def _refresh(ui, repo, origstatus, origsparsematch, force):
745 """Refreshes which files are on disk by comparing the old status and
746 """Refreshes which files are on disk by comparing the old status and
746 sparsematch with the new sparsematch.
747 sparsematch with the new sparsematch.
747
748
748 Will raise an exception if a file with pending changes is being excluded
749 Will raise an exception if a file with pending changes is being excluded
749 or included (unless force=True).
750 or included (unless force=True).
750 """
751 """
751 modified, added, removed, deleted, unknown, ignored, clean = origstatus
752 modified, added, removed, deleted, unknown, ignored, clean = origstatus
752
753
753 # Verify there are no pending changes
754 # Verify there are no pending changes
754 pending = set()
755 pending = set()
755 pending.update(modified)
756 pending.update(modified)
756 pending.update(added)
757 pending.update(added)
757 pending.update(removed)
758 pending.update(removed)
758 sparsematch = repo.sparsematch()
759 sparsematch = repo.sparsematch()
759 abort = False
760 abort = False
760 for file in pending:
761 for file in pending:
761 if not sparsematch(file):
762 if not sparsematch(file):
762 ui.warn(_("pending changes to '%s'\n") % file)
763 ui.warn(_("pending changes to '%s'\n") % file)
763 abort = not force
764 abort = not force
764 if abort:
765 if abort:
765 raise error.Abort(_("could not update sparseness due to " +
766 raise error.Abort(_("could not update sparseness due to " +
766 "pending changes"))
767 "pending changes"))
767
768
768 # Calculate actions
769 # Calculate actions
769 dirstate = repo.dirstate
770 dirstate = repo.dirstate
770 ctx = repo['.']
771 ctx = repo['.']
771 added = []
772 added = []
772 lookup = []
773 lookup = []
773 dropped = []
774 dropped = []
774 mf = ctx.manifest()
775 mf = ctx.manifest()
775 files = set(mf)
776 files = set(mf)
776
777
777 actions = {}
778 actions = {}
778
779
779 for file in files:
780 for file in files:
780 old = origsparsematch(file)
781 old = origsparsematch(file)
781 new = sparsematch(file)
782 new = sparsematch(file)
782 # Add files that are newly included, or that don't exist in
783 # Add files that are newly included, or that don't exist in
783 # the dirstate yet.
784 # the dirstate yet.
784 if (new and not old) or (old and new and not file in dirstate):
785 if (new and not old) or (old and new and not file in dirstate):
785 fl = mf.flags(file)
786 fl = mf.flags(file)
786 if repo.wvfs.exists(file):
787 if repo.wvfs.exists(file):
787 actions[file] = ('e', (fl,), '')
788 actions[file] = ('e', (fl,), '')
788 lookup.append(file)
789 lookup.append(file)
789 else:
790 else:
790 actions[file] = ('g', (fl, False), '')
791 actions[file] = ('g', (fl, False), '')
791 added.append(file)
792 added.append(file)
792 # Drop files that are newly excluded, or that still exist in
793 # Drop files that are newly excluded, or that still exist in
793 # the dirstate.
794 # the dirstate.
794 elif (old and not new) or (not old and not new and file in dirstate):
795 elif (old and not new) or (not old and not new and file in dirstate):
795 dropped.append(file)
796 dropped.append(file)
796 if file not in pending:
797 if file not in pending:
797 actions[file] = ('r', [], '')
798 actions[file] = ('r', [], '')
798
799
799 # Verify there are no pending changes in newly included files
800 # Verify there are no pending changes in newly included files
800 abort = False
801 abort = False
801 for file in lookup:
802 for file in lookup:
802 ui.warn(_("pending changes to '%s'\n") % file)
803 ui.warn(_("pending changes to '%s'\n") % file)
803 abort = not force
804 abort = not force
804 if abort:
805 if abort:
805 raise error.Abort(_("cannot change sparseness due to " +
806 raise error.Abort(_("cannot change sparseness due to " +
806 "pending changes (delete the files or use --force " +
807 "pending changes (delete the files or use --force " +
807 "to bring them back dirty)"))
808 "to bring them back dirty)"))
808
809
809 # Check for files that were only in the dirstate.
810 # Check for files that were only in the dirstate.
810 for file, state in dirstate.iteritems():
811 for file, state in dirstate.iteritems():
811 if not file in files:
812 if not file in files:
812 old = origsparsematch(file)
813 old = origsparsematch(file)
813 new = sparsematch(file)
814 new = sparsematch(file)
814 if old and not new:
815 if old and not new:
815 dropped.append(file)
816 dropped.append(file)
816
817
817 # Apply changes to disk
818 # Apply changes to disk
818 typeactions = dict((m, []) for m in 'a f g am cd dc r dm dg m e k'.split())
819 typeactions = dict((m, []) for m in 'a f g am cd dc r dm dg m e k'.split())
819 for f, (m, args, msg) in actions.iteritems():
820 for f, (m, args, msg) in actions.iteritems():
820 if m not in typeactions:
821 if m not in typeactions:
821 typeactions[m] = []
822 typeactions[m] = []
822 typeactions[m].append((f, args, msg))
823 typeactions[m].append((f, args, msg))
823 mergemod.applyupdates(repo, typeactions, repo[None], repo['.'], False)
824 mergemod.applyupdates(repo, typeactions, repo[None], repo['.'], False)
824
825
825 # Fix dirstate
826 # Fix dirstate
826 for file in added:
827 for file in added:
827 dirstate.normal(file)
828 dirstate.normal(file)
828
829
829 for file in dropped:
830 for file in dropped:
830 dirstate.drop(file)
831 dirstate.drop(file)
831
832
832 for file in lookup:
833 for file in lookup:
833 # File exists on disk, and we're bringing it back in an unknown state.
834 # File exists on disk, and we're bringing it back in an unknown state.
834 dirstate.normallookup(file)
835 dirstate.normallookup(file)
835
836
836 return added, dropped, lookup
837 return added, dropped, lookup
837
838
838 def _verbose_output(ui, opts, profilecount, includecount, excludecount, added,
839 def _verbose_output(ui, opts, profilecount, includecount, excludecount, added,
839 dropped, lookup):
840 dropped, lookup):
840 """Produce --verbose and templatable output
841 """Produce --verbose and templatable output
841
842
842 This specifically enables -Tjson, providing machine-readable stats on how
843 This specifically enables -Tjson, providing machine-readable stats on how
843 the sparse profile changed.
844 the sparse profile changed.
844
845
845 """
846 """
846 with ui.formatter('sparse', opts) as fm:
847 with ui.formatter('sparse', opts) as fm:
847 fm.startitem()
848 fm.startitem()
848 fm.condwrite(ui.verbose, 'profiles_added', 'Profile # change: %d\n',
849 fm.condwrite(ui.verbose, 'profiles_added', 'Profile # change: %d\n',
849 profilecount)
850 profilecount)
850 fm.condwrite(ui.verbose, 'include_rules_added',
851 fm.condwrite(ui.verbose, 'include_rules_added',
851 'Include rule # change: %d\n', includecount)
852 'Include rule # change: %d\n', includecount)
852 fm.condwrite(ui.verbose, 'exclude_rules_added',
853 fm.condwrite(ui.verbose, 'exclude_rules_added',
853 'Exclude rule # change: %d\n', excludecount)
854 'Exclude rule # change: %d\n', excludecount)
854 # In 'plain' verbose mode, mergemod.applyupdates already outputs what
855 # In 'plain' verbose mode, mergemod.applyupdates already outputs what
855 # files are added or removed outside of the templating formatter
856 # files are added or removed outside of the templating formatter
856 # framework. No point in repeating ourselves in that case.
857 # framework. No point in repeating ourselves in that case.
857 if not fm.isplain():
858 if not fm.isplain():
858 fm.condwrite(ui.verbose, 'files_added', 'Files added: %d\n',
859 fm.condwrite(ui.verbose, 'files_added', 'Files added: %d\n',
859 added)
860 added)
860 fm.condwrite(ui.verbose, 'files_dropped', 'Files dropped: %d\n',
861 fm.condwrite(ui.verbose, 'files_dropped', 'Files dropped: %d\n',
861 dropped)
862 dropped)
862 fm.condwrite(ui.verbose, 'files_conflicting',
863 fm.condwrite(ui.verbose, 'files_conflicting',
863 'Files conflicting: %d\n', lookup)
864 'Files conflicting: %d\n', lookup)
864
865 class forceincludematcher(object):
866 """A matcher that returns true for any of the forced includes before testing
867 against the actual matcher."""
868 def __init__(self, matcher, includes):
869 self._matcher = matcher
870 self._includes = includes
871
872 def __call__(self, value):
873 return value in self._includes or self._matcher(value)
874
875 def always(self):
876 return False
877
878 def files(self):
879 return []
880
881 def isexact(self):
882 return False
883
884 def anypats(self):
885 return True
886
887 def prefix(self):
888 return False
889
890 def __repr__(self):
891 return ('<forceincludematcher matcher=%r, includes=%r>' %
892 (self._matcher, sorted(self._includes)))
893
894 class unionmatcher(object):
895 """A matcher that is the union of several matchers."""
896 def __init__(self, matchers):
897 self._matchers = matchers
898
899 def __call__(self, value):
900 for match in self._matchers:
901 if match(value):
902 return True
903 return False
904
905 def always(self):
906 return False
907
908 def files(self):
909 return []
910
911 def isexact(self):
912 return False
913
914 def anypats(self):
915 return True
916
917 def prefix(self):
918 return False
919
920 def __repr__(self):
921 return ('<unionmatcher matchers=%r>' % self._matchers)
922
923 class negatematcher(object):
924 def __init__(self, matcher):
925 self._matcher = matcher
926
927 def __call__(self, value):
928 return not self._matcher(value)
929
930 def always(self):
931 return False
932
933 def files(self):
934 return []
935
936 def isexact(self):
937 return False
938
939 def anypats(self):
940 return True
941
942 def __repr__(self):
943 return ('<negatematcher matcher=%r>' % self._matcher)
@@ -1,983 +1,1036
1 # match.py - filename matching
1 # match.py - filename matching
2 #
2 #
3 # Copyright 2008, 2009 Matt Mackall <mpm@selenic.com> and others
3 # Copyright 2008, 2009 Matt Mackall <mpm@selenic.com> and others
4 #
4 #
5 # This software may be used and distributed according to the terms of the
5 # This software may be used and distributed according to the terms of the
6 # GNU General Public License version 2 or any later version.
6 # GNU General Public License version 2 or any later version.
7
7
8 from __future__ import absolute_import
8 from __future__ import absolute_import
9
9
10 import copy
10 import copy
11 import os
11 import os
12 import re
12 import re
13
13
14 from .i18n import _
14 from .i18n import _
15 from . import (
15 from . import (
16 error,
16 error,
17 pathutil,
17 pathutil,
18 util,
18 util,
19 )
19 )
20
20
21 propertycache = util.propertycache
21 propertycache = util.propertycache
22
22
23 def _rematcher(regex):
23 def _rematcher(regex):
24 '''compile the regexp with the best available regexp engine and return a
24 '''compile the regexp with the best available regexp engine and return a
25 matcher function'''
25 matcher function'''
26 m = util.re.compile(regex)
26 m = util.re.compile(regex)
27 try:
27 try:
28 # slightly faster, provided by facebook's re2 bindings
28 # slightly faster, provided by facebook's re2 bindings
29 return m.test_match
29 return m.test_match
30 except AttributeError:
30 except AttributeError:
31 return m.match
31 return m.match
32
32
33 def _expandsets(kindpats, ctx, listsubrepos):
33 def _expandsets(kindpats, ctx, listsubrepos):
34 '''Returns the kindpats list with the 'set' patterns expanded.'''
34 '''Returns the kindpats list with the 'set' patterns expanded.'''
35 fset = set()
35 fset = set()
36 other = []
36 other = []
37
37
38 for kind, pat, source in kindpats:
38 for kind, pat, source in kindpats:
39 if kind == 'set':
39 if kind == 'set':
40 if not ctx:
40 if not ctx:
41 raise error.ProgrammingError("fileset expression with no "
41 raise error.ProgrammingError("fileset expression with no "
42 "context")
42 "context")
43 s = ctx.getfileset(pat)
43 s = ctx.getfileset(pat)
44 fset.update(s)
44 fset.update(s)
45
45
46 if listsubrepos:
46 if listsubrepos:
47 for subpath in ctx.substate:
47 for subpath in ctx.substate:
48 s = ctx.sub(subpath).getfileset(pat)
48 s = ctx.sub(subpath).getfileset(pat)
49 fset.update(subpath + '/' + f for f in s)
49 fset.update(subpath + '/' + f for f in s)
50
50
51 continue
51 continue
52 other.append((kind, pat, source))
52 other.append((kind, pat, source))
53 return fset, other
53 return fset, other
54
54
55 def _expandsubinclude(kindpats, root):
55 def _expandsubinclude(kindpats, root):
56 '''Returns the list of subinclude matcher args and the kindpats without the
56 '''Returns the list of subinclude matcher args and the kindpats without the
57 subincludes in it.'''
57 subincludes in it.'''
58 relmatchers = []
58 relmatchers = []
59 other = []
59 other = []
60
60
61 for kind, pat, source in kindpats:
61 for kind, pat, source in kindpats:
62 if kind == 'subinclude':
62 if kind == 'subinclude':
63 sourceroot = pathutil.dirname(util.normpath(source))
63 sourceroot = pathutil.dirname(util.normpath(source))
64 pat = util.pconvert(pat)
64 pat = util.pconvert(pat)
65 path = pathutil.join(sourceroot, pat)
65 path = pathutil.join(sourceroot, pat)
66
66
67 newroot = pathutil.dirname(path)
67 newroot = pathutil.dirname(path)
68 matcherargs = (newroot, '', [], ['include:%s' % path])
68 matcherargs = (newroot, '', [], ['include:%s' % path])
69
69
70 prefix = pathutil.canonpath(root, root, newroot)
70 prefix = pathutil.canonpath(root, root, newroot)
71 if prefix:
71 if prefix:
72 prefix += '/'
72 prefix += '/'
73 relmatchers.append((prefix, matcherargs))
73 relmatchers.append((prefix, matcherargs))
74 else:
74 else:
75 other.append((kind, pat, source))
75 other.append((kind, pat, source))
76
76
77 return relmatchers, other
77 return relmatchers, other
78
78
79 def _kindpatsalwaysmatch(kindpats):
79 def _kindpatsalwaysmatch(kindpats):
80 """"Checks whether the kindspats match everything, as e.g.
80 """"Checks whether the kindspats match everything, as e.g.
81 'relpath:.' does.
81 'relpath:.' does.
82 """
82 """
83 for kind, pat, source in kindpats:
83 for kind, pat, source in kindpats:
84 if pat != '' or kind not in ['relpath', 'glob']:
84 if pat != '' or kind not in ['relpath', 'glob']:
85 return False
85 return False
86 return True
86 return True
87
87
88 def match(root, cwd, patterns=None, include=None, exclude=None, default='glob',
88 def match(root, cwd, patterns=None, include=None, exclude=None, default='glob',
89 exact=False, auditor=None, ctx=None, listsubrepos=False, warn=None,
89 exact=False, auditor=None, ctx=None, listsubrepos=False, warn=None,
90 badfn=None, icasefs=False):
90 badfn=None, icasefs=False):
91 """build an object to match a set of file patterns
91 """build an object to match a set of file patterns
92
92
93 arguments:
93 arguments:
94 root - the canonical root of the tree you're matching against
94 root - the canonical root of the tree you're matching against
95 cwd - the current working directory, if relevant
95 cwd - the current working directory, if relevant
96 patterns - patterns to find
96 patterns - patterns to find
97 include - patterns to include (unless they are excluded)
97 include - patterns to include (unless they are excluded)
98 exclude - patterns to exclude (even if they are included)
98 exclude - patterns to exclude (even if they are included)
99 default - if a pattern in patterns has no explicit type, assume this one
99 default - if a pattern in patterns has no explicit type, assume this one
100 exact - patterns are actually filenames (include/exclude still apply)
100 exact - patterns are actually filenames (include/exclude still apply)
101 warn - optional function used for printing warnings
101 warn - optional function used for printing warnings
102 badfn - optional bad() callback for this matcher instead of the default
102 badfn - optional bad() callback for this matcher instead of the default
103 icasefs - make a matcher for wdir on case insensitive filesystems, which
103 icasefs - make a matcher for wdir on case insensitive filesystems, which
104 normalizes the given patterns to the case in the filesystem
104 normalizes the given patterns to the case in the filesystem
105
105
106 a pattern is one of:
106 a pattern is one of:
107 'glob:<glob>' - a glob relative to cwd
107 'glob:<glob>' - a glob relative to cwd
108 're:<regexp>' - a regular expression
108 're:<regexp>' - a regular expression
109 'path:<path>' - a path relative to repository root, which is matched
109 'path:<path>' - a path relative to repository root, which is matched
110 recursively
110 recursively
111 'rootfilesin:<path>' - a path relative to repository root, which is
111 'rootfilesin:<path>' - a path relative to repository root, which is
112 matched non-recursively (will not match subdirectories)
112 matched non-recursively (will not match subdirectories)
113 'relglob:<glob>' - an unrooted glob (*.c matches C files in all dirs)
113 'relglob:<glob>' - an unrooted glob (*.c matches C files in all dirs)
114 'relpath:<path>' - a path relative to cwd
114 'relpath:<path>' - a path relative to cwd
115 'relre:<regexp>' - a regexp that needn't match the start of a name
115 'relre:<regexp>' - a regexp that needn't match the start of a name
116 'set:<fileset>' - a fileset expression
116 'set:<fileset>' - a fileset expression
117 'include:<path>' - a file of patterns to read and include
117 'include:<path>' - a file of patterns to read and include
118 'subinclude:<path>' - a file of patterns to match against files under
118 'subinclude:<path>' - a file of patterns to match against files under
119 the same directory
119 the same directory
120 '<something>' - a pattern of the specified default type
120 '<something>' - a pattern of the specified default type
121 """
121 """
122 normalize = _donormalize
122 normalize = _donormalize
123 if icasefs:
123 if icasefs:
124 if exact:
124 if exact:
125 raise error.ProgrammingError("a case-insensitive exact matcher "
125 raise error.ProgrammingError("a case-insensitive exact matcher "
126 "doesn't make sense")
126 "doesn't make sense")
127 dirstate = ctx.repo().dirstate
127 dirstate = ctx.repo().dirstate
128 dsnormalize = dirstate.normalize
128 dsnormalize = dirstate.normalize
129
129
130 def normalize(patterns, default, root, cwd, auditor, warn):
130 def normalize(patterns, default, root, cwd, auditor, warn):
131 kp = _donormalize(patterns, default, root, cwd, auditor, warn)
131 kp = _donormalize(patterns, default, root, cwd, auditor, warn)
132 kindpats = []
132 kindpats = []
133 for kind, pats, source in kp:
133 for kind, pats, source in kp:
134 if kind not in ('re', 'relre'): # regex can't be normalized
134 if kind not in ('re', 'relre'): # regex can't be normalized
135 p = pats
135 p = pats
136 pats = dsnormalize(pats)
136 pats = dsnormalize(pats)
137
137
138 # Preserve the original to handle a case only rename.
138 # Preserve the original to handle a case only rename.
139 if p != pats and p in dirstate:
139 if p != pats and p in dirstate:
140 kindpats.append((kind, p, source))
140 kindpats.append((kind, p, source))
141
141
142 kindpats.append((kind, pats, source))
142 kindpats.append((kind, pats, source))
143 return kindpats
143 return kindpats
144
144
145 if exact:
145 if exact:
146 m = exactmatcher(root, cwd, patterns, badfn)
146 m = exactmatcher(root, cwd, patterns, badfn)
147 elif patterns:
147 elif patterns:
148 kindpats = normalize(patterns, default, root, cwd, auditor, warn)
148 kindpats = normalize(patterns, default, root, cwd, auditor, warn)
149 if _kindpatsalwaysmatch(kindpats):
149 if _kindpatsalwaysmatch(kindpats):
150 m = alwaysmatcher(root, cwd, badfn, relativeuipath=True)
150 m = alwaysmatcher(root, cwd, badfn, relativeuipath=True)
151 else:
151 else:
152 m = patternmatcher(root, cwd, kindpats, ctx=ctx,
152 m = patternmatcher(root, cwd, kindpats, ctx=ctx,
153 listsubrepos=listsubrepos, badfn=badfn)
153 listsubrepos=listsubrepos, badfn=badfn)
154 else:
154 else:
155 # It's a little strange that no patterns means to match everything.
155 # It's a little strange that no patterns means to match everything.
156 # Consider changing this to match nothing (probably using nevermatcher).
156 # Consider changing this to match nothing (probably using nevermatcher).
157 m = alwaysmatcher(root, cwd, badfn)
157 m = alwaysmatcher(root, cwd, badfn)
158
158
159 if include:
159 if include:
160 kindpats = normalize(include, 'glob', root, cwd, auditor, warn)
160 kindpats = normalize(include, 'glob', root, cwd, auditor, warn)
161 im = includematcher(root, cwd, kindpats, ctx=ctx,
161 im = includematcher(root, cwd, kindpats, ctx=ctx,
162 listsubrepos=listsubrepos, badfn=None)
162 listsubrepos=listsubrepos, badfn=None)
163 m = intersectmatchers(m, im)
163 m = intersectmatchers(m, im)
164 if exclude:
164 if exclude:
165 kindpats = normalize(exclude, 'glob', root, cwd, auditor, warn)
165 kindpats = normalize(exclude, 'glob', root, cwd, auditor, warn)
166 em = includematcher(root, cwd, kindpats, ctx=ctx,
166 em = includematcher(root, cwd, kindpats, ctx=ctx,
167 listsubrepos=listsubrepos, badfn=None)
167 listsubrepos=listsubrepos, badfn=None)
168 m = differencematcher(m, em)
168 m = differencematcher(m, em)
169 return m
169 return m
170
170
171 def exact(root, cwd, files, badfn=None):
171 def exact(root, cwd, files, badfn=None):
172 return exactmatcher(root, cwd, files, badfn=badfn)
172 return exactmatcher(root, cwd, files, badfn=badfn)
173
173
174 def always(root, cwd):
174 def always(root, cwd):
175 return alwaysmatcher(root, cwd)
175 return alwaysmatcher(root, cwd)
176
176
177 def never(root, cwd):
177 def never(root, cwd):
178 return nevermatcher(root, cwd)
178 return nevermatcher(root, cwd)
179
179
180 def badmatch(match, badfn):
180 def badmatch(match, badfn):
181 """Make a copy of the given matcher, replacing its bad method with the given
181 """Make a copy of the given matcher, replacing its bad method with the given
182 one.
182 one.
183 """
183 """
184 m = copy.copy(match)
184 m = copy.copy(match)
185 m.bad = badfn
185 m.bad = badfn
186 return m
186 return m
187
187
188 def _donormalize(patterns, default, root, cwd, auditor, warn):
188 def _donormalize(patterns, default, root, cwd, auditor, warn):
189 '''Convert 'kind:pat' from the patterns list to tuples with kind and
189 '''Convert 'kind:pat' from the patterns list to tuples with kind and
190 normalized and rooted patterns and with listfiles expanded.'''
190 normalized and rooted patterns and with listfiles expanded.'''
191 kindpats = []
191 kindpats = []
192 for kind, pat in [_patsplit(p, default) for p in patterns]:
192 for kind, pat in [_patsplit(p, default) for p in patterns]:
193 if kind in ('glob', 'relpath'):
193 if kind in ('glob', 'relpath'):
194 pat = pathutil.canonpath(root, cwd, pat, auditor)
194 pat = pathutil.canonpath(root, cwd, pat, auditor)
195 elif kind in ('relglob', 'path', 'rootfilesin'):
195 elif kind in ('relglob', 'path', 'rootfilesin'):
196 pat = util.normpath(pat)
196 pat = util.normpath(pat)
197 elif kind in ('listfile', 'listfile0'):
197 elif kind in ('listfile', 'listfile0'):
198 try:
198 try:
199 files = util.readfile(pat)
199 files = util.readfile(pat)
200 if kind == 'listfile0':
200 if kind == 'listfile0':
201 files = files.split('\0')
201 files = files.split('\0')
202 else:
202 else:
203 files = files.splitlines()
203 files = files.splitlines()
204 files = [f for f in files if f]
204 files = [f for f in files if f]
205 except EnvironmentError:
205 except EnvironmentError:
206 raise error.Abort(_("unable to read file list (%s)") % pat)
206 raise error.Abort(_("unable to read file list (%s)") % pat)
207 for k, p, source in _donormalize(files, default, root, cwd,
207 for k, p, source in _donormalize(files, default, root, cwd,
208 auditor, warn):
208 auditor, warn):
209 kindpats.append((k, p, pat))
209 kindpats.append((k, p, pat))
210 continue
210 continue
211 elif kind == 'include':
211 elif kind == 'include':
212 try:
212 try:
213 fullpath = os.path.join(root, util.localpath(pat))
213 fullpath = os.path.join(root, util.localpath(pat))
214 includepats = readpatternfile(fullpath, warn)
214 includepats = readpatternfile(fullpath, warn)
215 for k, p, source in _donormalize(includepats, default,
215 for k, p, source in _donormalize(includepats, default,
216 root, cwd, auditor, warn):
216 root, cwd, auditor, warn):
217 kindpats.append((k, p, source or pat))
217 kindpats.append((k, p, source or pat))
218 except error.Abort as inst:
218 except error.Abort as inst:
219 raise error.Abort('%s: %s' % (pat, inst[0]))
219 raise error.Abort('%s: %s' % (pat, inst[0]))
220 except IOError as inst:
220 except IOError as inst:
221 if warn:
221 if warn:
222 warn(_("skipping unreadable pattern file '%s': %s\n") %
222 warn(_("skipping unreadable pattern file '%s': %s\n") %
223 (pat, inst.strerror))
223 (pat, inst.strerror))
224 continue
224 continue
225 # else: re or relre - which cannot be normalized
225 # else: re or relre - which cannot be normalized
226 kindpats.append((kind, pat, ''))
226 kindpats.append((kind, pat, ''))
227 return kindpats
227 return kindpats
228
228
229 class basematcher(object):
229 class basematcher(object):
230
230
231 def __init__(self, root, cwd, badfn=None, relativeuipath=True):
231 def __init__(self, root, cwd, badfn=None, relativeuipath=True):
232 self._root = root
232 self._root = root
233 self._cwd = cwd
233 self._cwd = cwd
234 if badfn is not None:
234 if badfn is not None:
235 self.bad = badfn
235 self.bad = badfn
236 self._relativeuipath = relativeuipath
236 self._relativeuipath = relativeuipath
237
237
238 def __call__(self, fn):
238 def __call__(self, fn):
239 return self.matchfn(fn)
239 return self.matchfn(fn)
240 def __iter__(self):
240 def __iter__(self):
241 for f in self._files:
241 for f in self._files:
242 yield f
242 yield f
243 # Callbacks related to how the matcher is used by dirstate.walk.
243 # Callbacks related to how the matcher is used by dirstate.walk.
244 # Subscribers to these events must monkeypatch the matcher object.
244 # Subscribers to these events must monkeypatch the matcher object.
245 def bad(self, f, msg):
245 def bad(self, f, msg):
246 '''Callback from dirstate.walk for each explicit file that can't be
246 '''Callback from dirstate.walk for each explicit file that can't be
247 found/accessed, with an error message.'''
247 found/accessed, with an error message.'''
248 pass
248 pass
249
249
250 # If an explicitdir is set, it will be called when an explicitly listed
250 # If an explicitdir is set, it will be called when an explicitly listed
251 # directory is visited.
251 # directory is visited.
252 explicitdir = None
252 explicitdir = None
253
253
254 # If an traversedir is set, it will be called when a directory discovered
254 # If an traversedir is set, it will be called when a directory discovered
255 # by recursive traversal is visited.
255 # by recursive traversal is visited.
256 traversedir = None
256 traversedir = None
257
257
258 def abs(self, f):
258 def abs(self, f):
259 '''Convert a repo path back to path that is relative to the root of the
259 '''Convert a repo path back to path that is relative to the root of the
260 matcher.'''
260 matcher.'''
261 return f
261 return f
262
262
263 def rel(self, f):
263 def rel(self, f):
264 '''Convert repo path back to path that is relative to cwd of matcher.'''
264 '''Convert repo path back to path that is relative to cwd of matcher.'''
265 return util.pathto(self._root, self._cwd, f)
265 return util.pathto(self._root, self._cwd, f)
266
266
267 def uipath(self, f):
267 def uipath(self, f):
268 '''Convert repo path to a display path. If patterns or -I/-X were used
268 '''Convert repo path to a display path. If patterns or -I/-X were used
269 to create this matcher, the display path will be relative to cwd.
269 to create this matcher, the display path will be relative to cwd.
270 Otherwise it is relative to the root of the repo.'''
270 Otherwise it is relative to the root of the repo.'''
271 return (self._relativeuipath and self.rel(f)) or self.abs(f)
271 return (self._relativeuipath and self.rel(f)) or self.abs(f)
272
272
273 @propertycache
273 @propertycache
274 def _files(self):
274 def _files(self):
275 return []
275 return []
276
276
277 def files(self):
277 def files(self):
278 '''Explicitly listed files or patterns or roots:
278 '''Explicitly listed files or patterns or roots:
279 if no patterns or .always(): empty list,
279 if no patterns or .always(): empty list,
280 if exact: list exact files,
280 if exact: list exact files,
281 if not .anypats(): list all files and dirs,
281 if not .anypats(): list all files and dirs,
282 else: optimal roots'''
282 else: optimal roots'''
283 return self._files
283 return self._files
284
284
285 @propertycache
285 @propertycache
286 def _fileset(self):
286 def _fileset(self):
287 return set(self._files)
287 return set(self._files)
288
288
289 def exact(self, f):
289 def exact(self, f):
290 '''Returns True if f is in .files().'''
290 '''Returns True if f is in .files().'''
291 return f in self._fileset
291 return f in self._fileset
292
292
293 def matchfn(self, f):
293 def matchfn(self, f):
294 return False
294 return False
295
295
296 def visitdir(self, dir):
296 def visitdir(self, dir):
297 '''Decides whether a directory should be visited based on whether it
297 '''Decides whether a directory should be visited based on whether it
298 has potential matches in it or one of its subdirectories. This is
298 has potential matches in it or one of its subdirectories. This is
299 based on the match's primary, included, and excluded patterns.
299 based on the match's primary, included, and excluded patterns.
300
300
301 Returns the string 'all' if the given directory and all subdirectories
301 Returns the string 'all' if the given directory and all subdirectories
302 should be visited. Otherwise returns True or False indicating whether
302 should be visited. Otherwise returns True or False indicating whether
303 the given directory should be visited.
303 the given directory should be visited.
304
304
305 This function's behavior is undefined if it has returned False for
305 This function's behavior is undefined if it has returned False for
306 one of the dir's parent directories.
306 one of the dir's parent directories.
307 '''
307 '''
308 return False
308 return False
309
309
310 def anypats(self):
310 def anypats(self):
311 '''Matcher uses patterns or include/exclude.'''
311 '''Matcher uses patterns or include/exclude.'''
312 return False
312 return False
313
313
314 def always(self):
314 def always(self):
315 '''Matcher will match everything and .files() will be empty
315 '''Matcher will match everything and .files() will be empty
316 - optimization might be possible and necessary.'''
316 - optimization might be possible and necessary.'''
317 return False
317 return False
318
318
319 def isexact(self):
319 def isexact(self):
320 return False
320 return False
321
321
322 def prefix(self):
322 def prefix(self):
323 return not self.always() and not self.isexact() and not self.anypats()
323 return not self.always() and not self.isexact() and not self.anypats()
324
324
325 class alwaysmatcher(basematcher):
325 class alwaysmatcher(basematcher):
326 '''Matches everything.'''
326 '''Matches everything.'''
327
327
328 def __init__(self, root, cwd, badfn=None, relativeuipath=False):
328 def __init__(self, root, cwd, badfn=None, relativeuipath=False):
329 super(alwaysmatcher, self).__init__(root, cwd, badfn,
329 super(alwaysmatcher, self).__init__(root, cwd, badfn,
330 relativeuipath=relativeuipath)
330 relativeuipath=relativeuipath)
331
331
332 def always(self):
332 def always(self):
333 return True
333 return True
334
334
335 def matchfn(self, f):
335 def matchfn(self, f):
336 return True
336 return True
337
337
338 def visitdir(self, dir):
338 def visitdir(self, dir):
339 return 'all'
339 return 'all'
340
340
341 def __repr__(self):
341 def __repr__(self):
342 return '<alwaysmatcher>'
342 return '<alwaysmatcher>'
343
343
344 class nevermatcher(basematcher):
344 class nevermatcher(basematcher):
345 '''Matches nothing.'''
345 '''Matches nothing.'''
346
346
347 def __init__(self, root, cwd, badfn=None):
347 def __init__(self, root, cwd, badfn=None):
348 super(nevermatcher, self).__init__(root, cwd, badfn)
348 super(nevermatcher, self).__init__(root, cwd, badfn)
349
349
350 def __repr__(self):
350 def __repr__(self):
351 return '<nevermatcher>'
351 return '<nevermatcher>'
352
352
353 class patternmatcher(basematcher):
353 class patternmatcher(basematcher):
354
354
355 def __init__(self, root, cwd, kindpats, ctx=None, listsubrepos=False,
355 def __init__(self, root, cwd, kindpats, ctx=None, listsubrepos=False,
356 badfn=None):
356 badfn=None):
357 super(patternmatcher, self).__init__(root, cwd, badfn)
357 super(patternmatcher, self).__init__(root, cwd, badfn)
358
358
359 self._files = _explicitfiles(kindpats)
359 self._files = _explicitfiles(kindpats)
360 self._anypats = _anypats(kindpats)
360 self._anypats = _anypats(kindpats)
361 self._pats, self.matchfn = _buildmatch(ctx, kindpats, '$', listsubrepos,
361 self._pats, self.matchfn = _buildmatch(ctx, kindpats, '$', listsubrepos,
362 root)
362 root)
363
363
364 @propertycache
364 @propertycache
365 def _dirs(self):
365 def _dirs(self):
366 return set(util.dirs(self._fileset)) | {'.'}
366 return set(util.dirs(self._fileset)) | {'.'}
367
367
368 def visitdir(self, dir):
368 def visitdir(self, dir):
369 if self.prefix() and dir in self._fileset:
369 if self.prefix() and dir in self._fileset:
370 return 'all'
370 return 'all'
371 return ('.' in self._fileset or
371 return ('.' in self._fileset or
372 dir in self._fileset or
372 dir in self._fileset or
373 dir in self._dirs or
373 dir in self._dirs or
374 any(parentdir in self._fileset
374 any(parentdir in self._fileset
375 for parentdir in util.finddirs(dir)))
375 for parentdir in util.finddirs(dir)))
376
376
377 def anypats(self):
377 def anypats(self):
378 return self._anypats
378 return self._anypats
379
379
380 def __repr__(self):
380 def __repr__(self):
381 return ('<patternmatcher patterns=%r>' % self._pats)
381 return ('<patternmatcher patterns=%r>' % self._pats)
382
382
383 class includematcher(basematcher):
383 class includematcher(basematcher):
384
384
385 def __init__(self, root, cwd, kindpats, ctx=None, listsubrepos=False,
385 def __init__(self, root, cwd, kindpats, ctx=None, listsubrepos=False,
386 badfn=None):
386 badfn=None):
387 super(includematcher, self).__init__(root, cwd, badfn)
387 super(includematcher, self).__init__(root, cwd, badfn)
388
388
389 self._pats, self.matchfn = _buildmatch(ctx, kindpats, '(?:/|$)',
389 self._pats, self.matchfn = _buildmatch(ctx, kindpats, '(?:/|$)',
390 listsubrepos, root)
390 listsubrepos, root)
391 self._anypats = _anypats(kindpats)
391 self._anypats = _anypats(kindpats)
392 roots, dirs = _rootsanddirs(kindpats)
392 roots, dirs = _rootsanddirs(kindpats)
393 # roots are directories which are recursively included.
393 # roots are directories which are recursively included.
394 self._roots = set(roots)
394 self._roots = set(roots)
395 # dirs are directories which are non-recursively included.
395 # dirs are directories which are non-recursively included.
396 self._dirs = set(dirs)
396 self._dirs = set(dirs)
397
397
398 def visitdir(self, dir):
398 def visitdir(self, dir):
399 if not self._anypats and dir in self._roots:
399 if not self._anypats and dir in self._roots:
400 # The condition above is essentially self.prefix() for includes
400 # The condition above is essentially self.prefix() for includes
401 return 'all'
401 return 'all'
402 return ('.' in self._roots or
402 return ('.' in self._roots or
403 dir in self._roots or
403 dir in self._roots or
404 dir in self._dirs or
404 dir in self._dirs or
405 any(parentdir in self._roots
405 any(parentdir in self._roots
406 for parentdir in util.finddirs(dir)))
406 for parentdir in util.finddirs(dir)))
407
407
408 def anypats(self):
408 def anypats(self):
409 return True
409 return True
410
410
411 def __repr__(self):
411 def __repr__(self):
412 return ('<includematcher includes=%r>' % self._pats)
412 return ('<includematcher includes=%r>' % self._pats)
413
413
414 class exactmatcher(basematcher):
414 class exactmatcher(basematcher):
415 '''Matches the input files exactly. They are interpreted as paths, not
415 '''Matches the input files exactly. They are interpreted as paths, not
416 patterns (so no kind-prefixes).
416 patterns (so no kind-prefixes).
417 '''
417 '''
418
418
419 def __init__(self, root, cwd, files, badfn=None):
419 def __init__(self, root, cwd, files, badfn=None):
420 super(exactmatcher, self).__init__(root, cwd, badfn)
420 super(exactmatcher, self).__init__(root, cwd, badfn)
421
421
422 if isinstance(files, list):
422 if isinstance(files, list):
423 self._files = files
423 self._files = files
424 else:
424 else:
425 self._files = list(files)
425 self._files = list(files)
426
426
427 matchfn = basematcher.exact
427 matchfn = basematcher.exact
428
428
429 @propertycache
429 @propertycache
430 def _dirs(self):
430 def _dirs(self):
431 return set(util.dirs(self._fileset)) | {'.'}
431 return set(util.dirs(self._fileset)) | {'.'}
432
432
433 def visitdir(self, dir):
433 def visitdir(self, dir):
434 return dir in self._dirs
434 return dir in self._dirs
435
435
436 def isexact(self):
436 def isexact(self):
437 return True
437 return True
438
438
439 def __repr__(self):
439 def __repr__(self):
440 return ('<exactmatcher files=%r>' % self._files)
440 return ('<exactmatcher files=%r>' % self._files)
441
441
442 class differencematcher(basematcher):
442 class differencematcher(basematcher):
443 '''Composes two matchers by matching if the first matches and the second
443 '''Composes two matchers by matching if the first matches and the second
444 does not. Well, almost... If the user provides a pattern like "-X foo foo",
444 does not. Well, almost... If the user provides a pattern like "-X foo foo",
445 Mercurial actually does match "foo" against that. That's because exact
445 Mercurial actually does match "foo" against that. That's because exact
446 matches are treated specially. So, since this differencematcher is used for
446 matches are treated specially. So, since this differencematcher is used for
447 excludes, it needs to special-case exact matching.
447 excludes, it needs to special-case exact matching.
448
448
449 The second matcher's non-matching-attributes (root, cwd, bad, explicitdir,
449 The second matcher's non-matching-attributes (root, cwd, bad, explicitdir,
450 traversedir) are ignored.
450 traversedir) are ignored.
451
451
452 TODO: If we want to keep the behavior described above for exact matches, we
452 TODO: If we want to keep the behavior described above for exact matches, we
453 should consider instead treating the above case something like this:
453 should consider instead treating the above case something like this:
454 union(exact(foo), difference(pattern(foo), include(foo)))
454 union(exact(foo), difference(pattern(foo), include(foo)))
455 '''
455 '''
456 def __init__(self, m1, m2):
456 def __init__(self, m1, m2):
457 super(differencematcher, self).__init__(m1._root, m1._cwd)
457 super(differencematcher, self).__init__(m1._root, m1._cwd)
458 self._m1 = m1
458 self._m1 = m1
459 self._m2 = m2
459 self._m2 = m2
460 self.bad = m1.bad
460 self.bad = m1.bad
461 self.explicitdir = m1.explicitdir
461 self.explicitdir = m1.explicitdir
462 self.traversedir = m1.traversedir
462 self.traversedir = m1.traversedir
463
463
464 def matchfn(self, f):
464 def matchfn(self, f):
465 return self._m1(f) and (not self._m2(f) or self._m1.exact(f))
465 return self._m1(f) and (not self._m2(f) or self._m1.exact(f))
466
466
467 @propertycache
467 @propertycache
468 def _files(self):
468 def _files(self):
469 if self.isexact():
469 if self.isexact():
470 return [f for f in self._m1.files() if self(f)]
470 return [f for f in self._m1.files() if self(f)]
471 # If m1 is not an exact matcher, we can't easily figure out the set of
471 # If m1 is not an exact matcher, we can't easily figure out the set of
472 # files, because its files() are not always files. For example, if
472 # files, because its files() are not always files. For example, if
473 # m1 is "path:dir" and m2 is "rootfileins:.", we don't
473 # m1 is "path:dir" and m2 is "rootfileins:.", we don't
474 # want to remove "dir" from the set even though it would match m2,
474 # want to remove "dir" from the set even though it would match m2,
475 # because the "dir" in m1 may not be a file.
475 # because the "dir" in m1 may not be a file.
476 return self._m1.files()
476 return self._m1.files()
477
477
478 def visitdir(self, dir):
478 def visitdir(self, dir):
479 if self._m2.visitdir(dir) == 'all':
479 if self._m2.visitdir(dir) == 'all':
480 # There's a bug here: If m1 matches file 'dir/file' and m2 excludes
480 # There's a bug here: If m1 matches file 'dir/file' and m2 excludes
481 # 'dir' (recursively), we should still visit 'dir' due to the
481 # 'dir' (recursively), we should still visit 'dir' due to the
482 # exception we have for exact matches.
482 # exception we have for exact matches.
483 return False
483 return False
484 return bool(self._m1.visitdir(dir))
484 return bool(self._m1.visitdir(dir))
485
485
486 def isexact(self):
486 def isexact(self):
487 return self._m1.isexact()
487 return self._m1.isexact()
488
488
489 def anypats(self):
489 def anypats(self):
490 return self._m1.anypats() or self._m2.anypats()
490 return self._m1.anypats() or self._m2.anypats()
491
491
492 def __repr__(self):
492 def __repr__(self):
493 return ('<differencematcher m1=%r, m2=%r>' % (self._m1, self._m2))
493 return ('<differencematcher m1=%r, m2=%r>' % (self._m1, self._m2))
494
494
495 def intersectmatchers(m1, m2):
495 def intersectmatchers(m1, m2):
496 '''Composes two matchers by matching if both of them match.
496 '''Composes two matchers by matching if both of them match.
497
497
498 The second matcher's non-matching-attributes (root, cwd, bad, explicitdir,
498 The second matcher's non-matching-attributes (root, cwd, bad, explicitdir,
499 traversedir) are ignored.
499 traversedir) are ignored.
500 '''
500 '''
501 if m1 is None or m2 is None:
501 if m1 is None or m2 is None:
502 return m1 or m2
502 return m1 or m2
503 if m1.always():
503 if m1.always():
504 m = copy.copy(m2)
504 m = copy.copy(m2)
505 # TODO: Consider encapsulating these things in a class so there's only
505 # TODO: Consider encapsulating these things in a class so there's only
506 # one thing to copy from m1.
506 # one thing to copy from m1.
507 m.bad = m1.bad
507 m.bad = m1.bad
508 m.explicitdir = m1.explicitdir
508 m.explicitdir = m1.explicitdir
509 m.traversedir = m1.traversedir
509 m.traversedir = m1.traversedir
510 m.abs = m1.abs
510 m.abs = m1.abs
511 m.rel = m1.rel
511 m.rel = m1.rel
512 m._relativeuipath |= m1._relativeuipath
512 m._relativeuipath |= m1._relativeuipath
513 return m
513 return m
514 if m2.always():
514 if m2.always():
515 m = copy.copy(m1)
515 m = copy.copy(m1)
516 m._relativeuipath |= m2._relativeuipath
516 m._relativeuipath |= m2._relativeuipath
517 return m
517 return m
518 return intersectionmatcher(m1, m2)
518 return intersectionmatcher(m1, m2)
519
519
520 class intersectionmatcher(basematcher):
520 class intersectionmatcher(basematcher):
521 def __init__(self, m1, m2):
521 def __init__(self, m1, m2):
522 super(intersectionmatcher, self).__init__(m1._root, m1._cwd)
522 super(intersectionmatcher, self).__init__(m1._root, m1._cwd)
523 self._m1 = m1
523 self._m1 = m1
524 self._m2 = m2
524 self._m2 = m2
525 self.bad = m1.bad
525 self.bad = m1.bad
526 self.explicitdir = m1.explicitdir
526 self.explicitdir = m1.explicitdir
527 self.traversedir = m1.traversedir
527 self.traversedir = m1.traversedir
528
528
529 @propertycache
529 @propertycache
530 def _files(self):
530 def _files(self):
531 if self.isexact():
531 if self.isexact():
532 m1, m2 = self._m1, self._m2
532 m1, m2 = self._m1, self._m2
533 if not m1.isexact():
533 if not m1.isexact():
534 m1, m2 = m2, m1
534 m1, m2 = m2, m1
535 return [f for f in m1.files() if m2(f)]
535 return [f for f in m1.files() if m2(f)]
536 # It neither m1 nor m2 is an exact matcher, we can't easily intersect
536 # It neither m1 nor m2 is an exact matcher, we can't easily intersect
537 # the set of files, because their files() are not always files. For
537 # the set of files, because their files() are not always files. For
538 # example, if intersecting a matcher "-I glob:foo.txt" with matcher of
538 # example, if intersecting a matcher "-I glob:foo.txt" with matcher of
539 # "path:dir2", we don't want to remove "dir2" from the set.
539 # "path:dir2", we don't want to remove "dir2" from the set.
540 return self._m1.files() + self._m2.files()
540 return self._m1.files() + self._m2.files()
541
541
542 def matchfn(self, f):
542 def matchfn(self, f):
543 return self._m1(f) and self._m2(f)
543 return self._m1(f) and self._m2(f)
544
544
545 def visitdir(self, dir):
545 def visitdir(self, dir):
546 visit1 = self._m1.visitdir(dir)
546 visit1 = self._m1.visitdir(dir)
547 if visit1 == 'all':
547 if visit1 == 'all':
548 return self._m2.visitdir(dir)
548 return self._m2.visitdir(dir)
549 # bool() because visit1=True + visit2='all' should not be 'all'
549 # bool() because visit1=True + visit2='all' should not be 'all'
550 return bool(visit1 and self._m2.visitdir(dir))
550 return bool(visit1 and self._m2.visitdir(dir))
551
551
552 def always(self):
552 def always(self):
553 return self._m1.always() and self._m2.always()
553 return self._m1.always() and self._m2.always()
554
554
555 def isexact(self):
555 def isexact(self):
556 return self._m1.isexact() or self._m2.isexact()
556 return self._m1.isexact() or self._m2.isexact()
557
557
558 def anypats(self):
558 def anypats(self):
559 return self._m1.anypats() or self._m2.anypats()
559 return self._m1.anypats() or self._m2.anypats()
560
560
561 def __repr__(self):
561 def __repr__(self):
562 return ('<intersectionmatcher m1=%r, m2=%r>' % (self._m1, self._m2))
562 return ('<intersectionmatcher m1=%r, m2=%r>' % (self._m1, self._m2))
563
563
564 class subdirmatcher(basematcher):
564 class subdirmatcher(basematcher):
565 """Adapt a matcher to work on a subdirectory only.
565 """Adapt a matcher to work on a subdirectory only.
566
566
567 The paths are remapped to remove/insert the path as needed:
567 The paths are remapped to remove/insert the path as needed:
568
568
569 >>> m1 = match('root', '', ['a.txt', 'sub/b.txt'])
569 >>> m1 = match('root', '', ['a.txt', 'sub/b.txt'])
570 >>> m2 = subdirmatcher('sub', m1)
570 >>> m2 = subdirmatcher('sub', m1)
571 >>> bool(m2('a.txt'))
571 >>> bool(m2('a.txt'))
572 False
572 False
573 >>> bool(m2('b.txt'))
573 >>> bool(m2('b.txt'))
574 True
574 True
575 >>> bool(m2.matchfn('a.txt'))
575 >>> bool(m2.matchfn('a.txt'))
576 False
576 False
577 >>> bool(m2.matchfn('b.txt'))
577 >>> bool(m2.matchfn('b.txt'))
578 True
578 True
579 >>> m2.files()
579 >>> m2.files()
580 ['b.txt']
580 ['b.txt']
581 >>> m2.exact('b.txt')
581 >>> m2.exact('b.txt')
582 True
582 True
583 >>> util.pconvert(m2.rel('b.txt'))
583 >>> util.pconvert(m2.rel('b.txt'))
584 'sub/b.txt'
584 'sub/b.txt'
585 >>> def bad(f, msg):
585 >>> def bad(f, msg):
586 ... print "%s: %s" % (f, msg)
586 ... print "%s: %s" % (f, msg)
587 >>> m1.bad = bad
587 >>> m1.bad = bad
588 >>> m2.bad('x.txt', 'No such file')
588 >>> m2.bad('x.txt', 'No such file')
589 sub/x.txt: No such file
589 sub/x.txt: No such file
590 >>> m2.abs('c.txt')
590 >>> m2.abs('c.txt')
591 'sub/c.txt'
591 'sub/c.txt'
592 """
592 """
593
593
594 def __init__(self, path, matcher):
594 def __init__(self, path, matcher):
595 super(subdirmatcher, self).__init__(matcher._root, matcher._cwd)
595 super(subdirmatcher, self).__init__(matcher._root, matcher._cwd)
596 self._path = path
596 self._path = path
597 self._matcher = matcher
597 self._matcher = matcher
598 self._always = matcher.always()
598 self._always = matcher.always()
599
599
600 self._files = [f[len(path) + 1:] for f in matcher._files
600 self._files = [f[len(path) + 1:] for f in matcher._files
601 if f.startswith(path + "/")]
601 if f.startswith(path + "/")]
602
602
603 # If the parent repo had a path to this subrepo and the matcher is
603 # If the parent repo had a path to this subrepo and the matcher is
604 # a prefix matcher, this submatcher always matches.
604 # a prefix matcher, this submatcher always matches.
605 if matcher.prefix():
605 if matcher.prefix():
606 self._always = any(f == path for f in matcher._files)
606 self._always = any(f == path for f in matcher._files)
607
607
608 def bad(self, f, msg):
608 def bad(self, f, msg):
609 self._matcher.bad(self._path + "/" + f, msg)
609 self._matcher.bad(self._path + "/" + f, msg)
610
610
611 def abs(self, f):
611 def abs(self, f):
612 return self._matcher.abs(self._path + "/" + f)
612 return self._matcher.abs(self._path + "/" + f)
613
613
614 def rel(self, f):
614 def rel(self, f):
615 return self._matcher.rel(self._path + "/" + f)
615 return self._matcher.rel(self._path + "/" + f)
616
616
617 def uipath(self, f):
617 def uipath(self, f):
618 return self._matcher.uipath(self._path + "/" + f)
618 return self._matcher.uipath(self._path + "/" + f)
619
619
620 def matchfn(self, f):
620 def matchfn(self, f):
621 # Some information is lost in the superclass's constructor, so we
621 # Some information is lost in the superclass's constructor, so we
622 # can not accurately create the matching function for the subdirectory
622 # can not accurately create the matching function for the subdirectory
623 # from the inputs. Instead, we override matchfn() and visitdir() to
623 # from the inputs. Instead, we override matchfn() and visitdir() to
624 # call the original matcher with the subdirectory path prepended.
624 # call the original matcher with the subdirectory path prepended.
625 return self._matcher.matchfn(self._path + "/" + f)
625 return self._matcher.matchfn(self._path + "/" + f)
626
626
627 def visitdir(self, dir):
627 def visitdir(self, dir):
628 if dir == '.':
628 if dir == '.':
629 dir = self._path
629 dir = self._path
630 else:
630 else:
631 dir = self._path + "/" + dir
631 dir = self._path + "/" + dir
632 return self._matcher.visitdir(dir)
632 return self._matcher.visitdir(dir)
633
633
634 def always(self):
634 def always(self):
635 return self._always
635 return self._always
636
636
637 def anypats(self):
637 def anypats(self):
638 return self._matcher.anypats()
638 return self._matcher.anypats()
639
639
640 def __repr__(self):
640 def __repr__(self):
641 return ('<subdirmatcher path=%r, matcher=%r>' %
641 return ('<subdirmatcher path=%r, matcher=%r>' %
642 (self._path, self._matcher))
642 (self._path, self._matcher))
643
643
644 class forceincludematcher(basematcher):
645 """A matcher that returns true for any of the forced includes before testing
646 against the actual matcher."""
647 def __init__(self, matcher, includes):
648 self._matcher = matcher
649 self._includes = includes
650
651 def __call__(self, value):
652 return value in self._includes or self._matcher(value)
653
654 def anypats(self):
655 return True
656
657 def prefix(self):
658 return False
659
660 def __repr__(self):
661 return ('<forceincludematcher matcher=%r, includes=%r>' %
662 (self._matcher, sorted(self._includes)))
663
664 class unionmatcher(basematcher):
665 """A matcher that is the union of several matchers."""
666 def __init__(self, matchers):
667 self._matchers = matchers
668
669 def __call__(self, value):
670 for match in self._matchers:
671 if match(value):
672 return True
673 return False
674
675 def anypats(self):
676 return True
677
678 def prefix(self):
679 return False
680
681 def __repr__(self):
682 return ('<unionmatcher matchers=%r>' % self._matchers)
683
684 class negatematcher(basematcher):
685 def __init__(self, matcher):
686 self._matcher = matcher
687
688 def __call__(self, value):
689 return not self._matcher(value)
690
691 def anypats(self):
692 return True
693
694 def __repr__(self):
695 return ('<negatematcher matcher=%r>' % self._matcher)
696
644 def patkind(pattern, default=None):
697 def patkind(pattern, default=None):
645 '''If pattern is 'kind:pat' with a known kind, return kind.'''
698 '''If pattern is 'kind:pat' with a known kind, return kind.'''
646 return _patsplit(pattern, default)[0]
699 return _patsplit(pattern, default)[0]
647
700
648 def _patsplit(pattern, default):
701 def _patsplit(pattern, default):
649 """Split a string into the optional pattern kind prefix and the actual
702 """Split a string into the optional pattern kind prefix and the actual
650 pattern."""
703 pattern."""
651 if ':' in pattern:
704 if ':' in pattern:
652 kind, pat = pattern.split(':', 1)
705 kind, pat = pattern.split(':', 1)
653 if kind in ('re', 'glob', 'path', 'relglob', 'relpath', 'relre',
706 if kind in ('re', 'glob', 'path', 'relglob', 'relpath', 'relre',
654 'listfile', 'listfile0', 'set', 'include', 'subinclude',
707 'listfile', 'listfile0', 'set', 'include', 'subinclude',
655 'rootfilesin'):
708 'rootfilesin'):
656 return kind, pat
709 return kind, pat
657 return default, pattern
710 return default, pattern
658
711
659 def _globre(pat):
712 def _globre(pat):
660 r'''Convert an extended glob string to a regexp string.
713 r'''Convert an extended glob string to a regexp string.
661
714
662 >>> print _globre(r'?')
715 >>> print _globre(r'?')
663 .
716 .
664 >>> print _globre(r'*')
717 >>> print _globre(r'*')
665 [^/]*
718 [^/]*
666 >>> print _globre(r'**')
719 >>> print _globre(r'**')
667 .*
720 .*
668 >>> print _globre(r'**/a')
721 >>> print _globre(r'**/a')
669 (?:.*/)?a
722 (?:.*/)?a
670 >>> print _globre(r'a/**/b')
723 >>> print _globre(r'a/**/b')
671 a\/(?:.*/)?b
724 a\/(?:.*/)?b
672 >>> print _globre(r'[a*?!^][^b][!c]')
725 >>> print _globre(r'[a*?!^][^b][!c]')
673 [a*?!^][\^b][^c]
726 [a*?!^][\^b][^c]
674 >>> print _globre(r'{a,b}')
727 >>> print _globre(r'{a,b}')
675 (?:a|b)
728 (?:a|b)
676 >>> print _globre(r'.\*\?')
729 >>> print _globre(r'.\*\?')
677 \.\*\?
730 \.\*\?
678 '''
731 '''
679 i, n = 0, len(pat)
732 i, n = 0, len(pat)
680 res = ''
733 res = ''
681 group = 0
734 group = 0
682 escape = util.re.escape
735 escape = util.re.escape
683 def peek():
736 def peek():
684 return i < n and pat[i:i + 1]
737 return i < n and pat[i:i + 1]
685 while i < n:
738 while i < n:
686 c = pat[i:i + 1]
739 c = pat[i:i + 1]
687 i += 1
740 i += 1
688 if c not in '*?[{},\\':
741 if c not in '*?[{},\\':
689 res += escape(c)
742 res += escape(c)
690 elif c == '*':
743 elif c == '*':
691 if peek() == '*':
744 if peek() == '*':
692 i += 1
745 i += 1
693 if peek() == '/':
746 if peek() == '/':
694 i += 1
747 i += 1
695 res += '(?:.*/)?'
748 res += '(?:.*/)?'
696 else:
749 else:
697 res += '.*'
750 res += '.*'
698 else:
751 else:
699 res += '[^/]*'
752 res += '[^/]*'
700 elif c == '?':
753 elif c == '?':
701 res += '.'
754 res += '.'
702 elif c == '[':
755 elif c == '[':
703 j = i
756 j = i
704 if j < n and pat[j:j + 1] in '!]':
757 if j < n and pat[j:j + 1] in '!]':
705 j += 1
758 j += 1
706 while j < n and pat[j:j + 1] != ']':
759 while j < n and pat[j:j + 1] != ']':
707 j += 1
760 j += 1
708 if j >= n:
761 if j >= n:
709 res += '\\['
762 res += '\\['
710 else:
763 else:
711 stuff = pat[i:j].replace('\\','\\\\')
764 stuff = pat[i:j].replace('\\','\\\\')
712 i = j + 1
765 i = j + 1
713 if stuff[0:1] == '!':
766 if stuff[0:1] == '!':
714 stuff = '^' + stuff[1:]
767 stuff = '^' + stuff[1:]
715 elif stuff[0:1] == '^':
768 elif stuff[0:1] == '^':
716 stuff = '\\' + stuff
769 stuff = '\\' + stuff
717 res = '%s[%s]' % (res, stuff)
770 res = '%s[%s]' % (res, stuff)
718 elif c == '{':
771 elif c == '{':
719 group += 1
772 group += 1
720 res += '(?:'
773 res += '(?:'
721 elif c == '}' and group:
774 elif c == '}' and group:
722 res += ')'
775 res += ')'
723 group -= 1
776 group -= 1
724 elif c == ',' and group:
777 elif c == ',' and group:
725 res += '|'
778 res += '|'
726 elif c == '\\':
779 elif c == '\\':
727 p = peek()
780 p = peek()
728 if p:
781 if p:
729 i += 1
782 i += 1
730 res += escape(p)
783 res += escape(p)
731 else:
784 else:
732 res += escape(c)
785 res += escape(c)
733 else:
786 else:
734 res += escape(c)
787 res += escape(c)
735 return res
788 return res
736
789
737 def _regex(kind, pat, globsuffix):
790 def _regex(kind, pat, globsuffix):
738 '''Convert a (normalized) pattern of any kind into a regular expression.
791 '''Convert a (normalized) pattern of any kind into a regular expression.
739 globsuffix is appended to the regexp of globs.'''
792 globsuffix is appended to the regexp of globs.'''
740 if not pat:
793 if not pat:
741 return ''
794 return ''
742 if kind == 're':
795 if kind == 're':
743 return pat
796 return pat
744 if kind == 'path':
797 if kind == 'path':
745 if pat == '.':
798 if pat == '.':
746 return ''
799 return ''
747 return '^' + util.re.escape(pat) + '(?:/|$)'
800 return '^' + util.re.escape(pat) + '(?:/|$)'
748 if kind == 'rootfilesin':
801 if kind == 'rootfilesin':
749 if pat == '.':
802 if pat == '.':
750 escaped = ''
803 escaped = ''
751 else:
804 else:
752 # Pattern is a directory name.
805 # Pattern is a directory name.
753 escaped = util.re.escape(pat) + '/'
806 escaped = util.re.escape(pat) + '/'
754 # Anything after the pattern must be a non-directory.
807 # Anything after the pattern must be a non-directory.
755 return '^' + escaped + '[^/]+$'
808 return '^' + escaped + '[^/]+$'
756 if kind == 'relglob':
809 if kind == 'relglob':
757 return '(?:|.*/)' + _globre(pat) + globsuffix
810 return '(?:|.*/)' + _globre(pat) + globsuffix
758 if kind == 'relpath':
811 if kind == 'relpath':
759 return util.re.escape(pat) + '(?:/|$)'
812 return util.re.escape(pat) + '(?:/|$)'
760 if kind == 'relre':
813 if kind == 'relre':
761 if pat.startswith('^'):
814 if pat.startswith('^'):
762 return pat
815 return pat
763 return '.*' + pat
816 return '.*' + pat
764 return _globre(pat) + globsuffix
817 return _globre(pat) + globsuffix
765
818
766 def _buildmatch(ctx, kindpats, globsuffix, listsubrepos, root):
819 def _buildmatch(ctx, kindpats, globsuffix, listsubrepos, root):
767 '''Return regexp string and a matcher function for kindpats.
820 '''Return regexp string and a matcher function for kindpats.
768 globsuffix is appended to the regexp of globs.'''
821 globsuffix is appended to the regexp of globs.'''
769 matchfuncs = []
822 matchfuncs = []
770
823
771 subincludes, kindpats = _expandsubinclude(kindpats, root)
824 subincludes, kindpats = _expandsubinclude(kindpats, root)
772 if subincludes:
825 if subincludes:
773 submatchers = {}
826 submatchers = {}
774 def matchsubinclude(f):
827 def matchsubinclude(f):
775 for prefix, matcherargs in subincludes:
828 for prefix, matcherargs in subincludes:
776 if f.startswith(prefix):
829 if f.startswith(prefix):
777 mf = submatchers.get(prefix)
830 mf = submatchers.get(prefix)
778 if mf is None:
831 if mf is None:
779 mf = match(*matcherargs)
832 mf = match(*matcherargs)
780 submatchers[prefix] = mf
833 submatchers[prefix] = mf
781
834
782 if mf(f[len(prefix):]):
835 if mf(f[len(prefix):]):
783 return True
836 return True
784 return False
837 return False
785 matchfuncs.append(matchsubinclude)
838 matchfuncs.append(matchsubinclude)
786
839
787 fset, kindpats = _expandsets(kindpats, ctx, listsubrepos)
840 fset, kindpats = _expandsets(kindpats, ctx, listsubrepos)
788 if fset:
841 if fset:
789 matchfuncs.append(fset.__contains__)
842 matchfuncs.append(fset.__contains__)
790
843
791 regex = ''
844 regex = ''
792 if kindpats:
845 if kindpats:
793 regex, mf = _buildregexmatch(kindpats, globsuffix)
846 regex, mf = _buildregexmatch(kindpats, globsuffix)
794 matchfuncs.append(mf)
847 matchfuncs.append(mf)
795
848
796 if len(matchfuncs) == 1:
849 if len(matchfuncs) == 1:
797 return regex, matchfuncs[0]
850 return regex, matchfuncs[0]
798 else:
851 else:
799 return regex, lambda f: any(mf(f) for mf in matchfuncs)
852 return regex, lambda f: any(mf(f) for mf in matchfuncs)
800
853
801 def _buildregexmatch(kindpats, globsuffix):
854 def _buildregexmatch(kindpats, globsuffix):
802 """Build a match function from a list of kinds and kindpats,
855 """Build a match function from a list of kinds and kindpats,
803 return regexp string and a matcher function."""
856 return regexp string and a matcher function."""
804 try:
857 try:
805 regex = '(?:%s)' % '|'.join([_regex(k, p, globsuffix)
858 regex = '(?:%s)' % '|'.join([_regex(k, p, globsuffix)
806 for (k, p, s) in kindpats])
859 for (k, p, s) in kindpats])
807 if len(regex) > 20000:
860 if len(regex) > 20000:
808 raise OverflowError
861 raise OverflowError
809 return regex, _rematcher(regex)
862 return regex, _rematcher(regex)
810 except OverflowError:
863 except OverflowError:
811 # We're using a Python with a tiny regex engine and we
864 # We're using a Python with a tiny regex engine and we
812 # made it explode, so we'll divide the pattern list in two
865 # made it explode, so we'll divide the pattern list in two
813 # until it works
866 # until it works
814 l = len(kindpats)
867 l = len(kindpats)
815 if l < 2:
868 if l < 2:
816 raise
869 raise
817 regexa, a = _buildregexmatch(kindpats[:l//2], globsuffix)
870 regexa, a = _buildregexmatch(kindpats[:l//2], globsuffix)
818 regexb, b = _buildregexmatch(kindpats[l//2:], globsuffix)
871 regexb, b = _buildregexmatch(kindpats[l//2:], globsuffix)
819 return regex, lambda s: a(s) or b(s)
872 return regex, lambda s: a(s) or b(s)
820 except re.error:
873 except re.error:
821 for k, p, s in kindpats:
874 for k, p, s in kindpats:
822 try:
875 try:
823 _rematcher('(?:%s)' % _regex(k, p, globsuffix))
876 _rematcher('(?:%s)' % _regex(k, p, globsuffix))
824 except re.error:
877 except re.error:
825 if s:
878 if s:
826 raise error.Abort(_("%s: invalid pattern (%s): %s") %
879 raise error.Abort(_("%s: invalid pattern (%s): %s") %
827 (s, k, p))
880 (s, k, p))
828 else:
881 else:
829 raise error.Abort(_("invalid pattern (%s): %s") % (k, p))
882 raise error.Abort(_("invalid pattern (%s): %s") % (k, p))
830 raise error.Abort(_("invalid pattern"))
883 raise error.Abort(_("invalid pattern"))
831
884
832 def _patternrootsanddirs(kindpats):
885 def _patternrootsanddirs(kindpats):
833 '''Returns roots and directories corresponding to each pattern.
886 '''Returns roots and directories corresponding to each pattern.
834
887
835 This calculates the roots and directories exactly matching the patterns and
888 This calculates the roots and directories exactly matching the patterns and
836 returns a tuple of (roots, dirs) for each. It does not return other
889 returns a tuple of (roots, dirs) for each. It does not return other
837 directories which may also need to be considered, like the parent
890 directories which may also need to be considered, like the parent
838 directories.
891 directories.
839 '''
892 '''
840 r = []
893 r = []
841 d = []
894 d = []
842 for kind, pat, source in kindpats:
895 for kind, pat, source in kindpats:
843 if kind == 'glob': # find the non-glob prefix
896 if kind == 'glob': # find the non-glob prefix
844 root = []
897 root = []
845 for p in pat.split('/'):
898 for p in pat.split('/'):
846 if '[' in p or '{' in p or '*' in p or '?' in p:
899 if '[' in p or '{' in p or '*' in p or '?' in p:
847 break
900 break
848 root.append(p)
901 root.append(p)
849 r.append('/'.join(root) or '.')
902 r.append('/'.join(root) or '.')
850 elif kind in ('relpath', 'path'):
903 elif kind in ('relpath', 'path'):
851 r.append(pat or '.')
904 r.append(pat or '.')
852 elif kind in ('rootfilesin',):
905 elif kind in ('rootfilesin',):
853 d.append(pat or '.')
906 d.append(pat or '.')
854 else: # relglob, re, relre
907 else: # relglob, re, relre
855 r.append('.')
908 r.append('.')
856 return r, d
909 return r, d
857
910
858 def _roots(kindpats):
911 def _roots(kindpats):
859 '''Returns root directories to match recursively from the given patterns.'''
912 '''Returns root directories to match recursively from the given patterns.'''
860 roots, dirs = _patternrootsanddirs(kindpats)
913 roots, dirs = _patternrootsanddirs(kindpats)
861 return roots
914 return roots
862
915
863 def _rootsanddirs(kindpats):
916 def _rootsanddirs(kindpats):
864 '''Returns roots and exact directories from patterns.
917 '''Returns roots and exact directories from patterns.
865
918
866 roots are directories to match recursively, whereas exact directories should
919 roots are directories to match recursively, whereas exact directories should
867 be matched non-recursively. The returned (roots, dirs) tuple will also
920 be matched non-recursively. The returned (roots, dirs) tuple will also
868 include directories that need to be implicitly considered as either, such as
921 include directories that need to be implicitly considered as either, such as
869 parent directories.
922 parent directories.
870
923
871 >>> _rootsanddirs(\
924 >>> _rootsanddirs(\
872 [('glob', 'g/h/*', ''), ('glob', 'g/h', ''), ('glob', 'g*', '')])
925 [('glob', 'g/h/*', ''), ('glob', 'g/h', ''), ('glob', 'g*', '')])
873 (['g/h', 'g/h', '.'], ['g', '.'])
926 (['g/h', 'g/h', '.'], ['g', '.'])
874 >>> _rootsanddirs(\
927 >>> _rootsanddirs(\
875 [('rootfilesin', 'g/h', ''), ('rootfilesin', '', '')])
928 [('rootfilesin', 'g/h', ''), ('rootfilesin', '', '')])
876 ([], ['g/h', '.', 'g', '.'])
929 ([], ['g/h', '.', 'g', '.'])
877 >>> _rootsanddirs(\
930 >>> _rootsanddirs(\
878 [('relpath', 'r', ''), ('path', 'p/p', ''), ('path', '', '')])
931 [('relpath', 'r', ''), ('path', 'p/p', ''), ('path', '', '')])
879 (['r', 'p/p', '.'], ['p', '.'])
932 (['r', 'p/p', '.'], ['p', '.'])
880 >>> _rootsanddirs(\
933 >>> _rootsanddirs(\
881 [('relglob', 'rg*', ''), ('re', 're/', ''), ('relre', 'rr', '')])
934 [('relglob', 'rg*', ''), ('re', 're/', ''), ('relre', 'rr', '')])
882 (['.', '.', '.'], ['.'])
935 (['.', '.', '.'], ['.'])
883 '''
936 '''
884 r, d = _patternrootsanddirs(kindpats)
937 r, d = _patternrootsanddirs(kindpats)
885
938
886 # Append the parents as non-recursive/exact directories, since they must be
939 # Append the parents as non-recursive/exact directories, since they must be
887 # scanned to get to either the roots or the other exact directories.
940 # scanned to get to either the roots or the other exact directories.
888 d.extend(util.dirs(d))
941 d.extend(util.dirs(d))
889 d.extend(util.dirs(r))
942 d.extend(util.dirs(r))
890 # util.dirs() does not include the root directory, so add it manually
943 # util.dirs() does not include the root directory, so add it manually
891 d.append('.')
944 d.append('.')
892
945
893 return r, d
946 return r, d
894
947
895 def _explicitfiles(kindpats):
948 def _explicitfiles(kindpats):
896 '''Returns the potential explicit filenames from the patterns.
949 '''Returns the potential explicit filenames from the patterns.
897
950
898 >>> _explicitfiles([('path', 'foo/bar', '')])
951 >>> _explicitfiles([('path', 'foo/bar', '')])
899 ['foo/bar']
952 ['foo/bar']
900 >>> _explicitfiles([('rootfilesin', 'foo/bar', '')])
953 >>> _explicitfiles([('rootfilesin', 'foo/bar', '')])
901 []
954 []
902 '''
955 '''
903 # Keep only the pattern kinds where one can specify filenames (vs only
956 # Keep only the pattern kinds where one can specify filenames (vs only
904 # directory names).
957 # directory names).
905 filable = [kp for kp in kindpats if kp[0] not in ('rootfilesin',)]
958 filable = [kp for kp in kindpats if kp[0] not in ('rootfilesin',)]
906 return _roots(filable)
959 return _roots(filable)
907
960
908 def _anypats(kindpats):
961 def _anypats(kindpats):
909 for kind, pat, source in kindpats:
962 for kind, pat, source in kindpats:
910 if kind in ('glob', 're', 'relglob', 'relre', 'set', 'rootfilesin'):
963 if kind in ('glob', 're', 'relglob', 'relre', 'set', 'rootfilesin'):
911 return True
964 return True
912
965
913 _commentre = None
966 _commentre = None
914
967
915 def readpatternfile(filepath, warn, sourceinfo=False):
968 def readpatternfile(filepath, warn, sourceinfo=False):
916 '''parse a pattern file, returning a list of
969 '''parse a pattern file, returning a list of
917 patterns. These patterns should be given to compile()
970 patterns. These patterns should be given to compile()
918 to be validated and converted into a match function.
971 to be validated and converted into a match function.
919
972
920 trailing white space is dropped.
973 trailing white space is dropped.
921 the escape character is backslash.
974 the escape character is backslash.
922 comments start with #.
975 comments start with #.
923 empty lines are skipped.
976 empty lines are skipped.
924
977
925 lines can be of the following formats:
978 lines can be of the following formats:
926
979
927 syntax: regexp # defaults following lines to non-rooted regexps
980 syntax: regexp # defaults following lines to non-rooted regexps
928 syntax: glob # defaults following lines to non-rooted globs
981 syntax: glob # defaults following lines to non-rooted globs
929 re:pattern # non-rooted regular expression
982 re:pattern # non-rooted regular expression
930 glob:pattern # non-rooted glob
983 glob:pattern # non-rooted glob
931 pattern # pattern of the current default type
984 pattern # pattern of the current default type
932
985
933 if sourceinfo is set, returns a list of tuples:
986 if sourceinfo is set, returns a list of tuples:
934 (pattern, lineno, originalline). This is useful to debug ignore patterns.
987 (pattern, lineno, originalline). This is useful to debug ignore patterns.
935 '''
988 '''
936
989
937 syntaxes = {'re': 'relre:', 'regexp': 'relre:', 'glob': 'relglob:',
990 syntaxes = {'re': 'relre:', 'regexp': 'relre:', 'glob': 'relglob:',
938 'include': 'include', 'subinclude': 'subinclude'}
991 'include': 'include', 'subinclude': 'subinclude'}
939 syntax = 'relre:'
992 syntax = 'relre:'
940 patterns = []
993 patterns = []
941
994
942 fp = open(filepath, 'rb')
995 fp = open(filepath, 'rb')
943 for lineno, line in enumerate(util.iterfile(fp), start=1):
996 for lineno, line in enumerate(util.iterfile(fp), start=1):
944 if "#" in line:
997 if "#" in line:
945 global _commentre
998 global _commentre
946 if not _commentre:
999 if not _commentre:
947 _commentre = util.re.compile(br'((?:^|[^\\])(?:\\\\)*)#.*')
1000 _commentre = util.re.compile(br'((?:^|[^\\])(?:\\\\)*)#.*')
948 # remove comments prefixed by an even number of escapes
1001 # remove comments prefixed by an even number of escapes
949 m = _commentre.search(line)
1002 m = _commentre.search(line)
950 if m:
1003 if m:
951 line = line[:m.end(1)]
1004 line = line[:m.end(1)]
952 # fixup properly escaped comments that survived the above
1005 # fixup properly escaped comments that survived the above
953 line = line.replace("\\#", "#")
1006 line = line.replace("\\#", "#")
954 line = line.rstrip()
1007 line = line.rstrip()
955 if not line:
1008 if not line:
956 continue
1009 continue
957
1010
958 if line.startswith('syntax:'):
1011 if line.startswith('syntax:'):
959 s = line[7:].strip()
1012 s = line[7:].strip()
960 try:
1013 try:
961 syntax = syntaxes[s]
1014 syntax = syntaxes[s]
962 except KeyError:
1015 except KeyError:
963 if warn:
1016 if warn:
964 warn(_("%s: ignoring invalid syntax '%s'\n") %
1017 warn(_("%s: ignoring invalid syntax '%s'\n") %
965 (filepath, s))
1018 (filepath, s))
966 continue
1019 continue
967
1020
968 linesyntax = syntax
1021 linesyntax = syntax
969 for s, rels in syntaxes.iteritems():
1022 for s, rels in syntaxes.iteritems():
970 if line.startswith(rels):
1023 if line.startswith(rels):
971 linesyntax = rels
1024 linesyntax = rels
972 line = line[len(rels):]
1025 line = line[len(rels):]
973 break
1026 break
974 elif line.startswith(s+':'):
1027 elif line.startswith(s+':'):
975 linesyntax = rels
1028 linesyntax = rels
976 line = line[len(s) + 1:]
1029 line = line[len(s) + 1:]
977 break
1030 break
978 if sourceinfo:
1031 if sourceinfo:
979 patterns.append((linesyntax + line, lineno, line))
1032 patterns.append((linesyntax + line, lineno, line))
980 else:
1033 else:
981 patterns.append(linesyntax + line)
1034 patterns.append(linesyntax + line)
982 fp.close()
1035 fp.close()
983 return patterns
1036 return patterns
General Comments 0
You need to be logged in to leave comments. Login now