##// END OF EJS Templates
convert: add --sourcesort option for source specific sort...
Patrick Mezard -
r8690:c5b4f662 default
parent child Browse files
Show More
@@ -1,274 +1,275 b''
1 1 # convert.py Foreign SCM converter
2 2 #
3 3 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
4 4 #
5 5 # This software may be used and distributed according to the terms of the
6 6 # GNU General Public License version 2, incorporated herein by reference.
7 7
8 8 '''converting foreign VCS repositories to Mercurial'''
9 9
10 10 import convcmd
11 11 import cvsps
12 12 import subversion
13 13 from mercurial import commands
14 14 from mercurial.i18n import _
15 15
16 16 # Commands definition was moved elsewhere to ease demandload job.
17 17
18 18 def convert(ui, src, dest=None, revmapfile=None, **opts):
19 19 """convert a foreign SCM repository to a Mercurial one.
20 20
21 21 Accepted source formats [identifiers]:
22 22 - Mercurial [hg]
23 23 - CVS [cvs]
24 24 - Darcs [darcs]
25 25 - git [git]
26 26 - Subversion [svn]
27 27 - Monotone [mtn]
28 28 - GNU Arch [gnuarch]
29 29 - Bazaar [bzr]
30 30 - Perforce [p4]
31 31
32 32 Accepted destination formats [identifiers]:
33 33 - Mercurial [hg]
34 34 - Subversion [svn] (history on branches is not preserved)
35 35
36 36 If no revision is given, all revisions will be converted.
37 37 Otherwise, convert will only import up to the named revision
38 38 (given in a format understood by the source).
39 39
40 40 If no destination directory name is specified, it defaults to the
41 41 basename of the source with '-hg' appended. If the destination
42 42 repository doesn't exist, it will be created.
43 43
44 44 If <REVMAP> isn't given, it will be put in a default location
45 45 (<dest>/.hg/shamap by default). The <REVMAP> is a simple text file
46 46 that maps each source commit ID to the destination ID for that
47 47 revision, like so:
48 48 <source ID> <destination ID>
49 49
50 50 If the file doesn't exist, it's automatically created. It's
51 51 updated on each commit copied, so convert-repo can be interrupted
52 52 and can be run repeatedly to copy new commits.
53 53
54 54 The [username mapping] file is a simple text file that maps each
55 55 source commit author to a destination commit author. It is handy
56 56 for source SCMs that use unix logins to identify authors (eg:
57 57 CVS). One line per author mapping and the line format is:
58 58 srcauthor=whatever string you want
59 59
60 60 The filemap is a file that allows filtering and remapping of files
61 61 and directories. Comment lines start with '#'. Each line can
62 62 contain one of the following directives:
63 63
64 64 include path/to/file
65 65
66 66 exclude path/to/file
67 67
68 68 rename from/file to/file
69 69
70 70 The 'include' directive causes a file, or all files under a
71 71 directory, to be included in the destination repository, and the
72 72 exclusion of all other files and directories not explicitly included.
73 73 The 'exclude' directive causes files or directories to be omitted.
74 74 The 'rename' directive renames a file or directory. To rename from
75 75 a subdirectory into the root of the repository, use '.' as the
76 76 path to rename to.
77 77
78 78 The splicemap is a file that allows insertion of synthetic
79 79 history, letting you specify the parents of a revision. This is
80 80 useful if you want to e.g. give a Subversion merge two parents, or
81 81 graft two disconnected series of history together. Each entry
82 82 contains a key, followed by a space, followed by one or two
83 83 comma-separated values. The key is the revision ID in the source
84 84 revision control system whose parents should be modified (same
85 85 format as a key in .hg/shamap). The values are the revision IDs
86 86 (in either the source or destination revision control system) that
87 87 should be used as the new parents for that node.
88 88
89 89 The branchmap is a file that allows you to rename a branch when it is
90 90 being brought in from whatever external repository. When used in
91 91 conjunction with a splicemap, it allows for a powerful combination
92 92 to help fix even the most badly mismanaged repositories and turn them
93 93 into nicely structured Mercurial repositories. The branchmap contains
94 94 lines of the form "original_branch_name new_branch_name".
95 95 "original_branch_name" is the name of the branch in the source
96 96 repository, and "new_branch_name" is the name of the branch is the
97 97 destination repository. This can be used to (for instance) move code
98 98 in one repository from "default" to a named branch.
99 99
100 100 Mercurial Source
101 101 -----------------
102 102
103 103 --config convert.hg.ignoreerrors=False (boolean)
104 104 ignore integrity errors when reading. Use it to fix Mercurial
105 105 repositories with missing revlogs, by converting from and to
106 106 Mercurial.
107 107 --config convert.hg.saverev=False (boolean)
108 108 store original revision ID in changeset (forces target IDs to
109 109 change)
110 110 --config convert.hg.startrev=0 (hg revision identifier)
111 111 convert start revision and its descendants
112 112
113 113 CVS Source
114 114 ----------
115 115
116 116 CVS source will use a sandbox (i.e. a checked-out copy) from CVS
117 117 to indicate the starting point of what will be converted. Direct
118 118 access to the repository files is not needed, unless of course the
119 119 repository is :local:. The conversion uses the top level directory
120 120 in the sandbox to find the CVS repository, and then uses CVS rlog
121 121 commands to find files to convert. This means that unless a
122 122 filemap is given, all files under the starting directory will be
123 123 converted, and that any directory reorganization in the CVS
124 124 sandbox is ignored.
125 125
126 126 Because CVS does not have changesets, it is necessary to collect
127 127 individual commits to CVS and merge them into changesets. CVS
128 128 source uses its internal changeset merging code by default but can
129 129 be configured to call the external 'cvsps' program by setting:
130 130 --config convert.cvsps='cvsps -A -u --cvs-direct -q'
131 131 This option is deprecated and will be removed in Mercurial 1.4.
132 132
133 133 The options shown are the defaults.
134 134
135 135 Internal cvsps is selected by setting
136 136 --config convert.cvsps=builtin
137 137 and has a few more configurable options:
138 138 --config convert.cvsps.cache=True (boolean)
139 139 Set to False to disable remote log caching, for testing and
140 140 debugging purposes.
141 141 --config convert.cvsps.fuzz=60 (integer)
142 142 Specify the maximum time (in seconds) that is allowed
143 143 between commits with identical user and log message in a
144 144 single changeset. When very large files were checked in as
145 145 part of a changeset then the default may not be long
146 146 enough.
147 147 --config convert.cvsps.mergeto='{{mergetobranch ([-\\w]+)}}'
148 148 Specify a regular expression to which commit log messages
149 149 are matched. If a match occurs, then the conversion
150 150 process will insert a dummy revision merging the branch on
151 151 which this log message occurs to the branch indicated in
152 152 the regex.
153 153 --config convert.cvsps.mergefrom='{{mergefrombranch ([-\\w]+)}}'
154 154 Specify a regular expression to which commit log messages
155 155 are matched. If a match occurs, then the conversion
156 156 process will add the most recent revision on the branch
157 157 indicated in the regex as the second parent of the
158 158 changeset.
159 159
160 160 The hgext/convert/cvsps wrapper script allows the builtin
161 161 changeset merging code to be run without doing a conversion. Its
162 162 parameters and output are similar to that of cvsps 2.1.
163 163
164 164 Subversion Source
165 165 -----------------
166 166
167 167 Subversion source detects classical trunk/branches/tags layouts.
168 168 By default, the supplied "svn://repo/path/" source URL is
169 169 converted as a single branch. If "svn://repo/path/trunk" exists it
170 170 replaces the default branch. If "svn://repo/path/branches" exists,
171 171 its subdirectories are listed as possible branches. If
172 172 "svn://repo/path/tags" exists, it is looked for tags referencing
173 173 converted branches. Default "trunk", "branches" and "tags" values
174 174 can be overridden with following options. Set them to paths
175 175 relative to the source URL, or leave them blank to disable auto
176 176 detection.
177 177
178 178 --config convert.svn.branches=branches (directory name)
179 179 specify the directory containing branches
180 180 --config convert.svn.tags=tags (directory name)
181 181 specify the directory containing tags
182 182 --config convert.svn.trunk=trunk (directory name)
183 183 specify the name of the trunk branch
184 184
185 185 Source history can be retrieved starting at a specific revision,
186 186 instead of being integrally converted. Only single branch
187 187 conversions are supported.
188 188
189 189 --config convert.svn.startrev=0 (svn revision number)
190 190 specify start Subversion revision.
191 191
192 192 Perforce Source
193 193 ---------------
194 194
195 195 The Perforce (P4) importer can be given a p4 depot path or a
196 196 client specification as source. It will convert all files in the
197 197 source to a flat Mercurial repository, ignoring labels, branches
198 198 and integrations. Note that when a depot path is given you then
199 199 usually should specify a target directory, because otherwise the
200 200 target may be named ...-hg.
201 201
202 202 It is possible to limit the amount of source history to be
203 203 converted by specifying an initial Perforce revision.
204 204
205 205 --config convert.p4.startrev=0 (perforce changelist number)
206 206 specify initial Perforce revision.
207 207
208 208
209 209 Mercurial Destination
210 210 ---------------------
211 211
212 212 --config convert.hg.clonebranches=False (boolean)
213 213 dispatch source branches in separate clones.
214 214 --config convert.hg.tagsbranch=default (branch name)
215 215 tag revisions branch name
216 216 --config convert.hg.usebranchnames=True (boolean)
217 217 preserve branch names
218 218
219 219 """
220 220 return convcmd.convert(ui, src, dest, revmapfile, **opts)
221 221
222 222 def debugsvnlog(ui, **opts):
223 223 return subversion.debugsvnlog(ui, **opts)
224 224
225 225 def debugcvsps(ui, *args, **opts):
226 226 '''create changeset information from CVS
227 227
228 228 This command is intended as a debugging tool for the CVS to
229 229 Mercurial converter, and can be used as a direct replacement for
230 230 cvsps.
231 231
232 232 Hg debugcvsps reads the CVS rlog for current directory (or any
233 233 named directory) in the CVS repository, and converts the log to a
234 234 series of changesets based on matching commit log entries and
235 235 dates.'''
236 236 return cvsps.debugcvsps(ui, *args, **opts)
237 237
238 238 commands.norepo += " convert debugsvnlog debugcvsps"
239 239
240 240 cmdtable = {
241 241 "convert":
242 242 (convert,
243 243 [('A', 'authors', '', _('username mapping filename')),
244 244 ('d', 'dest-type', '', _('destination repository type')),
245 245 ('', 'filemap', '', _('remap file names using contents of file')),
246 246 ('r', 'rev', '', _('import up to target revision REV')),
247 247 ('s', 'source-type', '', _('source repository type')),
248 248 ('', 'splicemap', '', _('splice synthesized history into place')),
249 249 ('', 'branchmap', '', _('change branch names while converting')),
250 ('', 'datesort', None, _('try to sort changesets by date'))],
250 ('', 'datesort', None, _('try to sort changesets by date')),
251 ('', 'sourcesort', None, _('preserve source changesets order'))],
251 252 _('hg convert [OPTION]... SOURCE [DEST [REVMAP]]')),
252 253 "debugsvnlog":
253 254 (debugsvnlog,
254 255 [],
255 256 'hg debugsvnlog'),
256 257 "debugcvsps":
257 258 (debugcvsps,
258 259 [
259 260 # Main options shared with cvsps-2.1
260 261 ('b', 'branches', [], _('only return changes on specified branches')),
261 262 ('p', 'prefix', '', _('prefix to remove from file names')),
262 263 ('r', 'revisions', [], _('only return changes after or between specified tags')),
263 264 ('u', 'update-cache', None, _("update cvs log cache")),
264 265 ('x', 'new-cache', None, _("create new cvs log cache")),
265 266 ('z', 'fuzz', 60, _('set commit time fuzz in seconds')),
266 267 ('', 'root', '', _('specify cvsroot')),
267 268 # Options specific to builtin cvsps
268 269 ('', 'parents', '', _('show parent changesets')),
269 270 ('', 'ancestors', '', _('show current changeset in ancestor branches')),
270 271 # Options that are ignored for compatibility with cvsps-2.1
271 272 ('A', 'cvs-direct', None, _('ignored for compatibility')),
272 273 ],
273 274 _('hg debugcvsps [OPTION]... [PATH]...')),
274 275 }
@@ -1,368 +1,369 b''
1 1 # common.py - common code for the convert extension
2 2 #
3 3 # Copyright 2005-2009 Matt Mackall <mpm@selenic.com> and others
4 4 #
5 5 # This software may be used and distributed according to the terms of the
6 6 # GNU General Public License version 2, incorporated herein by reference.
7 7
8 8 import base64, errno
9 9 import os
10 10 import cPickle as pickle
11 11 from mercurial import util
12 12 from mercurial.i18n import _
13 13
14 14 def encodeargs(args):
15 15 def encodearg(s):
16 16 lines = base64.encodestring(s)
17 17 lines = [l.splitlines()[0] for l in lines]
18 18 return ''.join(lines)
19 19
20 20 s = pickle.dumps(args)
21 21 return encodearg(s)
22 22
23 23 def decodeargs(s):
24 24 s = base64.decodestring(s)
25 25 return pickle.loads(s)
26 26
27 27 class MissingTool(Exception): pass
28 28
29 29 def checktool(exe, name=None, abort=True):
30 30 name = name or exe
31 31 if not util.find_exe(exe):
32 32 exc = abort and util.Abort or MissingTool
33 33 raise exc(_('cannot find required "%s" tool') % name)
34 34
35 35 class NoRepo(Exception): pass
36 36
37 37 SKIPREV = 'SKIP'
38 38
39 39 class commit(object):
40 40 def __init__(self, author, date, desc, parents, branch=None, rev=None,
41 extra={}):
41 extra={}, sortkey=None):
42 42 self.author = author or 'unknown'
43 43 self.date = date or '0 0'
44 44 self.desc = desc
45 45 self.parents = parents
46 46 self.branch = branch
47 47 self.rev = rev
48 48 self.extra = extra
49 self.sortkey = sortkey
49 50
50 51 class converter_source(object):
51 52 """Conversion source interface"""
52 53
53 54 def __init__(self, ui, path=None, rev=None):
54 55 """Initialize conversion source (or raise NoRepo("message")
55 56 exception if path is not a valid repository)"""
56 57 self.ui = ui
57 58 self.path = path
58 59 self.rev = rev
59 60
60 61 self.encoding = 'utf-8'
61 62
62 63 def before(self):
63 64 pass
64 65
65 66 def after(self):
66 67 pass
67 68
68 69 def setrevmap(self, revmap):
69 70 """set the map of already-converted revisions"""
70 71 pass
71 72
72 73 def getheads(self):
73 74 """Return a list of this repository's heads"""
74 75 raise NotImplementedError()
75 76
76 77 def getfile(self, name, rev):
77 78 """Return file contents as a string. rev is the identifier returned
78 79 by a previous call to getchanges(). Raise IOError to indicate that
79 80 name was deleted in rev.
80 81 """
81 82 raise NotImplementedError()
82 83
83 84 def getmode(self, name, rev):
84 85 """Return file mode, eg. '', 'x', or 'l'. rev is the identifier
85 86 returned by a previous call to getchanges().
86 87 """
87 88 raise NotImplementedError()
88 89
89 90 def getchanges(self, version):
90 91 """Returns a tuple of (files, copies).
91 92
92 93 files is a sorted list of (filename, id) tuples for all files
93 94 changed between version and its first parent returned by
94 95 getcommit(). id is the source revision id of the file.
95 96
96 97 copies is a dictionary of dest: source
97 98 """
98 99 raise NotImplementedError()
99 100
100 101 def getcommit(self, version):
101 102 """Return the commit object for version"""
102 103 raise NotImplementedError()
103 104
104 105 def gettags(self):
105 106 """Return the tags as a dictionary of name: revision"""
106 107 raise NotImplementedError()
107 108
108 109 def recode(self, s, encoding=None):
109 110 if not encoding:
110 111 encoding = self.encoding or 'utf-8'
111 112
112 113 if isinstance(s, unicode):
113 114 return s.encode("utf-8")
114 115 try:
115 116 return s.decode(encoding).encode("utf-8")
116 117 except:
117 118 try:
118 119 return s.decode("latin-1").encode("utf-8")
119 120 except:
120 121 return s.decode(encoding, "replace").encode("utf-8")
121 122
122 123 def getchangedfiles(self, rev, i):
123 124 """Return the files changed by rev compared to parent[i].
124 125
125 126 i is an index selecting one of the parents of rev. The return
126 127 value should be the list of files that are different in rev and
127 128 this parent.
128 129
129 130 If rev has no parents, i is None.
130 131
131 132 This function is only needed to support --filemap
132 133 """
133 134 raise NotImplementedError()
134 135
135 136 def converted(self, rev, sinkrev):
136 137 '''Notify the source that a revision has been converted.'''
137 138 pass
138 139
139 140
140 141 class converter_sink(object):
141 142 """Conversion sink (target) interface"""
142 143
143 144 def __init__(self, ui, path):
144 145 """Initialize conversion sink (or raise NoRepo("message")
145 146 exception if path is not a valid repository)
146 147
147 148 created is a list of paths to remove if a fatal error occurs
148 149 later"""
149 150 self.ui = ui
150 151 self.path = path
151 152 self.created = []
152 153
153 154 def getheads(self):
154 155 """Return a list of this repository's heads"""
155 156 raise NotImplementedError()
156 157
157 158 def revmapfile(self):
158 159 """Path to a file that will contain lines
159 160 source_rev_id sink_rev_id
160 161 mapping equivalent revision identifiers for each system."""
161 162 raise NotImplementedError()
162 163
163 164 def authorfile(self):
164 165 """Path to a file that will contain lines
165 166 srcauthor=dstauthor
166 167 mapping equivalent authors identifiers for each system."""
167 168 return None
168 169
169 170 def putcommit(self, files, copies, parents, commit, source):
170 171 """Create a revision with all changed files listed in 'files'
171 172 and having listed parents. 'commit' is a commit object containing
172 173 at a minimum the author, date, and message for this changeset.
173 174 'files' is a list of (path, version) tuples, 'copies'is a dictionary
174 175 mapping destinations to sources, and 'source' is the source repository.
175 176 Only getfile() and getmode() should be called on 'source'.
176 177
177 178 Note that the sink repository is not told to update itself to
178 179 a particular revision (or even what that revision would be)
179 180 before it receives the file data.
180 181 """
181 182 raise NotImplementedError()
182 183
183 184 def puttags(self, tags):
184 185 """Put tags into sink.
185 186 tags: {tagname: sink_rev_id, ...}"""
186 187 raise NotImplementedError()
187 188
188 189 def setbranch(self, branch, pbranches):
189 190 """Set the current branch name. Called before the first putcommit
190 191 on the branch.
191 192 branch: branch name for subsequent commits
192 193 pbranches: (converted parent revision, parent branch) tuples"""
193 194 pass
194 195
195 196 def setfilemapmode(self, active):
196 197 """Tell the destination that we're using a filemap
197 198
198 199 Some converter_sources (svn in particular) can claim that a file
199 200 was changed in a revision, even if there was no change. This method
200 201 tells the destination that we're using a filemap and that it should
201 202 filter empty revisions.
202 203 """
203 204 pass
204 205
205 206 def before(self):
206 207 pass
207 208
208 209 def after(self):
209 210 pass
210 211
211 212
212 213 class commandline(object):
213 214 def __init__(self, ui, command):
214 215 self.ui = ui
215 216 self.command = command
216 217
217 218 def prerun(self):
218 219 pass
219 220
220 221 def postrun(self):
221 222 pass
222 223
223 224 def _cmdline(self, cmd, *args, **kwargs):
224 225 cmdline = [self.command, cmd] + list(args)
225 226 for k, v in kwargs.iteritems():
226 227 if len(k) == 1:
227 228 cmdline.append('-' + k)
228 229 else:
229 230 cmdline.append('--' + k.replace('_', '-'))
230 231 try:
231 232 if len(k) == 1:
232 233 cmdline.append('' + v)
233 234 else:
234 235 cmdline[-1] += '=' + v
235 236 except TypeError:
236 237 pass
237 238 cmdline = [util.shellquote(arg) for arg in cmdline]
238 239 if not self.ui.debugflag:
239 240 cmdline += ['2>', util.nulldev]
240 241 cmdline += ['<', util.nulldev]
241 242 cmdline = ' '.join(cmdline)
242 243 return cmdline
243 244
244 245 def _run(self, cmd, *args, **kwargs):
245 246 cmdline = self._cmdline(cmd, *args, **kwargs)
246 247 self.ui.debug(_('running: %s\n') % (cmdline,))
247 248 self.prerun()
248 249 try:
249 250 return util.popen(cmdline)
250 251 finally:
251 252 self.postrun()
252 253
253 254 def run(self, cmd, *args, **kwargs):
254 255 fp = self._run(cmd, *args, **kwargs)
255 256 output = fp.read()
256 257 self.ui.debug(output)
257 258 return output, fp.close()
258 259
259 260 def runlines(self, cmd, *args, **kwargs):
260 261 fp = self._run(cmd, *args, **kwargs)
261 262 output = fp.readlines()
262 263 self.ui.debug(''.join(output))
263 264 return output, fp.close()
264 265
265 266 def checkexit(self, status, output=''):
266 267 if status:
267 268 if output:
268 269 self.ui.warn(_('%s error:\n') % self.command)
269 270 self.ui.warn(output)
270 271 msg = util.explain_exit(status)[0]
271 272 raise util.Abort(_('%s %s') % (self.command, msg))
272 273
273 274 def run0(self, cmd, *args, **kwargs):
274 275 output, status = self.run(cmd, *args, **kwargs)
275 276 self.checkexit(status, output)
276 277 return output
277 278
278 279 def runlines0(self, cmd, *args, **kwargs):
279 280 output, status = self.runlines(cmd, *args, **kwargs)
280 281 self.checkexit(status, ''.join(output))
281 282 return output
282 283
283 284 def getargmax(self):
284 285 if '_argmax' in self.__dict__:
285 286 return self._argmax
286 287
287 288 # POSIX requires at least 4096 bytes for ARG_MAX
288 289 self._argmax = 4096
289 290 try:
290 291 self._argmax = os.sysconf("SC_ARG_MAX")
291 292 except:
292 293 pass
293 294
294 295 # Windows shells impose their own limits on command line length,
295 296 # down to 2047 bytes for cmd.exe under Windows NT/2k and 2500 bytes
296 297 # for older 4nt.exe. See http://support.microsoft.com/kb/830473 for
297 298 # details about cmd.exe limitations.
298 299
299 300 # Since ARG_MAX is for command line _and_ environment, lower our limit
300 301 # (and make happy Windows shells while doing this).
301 302
302 303 self._argmax = self._argmax/2 - 1
303 304 return self._argmax
304 305
305 306 def limit_arglist(self, arglist, cmd, *args, **kwargs):
306 307 limit = self.getargmax() - len(self._cmdline(cmd, *args, **kwargs))
307 308 bytes = 0
308 309 fl = []
309 310 for fn in arglist:
310 311 b = len(fn) + 3
311 312 if bytes + b < limit or len(fl) == 0:
312 313 fl.append(fn)
313 314 bytes += b
314 315 else:
315 316 yield fl
316 317 fl = [fn]
317 318 bytes = b
318 319 if fl:
319 320 yield fl
320 321
321 322 def xargs(self, arglist, cmd, *args, **kwargs):
322 323 for l in self.limit_arglist(arglist, cmd, *args, **kwargs):
323 324 self.run0(cmd, *(list(args) + l), **kwargs)
324 325
325 326 class mapfile(dict):
326 327 def __init__(self, ui, path):
327 328 super(mapfile, self).__init__()
328 329 self.ui = ui
329 330 self.path = path
330 331 self.fp = None
331 332 self.order = []
332 333 self._read()
333 334
334 335 def _read(self):
335 336 if not self.path:
336 337 return
337 338 try:
338 339 fp = open(self.path, 'r')
339 340 except IOError, err:
340 341 if err.errno != errno.ENOENT:
341 342 raise
342 343 return
343 344 for i, line in enumerate(fp):
344 345 try:
345 346 key, value = line[:-1].rsplit(' ', 1)
346 347 except ValueError:
347 348 raise util.Abort(_('syntax error in %s(%d): key/value pair expected')
348 349 % (self.path, i+1))
349 350 if key not in self:
350 351 self.order.append(key)
351 352 super(mapfile, self).__setitem__(key, value)
352 353 fp.close()
353 354
354 355 def __setitem__(self, key, value):
355 356 if self.fp is None:
356 357 try:
357 358 self.fp = open(self.path, 'a')
358 359 except IOError, err:
359 360 raise util.Abort(_('could not open map file %r: %s') %
360 361 (self.path, err.strerror))
361 362 self.fp.write('%s %s\n' % (key, value))
362 363 self.fp.flush()
363 364 super(mapfile, self).__setitem__(key, value)
364 365
365 366 def close(self):
366 367 if self.fp:
367 368 self.fp.close()
368 369 self.fp = None
@@ -1,382 +1,393 b''
1 1 # convcmd - convert extension commands definition
2 2 #
3 3 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
4 4 #
5 5 # This software may be used and distributed according to the terms of the
6 6 # GNU General Public License version 2, incorporated herein by reference.
7 7
8 8 from common import NoRepo, MissingTool, SKIPREV, mapfile
9 9 from cvs import convert_cvs
10 10 from darcs import darcs_source
11 11 from git import convert_git
12 12 from hg import mercurial_source, mercurial_sink
13 13 from subversion import svn_source, svn_sink
14 14 from monotone import monotone_source
15 15 from gnuarch import gnuarch_source
16 16 from bzr import bzr_source
17 17 from p4 import p4_source
18 18 import filemap
19 19
20 20 import os, shutil
21 21 from mercurial import hg, util, encoding
22 22 from mercurial.i18n import _
23 23
24 24 orig_encoding = 'ascii'
25 25
26 26 def recode(s):
27 27 if isinstance(s, unicode):
28 28 return s.encode(orig_encoding, 'replace')
29 29 else:
30 30 return s.decode('utf-8').encode(orig_encoding, 'replace')
31 31
32 32 source_converters = [
33 33 ('cvs', convert_cvs),
34 34 ('git', convert_git),
35 35 ('svn', svn_source),
36 36 ('hg', mercurial_source),
37 37 ('darcs', darcs_source),
38 38 ('mtn', monotone_source),
39 39 ('gnuarch', gnuarch_source),
40 40 ('bzr', bzr_source),
41 41 ('p4', p4_source),
42 42 ]
43 43
44 44 sink_converters = [
45 45 ('hg', mercurial_sink),
46 46 ('svn', svn_sink),
47 47 ]
48 48
49 49 def convertsource(ui, path, type, rev):
50 50 exceptions = []
51 51 for name, source in source_converters:
52 52 try:
53 53 if not type or name == type:
54 54 return source(ui, path, rev)
55 55 except (NoRepo, MissingTool), inst:
56 56 exceptions.append(inst)
57 57 if not ui.quiet:
58 58 for inst in exceptions:
59 59 ui.write("%s\n" % inst)
60 60 raise util.Abort(_('%s: missing or unsupported repository') % path)
61 61
62 62 def convertsink(ui, path, type):
63 63 for name, sink in sink_converters:
64 64 try:
65 65 if not type or name == type:
66 66 return sink(ui, path)
67 67 except NoRepo, inst:
68 68 ui.note(_("convert: %s\n") % inst)
69 69 raise util.Abort(_('%s: unknown repository type') % path)
70 70
71 71 class converter(object):
72 72 def __init__(self, ui, source, dest, revmapfile, opts):
73 73
74 74 self.source = source
75 75 self.dest = dest
76 76 self.ui = ui
77 77 self.opts = opts
78 78 self.commitcache = {}
79 79 self.authors = {}
80 80 self.authorfile = None
81 81
82 82 # Record converted revisions persistently: maps source revision
83 83 # ID to target revision ID (both strings). (This is how
84 84 # incremental conversions work.)
85 85 self.map = mapfile(ui, revmapfile)
86 86
87 87 # Read first the dst author map if any
88 88 authorfile = self.dest.authorfile()
89 89 if authorfile and os.path.exists(authorfile):
90 90 self.readauthormap(authorfile)
91 91 # Extend/Override with new author map if necessary
92 92 if opts.get('authors'):
93 93 self.readauthormap(opts.get('authors'))
94 94 self.authorfile = self.dest.authorfile()
95 95
96 96 self.splicemap = mapfile(ui, opts.get('splicemap'))
97 97 self.branchmap = mapfile(ui, opts.get('branchmap'))
98 98
99 99 def walktree(self, heads):
100 100 '''Return a mapping that identifies the uncommitted parents of every
101 101 uncommitted changeset.'''
102 102 visit = heads
103 103 known = set()
104 104 parents = {}
105 105 while visit:
106 106 n = visit.pop(0)
107 107 if n in known or n in self.map: continue
108 108 known.add(n)
109 109 commit = self.cachecommit(n)
110 110 parents[n] = []
111 111 for p in commit.parents:
112 112 parents[n].append(p)
113 113 visit.append(p)
114 114
115 115 return parents
116 116
117 117 def toposort(self, parents, sortmode):
118 118 '''Return an ordering such that every uncommitted changeset is
119 119 preceeded by all its uncommitted ancestors.'''
120 120
121 121 def mapchildren(parents):
122 122 """Return a (children, roots) tuple where 'children' maps parent
123 123 revision identifiers to children ones, and 'roots' is the list of
124 124 revisions without parents. 'parents' must be a mapping of revision
125 125 identifier to its parents ones.
126 126 """
127 127 visit = parents.keys()
128 128 seen = set()
129 129 children = {}
130 130 roots = []
131 131
132 132 while visit:
133 133 n = visit.pop(0)
134 134 if n in seen:
135 135 continue
136 136 seen.add(n)
137 137 # Ensure that nodes without parents are present in the
138 138 # 'children' mapping.
139 139 children.setdefault(n, [])
140 140 hasparent = False
141 141 for p in parents[n]:
142 142 if not p in self.map:
143 143 visit.append(p)
144 144 hasparent = True
145 145 children.setdefault(p, []).append(n)
146 146 if not hasparent:
147 147 roots.append(n)
148 148
149 149 return children, roots
150 150
151 151 # Sort functions are supposed to take a list of revisions which
152 152 # can be converted immediately and pick one
153 153
154 154 def makebranchsorter():
155 155 """If the previously converted revision has a child in the
156 156 eligible revisions list, pick it. Return the list head
157 157 otherwise. Branch sort attempts to minimize branch
158 158 switching, which is harmful for Mercurial backend
159 159 compression.
160 160 """
161 161 prev = [None]
162 162 def picknext(nodes):
163 163 next = nodes[0]
164 164 for n in nodes:
165 165 if prev[0] in parents[n]:
166 166 next = n
167 167 break
168 168 prev[0] = next
169 169 return next
170 170 return picknext
171 171
172 def makesourcesorter():
173 """Source specific sort."""
174 keyfn = lambda n: self.commitcache[n].sortkey
175 def picknext(nodes):
176 return sorted(nodes, key=keyfn)[0]
177 return picknext
178
172 179 def makedatesorter():
173 180 """Sort revisions by date."""
174 181 dates = {}
175 182 def getdate(n):
176 183 if n not in dates:
177 184 dates[n] = util.parsedate(self.commitcache[n].date)
178 185 return dates[n]
179 186
180 187 def picknext(nodes):
181 188 return min([(getdate(n), n) for n in nodes])[1]
182 189
183 190 return picknext
184 191
185 192 if sortmode == 'branchsort':
186 193 picknext = makebranchsorter()
187 194 elif sortmode == 'datesort':
188 195 picknext = makedatesorter()
196 elif sortmode == 'sourcesort':
197 picknext = makesourcesorter()
189 198 else:
190 199 raise util.Abort(_('unknown sort mode: %s') % sortmode)
191 200
192 201 children, actives = mapchildren(parents)
193 202
194 203 s = []
195 204 pendings = {}
196 205 while actives:
197 206 n = picknext(actives)
198 207 actives.remove(n)
199 208 s.append(n)
200 209
201 210 # Update dependents list
202 211 for c in children.get(n, []):
203 212 if c not in pendings:
204 213 pendings[c] = [p for p in parents[c] if p not in self.map]
205 214 try:
206 215 pendings[c].remove(n)
207 216 except ValueError:
208 217 raise util.Abort(_('cycle detected between %s and %s')
209 218 % (recode(c), recode(n)))
210 219 if not pendings[c]:
211 220 # Parents are converted, node is eligible
212 221 actives.insert(0, c)
213 222 pendings[c] = None
214 223
215 224 if len(s) != len(parents):
216 225 raise util.Abort(_("not all revisions were sorted"))
217 226
218 227 return s
219 228
220 229 def writeauthormap(self):
221 230 authorfile = self.authorfile
222 231 if authorfile:
223 232 self.ui.status(_('Writing author map file %s\n') % authorfile)
224 233 ofile = open(authorfile, 'w+')
225 234 for author in self.authors:
226 235 ofile.write("%s=%s\n" % (author, self.authors[author]))
227 236 ofile.close()
228 237
229 238 def readauthormap(self, authorfile):
230 239 afile = open(authorfile, 'r')
231 240 for line in afile:
232 241
233 242 line = line.strip()
234 243 if not line or line.startswith('#'):
235 244 continue
236 245
237 246 try:
238 247 srcauthor, dstauthor = line.split('=', 1)
239 248 except ValueError:
240 249 msg = _('Ignoring bad line in author map file %s: %s\n')
241 250 self.ui.warn(msg % (authorfile, line.rstrip()))
242 251 continue
243 252
244 253 srcauthor = srcauthor.strip()
245 254 dstauthor = dstauthor.strip()
246 255 if self.authors.get(srcauthor) in (None, dstauthor):
247 256 msg = _('mapping author %s to %s\n')
248 257 self.ui.debug(msg % (srcauthor, dstauthor))
249 258 self.authors[srcauthor] = dstauthor
250 259 continue
251 260
252 261 m = _('overriding mapping for author %s, was %s, will be %s\n')
253 262 self.ui.status(m % (srcauthor, self.authors[srcauthor], dstauthor))
254 263
255 264 afile.close()
256 265
257 266 def cachecommit(self, rev):
258 267 commit = self.source.getcommit(rev)
259 268 commit.author = self.authors.get(commit.author, commit.author)
260 269 commit.branch = self.branchmap.get(commit.branch, commit.branch)
261 270 self.commitcache[rev] = commit
262 271 return commit
263 272
264 273 def copy(self, rev):
265 274 commit = self.commitcache[rev]
266 275
267 276 changes = self.source.getchanges(rev)
268 277 if isinstance(changes, basestring):
269 278 if changes == SKIPREV:
270 279 dest = SKIPREV
271 280 else:
272 281 dest = self.map[changes]
273 282 self.map[rev] = dest
274 283 return
275 284 files, copies = changes
276 285 pbranches = []
277 286 if commit.parents:
278 287 for prev in commit.parents:
279 288 if prev not in self.commitcache:
280 289 self.cachecommit(prev)
281 290 pbranches.append((self.map[prev],
282 291 self.commitcache[prev].branch))
283 292 self.dest.setbranch(commit.branch, pbranches)
284 293 try:
285 294 parents = self.splicemap[rev].replace(',', ' ').split()
286 295 self.ui.status(_('spliced in %s as parents of %s\n') %
287 296 (parents, rev))
288 297 parents = [self.map.get(p, p) for p in parents]
289 298 except KeyError:
290 299 parents = [b[0] for b in pbranches]
291 300 newnode = self.dest.putcommit(files, copies, parents, commit, self.source)
292 301 self.source.converted(rev, newnode)
293 302 self.map[rev] = newnode
294 303
295 304 def convert(self, sortmode):
296 305 try:
297 306 self.source.before()
298 307 self.dest.before()
299 308 self.source.setrevmap(self.map)
300 309 self.ui.status(_("scanning source...\n"))
301 310 heads = self.source.getheads()
302 311 parents = self.walktree(heads)
303 312 self.ui.status(_("sorting...\n"))
304 313 t = self.toposort(parents, sortmode)
305 314 num = len(t)
306 315 c = None
307 316
308 317 self.ui.status(_("converting...\n"))
309 318 for c in t:
310 319 num -= 1
311 320 desc = self.commitcache[c].desc
312 321 if "\n" in desc:
313 322 desc = desc.splitlines()[0]
314 323 # convert log message to local encoding without using
315 324 # tolocal() because encoding.encoding conver() use it as
316 325 # 'utf-8'
317 326 self.ui.status("%d %s\n" % (num, recode(desc)))
318 327 self.ui.note(_("source: %s\n") % recode(c))
319 328 self.copy(c)
320 329
321 330 tags = self.source.gettags()
322 331 ctags = {}
323 332 for k in tags:
324 333 v = tags[k]
325 334 if self.map.get(v, SKIPREV) != SKIPREV:
326 335 ctags[k] = self.map[v]
327 336
328 337 if c and ctags:
329 338 nrev = self.dest.puttags(ctags)
330 339 # write another hash correspondence to override the previous
331 340 # one so we don't end up with extra tag heads
332 341 if nrev:
333 342 self.map[c] = nrev
334 343
335 344 self.writeauthormap()
336 345 finally:
337 346 self.cleanup()
338 347
339 348 def cleanup(self):
340 349 try:
341 350 self.dest.after()
342 351 finally:
343 352 self.source.after()
344 353 self.map.close()
345 354
346 355 def convert(ui, src, dest=None, revmapfile=None, **opts):
347 356 global orig_encoding
348 357 orig_encoding = encoding.encoding
349 358 encoding.encoding = 'UTF-8'
350 359
351 360 if not dest:
352 361 dest = hg.defaultdest(src) + "-hg"
353 362 ui.status(_("assuming destination %s\n") % dest)
354 363
355 364 destc = convertsink(ui, dest, opts.get('dest_type'))
356 365
357 366 try:
358 367 srcc = convertsource(ui, src, opts.get('source_type'),
359 368 opts.get('rev'))
360 369 except Exception:
361 370 for path in destc.created:
362 371 shutil.rmtree(path, True)
363 372 raise
364 373
365 sortmode = 'branchsort'
366 if opts.get('datesort'):
367 sortmode = 'datesort'
374 sortmodes = ('datesort', 'sourcesort')
375 sortmode = [m for m in sortmodes if opts.get(m)]
376 if len(sortmode) > 1:
377 raise util.Abort(_('more than one sort mode specified'))
378 sortmode = sortmode and sortmode[0] or 'branchsort'
368 379
369 380 fmap = opts.get('filemap')
370 381 if fmap:
371 382 srcc = filemap.filemap_source(ui, srcc, fmap)
372 383 destc.setfilemapmode(True)
373 384
374 385 if not revmapfile:
375 386 try:
376 387 revmapfile = destc.revmapfile()
377 388 except:
378 389 revmapfile = os.path.join(destc, "map")
379 390
380 391 c = converter(ui, srcc, destc, revmapfile, opts)
381 392 c.convert(sortmode)
382 393
@@ -1,339 +1,340 b''
1 1 # hg.py - hg backend for convert extension
2 2 #
3 3 # Copyright 2005-2009 Matt Mackall <mpm@selenic.com> and others
4 4 #
5 5 # This software may be used and distributed according to the terms of the
6 6 # GNU General Public License version 2, incorporated herein by reference.
7 7
8 8 # Notes for hg->hg conversion:
9 9 #
10 10 # * Old versions of Mercurial didn't trim the whitespace from the ends
11 11 # of commit messages, but new versions do. Changesets created by
12 12 # those older versions, then converted, may thus have different
13 13 # hashes for changesets that are otherwise identical.
14 14 #
15 15 # * Using "--config convert.hg.saverev=true" will make the source
16 16 # identifier to be stored in the converted revision. This will cause
17 17 # the converted revision to have a different identity than the
18 18 # source.
19 19
20 20
21 21 import os, time
22 22 from mercurial.i18n import _
23 23 from mercurial.node import bin, hex, nullid
24 24 from mercurial import hg, util, context, error
25 25
26 26 from common import NoRepo, commit, converter_source, converter_sink
27 27
28 28 class mercurial_sink(converter_sink):
29 29 def __init__(self, ui, path):
30 30 converter_sink.__init__(self, ui, path)
31 31 self.branchnames = ui.configbool('convert', 'hg.usebranchnames', True)
32 32 self.clonebranches = ui.configbool('convert', 'hg.clonebranches', False)
33 33 self.tagsbranch = ui.config('convert', 'hg.tagsbranch', 'default')
34 34 self.lastbranch = None
35 35 if os.path.isdir(path) and len(os.listdir(path)) > 0:
36 36 try:
37 37 self.repo = hg.repository(self.ui, path)
38 38 if not self.repo.local():
39 39 raise NoRepo(_('%s is not a local Mercurial repo') % path)
40 40 except error.RepoError, err:
41 41 ui.traceback()
42 42 raise NoRepo(err.args[0])
43 43 else:
44 44 try:
45 45 ui.status(_('initializing destination %s repository\n') % path)
46 46 self.repo = hg.repository(self.ui, path, create=True)
47 47 if not self.repo.local():
48 48 raise NoRepo(_('%s is not a local Mercurial repo') % path)
49 49 self.created.append(path)
50 50 except error.RepoError:
51 51 ui.traceback()
52 52 raise NoRepo("could not create hg repo %s as sink" % path)
53 53 self.lock = None
54 54 self.wlock = None
55 55 self.filemapmode = False
56 56
57 57 def before(self):
58 58 self.ui.debug(_('run hg sink pre-conversion action\n'))
59 59 self.wlock = self.repo.wlock()
60 60 self.lock = self.repo.lock()
61 61
62 62 def after(self):
63 63 self.ui.debug(_('run hg sink post-conversion action\n'))
64 64 self.lock.release()
65 65 self.wlock.release()
66 66
67 67 def revmapfile(self):
68 68 return os.path.join(self.path, ".hg", "shamap")
69 69
70 70 def authorfile(self):
71 71 return os.path.join(self.path, ".hg", "authormap")
72 72
73 73 def getheads(self):
74 74 h = self.repo.changelog.heads()
75 75 return [ hex(x) for x in h ]
76 76
77 77 def setbranch(self, branch, pbranches):
78 78 if not self.clonebranches:
79 79 return
80 80
81 81 setbranch = (branch != self.lastbranch)
82 82 self.lastbranch = branch
83 83 if not branch:
84 84 branch = 'default'
85 85 pbranches = [(b[0], b[1] and b[1] or 'default') for b in pbranches]
86 86 pbranch = pbranches and pbranches[0][1] or 'default'
87 87
88 88 branchpath = os.path.join(self.path, branch)
89 89 if setbranch:
90 90 self.after()
91 91 try:
92 92 self.repo = hg.repository(self.ui, branchpath)
93 93 except:
94 94 self.repo = hg.repository(self.ui, branchpath, create=True)
95 95 self.before()
96 96
97 97 # pbranches may bring revisions from other branches (merge parents)
98 98 # Make sure we have them, or pull them.
99 99 missings = {}
100 100 for b in pbranches:
101 101 try:
102 102 self.repo.lookup(b[0])
103 103 except:
104 104 missings.setdefault(b[1], []).append(b[0])
105 105
106 106 if missings:
107 107 self.after()
108 108 for pbranch, heads in missings.iteritems():
109 109 pbranchpath = os.path.join(self.path, pbranch)
110 110 prepo = hg.repository(self.ui, pbranchpath)
111 111 self.ui.note(_('pulling from %s into %s\n') % (pbranch, branch))
112 112 self.repo.pull(prepo, [prepo.lookup(h) for h in heads])
113 113 self.before()
114 114
115 115 def putcommit(self, files, copies, parents, commit, source):
116 116
117 117 files = dict(files)
118 118 def getfilectx(repo, memctx, f):
119 119 v = files[f]
120 120 data = source.getfile(f, v)
121 121 e = source.getmode(f, v)
122 122 return context.memfilectx(f, data, 'l' in e, 'x' in e, copies.get(f))
123 123
124 124 pl = []
125 125 for p in parents:
126 126 if p not in pl:
127 127 pl.append(p)
128 128 parents = pl
129 129 nparents = len(parents)
130 130 if self.filemapmode and nparents == 1:
131 131 m1node = self.repo.changelog.read(bin(parents[0]))[0]
132 132 parent = parents[0]
133 133
134 134 if len(parents) < 2: parents.append(nullid)
135 135 if len(parents) < 2: parents.append(nullid)
136 136 p2 = parents.pop(0)
137 137
138 138 text = commit.desc
139 139 extra = commit.extra.copy()
140 140 if self.branchnames and commit.branch:
141 141 extra['branch'] = commit.branch
142 142 if commit.rev:
143 143 extra['convert_revision'] = commit.rev
144 144
145 145 while parents:
146 146 p1 = p2
147 147 p2 = parents.pop(0)
148 148 ctx = context.memctx(self.repo, (p1, p2), text, files.keys(), getfilectx,
149 149 commit.author, commit.date, extra)
150 150 self.repo.commitctx(ctx)
151 151 text = "(octopus merge fixup)\n"
152 152 p2 = hex(self.repo.changelog.tip())
153 153
154 154 if self.filemapmode and nparents == 1:
155 155 man = self.repo.manifest
156 156 mnode = self.repo.changelog.read(bin(p2))[0]
157 157 if not man.cmp(m1node, man.revision(mnode)):
158 158 self.ui.status(_("filtering out empty revision\n"))
159 159 self.repo.rollback()
160 160 return parent
161 161 return p2
162 162
163 163 def puttags(self, tags):
164 164 try:
165 165 parentctx = self.repo[self.tagsbranch]
166 166 tagparent = parentctx.node()
167 167 except error.RepoError:
168 168 parentctx = None
169 169 tagparent = nullid
170 170
171 171 try:
172 172 oldlines = sorted(parentctx['.hgtags'].data().splitlines(1))
173 173 except:
174 174 oldlines = []
175 175
176 176 newlines = sorted([("%s %s\n" % (tags[tag], tag)) for tag in tags])
177 177 if newlines == oldlines:
178 178 return None
179 179 data = "".join(newlines)
180 180 def getfilectx(repo, memctx, f):
181 181 return context.memfilectx(f, data, False, False, None)
182 182
183 183 self.ui.status(_("updating tags\n"))
184 184 date = "%s 0" % int(time.mktime(time.gmtime()))
185 185 extra = {'branch': self.tagsbranch}
186 186 ctx = context.memctx(self.repo, (tagparent, None), "update tags",
187 187 [".hgtags"], getfilectx, "convert-repo", date,
188 188 extra)
189 189 self.repo.commitctx(ctx)
190 190 return hex(self.repo.changelog.tip())
191 191
192 192 def setfilemapmode(self, active):
193 193 self.filemapmode = active
194 194
195 195 class mercurial_source(converter_source):
196 196 def __init__(self, ui, path, rev=None):
197 197 converter_source.__init__(self, ui, path, rev)
198 198 self.ignoreerrors = ui.configbool('convert', 'hg.ignoreerrors', False)
199 199 self.ignored = set()
200 200 self.saverev = ui.configbool('convert', 'hg.saverev', False)
201 201 try:
202 202 self.repo = hg.repository(self.ui, path)
203 203 # try to provoke an exception if this isn't really a hg
204 204 # repo, but some other bogus compatible-looking url
205 205 if not self.repo.local():
206 206 raise error.RepoError()
207 207 except error.RepoError:
208 208 ui.traceback()
209 209 raise NoRepo("%s is not a local Mercurial repo" % path)
210 210 self.lastrev = None
211 211 self.lastctx = None
212 212 self._changescache = None
213 213 self.convertfp = None
214 214 # Restrict converted revisions to startrev descendants
215 215 startnode = ui.config('convert', 'hg.startrev')
216 216 if startnode is not None:
217 217 try:
218 218 startnode = self.repo.lookup(startnode)
219 219 except error.RepoError:
220 220 raise util.Abort(_('%s is not a valid start revision')
221 221 % startnode)
222 222 startrev = self.repo.changelog.rev(startnode)
223 223 children = {startnode: 1}
224 224 for rev in self.repo.changelog.descendants(startrev):
225 225 children[self.repo.changelog.node(rev)] = 1
226 226 self.keep = children.__contains__
227 227 else:
228 228 self.keep = util.always
229 229
230 230 def changectx(self, rev):
231 231 if self.lastrev != rev:
232 232 self.lastctx = self.repo[rev]
233 233 self.lastrev = rev
234 234 return self.lastctx
235 235
236 236 def parents(self, ctx):
237 237 return [p.node() for p in ctx.parents()
238 238 if p and self.keep(p.node())]
239 239
240 240 def getheads(self):
241 241 if self.rev:
242 242 heads = [self.repo[self.rev].node()]
243 243 else:
244 244 heads = self.repo.heads()
245 245 return [hex(h) for h in heads if self.keep(h)]
246 246
247 247 def getfile(self, name, rev):
248 248 try:
249 249 return self.changectx(rev)[name].data()
250 250 except error.LookupError, err:
251 251 raise IOError(err)
252 252
253 253 def getmode(self, name, rev):
254 254 return self.changectx(rev).manifest().flags(name)
255 255
256 256 def getchanges(self, rev):
257 257 ctx = self.changectx(rev)
258 258 parents = self.parents(ctx)
259 259 if not parents:
260 260 files = sorted(ctx.manifest())
261 261 if self.ignoreerrors:
262 262 # calling getcopies() is a simple way to detect missing
263 263 # revlogs and populate self.ignored
264 264 self.getcopies(ctx, files)
265 265 return [(f, rev) for f in files if f not in self.ignored], {}
266 266 if self._changescache and self._changescache[0] == rev:
267 267 m, a, r = self._changescache[1]
268 268 else:
269 269 m, a, r = self.repo.status(parents[0], ctx.node())[:3]
270 270 # getcopies() detects missing revlogs early, run it before
271 271 # filtering the changes.
272 272 copies = self.getcopies(ctx, m + a)
273 273 changes = [(name, rev) for name in m + a + r
274 274 if name not in self.ignored]
275 275 return sorted(changes), copies
276 276
277 277 def getcopies(self, ctx, files):
278 278 copies = {}
279 279 for name in files:
280 280 if name in self.ignored:
281 281 continue
282 282 try:
283 283 copysource, copynode = ctx.filectx(name).renamed()
284 284 if copysource in self.ignored or not self.keep(copynode):
285 285 continue
286 286 copies[name] = copysource
287 287 except TypeError:
288 288 pass
289 289 except error.LookupError, e:
290 290 if not self.ignoreerrors:
291 291 raise
292 292 self.ignored.add(name)
293 293 self.ui.warn(_('ignoring: %s\n') % e)
294 294 return copies
295 295
296 296 def getcommit(self, rev):
297 297 ctx = self.changectx(rev)
298 298 parents = [hex(p) for p in self.parents(ctx)]
299 299 if self.saverev:
300 300 crev = rev
301 301 else:
302 302 crev = None
303 303 return commit(author=ctx.user(), date=util.datestr(ctx.date()),
304 304 desc=ctx.description(), rev=crev, parents=parents,
305 branch=ctx.branch(), extra=ctx.extra())
305 branch=ctx.branch(), extra=ctx.extra(),
306 sortkey=ctx.rev())
306 307
307 308 def gettags(self):
308 309 tags = [t for t in self.repo.tagslist() if t[0] != 'tip']
309 310 return dict([(name, hex(node)) for name, node in tags
310 311 if self.keep(node)])
311 312
312 313 def getchangedfiles(self, rev, i):
313 314 ctx = self.changectx(rev)
314 315 parents = self.parents(ctx)
315 316 if not parents and i is None:
316 317 i = 0
317 318 changes = [], ctx.manifest().keys(), []
318 319 else:
319 320 i = i or 0
320 321 changes = self.repo.status(parents[i], ctx.node())[:3]
321 322 changes = [[f for f in l if f not in self.ignored] for l in changes]
322 323
323 324 if i == 0:
324 325 self._changescache = (rev, changes)
325 326
326 327 return changes[0] + changes[1] + changes[2]
327 328
328 329 def converted(self, rev, destrev):
329 330 if self.convertfp is None:
330 331 self.convertfp = open(os.path.join(self.path, '.hg', 'shamap'),
331 332 'a')
332 333 self.convertfp.write('%s %s\n' % (destrev, rev))
333 334 self.convertfp.flush()
334 335
335 336 def before(self):
336 337 self.ui.debug(_('run hg source pre-conversion action\n'))
337 338
338 339 def after(self):
339 340 self.ui.debug(_('run hg source post-conversion action\n'))
@@ -1,40 +1,45 b''
1 1 #!/bin/sh
2 2
3 3 cat >> $HGRCPATH <<EOF
4 4 [extensions]
5 5 convert=
6 6 graphlog=
7 7 EOF
8 8
9 9 hg init t
10 10 cd t
11 11 echo a >> a
12 12 hg ci -Am a0 -d '1 0'
13 13 hg branch brancha
14 14 echo a >> a
15 15 hg ci -m a1 -d '2 0'
16 16 echo a >> a
17 17 hg ci -m a2 -d '3 0'
18 18 echo a >> a
19 19 hg ci -m a3 -d '4 0'
20 20 hg up -C 0
21 21 hg branch branchb
22 22 echo b >> b
23 hg ci -Am b0 -d '5 0'
23 hg ci -Am b0 -d '6 0'
24 24 hg up -C brancha
25 25 echo a >> a
26 hg ci -m a4 -d '6 0'
26 hg ci -m a4 -d '5 0'
27 27 echo a >> a
28 28 hg ci -m a5 -d '7 0'
29 29 echo a >> a
30 30 hg ci -m a6 -d '8 0'
31 31 hg up -C branchb
32 32 echo b >> b
33 33 hg ci -m b1 -d '9 0'
34 34 cd ..
35 35
36 36 echo % convert with datesort
37 hg convert --datesort t t2
37 hg convert --datesort t t-datesort
38 38 echo % graph converted repo
39 hg -R t2 glog --template '{rev} "{desc}"\n'
39 hg -R t-datesort glog --template '{rev} "{desc}"\n'
40 40
41 echo % convert with datesort
42 hg convert --sourcesort t t-sourcesort
43 echo % graph converted repo
44 hg -R t-sourcesort glog --template '{rev} "{desc}"\n'
45
@@ -1,41 +1,74 b''
1 1 adding a
2 2 marked working directory as branch brancha
3 3 1 files updated, 0 files merged, 0 files removed, 0 files unresolved
4 4 marked working directory as branch branchb
5 5 adding b
6 6 created new head
7 7 1 files updated, 0 files merged, 1 files removed, 0 files unresolved
8 8 2 files updated, 0 files merged, 0 files removed, 0 files unresolved
9 9 % convert with datesort
10 initializing destination t2 repository
10 initializing destination t-datesort repository
11 scanning source...
12 sorting...
13 converting...
14 8 a0
15 7 a1
16 6 a2
17 5 a3
18 4 a4
19 3 b0
20 2 a5
21 1 a6
22 0 b1
23 % graph converted repo
24 o 8 "b1"
25 |
26 | o 7 "a6"
27 | |
28 | o 6 "a5"
29 | |
30 o | 5 "b0"
31 | |
32 | o 4 "a4"
33 | |
34 | o 3 "a3"
35 | |
36 | o 2 "a2"
37 | |
38 | o 1 "a1"
39 |/
40 o 0 "a0"
41
42 % convert with datesort
43 initializing destination t-sourcesort repository
11 44 scanning source...
12 45 sorting...
13 46 converting...
14 47 8 a0
15 48 7 a1
16 49 6 a2
17 50 5 a3
18 51 4 b0
19 52 3 a4
20 53 2 a5
21 54 1 a6
22 55 0 b1
23 56 % graph converted repo
24 57 o 8 "b1"
25 58 |
26 59 | o 7 "a6"
27 60 | |
28 61 | o 6 "a5"
29 62 | |
30 63 | o 5 "a4"
31 64 | |
32 65 o | 4 "b0"
33 66 | |
34 67 | o 3 "a3"
35 68 | |
36 69 | o 2 "a2"
37 70 | |
38 71 | o 1 "a1"
39 72 |/
40 73 o 0 "a0"
41 74
@@ -1,261 +1,262 b''
1 1 hg convert [OPTION]... SOURCE [DEST [REVMAP]]
2 2
3 3 convert a foreign SCM repository to a Mercurial one.
4 4
5 5 Accepted source formats [identifiers]:
6 6 - Mercurial [hg]
7 7 - CVS [cvs]
8 8 - Darcs [darcs]
9 9 - git [git]
10 10 - Subversion [svn]
11 11 - Monotone [mtn]
12 12 - GNU Arch [gnuarch]
13 13 - Bazaar [bzr]
14 14 - Perforce [p4]
15 15
16 16 Accepted destination formats [identifiers]:
17 17 - Mercurial [hg]
18 18 - Subversion [svn] (history on branches is not preserved)
19 19
20 20 If no revision is given, all revisions will be converted.
21 21 Otherwise, convert will only import up to the named revision
22 22 (given in a format understood by the source).
23 23
24 24 If no destination directory name is specified, it defaults to the
25 25 basename of the source with '-hg' appended. If the destination
26 26 repository doesn't exist, it will be created.
27 27
28 28 If <REVMAP> isn't given, it will be put in a default location
29 29 (<dest>/.hg/shamap by default). The <REVMAP> is a simple text file
30 30 that maps each source commit ID to the destination ID for that
31 31 revision, like so:
32 32 <source ID> <destination ID>
33 33
34 34 If the file doesn't exist, it's automatically created. It's
35 35 updated on each commit copied, so convert-repo can be interrupted
36 36 and can be run repeatedly to copy new commits.
37 37
38 38 The [username mapping] file is a simple text file that maps each
39 39 source commit author to a destination commit author. It is handy
40 40 for source SCMs that use unix logins to identify authors (eg:
41 41 CVS). One line per author mapping and the line format is:
42 42 srcauthor=whatever string you want
43 43
44 44 The filemap is a file that allows filtering and remapping of files
45 45 and directories. Comment lines start with '#'. Each line can
46 46 contain one of the following directives:
47 47
48 48 include path/to/file
49 49
50 50 exclude path/to/file
51 51
52 52 rename from/file to/file
53 53
54 54 The 'include' directive causes a file, or all files under a
55 55 directory, to be included in the destination repository, and the
56 56 exclusion of all other files and directories not explicitly included.
57 57 The 'exclude' directive causes files or directories to be omitted.
58 58 The 'rename' directive renames a file or directory. To rename from
59 59 a subdirectory into the root of the repository, use '.' as the
60 60 path to rename to.
61 61
62 62 The splicemap is a file that allows insertion of synthetic
63 63 history, letting you specify the parents of a revision. This is
64 64 useful if you want to e.g. give a Subversion merge two parents, or
65 65 graft two disconnected series of history together. Each entry
66 66 contains a key, followed by a space, followed by one or two
67 67 comma-separated values. The key is the revision ID in the source
68 68 revision control system whose parents should be modified (same
69 69 format as a key in .hg/shamap). The values are the revision IDs
70 70 (in either the source or destination revision control system) that
71 71 should be used as the new parents for that node.
72 72
73 73 The branchmap is a file that allows you to rename a branch when it is
74 74 being brought in from whatever external repository. When used in
75 75 conjunction with a splicemap, it allows for a powerful combination
76 76 to help fix even the most badly mismanaged repositories and turn them
77 77 into nicely structured Mercurial repositories. The branchmap contains
78 78 lines of the form "original_branch_name new_branch_name".
79 79 "original_branch_name" is the name of the branch in the source
80 80 repository, and "new_branch_name" is the name of the branch is the
81 81 destination repository. This can be used to (for instance) move code
82 82 in one repository from "default" to a named branch.
83 83
84 84 Mercurial Source
85 85 -----------------
86 86
87 87 --config convert.hg.ignoreerrors=False (boolean)
88 88 ignore integrity errors when reading. Use it to fix Mercurial
89 89 repositories with missing revlogs, by converting from and to
90 90 Mercurial.
91 91 --config convert.hg.saverev=False (boolean)
92 92 store original revision ID in changeset (forces target IDs to
93 93 change)
94 94 --config convert.hg.startrev=0 (hg revision identifier)
95 95 convert start revision and its descendants
96 96
97 97 CVS Source
98 98 ----------
99 99
100 100 CVS source will use a sandbox (i.e. a checked-out copy) from CVS
101 101 to indicate the starting point of what will be converted. Direct
102 102 access to the repository files is not needed, unless of course the
103 103 repository is :local:. The conversion uses the top level directory
104 104 in the sandbox to find the CVS repository, and then uses CVS rlog
105 105 commands to find files to convert. This means that unless a
106 106 filemap is given, all files under the starting directory will be
107 107 converted, and that any directory reorganization in the CVS
108 108 sandbox is ignored.
109 109
110 110 Because CVS does not have changesets, it is necessary to collect
111 111 individual commits to CVS and merge them into changesets. CVS
112 112 source uses its internal changeset merging code by default but can
113 113 be configured to call the external 'cvsps' program by setting:
114 114 --config convert.cvsps='cvsps -A -u --cvs-direct -q'
115 115 This option is deprecated and will be removed in Mercurial 1.4.
116 116
117 117 The options shown are the defaults.
118 118
119 119 Internal cvsps is selected by setting
120 120 --config convert.cvsps=builtin
121 121 and has a few more configurable options:
122 122 --config convert.cvsps.cache=True (boolean)
123 123 Set to False to disable remote log caching, for testing and
124 124 debugging purposes.
125 125 --config convert.cvsps.fuzz=60 (integer)
126 126 Specify the maximum time (in seconds) that is allowed
127 127 between commits with identical user and log message in a
128 128 single changeset. When very large files were checked in as
129 129 part of a changeset then the default may not be long
130 130 enough.
131 131 --config convert.cvsps.mergeto='{{mergetobranch ([-\w]+)}}'
132 132 Specify a regular expression to which commit log messages
133 133 are matched. If a match occurs, then the conversion
134 134 process will insert a dummy revision merging the branch on
135 135 which this log message occurs to the branch indicated in
136 136 the regex.
137 137 --config convert.cvsps.mergefrom='{{mergefrombranch ([-\w]+)}}'
138 138 Specify a regular expression to which commit log messages
139 139 are matched. If a match occurs, then the conversion
140 140 process will add the most recent revision on the branch
141 141 indicated in the regex as the second parent of the
142 142 changeset.
143 143
144 144 The hgext/convert/cvsps wrapper script allows the builtin
145 145 changeset merging code to be run without doing a conversion. Its
146 146 parameters and output are similar to that of cvsps 2.1.
147 147
148 148 Subversion Source
149 149 -----------------
150 150
151 151 Subversion source detects classical trunk/branches/tags layouts.
152 152 By default, the supplied "svn://repo/path/" source URL is
153 153 converted as a single branch. If "svn://repo/path/trunk" exists it
154 154 replaces the default branch. If "svn://repo/path/branches" exists,
155 155 its subdirectories are listed as possible branches. If
156 156 "svn://repo/path/tags" exists, it is looked for tags referencing
157 157 converted branches. Default "trunk", "branches" and "tags" values
158 158 can be overridden with following options. Set them to paths
159 159 relative to the source URL, or leave them blank to disable auto
160 160 detection.
161 161
162 162 --config convert.svn.branches=branches (directory name)
163 163 specify the directory containing branches
164 164 --config convert.svn.tags=tags (directory name)
165 165 specify the directory containing tags
166 166 --config convert.svn.trunk=trunk (directory name)
167 167 specify the name of the trunk branch
168 168
169 169 Source history can be retrieved starting at a specific revision,
170 170 instead of being integrally converted. Only single branch
171 171 conversions are supported.
172 172
173 173 --config convert.svn.startrev=0 (svn revision number)
174 174 specify start Subversion revision.
175 175
176 176 Perforce Source
177 177 ---------------
178 178
179 179 The Perforce (P4) importer can be given a p4 depot path or a
180 180 client specification as source. It will convert all files in the
181 181 source to a flat Mercurial repository, ignoring labels, branches
182 182 and integrations. Note that when a depot path is given you then
183 183 usually should specify a target directory, because otherwise the
184 184 target may be named ...-hg.
185 185
186 186 It is possible to limit the amount of source history to be
187 187 converted by specifying an initial Perforce revision.
188 188
189 189 --config convert.p4.startrev=0 (perforce changelist number)
190 190 specify initial Perforce revision.
191 191
192 192
193 193 Mercurial Destination
194 194 ---------------------
195 195
196 196 --config convert.hg.clonebranches=False (boolean)
197 197 dispatch source branches in separate clones.
198 198 --config convert.hg.tagsbranch=default (branch name)
199 199 tag revisions branch name
200 200 --config convert.hg.usebranchnames=True (boolean)
201 201 preserve branch names
202 202
203 203 options:
204 204
205 205 -A --authors username mapping filename
206 206 -d --dest-type destination repository type
207 207 --filemap remap file names using contents of file
208 208 -r --rev import up to target revision REV
209 209 -s --source-type source repository type
210 210 --splicemap splice synthesized history into place
211 211 --branchmap change branch names while converting
212 212 --datesort try to sort changesets by date
213 --sourcesort preserve source changesets order
213 214
214 215 use "hg -v help convert" to show global options
215 216 adding a
216 217 assuming destination a-hg
217 218 initializing destination a-hg repository
218 219 scanning source...
219 220 sorting...
220 221 converting...
221 222 4 a
222 223 3 b
223 224 2 c
224 225 1 d
225 226 0 e
226 227 pulling from ../a
227 228 searching for changes
228 229 no changes found
229 230 % should fail
230 231 initializing destination bogusfile repository
231 232 abort: cannot create new bundle repository
232 233 % should fail
233 234 abort: Permission denied: bogusdir
234 235 % should succeed
235 236 initializing destination bogusdir repository
236 237 scanning source...
237 238 sorting...
238 239 converting...
239 240 4 a
240 241 3 b
241 242 2 c
242 243 1 d
243 244 0 e
244 245 % test pre and post conversion actions
245 246 run hg source pre-conversion action
246 247 run hg sink pre-conversion action
247 248 run hg sink post-conversion action
248 249 run hg source post-conversion action
249 250 % converting empty dir should fail nicely
250 251 assuming destination emptydir-hg
251 252 initializing destination emptydir-hg repository
252 253 emptydir does not look like a CVS checkout
253 254 emptydir does not look like a Git repo
254 255 emptydir does not look like a Subversion repo
255 256 emptydir is not a local Mercurial repo
256 257 emptydir does not look like a darcs repo
257 258 emptydir does not look like a monotone repo
258 259 emptydir does not look like a GNU Arch repo
259 260 emptydir does not look like a Bazaar repo
260 261 emptydir does not look like a P4 repo
261 262 abort: emptydir: missing or unsupported repository
General Comments 0
You need to be logged in to leave comments. Login now