##// END OF EJS Templates
convert: add closesort algorithm to mercurial sources...
Constantine Linnick -
r18819:05acdf8e default
parent child Browse files
Show More
@@ -1,378 +1,383 b''
1 1 # convert.py Foreign SCM converter
2 2 #
3 3 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
4 4 #
5 5 # This software may be used and distributed according to the terms of the
6 6 # GNU General Public License version 2 or any later version.
7 7
8 8 '''import revisions from foreign VCS repositories into Mercurial'''
9 9
10 10 import convcmd
11 11 import cvsps
12 12 import subversion
13 13 from mercurial import commands, templatekw
14 14 from mercurial.i18n import _
15 15
16 16 testedwith = 'internal'
17 17
18 18 # Commands definition was moved elsewhere to ease demandload job.
19 19
20 20 def convert(ui, src, dest=None, revmapfile=None, **opts):
21 21 """convert a foreign SCM repository to a Mercurial one.
22 22
23 23 Accepted source formats [identifiers]:
24 24
25 25 - Mercurial [hg]
26 26 - CVS [cvs]
27 27 - Darcs [darcs]
28 28 - git [git]
29 29 - Subversion [svn]
30 30 - Monotone [mtn]
31 31 - GNU Arch [gnuarch]
32 32 - Bazaar [bzr]
33 33 - Perforce [p4]
34 34
35 35 Accepted destination formats [identifiers]:
36 36
37 37 - Mercurial [hg]
38 38 - Subversion [svn] (history on branches is not preserved)
39 39
40 40 If no revision is given, all revisions will be converted.
41 41 Otherwise, convert will only import up to the named revision
42 42 (given in a format understood by the source).
43 43
44 44 If no destination directory name is specified, it defaults to the
45 45 basename of the source with ``-hg`` appended. If the destination
46 46 repository doesn't exist, it will be created.
47 47
48 48 By default, all sources except Mercurial will use --branchsort.
49 49 Mercurial uses --sourcesort to preserve original revision numbers
50 50 order. Sort modes have the following effects:
51 51
52 52 --branchsort convert from parent to child revision when possible,
53 53 which means branches are usually converted one after
54 54 the other. It generates more compact repositories.
55 55
56 56 --datesort sort revisions by date. Converted repositories have
57 57 good-looking changelogs but are often an order of
58 58 magnitude larger than the same ones generated by
59 59 --branchsort.
60 60
61 61 --sourcesort try to preserve source revisions order, only
62 62 supported by Mercurial sources.
63 63
64 --closesort try to move closed revisions as close as possible
65 to parent branches, only supported by Mercurial
66 sources.
67
64 68 If ``REVMAP`` isn't given, it will be put in a default location
65 69 (``<dest>/.hg/shamap`` by default). The ``REVMAP`` is a simple
66 70 text file that maps each source commit ID to the destination ID
67 71 for that revision, like so::
68 72
69 73 <source ID> <destination ID>
70 74
71 75 If the file doesn't exist, it's automatically created. It's
72 76 updated on each commit copied, so :hg:`convert` can be interrupted
73 77 and can be run repeatedly to copy new commits.
74 78
75 79 The authormap is a simple text file that maps each source commit
76 80 author to a destination commit author. It is handy for source SCMs
77 81 that use unix logins to identify authors (e.g.: CVS). One line per
78 82 author mapping and the line format is::
79 83
80 84 source author = destination author
81 85
82 86 Empty lines and lines starting with a ``#`` are ignored.
83 87
84 88 The filemap is a file that allows filtering and remapping of files
85 89 and directories. Each line can contain one of the following
86 90 directives::
87 91
88 92 include path/to/file-or-dir
89 93
90 94 exclude path/to/file-or-dir
91 95
92 96 rename path/to/source path/to/destination
93 97
94 98 Comment lines start with ``#``. A specified path matches if it
95 99 equals the full relative name of a file or one of its parent
96 100 directories. The ``include`` or ``exclude`` directive with the
97 101 longest matching path applies, so line order does not matter.
98 102
99 103 The ``include`` directive causes a file, or all files under a
100 104 directory, to be included in the destination repository, and the
101 105 exclusion of all other files and directories not explicitly
102 106 included. The ``exclude`` directive causes files or directories to
103 107 be omitted. The ``rename`` directive renames a file or directory if
104 108 it is converted. To rename from a subdirectory into the root of
105 109 the repository, use ``.`` as the path to rename to.
106 110
107 111 The splicemap is a file that allows insertion of synthetic
108 112 history, letting you specify the parents of a revision. This is
109 113 useful if you want to e.g. give a Subversion merge two parents, or
110 114 graft two disconnected series of history together. Each entry
111 115 contains a key, followed by a space, followed by one or two
112 116 comma-separated values::
113 117
114 118 key parent1, parent2
115 119
116 120 The key is the revision ID in the source
117 121 revision control system whose parents should be modified (same
118 122 format as a key in .hg/shamap). The values are the revision IDs
119 123 (in either the source or destination revision control system) that
120 124 should be used as the new parents for that node. For example, if
121 125 you have merged "release-1.0" into "trunk", then you should
122 126 specify the revision on "trunk" as the first parent and the one on
123 127 the "release-1.0" branch as the second.
124 128
125 129 The branchmap is a file that allows you to rename a branch when it is
126 130 being brought in from whatever external repository. When used in
127 131 conjunction with a splicemap, it allows for a powerful combination
128 132 to help fix even the most badly mismanaged repositories and turn them
129 133 into nicely structured Mercurial repositories. The branchmap contains
130 134 lines of the form::
131 135
132 136 original_branch_name new_branch_name
133 137
134 138 where "original_branch_name" is the name of the branch in the
135 139 source repository, and "new_branch_name" is the name of the branch
136 140 is the destination repository. No whitespace is allowed in the
137 141 branch names. This can be used to (for instance) move code in one
138 142 repository from "default" to a named branch.
139 143
140 144 Mercurial Source
141 145 ################
142 146
143 147 The Mercurial source recognizes the following configuration
144 148 options, which you can set on the command line with ``--config``:
145 149
146 150 :convert.hg.ignoreerrors: ignore integrity errors when reading.
147 151 Use it to fix Mercurial repositories with missing revlogs, by
148 152 converting from and to Mercurial. Default is False.
149 153
150 154 :convert.hg.saverev: store original revision ID in changeset
151 155 (forces target IDs to change). It takes a boolean argument and
152 156 defaults to False.
153 157
154 158 :convert.hg.startrev: convert start revision and its descendants.
155 159 It takes a hg revision identifier and defaults to 0.
156 160
157 161 CVS Source
158 162 ##########
159 163
160 164 CVS source will use a sandbox (i.e. a checked-out copy) from CVS
161 165 to indicate the starting point of what will be converted. Direct
162 166 access to the repository files is not needed, unless of course the
163 167 repository is ``:local:``. The conversion uses the top level
164 168 directory in the sandbox to find the CVS repository, and then uses
165 169 CVS rlog commands to find files to convert. This means that unless
166 170 a filemap is given, all files under the starting directory will be
167 171 converted, and that any directory reorganization in the CVS
168 172 sandbox is ignored.
169 173
170 174 The following options can be used with ``--config``:
171 175
172 176 :convert.cvsps.cache: Set to False to disable remote log caching,
173 177 for testing and debugging purposes. Default is True.
174 178
175 179 :convert.cvsps.fuzz: Specify the maximum time (in seconds) that is
176 180 allowed between commits with identical user and log message in
177 181 a single changeset. When very large files were checked in as
178 182 part of a changeset then the default may not be long enough.
179 183 The default is 60.
180 184
181 185 :convert.cvsps.mergeto: Specify a regular expression to which
182 186 commit log messages are matched. If a match occurs, then the
183 187 conversion process will insert a dummy revision merging the
184 188 branch on which this log message occurs to the branch
185 189 indicated in the regex. Default is ``{{mergetobranch
186 190 ([-\\w]+)}}``
187 191
188 192 :convert.cvsps.mergefrom: Specify a regular expression to which
189 193 commit log messages are matched. If a match occurs, then the
190 194 conversion process will add the most recent revision on the
191 195 branch indicated in the regex as the second parent of the
192 196 changeset. Default is ``{{mergefrombranch ([-\\w]+)}}``
193 197
194 198 :convert.localtimezone: use local time (as determined by the TZ
195 199 environment variable) for changeset date/times. The default
196 200 is False (use UTC).
197 201
198 202 :hooks.cvslog: Specify a Python function to be called at the end of
199 203 gathering the CVS log. The function is passed a list with the
200 204 log entries, and can modify the entries in-place, or add or
201 205 delete them.
202 206
203 207 :hooks.cvschangesets: Specify a Python function to be called after
204 208 the changesets are calculated from the CVS log. The
205 209 function is passed a list with the changeset entries, and can
206 210 modify the changesets in-place, or add or delete them.
207 211
208 212 An additional "debugcvsps" Mercurial command allows the builtin
209 213 changeset merging code to be run without doing a conversion. Its
210 214 parameters and output are similar to that of cvsps 2.1. Please see
211 215 the command help for more details.
212 216
213 217 Subversion Source
214 218 #################
215 219
216 220 Subversion source detects classical trunk/branches/tags layouts.
217 221 By default, the supplied ``svn://repo/path/`` source URL is
218 222 converted as a single branch. If ``svn://repo/path/trunk`` exists
219 223 it replaces the default branch. If ``svn://repo/path/branches``
220 224 exists, its subdirectories are listed as possible branches. If
221 225 ``svn://repo/path/tags`` exists, it is looked for tags referencing
222 226 converted branches. Default ``trunk``, ``branches`` and ``tags``
223 227 values can be overridden with following options. Set them to paths
224 228 relative to the source URL, or leave them blank to disable auto
225 229 detection.
226 230
227 231 The following options can be set with ``--config``:
228 232
229 233 :convert.svn.branches: specify the directory containing branches.
230 234 The default is ``branches``.
231 235
232 236 :convert.svn.tags: specify the directory containing tags. The
233 237 default is ``tags``.
234 238
235 239 :convert.svn.trunk: specify the name of the trunk branch. The
236 240 default is ``trunk``.
237 241
238 242 :convert.localtimezone: use local time (as determined by the TZ
239 243 environment variable) for changeset date/times. The default
240 244 is False (use UTC).
241 245
242 246 Source history can be retrieved starting at a specific revision,
243 247 instead of being integrally converted. Only single branch
244 248 conversions are supported.
245 249
246 250 :convert.svn.startrev: specify start Subversion revision number.
247 251 The default is 0.
248 252
249 253 Perforce Source
250 254 ###############
251 255
252 256 The Perforce (P4) importer can be given a p4 depot path or a
253 257 client specification as source. It will convert all files in the
254 258 source to a flat Mercurial repository, ignoring labels, branches
255 259 and integrations. Note that when a depot path is given you then
256 260 usually should specify a target directory, because otherwise the
257 261 target may be named ``...-hg``.
258 262
259 263 It is possible to limit the amount of source history to be
260 264 converted by specifying an initial Perforce revision:
261 265
262 266 :convert.p4.startrev: specify initial Perforce revision (a
263 267 Perforce changelist number).
264 268
265 269 Mercurial Destination
266 270 #####################
267 271
268 272 The following options are supported:
269 273
270 274 :convert.hg.clonebranches: dispatch source branches in separate
271 275 clones. The default is False.
272 276
273 277 :convert.hg.tagsbranch: branch name for tag revisions, defaults to
274 278 ``default``.
275 279
276 280 :convert.hg.usebranchnames: preserve branch names. The default is
277 281 True.
278 282 """
279 283 return convcmd.convert(ui, src, dest, revmapfile, **opts)
280 284
281 285 def debugsvnlog(ui, **opts):
282 286 return subversion.debugsvnlog(ui, **opts)
283 287
284 288 def debugcvsps(ui, *args, **opts):
285 289 '''create changeset information from CVS
286 290
287 291 This command is intended as a debugging tool for the CVS to
288 292 Mercurial converter, and can be used as a direct replacement for
289 293 cvsps.
290 294
291 295 Hg debugcvsps reads the CVS rlog for current directory (or any
292 296 named directory) in the CVS repository, and converts the log to a
293 297 series of changesets based on matching commit log entries and
294 298 dates.'''
295 299 return cvsps.debugcvsps(ui, *args, **opts)
296 300
297 301 commands.norepo += " convert debugsvnlog debugcvsps"
298 302
299 303 cmdtable = {
300 304 "convert":
301 305 (convert,
302 306 [('', 'authors', '',
303 307 _('username mapping filename (DEPRECATED, use --authormap instead)'),
304 308 _('FILE')),
305 309 ('s', 'source-type', '',
306 310 _('source repository type'), _('TYPE')),
307 311 ('d', 'dest-type', '',
308 312 _('destination repository type'), _('TYPE')),
309 313 ('r', 'rev', '',
310 314 _('import up to target revision REV'), _('REV')),
311 315 ('A', 'authormap', '',
312 316 _('remap usernames using this file'), _('FILE')),
313 317 ('', 'filemap', '',
314 318 _('remap file names using contents of file'), _('FILE')),
315 319 ('', 'splicemap', '',
316 320 _('splice synthesized history into place'), _('FILE')),
317 321 ('', 'branchmap', '',
318 322 _('change branch names while converting'), _('FILE')),
319 323 ('', 'branchsort', None, _('try to sort changesets by branches')),
320 324 ('', 'datesort', None, _('try to sort changesets by date')),
321 ('', 'sourcesort', None, _('preserve source changesets order'))],
325 ('', 'sourcesort', None, _('preserve source changesets order')),
326 ('', 'closesort', None, _('try to reorder closed revisions'))],
322 327 _('hg convert [OPTION]... SOURCE [DEST [REVMAP]]')),
323 328 "debugsvnlog":
324 329 (debugsvnlog,
325 330 [],
326 331 'hg debugsvnlog'),
327 332 "debugcvsps":
328 333 (debugcvsps,
329 334 [
330 335 # Main options shared with cvsps-2.1
331 336 ('b', 'branches', [], _('only return changes on specified branches')),
332 337 ('p', 'prefix', '', _('prefix to remove from file names')),
333 338 ('r', 'revisions', [],
334 339 _('only return changes after or between specified tags')),
335 340 ('u', 'update-cache', None, _("update cvs log cache")),
336 341 ('x', 'new-cache', None, _("create new cvs log cache")),
337 342 ('z', 'fuzz', 60, _('set commit time fuzz in seconds')),
338 343 ('', 'root', '', _('specify cvsroot')),
339 344 # Options specific to builtin cvsps
340 345 ('', 'parents', '', _('show parent changesets')),
341 346 ('', 'ancestors', '',
342 347 _('show current changeset in ancestor branches')),
343 348 # Options that are ignored for compatibility with cvsps-2.1
344 349 ('A', 'cvs-direct', None, _('ignored for compatibility')),
345 350 ],
346 351 _('hg debugcvsps [OPTION]... [PATH]...')),
347 352 }
348 353
349 354 def kwconverted(ctx, name):
350 355 rev = ctx.extra().get('convert_revision', '')
351 356 if rev.startswith('svn:'):
352 357 if name == 'svnrev':
353 358 return str(subversion.revsplit(rev)[2])
354 359 elif name == 'svnpath':
355 360 return subversion.revsplit(rev)[1]
356 361 elif name == 'svnuuid':
357 362 return subversion.revsplit(rev)[0]
358 363 return rev
359 364
360 365 def kwsvnrev(repo, ctx, **args):
361 366 """:svnrev: String. Converted subversion revision number."""
362 367 return kwconverted(ctx, 'svnrev')
363 368
364 369 def kwsvnpath(repo, ctx, **args):
365 370 """:svnpath: String. Converted subversion revision project path."""
366 371 return kwconverted(ctx, 'svnpath')
367 372
368 373 def kwsvnuuid(repo, ctx, **args):
369 374 """:svnuuid: String. Converted subversion revision repository identifier."""
370 375 return kwconverted(ctx, 'svnuuid')
371 376
372 377 def extsetup(ui):
373 378 templatekw.keywords['svnrev'] = kwsvnrev
374 379 templatekw.keywords['svnpath'] = kwsvnpath
375 380 templatekw.keywords['svnuuid'] = kwsvnuuid
376 381
377 382 # tell hggettext to extract docstrings from these functions:
378 383 i18nfunctions = [kwsvnrev, kwsvnpath, kwsvnuuid]
@@ -1,455 +1,460 b''
1 1 # common.py - common code for the convert extension
2 2 #
3 3 # Copyright 2005-2009 Matt Mackall <mpm@selenic.com> and others
4 4 #
5 5 # This software may be used and distributed according to the terms of the
6 6 # GNU General Public License version 2 or any later version.
7 7
8 8 import base64, errno, subprocess, os, datetime
9 9 import cPickle as pickle
10 10 from mercurial import util
11 11 from mercurial.i18n import _
12 12
13 13 propertycache = util.propertycache
14 14
15 15 def encodeargs(args):
16 16 def encodearg(s):
17 17 lines = base64.encodestring(s)
18 18 lines = [l.splitlines()[0] for l in lines]
19 19 return ''.join(lines)
20 20
21 21 s = pickle.dumps(args)
22 22 return encodearg(s)
23 23
24 24 def decodeargs(s):
25 25 s = base64.decodestring(s)
26 26 return pickle.loads(s)
27 27
28 28 class MissingTool(Exception):
29 29 pass
30 30
31 31 def checktool(exe, name=None, abort=True):
32 32 name = name or exe
33 33 if not util.findexe(exe):
34 34 exc = abort and util.Abort or MissingTool
35 35 raise exc(_('cannot find required "%s" tool') % name)
36 36
37 37 class NoRepo(Exception):
38 38 pass
39 39
40 40 SKIPREV = 'SKIP'
41 41
42 42 class commit(object):
43 43 def __init__(self, author, date, desc, parents, branch=None, rev=None,
44 44 extra={}, sortkey=None):
45 45 self.author = author or 'unknown'
46 46 self.date = date or '0 0'
47 47 self.desc = desc
48 48 self.parents = parents
49 49 self.branch = branch
50 50 self.rev = rev
51 51 self.extra = extra
52 52 self.sortkey = sortkey
53 53
54 54 class converter_source(object):
55 55 """Conversion source interface"""
56 56
57 57 def __init__(self, ui, path=None, rev=None):
58 58 """Initialize conversion source (or raise NoRepo("message")
59 59 exception if path is not a valid repository)"""
60 60 self.ui = ui
61 61 self.path = path
62 62 self.rev = rev
63 63
64 64 self.encoding = 'utf-8'
65 65
66 66 def before(self):
67 67 pass
68 68
69 69 def after(self):
70 70 pass
71 71
72 72 def setrevmap(self, revmap):
73 73 """set the map of already-converted revisions"""
74 74 pass
75 75
76 76 def getheads(self):
77 77 """Return a list of this repository's heads"""
78 78 raise NotImplementedError
79 79
80 80 def getfile(self, name, rev):
81 81 """Return a pair (data, mode) where data is the file content
82 82 as a string and mode one of '', 'x' or 'l'. rev is the
83 83 identifier returned by a previous call to getchanges(). Raise
84 84 IOError to indicate that name was deleted in rev.
85 85 """
86 86 raise NotImplementedError
87 87
88 88 def getchanges(self, version):
89 89 """Returns a tuple of (files, copies).
90 90
91 91 files is a sorted list of (filename, id) tuples for all files
92 92 changed between version and its first parent returned by
93 93 getcommit(). id is the source revision id of the file.
94 94
95 95 copies is a dictionary of dest: source
96 96 """
97 97 raise NotImplementedError
98 98
99 99 def getcommit(self, version):
100 100 """Return the commit object for version"""
101 101 raise NotImplementedError
102 102
103 103 def gettags(self):
104 104 """Return the tags as a dictionary of name: revision
105 105
106 106 Tag names must be UTF-8 strings.
107 107 """
108 108 raise NotImplementedError
109 109
110 110 def recode(self, s, encoding=None):
111 111 if not encoding:
112 112 encoding = self.encoding or 'utf-8'
113 113
114 114 if isinstance(s, unicode):
115 115 return s.encode("utf-8")
116 116 try:
117 117 return s.decode(encoding).encode("utf-8")
118 118 except UnicodeError:
119 119 try:
120 120 return s.decode("latin-1").encode("utf-8")
121 121 except UnicodeError:
122 122 return s.decode(encoding, "replace").encode("utf-8")
123 123
124 124 def getchangedfiles(self, rev, i):
125 125 """Return the files changed by rev compared to parent[i].
126 126
127 127 i is an index selecting one of the parents of rev. The return
128 128 value should be the list of files that are different in rev and
129 129 this parent.
130 130
131 131 If rev has no parents, i is None.
132 132
133 133 This function is only needed to support --filemap
134 134 """
135 135 raise NotImplementedError
136 136
137 137 def converted(self, rev, sinkrev):
138 138 '''Notify the source that a revision has been converted.'''
139 139 pass
140 140
141 141 def hasnativeorder(self):
142 142 """Return true if this source has a meaningful, native revision
143 143 order. For instance, Mercurial revisions are store sequentially
144 144 while there is no such global ordering with Darcs.
145 145 """
146 146 return False
147 147
148 def hasnativeclose(self):
149 """Return true if this source has ability to close branch.
150 """
151 return False
152
148 153 def lookuprev(self, rev):
149 154 """If rev is a meaningful revision reference in source, return
150 155 the referenced identifier in the same format used by getcommit().
151 156 return None otherwise.
152 157 """
153 158 return None
154 159
155 160 def getbookmarks(self):
156 161 """Return the bookmarks as a dictionary of name: revision
157 162
158 163 Bookmark names are to be UTF-8 strings.
159 164 """
160 165 return {}
161 166
162 167 class converter_sink(object):
163 168 """Conversion sink (target) interface"""
164 169
165 170 def __init__(self, ui, path):
166 171 """Initialize conversion sink (or raise NoRepo("message")
167 172 exception if path is not a valid repository)
168 173
169 174 created is a list of paths to remove if a fatal error occurs
170 175 later"""
171 176 self.ui = ui
172 177 self.path = path
173 178 self.created = []
174 179
175 180 def getheads(self):
176 181 """Return a list of this repository's heads"""
177 182 raise NotImplementedError
178 183
179 184 def revmapfile(self):
180 185 """Path to a file that will contain lines
181 186 source_rev_id sink_rev_id
182 187 mapping equivalent revision identifiers for each system."""
183 188 raise NotImplementedError
184 189
185 190 def authorfile(self):
186 191 """Path to a file that will contain lines
187 192 srcauthor=dstauthor
188 193 mapping equivalent authors identifiers for each system."""
189 194 return None
190 195
191 196 def putcommit(self, files, copies, parents, commit, source, revmap):
192 197 """Create a revision with all changed files listed in 'files'
193 198 and having listed parents. 'commit' is a commit object
194 199 containing at a minimum the author, date, and message for this
195 200 changeset. 'files' is a list of (path, version) tuples,
196 201 'copies' is a dictionary mapping destinations to sources,
197 202 'source' is the source repository, and 'revmap' is a mapfile
198 203 of source revisions to converted revisions. Only getfile() and
199 204 lookuprev() should be called on 'source'.
200 205
201 206 Note that the sink repository is not told to update itself to
202 207 a particular revision (or even what that revision would be)
203 208 before it receives the file data.
204 209 """
205 210 raise NotImplementedError
206 211
207 212 def puttags(self, tags):
208 213 """Put tags into sink.
209 214
210 215 tags: {tagname: sink_rev_id, ...} where tagname is an UTF-8 string.
211 216 Return a pair (tag_revision, tag_parent_revision), or (None, None)
212 217 if nothing was changed.
213 218 """
214 219 raise NotImplementedError
215 220
216 221 def setbranch(self, branch, pbranches):
217 222 """Set the current branch name. Called before the first putcommit
218 223 on the branch.
219 224 branch: branch name for subsequent commits
220 225 pbranches: (converted parent revision, parent branch) tuples"""
221 226 pass
222 227
223 228 def setfilemapmode(self, active):
224 229 """Tell the destination that we're using a filemap
225 230
226 231 Some converter_sources (svn in particular) can claim that a file
227 232 was changed in a revision, even if there was no change. This method
228 233 tells the destination that we're using a filemap and that it should
229 234 filter empty revisions.
230 235 """
231 236 pass
232 237
233 238 def before(self):
234 239 pass
235 240
236 241 def after(self):
237 242 pass
238 243
239 244 def putbookmarks(self, bookmarks):
240 245 """Put bookmarks into sink.
241 246
242 247 bookmarks: {bookmarkname: sink_rev_id, ...}
243 248 where bookmarkname is an UTF-8 string.
244 249 """
245 250 pass
246 251
247 252 def hascommit(self, rev):
248 253 """Return True if the sink contains rev"""
249 254 raise NotImplementedError
250 255
251 256 class commandline(object):
252 257 def __init__(self, ui, command):
253 258 self.ui = ui
254 259 self.command = command
255 260
256 261 def prerun(self):
257 262 pass
258 263
259 264 def postrun(self):
260 265 pass
261 266
262 267 def _cmdline(self, cmd, *args, **kwargs):
263 268 cmdline = [self.command, cmd] + list(args)
264 269 for k, v in kwargs.iteritems():
265 270 if len(k) == 1:
266 271 cmdline.append('-' + k)
267 272 else:
268 273 cmdline.append('--' + k.replace('_', '-'))
269 274 try:
270 275 if len(k) == 1:
271 276 cmdline.append('' + v)
272 277 else:
273 278 cmdline[-1] += '=' + v
274 279 except TypeError:
275 280 pass
276 281 cmdline = [util.shellquote(arg) for arg in cmdline]
277 282 if not self.ui.debugflag:
278 283 cmdline += ['2>', os.devnull]
279 284 cmdline = ' '.join(cmdline)
280 285 return cmdline
281 286
282 287 def _run(self, cmd, *args, **kwargs):
283 288 def popen(cmdline):
284 289 p = subprocess.Popen(cmdline, shell=True, bufsize=-1,
285 290 close_fds=util.closefds,
286 291 stdout=subprocess.PIPE)
287 292 return p
288 293 return self._dorun(popen, cmd, *args, **kwargs)
289 294
290 295 def _run2(self, cmd, *args, **kwargs):
291 296 return self._dorun(util.popen2, cmd, *args, **kwargs)
292 297
293 298 def _dorun(self, openfunc, cmd, *args, **kwargs):
294 299 cmdline = self._cmdline(cmd, *args, **kwargs)
295 300 self.ui.debug('running: %s\n' % (cmdline,))
296 301 self.prerun()
297 302 try:
298 303 return openfunc(cmdline)
299 304 finally:
300 305 self.postrun()
301 306
302 307 def run(self, cmd, *args, **kwargs):
303 308 p = self._run(cmd, *args, **kwargs)
304 309 output = p.communicate()[0]
305 310 self.ui.debug(output)
306 311 return output, p.returncode
307 312
308 313 def runlines(self, cmd, *args, **kwargs):
309 314 p = self._run(cmd, *args, **kwargs)
310 315 output = p.stdout.readlines()
311 316 p.wait()
312 317 self.ui.debug(''.join(output))
313 318 return output, p.returncode
314 319
315 320 def checkexit(self, status, output=''):
316 321 if status:
317 322 if output:
318 323 self.ui.warn(_('%s error:\n') % self.command)
319 324 self.ui.warn(output)
320 325 msg = util.explainexit(status)[0]
321 326 raise util.Abort('%s %s' % (self.command, msg))
322 327
323 328 def run0(self, cmd, *args, **kwargs):
324 329 output, status = self.run(cmd, *args, **kwargs)
325 330 self.checkexit(status, output)
326 331 return output
327 332
328 333 def runlines0(self, cmd, *args, **kwargs):
329 334 output, status = self.runlines(cmd, *args, **kwargs)
330 335 self.checkexit(status, ''.join(output))
331 336 return output
332 337
333 338 @propertycache
334 339 def argmax(self):
335 340 # POSIX requires at least 4096 bytes for ARG_MAX
336 341 argmax = 4096
337 342 try:
338 343 argmax = os.sysconf("SC_ARG_MAX")
339 344 except (AttributeError, ValueError):
340 345 pass
341 346
342 347 # Windows shells impose their own limits on command line length,
343 348 # down to 2047 bytes for cmd.exe under Windows NT/2k and 2500 bytes
344 349 # for older 4nt.exe. See http://support.microsoft.com/kb/830473 for
345 350 # details about cmd.exe limitations.
346 351
347 352 # Since ARG_MAX is for command line _and_ environment, lower our limit
348 353 # (and make happy Windows shells while doing this).
349 354 return argmax // 2 - 1
350 355
351 356 def _limit_arglist(self, arglist, cmd, *args, **kwargs):
352 357 cmdlen = len(self._cmdline(cmd, *args, **kwargs))
353 358 limit = self.argmax - cmdlen
354 359 bytes = 0
355 360 fl = []
356 361 for fn in arglist:
357 362 b = len(fn) + 3
358 363 if bytes + b < limit or len(fl) == 0:
359 364 fl.append(fn)
360 365 bytes += b
361 366 else:
362 367 yield fl
363 368 fl = [fn]
364 369 bytes = b
365 370 if fl:
366 371 yield fl
367 372
368 373 def xargs(self, arglist, cmd, *args, **kwargs):
369 374 for l in self._limit_arglist(arglist, cmd, *args, **kwargs):
370 375 self.run0(cmd, *(list(args) + l), **kwargs)
371 376
372 377 class mapfile(dict):
373 378 def __init__(self, ui, path):
374 379 super(mapfile, self).__init__()
375 380 self.ui = ui
376 381 self.path = path
377 382 self.fp = None
378 383 self.order = []
379 384 self._read()
380 385
381 386 def _read(self):
382 387 if not self.path:
383 388 return
384 389 try:
385 390 fp = open(self.path, 'r')
386 391 except IOError, err:
387 392 if err.errno != errno.ENOENT:
388 393 raise
389 394 return
390 395 for i, line in enumerate(fp):
391 396 line = line.splitlines()[0].rstrip()
392 397 if not line:
393 398 # Ignore blank lines
394 399 continue
395 400 try:
396 401 key, value = line.rsplit(' ', 1)
397 402 except ValueError:
398 403 raise util.Abort(
399 404 _('syntax error in %s(%d): key/value pair expected')
400 405 % (self.path, i + 1))
401 406 if key not in self:
402 407 self.order.append(key)
403 408 super(mapfile, self).__setitem__(key, value)
404 409 fp.close()
405 410
406 411 def __setitem__(self, key, value):
407 412 if self.fp is None:
408 413 try:
409 414 self.fp = open(self.path, 'a')
410 415 except IOError, err:
411 416 raise util.Abort(_('could not open map file %r: %s') %
412 417 (self.path, err.strerror))
413 418 self.fp.write('%s %s\n' % (key, value))
414 419 self.fp.flush()
415 420 super(mapfile, self).__setitem__(key, value)
416 421
417 422 def close(self):
418 423 if self.fp:
419 424 self.fp.close()
420 425 self.fp = None
421 426
422 427 def parsesplicemap(path):
423 428 """Parse a splicemap, return a child/parents dictionary."""
424 429 if not path:
425 430 return {}
426 431 m = {}
427 432 try:
428 433 fp = open(path, 'r')
429 434 for i, line in enumerate(fp):
430 435 line = line.splitlines()[0].rstrip()
431 436 if not line:
432 437 # Ignore blank lines
433 438 continue
434 439 try:
435 440 child, parents = line.split(' ', 1)
436 441 parents = parents.replace(',', ' ').split()
437 442 except ValueError:
438 443 raise util.Abort(_('syntax error in %s(%d): child parent1'
439 444 '[,parent2] expected') % (path, i + 1))
440 445 pp = []
441 446 for p in parents:
442 447 if p not in pp:
443 448 pp.append(p)
444 449 m[child] = pp
445 450 except IOError, e:
446 451 if e.errno != errno.ENOENT:
447 452 raise
448 453 return m
449 454
450 455 def makedatetimestamp(t):
451 456 """Like util.makedate() but for time t instead of current time"""
452 457 delta = (datetime.datetime.utcfromtimestamp(t) -
453 458 datetime.datetime.fromtimestamp(t))
454 459 tz = delta.days * 86400 + delta.seconds
455 460 return t, tz
@@ -1,470 +1,482 b''
1 1 # convcmd - convert extension commands definition
2 2 #
3 3 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
4 4 #
5 5 # This software may be used and distributed according to the terms of the
6 6 # GNU General Public License version 2 or any later version.
7 7
8 8 from common import NoRepo, MissingTool, SKIPREV, mapfile
9 9 from cvs import convert_cvs
10 10 from darcs import darcs_source
11 11 from git import convert_git
12 12 from hg import mercurial_source, mercurial_sink
13 13 from subversion import svn_source, svn_sink
14 14 from monotone import monotone_source
15 15 from gnuarch import gnuarch_source
16 16 from bzr import bzr_source
17 17 from p4 import p4_source
18 18 import filemap, common
19 19
20 20 import os, shutil
21 21 from mercurial import hg, util, encoding
22 22 from mercurial.i18n import _
23 23
24 24 orig_encoding = 'ascii'
25 25
26 26 def recode(s):
27 27 if isinstance(s, unicode):
28 28 return s.encode(orig_encoding, 'replace')
29 29 else:
30 30 return s.decode('utf-8').encode(orig_encoding, 'replace')
31 31
32 32 source_converters = [
33 33 ('cvs', convert_cvs, 'branchsort'),
34 34 ('git', convert_git, 'branchsort'),
35 35 ('svn', svn_source, 'branchsort'),
36 36 ('hg', mercurial_source, 'sourcesort'),
37 37 ('darcs', darcs_source, 'branchsort'),
38 38 ('mtn', monotone_source, 'branchsort'),
39 39 ('gnuarch', gnuarch_source, 'branchsort'),
40 40 ('bzr', bzr_source, 'branchsort'),
41 41 ('p4', p4_source, 'branchsort'),
42 42 ]
43 43
44 44 sink_converters = [
45 45 ('hg', mercurial_sink),
46 46 ('svn', svn_sink),
47 47 ]
48 48
49 49 def convertsource(ui, path, type, rev):
50 50 exceptions = []
51 51 if type and type not in [s[0] for s in source_converters]:
52 52 raise util.Abort(_('%s: invalid source repository type') % type)
53 53 for name, source, sortmode in source_converters:
54 54 try:
55 55 if not type or name == type:
56 56 return source(ui, path, rev), sortmode
57 57 except (NoRepo, MissingTool), inst:
58 58 exceptions.append(inst)
59 59 if not ui.quiet:
60 60 for inst in exceptions:
61 61 ui.write("%s\n" % inst)
62 62 raise util.Abort(_('%s: missing or unsupported repository') % path)
63 63
64 64 def convertsink(ui, path, type):
65 65 if type and type not in [s[0] for s in sink_converters]:
66 66 raise util.Abort(_('%s: invalid destination repository type') % type)
67 67 for name, sink in sink_converters:
68 68 try:
69 69 if not type or name == type:
70 70 return sink(ui, path)
71 71 except NoRepo, inst:
72 72 ui.note(_("convert: %s\n") % inst)
73 73 except MissingTool, inst:
74 74 raise util.Abort('%s\n' % inst)
75 75 raise util.Abort(_('%s: unknown repository type') % path)
76 76
77 77 class progresssource(object):
78 78 def __init__(self, ui, source, filecount):
79 79 self.ui = ui
80 80 self.source = source
81 81 self.filecount = filecount
82 82 self.retrieved = 0
83 83
84 84 def getfile(self, file, rev):
85 85 self.retrieved += 1
86 86 self.ui.progress(_('getting files'), self.retrieved,
87 87 item=file, total=self.filecount)
88 88 return self.source.getfile(file, rev)
89 89
90 90 def lookuprev(self, rev):
91 91 return self.source.lookuprev(rev)
92 92
93 93 def close(self):
94 94 self.ui.progress(_('getting files'), None)
95 95
96 96 class converter(object):
97 97 def __init__(self, ui, source, dest, revmapfile, opts):
98 98
99 99 self.source = source
100 100 self.dest = dest
101 101 self.ui = ui
102 102 self.opts = opts
103 103 self.commitcache = {}
104 104 self.authors = {}
105 105 self.authorfile = None
106 106
107 107 # Record converted revisions persistently: maps source revision
108 108 # ID to target revision ID (both strings). (This is how
109 109 # incremental conversions work.)
110 110 self.map = mapfile(ui, revmapfile)
111 111
112 112 # Read first the dst author map if any
113 113 authorfile = self.dest.authorfile()
114 114 if authorfile and os.path.exists(authorfile):
115 115 self.readauthormap(authorfile)
116 116 # Extend/Override with new author map if necessary
117 117 if opts.get('authormap'):
118 118 self.readauthormap(opts.get('authormap'))
119 119 self.authorfile = self.dest.authorfile()
120 120
121 121 self.splicemap = common.parsesplicemap(opts.get('splicemap'))
122 122 self.branchmap = mapfile(ui, opts.get('branchmap'))
123 123
124 124 def walktree(self, heads):
125 125 '''Return a mapping that identifies the uncommitted parents of every
126 126 uncommitted changeset.'''
127 127 visit = heads
128 128 known = set()
129 129 parents = {}
130 130 while visit:
131 131 n = visit.pop(0)
132 132 if n in known or n in self.map:
133 133 continue
134 134 known.add(n)
135 135 self.ui.progress(_('scanning'), len(known), unit=_('revisions'))
136 136 commit = self.cachecommit(n)
137 137 parents[n] = []
138 138 for p in commit.parents:
139 139 parents[n].append(p)
140 140 visit.append(p)
141 141 self.ui.progress(_('scanning'), None)
142 142
143 143 return parents
144 144
145 145 def mergesplicemap(self, parents, splicemap):
146 146 """A splicemap redefines child/parent relationships. Check the
147 147 map contains valid revision identifiers and merge the new
148 148 links in the source graph.
149 149 """
150 150 for c in sorted(splicemap):
151 151 if c not in parents:
152 152 if not self.dest.hascommit(self.map.get(c, c)):
153 153 # Could be in source but not converted during this run
154 154 self.ui.warn(_('splice map revision %s is not being '
155 155 'converted, ignoring\n') % c)
156 156 continue
157 157 pc = []
158 158 for p in splicemap[c]:
159 159 # We do not have to wait for nodes already in dest.
160 160 if self.dest.hascommit(self.map.get(p, p)):
161 161 continue
162 162 # Parent is not in dest and not being converted, not good
163 163 if p not in parents:
164 164 raise util.Abort(_('unknown splice map parent: %s') % p)
165 165 pc.append(p)
166 166 parents[c] = pc
167 167
168 168 def toposort(self, parents, sortmode):
169 169 '''Return an ordering such that every uncommitted changeset is
170 170 preceded by all its uncommitted ancestors.'''
171 171
172 172 def mapchildren(parents):
173 173 """Return a (children, roots) tuple where 'children' maps parent
174 174 revision identifiers to children ones, and 'roots' is the list of
175 175 revisions without parents. 'parents' must be a mapping of revision
176 176 identifier to its parents ones.
177 177 """
178 178 visit = sorted(parents)
179 179 seen = set()
180 180 children = {}
181 181 roots = []
182 182
183 183 while visit:
184 184 n = visit.pop(0)
185 185 if n in seen:
186 186 continue
187 187 seen.add(n)
188 188 # Ensure that nodes without parents are present in the
189 189 # 'children' mapping.
190 190 children.setdefault(n, [])
191 191 hasparent = False
192 192 for p in parents[n]:
193 193 if p not in self.map:
194 194 visit.append(p)
195 195 hasparent = True
196 196 children.setdefault(p, []).append(n)
197 197 if not hasparent:
198 198 roots.append(n)
199 199
200 200 return children, roots
201 201
202 202 # Sort functions are supposed to take a list of revisions which
203 203 # can be converted immediately and pick one
204 204
205 205 def makebranchsorter():
206 206 """If the previously converted revision has a child in the
207 207 eligible revisions list, pick it. Return the list head
208 208 otherwise. Branch sort attempts to minimize branch
209 209 switching, which is harmful for Mercurial backend
210 210 compression.
211 211 """
212 212 prev = [None]
213 213 def picknext(nodes):
214 214 next = nodes[0]
215 215 for n in nodes:
216 216 if prev[0] in parents[n]:
217 217 next = n
218 218 break
219 219 prev[0] = next
220 220 return next
221 221 return picknext
222 222
223 223 def makesourcesorter():
224 224 """Source specific sort."""
225 225 keyfn = lambda n: self.commitcache[n].sortkey
226 226 def picknext(nodes):
227 227 return sorted(nodes, key=keyfn)[0]
228 228 return picknext
229 229
230 def makeclosesorter():
231 """Close order sort."""
232 keyfn = lambda n: ('close' not in self.commitcache[n].extra,
233 self.commitcache[n].sortkey)
234 def picknext(nodes):
235 return sorted(nodes, key=keyfn)[0]
236 return picknext
237
230 238 def makedatesorter():
231 239 """Sort revisions by date."""
232 240 dates = {}
233 241 def getdate(n):
234 242 if n not in dates:
235 243 dates[n] = util.parsedate(self.commitcache[n].date)
236 244 return dates[n]
237 245
238 246 def picknext(nodes):
239 247 return min([(getdate(n), n) for n in nodes])[1]
240 248
241 249 return picknext
242 250
243 251 if sortmode == 'branchsort':
244 252 picknext = makebranchsorter()
245 253 elif sortmode == 'datesort':
246 254 picknext = makedatesorter()
247 255 elif sortmode == 'sourcesort':
248 256 picknext = makesourcesorter()
257 elif sortmode == 'closesort':
258 picknext = makeclosesorter()
249 259 else:
250 260 raise util.Abort(_('unknown sort mode: %s') % sortmode)
251 261
252 262 children, actives = mapchildren(parents)
253 263
254 264 s = []
255 265 pendings = {}
256 266 while actives:
257 267 n = picknext(actives)
258 268 actives.remove(n)
259 269 s.append(n)
260 270
261 271 # Update dependents list
262 272 for c in children.get(n, []):
263 273 if c not in pendings:
264 274 pendings[c] = [p for p in parents[c] if p not in self.map]
265 275 try:
266 276 pendings[c].remove(n)
267 277 except ValueError:
268 278 raise util.Abort(_('cycle detected between %s and %s')
269 279 % (recode(c), recode(n)))
270 280 if not pendings[c]:
271 281 # Parents are converted, node is eligible
272 282 actives.insert(0, c)
273 283 pendings[c] = None
274 284
275 285 if len(s) != len(parents):
276 286 raise util.Abort(_("not all revisions were sorted"))
277 287
278 288 return s
279 289
280 290 def writeauthormap(self):
281 291 authorfile = self.authorfile
282 292 if authorfile:
283 293 self.ui.status(_('writing author map file %s\n') % authorfile)
284 294 ofile = open(authorfile, 'w+')
285 295 for author in self.authors:
286 296 ofile.write("%s=%s\n" % (author, self.authors[author]))
287 297 ofile.close()
288 298
289 299 def readauthormap(self, authorfile):
290 300 afile = open(authorfile, 'r')
291 301 for line in afile:
292 302
293 303 line = line.strip()
294 304 if not line or line.startswith('#'):
295 305 continue
296 306
297 307 try:
298 308 srcauthor, dstauthor = line.split('=', 1)
299 309 except ValueError:
300 310 msg = _('ignoring bad line in author map file %s: %s\n')
301 311 self.ui.warn(msg % (authorfile, line.rstrip()))
302 312 continue
303 313
304 314 srcauthor = srcauthor.strip()
305 315 dstauthor = dstauthor.strip()
306 316 if self.authors.get(srcauthor) in (None, dstauthor):
307 317 msg = _('mapping author %s to %s\n')
308 318 self.ui.debug(msg % (srcauthor, dstauthor))
309 319 self.authors[srcauthor] = dstauthor
310 320 continue
311 321
312 322 m = _('overriding mapping for author %s, was %s, will be %s\n')
313 323 self.ui.status(m % (srcauthor, self.authors[srcauthor], dstauthor))
314 324
315 325 afile.close()
316 326
317 327 def cachecommit(self, rev):
318 328 commit = self.source.getcommit(rev)
319 329 commit.author = self.authors.get(commit.author, commit.author)
320 330 commit.branch = self.branchmap.get(commit.branch, commit.branch)
321 331 self.commitcache[rev] = commit
322 332 return commit
323 333
324 334 def copy(self, rev):
325 335 commit = self.commitcache[rev]
326 336
327 337 changes = self.source.getchanges(rev)
328 338 if isinstance(changes, basestring):
329 339 if changes == SKIPREV:
330 340 dest = SKIPREV
331 341 else:
332 342 dest = self.map[changes]
333 343 self.map[rev] = dest
334 344 return
335 345 files, copies = changes
336 346 pbranches = []
337 347 if commit.parents:
338 348 for prev in commit.parents:
339 349 if prev not in self.commitcache:
340 350 self.cachecommit(prev)
341 351 pbranches.append((self.map[prev],
342 352 self.commitcache[prev].branch))
343 353 self.dest.setbranch(commit.branch, pbranches)
344 354 try:
345 355 parents = self.splicemap[rev]
346 356 self.ui.status(_('spliced in %s as parents of %s\n') %
347 357 (parents, rev))
348 358 parents = [self.map.get(p, p) for p in parents]
349 359 except KeyError:
350 360 parents = [b[0] for b in pbranches]
351 361 source = progresssource(self.ui, self.source, len(files))
352 362 newnode = self.dest.putcommit(files, copies, parents, commit,
353 363 source, self.map)
354 364 source.close()
355 365 self.source.converted(rev, newnode)
356 366 self.map[rev] = newnode
357 367
358 368 def convert(self, sortmode):
359 369 try:
360 370 self.source.before()
361 371 self.dest.before()
362 372 self.source.setrevmap(self.map)
363 373 self.ui.status(_("scanning source...\n"))
364 374 heads = self.source.getheads()
365 375 parents = self.walktree(heads)
366 376 self.mergesplicemap(parents, self.splicemap)
367 377 self.ui.status(_("sorting...\n"))
368 378 t = self.toposort(parents, sortmode)
369 379 num = len(t)
370 380 c = None
371 381
372 382 self.ui.status(_("converting...\n"))
373 383 for i, c in enumerate(t):
374 384 num -= 1
375 385 desc = self.commitcache[c].desc
376 386 if "\n" in desc:
377 387 desc = desc.splitlines()[0]
378 388 # convert log message to local encoding without using
379 389 # tolocal() because the encoding.encoding convert()
380 390 # uses is 'utf-8'
381 391 self.ui.status("%d %s\n" % (num, recode(desc)))
382 392 self.ui.note(_("source: %s\n") % recode(c))
383 393 self.ui.progress(_('converting'), i, unit=_('revisions'),
384 394 total=len(t))
385 395 self.copy(c)
386 396 self.ui.progress(_('converting'), None)
387 397
388 398 tags = self.source.gettags()
389 399 ctags = {}
390 400 for k in tags:
391 401 v = tags[k]
392 402 if self.map.get(v, SKIPREV) != SKIPREV:
393 403 ctags[k] = self.map[v]
394 404
395 405 if c and ctags:
396 406 nrev, tagsparent = self.dest.puttags(ctags)
397 407 if nrev and tagsparent:
398 408 # write another hash correspondence to override the previous
399 409 # one so we don't end up with extra tag heads
400 410 tagsparents = [e for e in self.map.iteritems()
401 411 if e[1] == tagsparent]
402 412 if tagsparents:
403 413 self.map[tagsparents[0][0]] = nrev
404 414
405 415 bookmarks = self.source.getbookmarks()
406 416 cbookmarks = {}
407 417 for k in bookmarks:
408 418 v = bookmarks[k]
409 419 if self.map.get(v, SKIPREV) != SKIPREV:
410 420 cbookmarks[k] = self.map[v]
411 421
412 422 if c and cbookmarks:
413 423 self.dest.putbookmarks(cbookmarks)
414 424
415 425 self.writeauthormap()
416 426 finally:
417 427 self.cleanup()
418 428
419 429 def cleanup(self):
420 430 try:
421 431 self.dest.after()
422 432 finally:
423 433 self.source.after()
424 434 self.map.close()
425 435
426 436 def convert(ui, src, dest=None, revmapfile=None, **opts):
427 437 global orig_encoding
428 438 orig_encoding = encoding.encoding
429 439 encoding.encoding = 'UTF-8'
430 440
431 441 # support --authors as an alias for --authormap
432 442 if not opts.get('authormap'):
433 443 opts['authormap'] = opts.get('authors')
434 444
435 445 if not dest:
436 446 dest = hg.defaultdest(src) + "-hg"
437 447 ui.status(_("assuming destination %s\n") % dest)
438 448
439 449 destc = convertsink(ui, dest, opts.get('dest_type'))
440 450
441 451 try:
442 452 srcc, defaultsort = convertsource(ui, src, opts.get('source_type'),
443 453 opts.get('rev'))
444 454 except Exception:
445 455 for path in destc.created:
446 456 shutil.rmtree(path, True)
447 457 raise
448 458
449 sortmodes = ('branchsort', 'datesort', 'sourcesort')
459 sortmodes = ('branchsort', 'datesort', 'sourcesort', 'closesort')
450 460 sortmode = [m for m in sortmodes if opts.get(m)]
451 461 if len(sortmode) > 1:
452 462 raise util.Abort(_('more than one sort mode specified'))
453 463 sortmode = sortmode and sortmode[0] or defaultsort
454 464 if sortmode == 'sourcesort' and not srcc.hasnativeorder():
455 465 raise util.Abort(_('--sourcesort is not supported by this data source'))
466 if sortmode == 'closesort' and not srcc.hasnativeclose():
467 raise util.Abort(_('--closesort is not supported by this data source'))
456 468
457 469 fmap = opts.get('filemap')
458 470 if fmap:
459 471 srcc = filemap.filemap_source(ui, srcc, fmap)
460 472 destc.setfilemapmode(True)
461 473
462 474 if not revmapfile:
463 475 try:
464 476 revmapfile = destc.revmapfile()
465 477 except Exception:
466 478 revmapfile = os.path.join(destc, "map")
467 479
468 480 c = converter(ui, srcc, destc, revmapfile, opts)
469 481 c.convert(sortmode)
470 482
@@ -1,396 +1,399 b''
1 1 # hg.py - hg backend for convert extension
2 2 #
3 3 # Copyright 2005-2009 Matt Mackall <mpm@selenic.com> and others
4 4 #
5 5 # This software may be used and distributed according to the terms of the
6 6 # GNU General Public License version 2 or any later version.
7 7
8 8 # Notes for hg->hg conversion:
9 9 #
10 10 # * Old versions of Mercurial didn't trim the whitespace from the ends
11 11 # of commit messages, but new versions do. Changesets created by
12 12 # those older versions, then converted, may thus have different
13 13 # hashes for changesets that are otherwise identical.
14 14 #
15 15 # * Using "--config convert.hg.saverev=true" will make the source
16 16 # identifier to be stored in the converted revision. This will cause
17 17 # the converted revision to have a different identity than the
18 18 # source.
19 19
20 20
21 21 import os, time, cStringIO
22 22 from mercurial.i18n import _
23 23 from mercurial.node import bin, hex, nullid
24 24 from mercurial import hg, util, context, bookmarks, error
25 25
26 26 from common import NoRepo, commit, converter_source, converter_sink
27 27
28 28 class mercurial_sink(converter_sink):
29 29 def __init__(self, ui, path):
30 30 converter_sink.__init__(self, ui, path)
31 31 self.branchnames = ui.configbool('convert', 'hg.usebranchnames', True)
32 32 self.clonebranches = ui.configbool('convert', 'hg.clonebranches', False)
33 33 self.tagsbranch = ui.config('convert', 'hg.tagsbranch', 'default')
34 34 self.lastbranch = None
35 35 if os.path.isdir(path) and len(os.listdir(path)) > 0:
36 36 try:
37 37 self.repo = hg.repository(self.ui, path)
38 38 if not self.repo.local():
39 39 raise NoRepo(_('%s is not a local Mercurial repository')
40 40 % path)
41 41 except error.RepoError, err:
42 42 ui.traceback()
43 43 raise NoRepo(err.args[0])
44 44 else:
45 45 try:
46 46 ui.status(_('initializing destination %s repository\n') % path)
47 47 self.repo = hg.repository(self.ui, path, create=True)
48 48 if not self.repo.local():
49 49 raise NoRepo(_('%s is not a local Mercurial repository')
50 50 % path)
51 51 self.created.append(path)
52 52 except error.RepoError:
53 53 ui.traceback()
54 54 raise NoRepo(_("could not create hg repository %s as sink")
55 55 % path)
56 56 self.lock = None
57 57 self.wlock = None
58 58 self.filemapmode = False
59 59
60 60 def before(self):
61 61 self.ui.debug('run hg sink pre-conversion action\n')
62 62 self.wlock = self.repo.wlock()
63 63 self.lock = self.repo.lock()
64 64
65 65 def after(self):
66 66 self.ui.debug('run hg sink post-conversion action\n')
67 67 if self.lock:
68 68 self.lock.release()
69 69 if self.wlock:
70 70 self.wlock.release()
71 71
72 72 def revmapfile(self):
73 73 return self.repo.join("shamap")
74 74
75 75 def authorfile(self):
76 76 return self.repo.join("authormap")
77 77
78 78 def getheads(self):
79 79 h = self.repo.changelog.heads()
80 80 return [hex(x) for x in h]
81 81
82 82 def setbranch(self, branch, pbranches):
83 83 if not self.clonebranches:
84 84 return
85 85
86 86 setbranch = (branch != self.lastbranch)
87 87 self.lastbranch = branch
88 88 if not branch:
89 89 branch = 'default'
90 90 pbranches = [(b[0], b[1] and b[1] or 'default') for b in pbranches]
91 91 pbranch = pbranches and pbranches[0][1] or 'default'
92 92
93 93 branchpath = os.path.join(self.path, branch)
94 94 if setbranch:
95 95 self.after()
96 96 try:
97 97 self.repo = hg.repository(self.ui, branchpath)
98 98 except Exception:
99 99 self.repo = hg.repository(self.ui, branchpath, create=True)
100 100 self.before()
101 101
102 102 # pbranches may bring revisions from other branches (merge parents)
103 103 # Make sure we have them, or pull them.
104 104 missings = {}
105 105 for b in pbranches:
106 106 try:
107 107 self.repo.lookup(b[0])
108 108 except Exception:
109 109 missings.setdefault(b[1], []).append(b[0])
110 110
111 111 if missings:
112 112 self.after()
113 113 for pbranch, heads in sorted(missings.iteritems()):
114 114 pbranchpath = os.path.join(self.path, pbranch)
115 115 prepo = hg.peer(self.ui, {}, pbranchpath)
116 116 self.ui.note(_('pulling from %s into %s\n') % (pbranch, branch))
117 117 self.repo.pull(prepo, [prepo.lookup(h) for h in heads])
118 118 self.before()
119 119
120 120 def _rewritetags(self, source, revmap, data):
121 121 fp = cStringIO.StringIO()
122 122 for line in data.splitlines():
123 123 s = line.split(' ', 1)
124 124 if len(s) != 2:
125 125 continue
126 126 revid = revmap.get(source.lookuprev(s[0]))
127 127 if not revid:
128 128 continue
129 129 fp.write('%s %s\n' % (revid, s[1]))
130 130 return fp.getvalue()
131 131
132 132 def putcommit(self, files, copies, parents, commit, source, revmap):
133 133
134 134 files = dict(files)
135 135 def getfilectx(repo, memctx, f):
136 136 v = files[f]
137 137 data, mode = source.getfile(f, v)
138 138 if f == '.hgtags':
139 139 data = self._rewritetags(source, revmap, data)
140 140 return context.memfilectx(f, data, 'l' in mode, 'x' in mode,
141 141 copies.get(f))
142 142
143 143 pl = []
144 144 for p in parents:
145 145 if p not in pl:
146 146 pl.append(p)
147 147 parents = pl
148 148 nparents = len(parents)
149 149 if self.filemapmode and nparents == 1:
150 150 m1node = self.repo.changelog.read(bin(parents[0]))[0]
151 151 parent = parents[0]
152 152
153 153 if len(parents) < 2:
154 154 parents.append(nullid)
155 155 if len(parents) < 2:
156 156 parents.append(nullid)
157 157 p2 = parents.pop(0)
158 158
159 159 text = commit.desc
160 160 extra = commit.extra.copy()
161 161 if self.branchnames and commit.branch:
162 162 extra['branch'] = commit.branch
163 163 if commit.rev:
164 164 extra['convert_revision'] = commit.rev
165 165
166 166 while parents:
167 167 p1 = p2
168 168 p2 = parents.pop(0)
169 169 ctx = context.memctx(self.repo, (p1, p2), text, files.keys(),
170 170 getfilectx, commit.author, commit.date, extra)
171 171 self.repo.commitctx(ctx)
172 172 text = "(octopus merge fixup)\n"
173 173 p2 = hex(self.repo.changelog.tip())
174 174
175 175 if self.filemapmode and nparents == 1:
176 176 man = self.repo.manifest
177 177 mnode = self.repo.changelog.read(bin(p2))[0]
178 178 closed = 'close' in commit.extra
179 179 if not closed and not man.cmp(m1node, man.revision(mnode)):
180 180 self.ui.status(_("filtering out empty revision\n"))
181 181 self.repo.rollback(force=True)
182 182 return parent
183 183 return p2
184 184
185 185 def puttags(self, tags):
186 186 try:
187 187 parentctx = self.repo[self.tagsbranch]
188 188 tagparent = parentctx.node()
189 189 except error.RepoError:
190 190 parentctx = None
191 191 tagparent = nullid
192 192
193 193 try:
194 194 oldlines = sorted(parentctx['.hgtags'].data().splitlines(True))
195 195 except Exception:
196 196 oldlines = []
197 197
198 198 newlines = sorted([("%s %s\n" % (tags[tag], tag)) for tag in tags])
199 199 if newlines == oldlines:
200 200 return None, None
201 201 data = "".join(newlines)
202 202 def getfilectx(repo, memctx, f):
203 203 return context.memfilectx(f, data, False, False, None)
204 204
205 205 self.ui.status(_("updating tags\n"))
206 206 date = "%s 0" % int(time.mktime(time.gmtime()))
207 207 extra = {'branch': self.tagsbranch}
208 208 ctx = context.memctx(self.repo, (tagparent, None), "update tags",
209 209 [".hgtags"], getfilectx, "convert-repo", date,
210 210 extra)
211 211 self.repo.commitctx(ctx)
212 212 return hex(self.repo.changelog.tip()), hex(tagparent)
213 213
214 214 def setfilemapmode(self, active):
215 215 self.filemapmode = active
216 216
217 217 def putbookmarks(self, updatedbookmark):
218 218 if not len(updatedbookmark):
219 219 return
220 220
221 221 self.ui.status(_("updating bookmarks\n"))
222 222 destmarks = self.repo._bookmarks
223 223 for bookmark in updatedbookmark:
224 224 destmarks[bookmark] = bin(updatedbookmark[bookmark])
225 225 destmarks.write()
226 226
227 227 def hascommit(self, rev):
228 228 if rev not in self.repo and self.clonebranches:
229 229 raise util.Abort(_('revision %s not found in destination '
230 230 'repository (lookups with clonebranches=true '
231 231 'are not implemented)') % rev)
232 232 return rev in self.repo
233 233
234 234 class mercurial_source(converter_source):
235 235 def __init__(self, ui, path, rev=None):
236 236 converter_source.__init__(self, ui, path, rev)
237 237 self.ignoreerrors = ui.configbool('convert', 'hg.ignoreerrors', False)
238 238 self.ignored = set()
239 239 self.saverev = ui.configbool('convert', 'hg.saverev', False)
240 240 try:
241 241 self.repo = hg.repository(self.ui, path)
242 242 # try to provoke an exception if this isn't really a hg
243 243 # repo, but some other bogus compatible-looking url
244 244 if not self.repo.local():
245 245 raise error.RepoError
246 246 except error.RepoError:
247 247 ui.traceback()
248 248 raise NoRepo(_("%s is not a local Mercurial repository") % path)
249 249 self.lastrev = None
250 250 self.lastctx = None
251 251 self._changescache = None
252 252 self.convertfp = None
253 253 # Restrict converted revisions to startrev descendants
254 254 startnode = ui.config('convert', 'hg.startrev')
255 255 if startnode is not None:
256 256 try:
257 257 startnode = self.repo.lookup(startnode)
258 258 except error.RepoError:
259 259 raise util.Abort(_('%s is not a valid start revision')
260 260 % startnode)
261 261 startrev = self.repo.changelog.rev(startnode)
262 262 children = {startnode: 1}
263 263 for rev in self.repo.changelog.descendants([startrev]):
264 264 children[self.repo.changelog.node(rev)] = 1
265 265 self.keep = children.__contains__
266 266 else:
267 267 self.keep = util.always
268 268
269 269 def changectx(self, rev):
270 270 if self.lastrev != rev:
271 271 self.lastctx = self.repo[rev]
272 272 self.lastrev = rev
273 273 return self.lastctx
274 274
275 275 def parents(self, ctx):
276 276 return [p for p in ctx.parents() if p and self.keep(p.node())]
277 277
278 278 def getheads(self):
279 279 if self.rev:
280 280 heads = [self.repo[self.rev].node()]
281 281 else:
282 282 heads = self.repo.heads()
283 283 return [hex(h) for h in heads if self.keep(h)]
284 284
285 285 def getfile(self, name, rev):
286 286 try:
287 287 fctx = self.changectx(rev)[name]
288 288 return fctx.data(), fctx.flags()
289 289 except error.LookupError, err:
290 290 raise IOError(err)
291 291
292 292 def getchanges(self, rev):
293 293 ctx = self.changectx(rev)
294 294 parents = self.parents(ctx)
295 295 if not parents:
296 296 files = sorted(ctx.manifest())
297 297 # getcopies() is not needed for roots, but it is a simple way to
298 298 # detect missing revlogs and abort on errors or populate
299 299 # self.ignored
300 300 self.getcopies(ctx, parents, files)
301 301 return [(f, rev) for f in files if f not in self.ignored], {}
302 302 if self._changescache and self._changescache[0] == rev:
303 303 m, a, r = self._changescache[1]
304 304 else:
305 305 m, a, r = self.repo.status(parents[0].node(), ctx.node())[:3]
306 306 # getcopies() detects missing revlogs early, run it before
307 307 # filtering the changes.
308 308 copies = self.getcopies(ctx, parents, m + a)
309 309 changes = [(name, rev) for name in m + a + r
310 310 if name not in self.ignored]
311 311 return sorted(changes), copies
312 312
313 313 def getcopies(self, ctx, parents, files):
314 314 copies = {}
315 315 for name in files:
316 316 if name in self.ignored:
317 317 continue
318 318 try:
319 319 copysource, copynode = ctx.filectx(name).renamed()
320 320 if copysource in self.ignored or not self.keep(copynode):
321 321 continue
322 322 # Ignore copy sources not in parent revisions
323 323 found = False
324 324 for p in parents:
325 325 if copysource in p:
326 326 found = True
327 327 break
328 328 if not found:
329 329 continue
330 330 copies[name] = copysource
331 331 except TypeError:
332 332 pass
333 333 except error.LookupError, e:
334 334 if not self.ignoreerrors:
335 335 raise
336 336 self.ignored.add(name)
337 337 self.ui.warn(_('ignoring: %s\n') % e)
338 338 return copies
339 339
340 340 def getcommit(self, rev):
341 341 ctx = self.changectx(rev)
342 342 parents = [p.hex() for p in self.parents(ctx)]
343 343 if self.saverev:
344 344 crev = rev
345 345 else:
346 346 crev = None
347 347 return commit(author=ctx.user(),
348 348 date=util.datestr(ctx.date(), '%Y-%m-%d %H:%M:%S %1%2'),
349 349 desc=ctx.description(), rev=crev, parents=parents,
350 350 branch=ctx.branch(), extra=ctx.extra(),
351 351 sortkey=ctx.rev())
352 352
353 353 def gettags(self):
354 354 tags = [t for t in self.repo.tagslist() if t[0] != 'tip']
355 355 return dict([(name, hex(node)) for name, node in tags
356 356 if self.keep(node)])
357 357
358 358 def getchangedfiles(self, rev, i):
359 359 ctx = self.changectx(rev)
360 360 parents = self.parents(ctx)
361 361 if not parents and i is None:
362 362 i = 0
363 363 changes = [], ctx.manifest().keys(), []
364 364 else:
365 365 i = i or 0
366 366 changes = self.repo.status(parents[i].node(), ctx.node())[:3]
367 367 changes = [[f for f in l if f not in self.ignored] for l in changes]
368 368
369 369 if i == 0:
370 370 self._changescache = (rev, changes)
371 371
372 372 return changes[0] + changes[1] + changes[2]
373 373
374 374 def converted(self, rev, destrev):
375 375 if self.convertfp is None:
376 376 self.convertfp = open(self.repo.join('shamap'), 'a')
377 377 self.convertfp.write('%s %s\n' % (destrev, rev))
378 378 self.convertfp.flush()
379 379
380 380 def before(self):
381 381 self.ui.debug('run hg source pre-conversion action\n')
382 382
383 383 def after(self):
384 384 self.ui.debug('run hg source post-conversion action\n')
385 385
386 386 def hasnativeorder(self):
387 387 return True
388 388
389 def hasnativeclose(self):
390 return True
391
389 392 def lookuprev(self, rev):
390 393 try:
391 394 return hex(self.repo.lookup(rev))
392 395 except error.RepoError:
393 396 return None
394 397
395 398 def getbookmarks(self):
396 399 return bookmarks.listbookmarks(self.repo)
@@ -1,119 +1,214 b''
1 1
2 2 $ cat >> $HGRCPATH <<EOF
3 3 > [extensions]
4 4 > convert=
5 5 > graphlog=
6 6 > EOF
7 7 $ hg init t
8 8 $ cd t
9 9 $ echo a >> a
10 10 $ hg ci -Am a0 -d '1 0'
11 11 adding a
12 12 $ hg branch brancha
13 13 marked working directory as branch brancha
14 14 (branches are permanent and global, did you want a bookmark?)
15 15 $ echo a >> a
16 16 $ hg ci -m a1 -d '2 0'
17 17 $ echo a >> a
18 18 $ hg ci -m a2 -d '3 0'
19 19 $ echo a >> a
20 20 $ hg ci -m a3 -d '4 0'
21 21 $ hg up -C 0
22 22 1 files updated, 0 files merged, 0 files removed, 0 files unresolved
23 23 $ hg branch branchb
24 24 marked working directory as branch branchb
25 25 (branches are permanent and global, did you want a bookmark?)
26 26 $ echo b >> b
27 27 $ hg ci -Am b0 -d '6 0'
28 28 adding b
29 29 $ hg up -C brancha
30 30 1 files updated, 0 files merged, 1 files removed, 0 files unresolved
31 31 $ echo a >> a
32 32 $ hg ci -m a4 -d '5 0'
33 33 $ echo a >> a
34 34 $ hg ci -m a5 -d '7 0'
35 35 $ echo a >> a
36 36 $ hg ci -m a6 -d '8 0'
37 37 $ hg up -C branchb
38 38 2 files updated, 0 files merged, 0 files removed, 0 files unresolved
39 39 $ echo b >> b
40 40 $ hg ci -m b1 -d '9 0'
41 $ hg up -C 0
42 0 files updated, 0 files merged, 1 files removed, 0 files unresolved
43 $ echo c >> c
44 $ hg branch branchc
45 marked working directory as branch branchc
46 (branches are permanent and global, did you want a bookmark?)
47 $ hg ci -Am c0 -d '10 0'
48 adding c
49 $ hg up -C brancha
50 1 files updated, 0 files merged, 1 files removed, 0 files unresolved
51 $ hg ci --close-branch -m a7x -d '11 0'
52 $ hg up -C branchb
53 2 files updated, 0 files merged, 0 files removed, 0 files unresolved
54 $ hg ci --close-branch -m b2x -d '12 0'
55 $ hg up -C branchc
56 1 files updated, 0 files merged, 1 files removed, 0 files unresolved
57 $ hg merge branchb
58 1 files updated, 0 files merged, 0 files removed, 0 files unresolved
59 (branch merge, don't forget to commit)
60 $ hg ci -m c1 -d '13 0'
41 61 $ cd ..
42 62
43 63 convert with datesort
44 64
45 65 $ hg convert --datesort t t-datesort
46 66 initializing destination t-datesort repository
47 67 scanning source...
48 68 sorting...
49 69 converting...
50 8 a0
51 7 a1
52 6 a2
53 5 a3
54 4 a4
55 3 b0
56 2 a5
57 1 a6
58 0 b1
70 12 a0
71 11 a1
72 10 a2
73 9 a3
74 8 a4
75 7 b0
76 6 a5
77 5 a6
78 4 b1
79 3 c0
80 2 a7x
81 1 b2x
82 0 c1
59 83
60 84 graph converted repo
61 85
62 86 $ hg -R t-datesort glog --template '{rev} "{desc}"\n'
63 o 8 "b1"
64 |
65 | o 7 "a6"
87 o 12 "c1"
88 |\
89 | o 11 "b2x"
66 90 | |
67 | o 6 "a5"
68 | |
69 o | 5 "b0"
70 | |
91 | | o 10 "a7x"
92 | | |
93 o | | 9 "c0"
94 | | |
95 | o | 8 "b1"
96 | | |
97 | | o 7 "a6"
98 | | |
99 | | o 6 "a5"
100 | | |
101 | o | 5 "b0"
102 |/ /
71 103 | o 4 "a4"
72 104 | |
73 105 | o 3 "a3"
74 106 | |
75 107 | o 2 "a2"
76 108 | |
77 109 | o 1 "a1"
78 110 |/
79 111 o 0 "a0"
80 112
81 113
82 114 convert with datesort (default mode)
83 115
84 116 $ hg convert t t-sourcesort
85 117 initializing destination t-sourcesort repository
86 118 scanning source...
87 119 sorting...
88 120 converting...
89 8 a0
90 7 a1
91 6 a2
92 5 a3
93 4 b0
94 3 a4
95 2 a5
96 1 a6
97 0 b1
121 12 a0
122 11 a1
123 10 a2
124 9 a3
125 8 b0
126 7 a4
127 6 a5
128 5 a6
129 4 b1
130 3 c0
131 2 a7x
132 1 b2x
133 0 c1
98 134
99 135 graph converted repo
100 136
101 137 $ hg -R t-sourcesort glog --template '{rev} "{desc}"\n'
102 o 8 "b1"
103 |
104 | o 7 "a6"
138 o 12 "c1"
139 |\
140 | o 11 "b2x"
105 141 | |
106 | o 6 "a5"
107 | |
108 | o 5 "a4"
109 | |
110 o | 4 "b0"
111 | |
142 | | o 10 "a7x"
143 | | |
144 o | | 9 "c0"
145 | | |
146 | o | 8 "b1"
147 | | |
148 | | o 7 "a6"
149 | | |
150 | | o 6 "a5"
151 | | |
152 | | o 5 "a4"
153 | | |
154 | o | 4 "b0"
155 |/ /
112 156 | o 3 "a3"
113 157 | |
114 158 | o 2 "a2"
115 159 | |
116 160 | o 1 "a1"
117 161 |/
118 162 o 0 "a0"
119 163
164
165 convert with closesort
166
167 $ hg convert --closesort t t-closesort
168 initializing destination t-closesort repository
169 scanning source...
170 sorting...
171 converting...
172 12 a0
173 11 a1
174 10 a2
175 9 a3
176 8 b0
177 7 a4
178 6 a5
179 5 a6
180 4 a7x
181 3 b1
182 2 b2x
183 1 c0
184 0 c1
185
186 graph converted repo
187
188 $ hg -R t-closesort glog --template '{rev} "{desc}"\n'
189 o 12 "c1"
190 |\
191 | o 11 "c0"
192 | |
193 o | 10 "b2x"
194 | |
195 o | 9 "b1"
196 | |
197 | | o 8 "a7x"
198 | | |
199 | | o 7 "a6"
200 | | |
201 | | o 6 "a5"
202 | | |
203 | | o 5 "a4"
204 | | |
205 o | | 4 "b0"
206 |/ /
207 | o 3 "a3"
208 | |
209 | o 2 "a2"
210 | |
211 | o 1 "a1"
212 |/
213 o 0 "a0"
214
@@ -1,455 +1,458 b''
1 1 $ cat >> $HGRCPATH <<EOF
2 2 > [extensions]
3 3 > convert=
4 4 > [convert]
5 5 > hg.saverev=False
6 6 > EOF
7 7 $ hg help convert
8 8 hg convert [OPTION]... SOURCE [DEST [REVMAP]]
9 9
10 10 convert a foreign SCM repository to a Mercurial one.
11 11
12 12 Accepted source formats [identifiers]:
13 13
14 14 - Mercurial [hg]
15 15 - CVS [cvs]
16 16 - Darcs [darcs]
17 17 - git [git]
18 18 - Subversion [svn]
19 19 - Monotone [mtn]
20 20 - GNU Arch [gnuarch]
21 21 - Bazaar [bzr]
22 22 - Perforce [p4]
23 23
24 24 Accepted destination formats [identifiers]:
25 25
26 26 - Mercurial [hg]
27 27 - Subversion [svn] (history on branches is not preserved)
28 28
29 29 If no revision is given, all revisions will be converted. Otherwise,
30 30 convert will only import up to the named revision (given in a format
31 31 understood by the source).
32 32
33 33 If no destination directory name is specified, it defaults to the basename
34 34 of the source with "-hg" appended. If the destination repository doesn't
35 35 exist, it will be created.
36 36
37 37 By default, all sources except Mercurial will use --branchsort. Mercurial
38 38 uses --sourcesort to preserve original revision numbers order. Sort modes
39 39 have the following effects:
40 40
41 41 --branchsort convert from parent to child revision when possible, which
42 42 means branches are usually converted one after the other.
43 43 It generates more compact repositories.
44 44 --datesort sort revisions by date. Converted repositories have good-
45 45 looking changelogs but are often an order of magnitude
46 46 larger than the same ones generated by --branchsort.
47 47 --sourcesort try to preserve source revisions order, only supported by
48 48 Mercurial sources.
49 --closesort try to move closed revisions as close as possible to parent
50 branches, only supported by Mercurial sources.
49 51
50 52 If "REVMAP" isn't given, it will be put in a default location
51 53 ("<dest>/.hg/shamap" by default). The "REVMAP" is a simple text file that
52 54 maps each source commit ID to the destination ID for that revision, like
53 55 so:
54 56
55 57 <source ID> <destination ID>
56 58
57 59 If the file doesn't exist, it's automatically created. It's updated on
58 60 each commit copied, so "hg convert" can be interrupted and can be run
59 61 repeatedly to copy new commits.
60 62
61 63 The authormap is a simple text file that maps each source commit author to
62 64 a destination commit author. It is handy for source SCMs that use unix
63 65 logins to identify authors (e.g.: CVS). One line per author mapping and
64 66 the line format is:
65 67
66 68 source author = destination author
67 69
68 70 Empty lines and lines starting with a "#" are ignored.
69 71
70 72 The filemap is a file that allows filtering and remapping of files and
71 73 directories. Each line can contain one of the following directives:
72 74
73 75 include path/to/file-or-dir
74 76
75 77 exclude path/to/file-or-dir
76 78
77 79 rename path/to/source path/to/destination
78 80
79 81 Comment lines start with "#". A specified path matches if it equals the
80 82 full relative name of a file or one of its parent directories. The
81 83 "include" or "exclude" directive with the longest matching path applies,
82 84 so line order does not matter.
83 85
84 86 The "include" directive causes a file, or all files under a directory, to
85 87 be included in the destination repository, and the exclusion of all other
86 88 files and directories not explicitly included. The "exclude" directive
87 89 causes files or directories to be omitted. The "rename" directive renames
88 90 a file or directory if it is converted. To rename from a subdirectory into
89 91 the root of the repository, use "." as the path to rename to.
90 92
91 93 The splicemap is a file that allows insertion of synthetic history,
92 94 letting you specify the parents of a revision. This is useful if you want
93 95 to e.g. give a Subversion merge two parents, or graft two disconnected
94 96 series of history together. Each entry contains a key, followed by a
95 97 space, followed by one or two comma-separated values:
96 98
97 99 key parent1, parent2
98 100
99 101 The key is the revision ID in the source revision control system whose
100 102 parents should be modified (same format as a key in .hg/shamap). The
101 103 values are the revision IDs (in either the source or destination revision
102 104 control system) that should be used as the new parents for that node. For
103 105 example, if you have merged "release-1.0" into "trunk", then you should
104 106 specify the revision on "trunk" as the first parent and the one on the
105 107 "release-1.0" branch as the second.
106 108
107 109 The branchmap is a file that allows you to rename a branch when it is
108 110 being brought in from whatever external repository. When used in
109 111 conjunction with a splicemap, it allows for a powerful combination to help
110 112 fix even the most badly mismanaged repositories and turn them into nicely
111 113 structured Mercurial repositories. The branchmap contains lines of the
112 114 form:
113 115
114 116 original_branch_name new_branch_name
115 117
116 118 where "original_branch_name" is the name of the branch in the source
117 119 repository, and "new_branch_name" is the name of the branch is the
118 120 destination repository. No whitespace is allowed in the branch names. This
119 121 can be used to (for instance) move code in one repository from "default"
120 122 to a named branch.
121 123
122 124 Mercurial Source
123 125 ################
124 126
125 127 The Mercurial source recognizes the following configuration options, which
126 128 you can set on the command line with "--config":
127 129
128 130 convert.hg.ignoreerrors
129 131 ignore integrity errors when reading. Use it to fix
130 132 Mercurial repositories with missing revlogs, by converting
131 133 from and to Mercurial. Default is False.
132 134 convert.hg.saverev
133 135 store original revision ID in changeset (forces target IDs
134 136 to change). It takes a boolean argument and defaults to
135 137 False.
136 138 convert.hg.startrev
137 139 convert start revision and its descendants. It takes a hg
138 140 revision identifier and defaults to 0.
139 141
140 142 CVS Source
141 143 ##########
142 144
143 145 CVS source will use a sandbox (i.e. a checked-out copy) from CVS to
144 146 indicate the starting point of what will be converted. Direct access to
145 147 the repository files is not needed, unless of course the repository is
146 148 ":local:". The conversion uses the top level directory in the sandbox to
147 149 find the CVS repository, and then uses CVS rlog commands to find files to
148 150 convert. This means that unless a filemap is given, all files under the
149 151 starting directory will be converted, and that any directory
150 152 reorganization in the CVS sandbox is ignored.
151 153
152 154 The following options can be used with "--config":
153 155
154 156 convert.cvsps.cache
155 157 Set to False to disable remote log caching, for testing and
156 158 debugging purposes. Default is True.
157 159 convert.cvsps.fuzz
158 160 Specify the maximum time (in seconds) that is allowed
159 161 between commits with identical user and log message in a
160 162 single changeset. When very large files were checked in as
161 163 part of a changeset then the default may not be long enough.
162 164 The default is 60.
163 165 convert.cvsps.mergeto
164 166 Specify a regular expression to which commit log messages
165 167 are matched. If a match occurs, then the conversion process
166 168 will insert a dummy revision merging the branch on which
167 169 this log message occurs to the branch indicated in the
168 170 regex. Default is "{{mergetobranch ([-\w]+)}}"
169 171 convert.cvsps.mergefrom
170 172 Specify a regular expression to which commit log messages
171 173 are matched. If a match occurs, then the conversion process
172 174 will add the most recent revision on the branch indicated in
173 175 the regex as the second parent of the changeset. Default is
174 176 "{{mergefrombranch ([-\w]+)}}"
175 177 convert.localtimezone
176 178 use local time (as determined by the TZ environment
177 179 variable) for changeset date/times. The default is False
178 180 (use UTC).
179 181 hooks.cvslog Specify a Python function to be called at the end of
180 182 gathering the CVS log. The function is passed a list with
181 183 the log entries, and can modify the entries in-place, or add
182 184 or delete them.
183 185 hooks.cvschangesets
184 186 Specify a Python function to be called after the changesets
185 187 are calculated from the CVS log. The function is passed a
186 188 list with the changeset entries, and can modify the
187 189 changesets in-place, or add or delete them.
188 190
189 191 An additional "debugcvsps" Mercurial command allows the builtin changeset
190 192 merging code to be run without doing a conversion. Its parameters and
191 193 output are similar to that of cvsps 2.1. Please see the command help for
192 194 more details.
193 195
194 196 Subversion Source
195 197 #################
196 198
197 199 Subversion source detects classical trunk/branches/tags layouts. By
198 200 default, the supplied "svn://repo/path/" source URL is converted as a
199 201 single branch. If "svn://repo/path/trunk" exists it replaces the default
200 202 branch. If "svn://repo/path/branches" exists, its subdirectories are
201 203 listed as possible branches. If "svn://repo/path/tags" exists, it is
202 204 looked for tags referencing converted branches. Default "trunk",
203 205 "branches" and "tags" values can be overridden with following options. Set
204 206 them to paths relative to the source URL, or leave them blank to disable
205 207 auto detection.
206 208
207 209 The following options can be set with "--config":
208 210
209 211 convert.svn.branches
210 212 specify the directory containing branches. The default is
211 213 "branches".
212 214 convert.svn.tags
213 215 specify the directory containing tags. The default is
214 216 "tags".
215 217 convert.svn.trunk
216 218 specify the name of the trunk branch. The default is
217 219 "trunk".
218 220 convert.localtimezone
219 221 use local time (as determined by the TZ environment
220 222 variable) for changeset date/times. The default is False
221 223 (use UTC).
222 224
223 225 Source history can be retrieved starting at a specific revision, instead
224 226 of being integrally converted. Only single branch conversions are
225 227 supported.
226 228
227 229 convert.svn.startrev
228 230 specify start Subversion revision number. The default is 0.
229 231
230 232 Perforce Source
231 233 ###############
232 234
233 235 The Perforce (P4) importer can be given a p4 depot path or a client
234 236 specification as source. It will convert all files in the source to a flat
235 237 Mercurial repository, ignoring labels, branches and integrations. Note
236 238 that when a depot path is given you then usually should specify a target
237 239 directory, because otherwise the target may be named "...-hg".
238 240
239 241 It is possible to limit the amount of source history to be converted by
240 242 specifying an initial Perforce revision:
241 243
242 244 convert.p4.startrev
243 245 specify initial Perforce revision (a Perforce changelist
244 246 number).
245 247
246 248 Mercurial Destination
247 249 #####################
248 250
249 251 The following options are supported:
250 252
251 253 convert.hg.clonebranches
252 254 dispatch source branches in separate clones. The default is
253 255 False.
254 256 convert.hg.tagsbranch
255 257 branch name for tag revisions, defaults to "default".
256 258 convert.hg.usebranchnames
257 259 preserve branch names. The default is True.
258 260
259 261 options:
260 262
261 263 -s --source-type TYPE source repository type
262 264 -d --dest-type TYPE destination repository type
263 265 -r --rev REV import up to target revision REV
264 266 -A --authormap FILE remap usernames using this file
265 267 --filemap FILE remap file names using contents of file
266 268 --splicemap FILE splice synthesized history into place
267 269 --branchmap FILE change branch names while converting
268 270 --branchsort try to sort changesets by branches
269 271 --datesort try to sort changesets by date
270 272 --sourcesort preserve source changesets order
273 --closesort try to reorder closed revisions
271 274
272 275 use "hg -v help convert" to show the global options
273 276 $ hg init a
274 277 $ cd a
275 278 $ echo a > a
276 279 $ hg ci -d'0 0' -Ama
277 280 adding a
278 281 $ hg cp a b
279 282 $ hg ci -d'1 0' -mb
280 283 $ hg rm a
281 284 $ hg ci -d'2 0' -mc
282 285 $ hg mv b a
283 286 $ hg ci -d'3 0' -md
284 287 $ echo a >> a
285 288 $ hg ci -d'4 0' -me
286 289 $ cd ..
287 290 $ hg convert a 2>&1 | grep -v 'subversion python bindings could not be loaded'
288 291 assuming destination a-hg
289 292 initializing destination a-hg repository
290 293 scanning source...
291 294 sorting...
292 295 converting...
293 296 4 a
294 297 3 b
295 298 2 c
296 299 1 d
297 300 0 e
298 301 $ hg --cwd a-hg pull ../a
299 302 pulling from ../a
300 303 searching for changes
301 304 no changes found
302 305
303 306 conversion to existing file should fail
304 307
305 308 $ touch bogusfile
306 309 $ hg convert a bogusfile
307 310 initializing destination bogusfile repository
308 311 abort: cannot create new bundle repository
309 312 [255]
310 313
311 314 #if unix-permissions
312 315
313 316 conversion to dir without permissions should fail
314 317
315 318 $ mkdir bogusdir
316 319 $ chmod 000 bogusdir
317 320
318 321 $ hg convert a bogusdir
319 322 abort: Permission denied: 'bogusdir'
320 323 [255]
321 324
322 325 user permissions should succeed
323 326
324 327 $ chmod 700 bogusdir
325 328 $ hg convert a bogusdir
326 329 initializing destination bogusdir repository
327 330 scanning source...
328 331 sorting...
329 332 converting...
330 333 4 a
331 334 3 b
332 335 2 c
333 336 1 d
334 337 0 e
335 338
336 339 #endif
337 340
338 341 test pre and post conversion actions
339 342
340 343 $ echo 'include b' > filemap
341 344 $ hg convert --debug --filemap filemap a partialb | \
342 345 > grep 'run hg'
343 346 run hg source pre-conversion action
344 347 run hg sink pre-conversion action
345 348 run hg sink post-conversion action
346 349 run hg source post-conversion action
347 350
348 351 converting empty dir should fail "nicely
349 352
350 353 $ mkdir emptydir
351 354
352 355 override $PATH to ensure p4 not visible; use $PYTHON in case we're
353 356 running from a devel copy, not a temp installation
354 357
355 358 $ PATH="$BINDIR" $PYTHON "$BINDIR"/hg convert emptydir
356 359 assuming destination emptydir-hg
357 360 initializing destination emptydir-hg repository
358 361 emptydir does not look like a CVS checkout
359 362 emptydir does not look like a Git repository
360 363 emptydir does not look like a Subversion repository
361 364 emptydir is not a local Mercurial repository
362 365 emptydir does not look like a darcs repository
363 366 emptydir does not look like a monotone repository
364 367 emptydir does not look like a GNU Arch repository
365 368 emptydir does not look like a Bazaar repository
366 369 cannot find required "p4" tool
367 370 abort: emptydir: missing or unsupported repository
368 371 [255]
369 372
370 373 convert with imaginary source type
371 374
372 375 $ hg convert --source-type foo a a-foo
373 376 initializing destination a-foo repository
374 377 abort: foo: invalid source repository type
375 378 [255]
376 379
377 380 convert with imaginary sink type
378 381
379 382 $ hg convert --dest-type foo a a-foo
380 383 abort: foo: invalid destination repository type
381 384 [255]
382 385
383 386 testing: convert must not produce duplicate entries in fncache
384 387
385 388 $ hg convert a b
386 389 initializing destination b repository
387 390 scanning source...
388 391 sorting...
389 392 converting...
390 393 4 a
391 394 3 b
392 395 2 c
393 396 1 d
394 397 0 e
395 398
396 399 contents of fncache file:
397 400
398 401 $ cat b/.hg/store/fncache | sort
399 402 data/a.i
400 403 data/b.i
401 404
402 405 test bogus URL
403 406
404 407 $ hg convert -q bzr+ssh://foobar@selenic.com/baz baz
405 408 abort: bzr+ssh://foobar@selenic.com/baz: missing or unsupported repository
406 409 [255]
407 410
408 411 test revset converted() lookup
409 412
410 413 $ hg --config convert.hg.saverev=True convert a c
411 414 initializing destination c repository
412 415 scanning source...
413 416 sorting...
414 417 converting...
415 418 4 a
416 419 3 b
417 420 2 c
418 421 1 d
419 422 0 e
420 423 $ echo f > c/f
421 424 $ hg -R c ci -d'0 0' -Amf
422 425 adding f
423 426 created new head
424 427 $ hg -R c log -r "converted(09d945a62ce6)"
425 428 changeset: 1:98c3dd46a874
426 429 user: test
427 430 date: Thu Jan 01 00:00:01 1970 +0000
428 431 summary: b
429 432
430 433 $ hg -R c log -r "converted()"
431 434 changeset: 0:31ed57b2037c
432 435 user: test
433 436 date: Thu Jan 01 00:00:00 1970 +0000
434 437 summary: a
435 438
436 439 changeset: 1:98c3dd46a874
437 440 user: test
438 441 date: Thu Jan 01 00:00:01 1970 +0000
439 442 summary: b
440 443
441 444 changeset: 2:3b9ca06ef716
442 445 user: test
443 446 date: Thu Jan 01 00:00:02 1970 +0000
444 447 summary: c
445 448
446 449 changeset: 3:4e0debd37cf2
447 450 user: test
448 451 date: Thu Jan 01 00:00:03 1970 +0000
449 452 summary: d
450 453
451 454 changeset: 4:9de3bc9349c5
452 455 user: test
453 456 date: Thu Jan 01 00:00:04 1970 +0000
454 457 summary: e
455 458
General Comments 0
You need to be logged in to leave comments. Login now