##// END OF EJS Templates
convert: deprecate --authors in preference for --authormap...
Martin Geisler -
r12198:0c67a58f default
parent child Browse files
Show More
@@ -1,318 +1,321 b''
1 # convert.py Foreign SCM converter
1 # convert.py Foreign SCM converter
2 #
2 #
3 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
3 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
4 #
4 #
5 # This software may be used and distributed according to the terms of the
5 # This software may be used and distributed according to the terms of the
6 # GNU General Public License version 2 or any later version.
6 # GNU General Public License version 2 or any later version.
7
7
8 '''import revisions from foreign VCS repositories into Mercurial'''
8 '''import revisions from foreign VCS repositories into Mercurial'''
9
9
10 import convcmd
10 import convcmd
11 import cvsps
11 import cvsps
12 import subversion
12 import subversion
13 from mercurial import commands
13 from mercurial import commands
14 from mercurial.i18n import _
14 from mercurial.i18n import _
15
15
16 # Commands definition was moved elsewhere to ease demandload job.
16 # Commands definition was moved elsewhere to ease demandload job.
17
17
18 def convert(ui, src, dest=None, revmapfile=None, **opts):
18 def convert(ui, src, dest=None, revmapfile=None, **opts):
19 """convert a foreign SCM repository to a Mercurial one.
19 """convert a foreign SCM repository to a Mercurial one.
20
20
21 Accepted source formats [identifiers]:
21 Accepted source formats [identifiers]:
22
22
23 - Mercurial [hg]
23 - Mercurial [hg]
24 - CVS [cvs]
24 - CVS [cvs]
25 - Darcs [darcs]
25 - Darcs [darcs]
26 - git [git]
26 - git [git]
27 - Subversion [svn]
27 - Subversion [svn]
28 - Monotone [mtn]
28 - Monotone [mtn]
29 - GNU Arch [gnuarch]
29 - GNU Arch [gnuarch]
30 - Bazaar [bzr]
30 - Bazaar [bzr]
31 - Perforce [p4]
31 - Perforce [p4]
32
32
33 Accepted destination formats [identifiers]:
33 Accepted destination formats [identifiers]:
34
34
35 - Mercurial [hg]
35 - Mercurial [hg]
36 - Subversion [svn] (history on branches is not preserved)
36 - Subversion [svn] (history on branches is not preserved)
37
37
38 If no revision is given, all revisions will be converted.
38 If no revision is given, all revisions will be converted.
39 Otherwise, convert will only import up to the named revision
39 Otherwise, convert will only import up to the named revision
40 (given in a format understood by the source).
40 (given in a format understood by the source).
41
41
42 If no destination directory name is specified, it defaults to the
42 If no destination directory name is specified, it defaults to the
43 basename of the source with ``-hg`` appended. If the destination
43 basename of the source with ``-hg`` appended. If the destination
44 repository doesn't exist, it will be created.
44 repository doesn't exist, it will be created.
45
45
46 By default, all sources except Mercurial will use --branchsort.
46 By default, all sources except Mercurial will use --branchsort.
47 Mercurial uses --sourcesort to preserve original revision numbers
47 Mercurial uses --sourcesort to preserve original revision numbers
48 order. Sort modes have the following effects:
48 order. Sort modes have the following effects:
49
49
50 --branchsort convert from parent to child revision when possible,
50 --branchsort convert from parent to child revision when possible,
51 which means branches are usually converted one after
51 which means branches are usually converted one after
52 the other. It generates more compact repositories.
52 the other. It generates more compact repositories.
53
53
54 --datesort sort revisions by date. Converted repositories have
54 --datesort sort revisions by date. Converted repositories have
55 good-looking changelogs but are often an order of
55 good-looking changelogs but are often an order of
56 magnitude larger than the same ones generated by
56 magnitude larger than the same ones generated by
57 --branchsort.
57 --branchsort.
58
58
59 --sourcesort try to preserve source revisions order, only
59 --sourcesort try to preserve source revisions order, only
60 supported by Mercurial sources.
60 supported by Mercurial sources.
61
61
62 If <REVMAP> isn't given, it will be put in a default location
62 If <REVMAP> isn't given, it will be put in a default location
63 (<dest>/.hg/shamap by default). The <REVMAP> is a simple text file
63 (<dest>/.hg/shamap by default). The <REVMAP> is a simple text file
64 that maps each source commit ID to the destination ID for that
64 that maps each source commit ID to the destination ID for that
65 revision, like so::
65 revision, like so::
66
66
67 <source ID> <destination ID>
67 <source ID> <destination ID>
68
68
69 If the file doesn't exist, it's automatically created. It's
69 If the file doesn't exist, it's automatically created. It's
70 updated on each commit copied, so :hg:`convert` can be interrupted
70 updated on each commit copied, so :hg:`convert` can be interrupted
71 and can be run repeatedly to copy new commits.
71 and can be run repeatedly to copy new commits.
72
72
73 The username mapping file is a simple text file that maps each
73 The authormap is a simple text file that maps each source commit
74 source commit author to a destination commit author. It is handy
74 author to a destination commit author. It is handy for source SCMs
75 for source SCMs that use unix logins to identify authors (eg:
75 that use unix logins to identify authors (eg: CVS). One line per
76 CVS). One line per author mapping and the line format is::
76 author mapping and the line format is::
77
77
78 source author = destination author
78 source author = destination author
79
79
80 Empty lines and lines starting with a ``#`` are ignored.
80 Empty lines and lines starting with a ``#`` are ignored.
81
81
82 The filemap is a file that allows filtering and remapping of files
82 The filemap is a file that allows filtering and remapping of files
83 and directories. Each line can contain one of the following
83 and directories. Each line can contain one of the following
84 directives::
84 directives::
85
85
86 include path/to/file-or-dir
86 include path/to/file-or-dir
87
87
88 exclude path/to/file-or-dir
88 exclude path/to/file-or-dir
89
89
90 rename path/to/source path/to/destination
90 rename path/to/source path/to/destination
91
91
92 Comment lines start with ``#``. A specified path matches if it
92 Comment lines start with ``#``. A specified path matches if it
93 equals the full relative name of a file or one of its parent
93 equals the full relative name of a file or one of its parent
94 directories. The ``include`` or ``exclude`` directive with the
94 directories. The ``include`` or ``exclude`` directive with the
95 longest matching path applies, so line order does not matter.
95 longest matching path applies, so line order does not matter.
96
96
97 The ``include`` directive causes a file, or all files under a
97 The ``include`` directive causes a file, or all files under a
98 directory, to be included in the destination repository, and the
98 directory, to be included in the destination repository, and the
99 exclusion of all other files and directories not explicitly
99 exclusion of all other files and directories not explicitly
100 included. The ``exclude`` directive causes files or directories to
100 included. The ``exclude`` directive causes files or directories to
101 be omitted. The ``rename`` directive renames a file or directory if
101 be omitted. The ``rename`` directive renames a file or directory if
102 it is converted. To rename from a subdirectory into the root of
102 it is converted. To rename from a subdirectory into the root of
103 the repository, use ``.`` as the path to rename to.
103 the repository, use ``.`` as the path to rename to.
104
104
105 The splicemap is a file that allows insertion of synthetic
105 The splicemap is a file that allows insertion of synthetic
106 history, letting you specify the parents of a revision. This is
106 history, letting you specify the parents of a revision. This is
107 useful if you want to e.g. give a Subversion merge two parents, or
107 useful if you want to e.g. give a Subversion merge two parents, or
108 graft two disconnected series of history together. Each entry
108 graft two disconnected series of history together. Each entry
109 contains a key, followed by a space, followed by one or two
109 contains a key, followed by a space, followed by one or two
110 comma-separated values::
110 comma-separated values::
111
111
112 key parent1, parent2
112 key parent1, parent2
113
113
114 The key is the revision ID in the source
114 The key is the revision ID in the source
115 revision control system whose parents should be modified (same
115 revision control system whose parents should be modified (same
116 format as a key in .hg/shamap). The values are the revision IDs
116 format as a key in .hg/shamap). The values are the revision IDs
117 (in either the source or destination revision control system) that
117 (in either the source or destination revision control system) that
118 should be used as the new parents for that node. For example, if
118 should be used as the new parents for that node. For example, if
119 you have merged "release-1.0" into "trunk", then you should
119 you have merged "release-1.0" into "trunk", then you should
120 specify the revision on "trunk" as the first parent and the one on
120 specify the revision on "trunk" as the first parent and the one on
121 the "release-1.0" branch as the second.
121 the "release-1.0" branch as the second.
122
122
123 The branchmap is a file that allows you to rename a branch when it is
123 The branchmap is a file that allows you to rename a branch when it is
124 being brought in from whatever external repository. When used in
124 being brought in from whatever external repository. When used in
125 conjunction with a splicemap, it allows for a powerful combination
125 conjunction with a splicemap, it allows for a powerful combination
126 to help fix even the most badly mismanaged repositories and turn them
126 to help fix even the most badly mismanaged repositories and turn them
127 into nicely structured Mercurial repositories. The branchmap contains
127 into nicely structured Mercurial repositories. The branchmap contains
128 lines of the form::
128 lines of the form::
129
129
130 original_branch_name new_branch_name
130 original_branch_name new_branch_name
131
131
132 where "original_branch_name" is the name of the branch in the
132 where "original_branch_name" is the name of the branch in the
133 source repository, and "new_branch_name" is the name of the branch
133 source repository, and "new_branch_name" is the name of the branch
134 is the destination repository. No whitespace is allowed in the
134 is the destination repository. No whitespace is allowed in the
135 branch names. This can be used to (for instance) move code in one
135 branch names. This can be used to (for instance) move code in one
136 repository from "default" to a named branch.
136 repository from "default" to a named branch.
137
137
138 Mercurial Source
138 Mercurial Source
139 ----------------
139 ----------------
140
140
141 --config convert.hg.ignoreerrors=False (boolean)
141 --config convert.hg.ignoreerrors=False (boolean)
142 ignore integrity errors when reading. Use it to fix Mercurial
142 ignore integrity errors when reading. Use it to fix Mercurial
143 repositories with missing revlogs, by converting from and to
143 repositories with missing revlogs, by converting from and to
144 Mercurial.
144 Mercurial.
145 --config convert.hg.saverev=False (boolean)
145 --config convert.hg.saverev=False (boolean)
146 store original revision ID in changeset (forces target IDs to
146 store original revision ID in changeset (forces target IDs to
147 change)
147 change)
148 --config convert.hg.startrev=0 (hg revision identifier)
148 --config convert.hg.startrev=0 (hg revision identifier)
149 convert start revision and its descendants
149 convert start revision and its descendants
150
150
151 CVS Source
151 CVS Source
152 ----------
152 ----------
153
153
154 CVS source will use a sandbox (i.e. a checked-out copy) from CVS
154 CVS source will use a sandbox (i.e. a checked-out copy) from CVS
155 to indicate the starting point of what will be converted. Direct
155 to indicate the starting point of what will be converted. Direct
156 access to the repository files is not needed, unless of course the
156 access to the repository files is not needed, unless of course the
157 repository is :local:. The conversion uses the top level directory
157 repository is :local:. The conversion uses the top level directory
158 in the sandbox to find the CVS repository, and then uses CVS rlog
158 in the sandbox to find the CVS repository, and then uses CVS rlog
159 commands to find files to convert. This means that unless a
159 commands to find files to convert. This means that unless a
160 filemap is given, all files under the starting directory will be
160 filemap is given, all files under the starting directory will be
161 converted, and that any directory reorganization in the CVS
161 converted, and that any directory reorganization in the CVS
162 sandbox is ignored.
162 sandbox is ignored.
163
163
164 The options shown are the defaults.
164 The options shown are the defaults.
165
165
166 --config convert.cvsps.cache=True (boolean)
166 --config convert.cvsps.cache=True (boolean)
167 Set to False to disable remote log caching, for testing and
167 Set to False to disable remote log caching, for testing and
168 debugging purposes.
168 debugging purposes.
169 --config convert.cvsps.fuzz=60 (integer)
169 --config convert.cvsps.fuzz=60 (integer)
170 Specify the maximum time (in seconds) that is allowed between
170 Specify the maximum time (in seconds) that is allowed between
171 commits with identical user and log message in a single
171 commits with identical user and log message in a single
172 changeset. When very large files were checked in as part of a
172 changeset. When very large files were checked in as part of a
173 changeset then the default may not be long enough.
173 changeset then the default may not be long enough.
174 --config convert.cvsps.mergeto='{{mergetobranch ([-\\w]+)}}'
174 --config convert.cvsps.mergeto='{{mergetobranch ([-\\w]+)}}'
175 Specify a regular expression to which commit log messages are
175 Specify a regular expression to which commit log messages are
176 matched. If a match occurs, then the conversion process will
176 matched. If a match occurs, then the conversion process will
177 insert a dummy revision merging the branch on which this log
177 insert a dummy revision merging the branch on which this log
178 message occurs to the branch indicated in the regex.
178 message occurs to the branch indicated in the regex.
179 --config convert.cvsps.mergefrom='{{mergefrombranch ([-\\w]+)}}'
179 --config convert.cvsps.mergefrom='{{mergefrombranch ([-\\w]+)}}'
180 Specify a regular expression to which commit log messages are
180 Specify a regular expression to which commit log messages are
181 matched. If a match occurs, then the conversion process will
181 matched. If a match occurs, then the conversion process will
182 add the most recent revision on the branch indicated in the
182 add the most recent revision on the branch indicated in the
183 regex as the second parent of the changeset.
183 regex as the second parent of the changeset.
184 --config hook.cvslog
184 --config hook.cvslog
185 Specify a Python function to be called at the end of gathering
185 Specify a Python function to be called at the end of gathering
186 the CVS log. The function is passed a list with the log entries,
186 the CVS log. The function is passed a list with the log entries,
187 and can modify the entries in-place, or add or delete them.
187 and can modify the entries in-place, or add or delete them.
188 --config hook.cvschangesets
188 --config hook.cvschangesets
189 Specify a Python function to be called after the changesets
189 Specify a Python function to be called after the changesets
190 are calculated from the the CVS log. The function is passed
190 are calculated from the the CVS log. The function is passed
191 a list with the changeset entries, and can modify the changesets
191 a list with the changeset entries, and can modify the changesets
192 in-place, or add or delete them.
192 in-place, or add or delete them.
193
193
194 An additional "debugcvsps" Mercurial command allows the builtin
194 An additional "debugcvsps" Mercurial command allows the builtin
195 changeset merging code to be run without doing a conversion. Its
195 changeset merging code to be run without doing a conversion. Its
196 parameters and output are similar to that of cvsps 2.1. Please see
196 parameters and output are similar to that of cvsps 2.1. Please see
197 the command help for more details.
197 the command help for more details.
198
198
199 Subversion Source
199 Subversion Source
200 -----------------
200 -----------------
201
201
202 Subversion source detects classical trunk/branches/tags layouts.
202 Subversion source detects classical trunk/branches/tags layouts.
203 By default, the supplied "svn://repo/path/" source URL is
203 By default, the supplied "svn://repo/path/" source URL is
204 converted as a single branch. If "svn://repo/path/trunk" exists it
204 converted as a single branch. If "svn://repo/path/trunk" exists it
205 replaces the default branch. If "svn://repo/path/branches" exists,
205 replaces the default branch. If "svn://repo/path/branches" exists,
206 its subdirectories are listed as possible branches. If
206 its subdirectories are listed as possible branches. If
207 "svn://repo/path/tags" exists, it is looked for tags referencing
207 "svn://repo/path/tags" exists, it is looked for tags referencing
208 converted branches. Default "trunk", "branches" and "tags" values
208 converted branches. Default "trunk", "branches" and "tags" values
209 can be overridden with following options. Set them to paths
209 can be overridden with following options. Set them to paths
210 relative to the source URL, or leave them blank to disable auto
210 relative to the source URL, or leave them blank to disable auto
211 detection.
211 detection.
212
212
213 --config convert.svn.branches=branches (directory name)
213 --config convert.svn.branches=branches (directory name)
214 specify the directory containing branches
214 specify the directory containing branches
215 --config convert.svn.tags=tags (directory name)
215 --config convert.svn.tags=tags (directory name)
216 specify the directory containing tags
216 specify the directory containing tags
217 --config convert.svn.trunk=trunk (directory name)
217 --config convert.svn.trunk=trunk (directory name)
218 specify the name of the trunk branch
218 specify the name of the trunk branch
219
219
220 Source history can be retrieved starting at a specific revision,
220 Source history can be retrieved starting at a specific revision,
221 instead of being integrally converted. Only single branch
221 instead of being integrally converted. Only single branch
222 conversions are supported.
222 conversions are supported.
223
223
224 --config convert.svn.startrev=0 (svn revision number)
224 --config convert.svn.startrev=0 (svn revision number)
225 specify start Subversion revision.
225 specify start Subversion revision.
226
226
227 Perforce Source
227 Perforce Source
228 ---------------
228 ---------------
229
229
230 The Perforce (P4) importer can be given a p4 depot path or a
230 The Perforce (P4) importer can be given a p4 depot path or a
231 client specification as source. It will convert all files in the
231 client specification as source. It will convert all files in the
232 source to a flat Mercurial repository, ignoring labels, branches
232 source to a flat Mercurial repository, ignoring labels, branches
233 and integrations. Note that when a depot path is given you then
233 and integrations. Note that when a depot path is given you then
234 usually should specify a target directory, because otherwise the
234 usually should specify a target directory, because otherwise the
235 target may be named ...-hg.
235 target may be named ...-hg.
236
236
237 It is possible to limit the amount of source history to be
237 It is possible to limit the amount of source history to be
238 converted by specifying an initial Perforce revision.
238 converted by specifying an initial Perforce revision.
239
239
240 --config convert.p4.startrev=0 (perforce changelist number)
240 --config convert.p4.startrev=0 (perforce changelist number)
241 specify initial Perforce revision.
241 specify initial Perforce revision.
242
242
243 Mercurial Destination
243 Mercurial Destination
244 ---------------------
244 ---------------------
245
245
246 --config convert.hg.clonebranches=False (boolean)
246 --config convert.hg.clonebranches=False (boolean)
247 dispatch source branches in separate clones.
247 dispatch source branches in separate clones.
248 --config convert.hg.tagsbranch=default (branch name)
248 --config convert.hg.tagsbranch=default (branch name)
249 tag revisions branch name
249 tag revisions branch name
250 --config convert.hg.usebranchnames=True (boolean)
250 --config convert.hg.usebranchnames=True (boolean)
251 preserve branch names
251 preserve branch names
252
252
253 """
253 """
254 return convcmd.convert(ui, src, dest, revmapfile, **opts)
254 return convcmd.convert(ui, src, dest, revmapfile, **opts)
255
255
256 def debugsvnlog(ui, **opts):
256 def debugsvnlog(ui, **opts):
257 return subversion.debugsvnlog(ui, **opts)
257 return subversion.debugsvnlog(ui, **opts)
258
258
259 def debugcvsps(ui, *args, **opts):
259 def debugcvsps(ui, *args, **opts):
260 '''create changeset information from CVS
260 '''create changeset information from CVS
261
261
262 This command is intended as a debugging tool for the CVS to
262 This command is intended as a debugging tool for the CVS to
263 Mercurial converter, and can be used as a direct replacement for
263 Mercurial converter, and can be used as a direct replacement for
264 cvsps.
264 cvsps.
265
265
266 Hg debugcvsps reads the CVS rlog for current directory (or any
266 Hg debugcvsps reads the CVS rlog for current directory (or any
267 named directory) in the CVS repository, and converts the log to a
267 named directory) in the CVS repository, and converts the log to a
268 series of changesets based on matching commit log entries and
268 series of changesets based on matching commit log entries and
269 dates.'''
269 dates.'''
270 return cvsps.debugcvsps(ui, *args, **opts)
270 return cvsps.debugcvsps(ui, *args, **opts)
271
271
272 commands.norepo += " convert debugsvnlog debugcvsps"
272 commands.norepo += " convert debugsvnlog debugcvsps"
273
273
274 cmdtable = {
274 cmdtable = {
275 "convert":
275 "convert":
276 (convert,
276 (convert,
277 [('A', 'authors', '',
277 [('A', 'authors', '',
278 _('username mapping filename'), _('FILE')),
278 _('username mapping filename (DEPRECATED, use --authormap instead)'),
279 _('FILE')),
279 ('s', 'source-type', '',
280 ('s', 'source-type', '',
280 _('source repository type'), _('TYPE')),
281 _('source repository type'), _('TYPE')),
281 ('d', 'dest-type', '',
282 ('d', 'dest-type', '',
282 _('destination repository type'), _('TYPE')),
283 _('destination repository type'), _('TYPE')),
283 ('r', 'rev', '',
284 ('r', 'rev', '',
284 _('import up to target revision REV'), _('REV')),
285 _('import up to target revision REV'), _('REV')),
286 ('', 'authormap', '',
287 _('remap usernames using this file'), _('FILE')),
285 ('', 'filemap', '',
288 ('', 'filemap', '',
286 _('remap file names using contents of file'), _('FILE')),
289 _('remap file names using contents of file'), _('FILE')),
287 ('', 'splicemap', '',
290 ('', 'splicemap', '',
288 _('splice synthesized history into place'), _('FILE')),
291 _('splice synthesized history into place'), _('FILE')),
289 ('', 'branchmap', '',
292 ('', 'branchmap', '',
290 _('change branch names while converting'), _('FILE')),
293 _('change branch names while converting'), _('FILE')),
291 ('', 'branchsort', None, _('try to sort changesets by branches')),
294 ('', 'branchsort', None, _('try to sort changesets by branches')),
292 ('', 'datesort', None, _('try to sort changesets by date')),
295 ('', 'datesort', None, _('try to sort changesets by date')),
293 ('', 'sourcesort', None, _('preserve source changesets order'))],
296 ('', 'sourcesort', None, _('preserve source changesets order'))],
294 _('hg convert [OPTION]... SOURCE [DEST [REVMAP]]')),
297 _('hg convert [OPTION]... SOURCE [DEST [REVMAP]]')),
295 "debugsvnlog":
298 "debugsvnlog":
296 (debugsvnlog,
299 (debugsvnlog,
297 [],
300 [],
298 'hg debugsvnlog'),
301 'hg debugsvnlog'),
299 "debugcvsps":
302 "debugcvsps":
300 (debugcvsps,
303 (debugcvsps,
301 [
304 [
302 # Main options shared with cvsps-2.1
305 # Main options shared with cvsps-2.1
303 ('b', 'branches', [], _('only return changes on specified branches')),
306 ('b', 'branches', [], _('only return changes on specified branches')),
304 ('p', 'prefix', '', _('prefix to remove from file names')),
307 ('p', 'prefix', '', _('prefix to remove from file names')),
305 ('r', 'revisions', [],
308 ('r', 'revisions', [],
306 _('only return changes after or between specified tags')),
309 _('only return changes after or between specified tags')),
307 ('u', 'update-cache', None, _("update cvs log cache")),
310 ('u', 'update-cache', None, _("update cvs log cache")),
308 ('x', 'new-cache', None, _("create new cvs log cache")),
311 ('x', 'new-cache', None, _("create new cvs log cache")),
309 ('z', 'fuzz', 60, _('set commit time fuzz in seconds')),
312 ('z', 'fuzz', 60, _('set commit time fuzz in seconds')),
310 ('', 'root', '', _('specify cvsroot')),
313 ('', 'root', '', _('specify cvsroot')),
311 # Options specific to builtin cvsps
314 # Options specific to builtin cvsps
312 ('', 'parents', '', _('show parent changesets')),
315 ('', 'parents', '', _('show parent changesets')),
313 ('', 'ancestors', '', _('show current changeset in ancestor branches')),
316 ('', 'ancestors', '', _('show current changeset in ancestor branches')),
314 # Options that are ignored for compatibility with cvsps-2.1
317 # Options that are ignored for compatibility with cvsps-2.1
315 ('A', 'cvs-direct', None, _('ignored for compatibility')),
318 ('A', 'cvs-direct', None, _('ignored for compatibility')),
316 ],
319 ],
317 _('hg debugcvsps [OPTION]... [PATH]...')),
320 _('hg debugcvsps [OPTION]... [PATH]...')),
318 }
321 }
@@ -1,430 +1,434 b''
1 # convcmd - convert extension commands definition
1 # convcmd - convert extension commands definition
2 #
2 #
3 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
3 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
4 #
4 #
5 # This software may be used and distributed according to the terms of the
5 # This software may be used and distributed according to the terms of the
6 # GNU General Public License version 2 or any later version.
6 # GNU General Public License version 2 or any later version.
7
7
8 from common import NoRepo, MissingTool, SKIPREV, mapfile
8 from common import NoRepo, MissingTool, SKIPREV, mapfile
9 from cvs import convert_cvs
9 from cvs import convert_cvs
10 from darcs import darcs_source
10 from darcs import darcs_source
11 from git import convert_git
11 from git import convert_git
12 from hg import mercurial_source, mercurial_sink
12 from hg import mercurial_source, mercurial_sink
13 from subversion import svn_source, svn_sink
13 from subversion import svn_source, svn_sink
14 from monotone import monotone_source
14 from monotone import monotone_source
15 from gnuarch import gnuarch_source
15 from gnuarch import gnuarch_source
16 from bzr import bzr_source
16 from bzr import bzr_source
17 from p4 import p4_source
17 from p4 import p4_source
18 import filemap
18 import filemap
19
19
20 import os, shutil
20 import os, shutil
21 from mercurial import hg, util, encoding
21 from mercurial import hg, util, encoding
22 from mercurial.i18n import _
22 from mercurial.i18n import _
23
23
24 orig_encoding = 'ascii'
24 orig_encoding = 'ascii'
25
25
26 def recode(s):
26 def recode(s):
27 if isinstance(s, unicode):
27 if isinstance(s, unicode):
28 return s.encode(orig_encoding, 'replace')
28 return s.encode(orig_encoding, 'replace')
29 else:
29 else:
30 return s.decode('utf-8').encode(orig_encoding, 'replace')
30 return s.decode('utf-8').encode(orig_encoding, 'replace')
31
31
32 source_converters = [
32 source_converters = [
33 ('cvs', convert_cvs, 'branchsort'),
33 ('cvs', convert_cvs, 'branchsort'),
34 ('git', convert_git, 'branchsort'),
34 ('git', convert_git, 'branchsort'),
35 ('svn', svn_source, 'branchsort'),
35 ('svn', svn_source, 'branchsort'),
36 ('hg', mercurial_source, 'sourcesort'),
36 ('hg', mercurial_source, 'sourcesort'),
37 ('darcs', darcs_source, 'branchsort'),
37 ('darcs', darcs_source, 'branchsort'),
38 ('mtn', monotone_source, 'branchsort'),
38 ('mtn', monotone_source, 'branchsort'),
39 ('gnuarch', gnuarch_source, 'branchsort'),
39 ('gnuarch', gnuarch_source, 'branchsort'),
40 ('bzr', bzr_source, 'branchsort'),
40 ('bzr', bzr_source, 'branchsort'),
41 ('p4', p4_source, 'branchsort'),
41 ('p4', p4_source, 'branchsort'),
42 ]
42 ]
43
43
44 sink_converters = [
44 sink_converters = [
45 ('hg', mercurial_sink),
45 ('hg', mercurial_sink),
46 ('svn', svn_sink),
46 ('svn', svn_sink),
47 ]
47 ]
48
48
49 def convertsource(ui, path, type, rev):
49 def convertsource(ui, path, type, rev):
50 exceptions = []
50 exceptions = []
51 if type and type not in [s[0] for s in source_converters]:
51 if type and type not in [s[0] for s in source_converters]:
52 raise util.Abort(_('%s: invalid source repository type') % type)
52 raise util.Abort(_('%s: invalid source repository type') % type)
53 for name, source, sortmode in source_converters:
53 for name, source, sortmode in source_converters:
54 try:
54 try:
55 if not type or name == type:
55 if not type or name == type:
56 return source(ui, path, rev), sortmode
56 return source(ui, path, rev), sortmode
57 except (NoRepo, MissingTool), inst:
57 except (NoRepo, MissingTool), inst:
58 exceptions.append(inst)
58 exceptions.append(inst)
59 if not ui.quiet:
59 if not ui.quiet:
60 for inst in exceptions:
60 for inst in exceptions:
61 ui.write("%s\n" % inst)
61 ui.write("%s\n" % inst)
62 raise util.Abort(_('%s: missing or unsupported repository') % path)
62 raise util.Abort(_('%s: missing or unsupported repository') % path)
63
63
64 def convertsink(ui, path, type):
64 def convertsink(ui, path, type):
65 if type and type not in [s[0] for s in sink_converters]:
65 if type and type not in [s[0] for s in sink_converters]:
66 raise util.Abort(_('%s: invalid destination repository type') % type)
66 raise util.Abort(_('%s: invalid destination repository type') % type)
67 for name, sink in sink_converters:
67 for name, sink in sink_converters:
68 try:
68 try:
69 if not type or name == type:
69 if not type or name == type:
70 return sink(ui, path)
70 return sink(ui, path)
71 except NoRepo, inst:
71 except NoRepo, inst:
72 ui.note(_("convert: %s\n") % inst)
72 ui.note(_("convert: %s\n") % inst)
73 raise util.Abort(_('%s: unknown repository type') % path)
73 raise util.Abort(_('%s: unknown repository type') % path)
74
74
75 class progresssource(object):
75 class progresssource(object):
76 def __init__(self, ui, source, filecount):
76 def __init__(self, ui, source, filecount):
77 self.ui = ui
77 self.ui = ui
78 self.source = source
78 self.source = source
79 self.filecount = filecount
79 self.filecount = filecount
80 self.retrieved = 0
80 self.retrieved = 0
81
81
82 def getfile(self, file, rev):
82 def getfile(self, file, rev):
83 self.retrieved += 1
83 self.retrieved += 1
84 self.ui.progress(_('getting files'), self.retrieved,
84 self.ui.progress(_('getting files'), self.retrieved,
85 item=file, total=self.filecount)
85 item=file, total=self.filecount)
86 return self.source.getfile(file, rev)
86 return self.source.getfile(file, rev)
87
87
88 def lookuprev(self, rev):
88 def lookuprev(self, rev):
89 return self.source.lookuprev(rev)
89 return self.source.lookuprev(rev)
90
90
91 def close(self):
91 def close(self):
92 self.ui.progress(_('getting files'), None)
92 self.ui.progress(_('getting files'), None)
93
93
94 class converter(object):
94 class converter(object):
95 def __init__(self, ui, source, dest, revmapfile, opts):
95 def __init__(self, ui, source, dest, revmapfile, opts):
96
96
97 self.source = source
97 self.source = source
98 self.dest = dest
98 self.dest = dest
99 self.ui = ui
99 self.ui = ui
100 self.opts = opts
100 self.opts = opts
101 self.commitcache = {}
101 self.commitcache = {}
102 self.authors = {}
102 self.authors = {}
103 self.authorfile = None
103 self.authorfile = None
104
104
105 # Record converted revisions persistently: maps source revision
105 # Record converted revisions persistently: maps source revision
106 # ID to target revision ID (both strings). (This is how
106 # ID to target revision ID (both strings). (This is how
107 # incremental conversions work.)
107 # incremental conversions work.)
108 self.map = mapfile(ui, revmapfile)
108 self.map = mapfile(ui, revmapfile)
109
109
110 # Read first the dst author map if any
110 # Read first the dst author map if any
111 authorfile = self.dest.authorfile()
111 authorfile = self.dest.authorfile()
112 if authorfile and os.path.exists(authorfile):
112 if authorfile and os.path.exists(authorfile):
113 self.readauthormap(authorfile)
113 self.readauthormap(authorfile)
114 # Extend/Override with new author map if necessary
114 # Extend/Override with new author map if necessary
115 if opts.get('authors'):
115 if opts.get('authormap'):
116 self.readauthormap(opts.get('authors'))
116 self.readauthormap(opts.get('authormap'))
117 self.authorfile = self.dest.authorfile()
117 self.authorfile = self.dest.authorfile()
118
118
119 self.splicemap = mapfile(ui, opts.get('splicemap'))
119 self.splicemap = mapfile(ui, opts.get('splicemap'))
120 self.branchmap = mapfile(ui, opts.get('branchmap'))
120 self.branchmap = mapfile(ui, opts.get('branchmap'))
121
121
122 def walktree(self, heads):
122 def walktree(self, heads):
123 '''Return a mapping that identifies the uncommitted parents of every
123 '''Return a mapping that identifies the uncommitted parents of every
124 uncommitted changeset.'''
124 uncommitted changeset.'''
125 visit = heads
125 visit = heads
126 known = set()
126 known = set()
127 parents = {}
127 parents = {}
128 while visit:
128 while visit:
129 n = visit.pop(0)
129 n = visit.pop(0)
130 if n in known or n in self.map:
130 if n in known or n in self.map:
131 continue
131 continue
132 known.add(n)
132 known.add(n)
133 self.ui.progress(_('scanning'), len(known), unit=_('revisions'))
133 self.ui.progress(_('scanning'), len(known), unit=_('revisions'))
134 commit = self.cachecommit(n)
134 commit = self.cachecommit(n)
135 parents[n] = []
135 parents[n] = []
136 for p in commit.parents:
136 for p in commit.parents:
137 parents[n].append(p)
137 parents[n].append(p)
138 visit.append(p)
138 visit.append(p)
139 self.ui.progress(_('scanning'), None)
139 self.ui.progress(_('scanning'), None)
140
140
141 return parents
141 return parents
142
142
143 def toposort(self, parents, sortmode):
143 def toposort(self, parents, sortmode):
144 '''Return an ordering such that every uncommitted changeset is
144 '''Return an ordering such that every uncommitted changeset is
145 preceeded by all its uncommitted ancestors.'''
145 preceeded by all its uncommitted ancestors.'''
146
146
147 def mapchildren(parents):
147 def mapchildren(parents):
148 """Return a (children, roots) tuple where 'children' maps parent
148 """Return a (children, roots) tuple where 'children' maps parent
149 revision identifiers to children ones, and 'roots' is the list of
149 revision identifiers to children ones, and 'roots' is the list of
150 revisions without parents. 'parents' must be a mapping of revision
150 revisions without parents. 'parents' must be a mapping of revision
151 identifier to its parents ones.
151 identifier to its parents ones.
152 """
152 """
153 visit = parents.keys()
153 visit = parents.keys()
154 seen = set()
154 seen = set()
155 children = {}
155 children = {}
156 roots = []
156 roots = []
157
157
158 while visit:
158 while visit:
159 n = visit.pop(0)
159 n = visit.pop(0)
160 if n in seen:
160 if n in seen:
161 continue
161 continue
162 seen.add(n)
162 seen.add(n)
163 # Ensure that nodes without parents are present in the
163 # Ensure that nodes without parents are present in the
164 # 'children' mapping.
164 # 'children' mapping.
165 children.setdefault(n, [])
165 children.setdefault(n, [])
166 hasparent = False
166 hasparent = False
167 for p in parents[n]:
167 for p in parents[n]:
168 if not p in self.map:
168 if not p in self.map:
169 visit.append(p)
169 visit.append(p)
170 hasparent = True
170 hasparent = True
171 children.setdefault(p, []).append(n)
171 children.setdefault(p, []).append(n)
172 if not hasparent:
172 if not hasparent:
173 roots.append(n)
173 roots.append(n)
174
174
175 return children, roots
175 return children, roots
176
176
177 # Sort functions are supposed to take a list of revisions which
177 # Sort functions are supposed to take a list of revisions which
178 # can be converted immediately and pick one
178 # can be converted immediately and pick one
179
179
180 def makebranchsorter():
180 def makebranchsorter():
181 """If the previously converted revision has a child in the
181 """If the previously converted revision has a child in the
182 eligible revisions list, pick it. Return the list head
182 eligible revisions list, pick it. Return the list head
183 otherwise. Branch sort attempts to minimize branch
183 otherwise. Branch sort attempts to minimize branch
184 switching, which is harmful for Mercurial backend
184 switching, which is harmful for Mercurial backend
185 compression.
185 compression.
186 """
186 """
187 prev = [None]
187 prev = [None]
188 def picknext(nodes):
188 def picknext(nodes):
189 next = nodes[0]
189 next = nodes[0]
190 for n in nodes:
190 for n in nodes:
191 if prev[0] in parents[n]:
191 if prev[0] in parents[n]:
192 next = n
192 next = n
193 break
193 break
194 prev[0] = next
194 prev[0] = next
195 return next
195 return next
196 return picknext
196 return picknext
197
197
198 def makesourcesorter():
198 def makesourcesorter():
199 """Source specific sort."""
199 """Source specific sort."""
200 keyfn = lambda n: self.commitcache[n].sortkey
200 keyfn = lambda n: self.commitcache[n].sortkey
201 def picknext(nodes):
201 def picknext(nodes):
202 return sorted(nodes, key=keyfn)[0]
202 return sorted(nodes, key=keyfn)[0]
203 return picknext
203 return picknext
204
204
205 def makedatesorter():
205 def makedatesorter():
206 """Sort revisions by date."""
206 """Sort revisions by date."""
207 dates = {}
207 dates = {}
208 def getdate(n):
208 def getdate(n):
209 if n not in dates:
209 if n not in dates:
210 dates[n] = util.parsedate(self.commitcache[n].date)
210 dates[n] = util.parsedate(self.commitcache[n].date)
211 return dates[n]
211 return dates[n]
212
212
213 def picknext(nodes):
213 def picknext(nodes):
214 return min([(getdate(n), n) for n in nodes])[1]
214 return min([(getdate(n), n) for n in nodes])[1]
215
215
216 return picknext
216 return picknext
217
217
218 if sortmode == 'branchsort':
218 if sortmode == 'branchsort':
219 picknext = makebranchsorter()
219 picknext = makebranchsorter()
220 elif sortmode == 'datesort':
220 elif sortmode == 'datesort':
221 picknext = makedatesorter()
221 picknext = makedatesorter()
222 elif sortmode == 'sourcesort':
222 elif sortmode == 'sourcesort':
223 picknext = makesourcesorter()
223 picknext = makesourcesorter()
224 else:
224 else:
225 raise util.Abort(_('unknown sort mode: %s') % sortmode)
225 raise util.Abort(_('unknown sort mode: %s') % sortmode)
226
226
227 children, actives = mapchildren(parents)
227 children, actives = mapchildren(parents)
228
228
229 s = []
229 s = []
230 pendings = {}
230 pendings = {}
231 while actives:
231 while actives:
232 n = picknext(actives)
232 n = picknext(actives)
233 actives.remove(n)
233 actives.remove(n)
234 s.append(n)
234 s.append(n)
235
235
236 # Update dependents list
236 # Update dependents list
237 for c in children.get(n, []):
237 for c in children.get(n, []):
238 if c not in pendings:
238 if c not in pendings:
239 pendings[c] = [p for p in parents[c] if p not in self.map]
239 pendings[c] = [p for p in parents[c] if p not in self.map]
240 try:
240 try:
241 pendings[c].remove(n)
241 pendings[c].remove(n)
242 except ValueError:
242 except ValueError:
243 raise util.Abort(_('cycle detected between %s and %s')
243 raise util.Abort(_('cycle detected between %s and %s')
244 % (recode(c), recode(n)))
244 % (recode(c), recode(n)))
245 if not pendings[c]:
245 if not pendings[c]:
246 # Parents are converted, node is eligible
246 # Parents are converted, node is eligible
247 actives.insert(0, c)
247 actives.insert(0, c)
248 pendings[c] = None
248 pendings[c] = None
249
249
250 if len(s) != len(parents):
250 if len(s) != len(parents):
251 raise util.Abort(_("not all revisions were sorted"))
251 raise util.Abort(_("not all revisions were sorted"))
252
252
253 return s
253 return s
254
254
255 def writeauthormap(self):
255 def writeauthormap(self):
256 authorfile = self.authorfile
256 authorfile = self.authorfile
257 if authorfile:
257 if authorfile:
258 self.ui.status(_('Writing author map file %s\n') % authorfile)
258 self.ui.status(_('Writing author map file %s\n') % authorfile)
259 ofile = open(authorfile, 'w+')
259 ofile = open(authorfile, 'w+')
260 for author in self.authors:
260 for author in self.authors:
261 ofile.write("%s=%s\n" % (author, self.authors[author]))
261 ofile.write("%s=%s\n" % (author, self.authors[author]))
262 ofile.close()
262 ofile.close()
263
263
264 def readauthormap(self, authorfile):
264 def readauthormap(self, authorfile):
265 afile = open(authorfile, 'r')
265 afile = open(authorfile, 'r')
266 for line in afile:
266 for line in afile:
267
267
268 line = line.strip()
268 line = line.strip()
269 if not line or line.startswith('#'):
269 if not line or line.startswith('#'):
270 continue
270 continue
271
271
272 try:
272 try:
273 srcauthor, dstauthor = line.split('=', 1)
273 srcauthor, dstauthor = line.split('=', 1)
274 except ValueError:
274 except ValueError:
275 msg = _('Ignoring bad line in author map file %s: %s\n')
275 msg = _('Ignoring bad line in author map file %s: %s\n')
276 self.ui.warn(msg % (authorfile, line.rstrip()))
276 self.ui.warn(msg % (authorfile, line.rstrip()))
277 continue
277 continue
278
278
279 srcauthor = srcauthor.strip()
279 srcauthor = srcauthor.strip()
280 dstauthor = dstauthor.strip()
280 dstauthor = dstauthor.strip()
281 if self.authors.get(srcauthor) in (None, dstauthor):
281 if self.authors.get(srcauthor) in (None, dstauthor):
282 msg = _('mapping author %s to %s\n')
282 msg = _('mapping author %s to %s\n')
283 self.ui.debug(msg % (srcauthor, dstauthor))
283 self.ui.debug(msg % (srcauthor, dstauthor))
284 self.authors[srcauthor] = dstauthor
284 self.authors[srcauthor] = dstauthor
285 continue
285 continue
286
286
287 m = _('overriding mapping for author %s, was %s, will be %s\n')
287 m = _('overriding mapping for author %s, was %s, will be %s\n')
288 self.ui.status(m % (srcauthor, self.authors[srcauthor], dstauthor))
288 self.ui.status(m % (srcauthor, self.authors[srcauthor], dstauthor))
289
289
290 afile.close()
290 afile.close()
291
291
292 def cachecommit(self, rev):
292 def cachecommit(self, rev):
293 commit = self.source.getcommit(rev)
293 commit = self.source.getcommit(rev)
294 commit.author = self.authors.get(commit.author, commit.author)
294 commit.author = self.authors.get(commit.author, commit.author)
295 commit.branch = self.branchmap.get(commit.branch, commit.branch)
295 commit.branch = self.branchmap.get(commit.branch, commit.branch)
296 self.commitcache[rev] = commit
296 self.commitcache[rev] = commit
297 return commit
297 return commit
298
298
299 def copy(self, rev):
299 def copy(self, rev):
300 commit = self.commitcache[rev]
300 commit = self.commitcache[rev]
301
301
302 changes = self.source.getchanges(rev)
302 changes = self.source.getchanges(rev)
303 if isinstance(changes, basestring):
303 if isinstance(changes, basestring):
304 if changes == SKIPREV:
304 if changes == SKIPREV:
305 dest = SKIPREV
305 dest = SKIPREV
306 else:
306 else:
307 dest = self.map[changes]
307 dest = self.map[changes]
308 self.map[rev] = dest
308 self.map[rev] = dest
309 return
309 return
310 files, copies = changes
310 files, copies = changes
311 pbranches = []
311 pbranches = []
312 if commit.parents:
312 if commit.parents:
313 for prev in commit.parents:
313 for prev in commit.parents:
314 if prev not in self.commitcache:
314 if prev not in self.commitcache:
315 self.cachecommit(prev)
315 self.cachecommit(prev)
316 pbranches.append((self.map[prev],
316 pbranches.append((self.map[prev],
317 self.commitcache[prev].branch))
317 self.commitcache[prev].branch))
318 self.dest.setbranch(commit.branch, pbranches)
318 self.dest.setbranch(commit.branch, pbranches)
319 try:
319 try:
320 parents = self.splicemap[rev].replace(',', ' ').split()
320 parents = self.splicemap[rev].replace(',', ' ').split()
321 self.ui.status(_('spliced in %s as parents of %s\n') %
321 self.ui.status(_('spliced in %s as parents of %s\n') %
322 (parents, rev))
322 (parents, rev))
323 parents = [self.map.get(p, p) for p in parents]
323 parents = [self.map.get(p, p) for p in parents]
324 except KeyError:
324 except KeyError:
325 parents = [b[0] for b in pbranches]
325 parents = [b[0] for b in pbranches]
326 source = progresssource(self.ui, self.source, len(files))
326 source = progresssource(self.ui, self.source, len(files))
327 newnode = self.dest.putcommit(files, copies, parents, commit,
327 newnode = self.dest.putcommit(files, copies, parents, commit,
328 source, self.map)
328 source, self.map)
329 source.close()
329 source.close()
330 self.source.converted(rev, newnode)
330 self.source.converted(rev, newnode)
331 self.map[rev] = newnode
331 self.map[rev] = newnode
332
332
333 def convert(self, sortmode):
333 def convert(self, sortmode):
334 try:
334 try:
335 self.source.before()
335 self.source.before()
336 self.dest.before()
336 self.dest.before()
337 self.source.setrevmap(self.map)
337 self.source.setrevmap(self.map)
338 self.ui.status(_("scanning source...\n"))
338 self.ui.status(_("scanning source...\n"))
339 heads = self.source.getheads()
339 heads = self.source.getheads()
340 parents = self.walktree(heads)
340 parents = self.walktree(heads)
341 self.ui.status(_("sorting...\n"))
341 self.ui.status(_("sorting...\n"))
342 t = self.toposort(parents, sortmode)
342 t = self.toposort(parents, sortmode)
343 num = len(t)
343 num = len(t)
344 c = None
344 c = None
345
345
346 self.ui.status(_("converting...\n"))
346 self.ui.status(_("converting...\n"))
347 for i, c in enumerate(t):
347 for i, c in enumerate(t):
348 num -= 1
348 num -= 1
349 desc = self.commitcache[c].desc
349 desc = self.commitcache[c].desc
350 if "\n" in desc:
350 if "\n" in desc:
351 desc = desc.splitlines()[0]
351 desc = desc.splitlines()[0]
352 # convert log message to local encoding without using
352 # convert log message to local encoding without using
353 # tolocal() because encoding.encoding conver() use it as
353 # tolocal() because encoding.encoding conver() use it as
354 # 'utf-8'
354 # 'utf-8'
355 self.ui.status("%d %s\n" % (num, recode(desc)))
355 self.ui.status("%d %s\n" % (num, recode(desc)))
356 self.ui.note(_("source: %s\n") % recode(c))
356 self.ui.note(_("source: %s\n") % recode(c))
357 self.ui.progress(_('converting'), i, unit=_('revisions'),
357 self.ui.progress(_('converting'), i, unit=_('revisions'),
358 total=len(t))
358 total=len(t))
359 self.copy(c)
359 self.copy(c)
360 self.ui.progress(_('converting'), None)
360 self.ui.progress(_('converting'), None)
361
361
362 tags = self.source.gettags()
362 tags = self.source.gettags()
363 ctags = {}
363 ctags = {}
364 for k in tags:
364 for k in tags:
365 v = tags[k]
365 v = tags[k]
366 if self.map.get(v, SKIPREV) != SKIPREV:
366 if self.map.get(v, SKIPREV) != SKIPREV:
367 ctags[k] = self.map[v]
367 ctags[k] = self.map[v]
368
368
369 if c and ctags:
369 if c and ctags:
370 nrev, tagsparent = self.dest.puttags(ctags)
370 nrev, tagsparent = self.dest.puttags(ctags)
371 if nrev and tagsparent:
371 if nrev and tagsparent:
372 # write another hash correspondence to override the previous
372 # write another hash correspondence to override the previous
373 # one so we don't end up with extra tag heads
373 # one so we don't end up with extra tag heads
374 tagsparents = [e for e in self.map.iteritems()
374 tagsparents = [e for e in self.map.iteritems()
375 if e[1] == tagsparent]
375 if e[1] == tagsparent]
376 if tagsparents:
376 if tagsparents:
377 self.map[tagsparents[0][0]] = nrev
377 self.map[tagsparents[0][0]] = nrev
378
378
379 self.writeauthormap()
379 self.writeauthormap()
380 finally:
380 finally:
381 self.cleanup()
381 self.cleanup()
382
382
383 def cleanup(self):
383 def cleanup(self):
384 try:
384 try:
385 self.dest.after()
385 self.dest.after()
386 finally:
386 finally:
387 self.source.after()
387 self.source.after()
388 self.map.close()
388 self.map.close()
389
389
390 def convert(ui, src, dest=None, revmapfile=None, **opts):
390 def convert(ui, src, dest=None, revmapfile=None, **opts):
391 global orig_encoding
391 global orig_encoding
392 orig_encoding = encoding.encoding
392 orig_encoding = encoding.encoding
393 encoding.encoding = 'UTF-8'
393 encoding.encoding = 'UTF-8'
394
394
395 # support --authors as an alias for --authormap
396 if not opts.get('authormap'):
397 opts['authormap'] = opts.get('authors')
398
395 if not dest:
399 if not dest:
396 dest = hg.defaultdest(src) + "-hg"
400 dest = hg.defaultdest(src) + "-hg"
397 ui.status(_("assuming destination %s\n") % dest)
401 ui.status(_("assuming destination %s\n") % dest)
398
402
399 destc = convertsink(ui, dest, opts.get('dest_type'))
403 destc = convertsink(ui, dest, opts.get('dest_type'))
400
404
401 try:
405 try:
402 srcc, defaultsort = convertsource(ui, src, opts.get('source_type'),
406 srcc, defaultsort = convertsource(ui, src, opts.get('source_type'),
403 opts.get('rev'))
407 opts.get('rev'))
404 except Exception:
408 except Exception:
405 for path in destc.created:
409 for path in destc.created:
406 shutil.rmtree(path, True)
410 shutil.rmtree(path, True)
407 raise
411 raise
408
412
409 sortmodes = ('branchsort', 'datesort', 'sourcesort')
413 sortmodes = ('branchsort', 'datesort', 'sourcesort')
410 sortmode = [m for m in sortmodes if opts.get(m)]
414 sortmode = [m for m in sortmodes if opts.get(m)]
411 if len(sortmode) > 1:
415 if len(sortmode) > 1:
412 raise util.Abort(_('more than one sort mode specified'))
416 raise util.Abort(_('more than one sort mode specified'))
413 sortmode = sortmode and sortmode[0] or defaultsort
417 sortmode = sortmode and sortmode[0] or defaultsort
414 if sortmode == 'sourcesort' and not srcc.hasnativeorder():
418 if sortmode == 'sourcesort' and not srcc.hasnativeorder():
415 raise util.Abort(_('--sourcesort is not supported by this data source'))
419 raise util.Abort(_('--sourcesort is not supported by this data source'))
416
420
417 fmap = opts.get('filemap')
421 fmap = opts.get('filemap')
418 if fmap:
422 if fmap:
419 srcc = filemap.filemap_source(ui, srcc, fmap)
423 srcc = filemap.filemap_source(ui, srcc, fmap)
420 destc.setfilemapmode(True)
424 destc.setfilemapmode(True)
421
425
422 if not revmapfile:
426 if not revmapfile:
423 try:
427 try:
424 revmapfile = destc.revmapfile()
428 revmapfile = destc.revmapfile()
425 except:
429 except:
426 revmapfile = os.path.join(destc, "map")
430 revmapfile = os.path.join(destc, "map")
427
431
428 c = converter(ui, srcc, destc, revmapfile, opts)
432 c = converter(ui, srcc, destc, revmapfile, opts)
429 c.convert(sortmode)
433 c.convert(sortmode)
430
434
@@ -1,318 +1,318 b''
1 hg convert [OPTION]... SOURCE [DEST [REVMAP]]
1 hg convert [OPTION]... SOURCE [DEST [REVMAP]]
2
2
3 convert a foreign SCM repository to a Mercurial one.
3 convert a foreign SCM repository to a Mercurial one.
4
4
5 Accepted source formats [identifiers]:
5 Accepted source formats [identifiers]:
6
6
7 - Mercurial [hg]
7 - Mercurial [hg]
8 - CVS [cvs]
8 - CVS [cvs]
9 - Darcs [darcs]
9 - Darcs [darcs]
10 - git [git]
10 - git [git]
11 - Subversion [svn]
11 - Subversion [svn]
12 - Monotone [mtn]
12 - Monotone [mtn]
13 - GNU Arch [gnuarch]
13 - GNU Arch [gnuarch]
14 - Bazaar [bzr]
14 - Bazaar [bzr]
15 - Perforce [p4]
15 - Perforce [p4]
16
16
17 Accepted destination formats [identifiers]:
17 Accepted destination formats [identifiers]:
18
18
19 - Mercurial [hg]
19 - Mercurial [hg]
20 - Subversion [svn] (history on branches is not preserved)
20 - Subversion [svn] (history on branches is not preserved)
21
21
22 If no revision is given, all revisions will be converted. Otherwise,
22 If no revision is given, all revisions will be converted. Otherwise,
23 convert will only import up to the named revision (given in a format
23 convert will only import up to the named revision (given in a format
24 understood by the source).
24 understood by the source).
25
25
26 If no destination directory name is specified, it defaults to the basename
26 If no destination directory name is specified, it defaults to the basename
27 of the source with "-hg" appended. If the destination repository doesn't
27 of the source with "-hg" appended. If the destination repository doesn't
28 exist, it will be created.
28 exist, it will be created.
29
29
30 By default, all sources except Mercurial will use --branchsort. Mercurial
30 By default, all sources except Mercurial will use --branchsort. Mercurial
31 uses --sourcesort to preserve original revision numbers order. Sort modes
31 uses --sourcesort to preserve original revision numbers order. Sort modes
32 have the following effects:
32 have the following effects:
33
33
34 --branchsort convert from parent to child revision when possible, which
34 --branchsort convert from parent to child revision when possible, which
35 means branches are usually converted one after the other. It
35 means branches are usually converted one after the other. It
36 generates more compact repositories.
36 generates more compact repositories.
37 --datesort sort revisions by date. Converted repositories have good-
37 --datesort sort revisions by date. Converted repositories have good-
38 looking changelogs but are often an order of magnitude
38 looking changelogs but are often an order of magnitude
39 larger than the same ones generated by --branchsort.
39 larger than the same ones generated by --branchsort.
40 --sourcesort try to preserve source revisions order, only supported by
40 --sourcesort try to preserve source revisions order, only supported by
41 Mercurial sources.
41 Mercurial sources.
42
42
43 If <REVMAP> isn't given, it will be put in a default location
43 If <REVMAP> isn't given, it will be put in a default location
44 (<dest>/.hg/shamap by default). The <REVMAP> is a simple text file that
44 (<dest>/.hg/shamap by default). The <REVMAP> is a simple text file that
45 maps each source commit ID to the destination ID for that revision, like
45 maps each source commit ID to the destination ID for that revision, like
46 so:
46 so:
47
47
48 <source ID> <destination ID>
48 <source ID> <destination ID>
49
49
50 If the file doesn't exist, it's automatically created. It's updated on
50 If the file doesn't exist, it's automatically created. It's updated on
51 each commit copied, so "hg convert" can be interrupted and can be run
51 each commit copied, so "hg convert" can be interrupted and can be run
52 repeatedly to copy new commits.
52 repeatedly to copy new commits.
53
53
54 The username mapping file is a simple text file that maps each source
54 The authormap is a simple text file that maps each source commit author to
55 commit author to a destination commit author. It is handy for source SCMs
55 a destination commit author. It is handy for source SCMs that use unix
56 that use unix logins to identify authors (eg: CVS). One line per author
56 logins to identify authors (eg: CVS). One line per author mapping and the
57 mapping and the line format is:
57 line format is:
58
58
59 source author = destination author
59 source author = destination author
60
60
61 Empty lines and lines starting with a "#" are ignored.
61 Empty lines and lines starting with a "#" are ignored.
62
62
63 The filemap is a file that allows filtering and remapping of files and
63 The filemap is a file that allows filtering and remapping of files and
64 directories. Each line can contain one of the following directives:
64 directories. Each line can contain one of the following directives:
65
65
66 include path/to/file-or-dir
66 include path/to/file-or-dir
67
67
68 exclude path/to/file-or-dir
68 exclude path/to/file-or-dir
69
69
70 rename path/to/source path/to/destination
70 rename path/to/source path/to/destination
71
71
72 Comment lines start with "#". A specified path matches if it equals the
72 Comment lines start with "#". A specified path matches if it equals the
73 full relative name of a file or one of its parent directories. The
73 full relative name of a file or one of its parent directories. The
74 "include" or "exclude" directive with the longest matching path applies,
74 "include" or "exclude" directive with the longest matching path applies,
75 so line order does not matter.
75 so line order does not matter.
76
76
77 The "include" directive causes a file, or all files under a directory, to
77 The "include" directive causes a file, or all files under a directory, to
78 be included in the destination repository, and the exclusion of all other
78 be included in the destination repository, and the exclusion of all other
79 files and directories not explicitly included. The "exclude" directive
79 files and directories not explicitly included. The "exclude" directive
80 causes files or directories to be omitted. The "rename" directive renames
80 causes files or directories to be omitted. The "rename" directive renames
81 a file or directory if it is converted. To rename from a subdirectory into
81 a file or directory if it is converted. To rename from a subdirectory into
82 the root of the repository, use "." as the path to rename to.
82 the root of the repository, use "." as the path to rename to.
83
83
84 The splicemap is a file that allows insertion of synthetic history,
84 The splicemap is a file that allows insertion of synthetic history,
85 letting you specify the parents of a revision. This is useful if you want
85 letting you specify the parents of a revision. This is useful if you want
86 to e.g. give a Subversion merge two parents, or graft two disconnected
86 to e.g. give a Subversion merge two parents, or graft two disconnected
87 series of history together. Each entry contains a key, followed by a
87 series of history together. Each entry contains a key, followed by a
88 space, followed by one or two comma-separated values:
88 space, followed by one or two comma-separated values:
89
89
90 key parent1, parent2
90 key parent1, parent2
91
91
92 The key is the revision ID in the source revision control system whose
92 The key is the revision ID in the source revision control system whose
93 parents should be modified (same format as a key in .hg/shamap). The
93 parents should be modified (same format as a key in .hg/shamap). The
94 values are the revision IDs (in either the source or destination revision
94 values are the revision IDs (in either the source or destination revision
95 control system) that should be used as the new parents for that node. For
95 control system) that should be used as the new parents for that node. For
96 example, if you have merged "release-1.0" into "trunk", then you should
96 example, if you have merged "release-1.0" into "trunk", then you should
97 specify the revision on "trunk" as the first parent and the one on the
97 specify the revision on "trunk" as the first parent and the one on the
98 "release-1.0" branch as the second.
98 "release-1.0" branch as the second.
99
99
100 The branchmap is a file that allows you to rename a branch when it is
100 The branchmap is a file that allows you to rename a branch when it is
101 being brought in from whatever external repository. When used in
101 being brought in from whatever external repository. When used in
102 conjunction with a splicemap, it allows for a powerful combination to help
102 conjunction with a splicemap, it allows for a powerful combination to help
103 fix even the most badly mismanaged repositories and turn them into nicely
103 fix even the most badly mismanaged repositories and turn them into nicely
104 structured Mercurial repositories. The branchmap contains lines of the
104 structured Mercurial repositories. The branchmap contains lines of the
105 form:
105 form:
106
106
107 original_branch_name new_branch_name
107 original_branch_name new_branch_name
108
108
109 where "original_branch_name" is the name of the branch in the source
109 where "original_branch_name" is the name of the branch in the source
110 repository, and "new_branch_name" is the name of the branch is the
110 repository, and "new_branch_name" is the name of the branch is the
111 destination repository. No whitespace is allowed in the branch names. This
111 destination repository. No whitespace is allowed in the branch names. This
112 can be used to (for instance) move code in one repository from "default"
112 can be used to (for instance) move code in one repository from "default"
113 to a named branch.
113 to a named branch.
114
114
115 Mercurial Source
115 Mercurial Source
116 ----------------
116 ----------------
117
117
118 --config convert.hg.ignoreerrors=False (boolean)
118 --config convert.hg.ignoreerrors=False (boolean)
119 ignore integrity errors when reading. Use it to fix Mercurial
119 ignore integrity errors when reading. Use it to fix Mercurial
120 repositories with missing revlogs, by converting from and to
120 repositories with missing revlogs, by converting from and to
121 Mercurial.
121 Mercurial.
122
122
123 --config convert.hg.saverev=False (boolean)
123 --config convert.hg.saverev=False (boolean)
124 store original revision ID in changeset (forces target IDs to change)
124 store original revision ID in changeset (forces target IDs to change)
125
125
126 --config convert.hg.startrev=0 (hg revision identifier)
126 --config convert.hg.startrev=0 (hg revision identifier)
127 convert start revision and its descendants
127 convert start revision and its descendants
128
128
129 CVS Source
129 CVS Source
130 ----------
130 ----------
131
131
132 CVS source will use a sandbox (i.e. a checked-out copy) from CVS to
132 CVS source will use a sandbox (i.e. a checked-out copy) from CVS to
133 indicate the starting point of what will be converted. Direct access to
133 indicate the starting point of what will be converted. Direct access to
134 the repository files is not needed, unless of course the repository is
134 the repository files is not needed, unless of course the repository is
135 :local:. The conversion uses the top level directory in the sandbox to
135 :local:. The conversion uses the top level directory in the sandbox to
136 find the CVS repository, and then uses CVS rlog commands to find files to
136 find the CVS repository, and then uses CVS rlog commands to find files to
137 convert. This means that unless a filemap is given, all files under the
137 convert. This means that unless a filemap is given, all files under the
138 starting directory will be converted, and that any directory
138 starting directory will be converted, and that any directory
139 reorganization in the CVS sandbox is ignored.
139 reorganization in the CVS sandbox is ignored.
140
140
141 The options shown are the defaults.
141 The options shown are the defaults.
142
142
143 --config convert.cvsps.cache=True (boolean)
143 --config convert.cvsps.cache=True (boolean)
144 Set to False to disable remote log caching, for testing and debugging
144 Set to False to disable remote log caching, for testing and debugging
145 purposes.
145 purposes.
146
146
147 --config convert.cvsps.fuzz=60 (integer)
147 --config convert.cvsps.fuzz=60 (integer)
148 Specify the maximum time (in seconds) that is allowed between commits
148 Specify the maximum time (in seconds) that is allowed between commits
149 with identical user and log message in a single changeset. When very
149 with identical user and log message in a single changeset. When very
150 large files were checked in as part of a changeset then the default
150 large files were checked in as part of a changeset then the default
151 may not be long enough.
151 may not be long enough.
152
152
153 --config convert.cvsps.mergeto='{{mergetobranch ([-\w]+)}}'
153 --config convert.cvsps.mergeto='{{mergetobranch ([-\w]+)}}'
154 Specify a regular expression to which commit log messages are matched.
154 Specify a regular expression to which commit log messages are matched.
155 If a match occurs, then the conversion process will insert a dummy
155 If a match occurs, then the conversion process will insert a dummy
156 revision merging the branch on which this log message occurs to the
156 revision merging the branch on which this log message occurs to the
157 branch indicated in the regex.
157 branch indicated in the regex.
158
158
159 --config convert.cvsps.mergefrom='{{mergefrombranch ([-\w]+)}}'
159 --config convert.cvsps.mergefrom='{{mergefrombranch ([-\w]+)}}'
160 Specify a regular expression to which commit log messages are matched.
160 Specify a regular expression to which commit log messages are matched.
161 If a match occurs, then the conversion process will add the most
161 If a match occurs, then the conversion process will add the most
162 recent revision on the branch indicated in the regex as the second
162 recent revision on the branch indicated in the regex as the second
163 parent of the changeset.
163 parent of the changeset.
164
164
165 --config hook.cvslog
165 --config hook.cvslog
166 Specify a Python function to be called at the end of gathering the CVS
166 Specify a Python function to be called at the end of gathering the CVS
167 log. The function is passed a list with the log entries, and can
167 log. The function is passed a list with the log entries, and can
168 modify the entries in-place, or add or delete them.
168 modify the entries in-place, or add or delete them.
169
169
170 --config hook.cvschangesets
170 --config hook.cvschangesets
171 Specify a Python function to be called after the changesets are
171 Specify a Python function to be called after the changesets are
172 calculated from the the CVS log. The function is passed a list with
172 calculated from the the CVS log. The function is passed a list with
173 the changeset entries, and can modify the changesets in-place, or add
173 the changeset entries, and can modify the changesets in-place, or add
174 or delete them.
174 or delete them.
175
175
176 An additional "debugcvsps" Mercurial command allows the builtin changeset
176 An additional "debugcvsps" Mercurial command allows the builtin changeset
177 merging code to be run without doing a conversion. Its parameters and
177 merging code to be run without doing a conversion. Its parameters and
178 output are similar to that of cvsps 2.1. Please see the command help for
178 output are similar to that of cvsps 2.1. Please see the command help for
179 more details.
179 more details.
180
180
181 Subversion Source
181 Subversion Source
182 -----------------
182 -----------------
183
183
184 Subversion source detects classical trunk/branches/tags layouts. By
184 Subversion source detects classical trunk/branches/tags layouts. By
185 default, the supplied "svn://repo/path/" source URL is converted as a
185 default, the supplied "svn://repo/path/" source URL is converted as a
186 single branch. If "svn://repo/path/trunk" exists it replaces the default
186 single branch. If "svn://repo/path/trunk" exists it replaces the default
187 branch. If "svn://repo/path/branches" exists, its subdirectories are
187 branch. If "svn://repo/path/branches" exists, its subdirectories are
188 listed as possible branches. If "svn://repo/path/tags" exists, it is
188 listed as possible branches. If "svn://repo/path/tags" exists, it is
189 looked for tags referencing converted branches. Default "trunk",
189 looked for tags referencing converted branches. Default "trunk",
190 "branches" and "tags" values can be overridden with following options. Set
190 "branches" and "tags" values can be overridden with following options. Set
191 them to paths relative to the source URL, or leave them blank to disable
191 them to paths relative to the source URL, or leave them blank to disable
192 auto detection.
192 auto detection.
193
193
194 --config convert.svn.branches=branches (directory name)
194 --config convert.svn.branches=branches (directory name)
195 specify the directory containing branches
195 specify the directory containing branches
196
196
197 --config convert.svn.tags=tags (directory name)
197 --config convert.svn.tags=tags (directory name)
198 specify the directory containing tags
198 specify the directory containing tags
199
199
200 --config convert.svn.trunk=trunk (directory name)
200 --config convert.svn.trunk=trunk (directory name)
201 specify the name of the trunk branch
201 specify the name of the trunk branch
202
202
203 Source history can be retrieved starting at a specific revision, instead
203 Source history can be retrieved starting at a specific revision, instead
204 of being integrally converted. Only single branch conversions are
204 of being integrally converted. Only single branch conversions are
205 supported.
205 supported.
206
206
207 --config convert.svn.startrev=0 (svn revision number)
207 --config convert.svn.startrev=0 (svn revision number)
208 specify start Subversion revision.
208 specify start Subversion revision.
209
209
210 Perforce Source
210 Perforce Source
211 ---------------
211 ---------------
212
212
213 The Perforce (P4) importer can be given a p4 depot path or a client
213 The Perforce (P4) importer can be given a p4 depot path or a client
214 specification as source. It will convert all files in the source to a flat
214 specification as source. It will convert all files in the source to a flat
215 Mercurial repository, ignoring labels, branches and integrations. Note
215 Mercurial repository, ignoring labels, branches and integrations. Note
216 that when a depot path is given you then usually should specify a target
216 that when a depot path is given you then usually should specify a target
217 directory, because otherwise the target may be named ...-hg.
217 directory, because otherwise the target may be named ...-hg.
218
218
219 It is possible to limit the amount of source history to be converted by
219 It is possible to limit the amount of source history to be converted by
220 specifying an initial Perforce revision.
220 specifying an initial Perforce revision.
221
221
222 --config convert.p4.startrev=0 (perforce changelist number)
222 --config convert.p4.startrev=0 (perforce changelist number)
223 specify initial Perforce revision.
223 specify initial Perforce revision.
224
224
225 Mercurial Destination
225 Mercurial Destination
226 ---------------------
226 ---------------------
227
227
228 --config convert.hg.clonebranches=False (boolean)
228 --config convert.hg.clonebranches=False (boolean)
229 dispatch source branches in separate clones.
229 dispatch source branches in separate clones.
230
230
231 --config convert.hg.tagsbranch=default (branch name)
231 --config convert.hg.tagsbranch=default (branch name)
232 tag revisions branch name
232 tag revisions branch name
233
233
234 --config convert.hg.usebranchnames=True (boolean)
234 --config convert.hg.usebranchnames=True (boolean)
235 preserve branch names
235 preserve branch names
236
236
237 options:
237 options:
238
238
239 -A --authors FILE username mapping filename
240 -s --source-type TYPE source repository type
239 -s --source-type TYPE source repository type
241 -d --dest-type TYPE destination repository type
240 -d --dest-type TYPE destination repository type
242 -r --rev REV import up to target revision REV
241 -r --rev REV import up to target revision REV
242 --authormap FILE remap usernames using this file
243 --filemap FILE remap file names using contents of file
243 --filemap FILE remap file names using contents of file
244 --splicemap FILE splice synthesized history into place
244 --splicemap FILE splice synthesized history into place
245 --branchmap FILE change branch names while converting
245 --branchmap FILE change branch names while converting
246 --branchsort try to sort changesets by branches
246 --branchsort try to sort changesets by branches
247 --datesort try to sort changesets by date
247 --datesort try to sort changesets by date
248 --sourcesort preserve source changesets order
248 --sourcesort preserve source changesets order
249
249
250 use "hg -v help convert" to show global options
250 use "hg -v help convert" to show global options
251 adding a
251 adding a
252 assuming destination a-hg
252 assuming destination a-hg
253 initializing destination a-hg repository
253 initializing destination a-hg repository
254 scanning source...
254 scanning source...
255 sorting...
255 sorting...
256 converting...
256 converting...
257 4 a
257 4 a
258 3 b
258 3 b
259 2 c
259 2 c
260 1 d
260 1 d
261 0 e
261 0 e
262 pulling from ../a
262 pulling from ../a
263 searching for changes
263 searching for changes
264 no changes found
264 no changes found
265 % should fail
265 % should fail
266 initializing destination bogusfile repository
266 initializing destination bogusfile repository
267 abort: cannot create new bundle repository
267 abort: cannot create new bundle repository
268 % should fail
268 % should fail
269 abort: Permission denied: bogusdir
269 abort: Permission denied: bogusdir
270 % should succeed
270 % should succeed
271 initializing destination bogusdir repository
271 initializing destination bogusdir repository
272 scanning source...
272 scanning source...
273 sorting...
273 sorting...
274 converting...
274 converting...
275 4 a
275 4 a
276 3 b
276 3 b
277 2 c
277 2 c
278 1 d
278 1 d
279 0 e
279 0 e
280 % test pre and post conversion actions
280 % test pre and post conversion actions
281 run hg source pre-conversion action
281 run hg source pre-conversion action
282 run hg sink pre-conversion action
282 run hg sink pre-conversion action
283 run hg sink post-conversion action
283 run hg sink post-conversion action
284 run hg source post-conversion action
284 run hg source post-conversion action
285 % converting empty dir should fail nicely
285 % converting empty dir should fail nicely
286 assuming destination emptydir-hg
286 assuming destination emptydir-hg
287 initializing destination emptydir-hg repository
287 initializing destination emptydir-hg repository
288 emptydir does not look like a CVS checkout
288 emptydir does not look like a CVS checkout
289 emptydir does not look like a Git repository
289 emptydir does not look like a Git repository
290 emptydir does not look like a Subversion repository
290 emptydir does not look like a Subversion repository
291 emptydir is not a local Mercurial repository
291 emptydir is not a local Mercurial repository
292 emptydir does not look like a darcs repository
292 emptydir does not look like a darcs repository
293 emptydir does not look like a monotone repository
293 emptydir does not look like a monotone repository
294 emptydir does not look like a GNU Arch repository
294 emptydir does not look like a GNU Arch repository
295 emptydir does not look like a Bazaar repository
295 emptydir does not look like a Bazaar repository
296 cannot find required "p4" tool
296 cannot find required "p4" tool
297 abort: emptydir: missing or unsupported repository
297 abort: emptydir: missing or unsupported repository
298 % convert with imaginary source type
298 % convert with imaginary source type
299 initializing destination a-foo repository
299 initializing destination a-foo repository
300 abort: foo: invalid source repository type
300 abort: foo: invalid source repository type
301 % convert with imaginary sink type
301 % convert with imaginary sink type
302 abort: foo: invalid destination repository type
302 abort: foo: invalid destination repository type
303
303
304 % testing: convert must not produce duplicate entries in fncache
304 % testing: convert must not produce duplicate entries in fncache
305 initializing destination b repository
305 initializing destination b repository
306 scanning source...
306 scanning source...
307 sorting...
307 sorting...
308 converting...
308 converting...
309 4 a
309 4 a
310 3 b
310 3 b
311 2 c
311 2 c
312 1 d
312 1 d
313 0 e
313 0 e
314 % contents of fncache file:
314 % contents of fncache file:
315 data/a.i
315 data/a.i
316 data/b.i
316 data/b.i
317 % test bogus URL
317 % test bogus URL
318 abort: bzr+ssh://foobar@selenic.com/baz: missing or unsupported repository
318 abort: bzr+ssh://foobar@selenic.com/baz: missing or unsupported repository
General Comments 0
You need to be logged in to leave comments. Login now