##// END OF EJS Templates
largefiles: eliminate an unnecessary import of configitems...
Matt Harbison -
r41102:72d4a176 default
parent child Browse files
Show More
@@ -1,195 +1,194 b''
1 # Copyright 2009-2010 Gregory P. Ward
1 # Copyright 2009-2010 Gregory P. Ward
2 # Copyright 2009-2010 Intelerad Medical Systems Incorporated
2 # Copyright 2009-2010 Intelerad Medical Systems Incorporated
3 # Copyright 2010-2011 Fog Creek Software
3 # Copyright 2010-2011 Fog Creek Software
4 # Copyright 2010-2011 Unity Technologies
4 # Copyright 2010-2011 Unity Technologies
5 #
5 #
6 # This software may be used and distributed according to the terms of the
6 # This software may be used and distributed according to the terms of the
7 # GNU General Public License version 2 or any later version.
7 # GNU General Public License version 2 or any later version.
8
8
9 '''track large binary files
9 '''track large binary files
10
10
11 Large binary files tend to be not very compressible, not very
11 Large binary files tend to be not very compressible, not very
12 diffable, and not at all mergeable. Such files are not handled
12 diffable, and not at all mergeable. Such files are not handled
13 efficiently by Mercurial's storage format (revlog), which is based on
13 efficiently by Mercurial's storage format (revlog), which is based on
14 compressed binary deltas; storing large binary files as regular
14 compressed binary deltas; storing large binary files as regular
15 Mercurial files wastes bandwidth and disk space and increases
15 Mercurial files wastes bandwidth and disk space and increases
16 Mercurial's memory usage. The largefiles extension addresses these
16 Mercurial's memory usage. The largefiles extension addresses these
17 problems by adding a centralized client-server layer on top of
17 problems by adding a centralized client-server layer on top of
18 Mercurial: largefiles live in a *central store* out on the network
18 Mercurial: largefiles live in a *central store* out on the network
19 somewhere, and you only fetch the revisions that you need when you
19 somewhere, and you only fetch the revisions that you need when you
20 need them.
20 need them.
21
21
22 largefiles works by maintaining a "standin file" in .hglf/ for each
22 largefiles works by maintaining a "standin file" in .hglf/ for each
23 largefile. The standins are small (41 bytes: an SHA-1 hash plus
23 largefile. The standins are small (41 bytes: an SHA-1 hash plus
24 newline) and are tracked by Mercurial. Largefile revisions are
24 newline) and are tracked by Mercurial. Largefile revisions are
25 identified by the SHA-1 hash of their contents, which is written to
25 identified by the SHA-1 hash of their contents, which is written to
26 the standin. largefiles uses that revision ID to get/put largefile
26 the standin. largefiles uses that revision ID to get/put largefile
27 revisions from/to the central store. This saves both disk space and
27 revisions from/to the central store. This saves both disk space and
28 bandwidth, since you don't need to retrieve all historical revisions
28 bandwidth, since you don't need to retrieve all historical revisions
29 of large files when you clone or pull.
29 of large files when you clone or pull.
30
30
31 To start a new repository or add new large binary files, just add
31 To start a new repository or add new large binary files, just add
32 --large to your :hg:`add` command. For example::
32 --large to your :hg:`add` command. For example::
33
33
34 $ dd if=/dev/urandom of=randomdata count=2000
34 $ dd if=/dev/urandom of=randomdata count=2000
35 $ hg add --large randomdata
35 $ hg add --large randomdata
36 $ hg commit -m "add randomdata as a largefile"
36 $ hg commit -m "add randomdata as a largefile"
37
37
38 When you push a changeset that adds/modifies largefiles to a remote
38 When you push a changeset that adds/modifies largefiles to a remote
39 repository, its largefile revisions will be uploaded along with it.
39 repository, its largefile revisions will be uploaded along with it.
40 Note that the remote Mercurial must also have the largefiles extension
40 Note that the remote Mercurial must also have the largefiles extension
41 enabled for this to work.
41 enabled for this to work.
42
42
43 When you pull a changeset that affects largefiles from a remote
43 When you pull a changeset that affects largefiles from a remote
44 repository, the largefiles for the changeset will by default not be
44 repository, the largefiles for the changeset will by default not be
45 pulled down. However, when you update to such a revision, any
45 pulled down. However, when you update to such a revision, any
46 largefiles needed by that revision are downloaded and cached (if
46 largefiles needed by that revision are downloaded and cached (if
47 they have never been downloaded before). One way to pull largefiles
47 they have never been downloaded before). One way to pull largefiles
48 when pulling is thus to use --update, which will update your working
48 when pulling is thus to use --update, which will update your working
49 copy to the latest pulled revision (and thereby downloading any new
49 copy to the latest pulled revision (and thereby downloading any new
50 largefiles).
50 largefiles).
51
51
52 If you want to pull largefiles you don't need for update yet, then
52 If you want to pull largefiles you don't need for update yet, then
53 you can use pull with the `--lfrev` option or the :hg:`lfpull` command.
53 you can use pull with the `--lfrev` option or the :hg:`lfpull` command.
54
54
55 If you know you are pulling from a non-default location and want to
55 If you know you are pulling from a non-default location and want to
56 download all the largefiles that correspond to the new changesets at
56 download all the largefiles that correspond to the new changesets at
57 the same time, then you can pull with `--lfrev "pulled()"`.
57 the same time, then you can pull with `--lfrev "pulled()"`.
58
58
59 If you just want to ensure that you will have the largefiles needed to
59 If you just want to ensure that you will have the largefiles needed to
60 merge or rebase with new heads that you are pulling, then you can pull
60 merge or rebase with new heads that you are pulling, then you can pull
61 with `--lfrev "head(pulled())"` flag to pre-emptively download any largefiles
61 with `--lfrev "head(pulled())"` flag to pre-emptively download any largefiles
62 that are new in the heads you are pulling.
62 that are new in the heads you are pulling.
63
63
64 Keep in mind that network access may now be required to update to
64 Keep in mind that network access may now be required to update to
65 changesets that you have not previously updated to. The nature of the
65 changesets that you have not previously updated to. The nature of the
66 largefiles extension means that updating is no longer guaranteed to
66 largefiles extension means that updating is no longer guaranteed to
67 be a local-only operation.
67 be a local-only operation.
68
68
69 If you already have large files tracked by Mercurial without the
69 If you already have large files tracked by Mercurial without the
70 largefiles extension, you will need to convert your repository in
70 largefiles extension, you will need to convert your repository in
71 order to benefit from largefiles. This is done with the
71 order to benefit from largefiles. This is done with the
72 :hg:`lfconvert` command::
72 :hg:`lfconvert` command::
73
73
74 $ hg lfconvert --size 10 oldrepo newrepo
74 $ hg lfconvert --size 10 oldrepo newrepo
75
75
76 In repositories that already have largefiles in them, any new file
76 In repositories that already have largefiles in them, any new file
77 over 10MB will automatically be added as a largefile. To change this
77 over 10MB will automatically be added as a largefile. To change this
78 threshold, set ``largefiles.minsize`` in your Mercurial config file
78 threshold, set ``largefiles.minsize`` in your Mercurial config file
79 to the minimum size in megabytes to track as a largefile, or use the
79 to the minimum size in megabytes to track as a largefile, or use the
80 --lfsize option to the add command (also in megabytes)::
80 --lfsize option to the add command (also in megabytes)::
81
81
82 [largefiles]
82 [largefiles]
83 minsize = 2
83 minsize = 2
84
84
85 $ hg add --lfsize 2
85 $ hg add --lfsize 2
86
86
87 The ``largefiles.patterns`` config option allows you to specify a list
87 The ``largefiles.patterns`` config option allows you to specify a list
88 of filename patterns (see :hg:`help patterns`) that should always be
88 of filename patterns (see :hg:`help patterns`) that should always be
89 tracked as largefiles::
89 tracked as largefiles::
90
90
91 [largefiles]
91 [largefiles]
92 patterns =
92 patterns =
93 *.jpg
93 *.jpg
94 re:.*\\.(png|bmp)$
94 re:.*\\.(png|bmp)$
95 library.zip
95 library.zip
96 content/audio/*
96 content/audio/*
97
97
98 Files that match one of these patterns will be added as largefiles
98 Files that match one of these patterns will be added as largefiles
99 regardless of their size.
99 regardless of their size.
100
100
101 The ``largefiles.minsize`` and ``largefiles.patterns`` config options
101 The ``largefiles.minsize`` and ``largefiles.patterns`` config options
102 will be ignored for any repositories not already containing a
102 will be ignored for any repositories not already containing a
103 largefile. To add the first largefile to a repository, you must
103 largefile. To add the first largefile to a repository, you must
104 explicitly do so with the --large flag passed to the :hg:`add`
104 explicitly do so with the --large flag passed to the :hg:`add`
105 command.
105 command.
106 '''
106 '''
107 from __future__ import absolute_import
107 from __future__ import absolute_import
108
108
109 from mercurial import (
109 from mercurial import (
110 cmdutil,
110 cmdutil,
111 configitems,
112 extensions,
111 extensions,
113 exthelper,
112 exthelper,
114 hg,
113 hg,
115 httppeer,
114 httppeer,
116 localrepo,
115 localrepo,
117 sshpeer,
116 sshpeer,
118 wireprotov1server,
117 wireprotov1server,
119 )
118 )
120
119
121 from . import (
120 from . import (
122 lfcommands,
121 lfcommands,
123 overrides,
122 overrides,
124 proto,
123 proto,
125 reposetup,
124 reposetup,
126 )
125 )
127
126
128 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
127 # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
129 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
128 # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
130 # be specifying the version(s) of Mercurial they are tested with, or
129 # be specifying the version(s) of Mercurial they are tested with, or
131 # leave the attribute unspecified.
130 # leave the attribute unspecified.
132 testedwith = 'ships-with-hg-core'
131 testedwith = 'ships-with-hg-core'
133
132
134 eh = exthelper.exthelper()
133 eh = exthelper.exthelper()
135 eh.merge(lfcommands.eh)
134 eh.merge(lfcommands.eh)
136 eh.merge(overrides.eh)
135 eh.merge(overrides.eh)
137 eh.merge(proto.eh)
136 eh.merge(proto.eh)
138
137
139 eh.configitem('largefiles', 'minsize',
138 eh.configitem('largefiles', 'minsize',
140 default=configitems.dynamicdefault,
139 default=eh.configitem.dynamicdefault,
141 )
140 )
142 eh.configitem('largefiles', 'patterns',
141 eh.configitem('largefiles', 'patterns',
143 default=list,
142 default=list,
144 )
143 )
145 eh.configitem('largefiles', 'usercache',
144 eh.configitem('largefiles', 'usercache',
146 default=None,
145 default=None,
147 )
146 )
148
147
149 cmdtable = eh.cmdtable
148 cmdtable = eh.cmdtable
150 configtable = eh.configtable
149 configtable = eh.configtable
151 extsetup = eh.finalextsetup
150 extsetup = eh.finalextsetup
152 reposetup = reposetup.reposetup
151 reposetup = reposetup.reposetup
153 uisetup = eh.finaluisetup
152 uisetup = eh.finaluisetup
154
153
155 def featuresetup(ui, supported):
154 def featuresetup(ui, supported):
156 # don't die on seeing a repo with the largefiles requirement
155 # don't die on seeing a repo with the largefiles requirement
157 supported |= {'largefiles'}
156 supported |= {'largefiles'}
158
157
159 @eh.uisetup
158 @eh.uisetup
160 def _uisetup(ui):
159 def _uisetup(ui):
161 localrepo.featuresetupfuncs.add(featuresetup)
160 localrepo.featuresetupfuncs.add(featuresetup)
162 hg.wirepeersetupfuncs.append(proto.wirereposetup)
161 hg.wirepeersetupfuncs.append(proto.wirereposetup)
163
162
164 cmdutil.outgoinghooks.add('largefiles', overrides.outgoinghook)
163 cmdutil.outgoinghooks.add('largefiles', overrides.outgoinghook)
165 cmdutil.summaryremotehooks.add('largefiles', overrides.summaryremotehook)
164 cmdutil.summaryremotehooks.add('largefiles', overrides.summaryremotehook)
166
165
167 # create the new wireproto commands ...
166 # create the new wireproto commands ...
168 wireprotov1server.wireprotocommand('putlfile', 'sha', permission='push')(
167 wireprotov1server.wireprotocommand('putlfile', 'sha', permission='push')(
169 proto.putlfile)
168 proto.putlfile)
170 wireprotov1server.wireprotocommand('getlfile', 'sha', permission='pull')(
169 wireprotov1server.wireprotocommand('getlfile', 'sha', permission='pull')(
171 proto.getlfile)
170 proto.getlfile)
172 wireprotov1server.wireprotocommand('statlfile', 'sha', permission='pull')(
171 wireprotov1server.wireprotocommand('statlfile', 'sha', permission='pull')(
173 proto.statlfile)
172 proto.statlfile)
174 wireprotov1server.wireprotocommand('lheads', '', permission='pull')(
173 wireprotov1server.wireprotocommand('lheads', '', permission='pull')(
175 wireprotov1server.heads)
174 wireprotov1server.heads)
176
175
177 extensions.wrapfunction(wireprotov1server.commands['heads'], 'func',
176 extensions.wrapfunction(wireprotov1server.commands['heads'], 'func',
178 proto.heads)
177 proto.heads)
179 # TODO also wrap wireproto.commandsv2 once heads is implemented there.
178 # TODO also wrap wireproto.commandsv2 once heads is implemented there.
180
179
181 # can't do this in reposetup because it needs to have happened before
180 # can't do this in reposetup because it needs to have happened before
182 # wirerepo.__init__ is called
181 # wirerepo.__init__ is called
183 proto.ssholdcallstream = sshpeer.sshv1peer._callstream
182 proto.ssholdcallstream = sshpeer.sshv1peer._callstream
184 proto.httpoldcallstream = httppeer.httppeer._callstream
183 proto.httpoldcallstream = httppeer.httppeer._callstream
185 sshpeer.sshv1peer._callstream = proto.sshrepocallstream
184 sshpeer.sshv1peer._callstream = proto.sshrepocallstream
186 httppeer.httppeer._callstream = proto.httprepocallstream
185 httppeer.httppeer._callstream = proto.httprepocallstream
187
186
188 # override some extensions' stuff as well
187 # override some extensions' stuff as well
189 for name, module in extensions.extensions():
188 for name, module in extensions.extensions():
190 if name == 'rebase':
189 if name == 'rebase':
191 # TODO: teach exthelper to handle this
190 # TODO: teach exthelper to handle this
192 extensions.wrapfunction(module, 'rebase',
191 extensions.wrapfunction(module, 'rebase',
193 overrides.overriderebase)
192 overrides.overriderebase)
194
193
195 revsetpredicate = eh.revsetpredicate
194 revsetpredicate = eh.revsetpredicate
General Comments 0
You need to be logged in to leave comments. Login now