Show More
@@ -1,199 +1,200 b'' | |||||
1 | # Copyright 2009-2010 Gregory P. Ward |
|
1 | # Copyright 2009-2010 Gregory P. Ward | |
2 | # Copyright 2009-2010 Intelerad Medical Systems Incorporated |
|
2 | # Copyright 2009-2010 Intelerad Medical Systems Incorporated | |
3 | # Copyright 2010-2011 Fog Creek Software |
|
3 | # Copyright 2010-2011 Fog Creek Software | |
4 | # Copyright 2010-2011 Unity Technologies |
|
4 | # Copyright 2010-2011 Unity Technologies | |
5 | # |
|
5 | # | |
6 | # This software may be used and distributed according to the terms of the |
|
6 | # This software may be used and distributed according to the terms of the | |
7 | # GNU General Public License version 2 or any later version. |
|
7 | # GNU General Public License version 2 or any later version. | |
8 |
|
8 | |||
9 | '''track large binary files |
|
9 | '''track large binary files | |
10 |
|
10 | |||
11 | Large binary files tend to be not very compressible, not very |
|
11 | Large binary files tend to be not very compressible, not very | |
12 | diffable, and not at all mergeable. Such files are not handled |
|
12 | diffable, and not at all mergeable. Such files are not handled | |
13 | efficiently by Mercurial's storage format (revlog), which is based on |
|
13 | efficiently by Mercurial's storage format (revlog), which is based on | |
14 | compressed binary deltas; storing large binary files as regular |
|
14 | compressed binary deltas; storing large binary files as regular | |
15 | Mercurial files wastes bandwidth and disk space and increases |
|
15 | Mercurial files wastes bandwidth and disk space and increases | |
16 | Mercurial's memory usage. The largefiles extension addresses these |
|
16 | Mercurial's memory usage. The largefiles extension addresses these | |
17 | problems by adding a centralized client-server layer on top of |
|
17 | problems by adding a centralized client-server layer on top of | |
18 | Mercurial: largefiles live in a *central store* out on the network |
|
18 | Mercurial: largefiles live in a *central store* out on the network | |
19 | somewhere, and you only fetch the revisions that you need when you |
|
19 | somewhere, and you only fetch the revisions that you need when you | |
20 | need them. |
|
20 | need them. | |
21 |
|
21 | |||
22 | largefiles works by maintaining a "standin file" in .hglf/ for each |
|
22 | largefiles works by maintaining a "standin file" in .hglf/ for each | |
23 | largefile. The standins are small (41 bytes: an SHA-1 hash plus |
|
23 | largefile. The standins are small (41 bytes: an SHA-1 hash plus | |
24 | newline) and are tracked by Mercurial. Largefile revisions are |
|
24 | newline) and are tracked by Mercurial. Largefile revisions are | |
25 | identified by the SHA-1 hash of their contents, which is written to |
|
25 | identified by the SHA-1 hash of their contents, which is written to | |
26 | the standin. largefiles uses that revision ID to get/put largefile |
|
26 | the standin. largefiles uses that revision ID to get/put largefile | |
27 | revisions from/to the central store. This saves both disk space and |
|
27 | revisions from/to the central store. This saves both disk space and | |
28 | bandwidth, since you don't need to retrieve all historical revisions |
|
28 | bandwidth, since you don't need to retrieve all historical revisions | |
29 | of large files when you clone or pull. |
|
29 | of large files when you clone or pull. | |
30 |
|
30 | |||
31 | To start a new repository or add new large binary files, just add |
|
31 | To start a new repository or add new large binary files, just add | |
32 | --large to your :hg:`add` command. For example:: |
|
32 | --large to your :hg:`add` command. For example:: | |
33 |
|
33 | |||
34 | $ dd if=/dev/urandom of=randomdata count=2000 |
|
34 | $ dd if=/dev/urandom of=randomdata count=2000 | |
35 | $ hg add --large randomdata |
|
35 | $ hg add --large randomdata | |
36 | $ hg commit -m "add randomdata as a largefile" |
|
36 | $ hg commit -m "add randomdata as a largefile" | |
37 |
|
37 | |||
38 | When you push a changeset that adds/modifies largefiles to a remote |
|
38 | When you push a changeset that adds/modifies largefiles to a remote | |
39 | repository, its largefile revisions will be uploaded along with it. |
|
39 | repository, its largefile revisions will be uploaded along with it. | |
40 | Note that the remote Mercurial must also have the largefiles extension |
|
40 | Note that the remote Mercurial must also have the largefiles extension | |
41 | enabled for this to work. |
|
41 | enabled for this to work. | |
42 |
|
42 | |||
43 | When you pull a changeset that affects largefiles from a remote |
|
43 | When you pull a changeset that affects largefiles from a remote | |
44 | repository, the largefiles for the changeset will by default not be |
|
44 | repository, the largefiles for the changeset will by default not be | |
45 | pulled down. However, when you update to such a revision, any |
|
45 | pulled down. However, when you update to such a revision, any | |
46 | largefiles needed by that revision are downloaded and cached (if |
|
46 | largefiles needed by that revision are downloaded and cached (if | |
47 | they have never been downloaded before). One way to pull largefiles |
|
47 | they have never been downloaded before). One way to pull largefiles | |
48 | when pulling is thus to use --update, which will update your working |
|
48 | when pulling is thus to use --update, which will update your working | |
49 | copy to the latest pulled revision (and thereby downloading any new |
|
49 | copy to the latest pulled revision (and thereby downloading any new | |
50 | largefiles). |
|
50 | largefiles). | |
51 |
|
51 | |||
52 | If you want to pull largefiles you don't need for update yet, then |
|
52 | If you want to pull largefiles you don't need for update yet, then | |
53 | you can use pull with the `--lfrev` option or the :hg:`lfpull` command. |
|
53 | you can use pull with the `--lfrev` option or the :hg:`lfpull` command. | |
54 |
|
54 | |||
55 | If you know you are pulling from a non-default location and want to |
|
55 | If you know you are pulling from a non-default location and want to | |
56 | download all the largefiles that correspond to the new changesets at |
|
56 | download all the largefiles that correspond to the new changesets at | |
57 | the same time, then you can pull with `--lfrev "pulled()"`. |
|
57 | the same time, then you can pull with `--lfrev "pulled()"`. | |
58 |
|
58 | |||
59 | If you just want to ensure that you will have the largefiles needed to |
|
59 | If you just want to ensure that you will have the largefiles needed to | |
60 | merge or rebase with new heads that you are pulling, then you can pull |
|
60 | merge or rebase with new heads that you are pulling, then you can pull | |
61 | with `--lfrev "head(pulled())"` flag to pre-emptively download any largefiles |
|
61 | with `--lfrev "head(pulled())"` flag to pre-emptively download any largefiles | |
62 | that are new in the heads you are pulling. |
|
62 | that are new in the heads you are pulling. | |
63 |
|
63 | |||
64 | Keep in mind that network access may now be required to update to |
|
64 | Keep in mind that network access may now be required to update to | |
65 | changesets that you have not previously updated to. The nature of the |
|
65 | changesets that you have not previously updated to. The nature of the | |
66 | largefiles extension means that updating is no longer guaranteed to |
|
66 | largefiles extension means that updating is no longer guaranteed to | |
67 | be a local-only operation. |
|
67 | be a local-only operation. | |
68 |
|
68 | |||
69 | If you already have large files tracked by Mercurial without the |
|
69 | If you already have large files tracked by Mercurial without the | |
70 | largefiles extension, you will need to convert your repository in |
|
70 | largefiles extension, you will need to convert your repository in | |
71 | order to benefit from largefiles. This is done with the |
|
71 | order to benefit from largefiles. This is done with the | |
72 | :hg:`lfconvert` command:: |
|
72 | :hg:`lfconvert` command:: | |
73 |
|
73 | |||
74 | $ hg lfconvert --size 10 oldrepo newrepo |
|
74 | $ hg lfconvert --size 10 oldrepo newrepo | |
75 |
|
75 | |||
76 | In repositories that already have largefiles in them, any new file |
|
76 | In repositories that already have largefiles in them, any new file | |
77 | over 10MB will automatically be added as a largefile. To change this |
|
77 | over 10MB will automatically be added as a largefile. To change this | |
78 | threshold, set ``largefiles.minsize`` in your Mercurial config file |
|
78 | threshold, set ``largefiles.minsize`` in your Mercurial config file | |
79 | to the minimum size in megabytes to track as a largefile, or use the |
|
79 | to the minimum size in megabytes to track as a largefile, or use the | |
80 | --lfsize option to the add command (also in megabytes):: |
|
80 | --lfsize option to the add command (also in megabytes):: | |
81 |
|
81 | |||
82 | [largefiles] |
|
82 | [largefiles] | |
83 | minsize = 2 |
|
83 | minsize = 2 | |
84 |
|
84 | |||
85 | $ hg add --lfsize 2 |
|
85 | $ hg add --lfsize 2 | |
86 |
|
86 | |||
87 | The ``largefiles.patterns`` config option allows you to specify a list |
|
87 | The ``largefiles.patterns`` config option allows you to specify a list | |
88 | of filename patterns (see :hg:`help patterns`) that should always be |
|
88 | of filename patterns (see :hg:`help patterns`) that should always be | |
89 | tracked as largefiles:: |
|
89 | tracked as largefiles:: | |
90 |
|
90 | |||
91 | [largefiles] |
|
91 | [largefiles] | |
92 | patterns = |
|
92 | patterns = | |
93 | *.jpg |
|
93 | *.jpg | |
94 | re:.*\\.(png|bmp)$ |
|
94 | re:.*\\.(png|bmp)$ | |
95 | library.zip |
|
95 | library.zip | |
96 | content/audio/* |
|
96 | content/audio/* | |
97 |
|
97 | |||
98 | Files that match one of these patterns will be added as largefiles |
|
98 | Files that match one of these patterns will be added as largefiles | |
99 | regardless of their size. |
|
99 | regardless of their size. | |
100 |
|
100 | |||
101 | The ``largefiles.minsize`` and ``largefiles.patterns`` config options |
|
101 | The ``largefiles.minsize`` and ``largefiles.patterns`` config options | |
102 | will be ignored for any repositories not already containing a |
|
102 | will be ignored for any repositories not already containing a | |
103 | largefile. To add the first largefile to a repository, you must |
|
103 | largefile. To add the first largefile to a repository, you must | |
104 | explicitly do so with the --large flag passed to the :hg:`add` |
|
104 | explicitly do so with the --large flag passed to the :hg:`add` | |
105 | command. |
|
105 | command. | |
106 | ''' |
|
106 | ''' | |
107 |
|
107 | |||
108 | from mercurial import ( |
|
108 | from mercurial import ( | |
109 | cmdutil, |
|
109 | cmdutil, | |
|
110 | configitems, | |||
110 | extensions, |
|
111 | extensions, | |
111 | exthelper, |
|
112 | exthelper, | |
112 | hg, |
|
113 | hg, | |
113 | localrepo, |
|
114 | localrepo, | |
114 | wireprotov1server, |
|
115 | wireprotov1server, | |
115 | ) |
|
116 | ) | |
116 |
|
117 | |||
117 | from . import ( |
|
118 | from . import ( | |
118 | lfcommands, |
|
119 | lfcommands, | |
119 | overrides, |
|
120 | overrides, | |
120 | proto, |
|
121 | proto, | |
121 | reposetup, |
|
122 | reposetup, | |
122 | ) |
|
123 | ) | |
123 |
|
124 | |||
124 | # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for |
|
125 | # Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for | |
125 | # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should |
|
126 | # extensions which SHIP WITH MERCURIAL. Non-mainline extensions should | |
126 | # be specifying the version(s) of Mercurial they are tested with, or |
|
127 | # be specifying the version(s) of Mercurial they are tested with, or | |
127 | # leave the attribute unspecified. |
|
128 | # leave the attribute unspecified. | |
128 | testedwith = b'ships-with-hg-core' |
|
129 | testedwith = b'ships-with-hg-core' | |
129 |
|
130 | |||
130 | eh = exthelper.exthelper() |
|
131 | eh = exthelper.exthelper() | |
131 | eh.merge(lfcommands.eh) |
|
132 | eh.merge(lfcommands.eh) | |
132 | eh.merge(overrides.eh) |
|
133 | eh.merge(overrides.eh) | |
133 | eh.merge(proto.eh) |
|
134 | eh.merge(proto.eh) | |
134 |
|
135 | |||
135 | eh.configitem( |
|
136 | eh.configitem( | |
136 | b'largefiles', |
|
137 | b'largefiles', | |
137 | b'minsize', |
|
138 | b'minsize', | |
138 |
default= |
|
139 | default=configitems.dynamicdefault, | |
139 | ) |
|
140 | ) | |
140 | eh.configitem( |
|
141 | eh.configitem( | |
141 | b'largefiles', |
|
142 | b'largefiles', | |
142 | b'patterns', |
|
143 | b'patterns', | |
143 | default=list, |
|
144 | default=list, | |
144 | ) |
|
145 | ) | |
145 | eh.configitem( |
|
146 | eh.configitem( | |
146 | b'largefiles', |
|
147 | b'largefiles', | |
147 | b'usercache', |
|
148 | b'usercache', | |
148 | default=None, |
|
149 | default=None, | |
149 | ) |
|
150 | ) | |
150 |
|
151 | |||
151 | cmdtable = eh.cmdtable |
|
152 | cmdtable = eh.cmdtable | |
152 | configtable = eh.configtable |
|
153 | configtable = eh.configtable | |
153 | extsetup = eh.finalextsetup |
|
154 | extsetup = eh.finalextsetup | |
154 | reposetup = reposetup.reposetup |
|
155 | reposetup = reposetup.reposetup | |
155 | uisetup = eh.finaluisetup |
|
156 | uisetup = eh.finaluisetup | |
156 |
|
157 | |||
157 |
|
158 | |||
158 | def featuresetup(ui, supported): |
|
159 | def featuresetup(ui, supported): | |
159 | # don't die on seeing a repo with the largefiles requirement |
|
160 | # don't die on seeing a repo with the largefiles requirement | |
160 | supported |= {b'largefiles'} |
|
161 | supported |= {b'largefiles'} | |
161 |
|
162 | |||
162 |
|
163 | |||
163 | @eh.uisetup |
|
164 | @eh.uisetup | |
164 | def _uisetup(ui): |
|
165 | def _uisetup(ui): | |
165 | localrepo.featuresetupfuncs.add(featuresetup) |
|
166 | localrepo.featuresetupfuncs.add(featuresetup) | |
166 | hg.wirepeersetupfuncs.append(proto.wirereposetup) |
|
167 | hg.wirepeersetupfuncs.append(proto.wirereposetup) | |
167 |
|
168 | |||
168 | cmdutil.outgoinghooks.add(b'largefiles', overrides.outgoinghook) |
|
169 | cmdutil.outgoinghooks.add(b'largefiles', overrides.outgoinghook) | |
169 | cmdutil.summaryremotehooks.add(b'largefiles', overrides.summaryremotehook) |
|
170 | cmdutil.summaryremotehooks.add(b'largefiles', overrides.summaryremotehook) | |
170 |
|
171 | |||
171 | # create the new wireproto commands ... |
|
172 | # create the new wireproto commands ... | |
172 | wireprotov1server.wireprotocommand(b'putlfile', b'sha', permission=b'push')( |
|
173 | wireprotov1server.wireprotocommand(b'putlfile', b'sha', permission=b'push')( | |
173 | proto.putlfile |
|
174 | proto.putlfile | |
174 | ) |
|
175 | ) | |
175 | wireprotov1server.wireprotocommand(b'getlfile', b'sha', permission=b'pull')( |
|
176 | wireprotov1server.wireprotocommand(b'getlfile', b'sha', permission=b'pull')( | |
176 | proto.getlfile |
|
177 | proto.getlfile | |
177 | ) |
|
178 | ) | |
178 | wireprotov1server.wireprotocommand( |
|
179 | wireprotov1server.wireprotocommand( | |
179 | b'statlfile', b'sha', permission=b'pull' |
|
180 | b'statlfile', b'sha', permission=b'pull' | |
180 | )(proto.statlfile) |
|
181 | )(proto.statlfile) | |
181 | wireprotov1server.wireprotocommand(b'lheads', b'', permission=b'pull')( |
|
182 | wireprotov1server.wireprotocommand(b'lheads', b'', permission=b'pull')( | |
182 | wireprotov1server.heads |
|
183 | wireprotov1server.heads | |
183 | ) |
|
184 | ) | |
184 |
|
185 | |||
185 | extensions.wrapfunction( |
|
186 | extensions.wrapfunction( | |
186 | wireprotov1server.commands[b'heads'], b'func', proto.heads |
|
187 | wireprotov1server.commands[b'heads'], b'func', proto.heads | |
187 | ) |
|
188 | ) | |
188 | # TODO also wrap wireproto.commandsv2 once heads is implemented there. |
|
189 | # TODO also wrap wireproto.commandsv2 once heads is implemented there. | |
189 |
|
190 | |||
190 | # override some extensions' stuff as well |
|
191 | # override some extensions' stuff as well | |
191 | for name, module in extensions.extensions(): |
|
192 | for name, module in extensions.extensions(): | |
192 | if name == b'rebase': |
|
193 | if name == b'rebase': | |
193 | # TODO: teach exthelper to handle this |
|
194 | # TODO: teach exthelper to handle this | |
194 | extensions.wrapfunction( |
|
195 | extensions.wrapfunction( | |
195 | module, b'rebase', overrides.overriderebasecmd |
|
196 | module, b'rebase', overrides.overriderebasecmd | |
196 | ) |
|
197 | ) | |
197 |
|
198 | |||
198 |
|
199 | |||
199 | revsetpredicate = eh.revsetpredicate |
|
200 | revsetpredicate = eh.revsetpredicate |
General Comments 0
You need to be logged in to leave comments.
Login now