Show More
@@ -8,22 +8,76 b'' | |||||
8 |
|
8 | |||
9 | '''track large binary files |
|
9 | '''track large binary files | |
10 |
|
10 | |||
11 |
Large binary files tend to be not very compressible, not very |
|
11 | Large binary files tend to be not very compressible, not very | |
12 |
not at all mergeable. |
|
12 | diffable, and not at all mergeable. Such files are not handled | |
13 | format (revlog), which is based on compressed binary deltas. largefiles solves |
|
13 | efficiently by Mercurial's storage format (revlog), which is based on | |
14 | this problem by adding a centralized client-server layer on top of Mercurial: |
|
14 | compressed binary deltas; storing large binary files as regular | |
15 | largefiles live in a *central store* out on the network somewhere, and you only |
|
15 | Mercurial files wastes bandwidth and disk space and increases | |
16 | fetch the ones that you need when you need them. |
|
16 | Mercurial's memory usage. The largefiles extension addresses these | |
|
17 | problems by adding a centralized client-server layer on top of | |||
|
18 | Mercurial: largefiles live in a *central store* out on the network | |||
|
19 | somewhere, and you only fetch the revisions that you need when you | |||
|
20 | need them. | |||
|
21 | ||||
|
22 | largefiles works by maintaining a "standin file" in .hglf/ for each | |||
|
23 | largefile. The standins are small (41 bytes: an SHA-1 hash plus | |||
|
24 | newline) and are tracked by Mercurial. Largefile revisions are | |||
|
25 | identified by the SHA-1 hash of their contents, which is written to | |||
|
26 | the standin. largefiles uses that revision ID to get/put largefile | |||
|
27 | revisions from/to the central store. This saves both disk space and | |||
|
28 | bandwidth, since you don't need to retrieve all historical revisions | |||
|
29 | of large files when you clone or pull. | |||
|
30 | ||||
|
31 | To start a new repository or add new large binary files, just add | |||
|
32 | --large to your ``hg add`` command. For example:: | |||
|
33 | ||||
|
34 | $ dd if=/dev/urandom of=randomdata count=2000 | |||
|
35 | $ hg add --large randomdata | |||
|
36 | $ hg commit -m 'add randomdata as a largefile' | |||
|
37 | ||||
|
38 | When you push a changeset that adds/modifies largefiles to a remote | |||
|
39 | repository, its largefile revisions will be uploaded along with it. | |||
|
40 | Note that the remote Mercurial must also have the largefiles extension | |||
|
41 | enabled for this to work. | |||
17 |
|
42 | |||
18 | largefiles works by maintaining a *standin* in .hglf/ for each largefile. The |
|
43 | When you pull a changeset that affects largefiles from a remote | |
19 | standins are small (41 bytes: an SHA-1 hash plus newline) and are tracked by |
|
44 | repository, Mercurial behaves as normal. However, when you update to | |
20 | Mercurial. Largefile revisions are identified by the SHA-1 hash of their |
|
45 | such a revision, any largefiles needed by that revision are downloaded | |
21 | contents, which is written to the standin. largefiles uses that revision ID to |
|
46 | and cached (if they have never been downloaded before). This means | |
22 | get/put largefile revisions from/to the central store. |
|
47 | that network access may be required to update to changesets you have | |
|
48 | not previously updated to. | |||
|
49 | ||||
|
50 | If you already have large files tracked by Mercurial without the | |||
|
51 | largefiles extension, you will need to convert your repository in | |||
|
52 | order to benefit from largefiles. This is done with the 'hg lfconvert' | |||
|
53 | command:: | |||
|
54 | ||||
|
55 | $ hg lfconvert --size 10 oldrepo newrepo | |||
23 |
|
56 | |||
24 | A complete tutorial for using lfiles is included in ``usage.txt`` in the lfiles |
|
57 | In repositories that already have largefiles in them, any new file | |
25 | source distribution. See |
|
58 | over 10MB will automatically be added as a largefile. To change this | |
26 | https://developers.kilnhg.com/Repo/Kiln/largefiles/largefiles/File/usage.txt |
|
59 | threshhold, set ``largefiles.size`` in your Mercurial config file to | |
|
60 | the minimum size in megabytes to track as a largefile, or use the | |||
|
61 | --lfsize option to the add command (also in megabytes):: | |||
|
62 | ||||
|
63 | [largefiles] | |||
|
64 | size = 2 XXX wouldn't minsize be a better name? | |||
|
65 | ||||
|
66 | $ hg add --lfsize 2 | |||
|
67 | ||||
|
68 | The ``largefiles.patterns`` config option allows you to specify a list | |||
|
69 | of filename patterns (see ``hg help patterns``) that should always be | |||
|
70 | tracked as largefiles:: | |||
|
71 | ||||
|
72 | [largefiles] | |||
|
73 | patterns = | |||
|
74 | *.jpg | |||
|
75 | re:.*\.(png|bmp)$ | |||
|
76 | library.zip | |||
|
77 | content/audio/* | |||
|
78 | ||||
|
79 | Files that match one of these patterns will be added as largefiles | |||
|
80 | regardless of their size. | |||
27 | ''' |
|
81 | ''' | |
28 |
|
82 | |||
29 | from mercurial import commands |
|
83 | from mercurial import commands |
@@ -20,14 +20,23 b' import basestore' | |||||
20 | # -- Commands ---------------------------------------------------------- |
|
20 | # -- Commands ---------------------------------------------------------- | |
21 |
|
21 | |||
22 | def lfconvert(ui, src, dest, *pats, **opts): |
|
22 | def lfconvert(ui, src, dest, *pats, **opts): | |
23 |
''' |
|
23 | '''convert a normal repository to a largefiles repository | |
24 |
|
24 | |||
25 | Convert source repository creating an identical repository, except that all |
|
25 | Convert repository SOURCE to a new repository DEST, identical to | |
26 | files that match the patterns given, or are over the given size will be |
|
26 | SOURCE except that certain files will be converted as largefiles: | |
27 | added as largefiles. The size used to determine whether or not to track a |
|
27 | specifically, any file that matches any PATTERN *or* whose size is | |
28 | file as a largefile is the size of the first version of the file. After |
|
28 | above the minimum size threshold is converted as a largefile. The | |
29 | running this command you will need to make sure that largefiles is enabled |
|
29 | size used to determine whether or not to track a file as a | |
30 | anywhere you intend to push the new repository.''' |
|
30 | largefile is the size of the first version of the file. The | |
|
31 | minimum size can be specified either with --size or in | |||
|
32 | configuration as ``largefiles.size``. | |||
|
33 | ||||
|
34 | After running this command you will need to make sure that | |||
|
35 | largefiles is enabled anywhere you intend to push the new | |||
|
36 | repository. | |||
|
37 | ||||
|
38 | Use --tonormal to convert largefiles back to normal files; after | |||
|
39 | this, the DEST repository can be used without largefiles at all.''' | |||
31 |
|
40 | |||
32 | if opts['tonormal']: |
|
41 | if opts['tonormal']: | |
33 | tolfile = False |
|
42 | tolfile = False | |
@@ -464,10 +473,12 b' def _updatelfile(repo, lfdirstate, lfile' | |||||
464 |
|
473 | |||
465 | cmdtable = { |
|
474 | cmdtable = { | |
466 | 'lfconvert': (lfconvert, |
|
475 | 'lfconvert': (lfconvert, | |
467 |
[('s', 'size', '', |
|
476 | [('s', 'size', '', | |
468 | 'will be considered largefiles. This can also be specified ' |
|
477 | _('minimum size (MB) for files to be converted ' | |
469 |
' |
|
478 | 'as largefiles'), | |
470 |
|
|
479 | 'SIZE'), | |
471 | 'Convert from a largefiles repo to a normal repo')], |
|
480 | ('', 'tonormal', False, | |
|
481 | _('convert from a largefiles repo to a normal repo')), | |||
|
482 | ], | |||
472 | _('hg lfconvert SOURCE DEST [FILE ...]')), |
|
483 | _('hg lfconvert SOURCE DEST [FILE ...]')), | |
473 | } |
|
484 | } |
General Comments 0
You need to be logged in to leave comments.
Login now