upgrade: make sure we reclone all revlogs when updating to some format

Adding or removing some requirement (eg: sparserevlog), requires to reclone
revlog to use the new format. We cannot simply copy the original files.

In this case, we issue a warning to proceed with clone every revlogs.

mercurial/upgrade.py

@@ -28,6 +28,12 b' from .utils import ('
     compression,
 )
 
+# list of requirements that request a clone of all revlog if added/removed
+RECLONES_REQUIREMENTS = {
+    'generaldelta',
+    localrepo.SPARSEREVLOG_REQUIREMENT,
+}
+
 def requiredsourcerequirements(repo):
     """Obtain requirements required to be present to upgrade a repo.
 
@@ -952,6 +958,17 b' def upgraderepo(ui, repo, run=False, opt'
                    # determineactions could have added optimisation
                    if o not in actions)
 
+    removedreqs = repo.requirements - newreqs
+    addedreqs = newreqs - repo.requirements
+
+    if revlogs != UPGRADE_ALL_REVLOGS:
+        incompatible = RECLONES_REQUIREMENTS & (removedreqs | addedreqs)
+        if incompatible:
+            msg = _('ignoring revlogs selection flags, format requirements '
+                    'change: %s\n')
+            ui.warn(msg % ', '.join(sorted(incompatible)))
+            revlogs = UPGRADE_ALL_REVLOGS
+
     def printrequirements():
         ui.write(_('requirements\n'))
         ui.write(_('   preserved: %s\n') %

tests/test-upgrade-repo.t

@@ -728,6 +728,109 b' Check that we can select filelog only'
   checking files
   checked 3 changesets with 3 changes to 3 files
 
+
+Check you can't skip revlog clone during important format downgrade
+
+  $ echo "[format]" > .hg/hgrc
+  $ echo "sparse-revlog=no" >> .hg/hgrc
+  $ hg debugupgrade --optimize re-delta-parent --run --manifest --no-backup --debug --traceback
+  ignoring revlogs selection flags, format requirements change: sparserevlog
+  upgrade will perform the following actions:
+  
+  requirements
+     preserved: dotencode, fncache, generaldelta, revlogv1, store
+     removed: sparserevlog
+  
+  re-delta-parent
+     deltas within internal storage will choose a new base revision if needed
+  
+  beginning upgrade...
+  repository locked and read-only
+  creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob)
+  (it is safe to interrupt this process any time before data migration completes)
+  migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog)
+  migrating 917 bytes in store; 401 bytes tracked data
+  migrating 3 filelogs containing 3 revisions (192 bytes in store; 0 bytes tracked data)
+  cloning 1 revisions from data/f0.i
+  cloning 1 revisions from data/f1.i
+  cloning 1 revisions from data/f2.i
+  finished migrating 3 filelog revisions across 3 filelogs; change in size: 0 bytes
+  migrating 1 manifests containing 3 revisions (349 bytes in store; 220 bytes tracked data)
+  cloning 3 revisions from 00manifest.i
+  finished migrating 3 manifest revisions across 1 manifests; change in size: 0 bytes
+  migrating changelog containing 3 revisions (376 bytes in store; 181 bytes tracked data)
+  cloning 3 revisions from 00changelog.i
+  finished migrating 3 changelog revisions; change in size: 0 bytes
+  finished migrating 9 total revisions; total change in store size: 0 bytes
+  copying phaseroots
+  data fully migrated to temporary repository
+  marking source repository as being upgraded; clients will be unable to read from repository
+  starting in-place swap of repository data
+  replaced files will be backed up at $TESTTMP/upgradegd/.hg/upgradebackup.* (glob)
+  replacing store...
+  store replacement complete; repository was inconsistent for *s (glob)
+  finalizing requirements file and making repository readable again
+  removing old repository content$TESTTMP/upgradegd/.hg/upgradebackup.* (glob)
+  removing temporary repository $TESTTMP/upgradegd/.hg/upgrade.* (glob)
+  $ hg verify
+  checking changesets
+  checking manifests
+  crosschecking files in changesets and manifests
+  checking files
+  checked 3 changesets with 3 changes to 3 files
+
+Check you can't skip revlog clone during important format upgrade
+
+  $ echo "sparse-revlog=yes" >> .hg/hgrc
+  $ hg debugupgrade --optimize re-delta-parent --run --manifest --no-backup --debug --traceback
+  ignoring revlogs selection flags, format requirements change: sparserevlog
+  upgrade will perform the following actions:
+  
+  requirements
+     preserved: dotencode, fncache, generaldelta, revlogv1, store
+     added: sparserevlog
+  
+  sparserevlog
+     Revlog supports delta chain with more unused data between payload. These gaps will be skipped at read time. This allows for better delta chains, making a better compression and faster exchange with server.
+  
+  re-delta-parent
+     deltas within internal storage will choose a new base revision if needed
+  
+  beginning upgrade...
+  repository locked and read-only
+  creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob)
+  (it is safe to interrupt this process any time before data migration completes)
+  migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in changelog)
+  migrating 917 bytes in store; 401 bytes tracked data
+  migrating 3 filelogs containing 3 revisions (192 bytes in store; 0 bytes tracked data)
+  cloning 1 revisions from data/f0.i
+  cloning 1 revisions from data/f1.i
+  cloning 1 revisions from data/f2.i
+  finished migrating 3 filelog revisions across 3 filelogs; change in size: 0 bytes
+  migrating 1 manifests containing 3 revisions (349 bytes in store; 220 bytes tracked data)
+  cloning 3 revisions from 00manifest.i
+  finished migrating 3 manifest revisions across 1 manifests; change in size: 0 bytes
+  migrating changelog containing 3 revisions (376 bytes in store; 181 bytes tracked data)
+  cloning 3 revisions from 00changelog.i
+  finished migrating 3 changelog revisions; change in size: 0 bytes
+  finished migrating 9 total revisions; total change in store size: 0 bytes
+  copying phaseroots
+  data fully migrated to temporary repository
+  marking source repository as being upgraded; clients will be unable to read from repository
+  starting in-place swap of repository data
+  replaced files will be backed up at $TESTTMP/upgradegd/.hg/upgradebackup.* (glob)
+  replacing store...
+  store replacement complete; repository was inconsistent for *s (glob)
+  finalizing requirements file and making repository readable again
+  removing old repository content$TESTTMP/upgradegd/.hg/upgradebackup.* (glob)
+  removing temporary repository $TESTTMP/upgradegd/.hg/upgrade.* (glob)
+  $ hg verify
+  checking changesets
+  checking manifests
+  crosschecking files in changesets and manifests
+  checking files
+  checked 3 changesets with 3 changes to 3 files
+
   $ cd ..
 
 store files with special filenames aren't encoded during copy