repository: define and use revision flag constants

Revlogs have a per-revision 2 byte field holding integer flags that
define how revision data should be interpreted. For historical reasons,
these integer values are sent verbatim on the wire protocol as part of
changegroup data.

From a semantic standpoint, the flags that go out over the wire are
different from the flags stored internally by revlogs. Failure to
establish this semantic distinction creates unwanted strong coupling
between revlog's internals and the wire protocol.

This commit establishes new constants on the repository module that
define the revision flags used by the wire protocol (and by some
internal storage APIs, sadly). The changegroups internals documentation
has been updated to document them explicitly. Various references
throughout the repo now use the repository constants instead of the
revlog constants. This is done to make it clear that we're operating
on generic revision data and this isn't tied to revlogs.

Differential Revision: https://phab.mercurial-scm.org/D4860

mercurial/changegroup.py

@@ -26,7 +26,6 b' from . import ('
     phases,
     pycompat,
     repository,
-    revlog,
     util,
 )
 
@@ -716,7 +715,7 b' def deltagroup(repo, store, nodes, ischa'
                 p1node, p2node = adjustedparents[revision.node]
                 revision.p1node = p1node
                 revision.p2node = p2node
-                revision.flags |= revlog.REVIDX_ELLIPSIS
+                revision.flags |= repository.REVISION_FLAG_ELLIPSIS
 
         else:
             linknode = lookup(revision.node)

mercurial/help/internals/changegroups.txt

@@ -4,8 +4,8 b' filelogs.'
 
 There are 3 versions of changegroups: ``1``, ``2``, and ``3``. From a
 high-level, versions ``1`` and ``2`` are almost exactly the same, with the
-only difference being an additional item in the *delta header*.  Version
-``3`` adds support for revlog flags in the *delta header* and optionally
+only difference being an additional item in the *delta header*. Version
+``3`` adds support for storage flags in the *delta header* and optionally
 exchanging treemanifests (enabled by setting an option on the
 ``changegroup`` part in the bundle2).
 
@@ -127,6 +127,25 b' In version 2 and up, the delta base node'
 changegroup. This allows the delta to be expressed against any parent,
 which can result in smaller deltas and more efficient encoding of data.
 
+The *flags* field holds bitwise flags affecting the processing of revision
+data. The following flags are defined:
+
+32768
+   Censored revision. The revision's fulltext has been replaced by censor
+   metadata. May only occur on file revisions.
+16384
+   Ellipsis revision. Revision hash does not match data (likely due to rewritten
+   parents).
+8192
+   Externally stored. The revision fulltext contains ``key:value`` ``\n``
+   delimited metadata defining an object stored elsewhere. Used by the LFS
+   extension.
+
+For historical reasons, the integer values are identical to revlog version 1
+per-revision storage flags and correspond to bits being set in this 2-byte
+field. Bits were allocated starting from the most-significant bit, hence the
+reverse ordering and allocation of these flags.
+
 Changeset Segment
 =================
 

mercurial/repository.py

@@ -30,6 +30,13 b" REPO_FEATURE_LFS = b'lfs'"
 # Repository supports being stream cloned.
 REPO_FEATURE_STREAM_CLONE = b'streamclone'
 
+REVISION_FLAG_CENSORED = 1 << 15
+REVISION_FLAG_ELLIPSIS = 1 << 14
+REVISION_FLAG_EXTSTORED = 1 << 13
+
+REVISION_FLAGS_KNOWN = (
+    REVISION_FLAG_CENSORED | REVISION_FLAG_ELLIPSIS | REVISION_FLAG_EXTSTORED)
+
 class ipeerconnection(interfaceutil.Interface):
     """Represents a "connection" to a repository.
 
@@ -375,7 +382,10 b' class irevisiondelta(interfaceutil.Inter'
         """20 byte node of the changelog revision this node is linked to.""")
 
     flags = interfaceutil.Attribute(
-        """2 bytes of integer flags that apply to this revision.""")
+        """2 bytes of integer flags that apply to this revision.
+
+        This is a bitwise composition of the ``REVISION_FLAG_*`` constants.
+        """)
 
     basenode = interfaceutil.Attribute(
         """20 byte node of the revision this data is a delta against.
@@ -658,7 +668,8 b' class ifilemutation(interfaceutil.Interf'
         The data passed in already contains a metadata header, if any.
 
         ``node`` and ``flags`` can be used to define the expected node and
-        the flags to use with storage.
+        the flags to use with storage. ``flags`` is a bitwise value composed
+        of the various ``REVISION_FLAG_*`` constants.
 
         ``add()`` is usually called when adding files from e.g. the working
         directory. ``addrevision()`` is often called by ``add()`` and for

mercurial/revlogutils/constants.py

@@ -10,6 +10,7 b''
 from __future__ import absolute_import
 
 from .. import (
+    repository,
     util,
 )
 
@@ -28,9 +29,16 b' REVLOGV1_FLAGS = FLAG_INLINE_DATA | FLAG'
 REVLOGV2_FLAGS = REVLOGV1_FLAGS
 
 # revlog index flags
-REVIDX_ISCENSORED = (1 << 15) # revision has censor metadata, must be verified
-REVIDX_ELLIPSIS = (1 << 14) # revision hash does not match data (narrowhg)
-REVIDX_EXTSTORED = (1 << 13) # revision data is stored externally
+
+# For historical reasons, revlog's internal flags were exposed via the
+# wire protocol and are even exposed in parts of the storage APIs.
+
+# revision has censor metadata, must be verified
+REVIDX_ISCENSORED = repository.REVISION_FLAG_CENSORED
+# revision hash does not match data (narrowhg)
+REVIDX_ELLIPSIS = repository.REVISION_FLAG_ELLIPSIS
+# revision data is stored externally
+REVIDX_EXTSTORED = repository.REVISION_FLAG_EXTSTORED
 REVIDX_DEFAULT_FLAGS = 0
 # stable order in which flags need to be processed and their processors applied
 REVIDX_FLAGS_ORDER = [

mercurial/testing/storage.py

@@ -17,7 +17,7 b' from ..node import ('
 from .. import (
     error,
     mdiff,
-    revlog,
+    repository,
 )
 from ..utils import (
     storageutil,
@@ -874,7 +874,7 b' class ifiledatatests(basetestcase):'
         with self._maketransactionfn() as tr:
             node0 = f.add(b'foo', None, tr, 0, nullid, nullid)
             f.addrevision(stored1, tr, 1, node0, nullid,
-                          flags=revlog.REVIDX_ISCENSORED)
+                          flags=repository.REVISION_FLAG_CENSORED)
 
         self.assertTrue(f.iscensored(1))
 
@@ -914,7 +914,7 b' class ifilemutationtests(basetestcase):'
         f = self._makefilefn()
         with self._maketransactionfn() as tr:
             for i in range(15, 0, -1):
-                if (1 << i) & ~revlog.REVIDX_KNOWN_FLAGS:
+                if (1 << i) & ~repository.REVISION_FLAGS_KNOWN:
                     flags = 1 << i
                     break
 

tests/simplestorerepo.py

@@ -371,7 +371,7 b' class filestorage(object):'
     def iscensored(self, rev):
         validaterev(rev)
 
-        return self._flags(rev) & revlog.REVIDX_ISCENSORED
+        return self._flags(rev) & repository.REVISION_FLAG_CENSORED
 
     def commonancestorsheads(self, a, b):
         validatenode(a)

tests/test-help.t

@@ -1038,8 +1038,8 b' sub-topics can be accessed'
   
       There are 3 versions of changegroups: "1", "2", and "3". From a high-
       level, versions "1" and "2" are almost exactly the same, with the only
-      difference being an additional item in the *delta header*.  Version "3"
-      adds support for revlog flags in the *delta header* and optionally
+      difference being an additional item in the *delta header*. Version "3"
+      adds support for storage flags in the *delta header* and optionally
       exchanging treemanifests (enabled by setting an option on the
       "changegroup" part in the bundle2).
   
@@ -1162,6 +1162,27 b' sub-topics can be accessed'
       changegroup. This allows the delta to be expressed against any parent,
       which can result in smaller deltas and more efficient encoding of data.
   
+      The *flags* field holds bitwise flags affecting the processing of revision
+      data. The following flags are defined:
+  
+      32768
+         Censored revision. The revision's fulltext has been replaced by censor
+         metadata. May only occur on file revisions.
+  
+      16384
+         Ellipsis revision. Revision hash does not match data (likely due to
+         rewritten parents).
+  
+      8192
+         Externally stored. The revision fulltext contains "key:value" "\n"
+         delimited metadata defining an object stored elsewhere. Used by the LFS
+         extension.
+  
+      For historical reasons, the integer values are identical to revlog version
+      1 per-revision storage flags and correspond to bits being set in this
+      2-byte field. Bits were allocated starting from the most-significant bit,
+      hence the reverse ordering and allocation of these flags.
+  
       Changeset Segment
       =================
   
@@ -3435,8 +3456,8 b' Sub-topic topics rendered properly'
   <p>
   There are 3 versions of changegroups: &quot;1&quot;, &quot;2&quot;, and &quot;3&quot;. From a
   high-level, versions &quot;1&quot; and &quot;2&quot; are almost exactly the same, with the
-  only difference being an additional item in the *delta header*.  Version
-  &quot;3&quot; adds support for revlog flags in the *delta header* and optionally
+  only difference being an additional item in the *delta header*. Version
+  &quot;3&quot; adds support for storage flags in the *delta header* and optionally
   exchanging treemanifests (enabled by setting an option on the
   &quot;changegroup&quot; part in the bundle2).
   </p>
@@ -3582,6 +3603,24 b' Sub-topic topics rendered properly'
   changegroup. This allows the delta to be expressed against any parent,
   which can result in smaller deltas and more efficient encoding of data.
   </p>
+  <p>
+  The *flags* field holds bitwise flags affecting the processing of revision
+  data. The following flags are defined:
+  </p>
+  <dl>
+   <dt>32768
+   <dd>Censored revision. The revision's fulltext has been replaced by censor metadata. May only occur on file revisions.
+   <dt>16384
+   <dd>Ellipsis revision. Revision hash does not match data (likely due to rewritten parents).
+   <dt>8192
+   <dd>Externally stored. The revision fulltext contains &quot;key:value&quot; &quot;\n&quot; delimited metadata defining an object stored elsewhere. Used by the LFS extension.
+  </dl>
+  <p>
+  For historical reasons, the integer values are identical to revlog version 1
+  per-revision storage flags and correspond to bits being set in this 2-byte
+  field. Bits were allocated starting from the most-significant bit, hence the
+  reverse ordering and allocation of these flags.
+  </p>
   <h2>Changeset Segment</h2>
   <p>
   The *changeset segment* consists of a single *delta group* holding