##// END OF EJS Templates
upstream » mercurial-mirror
RSS

Clone from https://www.mercurial-scm.org/repo/hg-all

rust: new rust options in setup.py...
rust: new rust options in setup.py The --rust global option turns on usage (and by default compilation) of the rust-cpython based mercurial.rustext. Similarly to what's previously done for zstd, there is a --no-rust option for the build_ext subcommand in order not to build mercurial.rustext, allowing for an OS distribution to prebuild it. The HGWITHRUSTEXT environment variable is still honored, and has the same effect as before, but now it works mostly by making the --rust global option defaulting to True, with some special cases for the direct-ffi case (see more about that below) Coincidentally, the --rust flag can also be passed from the make commands, like actually all global options, in the PURE variable make local PURE=--rust This feels inappropriate, though, and we should follow up with a proper make variable for that case. Although the direct-ffi bindings aren't directly useful any more, we keep them at this stage because - they provide a short prototyping path for experiments in which a C extension module has to call into a Rust extension. The proper way of doing that would be to use capsules, and it's best to wait for our pull request onto rust-cpython for that: https://github.com/dgrunwald/rust-cpython/pull/169 - Build support for capsules defined in Rust will probably need to reuse some of what's currently in use for direct-ffi.
Matt Harbison - - Load All Authors

File last commit:

r42428:1756859a default
r42646:302acc78 default
Show More
TODO.rst
195 lines | 7.6 KiB | text/x-rst | RstLexer
/ hgext / lfs / TODO.rst
History | Annotation | Raw |Copy content |Copy permalink

Prior to removing (EXPERIMENTAL)

These things affect UI and/or behavior, and should probably be implemented (or ruled out) prior to taking off the experimental shrinkwrap.

  1. Finish the hg convert story
    • Add an argument to accept a rules file to apply during conversion? Currently lfs.track is the only way to affect the conversion.
    • drop lfs.track config settings
    • splice in .hglfs file for normal repo -> lfs conversions?
  2. Stop uploading blobs when pushing between local repos
    • Could probably hardlink directly to the other local repo's store
    • Support inferring lfs.url for local push/pull (currently only supports http)
  3. Stop uploading blobs on strip/amend/histedit/etc.
    • This seems to be a side effect of doing it for hg bundle, which probably makes sense.
  4. Handle a server with the extension loaded and a client without the extension more gracefully.
  5. Remove lfs.retry hack in client? This came from FB, but it's not clear why it is/was needed.
  6. hg export currently writes out the LFS blob. Should it write the pointer instead?
    • hg diff is similar, and probably shouldn't see the pointer file
  7. Fix https multiplexing, and re-enable workers.
  8. Show to-be-applied rules with hg files -r 'wdir()' 'set:lfs()'
    • debugignore can show file + line number, so a dedicated command could be useful too.
  9. Filesets, revsets and templates
    • A dedicated revset should be faster than 'file(set:lfs())'
    • Attach {lfsoid} and {lfspointer} to general keywords, IFF the file is a blob
    • Drop existing items that would be redundant with general support
  10. Can grep avoid downloading most things?
    • Add a command option to skip LFS blobs?
  11. Add a flag that's visible in hg files -v to indicate external storage?
  12. Server side issues
  13. Add locks on cache and blob store
    • This is complicated with a global store, and multiple potentially unrelated local repositories that reference the same blob.
    • Alternately, maybe just handle collisions when trying to create the same blob in the store somehow.
  14. Are proper file sizes reported in debugupgraderepo?
  15. Finish prefetching files
    • -T {rawdata}
    • verify
    • grep
  16. Output cleanup
    • Can we print the url when connecting to the blobstore? (A sudden connection refused after pulling commits looks confusing.) Problem is, 'pushing to main url' is printed, and then lfs wants to upload before going back to the main repo transfer, so then that could be confusing with extra output. (This is kinda improved with 380f5131ee7b and 9f78d10742af.)
    • Add more progress indicators? Uploading a large repo looks idle for a long time while it scans for blobs in each outgoing revision.
    • Print filenames instead of hashes in error messages
      • subrepo aware paths, where necessary
    • Is existing output at the right status/note/debug level?
  17. Can verify be done without downloading everything?
    • If we know that we are talking to an hg server, we can leverage the fact that it validates in the Batch API portion, and skip d/l altogether. OTOH, maybe we should download the files unconditionally for forensics. The alternative is to define a custom transfer handler that definitively verifies without transferring, and then cache those results. When verify comes looking, look in the cache instead of actually opening the file and processing it.
    • Yuya has concerns about when blob fetch takes place vs when revlog is verified. Since the visible hash matches the blob content, I don't think there's a way to verify the pointer file that's actually stored in the filelog (other than basic JSON checks). Full verification requires the blob. See https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-April/116133.html
    • Opening a corrupt pointer file aborts. It probably shouldn't for verify.

Future ideas/features/polishing

These aren't in any particular order, and are things that don't have obvious BC concerns.

  1. Garbage collection (issue5790)
    • This gets complicated because of the global cache, which may or may not consist of hardlinks to the repo, and may be in use by other repos. (So the gc may be pointless.)
  2. Compress blobs
    • 700MB repo becomes 2.5GB with all lfs blobs
    • What implications are there for filesystem paths that don't indicate compression? (i.e. how to share with global cache and other local repos?)
    • Probably needs to be stored under .hg/store/lfs/zstd, with a repo requirement.
    • Allow tuneable compression type and settings?
    • Support compression over the wire if both sides understand the compression?
    • debugupgraderepo to convert?
    • Probably not worth supporting compressed and uncompressed concurrently
  3. Determine things to upload with readfast()
    • Significantly faster when pushing an entire large repo to http.
    • Causes test changes to fileset and templates; may need both this and current methods of lookup.
  4. Is a command to download everything needed? This would allow copying the whole to a portable drive. Currently this can be effected by running hg verify.
  5. Stop reading in entire file into one buffer when passing through filelog interface
  6. Keep corrupt files around in 'store/lfs/incoming' for forensics?
    • Files should be downloaded to 'incoming', and moved to normal location when done.
  7. Client side path enhancements
  8. Server enhancements
  9. Handle 3rd party server storage.
    • Teach client to handle lfs verify action. This is needed after the server instructs the client to upload the file to another server, in order to tell the server that the upload completed.
    • Teach the server to send redirects if configured, and process verify requests.
  10. Is any hg-git work needed?