##// END OF EJS Templates
upgrade: no longer keep all revlogs in memory at any point...
upgrade: no longer keep all revlogs in memory at any point Keeping all object open is unsustainable, so we will open them on demand. This mean opening them multiple times, but this is a lesser evil. Each revlog consume a small amount of memory (index content, associated nodemap, etc). While there are few "big" revlog, the sheer amount of small filelog can become a significant issue memory wise, consuming multiple GB of memory. If you combines this extra usage with the use of multiprocessing, this usage can quickly get out of control. This can effectively block the upgrade of larger repository. This changeset fixes this issue.

File last commit:

r49857:c9f44fc9 stable
r50447:19948429 stable
Show More
status.rs
149 lines | 4.8 KiB | application/rls-services+xml | RustLexer
Raphaël Gomès
rust-dirstate-status: add first Rust implementation of `dirstate.status`...
r43565 // status.rs
//
// Copyright 2019 Raphaël Gomès <rgomes@octobus.net>
//
// This software may be used and distributed according to the terms of the
// GNU General Public License version 2 or any later version.
//! Rust implementation of dirstate.status (dirstate.py).
//! It is currently missing a lot of functionality compared to the Python one
//! and will only be triggered in narrow cases.
Simon Sapin
dirstate-v2: Apply SECOND_AMBIGUOUS to directory mtimes too...
r49332 use crate::dirstate::entry::TruncatedTimestamp;
Simon Sapin
dirstate-v2: Make more APIs fallible, returning Result...
r48126 use crate::dirstate_tree::on_disk::DirstateV2ParseError;
Raphaël Gomès
rust: introduce SIZE_FROM_OTHER_PARENT constant...
r44003 use crate::{
Simon Sapin
dirstate: Remove the flat Rust DirstateMap implementation...
r48882 utils::hg_path::{HgPath, HgPathError},
Raphaël Gomès
rust-status: add bare `hg status` support in hg-core...
r45015 PatternError,
Raphaël Gomès
rust: introduce SIZE_FROM_OTHER_PARENT constant...
r44003 };
Simon Sapin
dirstate: Remove the flat Rust DirstateMap implementation...
r48882
use std::{borrow::Cow, fmt};
Raphaël Gomès
rust-dirstate-status: add first Rust implementation of `dirstate.status`...
r43565
Raphaël Gomès
rust-status: add missing variants to `Dispatch` enum...
r45013 /// Wrong type of file from a `BadMatch`
/// Note: a lot of those don't exist on all platforms.
Raphaël Gomès
rust-status: move to recursive traversal to prepare for parallel traversal...
r45023 #[derive(Debug, Copy, Clone)]
Raphaël Gomès
rust-status: add missing variants to `Dispatch` enum...
r45013 pub enum BadType {
CharacterDevice,
BlockDevice,
FIFO,
Socket,
Directory,
Unknown,
}
Simon Sapin
rust: replace ToString impls with Display...
r47173 impl fmt::Display for BadType {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
f.write_str(match self {
Raphaël Gomès
rust-status: update rust-cpython bridge to account for the changes in core...
r45016 BadType::CharacterDevice => "character device",
BadType::BlockDevice => "block device",
BadType::FIFO => "fifo",
BadType::Socket => "socket",
BadType::Directory => "directory",
BadType::Unknown => "unknown",
Simon Sapin
rust: replace ToString impls with Display...
r47173 })
Raphaël Gomès
rust-status: update rust-cpython bridge to account for the changes in core...
r45016 }
}
Raphaël Gomès
rust-status: add missing variants to `Dispatch` enum...
r45013 /// Was explicitly matched but cannot be found/accessed
Raphaël Gomès
rust-status: move to recursive traversal to prepare for parallel traversal...
r45023 #[derive(Debug, Copy, Clone)]
Raphaël Gomès
rust-status: add missing variants to `Dispatch` enum...
r45013 pub enum BadMatch {
OsError(i32),
BadType(BadType),
}
Simon Sapin
dirstate-tree: Add the new `status()` algorithm...
r47883 /// `Box<dyn Trait>` is syntactic sugar for `Box<dyn Trait + 'static>`, so add
Raphaël Gomès
rust-status: only involve ignore mechanism when needed...
r45088 /// an explicit lifetime here to not fight `'static` bounds "out of nowhere".
Simon Sapin
dirstate-tree: Give to `status()` mutable access to the `DirstateMap`...
r47882 pub type IgnoreFnType<'a> =
Box<dyn for<'r> Fn(&'r HgPath) -> bool + Sync + 'a>;
Raphaël Gomès
rust-dirstate-status: add `walk_explicit` implementation, use `Matcher` trait...
r44367
Raphaël Gomès
rust-status: improve documentation and readability...
r45672 /// We have a good mix of owned (from directory traversal) and borrowed (from
/// the dirstate/explicit) paths, this comes up a lot.
Raphaël Gomès
hg-core: define a `dirstate_status` `Operation`...
r45673 pub type HgPathCow<'a> = Cow<'a, HgPath>;
Raphaël Gomès
rust-status: improve documentation and readability...
r45672
Raphaël Gomès
rust-status: refactor options into a `StatusOptions` struct...
r45011 #[derive(Debug, Copy, Clone)]
pub struct StatusOptions {
/// Whether we are on a filesystem with UNIX-like exec flags
pub check_exec: bool,
pub list_clean: bool,
Raphaël Gomès
rust-status: add function for sequential traversal of the working directory...
r45014 pub list_unknown: bool,
pub list_ignored: bool,
Simon Sapin
rhg: Add support for `rhg status --copies`...
r49285 /// Whether to populate `StatusPath::copy_source`
pub list_copies: bool,
Raphaël Gomès
rust-status: collect traversed directories if required...
r45353 /// Whether to collect traversed dirs for applying a callback later.
/// Used by `hg purge` for example.
pub collect_traversed_dirs: bool,
Raphaël Gomès
rust-status: add function for sequential traversal of the working directory...
r45014 }
Simon Sapin
rhg: Add support for `rhg status --copies`...
r49285 #[derive(Default)]
Raphaël Gomès
rust-status: rename `StatusResult` to `DirstateStatus`...
r45012 pub struct DirstateStatus<'a> {
Simon Sapin
rhg: Update the dirstate on disk after status...
r49250 /// The current time at the start of the `status()` algorithm, as measured
/// and possibly truncated by the filesystem.
Simon Sapin
dirstate-v2: Apply SECOND_AMBIGUOUS to directory mtimes too...
r49332 pub filesystem_time_at_status_start: Option<TruncatedTimestamp>,
Simon Sapin
rhg: Update the dirstate on disk after status...
r49250
Simon Sapin
rust: Add doc-comments to DirstateStatus fields...
r47881 /// Tracked files whose contents have changed since the parent revision
Simon Sapin
rhg: Add support for `rhg status --copies`...
r49285 pub modified: Vec<StatusPath<'a>>,
Simon Sapin
rust: Add doc-comments to DirstateStatus fields...
r47881
/// Newly-tracked files that were not present in the parent
Simon Sapin
rhg: Add support for `rhg status --copies`...
r49285 pub added: Vec<StatusPath<'a>>,
Simon Sapin
rust: Add doc-comments to DirstateStatus fields...
r47881
/// Previously-tracked files that have been (re)moved with an hg command
Simon Sapin
rhg: Add support for `rhg status --copies`...
r49285 pub removed: Vec<StatusPath<'a>>,
Simon Sapin
rust: Add doc-comments to DirstateStatus fields...
r47881
/// (Still) tracked files that are missing, (re)moved with an non-hg
/// command
Simon Sapin
rhg: Add support for `rhg status --copies`...
r49285 pub deleted: Vec<StatusPath<'a>>,
Simon Sapin
rust: Add doc-comments to DirstateStatus fields...
r47881
/// Tracked files that are up to date with the parent.
/// Only pupulated if `StatusOptions::list_clean` is true.
Simon Sapin
rhg: Add support for `rhg status --copies`...
r49285 pub clean: Vec<StatusPath<'a>>,
Simon Sapin
rust: Add doc-comments to DirstateStatus fields...
r47881
/// Files in the working directory that are ignored with `.hgignore`.
/// Only pupulated if `StatusOptions::list_ignored` is true.
Simon Sapin
rhg: Add support for `rhg status --copies`...
r49285 pub ignored: Vec<StatusPath<'a>>,
Simon Sapin
rust: Add doc-comments to DirstateStatus fields...
r47881
/// Files in the working directory that are neither tracked nor ignored.
/// Only pupulated if `StatusOptions::list_unknown` is true.
Simon Sapin
rhg: Add support for `rhg status --copies`...
r49285 pub unknown: Vec<StatusPath<'a>>,
Simon Sapin
rust: Add doc-comments to DirstateStatus fields...
r47881
/// Was explicitly matched but cannot be found/accessed
Raphaël Gomès
rust-status: improve documentation and readability...
r45672 pub bad: Vec<(HgPathCow<'a>, BadMatch)>,
Simon Sapin
rust: Add doc-comments to DirstateStatus fields...
r47881
Simon Sapin
rust: Move "lookup" a.k.a. "unsure" paths into `DirstateStatus` struct...
r47880 /// Either clean or modified, but we can’t tell from filesystem metadata
/// alone. The file contents need to be read and compared with that in
/// the parent.
Simon Sapin
rhg: Add support for `rhg status --copies`...
r49285 pub unsure: Vec<StatusPath<'a>>,
Simon Sapin
rust: Add doc-comments to DirstateStatus fields...
r47881
Raphaël Gomès
rust-status: collect traversed directories if required...
r45353 /// Only filled if `collect_traversed_dirs` is `true`
Simon Sapin
dirstate-tree: Change status() results to not borrow DirstateMap...
r48136 pub traversed: Vec<HgPathCow<'a>>,
Simon Sapin
dirstate-v2: Write .hg/dirstate back to disk on directory cache changes...
r48139
/// Whether `status()` made changed to the `DirstateMap` that should be
/// written back to disk
pub dirty: bool,
Raphaël Gomès
rust-dirstate-status: add first Rust implementation of `dirstate.status`...
r43565 }
Simon Sapin
rhg: Add support for `rhg status --copies`...
r49285 #[derive(Clone, PartialEq, Eq, PartialOrd, Ord)]
pub struct StatusPath<'a> {
pub path: HgPathCow<'a>,
pub copy_source: Option<HgPathCow<'a>>,
}
Simon Sapin
rust: replace trivial `impl From …` with `#[derive(derive_more::From)]`...
r47164 #[derive(Debug, derive_more::From)]
Raphaël Gomès
rust-status: refactor status into a struct...
r45671 pub enum StatusError {
Raphaël Gomès
rust-status: improve documentation and readability...
r45672 /// An invalid path that cannot be represented in Mercurial was found
Raphaël Gomès
rust-status: refactor status into a struct...
r45671 Path(HgPathError),
Raphaël Gomès
rust-status: improve documentation and readability...
r45672 /// An invalid "ignore" pattern was found
Raphaël Gomès
rust-status: refactor status into a struct...
r45671 Pattern(PatternError),
Simon Sapin
dirstate-v2: Make more APIs fallible, returning Result...
r48126 /// Corrupted dirstate
DirstateV2ParseError(DirstateV2ParseError),
Raphaël Gomès
rust-status: refactor status into a struct...
r45671 }
Simon Sapin
rust: replace ToString impls with Display...
r47173 impl fmt::Display for StatusError {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
Raphaël Gomès
rust-status: refactor status into a struct...
r45671 match self {
Simon Sapin
rust: replace ToString impls with Display...
r47173 StatusError::Path(error) => error.fmt(f),
StatusError::Pattern(error) => error.fmt(f),
Simon Sapin
dirstate-v2: Make more APIs fallible, returning Result...
r48126 StatusError::DirstateV2ParseError(_) => {
f.write_str("dirstate-v2 parse error")
}
Raphaël Gomès
rust-status: refactor status into a struct...
r45671 }
}
}