docs(misc): various docs improvements

docs/source/conf.py

@@ -1,80 +1,83 b''
 # Configuration file for the Sphinx documentation builder.
 #
 # For the full list of built-in configuration values, see the documentation:
 # https://www.sphinx-doc.org/en/master/usage/configuration.html
 
 # -- Project information -----------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
 import datetime
 
 
 def _get_version():
     with open("../../VERSION") as f:
         return f.read().strip()
 
 
 now = datetime.datetime.today()
 
 # The full project version, used as the replacement for |release| and e.g. in the HTML templates.
 # For example, for the Python documentation, this may be something like 2.6.0rc1.
 # If you don’t need the separation provided between version and release, just set them both to the same value.
 release = _get_version()
 
 
 project = 'RhodeCode rcstack'
 copyright = f'2010-{now.year}, RhodeCode Inc.'
 author = 'RhodeCode Inc.'
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
 
 extensions = [
     'sphinx.ext.autodoc',
     'sphinx.ext.intersphinx',
     'sphinx.ext.todo',
     'sphinx.ext.imgmath'
 ]
 
 templates_path = ['_templates']
 exclude_patterns = []
 
 # The name of the Pygments (syntax highlighting) style to use.
 pygments_style = 'sphinx'
 
 # -- Options for HTML output -------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
 
 # https://github.com/pradyunsg/furo/tree/main/src/furo/theme/furo
 html_theme = 'furo'
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
 html_static_path = ['_static']
 
 # If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
 # using the given strftime format.
 html_last_updated_fmt = ' %H:%m %b %d, %Y'
 
 # If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
 html_show_sphinx = False
 
 html_extra_path = ['extra']
 
 
 rst_epilog = '''
 .. |RCCEshort| replace:: Community
 .. |RCEEshort| replace:: Enterprise
 .. |git| replace:: Git
 .. |hg| replace:: Mercurial
 .. |svn| replace:: Subversion
-
+.. |repo| replace:: repository
+.. |repos| replace:: repositories
+.. |authtoken| replace:: Authentication Token
+.. |authtokens| replace:: **Auth Tokens**
 .. |RCC| replace:: RhodeCode Control
 .. |RCE| replace:: RhodeCode Enterprise
 .. |RCCE| replace:: RhodeCode Community
 .. |RCEE| replace:: RhodeCode Enterprise
 .. |RCT| replace:: RhodeCode Tools
 
 .. |RCEBOLD| replace:: **RhodeCode Enterprise**
 .. |RCEITALICS| replace:: `RhodeCode Enterprise`
 '''

docs/source/index.rst

@@ -1,184 +1,183 b''
 Welcome to RhodeCode rcstack documentation!
 ===========================================
 
 
 =================
 RhodeCode rcstack
 =================
 
 RhodeCode rcstack is a multi-node highly-scalable setup to run RhodeCode services.
 It consists of all RhodeCode services required with additionally
 Edge Router(Traefik with optional SSL) and Metrics Stack (Prometheus, Loki, Grafana) in single environment using Docker.
 
 .. only:: html
 
    Latest Version
    --------------
 
    The latest version of rcstack is |release|.
 
 
 Overview
 --------
 
 To quickly get and install rcstack run this command
 
 .. code-block:: bash
 
     mkdir docker-rhodecode && cd docker-rhodecode
     curl -L -s -o rcstack https://dls.rhodecode.com/get-rcstack && chmod +x rcstack
     ./rcstack get-started
 
 
 Using a docker-compose this setup creates following services for RhodeCode:
 
 
 
 **Router (mandatory to run):**
 
 
 .. code-block:: bash
 
     ./rcstack router --help
 
 
 `Traefik <https://doc.traefik.io/traefik/>`_ aka Edge Router is responsible for handling all outgoing and incoming traffic to whole stack.
 It can do SSL termination, and handle SSL traffic with certificate management.
 
 Traefik listens to few defaults ports and
 handles all incoming/outgoing traffic to the stack
 
 - 80   (HTTP)
 - 443  (HTTPS)
 - 9022 (SSH for RhodeCode traffic)
 - 3100 (Loki log aggregation)
 
 
 **Services (mandatory to run):**
 
 .. code-block:: bash
 
     ./rcstack services --help
 
 
 - Database (defaults to PostgreSQL) Optionally can be replaced by MySQL, or own external database if needed.
 - Redis, acts as cache and queue exchange
 - ChannelStream - live websocket communications
 - Elasticsearch (full text search backend)
 - Nginx (static file service) proxy serving RhodeCode static files
 
 
 **RhodeCode (mandatory to run):**
 
 .. code-block:: bash
 
     ./rcstack rhodecode --help
 
 
 - RhodeCode CE/EE web app
 - VCSServer for GIT/SVN/HG support
 - SSH server for cloning over SSH
 - SVN webserver for SVN protocol support
 - Celery workers for asynchronous tasks
 - Celery beat for scheduler and automation tasks
 
 
 **Metrics (Optional)**
 
 .. code-block:: bash
 
     ./rcstack metrics --help
 
 - Loki, logs aggregation
 - Grafana, metrics Dashboard
 - Prometheus, metrics time-series
 - Statsd-exporter - statsd to Prometheus bridge
 - Node-exporter - machine stats and usage
 - Promtail - log scraping
 
 
 System Requirements
 -------------------
 
 Hardware
 ++++++++
 
 2 CPU cores and 2GB RAM is typically sufficient for small teams/projects.
 We recommend 2-4 CPU cores and 4-8GB of RAM to unlock full potential of all services running in the
 RhodeCode Stack.
 
 
 OS Support
 ++++++++++
 
 
 rcstack works on any linux distribution, and has a beta support for MacOS.
 There are two requirements for OS to be able to run whole rcstack.
 
 - Ability to run Docker daemon
 - Ability to run a certain standard unix tools like:
 
   - curl
   - tar
   - find
   - gzip
 
 
 
 .. toctree::
     :maxdepth: 1
     :hidden:
     :caption: Installation
 
     install/installation.rst
     install/upgrade.rst
 
 
 .. toctree::
     :maxdepth: 1
     :hidden:
     :caption: Configuration
 
     configuration/configuration-overview.rst
     configuration/configuration-of-components.rst
     configuration/scaling-rhodecode.rst
     configuration/enable-ssl-on-traefik.rst
     configuration/ssl-certificates.rst
     configuration/data-volumes.rst
     configuration/set-license-file.rst
     configuration/set-specific-install-version.rst
 
-
 .. toctree::
     :maxdepth: 1
     :hidden:
     :caption: Usage
 
     usage/monitoring-unhealthy-containers.rst
-
+    usage/full-text-search-setup.rst
 
 .. toctree::
     :maxdepth: 1
     :hidden:
     :caption: Migration
 
     migration/migration-to-docker.rst
 
 
 .. toctree::
     :maxdepth: 1
     :hidden:
     :caption: Builds & Images
 
     builds/build-new-image.rst
     builds/building-images.rst
     dev/dev-setup.rst
     dev/step-by-step-example.rst
 
 
 .. toctree::
     :maxdepth: 1
     :hidden:
     :caption: Changelog
 
     changes.rst
  No newline at end of file

docs/source/install/upgrade.rst

@@ -1,84 +1,101 b''
 .. _upgrade:
 
 Upgrade rcstack
 ===============
 
 
 It's usually recommended to start with updating the installer itself by running:
 
 .. code-block:: bash
 
     ./rcstack self-update
 
 
+.. note::
+
+    It's possible to change the update channel, or update to only specific versions. This can be done by providing
+    extra flags to self-update
+
+    .. code-block:: bash
+
+        # upgrade to master branch release line
+        ./rcstack self-update --force --cli-revision master --docker-revision master
+
+        # upgrade to a specific version corresponding to a git tag
+        ./rcstack self-update --force --cli-revision v5.3.0 --docker-revision v5.3.0
+
+
+Upgrade of services
+===================
+
 Then upgrade each stack based on instructions provided from the help output:
 
 .. code-block:: bash
 
     ./rcstack stack-upgrade <STACK_NAME>
 
 
 .. note::
 
     ./rcstack stack-upgrade rhodecode is a 0 downtime upgrade procedure.
     This will spawn 2x more nodes to perform the upgrade and then remove the old instances.
     This way all incoming traffic will be handled continuously without any interruptions on upgrade
 
 
 Example for full upgrade procedure:
 
 
 .. code-block:: bash
 
     ./rcstack self-update
     ./rcstack stack-upgrade router
     ./rcstack stack-upgrade services
     ./rcstack stack-upgrade rhodecode
     ./rcstack stack-upgrade metrics
 
 
 .. note::
 
     ./rcstack stack-upgrade does scaling of services which might use more RAM to handle the double instance amount.
     To avoid that stop each stack before running stack upgrade. This way services get upgraded without 2x scaling.
     This method however would cause a downtime.
 
     .. code-block:: bash
 
         ./rcstack self-update
         ./rcstack stack router down && ./rcstack stack-upgrade router
         ./rcstack stack services down && ./rcstack stack-upgrade services
         ./rcstack stack rhodecode down && ./rcstack stack-upgrade rhodecode
         ./rcstack stack metrics down && ./rcstack stack-upgrade metrics
 
 
 The above upgrade uses default timeouts to wait for any currently ongoing traffic to stop to kill the workers.
 There are two additional flags that can be used for a faster upgrade that could let to end users errors, or small downtime
 but will enforce the upgrade to be performed as fast as possible.
 
 
 Fast upgrade with max 65s wait time after new upgraded containers start, and wait 5s before force killing old
 containers. This is still 0 downtime upgrade, but stoping existing containers is forced to 5s
 which might result in killed ongoing connections
 
 
 .. code-block:: bash
 
     ./rcstack stack-upgrade rhodecode --stop-wait 65 --docker-stop-wait 5
 
 
 
-Install specific version
-++++++++++++++++++++++++
+Install specific version of RhodeCode
++++++++++++++++++++++++++++++++++++++
 
 
 In order to run a specific version of RhodeCode stack simply follow this:
 
 
 .. code-block:: bash
 
 
     ./rcstack cli set-runtime-image 5.0.0
     ./rcstack stack all down
     ./rcstack stack all up -d
 

docs/source/migration/migration-to-docker.rst

@@ -1,301 +1,310 b''
 ===================
 Migration to Docker
 ===================
 
 
 Migrate old pre 4.28.0 installer based releases
 -----------------------------------------------
 
 
 1. Decide on components use
 +++++++++++++++++++++++++++
 
 By default, RhodeCode stack uses Redis and Postgres. 
 We recommend using those in Docker as things like
 backup/restore are much easier, and possible to run by our installer
 
 If you plan to use your own redis or DB, disable those services in
 :file:`.custom/docker-compose-services.override.yaml` by setting ``replicas: 0``
 
 see: :ref:`configuration-of-components`
 
 Select your DB type by commenting postgres or mysql db in :file:`.custom/docker-compose-services.override.yaml`
 For Sqlite, disable both DB types.
 
 .. note::
 
    Selecting your own Redis, DB etc would require pointing to those instance inside .ini config files. By default
    rcstack is configured to talk to docker based addresses of those services.
 
 
 2. bootstrap the environment for rcstack
 ++++++++++++++++++++++++++++++++++++++++
 
 .. code-block::
 
     ./rcstack init
 
 
 During the install/configure phase pick corresponding edition (ce/ee)
 transfer your license-token, and domain used in previous setup.
 
 
 Then ensure 4.28.0 version is set for proper migration
 
 .. code-block::
 
     ./rcstack cli set-runtime-image 4.28.0
 
 
 3. Copy over your old configuration into new shared config dir
 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
 The init command will generate a docker optimized configurations, those by default have
 all services configured to be run against docker stack. Few options should be copied from old
 .ini config files for best backward compatibility and smooth migration.
 
 The new .ini files are located in the :file:`config/_shared/` directory.
 This is located in directory where rcstack is installed.
 
 Assumes installation is inside a directory :file:`/home/rcdev/.rccontrol/enterprise-1`
 
 .. code-block::
 
     cp /home/rcdev/.rccontrol/enterprise-1/rhodecode.ini /root/docker-rhodecode/config/_shared/rhodecode_old.ini
     cp /home/rcdev/.rccontrol/vcsserver-1/vcsserver.ini /root/docker-rhodecode/config/_shared/vcsserver_old.ini
 
 Then compare the new files e.g :file:`rhodecode.ini` with :file:`rhodecode_old.ini` and adjust accordingly,
 most important items for review are:
 
 - location of storages paths (those should be left Dockerized)
 - redis/db connection urls (depends on choice of services, either Docker urls, or external urls)
 - app_instance_uuid = COPY_OLD_VALUE
 - rhodecode.encrypted_values.secret = COPY_OLD_VALUE
 - beaker.session.secret = COPY_OLD_VALUE
 - email configuration
 
 
 For example, if you leave on your *own* redis instance then no adjustment is needed, if you decide to use
 docker based redis, here's an example change you need to make:
 
 .. code-block:: ini
 
     #rc_cache.cache_perms.arguments.host = localhost # this was commented out
     # new options is change to connect to docker redis instance, hostname is just the name of compose service (redis)
     rc_cache.cache_perms.arguments.host = redis
     rc_cache.cache_perms.arguments.port = 6379
     rc_cache.cache_perms.arguments.db = 0
     rc_cache.cache_perms.arguments.socket_timeout = 30
 
 
 4. launch services stack
 ++++++++++++++++++++++++
 
 Launch the service stack with Database and all required services
 
 .. code-block:: bash
 
     ./rcstack stack services up -d
 
 Check the status of service stack by checking the status command
 
 .. code-block:: bash
 
     ./rcstack status
 
     Running hostname: https://code.rhodecode.com
     CONTAINER ID   NAMES                                 IMAGE                               STATUS                                 PORTS
     5746f723d892   rc_cluster_services-channelstream-1   channelstream/channelstream:0.7.1   Up About a minute (healthy)            8000/tcp
     bef9599eef0c   rc_cluster_services-database-1        postgres:14.9                       Up About a minute (healthy)            5432/tcp
     addf574b6f1c   rc_cluster_services-elasticsearch-1   elasticsearch:6.8.23                Up About a minute (healthy)            9200/tcp, 9300/tcp
     9f89816cc6dc   rc_cluster_services-nginx-errors-1    nginx:1.25.2                        Up About a minute                      80/tcp
     1c8e90e8ab7d   rc_cluster_services-nginx-statics-1   nginx:1.25.2                        Up About a minute (health)             80/tcp
     fcb8378cd506   rc_cluster_services-redis-1           redis:7.0.12                        Up About a minute (healthy)            6379/tcp
 
 
 
 5. make a db dump
 +++++++++++++++++
 
 .. note::
 
     Run this only if docker-based DB is selected. Skip otherwise
 
 create a backup of your existing database, so it can be migrated to docker instance.
 Place the backup file into :file:`.custom/db_dump`
 
 
 .. code-block:: bash
 
    # For MySQL DBs
    $ mysqldump -u <uname> -p <pass> rhodecode_db_name > .custom/db_dump/mysql-db-backup.sql
 
    # For PostgreSQL DBs
    $ PGPASSWORD=<pass> pg_dump --inserts -U <uname> -h localhost rhodecode_db_name > .custom/db_dump/postgresql-db-backup.sql
 
    # For SQLite
    $ sqlite3 rhodecode.db ‘.dump’ > .custom/db_dump/sqlite-db-backup.sql
 
 
 
 6. restore db
 +++++++++++++
 
 
 .. note::
 
     Run this only if docker-based DB is selected. Skip otherwise
 
 Cleanup previous db created by init script, and re-create it.
 Restore the database into docker container running DB.
 Run the ./rcstack cli db command:
 
 .. code-block:: bash
 
     ./rcstack cli db
 
     attaching pg_data 			        under: /var/lib/postgresql/data
     attaching $PWD/.custom/db_dump 		under: /var/rc-data-dump
 
     restore dump:
     \i /var/rc-data-dump/your_dump.sql
 
     psql (14.9 (Debian 14.9-1.pgdg120+1))
     Type "help" for help.
 
     rhodecode=# \connect template1
     template1=# drop database rhodecode;
     template1=# create database rhodecode;
     template1=# \connect rhodecode
     rhodecode=# \i /var/rc-data-dump/postgresql-db-backup.sql
 
 
 This will restore the previously stored DB dump into new dockerized DB.
 
 
 7. move data into docker storage
 ++++++++++++++++++++++++++++++++
 
 Step one is to move the required components to the :file:`.custom/storage` directory inside the docker
 compose stack. This directory is mounted later on so data can be copied over into docker storage engine volumes.
 
 .. note::
 
     The tarball cache can be omitted since RhodeCode uses a new system which is not compatible with older release
 
 .. note::
 
     The path for artifacts can be found inside the **old** .ini file and DB
 
     .. code-block:: ini
 
         #artifacts, %(here)s is current location of old .ini file
         file_store.storage_path = %(here)s/data/file_store
 
 
 7.1. Copy artifacts into docker shared dir
 
 .. code-block:: bash
 
     mv -v .rccontrol/enterprise-1/data/file_store .custom/storage/
 
 
 7.2. Copy gists into docker shared dir
 
 .. code-block:: bash
 
     mv -v /home/rhodecode/repos/.gist .custom/storage/repos/.gist
 
 
 7.3. Copy repositories into docker shared dir
 
 .. code-block:: bash
 
     mv -v /home/rhodecode/repos .custom/storage/
 
 
 Once that is done, we can now move from the host directory into docker storage engine volumes.
 
 .. code-block:: bash
 
     ./rcstack cli storage
 
     attaching rc_datavolume         under: /vol/datavolume
     attaching rc_reposvolume        under: /vol/repovolume
     attaching $PWD/.custom/storage  under: /vol/backupvolume
 
     root@a27697d13f44:/vol#
 
 7.4. Move artifacts
 
 .. code-block:: bash
 
     root@a27697d13f44:/vol# mv /vol/backupvolume/repos/* /vol/repovolume/
 
 
 7.5. Move gists
 
 .. code-block:: bash
 
     root@a27697d13f44:/vol# mv /vol/backupvolume/repos/* /vol/repovolume/
 
 7.6. Move repositories
 
 .. code-block:: bash
 
     root@a27697d13f44:/vol# mv /vol/backupvolume/repos/* /vol/repovolume/
 
 
 8. Ensure other old services are stopped
 ++++++++++++++++++++++++++++++++++++++++
 
 At this point we're going to run the whole stack, because the router binds to some of the common
 ports, we need to make sure old db/nginx are stopped before we proceed.
 
 It's also important to stop running RhodeCode services.
 
 This is where a first downtime of previously ran setup would occur.
 
 
 
 9. Start complete stack
 +++++++++++++++++++++++
 
 .. code-block:: bash
 
     ./rcstack stack all up -d
 
 
 
 10. Fix DB paths
 ++++++++++++++++
 
 Docker internally changed the paths used before, he're a way to change it using ishell. But this can
 be also adjusted in the UI
 
 .. code-block:: bash
 
     ./rcstack cli ishell
 
 
 .. code-block:: python
 
     ui1 = RhodeCodeUi.query().filter(RhodeCodeUi.ui_key=='usercache').scalar()
     ui2 = RhodeCodeUi.query().filter(RhodeCodeUi.ui_key=='store_location').scalar()
     ui3 = RhodeCodeUi.query().filter(RhodeCodeUi.ui_key=='/').scalar()
 
     ui1.ui_value = '/var/opt/rhodecode_repo_store/.cache/lfs_store'
     ui2.ui_value = '/var/opt/rhodecode_repo_store/.cache/largefiles'
     ui3.ui_value = '/var/opt/rhodecode_repo_store'
 
     Session().add(ui1);Session().commit()
     Session().add(ui2);Session().commit()
     Session().add(ui3);Session().commit()
 
 
+11. Run full remap & rescan
++++++++++++++++++++++++++++
+
+From RhodeCode web-interface, the last step is to run a full remap & rescan action.
+
+1. From the RhodeCode Web interface, open
+   :menuselection:`Admin --> Settings --> Remap and rescan`
+2. Select :guilabel:`Invalidate cache for all repositories` to ensure we cleanup old caches from previous installs
+3. Click :guilabel:`Rescan Repositories` action
 
 
 SSL Certificates
 ^^^^^^^^^^^^^^^^
 
 - see: :ref:`configuration-of-ssl-certificates`
  No newline at end of file