RhodeCode Cluster
RhodeCode Cluster is a multi-node highly-scalable setup to run
RhodeCode, Edge Router(Traefik + SSL) and Metrics Stack (Prometheus, Loki, Grafana) and
all its additional components in single environment using Docker.
Using a docker-compose this setup creates following services for RhodeCode:
Edge-Router:
- Traefik, Edge Router, SSL termination etc
Core Services:
- Database (defaults to PostgreSQL)
- Redis, acts as cache and queue exchange
- ChannelStream - live websocket communications
- Nginx (static/channelstream) proxy serving RhodeCode static files and channelstream communication
- Elasticsearch (full text search backend)
RhodeCode
- RhodeCode CE/EE
- VCSServer for GIT/SVN/HG support
- SSH Server for cloning over SSH
- SVN webserver for HTTP support over SVN
- Celery workers for asynchronous tasks
- Celery beat for automation tasks
Metrics
- Loki, logs aggregation
- Grafana, metrics Dashboard
- Prometheus, metrics time-series
- Statsd-exporter - statsd to Prometheus bridge
- Node-exporter - machine stats and usage
- Promtail - log scraping
##rccontrol for Linux, new docker based installer
To get started with RhodeCode get the new shell installer called rccontrol
mkdir docker-rhodecode && cd docker-rhodecode
curl -o rccontrol https://code.rhodecode.com/rhodecode-enterprise-docker/raw/master/scripts/rccontrol/rccontrol && chmod +x rccontrol
./rccontrol get-started
Pre requisites
To Run this stack Docker engine and Docker Compose needs to be installed on the host machine.
Please run ./rccontrol init to install docker using the installer, or
visit docker site and install docker (min version 20.10) and docker compose:
rccontrol installer can install docker on linux machine, but if manual installation is
required it's also possible by si ply installing docker before
Quick install tutorial
Those are step-by-step installation/run steps.
create configurations / docker definitions:
./rccontrol init
At this point a custom file under .custom/.runtime.env was created. Adjust it if required.
Start Traefik router that would handle all incoming traffic, load balance. A valid domain needs to be present in
.custom/.runtime.env to access the RhodeCOde
./rccontrol stack router up --detach
Start the database and bootstrap it
./rcontrol stack database up --detach
Start other services required
./rccontrol stack services up --detach
Start RhodeCode stack
./rccontrol stack rhodecode up --detach
Check stack status
./rccontrol status
Output should look similar like this:
---
CONTAINER ID IMAGE STATUS NAMES PORTS
ef54fc528e3a traefik:v2.9.5 Up 2 hours rc_cluster_router-traefik-1 0.0.0.0:80->80/tcp, :::80->80/tcp
f3ea0539e8b0 rhodecode/rhodecode-ee:4.28.0 Up 2 hours (healthy) rc_cluster_apps-rhodecode-1 0.0.0.0:10020->10020/tcp, :::10020->10020/tcp
2be52ba58ffe rhodecode/rhodecode-ee:4.28.0 Up 2 hours (healthy) rc_cluster_apps-vcsserver-1
7cd730ad3263 rhodecode/rhodecode-ee:4.28.0 Up 2 hours (healthy) rc_cluster_apps-celery-1
dfa231342c87 rhodecode/rhodecode-ee:4.28.0 Up 2 hours (healthy) rc_cluster_apps-celery-beat-1
d3d76ce2de96 rhodecode/rhodecode-ee:4.28.0 Up 2 hours (healthy) rc_cluster_apps-sshd-1
daaac329414b rhodecode/rhodecode-ee:4.28.0 Up 2 hours (healthy) rc_cluster_apps-svn-1
7b8504fb9acb nginx:1.23.2 Up 2 hours (healthy) rc_cluster_services-nginx-1 80/tcp
7279c25feb6b elasticsearch:6.8.23 Up 2 hours (healthy) rc_cluster_services-elasticsearch-1 9200/tcp, 9300/tcp
19fb93587493 redis:7.0.5 Up 2 hours (healthy) rc_cluster_services-redis-1 6379/tcp
fb77fb6496c6 channelstream/channelstream:0.7.1 Up 2 hours (healthy) rc_cluster_services-channelstream-1 8000/tcp
cb6c5c022f5b postgres:14.6 Up 2 hours (healthy) rc_cluster_services-database-1 5432/tcp
Standalone cluster build from installer
If you;d like to build your own custom image here's a quick how to.:
Follow these steps to build and run the RhodeCode Cluster via Docker-compose.
1) Run:
./rccontrol init
2) Run artifacts fetch like installer and certain needed build binaries:
Start by fetching required installer binaries. This is required to create both
simple build and full compose setup.
Please use the `--version-name VERSION_NAME` flag to adjust RhodeCode version if needed.
(e.g. --version-name "4.24.1")
./rccontrol get-build-artifacts
This will download required installer files and put them into the `.cache` directory.
This directory should look similar to that after downloads have finish:
drwxr-xr-x 8 rcdev rcdev 256B Feb 8 13:35 .
drwxr-xr-x 14 rcdev rcdev 448B Feb 8 10:40 ..
-rw-r--r-- 1 rcdev rcdev 0B Feb 8 20:44 .dirkeep
-rwxr-xr-x 1 rcdev rcdev 241M Feb 8 13:35 RhodeCode-installer-linux-build20210208_0800
-rw-r--r-- 1 rcdev rcdev 156M Feb 8 13:35 RhodeCodeCommunity-4.24.1+x86_64-linux_build20210208_0800.tar.bz2
-rw-r--r-- 1 rcdev rcdev 171M Feb 8 13:35 RhodeCodeEnterprise-4.24.1+x86_64-linux_build20210208_0800.tar.bz2
-rw-r--r-- 1 rcdev rcdev 145M Feb 8 13:35 RhodeCodeVCSServer-4.24.1+x86_64-linux_build20210208_0800.tar.bz2
-rw-r--r-- 1 rcdev rcdev 109M Feb 8 13:35 locale-archive
3) Create the build
./rccontrol build
Standalone cluster build from source code
There's an option to build the latest release from the source code as a docker installation.
If you;d like to build your own custom image based on the source code here's a quick how to.:
Follow these steps to build and run the RhodeCode Cluster via Docker-compose.
1) Run init to setup needed docker env and files:
./rccontrol init
2) Run artifacts fetch like installer and certain needed build binaries:
Start by fetching required installer binaries.
Only local-archive is required to be present, installer is disregarded for source build
./rccontrol get-build-artifacts
This will download required installer files and put them into the `.cache` directory.
This directory should look similar to that after downloads have finish:
drwxr-xr-x 8 rcdev rcdev 256B Feb 8 13:35 .
drwxr-xr-x 14 rcdev rcdev 448B Feb 8 10:40 ..
-rw-r--r-- 1 rcdev rcdev 0B Feb 8 20:44 .dirkeep
-rwxr-xr-x 1 rcdev rcdev 241M Feb 8 13:35 RhodeCode-installer-linux-build20210208_0800
-rw-r--r-- 1 rcdev rcdev 156M Feb 8 13:35 RhodeCodeCommunity-4.24.1+x86_64-linux_build20210208_0800.tar.bz2
-rw-r--r-- 1 rcdev rcdev 171M Feb 8 13:35 RhodeCodeEnterprise-4.24.1+x86_64-linux_build20210208_0800.tar.bz2
-rw-r--r-- 1 rcdev rcdev 145M Feb 8 13:35 RhodeCodeVCSServer-4.24.1+x86_64-linux_build20210208_0800.tar.bz2
-rw-r--r-- 1 rcdev rcdev 109M Feb 8 13:35 locale-archive
3) get source code needed to create a build
This step will create source code copies into the `.source/` path. So it will look like this:
```
-rw-r--r-- 1 docker docker 0 Nov 25 12:27 .dirkeep
drwxr-xr-x 1 docker docker 1184 Nov 25 12:27 rhodecode-enterprise-ce
drwxr-xr-x 1 docker docker 1120 Nov 25 12:27 rhodecode-enterprise-ee
drwxr-xr-x 1 docker docker 800 Nov 25 12:27 rhodecode-vcsserver
```
If you have the 3 required projects source code already, this step can be omitted, and the
sources can be copied to the `.source/` directory. note: symlinks don't work.
- <https://code.rhodecode.com/rhodecode-vcsserver>
- <https://code.rhodecode.com/rhodecode-enterprise-ce>
- <https://code.rhodecode.com/rhodecode-enterprise-ee> (assuming access is granted to this)
Run this to fetch the sources
./rccontrol get-build-source --revision=default --auth-token=xxxx --server-url=https://code.rhodecode.com/sources
4) Create the build
./rccontrol build-source --version-name 4.28.0.REL.2022.12.12
Operation
Data structure
There are 4 volumes defined:
-
/etc/rhodecode/confShared volume used for configuration files for rhodecode, vcsserver and supervisord, and some cache data
-
/var/opt/rhodecode_repo_storeUsed for main repository storage where repositories would be stored
-
/var/opt/rhodecode_dataData dir for rhodecode cache/lock files, or user sessions (for file backend)
-
/var/log/rhodecodeUsed to store all logs from RhodeCode
Set License for EE version
In order to install EE edition a license file is required to be present.
It can contain your current license, or when empty license can be applied via Web interface.
To apply it during build phase save your raw license data into a file
config/rhodecode_enterprise.license
If this file is present build phase will read it and license will be applied at creation.
This file can also be empty and license can be applied via a WEB interface after first login.
Run Docker compose build:
This will build RhodeCode based on downloaded installer packages.
To create a full stack we need to run the database container, so it's ready to
build the docker image.
Disk space problems?
docker system df
docker builder prune -f
# optionally
docker image prune -a
Creating & building images
1) start with running the required database for the build stage in the background.
docker-compose up --detach database
This will start our postgres database, and expose it to the network.
Postgres DB is configured to output logs into a file `/var/log/rhodecode/postgres`
2) We can now run the full installation. Database needs to be running for the next build command.
This will build the rhodecode base image used for rhodecode, vcsserver, celery, ssh, svn
Then it will build all other components required.
docker-compose build rhodecode
docker-compose build
3) Once we build all required containers, we can run the whole stack using docker-compose up
docker-compose up
If running locally you can access Running RhodeCode via Nginx under:
http://localhost:8888
localhost can be changed to the server IP where docker is running.
In case for bigger setups docker-compose can scale more rhodecode/vcsserver workers:
docker-compose up --scale vcsserver=3 rhodecode=3
Logging is pushed to stdout from all services.
Upgrade procedure:
- run ./rccontrol self-update
- run ./rccontrol stack-upgrade to get upgrade instructions
With this done, you can now proceed with every step of normal source installation (Creating & building images), but instead of using
just docker-compose command it needs to be replaced with docker-compose -f docker-compose.yaml -f docker-compose.source.yaml
For example to override the installer build with the source rhodecode based image, and also setting proper version, run:
RC_VERSION="4.28.0.SRC.2022.12.12.1" docker-compose -f docker-compose-apps.yaml -f docker-compose.source.yaml build --no-cache --progress=plain rhodecode
NOTE THAT it's recommended to keep rc_version and source_Ver the same
Simple community build
Build docker RhodeCode Community without any dependencies (redis, external db) using
simple sqlite database and file based caches.
This is a fully running instance good for small use with 3-5 users.
docker build -t rhodecode/rhodecode-ce:4.23.2 -f rhodecode.dockerfile \
-e RHODECODE_TYPE=Community \
-e RHODECODE_VERSION=4.23.2 \
-e RHODECODE_DB=sqlite \
-e RHODECODE_USER_NAME=admin \
-e RHODECODE_USER_PASS=secret4 \
-e RHODECODE_USER_EMAIL=support@rhodecode.com \
.
note: for debugging better to add --progress plain into the build command to obtain all the output from the build.
To Build against existing running Postgres or MySQL you can specify:
--build-arg RHODECODE_DB=postgresql://postgres:secret@database/rhodecode
--build-arg RHODECODE_DB=mysql://root:secret@localhost/rhodecode?charset=utf8
To copy over the data into volumes use such command:
docker run -v logvolume:/data --name data_vol busybox true
docker cp . data_vol:/data
docker rm data_vol
Run the container, mounting the required volumes. By default the application would be
available at http://localhost:10020, and default login is (unless specified differently in the build command)
user: admin
password: secret4
We've not built our image using specific version. It's time to run it:
We specify the run.ini by selecting config option we have locally
docker run \
--name rhodecode-container \
--publish 10020:10020 \
--restart unless-stopped \
--volume $PWD/config:/etc/rhodecode/conf \
--volume $PWD/config/rhodecode.ini:/etc/rhodecode/conf_build/rhodecode.ini \
--volume $PWD/logs:/var/log/rhodecode \
'rhodecode/rhodecode-ee:4.23.2'
Enter container
docker exec -it rhodecode-container /bin/bash
Enter interactive shell
docker exec -it rhodecode-container /usr/local/bin/rhodecode_bin/bin/rc-ishell /etc/rhodecode/conf/rhodecode.ini
Run Database migrations
docker exec -it rhodecode-container /usr/local/bin/rhodecode_bin/bin/rc-upgrade-db /etc/rhodecode/conf/rhodecode.ini --force-yes
Registry for docker swarm
docker run -d -p 5000:5000 --restart always --name registry registry:2