##// END OF EJS Templates
docs: updated old commands not longer available starting from 4.11
marcink -
r2628:b50c9994 stable
parent child Browse files
Show More
@@ -1,52 +1,52 b''
1 1 =======================
2 2 DB Schema and Migration
3 3 =======================
4 4
5 5 To create or alter tables in the database, it's necessary to change a couple of
6 6 files, apart from configuring the settings pointing to the latest database
7 7 schema.
8 8
9 9
10 10 Database Model and ORM
11 11 ----------------------
12 12
13 13 On ``rhodecode.model.db`` you will find the database definition of all tables and
14 14 fields. Any freshly installed database will be correctly created by the definitions
15 15 here. So, any change to this file will affect the tests without having to change
16 16 any other file.
17 17
18 18 A second layer are the business classes inside ``rhodecode.model``.
19 19
20 20
21 21 Database Migration
22 22 ------------------
23 23
24 24 Three files play a role when creating database migrations:
25 25
26 26 * Database schema inside ``rhodecode.lib.dbmigrate``
27 27 * Database version inside ``rhodecode.lib.dbmigrate``
28 28 * Configuration ``__dbversion__`` at ``rhodecode.__init__``
29 29
30 30
31 31 Schema is a snapshot of the database version BEFORE the migration. So, it's
32 32 the initial state before any changes were added. The name convention is
33 33 the latest release version where the snapshot was created, and not the
34 34 target version of this code.
35 35
36 36 Version is the method that will define how to UPGRADE/DOWNGRADE the database.
37 37
38 38 ``rhodecode.__init__`` contains only a variable that defines up to which version of
39 39 the database will be used to upgrade. Eg.: ``__dbversion__ = 45``
40 40
41 41
42 42 For examples on how to create those files, please see the existing code.
43 43
44 44
45 45 Migration Command
46 46 ^^^^^^^^^^^^^^^^^
47 47
48 48 After you've changed the database ORM and migration files, you can run::
49 49
50 paster upgrade-db <ini-file>
50 rc-upgrade-db <ini-file>
51 51
52 52 The database will be upgraded up to the version defined in the ``__init__`` file. No newline at end of file
@@ -1,219 +1,219 b''
1 1 .. _dev-setup:
2 2
3 3 ===================
4 4 Development setup
5 5 ===================
6 6
7 7
8 8 RhodeCode Enterprise runs inside a Nix managed environment. This ensures build
9 9 environment dependencies are correctly declared and installed during setup.
10 10 It also enables atomic upgrades, rollbacks, and multiple instances of RhodeCode
11 11 Enterprise running with isolation.
12 12
13 13 To set up RhodeCode Enterprise inside the Nix environment, use the following steps:
14 14
15 15
16 16
17 17 Setup Nix Package Manager
18 18 -------------------------
19 19
20 20 To install the Nix Package Manager, please run::
21 21
22 22 $ curl https://nixos.org/nix/install | sh
23 23
24 24 or go to https://nixos.org/nix/ and follow the installation instructions.
25 25 Once this is correctly set up on your system, you should be able to use the
26 26 following commands:
27 27
28 28 * `nix-env`
29 29
30 30 * `nix-shell`
31 31
32 32
33 33 .. tip::
34 34
35 35 Update your channels frequently by running ``nix-channel --update``.
36 36
37 37
38 38 Switch nix to the latest STABLE channel
39 39 ---------------------------------------
40 40
41 41 run::
42 42
43 43 nix-channel --add https://nixos.org/channels/nixos-16.03 nixpkgs
44 44
45 45 Followed by::
46 46
47 47 nix-channel --update
48 48
49 49
50 50 Install required binaries
51 51 -------------------------
52 52
53 53 We need some handy tools first.
54 54
55 55 run::
56 56
57 57 nix-env -i nix-prefetch-hg
58 58 nix-env -i nix-prefetch-git
59 59
60 60
61 61 Clone the required repositories
62 62 -------------------------------
63 63
64 64 After Nix is set up, clone the RhodeCode Enterprise Community Edition and
65 65 RhodeCode VCSServer repositories into the same directory.
66 66 RhodeCode currently is using Mercurial Version Control System, please make sure
67 67 you have it installed before continuing.
68 68
69 69 To obtain the required sources, use the following commands::
70 70
71 71 mkdir rhodecode-develop && cd rhodecode-develop
72 72 hg clone https://code.rhodecode.com/rhodecode-enterprise-ce
73 73 hg clone https://code.rhodecode.com/rhodecode-vcsserver
74 74
75 75 .. note::
76 76
77 77 If you cannot clone the repository, please contact us via support@rhodecode.com
78 78
79 79
80 80 Install some required libraries
81 81 -------------------------------
82 82
83 83 There are some required drivers and dev libraries that we need to install to
84 84 test RhodeCode under different types of databases. For example in Ubuntu we
85 85 need to install the following.
86 86
87 87 required libraries::
88 88
89 89 sudo apt-get install libapr1-dev libaprutil1-dev
90 90 sudo apt-get install libsvn-dev
91 91 sudo apt-get install mysql-server libmysqlclient-dev
92 92 sudo apt-get install postgresql postgresql-contrib libpq-dev
93 93 sudo apt-get install libcurl4-openssl-dev
94 94
95 95
96 96 Enter the Development Shell
97 97 ---------------------------
98 98
99 99 The final step is to start the development shells. To do this, run the
100 100 following command from inside the cloned repository::
101 101
102 102 #first, the vcsserver
103 103 cd ~/rhodecode-vcsserver
104 104 nix-shell
105 105
106 106 # then enterprise sources
107 107 cd ~/rhodecode-enterprise-ce
108 108 nix-shell
109 109
110 110 .. note::
111 111
112 112 On the first run, this will take a while to download and optionally compile
113 113 a few things. The following runs will be faster. The development shell works
114 114 fine on both MacOS and Linux platforms.
115 115
116 116
117 117 Create config.nix for development
118 118 ---------------------------------
119 119
120 120 In order to run proper tests and setup linking across projects, a config.nix
121 121 file needs to be setup::
122 122
123 123 # create config
124 124 mkdir -p ~/.nixpkgs
125 125 touch ~/.nixpkgs/config.nix
126 126
127 127 # put the below content into the ~/.nixpkgs/config.nix file
128 128 # adjusts, the path to where you cloned your repositories.
129 129
130 130 {
131 131 rc = {
132 132 sources = {
133 133 rhodecode-vcsserver = "/home/dev/rhodecode-vcsserver";
134 134 rhodecode-enterprise-ce = "/home/dev/rhodecode-enterprise-ce";
135 135 rhodecode-enterprise-ee = "/home/dev/rhodecode-enterprise-ee";
136 136 };
137 137 };
138 138 }
139 139
140 140
141 141
142 142 Creating a Development Configuration
143 143 ------------------------------------
144 144
145 145 To create a development environment for RhodeCode Enterprise,
146 146 use the following steps:
147 147
148 148 1. Create a copy of vcsserver config:
149 149 `cp ~/rhodecode-vcsserver/configs/development.ini ~/rhodecode-vcsserver/configs/dev.ini`
150 150 2. Create a copy of rhodocode config:
151 151 `cp ~/rhodecode-enterprise-ce/configs/development.ini ~/rhodecode-enterprise-ce/configs/dev.ini`
152 152 3. Adjust the configuration settings to your needs if needed.
153 153
154 154 .. note::
155 155
156 156 It is recommended to use the name `dev.ini` since it's included in .hgignore file.
157 157
158 158
159 159 Setup the Development Database
160 160 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
161 161
162 162 To create a development database, use the following example. This is a one
163 163 time operation executed from the nix-shell of rhodecode-enterprise-ce sources ::
164 164
165 paster setup-rhodecode dev.ini \
165 rc-setup-app dev.ini \
166 166 --user=admin --password=secret \
167 167 --email=admin@example.com \
168 168 --repos=~/my_dev_repos
169 169
170 170
171 171 Compile CSS and JavaScript
172 172 ^^^^^^^^^^^^^^^^^^^^^^^^^^
173 173
174 174 To use the application's frontend and prepare it for production deployment,
175 175 you will need to compile the CSS and JavaScript with Grunt.
176 176 This is easily done from within the nix-shell using the following command::
177 177
178 178 grunt
179 179
180 180 When developing new features you will need to recompile following any
181 181 changes made to the CSS or JavaScript files when developing the code::
182 182
183 183 grunt watch
184 184
185 185 This prepares the development (with comments/whitespace) versions of files.
186 186
187 187 Start the Development Servers
188 188 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
189 189
190 190 From the rhodecode-vcsserver directory, start the development server in another
191 191 nix-shell, using the following command::
192 192
193 193 pserve configs/dev.ini
194 194
195 195 In the adjacent nix-shell which you created for your development server, you may
196 196 now start CE with the following command::
197 197
198 198
199 199 pserve --reload configs/dev.ini
200 200
201 201 .. note::
202 202
203 203 `--reload` flag will automatically reload the server when source file changes.
204 204
205 205
206 206 Run the Environment Tests
207 207 ^^^^^^^^^^^^^^^^^^^^^^^^^
208 208
209 209 Please make sure that the tests are passing to verify that your environment is
210 210 set up correctly. RhodeCode uses py.test to run tests.
211 211 While your instance is running, start a new nix-shell and simply run
212 212 ``make test`` to run the basic test suite.
213 213
214 214
215 215 Need Help?
216 216 ^^^^^^^^^^
217 217
218 218 Join us on Slack via https://rhodecode.com/join or post questions in our
219 219 Community Portal at https://community.rhodecode.com
@@ -1,232 +1,232 b''
1 1 #
2 2 # About
3 3 # =====
4 4 #
5 5 # This file defines jobs for our CI system and the attribute "build" is used
6 6 # as the input for packaging.
7 7 #
8 8 #
9 9 # CI details
10 10 # ==========
11 11 #
12 12 # This file defines an attribute set of derivations. Each of these attributes is
13 13 # then used in our CI system as one job to run. This way we keep the
14 14 # configuration for the CI jobs as well under version control.
15 15 #
16 16 # Run CI jobs locally
17 17 # -------------------
18 18 #
19 19 # Since it is all based on normal Nix derivations, the jobs can be tested
20 20 # locally with a run of "nix-build" like the following example:
21 21 #
22 22 # nix-build release.nix -A test-api -I vcsserver=~/rhodecode-vcsserver
23 23 #
24 24 # Note: Replace "~/rhodecode-vcsserver" with a path where a clone of the
25 25 # vcsserver resides.
26 26
27 27 { pkgs ? import <nixpkgs> {}
28 28 , doCheck ? true
29 29 }:
30 30
31 31 let
32 32
33 33 inherit (pkgs)
34 34 stdenv
35 35 system;
36 36
37 37 testing = import <nixpkgs/nixos/lib/testing.nix> {
38 38 inherit system;
39 39 };
40 40
41 41 runInMachine = testing.runInMachine;
42 42
43 43 sphinx = import ./docs/default.nix {};
44 44
45 45 mkDocs = kind: stdenv.mkDerivation {
46 46 name = kind;
47 47 srcs = [
48 48 (./. + (builtins.toPath "/${kind}"))
49 49 (builtins.filterSource
50 50 (path: type: baseNameOf path == "VERSION")
51 51 ./rhodecode)
52 52 ];
53 53 sourceRoot = kind;
54 54 buildInputs = [ sphinx ];
55 55 configurePhase = null;
56 56 buildPhase = ''
57 57 make SPHINXBUILD=sphinx-build html
58 58 '';
59 59 installPhase = ''
60 60 mkdir -p $out
61 61 mv _build/html $out/
62 62
63 63 mkdir -p $out/nix-support
64 64 echo "doc manual $out/html index.html" >> \
65 65 "$out/nix-support/hydra-build-products"
66 66 '';
67 67 };
68 68
69 69 enterprise = import ./default.nix {
70 70 inherit
71 71 pkgs;
72 72
73 73 # TODO: for quick local testing
74 74 doCheck = false;
75 75 };
76 76
77 77 test-cfg = stdenv.mkDerivation {
78 78 name = "test-cfg";
79 79 unpackPhase = "true";
80 80 buildInputs = [
81 81 enterprise.src
82 82 ];
83 83 installPhase = ''
84 84 mkdir -p $out/etc
85 85 cp ${enterprise.src}/test.ini $out/etc/enterprise.ini
86 86 # TODO: johbo: Needed, so that the login works, this causes
87 87 # probably some side effects
88 88 substituteInPlace $out/etc/enterprise.ini --replace "is_test = True" ""
89 89
90 90 # Gevent configuration
91 91 cp $out/etc/enterprise.ini $out/etc/enterprise-gevent.ini;
92 92 cat >> $out/etc/enterprise-gevent.ini <<EOF
93 93
94 94 [server:main]
95 95 use = egg:gunicorn#main
96 96 worker_class = gevent
97 97 EOF
98 98
99 99 cp ${enterprise.src}/vcsserver/test.ini $out/etc/vcsserver.ini
100 100 '';
101 101 };
102 102
103 103 ac-test-drv = import ./acceptance_tests {
104 104 withExternals = false;
105 105 };
106 106
107 107 # TODO: johbo: Currently abusing buildPythonPackage to make the
108 108 # needed environment for the ac-test tools.
109 109 mkAcTests = {
110 110 # Path to an INI file which will be used to run Enterprise.
111 111 #
112 112 # Intended usage is to provide different configuration files to
113 113 # run the tests against a different configuration.
114 114 enterpriseCfg ? "${test-cfg}/etc/enterprise.ini"
115 115
116 116 # Path to an INI file which will be used to run the VCSServer.
117 117 , vcsserverCfg ? "${test-cfg}/etc/vcsserver.ini"
118 118 }: pkgs.pythonPackages.buildPythonPackage {
119 119 name = "enterprise-ac-tests";
120 120 src = ./acceptance_tests;
121 121
122 122 buildInputs = with pkgs; [
123 123 curl
124 124 enterprise
125 125 ac-test-drv
126 126 ];
127 127
128 128 buildPhase = ''
129 129 cp ${enterpriseCfg} enterprise.ini
130 130
131 131 echo "Creating a fake home directory"
132 132 mkdir fake-home
133 133 export HOME=$PWD/fake-home
134 134
135 135 echo "Creating a repository directory"
136 136 mkdir repos
137 137
138 138 echo "Preparing the database"
139 paster setup-rhodecode \
139 rc-setup-app \
140 140 --user=admin \
141 141 --email=admin@example.com \
142 142 --password=secret \
143 143 --api-key=9999999999999999999999999999999999999999 \
144 144 --force-yes \
145 145 --repos=$PWD/repos \
146 146 enterprise.ini > /dev/null
147 147
148 148 echo "Starting rc-server"
149 149 vcsserver --config ${vcsserverCfg} >vcsserver.log 2>&1 &
150 150 rc-server enterprise.ini >rc-server.log 2>&1 &
151 151
152 152 while ! curl -f -s http://localhost:5000 > /dev/null
153 153 do
154 154 echo "Waiting for server to be ready..."
155 155 sleep 3
156 156 done
157 157 echo "Webserver is ready."
158 158
159 159 echo "Starting the test run"
160 160 py.test -c example.ini -vs --maxfail=5 tests
161 161
162 162 echo "Kill rc-server"
163 163 kill %2
164 164 kill %1
165 165 '';
166 166
167 167 # TODO: johbo: Use the install phase again once the normal mkDerivation
168 168 # can be used again.
169 169 postInstall = ''
170 170 mkdir -p $out
171 171 cp enterprise.ini $out
172 172 cp ${vcsserverCfg} $out/vcsserver.ini
173 173 cp rc-server.log $out
174 174 cp vcsserver.log $out
175 175
176 176 mkdir -p $out/nix-support
177 177 echo "report config $out enterprise.ini" >> $out/nix-support/hydra-build-products
178 178 echo "report config $out vcsserver.ini" >> $out/nix-support/hydra-build-products
179 179 echo "report rc-server $out rc-server.log" >> $out/nix-support/hydra-build-products
180 180 echo "report vcsserver $out vcsserver.log" >> $out/nix-support/hydra-build-products
181 181 '';
182 182 };
183 183
184 184 vcsserver = import <vcsserver> {
185 185 inherit pkgs;
186 186
187 187 # TODO: johbo: Think of a more elegant solution to this problem
188 188 pythonExternalOverrides = self: super: (enterprise.myPythonPackagesUnfix self);
189 189 };
190 190
191 191 runTests = optionString: (enterprise.override (attrs: {
192 192 doCheck = true;
193 193 name = "test-run";
194 194 buildInputs = attrs.buildInputs ++ [
195 195 vcsserver
196 196 ];
197 197 checkPhase = ''
198 198 py.test ${optionString} -vv -ra
199 199 '';
200 200 buildPhase = attrs.shellHook;
201 201 installPhase = ''
202 202 echo "Intentionally not installing anything"
203 203 '';
204 204 meta.description = "Enterprise test run ${optionString}";
205 205 }));
206 206
207 207 jobs = {
208 208
209 209 build = enterprise;
210 210
211 211 # johbo: Currently this is simply running the tests against the sources. Nicer
212 212 # would be to run xdist and against the installed application, so that we also
213 213 # cover the impact of installing the application.
214 214 test-api = runTests "rhodecode/api";
215 215 test-functional = runTests "rhodecode/tests/functional";
216 216 test-rest = runTests "rhodecode/tests --ignore=rhodecode/tests/functional";
217 217 test-full = runTests "rhodecode";
218 218
219 219 docs = mkDocs "docs";
220 220
221 221 aggregate = pkgs.releaseTools.aggregate {
222 222 name = "aggregated-jobs";
223 223 constituents = [
224 224 jobs.build
225 225 jobs.test-api
226 226 jobs.test-rest
227 227 jobs.docs
228 228 ];
229 229 };
230 230 };
231 231
232 232 in jobs
General Comments 0
You need to be logged in to leave comments. Login now