release: merge back stable branch into default

docs/release-notes/release-notes-4.11.5.rst

@@ -0,0 +1,39 b''
+|RCE| 4.11.5 |RNS|
+------------------
+
+Release Date
+^^^^^^^^^^^^
+
+- 2018-02-21
+
+
+New Features
+^^^^^^^^^^^^
+
+
+
+General
+^^^^^^^
+
+
+
+Security
+^^^^^^^^
+
+
+
+Performance
+^^^^^^^^^^^
+
+
+
+Fixes
+^^^^^
+
+- Auth: fixed a regression with external auth new users creation.
+
+
+Upgrade notes
+^^^^^^^^^^^^^
+
+- Unscheduled bugfix release.

.hgtags

@@ -1,35 +1,36 b''
 1bd3e92b7e2e2d2024152b34bb88dff1db544a71 v4.0.0
 170c5398320ea6cddd50955e88d408794c21d43a v4.0.1
 c3fe200198f5aa34cf2e4066df2881a9cefe3704 v4.1.0
 7fd5c850745e2ea821fb4406af5f4bff9b0a7526 v4.1.1
 41c87da28a179953df86061d817bc35533c66dd2 v4.1.2
 baaf9f5bcea3bae0ef12ae20c8b270482e62abb6 v4.2.0
 32a70c7e56844a825f61df496ee5eaf8c3c4e189 v4.2.1
 fa695cdb411d294679ac081d595ac654e5613b03 v4.3.0
 0e4dc11b58cad833c513fe17bac39e6850edf959 v4.3.1
 8a876f48f5cb1d018b837db28ff928500cb32cfb v4.4.0
 8dd86b410b1aac086ffdfc524ef300f896af5047 v4.4.1
 d2514226abc8d3b4f6fb57765f47d1b6fb360a05 v4.4.2
 27d783325930af6dad2741476c0d0b1b7c8415c2 v4.5.0
 7f2016f352abcbdba4a19d4039c386e9629449da v4.5.1
 416fec799314c70a5c780fb28b3357b08869333a v4.5.2
 27c3b85fafc83143e6678fbc3da69e1615bcac55 v4.6.0
 5ad13deb9118c2a5243d4032d4d9cc174e5872db v4.6.1
 2be921e01fa24bb102696ada596f87464c3666f6 v4.7.0
 7198bdec29c2872c974431d55200d0398354cdb1 v4.7.1
 bd1c8d230fe741c2dfd7100a0ef39fd0774fd581 v4.7.2
 9731914f89765d9628dc4dddc84bc9402aa124c8 v4.8.0
 c5a2b7d0e4bbdebc4a62d7b624befe375207b659 v4.9.0
 d9aa3b27ac9f7e78359775c75fedf7bfece232f1 v4.9.1
 4ba4d74981cec5d6b28b158f875a2540952c2f74 v4.10.0
 0a6821cbd6b0b3c21503002f88800679fa35ab63 v4.10.1
 434ad90ec8d621f4416074b84f6e9ce03964defb v4.10.2
 68baee10e698da2724c6e0f698c03a6abb993bf2 v4.10.3
 00821d3afd1dce3f4767cc353f84a17f7d5218a1 v4.10.4
 22f6744ad8cc274311825f63f953e4dee2ea5cb9 v4.10.5
 96eb24bea2f5f9258775245e3f09f6fa0a4dda01 v4.10.6
 3121217a812c956d7dd5a5875821bd73e8002a32 v4.11.0
 fa98b454715ac5b912f39e84af54345909a2a805 v4.11.1
 3982abcfdcc229a723cebe52d3a9bcff10bba08e v4.11.2
 33195f145db9172f0a8f1487e09207178a6ab065 v4.11.3
 194c74f33e32bbae6fc4d71ec5a999cff3c13605 v4.11.4
+8fbd8b0c3ddc2fa4ac9e4ca16942a03eb593df2d v4.11.5

docs/contributing/db-schema.rst

@@ -1,52 +1,52 b''
 =======================
 DB Schema and Migration
 =======================
 
 To create or alter tables in the database, it's necessary to change a couple of
 files, apart from configuring the settings pointing to the latest database
 schema.
 
 
 Database Model and ORM
 ----------------------
 
 On ``rhodecode.model.db`` you will find the database definition of all tables and 
 fields. Any freshly installed database will be correctly created by the definitions 
 here. So, any change to this file will affect the tests without having to change
 any other file.
 
 A second layer are the business classes inside ``rhodecode.model``. 
 
 
 Database Migration
 ------------------
 
 Three files play a role when creating database migrations:
 
     * Database schema inside ``rhodecode.lib.dbmigrate``
     * Database version inside ``rhodecode.lib.dbmigrate`` 
     * Configuration ``__dbversion__`` at ``rhodecode.__init__``
 
 
 Schema is a snapshot of the database version BEFORE the migration. So, it's
 the initial state before any changes were added. The name convention is
 the latest release version where the snapshot was created, and not the 
 target version of this code.
 
 Version is the method that will define how to UPGRADE/DOWNGRADE the database.
 
 ``rhodecode.__init__`` contains only a variable that defines up to which version of 
 the database will be used to upgrade. Eg.: ``__dbversion__ = 45``
 
 
 For examples on how to create those files, please see the existing code.
 
 
 Migration Command
 ^^^^^^^^^^^^^^^^^
 
 After you've changed the database ORM and migration files, you can run::
 
-   paster upgrade-db <ini-file> 
+   rc-upgrade-db <ini-file>
 
 The database will be upgraded up to the version defined in the ``__init__`` file.
  No newline at end of file

docs/contributing/dev-setup.rst

@@ -1,226 +1,226 b''
 .. _dev-setup:
 
 ===================
  Development setup
 ===================
 
 
 RhodeCode Enterprise runs inside a Nix managed environment. This ensures build
 environment dependencies are correctly declared and installed during setup.
 It also enables atomic upgrades, rollbacks, and multiple instances of RhodeCode
 Enterprise running with isolation.
 
 To set up RhodeCode Enterprise inside the Nix environment, use the following steps:
 
 
 
 Setup Nix Package Manager
 -------------------------
 
 To install the Nix Package Manager, please run::
 
    $ curl https://nixos.org/nix/install | sh
 
 or go to https://nixos.org/nix/ and follow the installation instructions.
 Once this is correctly set up on your system, you should be able to use the
 following commands:
 
 * `nix-env`
 
 * `nix-shell`
 
 
 .. tip::
 
    Update your channels frequently by running ``nix-channel --update``.
 
 .. note::
 
    To uninstall nix run the following:
 
     remove the . "$HOME/.nix-profile/etc/profile.d/nix.sh" line in your ~/.profile or ~/.bash_profile
     rm -rf $HOME/{.nix-channels,.nix-defexpr,.nix-profile,.config/nixpkgs}
     sudo rm -rf /nix
 
 Switch nix to the latest STABLE channel
 ---------------------------------------
 
 run::
 
    nix-channel --add https://nixos.org/channels/nixos-16.03 nixpkgs
 
 Followed by::
 
    nix-channel --update
 
 
 Install required binaries
 -------------------------
 
 We need some handy tools first.
 
 run::
 
     nix-env -i nix-prefetch-hg
     nix-env -i nix-prefetch-git
 
 
 Clone the required repositories
 -------------------------------
 
 After Nix is set up, clone the RhodeCode Enterprise Community Edition and
 RhodeCode VCSServer repositories into the same directory.
 RhodeCode currently is using Mercurial Version Control System, please make sure
 you have it installed before continuing.
 
 To obtain the required sources, use the following commands::
 
     mkdir rhodecode-develop && cd rhodecode-develop
     hg clone https://code.rhodecode.com/rhodecode-enterprise-ce
     hg clone https://code.rhodecode.com/rhodecode-vcsserver
 
 .. note::
 
    If you cannot clone the repository, please contact us via [e-mail]
 
 
 Install some required libraries
 -------------------------------
 
 There are some required drivers and dev libraries that we need to install to
 test RhodeCode under different types of databases. For example in Ubuntu we
 need to install the following.
 
 required libraries::
 
     sudo apt-get install libapr1-dev libaprutil1-dev
     sudo apt-get install libsvn-dev
     sudo apt-get install mysql-server libmysqlclient-dev
     sudo apt-get install postgresql postgresql-contrib libpq-dev
     sudo apt-get install libcurl4-openssl-dev
 
 
 Enter the Development Shell
 ---------------------------
 
 The final step is to start the development shells. To do this, run the
 following command from inside the cloned repository::
 
    #first, the vcsserver
    cd ~/rhodecode-vcsserver
    nix-shell
 
    # then enterprise sources
    cd ~/rhodecode-enterprise-ce
    nix-shell
 
 .. note::
 
    On the first run, this will take a while to download and optionally compile
    a few things. The following runs will be faster. The development shell works
    fine on both MacOS and Linux platforms.
 
 
 Create config.nix for development
 ---------------------------------
 
 In order to run proper tests and setup linking across projects, a config.nix
 file needs to be setup::
 
     # create config
     mkdir -p ~/.nixpkgs
     touch ~/.nixpkgs/config.nix
 
     # put the below content into the ~/.nixpkgs/config.nix file
     # adjusts, the path to where you cloned your repositories.
 
     {
       rc = {
        sources = {
         rhodecode-vcsserver = "/home/dev/rhodecode-vcsserver";
         rhodecode-enterprise-ce = "/home/dev/rhodecode-enterprise-ce";
         rhodecode-enterprise-ee = "/home/dev/rhodecode-enterprise-ee";
        };
       };
     }
 
 
 
 Creating a Development Configuration
 ------------------------------------
 
 To create a development environment for RhodeCode Enterprise,
 use the following steps:
 
 1. Create a copy of vcsserver config:
     `cp ~/rhodecode-vcsserver/configs/development.ini ~/rhodecode-vcsserver/configs/dev.ini`
 2. Create a copy of rhodocode config:
     `cp ~/rhodecode-enterprise-ce/configs/development.ini ~/rhodecode-enterprise-ce/configs/dev.ini`
 3. Adjust the configuration settings to your needs if needed.
 
 .. note::
 
   It is recommended to use the name `dev.ini` since it's included in .hgignore file.
 
 
 Setup the Development Database
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To create a development database, use the following example. This is a one
 time operation executed from the nix-shell of rhodecode-enterprise-ce sources ::
 
-    paster setup-rhodecode dev.ini \
+    rc-setup-app dev.ini \
         --user=admin --password=secret \
         --email=[e-mail] \
         --repos=~/my_dev_repos
 
 
 Compile CSS and JavaScript
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To use the application's frontend and prepare it for production deployment,
 you will need to compile the CSS and JavaScript with Grunt.
 This is easily done from within the nix-shell using the following command::
 
     grunt
 
 When developing new features you will need to recompile following any
 changes made to the CSS or JavaScript files when developing the code::
 
     grunt watch
 
 This prepares the development (with comments/whitespace) versions of files.
 
 Start the Development Servers
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 From the rhodecode-vcsserver directory, start the development server in another
 nix-shell, using the following command::
 
       pserve configs/dev.ini
 
 In the adjacent nix-shell which you created for your development server, you may
 now start CE with the following command::
 
 
       pserve --reload configs/dev.ini
 
 .. note::
 
   `--reload` flag will automatically reload the server when source file changes.
 
 
 Run the Environment Tests
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Please make sure that the tests are passing to verify that your environment is
 set up correctly. RhodeCode uses py.test to run tests.
 While your instance is running, start a new nix-shell and simply run
 ``make test`` to run the basic test suite.
 
 
 Need Help?
 ^^^^^^^^^^
 
 Join us on Slack via https://rhodecode.com/join or post questions in our
 Community Portal at https://community.rhodecode.com

docs/release-notes/release-notes.rst

@@ -1,112 +1,113 b''
 .. _rhodecode-release-notes-ref:
 
 Release Notes
 =============
 
 |RCE| 4.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
+   release-notes-4.11.5.rst
    release-notes-4.11.4.rst
    release-notes-4.11.3.rst
    release-notes-4.11.2.rst
    release-notes-4.11.1.rst
    release-notes-4.11.0.rst
    release-notes-4.10.6.rst
    release-notes-4.10.5.rst
    release-notes-4.10.4.rst
    release-notes-4.10.3.rst
    release-notes-4.10.2.rst
    release-notes-4.10.1.rst
    release-notes-4.10.0.rst
    release-notes-4.9.1.rst
    release-notes-4.9.0.rst
    release-notes-4.8.0.rst
    release-notes-4.7.2.rst
    release-notes-4.7.1.rst
    release-notes-4.7.0.rst
    release-notes-4.6.1.rst
    release-notes-4.6.0.rst
    release-notes-4.5.2.rst
    release-notes-4.5.1.rst
    release-notes-4.5.0.rst
    release-notes-4.4.2.rst
    release-notes-4.4.1.rst
    release-notes-4.4.0.rst
    release-notes-4.3.1.rst
    release-notes-4.3.0.rst
    release-notes-4.2.1.rst
    release-notes-4.2.0.rst
    release-notes-4.1.2.rst
    release-notes-4.1.1.rst
    release-notes-4.1.0.rst
    release-notes-4.0.1.rst
    release-notes-4.0.0.rst
 
 |RCE| 3.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-3.8.4.rst
    release-notes-3.8.3.rst
    release-notes-3.8.2.rst
    release-notes-3.8.1.rst
    release-notes-3.8.0.rst
    release-notes-3.7.1.rst
    release-notes-3.7.0.rst
    release-notes-3.6.1.rst
    release-notes-3.6.0.rst
    release-notes-3.5.2.rst
    release-notes-3.5.1.rst
    release-notes-3.5.0.rst
    release-notes-3.4.1.rst
    release-notes-3.4.0.rst
    release-notes-3.3.4.rst
    release-notes-3.3.3.rst
    release-notes-3.3.2.rst
    release-notes-3.3.1.rst
    release-notes-3.3.0.rst
    release-notes-3.2.3.rst
    release-notes-3.2.2.rst
    release-notes-3.2.1.rst
    release-notes-3.2.0.rst
    release-notes-3.1.1.rst
    release-notes-3.1.0.rst
    release-notes-3.0.2.rst
    release-notes-3.0.1.rst
    release-notes-3.0.0.rst
 
 |RCE| 2.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-2.2.8.rst
    release-notes-2.2.7.rst
    release-notes-2.2.6.rst
    release-notes-2.2.5.rst
    release-notes-2.2.4.rst
    release-notes-2.2.3.rst
    release-notes-2.2.2.rst
    release-notes-2.2.1.rst
    release-notes-2.2.0.rst
    release-notes-2.1.0.rst
    release-notes-2.0.2.rst
    release-notes-2.0.1.rst
    release-notes-2.0.0.rst
 
 |RCE| 1.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-1.7.2.rst
    release-notes-1.7.1.rst
    release-notes-1.7.0.rst
    release-notes-1.6.0.rst

release.nix

@@ -1,232 +1,232 b''
 #
 # About
 # =====
 #
 # This file defines jobs for our CI system and the attribute "build" is used
 # as the input for packaging.
 #
 #
 # CI details
 # ==========
 #
 # This file defines an attribute set of derivations. Each of these attributes is
 # then used in our CI system as one job to run. This way we keep the
 # configuration for the CI jobs as well under version control.
 #
 # Run CI jobs locally
 # -------------------
 #
 # Since it is all based on normal Nix derivations, the jobs can be tested
 # locally with a run of "nix-build" like the following example:
 #
 #    nix-build release.nix -A test-api -I vcsserver=~/rhodecode-vcsserver
 #
 # Note: Replace "~/rhodecode-vcsserver" with a path where a clone of the
 # vcsserver resides.
 
 { pkgs ? import <nixpkgs> {}
 , doCheck ? true
 }:
 
 let
 
   inherit (pkgs)
     stdenv
     system;
 
   testing = import <nixpkgs/nixos/lib/testing.nix> {
     inherit system;
   };
 
   runInMachine = testing.runInMachine;
 
   sphinx = import ./docs/default.nix {};
 
   mkDocs = kind: stdenv.mkDerivation {
     name = kind;
     srcs = [
       (./. + (builtins.toPath "/${kind}"))
       (builtins.filterSource
         (path: type: baseNameOf path == "VERSION")
         ./rhodecode)
     ];
     sourceRoot = kind;
     buildInputs = [ sphinx ];
     configurePhase = null;
     buildPhase = ''
       make SPHINXBUILD=sphinx-build html
     '';
     installPhase = ''
       mkdir -p $out
       mv _build/html $out/
 
       mkdir -p $out/nix-support
       echo "doc manual $out/html index.html" >> \
         "$out/nix-support/hydra-build-products"
     '';
   };
 
   enterprise = import ./default.nix {
     inherit
       pkgs;
 
     # TODO: for quick local testing
     doCheck = false;
   };
 
   test-cfg = stdenv.mkDerivation {
     name = "test-cfg";
     unpackPhase = "true";
     buildInputs = [
       enterprise.src
     ];
     installPhase = ''
       mkdir -p $out/etc
       cp ${enterprise.src}/test.ini $out/etc/enterprise.ini
       # TODO: johbo: Needed, so that the login works, this causes
       # probably some side effects
       substituteInPlace $out/etc/enterprise.ini --replace "is_test = True" ""
 
       # Gevent configuration
       cp $out/etc/enterprise.ini $out/etc/enterprise-gevent.ini;
       cat >> $out/etc/enterprise-gevent.ini <<EOF
 
       [server:main]
       use = egg:gunicorn#main
       worker_class = gevent
       EOF
 
       cp ${enterprise.src}/vcsserver/test.ini $out/etc/vcsserver.ini
     '';
   };
 
   ac-test-drv = import ./acceptance_tests {
     withExternals = false;
   };
 
   # TODO: johbo: Currently abusing buildPythonPackage to make the
   # needed environment for the ac-test tools.
   mkAcTests = {
     # Path to an INI file which will be used to run Enterprise.
     #
     # Intended usage is to provide different configuration files to
     # run the tests against a different configuration.
     enterpriseCfg ? "${test-cfg}/etc/enterprise.ini"
 
     # Path to an INI file which will be used to run the VCSServer.
   , vcsserverCfg ? "${test-cfg}/etc/vcsserver.ini"
   }: pkgs.pythonPackages.buildPythonPackage {
     name = "enterprise-ac-tests";
     src = ./acceptance_tests;
 
     buildInputs = with pkgs; [
       curl
       enterprise
       ac-test-drv
     ];
 
     buildPhase = ''
       cp ${enterpriseCfg} enterprise.ini
 
       echo "Creating a fake home directory"
       mkdir fake-home
       export HOME=$PWD/fake-home
 
       echo "Creating a repository directory"
       mkdir repos
 
       echo "Preparing the database"
-      paster setup-rhodecode \
+      rc-setup-app \
         --user=admin \
         --email=[e-mail] \
         --password=secret \
         --api-key=9999999999999999999999999999999999999999 \
         --force-yes \
         --repos=$PWD/repos \
         enterprise.ini > /dev/null
 
       echo "Starting rc-server"
       vcsserver --config ${vcsserverCfg} >vcsserver.log 2>&1 &
       rc-server enterprise.ini >rc-server.log 2>&1 &
 
       while ! curl -f -s http://localhost:5000 > /dev/null
       do
           echo "Waiting for server to be ready..."
           sleep 3
       done
       echo "Webserver is ready."
 
       echo "Starting the test run"
       py.test -c example.ini -vs --maxfail=5 tests
 
       echo "Kill rc-server"
       kill %2
       kill %1
     '';
 
     # TODO: johbo: Use the install phase again once the normal mkDerivation
     # can be used again.
     postInstall = ''
       mkdir -p $out
       cp enterprise.ini $out
       cp ${vcsserverCfg} $out/vcsserver.ini
       cp rc-server.log $out
       cp vcsserver.log $out
 
       mkdir -p $out/nix-support
       echo "report config $out enterprise.ini" >> $out/nix-support/hydra-build-products
       echo "report config $out vcsserver.ini" >> $out/nix-support/hydra-build-products
       echo "report rc-server $out rc-server.log" >> $out/nix-support/hydra-build-products
       echo "report vcsserver $out vcsserver.log" >> $out/nix-support/hydra-build-products
     '';
   };
 
   vcsserver = import <vcsserver> {
     inherit pkgs;
 
     # TODO: johbo: Think of a more elegant solution to this problem
     pythonExternalOverrides = self: super: (enterprise.myPythonPackagesUnfix self);
   };
 
   runTests = optionString: (enterprise.override (attrs: {
     doCheck = true;
     name = "test-run";
     buildInputs = attrs.buildInputs ++ [
       vcsserver
     ];
     checkPhase = ''
       py.test ${optionString} -vv -ra
     '';
     buildPhase = attrs.shellHook;
     installPhase = ''
       echo "Intentionally not installing anything"
     '';
     meta.description = "Enterprise test run ${optionString}";
   }));
 
   jobs = {
 
     build = enterprise;
 
     # johbo: Currently this is simply running the tests against the sources. Nicer
     # would be to run xdist and against the installed application, so that we also
     # cover the impact of installing the application.
     test-api = runTests "rhodecode/api";
     test-functional = runTests "rhodecode/tests/functional";
     test-rest = runTests "rhodecode/tests --ignore=rhodecode/tests/functional";
     test-full = runTests "rhodecode";
 
     docs = mkDocs "docs";
 
     aggregate = pkgs.releaseTools.aggregate {
       name = "aggregated-jobs";
       constituents = [
         jobs.build
         jobs.test-api
         jobs.test-rest
         jobs.docs
       ];
     };
   };
 
 in jobs