release: Merge default into stable for release preparation

docs/admin/system_admin/tuning/tuning-rhodecode-memory-usage.rst

@@ -0,0 +1,26 b''
+.. _adjust-rhodecode-mem:
+
+RhodeCode Memory Usage
+----------------------
+
+Starting from Version 4.18.X RhodeCode has a builtin memory monitor for gunicorn workers.
+Enabling this can limit the maximum amount of memory system can use. Each worker
+for RhodeCode is monitored independently.
+To enable Memory management make sure to have following settings inside `[app:main] section` of
+:file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
+
+
+
+    ; Maximum memory usage that each worker can use before it will receive a
+    ; graceful restart signal 0 = memory monitoring is disabled
+    ; Examples: 268435456 (256MB), 536870912 (512MB)
+    ; 1073741824 (1GB), 2147483648 (2GB), 4294967296 (4GB)
+    memory_max_usage = 1073741824
+
+    ; How often in seconds to check for memory usage for each gunicorn worker
+    memory_usage_check_interval = 60
+
+    ; Threshold value for which we don't recycle worker if GarbageCollection
+    ; frees up enough resources. Before each restart we try to run GC on worker
+    ; in case we get enough free memory after that, restart will not happen.
+    memory_usage_recovery_threshold = 0.8

docs/release-notes/release-notes-4.18.0.rst

@@ -0,0 +1,230 b''
+|RCE| 4.18.0 |RNS|
+------------------
+
+Release Date
+^^^^^^^^^^^^
+
+- 2020-01-05
+
+
+New Features
+^^^^^^^^^^^^
+
+- Artifacts: are no longer in BETA. New info page is available for uploaded artifacts
+  which exposes some useful information like sha256, various access urls etc, and also
+  allows deletion of artifacts, and updating their description.
+- Artifacts: support new download url based on access to artifacts using new auth-token types.
+- Artifacts: added ability to store artifacts using API, and internal cli upload.
+  This allows uploading of artifacts that can have 100s of GBs in size efficiently.
+- Artifacts: added metadata logic to store various extra custom data for artifacts.
+- Comments: added support for adding comment attachments using the artifacts logic.
+  Logged in users can now pick or drag and drop attachments into comment forms.
+- Comments: enable linkification of certain patterns on comments in repo/pull request scopes.
+  This will render now active links to commits, pull-requests mentioned in comments body.
+- Jira: new update integration plugin.
+  Plugin now fetches possible transitions from tickets and show them to users in the interface.
+  Allow sending extra attributes during a transition like `resolution` message.
+- Navigation: Added new consistent and contextual way of creating new objects
+  likes gists, repositories, and repository groups using dedicated action (with a `+` sign)
+  available in the top navigation.
+- Hovercards: added new tooltips and hovercards to expose certain information for objects shown in UI.
+  RhodeCode usernames, issues, pull-requests will have active hovercard logic that will
+  load extra information about them and exposing them to users.
+- Files: all readme files found in repository file browser will be now rendered, allowing having readme per directory.
+- Search: expose line counts in search files information.
+- Audit-logs: expose download user audit logs as JSON file.
+- Users: added description field for users.
+  Allows users to write a short BIO, or description of their role in the organization.
+- Users: allow super-admins to change bound authentication type for users.
+  E.g internal rhodecode accounts can be changed to ldap easily from user settings page.
+- Pull requests: simplified the UI for display view, hide less important information and expose the most important ones.
+- Pull requests: add merge check that detects WIP marker in title.
+  Usually WIP in title means unfinished task that needs still some work, such marker will prevent accidental merges.
+- Pull requests: TODO comments have now a dedicated box below reviewers to keep track
+  of important TODOs that still need attention before review process is finalized.
+- Pull requests: participants of pull request will receive an email about update of a
+  pull requests with a small summary of changes made.
+- Pull requests: change the naming from #NUM into !NUM.
+  !NUM format is now parsed and linkified in comments and commit messages.
+- Pull requests: pull requests which state is changing can now be viewed with a limited view.
+- Pull requests: re-organize merge/close buttons and merge checks according to the new UI.
+- Pull requests: update commits button allows a force-refresh update now using dropdown option.
+- Pull requests: added quick filter to grid view to filter/search pull requests in a repository.
+- Pull requests: closing a pull-request without a merge requires additional confirmation now.
+- Pull requests: merge checks will now show which files caused conflicts and are blocking the merge.
+- Emails: updated all generated emails design and cleanup the data fields they expose.
+  a) More consistent UI for all types of emails. b) Improved formatting of plaintext emails
+  c) Added reply link to comment type emails for quicker response action.
+
+
+General
+^^^^^^^
+
+- Artifacts: don't show hidden artifacts, allow showing them via a GET ?hidden=1 flag.
+  Hidden artifacts are for example comment attachments.
+- UI: new commits page, according to the new design, which started on 4.17.X release lines
+- UI: use explicit named actions like "create user" instead of generic "save" which is bad UX.
+- UI: fixed problems with generating last change in repository groups.
+  There's now a new logic that checks all objects inside group for latest update time.
+- API: add artifact `get_info`, and `store_metadata` methods.
+- API: allowed to specify extra recipients for pr/commit comments api methods.
+- Vcsserver: set file based cache as default for vcsserver which can be shared
+  across multiple workers saving memory usage.
+- Vcsserver: added redis as possible cache backend for even greater performance.
+- Dependencies: bumped GIT version to 2.23.0
+- Dependencies: bumped SVN version to 1.12.2
+- Dependencies: bumped Mercurial version to 5.1.1 and hg-evolve to 9.1.0
+- Search: added logic for sorting ElasticSearch6 backend search results.
+- User bookmarks: make it easier to re-organize existing entries.
+- Data grids: hide pagination for single pages in grids.
+- Gists: UX, removed private/public gist buttons and replaced them with radio group.
+- Gunicorn: moved all configuration of gunicorn workers to .ini files.
+- Gunicorn: added worker memory management allowing setting maximum per-worker memory usage.
+- Automation: moved update groups task into celery task
+- Cache commits: add option to refresh caches manually from advanced pages.
+- Pull requests: add indication of state change in list of pull-requests and actually show them in the list.
+- Cache keys: register and self cleanup cache keys used for invalidation to prevent leaking lot of them into DB on worker recycle
+- Repo groups: removed locking inheritance flag from repo-groups. We'll deprecate this soon and this only brings in confusion
+- System snapshot: improved formatting for better readability
+- System info: expose data about vcsserver.
+- Packages: updated celery to 4.3.0 and switch default backend to redis instead of RabbitMQ.
+  Redis is stable enough and easier to install. Having Redis simplifies the stack as it's used in other parts of RhodeCode.
+- Dependencies: bumped alembic to 1.2.1
+- Dependencies: bumped amqp==2.5.2 and kombu==4.6.6
+- Dependencies: bumped atomicwrites==1.3.0
+- Dependencies: bumped cffi==1.12.3
+- Dependencies: bumped configparser==4.0.2
+- Dependencies: bumped deform==2.0.8
+- Dependencies: bumped dogpile.cache==0.9.0
+- Dependencies: bumped hupper==1.8.1
+- Dependencies: bumped mako to 1.1.0
+- Dependencies: bumped markupsafe to 1.1.1
+- Dependencies: bumped packaging==19.2
+- Dependencies: bumped paste==3.2.1
+- Dependencies: bumped pastescript==3.2.0
+- Dependencies: bumped pathlib2 to 2.3.4
+- Dependencies: bumped pluggy==0.13.0
+- Dependencies: bumped psutil to 5.6.3
+- Dependencies: bumped psutil==5.6.5
+- Dependencies: bumped psycopg2==2.8.4
+- Dependencies: bumped pycurl to 7.43.0.3
+- Dependencies: bumped pyotp==2.3.0
+- Dependencies: bumped pyparsing to 2.4.2
+- Dependencies: bumped pyramid-debugtoolbar==4.5.1
+- Dependencies: bumped pyramid-mako to 1.1.0
+- Dependencies: bumped redis to 3.3.8
+- Dependencies: bumped sqlalchemy to 1.3.8
+- Dependencies: bumped sqlalchemy==1.3.11
+- Dependencies: bumped test libraries.
+- Dependencies: freeze alembic==1.3.1
+- Dependencies: freeze python-dateutil
+- Dependencies: freeze redis==3.3.11
+- Dependencies: freeze supervisor==4.1.0
+
+
+Security
+^^^^^^^^
+
+- Security: fixed issues with exposing wrong http status (403) indicating repository with
+  given name exists and we don't have permissions to it. This was exposed in the redirection
+  logic of the global pull-request page. In case of redirection we also exposed
+  repository name in the URL.
+
+
+Performance
+^^^^^^^^^^^
+
+- Core: many various small improvements and optimizations to make rhodecode faster then before.
+- VCSServer: new cache implementation for remote functions.
+  Single worker shared caches that can use redis/file-cache.
+  This greatly improves performance on larger instances, and doesn't trigger cache
+  re-calculation on worker restarts.
+- GIT: switched internal git operations from Dulwich to libgit2 in order to obtain better performance and scalability.
+- SSH: skip loading unneeded application parts for SSH to make execution of ssh commands faster.
+- Main page: main page will now load repositories and repositories groups using partial DB calls instead of big JSON files.
+  In case of many repositories in root this could lead to very slow page rendering.
+- Admin pages: made all grids use same DB based partial loading logic. We'll no longer fetch
+  all objects into JSON for display purposes. This significantly improves speed of those pages in case
+  of many objects shown in them.
+- Summary page: use non-memory cache for readme, and cleanup cache for repo stats.
+  This change won't re-cache after worker restarts and can be shared across all workers
+- Files: only check for git_lfs/hg_largefiles if they are enabled.
+  This speeds up fetching of files if they are not LF and very big.
+- Vcsserver: added support for streaming data from the remote methods. This allows
+  to stream very large files without taking up memory, mostly for usage in SVN when
+  downloading large binaries from vcs system.
+- Files: added streaming remote attributes for vcsserver.
+  This change enables streaming raw content or raw downloads of large files without
+  transferring them over to enterprise for pack & repack using msgpack.
+  Msgpack has a limit of 2gb and generally pack+repack for ~2gb is very slow.
+- Files: ensure over size limit files never do any content fetching when viewing such files.
+- VCSServer: skip host verification to speed up pycurl calls.
+- User-bookmarks: cache fetching of bookmarks since this is quite expensive query to
+  make with joinedload on repos/repo groups.
+- Goto-switcher: reduce query data to only required attributes for speedups.
+- My account: owner/watched repos are now loaded only using DB queries.
+
+
+Fixes
+^^^^^
+
+- Mercurial: move imports from top-level to prevent from loading mercurial code on hook execution for svn/git.
+- GIT: limit sync-fetch logic to only retrieve tags/ and heads/ with default execution arguments.
+- GIT: fixed issue with git submodules detection.
+- SVN: fix checkout url for ssh+svn backend not having special prefix resulting in incorrect command shown.
+- SVN: fixed problem with showing empty directories.
+- OAuth: use a vendored version of `authomatic` library, and switch Bitbucket authentication to use oauth2.
+- Diffs: handle paths with quotes in diffs.
+- Diffs: fixed outdated files in pull-requests re-using the filediff raw_id for anchor generation. Fixes #5567
+- Diffs: toggle race condition on sticky vs wide-diff-mode that caused some display problems on larger diffs.
+- Pull requests: handle exceptions in state change and improve logging.
+- Pull requests: fixed title/description generation for single commits which are numbers.
+- Pull requests: changed the source of changes to be using shadow repos if it exists.
+  In case of `git push -f` and rebase we lost commits in the repo resulting in
+  problems of displaying versions of pull-requests.
+- Pull requests: handle case when removing existing files from a repository in compare versions diff.
+- Files: don't expose copy content helper in case of binary files.
+- Registration: properly expose first_name/last_name into email on user registration.
+- Markup renderers: fixed broken code highlight for rst files.
+- Ui: make super admin be named consistently across ui.
+- Audit logs: fixed search cases with special chars such as `-`.
+
+
+Upgrade notes
+^^^^^^^^^^^^^
+
+- New Automation task. We've changed the logic for updating latest change inside repository group.
+  New logic includes scanning for changes in all nested objects. Since this is a heavy task
+  a new dedicated scheduler task has been created to update it automatically on a scheduled base.
+  Please review in `admin > settings > automation` to enable this task.
+
+- New safer encryption algorithm. Some setting values are encrypted before storing it inside the database.
+  To keep full backward compatibility old AES algorithm is used.
+  If you wish to enable a safer option set fernet encryption instead inside rhodecode.ini
+  `rhodecode.encrypted_values.algorithm = fernet`
+
+- Pull requests UI changes. We've simplified the UI on pull requests page.
+  Please review the new UI to prevent surprises. All actions from old UI should be still possible with the new one.
+
+- Redis is now a default recommended backend for Celery and replaces previous rabbitmq.
+  Redis is generally easier to manage and install, and it's also very stable for usage
+  in the scheduler/celery async tasks. Since we also recommend Redis for caches the application
+  stack can be simplified by removing rabbitmq and replacing it with single Redis instance.
+
+- Recommendation for using Redis as the new cache backend on vcsserver.
+  Since Version 4.18.0 VCSServer has a new cache implementation for VCS data.
+  By default, for simplicity the cache type is file based. We strongly recommend using
+  Redis instead for better Performance and scalability
+  Please review vcsserver.ini settings under:
+  `rc_cache.repo_object.backend = dogpile.cache.rc.redis_msgpack`
+
+- New memory monitoring for Gunicorn workers. Starting from 4.18 release a option was added
+  to limit the maximum amount of memory used by a worker.
+  Please review new settings in `[server:main]` section for memory management in both
+  rhodecode.ini and vcsserver.ini::
+
+    ; Maximum memory usage that each worker can use before it will receive a
+    ; graceful restart signal 0 = memory monitoring is disabled
+    ; Examples: 268435456 (256MB), 536870912 (512MB)
+    ; 1073741824 (1GB), 2147483648 (2GB), 4294967296 (4GB)
+    memory_max_usage = 0

.bumpversion.cfg

@@ -1,6 +1,6 b''
 [bumpversion]
-current_version = 4.17.4
+current_version = 4.18.0
 message = release: Bump version {current_version} to {new_version}
 
 [bumpversion:file:rhodecode/VERSION]
 

.coveragerc

@@ -1,16 +1,17 b''
 [run]
 
 branch = True
 
 include =
     rhodecode/*
 
 omit =
     rhodecode/lib/dbmigrate/*
     rhodecode/lib/paster_commands/*
+    rhodecode/lib/_vendor/*
 
 [report]
 
 exclude_lines =
     raise NotImplementedError
 

.hgignore

@@ -1,67 +1,68 b''
 syntax: glob
 *.egg
 *.egg-info
 *.idea
 *.orig
 *.pyc
 *.sqlite-journal
 *.swp
 *.tox
 *.DS_Store*
 rhodecode/public/js/src/components/**/*.css
 
 syntax: regexp
 
 #.filename
 ^\.settings$
 ^\.project$
 ^\.pydevproject$
 ^\.coverage$
 ^\.cache.*$
 ^\.rhodecode$
 
 ^rcextensions
 ^.dev
 ^._dev
 ^build/
 ^coverage\.xml$
 ^data$
 ^\.eggs/
 ^configs/data$
 ^dev.ini$
 ^acceptance_tests/dev.*\.ini$
 ^dist/
 ^fabfile.py
 ^htmlcov
 ^junit\.xml$
 ^node_modules/
 ^node_binaries/
 ^pylint.log$
 ^rcextensions/
 ^result$
 ^rhodecode/public/css/style.css$
 ^rhodecode/public/css/style-polymer.css$
 ^rhodecode/public/css/style-ipython.css$
 ^rhodecode/public/js/rhodecode-components.html$
 ^rhodecode/public/js/rhodecode-components.js$
 ^rhodecode/public/js/scripts.js$
+^rhodecode/public/js/scripts.min.js$
 ^rhodecode/public/js/src/components/root-styles.gen.html$
 ^rhodecode/public/js/vendors/webcomponentsjs/
 ^rhodecode\.db$
 ^rhodecode\.log$
 ^rhodecode_dev\.log$
 ^test\.db$
 
 # ac-tests
 ^acceptance_tests/\.cache.*$
 ^acceptance_tests/externals
 ^acceptance_tests/ghostdriver.log$
 ^acceptance_tests/local(_.+)?\.ini$
 
 # docs
 ^docs/_build$
 ^docs/result$
 ^docs-internal/_build$
 
 # Cythonized things
 ^rhodecode/.*\.(c|so)$

.release.cfg

@@ -1,33 +1,28 b''
 [DEFAULT]
 done = false
 
 [task:bump_version]
 done = true
 
 [task:rc_tools_pinned]
-done = true
 
 [task:fixes_on_stable]
-done = true
 
 [task:pip2nix_generated]
-done = true
 
 [task:changelog_updated]
-done = true
 
 [task:generate_api_docs]
-done = true
+
+[task:updated_translation]
 
 [release]
-state = prepared
-version = 4.17.4
-
-[task:updated_translation]
+state = in_progress
+version = 4.18.0
 
 [task:generate_js_routes]
 
 [task:updated_trial_license]
 
 [task:generate_oss_licenses]
 

Gruntfile.js

@@ -1,21 +1,22 b''
 var gruntConfig = require('./grunt_config.json');
 var webpackConfig = require('./webpack.config');
 gruntConfig["webpack"] = {
     options: {
         stats: !process.env.NODE_ENV || process.env.NODE_ENV === 'development'
     },
     prod: webpackConfig,
     dev: Object.assign({ watch: false }, webpackConfig)
 };
 
 module.exports = function(grunt) {
   grunt.initConfig(gruntConfig);
 
   grunt.loadNpmTasks('grunt-contrib-less');
   grunt.loadNpmTasks('grunt-contrib-concat');
+  grunt.loadNpmTasks('grunt-contrib-uglify');
   grunt.loadNpmTasks('grunt-contrib-watch');
   grunt.loadNpmTasks('grunt-contrib-jshint');
   grunt.loadNpmTasks('grunt-contrib-copy');
   grunt.loadNpmTasks('grunt-webpack');
-  grunt.registerTask('default', ['less:production', 'less:components', 'copy', 'webpack', 'concat:dist']);
+  grunt.registerTask('default', ['less:production', 'less:components', 'copy', 'webpack', 'concat:dist', 'uglify:dist']);
 };

Makefile

@@ -1,64 +1,65 b''
 
 .PHONY: clean docs docs-clean docs-cleanup test test-clean test-only test-only-postgres test-only-mysql web-build generate-pkgs
 
 NODE_PATH=./node_modules
 WEBPACK=./node_binaries/webpack
 GRUNT=./node_binaries/grunt
 
 
 clean:
 	make test-clean
 	find . -type f \( -iname '*.c' -o -iname '*.pyc' -o -iname '*.so' -o -iname '*.orig' \) -exec rm '{}' ';'
 
 test:
 	make test-clean
 	make test-only
 
 test-clean:
 	rm -rf coverage.xml htmlcov junit.xml pylint.log result
 	find . -type d -name "__pycache__" -prune -exec rm -rf '{}' ';'
+	find . -type f \( -iname '.coverage.*' \) -exec rm '{}' ';'
 
 test-only:
 	PYTHONHASHSEED=random \
 	py.test -x -vv -r xw -p no:sugar --cov=rhodecode \
     --cov-report=term-missing --cov-report=html \
     rhodecode
 
 test-only-mysql:
 	PYTHONHASHSEED=random \
 	py.test -x -vv -r xw -p no:sugar --cov=rhodecode \
     --cov-report=term-missing --cov-report=html \
-    --ini-config-override='{"app:main": {"sqlalchemy.db1.url": "mysql://root:qweqwe@localhost/rhodecode_test"}}' \
+    --ini-config-override='{"app:main": {"sqlalchemy.db1.url": "mysql://root:qweqwe@localhost/rhodecode_test?charset=utf8"}}' \
     rhodecode
 
 test-only-postgres:
 	PYTHONHASHSEED=random \
 	py.test -x -vv -r xw -p no:sugar --cov=rhodecode \
     --cov-report=term-missing --cov-report=html \
     --ini-config-override='{"app:main": {"sqlalchemy.db1.url": "postgresql://postgres:qweqwe@localhost/rhodecode_test"}}' \
     rhodecode
 
 
 docs:
 	(cd docs; nix-build default.nix -o result; make clean html)
 
 docs-clean:
 	(cd docs; make clean)
 
 docs-cleanup:
 	(cd docs; make cleanup)
 
 web-build:
 	NODE_PATH=$(NODE_PATH) $(GRUNT)
 
 generate-pkgs:
 	nix-shell pkgs/shell-generate.nix --command "pip2nix generate --licenses"
 
 generate-js-pkgs:
 	rm -rf node_modules && \
 	nix-shell pkgs/shell-generate.nix --command "node2nix --input package.json -o pkgs/node-packages.nix -e pkgs/node-env.nix -c pkgs/node-default.nix -d --flatten --nodejs-8" && \
 	sed -i -e 's/http:\/\//https:\/\//g' pkgs/node-packages.nix
 
 generate-license-meta:
 	nix-build pkgs/license-generate.nix -o result-license && \
 	cat result-license/licenses.json | python -m json.tool > rhodecode/config/licenses.json
  No newline at end of file

README.rst

@@ -1,116 +1,121 b''
 =========
 RhodeCode
 =========
 
 About
 -----
 
-``RhodeCode`` is a fast and powerful management tool for Mercurial_ and GIT_
-and Subversion_ with a built in push/pull server, full text search,
-pull requests and powerful code-review system. It works on http/https, SSH and
-has a few unique features like:
+``RhodeCode`` is a fast and powerful source code management tool for
+Mercurial_, GIT_ and Subversion_. It's main features are:
+
 
-- plugable architecture from Pyramid web-framework.
-- advanced permission system with IP restrictions, inheritation, and user-groups.
+- built in push/pull server
+- SSH with key management support
+- full text search.
+- plugable authentication.
+- pull requests and powerful code-review system.
+- advanced permission system with IP restrictions, permission inheritation, and user-groups.
 - rich set of authentication plugins including LDAP, ActiveDirectory, SAML 2.0,
   Atlassian Crowd, Http-Headers, Pam, Token-Auth, OAuth.
 - live code-review chat, and reviewer rules.
 - full web based file editing.
 - unified multi vcs support.
 - snippets (gist) system.
-- integration framework for Slack, CI systems, Webhooks.
+- artfacts store for binaries.
+- integration framework for Slack, CI systems, Webhooks, Jira, Redmine etc.
 - integration with all 3rd party issue trackers.
 
 
 RhodeCode also provides rich API, and multiple event hooks so it's easy
 integrable with existing external systems.
 
 RhodeCode is similar in some respects to gitlab_, github_ or bitbucket_,
 however RhodeCode can be run as standalone hosted application on your own server.
 RhodeCode can be installed on \*nix or Windows systems.
 
 RhodeCode uses `PEP386 versioning <http://www.python.org/dev/peps/pep-0386/>`_
 
 Installation
 ------------
 Please visit https://docs.rhodecode.com/RhodeCode-Control/tasks/install-cli.html
 for more details
 
 
 Source code
 -----------
 
 The latest sources can be obtained from official RhodeCode instance
-https://code.rhodecode.com
+https://code.rhodecode.com/rhodecode-enterprise-ce
+https://code.rhodecode.com/rhodecode-vcsserver
 
 
 Contributions
 -------------
 
 RhodeCode is open-source; contributions are welcome!
 
 Please see the contribution documentation inside of the docs folder, which is
 also available at
 https://docs.rhodecode.com/RhodeCode-Enterprise/contributing/contributing.html
 
 For additional information about collaboration tools, our issue tracker,
 licensing, and contribution credit, visit https://rhodecode.com/open-source 
 
 
 RhodeCode Features
 ------------------
 
 Check out all features of RhodeCode at https://rhodecode.com/features
 
 License
 -------
 
 ``RhodeCode`` is dual-licensed with AGPLv3 and commercial license.
 Please see LICENSE.txt file for details.
 
 
 Getting help
 ------------
 
 Listed bellow are various support resources that should help.
 
 .. note::
 
    Please try to read the documentation before posting any issues, especially
    the **troubleshooting section**
 
 - Official issue tracker `RhodeCode Issue tracker <https://issues.rhodecode.com>`_
 
 - Official Slack Community Channel `RhodeCode Slack <https://rhodecode.com/join>`_
 
 - Search our community portal `Community portal <https://community.rhodecode.com>`_
 
 - You can also follow RhodeCode on twitter **@RhodeCode** where we often post
   news and other interesting stuff about RhodeCode.
 
 
 Online documentation
 --------------------
 
 Online documentation for the current version of RhodeCode is available at
  - http://rhodecode.com/docs
 
 You may also build the documentation for yourself - go into ``docs/`` and run::
 
     nix-build default.nix -o result && make clean html
 
 (You need to have sphinx_ installed to build the documentation. If you don't
 have sphinx_ installed you can install it via the command:
 ``pip install sphinx``)
 
 .. _virtualenv: http://pypi.python.org/pypi/virtualenv
 .. _python: http://www.python.org/
 .. _sphinx: http://sphinx.pocoo.org/
 .. _mercurial: http://mercurial.selenic.com/
 .. _bitbucket: http://bitbucket.org/
 .. _github: http://github.com/
 .. _gitlab: http://gitlab.com/
 .. _subversion: http://subversion.tigris.org/
 .. _git: http://git-scm.com/
 .. _celery: http://celeryproject.org/
 .. _vcs: http://pypi.python.org/pypi/vcs

configs/development.ini

@@ -1,741 +1,833 b''
-
+## -*- coding: utf-8 -*-
 
-################################################################################
-##                RHODECODE COMMUNITY  EDITION CONFIGURATION                  ##
-################################################################################
+; #########################################
+; RHODECODE COMMUNITY EDITION CONFIGURATION
+; #########################################
 
 [DEFAULT]
-## Debug flag sets all loggers to debug, and enables request tracking
+; Debug flag sets all loggers to debug, and enables request tracking
 debug = true
 
-################################################################################
-##                            EMAIL CONFIGURATION                             ##
-## Uncomment and replace with the email address which should receive          ##
-## any error reports after an application crash                               ##
-## Additionally these settings will be used by the RhodeCode mailing system   ##
-################################################################################
+; ########################################################################
+; EMAIL CONFIGURATION
+; These settings will be used by the RhodeCode mailing system
+; ########################################################################
 
-## prefix all emails subjects with given prefix, helps filtering out emails
+; prefix all emails subjects with given prefix, helps filtering out emails
 #email_prefix = [RhodeCode]
 
-## email FROM address all mails will be sent
+; email FROM address all mails will be sent
 #app_email_from = rhodecode-noreply@localhost
 
 #smtp_server = mail.server.com
 #smtp_username =
 #smtp_password =
 #smtp_port =
 #smtp_use_tls = false
 #smtp_use_ssl = true
 
 [server:main]
-## COMMON ##
+; COMMON HOST/IP CONFIG
 host = 127.0.0.1
 port = 5000
 
-###########################################################
-## WAITRESS WSGI SERVER - Recommended for Development  ####
-###########################################################
+; ##################################################
+; WAITRESS WSGI SERVER - Recommended for Development
+; ##################################################
 
+; use server type
 use = egg:waitress#main
-## number of worker threads
+
+; number of worker threads
 threads = 5
-## MAX BODY SIZE 100GB
+
+; MAX BODY SIZE 100GB
 max_request_body_size = 107374182400
-## Use poll instead of select, fixes file descriptors limits problems.
-## May not work on old windows systems.
+
+; Use poll instead of select, fixes file descriptors limits problems.
+; May not work on old windows systems.
 asyncore_use_poll = true
 
 
-##########################
-## GUNICORN WSGI SERVER ##
-##########################
-## run with gunicorn --log-config rhodecode.ini --paste rhodecode.ini
+; ###########################
+; GUNICORN APPLICATION SERVER
+; ###########################
 
+; run with gunicorn --log-config rhodecode.ini --paste rhodecode.ini
+
+; Module to use, this setting shouldn't be changed
 #use = egg:gunicorn#main
-## Sets the number of process workers. More workers means more concurrent connections
-## RhodeCode can handle at the same time. Each additional worker also it increases
-## memory usage as each has it's own set of caches.
-## Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
-## than 8-10 unless for really big deployments .e.g 700-1000 users.
-## `instance_id = *` must be set in the [app:main] section below (which is the default)
-## when using more than 1 worker.
+
+; Sets the number of process workers. More workers means more concurrent connections
+; RhodeCode can handle at the same time. Each additional worker also it increases
+; memory usage as each has it's own set of caches.
+; Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
+; than 8-10 unless for really big deployments .e.g 700-1000 users.
+; `instance_id = *` must be set in the [app:main] section below (which is the default)
+; when using more than 1 worker.
 #workers = 2
-## process name visible in process list
+
+; Gunicorn access log level
+#loglevel = info
+
+; Process name visible in process list
 #proc_name = rhodecode
-## type of worker class, one of sync, gevent
-## recommended for bigger setup is using of of other than sync one
+
+; Type of worker class, one of `sync`, `gevent`
+; Recommended type is `gevent`
 #worker_class = gevent
-## The maximum number of simultaneous clients. Valid only for Gevent
+
+; The maximum number of simultaneous clients. Valid only for gevent
 #worker_connections = 10
-## max number of requests that worker will handle before being gracefully
-## restarted, could prevent memory leaks
+
+; Max number of requests that worker will handle before being gracefully restarted.
+; Prevents memory leaks, jitter adds variability so not all workers are restarted at once.
 #max_requests = 1000
 #max_requests_jitter = 30
-## amount of time a worker can spend with handling a request before it
-## gets killed and restarted. Set to 6hrs
+
+; Amount of time a worker can spend with handling a request before it
+; gets killed and restarted. By default set to 21600 (6hrs)
+; Examples: 1800 (30min), 3600 (1hr), 7200 (2hr), 43200 (12h)
 #timeout = 21600
 
+; The maximum size of HTTP request line in bytes.
+; 0 for unlimited
+#limit_request_line = 0
+
+; Limit the number of HTTP headers fields in a request.
+; By default this value is 100 and can't be larger than 32768.
+#limit_request_fields = 32768
+
+; Limit the allowed size of an HTTP request header field.
+; Value is a positive number or 0.
+; Setting it to 0 will allow unlimited header field sizes.
+#limit_request_field_size = 0
+
+; Timeout for graceful workers restart.
+; After receiving a restart signal, workers have this much time to finish
+; serving requests. Workers still alive after the timeout (starting from the
+; receipt of the restart signal) are force killed.
+; Examples: 1800 (30min), 3600 (1hr), 7200 (2hr), 43200 (12h)
+#graceful_timeout = 3600
+
+# The number of seconds to wait for requests on a Keep-Alive connection.
+# Generally set in the 1-5 seconds range.
+#keepalive = 2
+
+; Maximum memory usage that each worker can use before it will receive a
+; graceful restart signal 0 = memory monitoring is disabled
+; Examples: 268435456 (256MB), 536870912 (512MB)
+; 1073741824 (1GB), 2147483648 (2GB), 4294967296 (4GB)
+#memory_max_usage = 0
+
+; How often in seconds to check for memory usage for each gunicorn worker
+#memory_usage_check_interval = 60
+
+; Threshold value for which we don't recycle worker if GarbageCollection
+; frees up enough resources. Before each restart we try to run GC on worker
+; in case we get enough free memory after that, restart will not happen.
+#memory_usage_recovery_threshold = 0.8
+
 
-## prefix middleware for RhodeCode.
-## recommended when using proxy setup.
-## allows to set RhodeCode under a prefix in server.
-## eg https://server.com/custom_prefix. Enable `filter-with =` option below as well.
-## And set your prefix like: `prefix = /custom_prefix`
-## be sure to also set beaker.session.cookie_path = /custom_prefix if you need
-## to make your cookies only work on prefix url
+; Prefix middleware for RhodeCode.
+; recommended when using proxy setup.
+; allows to set RhodeCode under a prefix in server.
+; eg https://server.com/custom_prefix. Enable `filter-with =` option below as well.
+; And set your prefix like: `prefix = /custom_prefix`
+; be sure to also set beaker.session.cookie_path = /custom_prefix if you need
+; to make your cookies only work on prefix url
 [filter:proxy-prefix]
 use = egg:PasteDeploy#prefix
 prefix = /
 
 [app:main]
-## The %(here)s variable will be replaced with the absolute path of parent directory
-## of this file
-## In addition ENVIRONMENT variables usage is possible, e.g
-## sqlalchemy.db1.url = {ENV_RC_DB_URL}
+; The %(here)s variable will be replaced with the absolute path of parent directory
+; of this file
+; In addition ENVIRONMENT variables usage is possible, e.g
+; sqlalchemy.db1.url = {ENV_RC_DB_URL}
 
 use = egg:rhodecode-enterprise-ce
 
-## enable proxy prefix middleware, defined above
+; enable proxy prefix middleware, defined above
 #filter-with = proxy-prefix
 
+; #############
+; DEBUG OPTIONS
+; #############
+
+pyramid.reload_templates = true
+
 # During development the we want to have the debug toolbar enabled
 pyramid.includes =
     pyramid_debugtoolbar
-    rhodecode.lib.middleware.request_wrapper
-
-pyramid.reload_templates = true
 
 debugtoolbar.hosts = 0.0.0.0/0
 debugtoolbar.exclude_prefixes =
     /css
     /fonts
     /images
     /js
 
 ## RHODECODE PLUGINS ##
 rhodecode.includes =
     rhodecode.api
 
 
 # api prefix url
 rhodecode.api.url = /_admin/api
 
-
-## END RHODECODE PLUGINS ##
+; enable debug style page
+debug_style = true
 
-## encryption key used to encrypt social plugin tokens,
-## remote_urls with credentials etc, if not set it defaults to
-## `beaker.session.secret`
+; #################
+; END DEBUG OPTIONS
+; #################
+
+; encryption key used to encrypt social plugin tokens,
+; remote_urls with credentials etc, if not set it defaults to
+; `beaker.session.secret`
 #rhodecode.encrypted_values.secret =
 
-## decryption strict mode (enabled by default). It controls if decryption raises
-## `SignatureVerificationError` in case of wrong key, or damaged encryption data.
+; decryption strict mode (enabled by default). It controls if decryption raises
+; `SignatureVerificationError` in case of wrong key, or damaged encryption data.
 #rhodecode.encrypted_values.strict = false
 
-## Pick algorithm for encryption. Either fernet (more secure) or aes (default)
-## fernet is safer, and we strongly recommend switching to it.
-## Due to backward compatibility aes is used as default.
+; Pick algorithm for encryption. Either fernet (more secure) or aes (default)
+; fernet is safer, and we strongly recommend switching to it.
+; Due to backward compatibility aes is used as default.
 #rhodecode.encrypted_values.algorithm = fernet
 
-## return gzipped responses from RhodeCode (static files/application)
+; Return gzipped responses from RhodeCode (static files/application)
 gzip_responses = false
 
-## auto-generate javascript routes file on startup
+; Auto-generate javascript routes file on startup
 generate_js_files = false
 
-## System global default language.
-## All available languages: en(default), be, de, es, fr, it, ja, pl, pt, ru, zh
+; System global default language.
+; All available languages: en (default), be, de, es, fr, it, ja, pl, pt, ru, zh
 lang = en
 
-## Perform a full repository scan and import on each server start.
-## Settings this to true could lead to very long startup time.
+; Perform a full repository scan and import on each server start.
+; Settings this to true could lead to very long startup time.
 startup.import_repos = false
 
-## Uncomment and set this path to use archive download cache.
-## Once enabled, generated archives will be cached at this location
-## and served from the cache during subsequent requests for the same archive of
-## the repository.
+; Uncomment and set this path to use archive download cache.
+; Once enabled, generated archives will be cached at this location
+; and served from the cache during subsequent requests for the same archive of
+; the repository.
 #archive_cache_dir = /tmp/tarballcache
 
-## URL at which the application is running. This is used for Bootstrapping
-## requests in context when no web request is available. Used in ishell, or
-## SSH calls. Set this for events to receive proper url for SSH calls.
+; URL at which the application is running. This is used for Bootstrapping
+; requests in context when no web request is available. Used in ishell, or
+; SSH calls. Set this for events to receive proper url for SSH calls.
 app.base_url = http://rhodecode.local
 
-## Unique application ID. Should be a random unique string for security.
+; Unique application ID. Should be a random unique string for security.
 app_instance_uuid = rc-production
 
-## Cut off limit for large diffs (size in bytes). If overall diff size on
-## commit, or pull request exceeds this limit this diff will be displayed
-## partially. E.g 512000 == 512Kb
+; Cut off limit for large diffs (size in bytes). If overall diff size on
+; commit, or pull request exceeds this limit this diff will be displayed
+; partially. E.g 512000 == 512Kb
 cut_off_limit_diff = 512000
 
-## Cut off limit for large files inside diffs (size in bytes). Each individual
-## file inside diff which exceeds this limit will be displayed partially.
-##  E.g 128000 == 128Kb
+; Cut off limit for large files inside diffs (size in bytes). Each individual
+; file inside diff which exceeds this limit will be displayed partially.
+;  E.g 128000 == 128Kb
 cut_off_limit_file = 128000
 
-## use cached version of vcs repositories everywhere. Recommended to be `true`
+; Use cached version of vcs repositories everywhere. Recommended to be `true`
 vcs_full_cache = true
 
-## Force https in RhodeCode, fixes https redirects, assumes it's always https.
-## Normally this is controlled by proper http flags sent from http server
+; Force https in RhodeCode, fixes https redirects, assumes it's always https.
+; Normally this is controlled by proper flags sent from http server such as Nginx or Apache
 force_https = false
 
-## use Strict-Transport-Security headers
+; use Strict-Transport-Security headers
 use_htsts = false
 
-## git rev filter option, --all is the default filter, if you need to
-## hide all refs in changelog switch this to --branches --tags
-git_rev_filter = --branches --tags
-
-# Set to true if your repos are exposed using the dumb protocol
+; Set to true if your repos are exposed using the dumb protocol
 git_update_server_info = false
 
-## RSS/ATOM feed options
+; RSS/ATOM feed options
 rss_cut_off_limit = 256000
 rss_items_per_page = 10
 rss_include_diff = false
 
-## gist URL alias, used to create nicer urls for gist. This should be an
-## url that does rewrites to _admin/gists/{gistid}.
-## example: http://gist.rhodecode.org/{gistid}. Empty means use the internal
-## RhodeCode url, ie. http[s]://rhodecode.server/_admin/gists/{gistid}
+; gist URL alias, used to create nicer urls for gist. This should be an
+; url that does rewrites to _admin/gists/{gistid}.
+; example: http://gist.rhodecode.org/{gistid}. Empty means use the internal
+; RhodeCode url, ie. http[s]://rhodecode.server/_admin/gists/{gistid}
 gist_alias_url =
 
-## List of views (using glob pattern syntax) that AUTH TOKENS could be
-## used for access.
-## Adding ?auth_token=TOKEN_HASH to the url authenticates this request as if it
-## came from the the logged in user who own this authentication token.
-## Additionally @TOKEN syntax can be used to bound the view to specific
-## authentication token. Such view would be only accessible when used together
-## with this authentication token
-##
-## list of all views can be found under `/_admin/permissions/auth_token_access`
-## The list should be "," separated and on a single line.
-##
-## Most common views to enable:
+; List of views (using glob pattern syntax) that AUTH TOKENS could be
+; used for access.
+; Adding ?auth_token=TOKEN_HASH to the url authenticates this request as if it
+; came from the the logged in user who own this authentication token.
+; Additionally @TOKEN syntax can be used to bound the view to specific
+; authentication token. Such view would be only accessible when used together
+; with this authentication token
+; list of all views can be found under `/_admin/permissions/auth_token_access`
+; The list should be "," separated and on a single line.
+; Most common views to enable:
+
 #    RepoCommitsView:repo_commit_download
 #    RepoCommitsView:repo_commit_patch
 #    RepoCommitsView:repo_commit_raw
 #    RepoCommitsView:repo_commit_raw@TOKEN
 #    RepoFilesView:repo_files_diff
 #    RepoFilesView:repo_archivefile
 #    RepoFilesView:repo_file_raw
 #    GistView:*
 api_access_controllers_whitelist =
 
-## Default encoding used to convert from and to unicode
-## can be also a comma separated list of encoding in case of mixed encodings
+; Default encoding used to convert from and to unicode
+; can be also a comma separated list of encoding in case of mixed encodings
 default_encoding = UTF-8
 
-## instance-id prefix
-## a prefix key for this instance used for cache invalidation when running
-## multiple instances of RhodeCode, make sure it's globally unique for
-## all running RhodeCode instances. Leave empty if you don't use it
+; instance-id prefix
+; a prefix key for this instance used for cache invalidation when running
+; multiple instances of RhodeCode, make sure it's globally unique for
+; all running RhodeCode instances. Leave empty if you don't use it
 instance_id =
 
-## Fallback authentication plugin. Set this to a plugin ID to force the usage
-## of an authentication plugin also if it is disabled by it's settings.
-## This could be useful if you are unable to log in to the system due to broken
-## authentication settings. Then you can enable e.g. the internal RhodeCode auth
-## module to log in again and fix the settings.
-##
-## Available builtin plugin IDs (hash is part of the ID):
-## egg:rhodecode-enterprise-ce#rhodecode
-## egg:rhodecode-enterprise-ce#pam
-## egg:rhodecode-enterprise-ce#ldap
-## egg:rhodecode-enterprise-ce#jasig_cas
-## egg:rhodecode-enterprise-ce#headers
-## egg:rhodecode-enterprise-ce#crowd
+; Fallback authentication plugin. Set this to a plugin ID to force the usage
+; of an authentication plugin also if it is disabled by it's settings.
+; This could be useful if you are unable to log in to the system due to broken
+; authentication settings. Then you can enable e.g. the internal RhodeCode auth
+; module to log in again and fix the settings.
+; Available builtin plugin IDs (hash is part of the ID):
+; egg:rhodecode-enterprise-ce#rhodecode
+; egg:rhodecode-enterprise-ce#pam
+; egg:rhodecode-enterprise-ce#ldap
+; egg:rhodecode-enterprise-ce#jasig_cas
+; egg:rhodecode-enterprise-ce#headers
+; egg:rhodecode-enterprise-ce#crowd
+
 #rhodecode.auth_plugin_fallback = egg:rhodecode-enterprise-ce#rhodecode
 
-## alternative return HTTP header for failed authentication. Default HTTP
-## response is 401 HTTPUnauthorized. Currently HG clients have troubles with
-## handling that causing a series of failed authentication calls.
-## Set this variable to 403 to return HTTPForbidden, or any other HTTP code
-## This will be served instead of default 401 on bad authentication
+; Flag to control loading of legacy plugins in py:/path format
+auth_plugin.import_legacy_plugins = true
+
+; alternative return HTTP header for failed authentication. Default HTTP
+; response is 401 HTTPUnauthorized. Currently HG clients have troubles with
+; handling that causing a series of failed authentication calls.
+; Set this variable to 403 to return HTTPForbidden, or any other HTTP code
+; This will be served instead of default 401 on bad authentication
 auth_ret_code =
 
-## use special detection method when serving auth_ret_code, instead of serving
-## ret_code directly, use 401 initially (Which triggers credentials prompt)
-## and then serve auth_ret_code to clients
+; use special detection method when serving auth_ret_code, instead of serving
+; ret_code directly, use 401 initially (Which triggers credentials prompt)
+; and then serve auth_ret_code to clients
 auth_ret_code_detection = false
 
-## locking return code. When repository is locked return this HTTP code. 2XX
-## codes don't break the transactions while 4XX codes do
+; locking return code. When repository is locked return this HTTP code. 2XX
+; codes don't break the transactions while 4XX codes do
 lock_ret_code = 423
 
-## allows to change the repository location in settings page
+; allows to change the repository location in settings page
 allow_repo_location_change = true
 
-## allows to setup custom hooks in settings page
+; allows to setup custom hooks in settings page
 allow_custom_hooks_settings = true
 
-## Generated license token required for EE edition license.
-## New generated token value can be found in Admin > settings > license page.
+; Generated license token required for EE edition license.
+; New generated token value can be found in Admin > settings > license page.
 license_token =
 
-## supervisor connection uri, for managing supervisor and logs.
+; This flag hides sensitive information on the license page such as token, and license data
+license.hide_license_info = false
+
+; supervisor connection uri, for managing supervisor and logs.
 supervisor.uri =
-## supervisord group name/id we only want this RC instance to handle
+
+; supervisord group name/id we only want this RC instance to handle
 supervisor.group_id = dev
 
-## Display extended labs settings
+; Display extended labs settings
 labs_settings_active = true
 
-## Custom exception store path, defaults to TMPDIR
-## This is used to store exception from RhodeCode in shared directory
+; Custom exception store path, defaults to TMPDIR
+; This is used to store exception from RhodeCode in shared directory
 #exception_tracker.store_path =
 
-## File store configuration. This is used to store and serve uploaded files
+; File store configuration. This is used to store and serve uploaded files
 file_store.enabled = true
-## Storage backend, available options are: local
+
+; Storage backend, available options are: local
 file_store.backend = local
-## path to store the uploaded binaries
+
+; path to store the uploaded binaries
 file_store.storage_path = %(here)s/data/file_store
 
 
-####################################
-###        CELERY CONFIG        ####
-####################################
-## run: /path/to/celery worker \
-##     -E --beat --app rhodecode.lib.celerylib.loader \
-##     --scheduler rhodecode.lib.celerylib.scheduler.RcScheduler \
-##     --loglevel DEBUG --ini /path/to/rhodecode.ini
+; #############
+; CELERY CONFIG
+; #############
+
+; manually run celery: /path/to/celery worker -E --beat --app rhodecode.lib.celerylib.loader --scheduler rhodecode.lib.celerylib.scheduler.RcScheduler --loglevel DEBUG --ini /path/to/rhodecode.ini
 
 use_celery = false
 
-## connection url to the message broker (default rabbitmq)
-celery.broker_url = amqp://rabbitmq:qweqwe@localhost:5672/rabbitmqhost
+; connection url to the message broker (default redis)
+celery.broker_url = redis://localhost:6379/8
 
-## maximum tasks to execute before worker restart
+; rabbitmq example
+#celery.broker_url = amqp://rabbitmq:qweqwe@localhost:5672/rabbitmqhost
+
+; maximum tasks to execute before worker restart
 celery.max_tasks_per_child = 100
 
-## tasks will never be sent to the queue, but executed locally instead.
+; tasks will never be sent to the queue, but executed locally instead.
 celery.task_always_eager = false
 
-#####################################
-###         DOGPILE CACHE        ####
-#####################################
-## Default cache dir for caches.  Putting this into a ramdisk
-## can boost performance, eg. /tmpfs/data_ramdisk, however this directory might require
-## large amount of space
+; #############
+; DOGPILE CACHE
+; #############
+
+; Default cache dir for caches. Putting this into a ramdisk can boost performance.
+; eg. /tmpfs/data_ramdisk, however this directory might require large amount of space
 cache_dir = %(here)s/data
 
-## `cache_perms` cache settings for permission tree, auth TTL.
+; *********************************************
+; `sql_cache_short` cache for heavy SQL queries
+; Only supported backend is `memory_lru`
+; *********************************************
+rc_cache.sql_cache_short.backend = dogpile.cache.rc.memory_lru
+rc_cache.sql_cache_short.expiration_time = 30
+
+
+; *****************************************************
+; `cache_repo_longterm` cache for repo object instances
+; Only supported backend is `memory_lru`
+; *****************************************************
+rc_cache.cache_repo_longterm.backend = dogpile.cache.rc.memory_lru
+; by default we use 30 Days, cache is still invalidated on push
+rc_cache.cache_repo_longterm.expiration_time = 2592000
+; max items in LRU cache, set to smaller number to save memory, and expire last used caches
+rc_cache.cache_repo_longterm.max_size = 10000
+
+
+; *************************************************
+; `cache_perms` cache for permission tree, auth TTL
+; *************************************************
 rc_cache.cache_perms.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_perms.expiration_time = 300
+; file cache store path. Defaults to `cache_dir =` value or tempdir if both values are not set
+#rc_cache.cache_perms.arguments.filename = /tmp/cache_perms.db
 
-## alternative `cache_perms` redis backend with distributed lock
+; alternative `cache_perms` redis backend with distributed lock
 #rc_cache.cache_perms.backend = dogpile.cache.rc.redis
 #rc_cache.cache_perms.expiration_time = 300
-## redis_expiration_time needs to be greater then expiration_time
+
+; redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_perms.arguments.redis_expiration_time = 7200
-#rc_cache.cache_perms.arguments.socket_timeout = 30
+
 #rc_cache.cache_perms.arguments.host = localhost
 #rc_cache.cache_perms.arguments.port = 6379
 #rc_cache.cache_perms.arguments.db = 0
-## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
+#rc_cache.cache_perms.arguments.socket_timeout = 30
+; more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_perms.arguments.distributed_lock = true
 
-## `cache_repo` cache settings for FileTree, Readme, RSS FEEDS
+
+; ***************************************************
+; `cache_repo` cache for file tree, Readme, RSS FEEDS
+; ***************************************************
 rc_cache.cache_repo.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_repo.expiration_time = 2592000
+; file cache store path. Defaults to `cache_dir =` value or tempdir if both values are not set
+#rc_cache.cache_repo.arguments.filename = /tmp/cache_repo.db
 
-## alternative `cache_repo` redis backend with distributed lock
+; alternative `cache_repo` redis backend with distributed lock
 #rc_cache.cache_repo.backend = dogpile.cache.rc.redis
 #rc_cache.cache_repo.expiration_time = 2592000
-## redis_expiration_time needs to be greater then expiration_time
+
+; redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_repo.arguments.redis_expiration_time = 2678400
-#rc_cache.cache_repo.arguments.socket_timeout = 30
+
 #rc_cache.cache_repo.arguments.host = localhost
 #rc_cache.cache_repo.arguments.port = 6379
 #rc_cache.cache_repo.arguments.db = 1
-## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
+#rc_cache.cache_repo.arguments.socket_timeout = 30
+; more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_repo.arguments.distributed_lock = true
 
-## cache settings for SQL queries, this needs to use memory type backend
-rc_cache.sql_cache_short.backend = dogpile.cache.rc.memory_lru
-rc_cache.sql_cache_short.expiration_time = 30
 
-## `cache_repo_longterm` cache for repo object instances, this needs to use memory
-## type backend as the objects kept are not pickle serializable
-rc_cache.cache_repo_longterm.backend = dogpile.cache.rc.memory_lru
-## by default we use 96H, this is using invalidation on push anyway
-rc_cache.cache_repo_longterm.expiration_time = 345600
-## max items in LRU cache, reduce this number to save memory, and expire last used
-## cached objects
-rc_cache.cache_repo_longterm.max_size = 10000
+; ##############
+; BEAKER SESSION
+; ##############
 
-
-####################################
-###       BEAKER SESSION        ####
-####################################
-
-## .session.type is type of storage options for the session, current allowed
-## types are file, ext:memcached, ext:redis, ext:database, and memory (default).
+; beaker.session.type is type of storage options for the logged users sessions. Current allowed
+; types are file, ext:redis, ext:database, ext:memcached, and memory (default if not specified).
+; Fastest ones are Redis and ext:database
 beaker.session.type = file
 beaker.session.data_dir = %(here)s/data/sessions
 
-## redis sessions
+; Redis based sessions
 #beaker.session.type = ext:redis
 #beaker.session.url = redis://127.0.0.1:6379/2
 
-## db based session, fast, and allows easy management over logged in users
+; DB based session, fast, and allows easy management over logged in users
 #beaker.session.type = ext:database
 #beaker.session.table_name = db_session
 #beaker.session.sa.url = postgresql://postgres:secret@localhost/rhodecode
 #beaker.session.sa.url = mysql://root:secret@127.0.0.1/rhodecode
 #beaker.session.sa.pool_recycle = 3600
 #beaker.session.sa.echo = false
 
 beaker.session.key = rhodecode
 beaker.session.secret = develop-rc-uytcxaz
 beaker.session.lock_dir = %(here)s/data/sessions/lock
 
-## Secure encrypted cookie. Requires AES and AES python libraries
-## you must disable beaker.session.secret to use this
+; Secure encrypted cookie. Requires AES and AES python libraries
+; you must disable beaker.session.secret to use this
 #beaker.session.encrypt_key = key_for_encryption
 #beaker.session.validate_key = validation_key
 
-## sets session as invalid(also logging out user) if it haven not been
-## accessed for given amount of time in seconds
+; Sets session as invalid (also logging out user) if it haven not been
+; accessed for given amount of time in seconds
 beaker.session.timeout = 2592000
 beaker.session.httponly = true
-## Path to use for the cookie. Set to prefix if you use prefix middleware
+
+; Path to use for the cookie. Set to prefix if you use prefix middleware
 #beaker.session.cookie_path = /custom_prefix
 
-## uncomment for https secure cookie
+; Set https secure cookie
 beaker.session.secure = false
 
-## auto save the session to not to use .save()
-beaker.session.auto = false
-
-## default cookie expiration time in seconds, set to `true` to set expire
-## at browser close
+; default cookie expiration time in seconds, set to `true` to set expire
+; at browser close
 #beaker.session.cookie_expires = 3600
 
-###################################
-## SEARCH INDEXING CONFIGURATION ##
-###################################
-## Full text search indexer is available in rhodecode-tools under
-## `rhodecode-tools index` command
+; #############################
+; SEARCH INDEXING CONFIGURATION
+; #############################
 
-## WHOOSH Backend, doesn't require additional services to run
-## it works good with few dozen repos
+; Full text search indexer is available in rhodecode-tools under
+; `rhodecode-tools index` command
+
+; WHOOSH Backend, doesn't require additional services to run
+; it works good with few dozen repos
 search.module = rhodecode.lib.index.whoosh
 search.location = %(here)s/data/index
 
-########################################
-###    CHANNELSTREAM CONFIG         ####
-########################################
-## channelstream enables persistent connections and live notification
-## in the system. It's also used by the chat system
+; ####################
+; CHANNELSTREAM CONFIG
+; ####################
+
+; channelstream enables persistent connections and live notification
+; in the system. It's also used by the chat system
 
 channelstream.enabled = false
 
-## server address for channelstream server on the backend
+; server address for channelstream server on the backend
 channelstream.server = 127.0.0.1:9800
 
-## location of the channelstream server from outside world
-## use ws:// for http or wss:// for https. This address needs to be handled
-## by external HTTP server such as Nginx or Apache
-## see Nginx/Apache configuration examples in our docs
+; location of the channelstream server from outside world
+; use ws:// for http or wss:// for https. This address needs to be handled
+; by external HTTP server such as Nginx or Apache
+; see Nginx/Apache configuration examples in our docs
 channelstream.ws_url = ws://rhodecode.yourserver.com/_channelstream
 channelstream.secret = secret
 channelstream.history.location = %(here)s/channelstream_history
 
-## Internal application path that Javascript uses to connect into.
-## If you use proxy-prefix the prefix should be added before /_channelstream
+; Internal application path that Javascript uses to connect into.
+; If you use proxy-prefix the prefix should be added before /_channelstream
 channelstream.proxy_path = /_channelstream
 
 
-###################################
-##       APPENLIGHT CONFIG       ##
-###################################
+; ##############################
+; MAIN RHODECODE DATABASE CONFIG
+; ##############################
+
+#sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
+#sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
+#sqlalchemy.db1.url = mysql://root:qweqwe@localhost/rhodecode?charset=utf8
+; pymysql is an alternative driver for MySQL, use in case of problems with default one
+#sqlalchemy.db1.url = mysql+pymysql://root:qweqwe@localhost/rhodecode
+
+sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
+
+; see sqlalchemy docs for other advanced settings
+; print the sql statements to output
+sqlalchemy.db1.echo = false
+
+; recycle the connections after this amount of seconds
+sqlalchemy.db1.pool_recycle = 3600
+sqlalchemy.db1.convert_unicode = true
+
+; the number of connections to keep open inside the connection pool.
+; 0 indicates no limit
+#sqlalchemy.db1.pool_size = 5
+
+; The number of connections to allow in connection pool "overflow", that is
+; connections that can be opened above and beyond the pool_size setting,
+; which defaults to five.
+#sqlalchemy.db1.max_overflow = 10
+
+; Connection check ping, used to detect broken database connections
+; could be enabled to better handle cases if MySQL has gone away errors
+#sqlalchemy.db1.ping_connection = true
+
+; ##########
+; VCS CONFIG
+; ##########
+vcs.server.enable = true
+vcs.server = localhost:9900
+
+; Web server connectivity protocol, responsible for web based VCS operations
+; Available protocols are:
+; `http` - use http-rpc backend (default)
+vcs.server.protocol = http
+
+; Push/Pull operations protocol, available options are:
+; `http` - use http-rpc backend (default)
+vcs.scm_app_implementation = http
+
+; Push/Pull operations hooks protocol, available options are:
+; `http` - use http-rpc backend (default)
+vcs.hooks.protocol = http
+
+; Host on which this instance is listening for hooks. If vcsserver is in other location
+; this should be adjusted.
+vcs.hooks.host = 127.0.0.1
+
+; Start VCSServer with this instance as a subprocess, useful for development
+vcs.start_server = false
+
+; List of enabled VCS backends, available options are:
+; `hg`  - mercurial
+; `git` - git
+; `svn` - subversion
+vcs.backends = hg, git, svn
+
+; Wait this number of seconds before killing connection to the vcsserver
+vcs.connection_timeout = 3600
+
+; Compatibility version when creating SVN repositories. Defaults to newest version when commented out.
+; Available options are: pre-1.4-compatible, pre-1.5-compatible, pre-1.6-compatible, pre-1.8-compatible, pre-1.9-compatible
+#vcs.svn.compatible_version = pre-1.8-compatible
+
 
-## Appenlight is tailored to work with RhodeCode, see
-## http://appenlight.com for details how to obtain an account
+; ####################################################
+; Subversion proxy support (mod_dav_svn)
+; Maps RhodeCode repo groups into SVN paths for Apache
+; ####################################################
+
+; Enable or disable the config file generation.
+svn.proxy.generate_config = false
+
+; Generate config file with `SVNListParentPath` set to `On`.
+svn.proxy.list_parent_path = true
+
+; Set location and file name of generated config file.
+svn.proxy.config_file_path = %(here)s/mod_dav_svn.conf
+
+; alternative mod_dav config template. This needs to be a valid mako template
+; Example template can be found in the source code:
+; rhodecode/apps/svn_support/templates/mod-dav-svn.conf.mako
+#svn.proxy.config_template = ~/.rccontrol/enterprise-1/custom_svn_conf.mako
+
+; Used as a prefix to the `Location` block in the generated config file.
+; In most cases it should be set to `/`.
+svn.proxy.location_root = /
+
+; Command to reload the mod dav svn configuration on change.
+; Example: `/etc/init.d/apache2 reload` or /home/USER/apache_reload.sh
+; Make sure user who runs RhodeCode process is allowed to reload Apache
+#svn.proxy.reload_cmd = /etc/init.d/apache2 reload
+
+; If the timeout expires before the reload command finishes, the command will
+; be killed. Setting it to zero means no timeout. Defaults to 10 seconds.
+#svn.proxy.reload_timeout = 10
+
+; ####################
+; SSH Support Settings
+; ####################
 
-## Appenlight integration enabled
+; Defines if a custom authorized_keys file should be created and written on
+; any change user ssh keys. Setting this to false also disables possibility
+; of adding SSH keys by users from web interface. Super admins can still
+; manage SSH Keys.
+ssh.generate_authorized_keyfile = false
+
+; Options for ssh, default is `no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding`
+# ssh.authorized_keys_ssh_opts =
+
+; Path to the authorized_keys file where the generate entries are placed.
+; It is possible to have multiple key files specified in `sshd_config` e.g.
+; AuthorizedKeysFile %h/.ssh/authorized_keys %h/.ssh/authorized_keys_rhodecode
+ssh.authorized_keys_file_path = ~/.ssh/authorized_keys_rhodecode
+
+; Command to execute the SSH wrapper. The binary is available in the
+; RhodeCode installation directory.
+; e.g ~/.rccontrol/community-1/profile/bin/rc-ssh-wrapper
+ssh.wrapper_cmd = ~/.rccontrol/community-1/rc-ssh-wrapper
+
+; Allow shell when executing the ssh-wrapper command
+ssh.wrapper_cmd_allow_shell = false
+
+; Enables logging, and detailed output send back to the client during SSH
+; operations. Useful for debugging, shouldn't be used in production.
+ssh.enable_debug_logging = true
+
+; Paths to binary executable, by default they are the names, but we can
+; override them if we want to use a custom one
+ssh.executable.hg = ~/.rccontrol/vcsserver-1/profile/bin/hg
+ssh.executable.git = ~/.rccontrol/vcsserver-1/profile/bin/git
+ssh.executable.svn = ~/.rccontrol/vcsserver-1/profile/bin/svnserve
+
+; Enables SSH key generator web interface. Disabling this still allows users
+; to add their own keys.
+ssh.enable_ui_key_generator = true
+
+
+; #################
+; APPENLIGHT CONFIG
+; #################
+
+; Appenlight is tailored to work with RhodeCode, see
+; http://appenlight.rhodecode.com for details how to obtain an account
+
+; Appenlight integration enabled
 appenlight = false
 
 appenlight.server_url = https://api.appenlight.com
 appenlight.api_key = YOUR_API_KEY
 #appenlight.transport_config = https://api.appenlight.com?threaded=1&timeout=5
 
-## used for JS client
+; used for JS client
 appenlight.api_public_key = YOUR_API_PUBLIC_KEY
 
-## TWEAK AMOUNT OF INFO SENT HERE
+; TWEAK AMOUNT OF INFO SENT HERE
 
-## enables 404 error logging (default False)
+; enables 404 error logging (default False)
 appenlight.report_404 = false
 
-## time in seconds after request is considered being slow (default 1)
+; time in seconds after request is considered being slow (default 1)
 appenlight.slow_request_time = 1
 
-## record slow requests in application
-## (needs to be enabled for slow datastore recording and time tracking)
+; record slow requests in application
+; (needs to be enabled for slow datastore recording and time tracking)
 appenlight.slow_requests = true
 
-## enable hooking to application loggers
+; enable hooking to application loggers
 appenlight.logging = true
 
-## minimum log level for log capture
+; minimum log level for log capture
 appenlight.logging.level = WARNING
 
-## send logs only from erroneous/slow requests
-## (saves API quota for intensive logging)
+; send logs only from erroneous/slow requests
+; (saves API quota for intensive logging)
 appenlight.logging_on_error = false
 
-## list of additional keywords that should be grabbed from environ object
-## can be string with comma separated list of words in lowercase
-## (by default client will always send following info:
-## 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
-## start with HTTP* this list be extended with additional keywords here
+; list of additional keywords that should be grabbed from environ object
+; can be string with comma separated list of words in lowercase
+; (by default client will always send following info:
+; 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
+; start with HTTP* this list be extended with additional keywords here
 appenlight.environ_keys_whitelist =
 
-## list of keywords that should be blanked from request object
-## can be string with comma separated list of words in lowercase
-## (by default client will always blank keys that contain following words
-## 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
-## this list be extended with additional keywords set here
+; list of keywords that should be blanked from request object
+; can be string with comma separated list of words in lowercase
+; (by default client will always blank keys that contain following words
+; 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
+; this list be extended with additional keywords set here
 appenlight.request_keys_blacklist =
 
-## list of namespaces that should be ignores when gathering log entries
-## can be string with comma separated list of namespaces
-## (by default the client ignores own entries: appenlight_client.client)
+; list of namespaces that should be ignores when gathering log entries
+; can be string with comma separated list of namespaces
+; (by default the client ignores own entries: appenlight_client.client)
 appenlight.log_namespace_blacklist =
 
-# enable debug style page
-debug_style = true
-
-###########################################
-###   MAIN RHODECODE DATABASE CONFIG    ###
-###########################################
-#sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
-#sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
-#sqlalchemy.db1.url = mysql://root:qweqwe@localhost/rhodecode?charset=utf8
-# pymysql is an alternative driver for MySQL, use in case of problems with default one
-#sqlalchemy.db1.url = mysql+pymysql://root:qweqwe@localhost/rhodecode
-
-sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
-
-# see sqlalchemy docs for other advanced settings
-
-## print the sql statements to output
-sqlalchemy.db1.echo = false
-## recycle the connections after this amount of seconds
-sqlalchemy.db1.pool_recycle = 3600
-sqlalchemy.db1.convert_unicode = true
-
-## the number of connections to keep open inside the connection pool.
-## 0 indicates no limit
-#sqlalchemy.db1.pool_size = 5
-
-## the number of connections to allow in connection pool "overflow", that is
-## connections that can be opened above and beyond the pool_size setting,
-## which defaults to five.
-#sqlalchemy.db1.max_overflow = 10
-
-## Connection check ping, used to detect broken database connections
-## could be enabled to better handle cases if MySQL has gone away errors
-#sqlalchemy.db1.ping_connection = true
-
-##################
-### VCS CONFIG ###
-##################
-vcs.server.enable = true
-vcs.server = localhost:9900
-
-## Web server connectivity protocol, responsible for web based VCS operations
-## Available protocols are:
-## `http` - use http-rpc backend (default)
-vcs.server.protocol = http
-
-## Push/Pull operations protocol, available options are:
-## `http` - use http-rpc backend (default)
-vcs.scm_app_implementation = http
-
-## Push/Pull operations hooks protocol, available options are:
-## `http` - use http-rpc backend (default)
-vcs.hooks.protocol = http
-
-## Host on which this instance is listening for hooks. If vcsserver is in other location
-## this should be adjusted.
-vcs.hooks.host = 127.0.0.1
-
-vcs.server.log_level = debug
-## Start VCSServer with this instance as a subprocess, useful for development
-vcs.start_server = false
-
-## List of enabled VCS backends, available options are:
-## `hg`  - mercurial
-## `git` - git
-## `svn` - subversion
-vcs.backends = hg, git, svn
-
-vcs.connection_timeout = 3600
-## Compatibility version when creating SVN repositories. Defaults to newest version when commented out.
-## Available options are: pre-1.4-compatible, pre-1.5-compatible, pre-1.6-compatible, pre-1.8-compatible, pre-1.9-compatible
-#vcs.svn.compatible_version = pre-1.8-compatible
-
-
-############################################################
-### Subversion proxy support (mod_dav_svn)               ###
-### Maps RhodeCode repo groups into SVN paths for Apache ###
-############################################################
-## Enable or disable the config file generation.
-svn.proxy.generate_config = false
-## Generate config file with `SVNListParentPath` set to `On`.
-svn.proxy.list_parent_path = true
-## Set location and file name of generated config file.
-svn.proxy.config_file_path = %(here)s/mod_dav_svn.conf
-## alternative mod_dav config template. This needs to be a mako template
-#svn.proxy.config_template = ~/.rccontrol/enterprise-1/custom_svn_conf.mako
-## Used as a prefix to the `Location` block in the generated config file.
-## In most cases it should be set to `/`.
-svn.proxy.location_root = /
-## Command to reload the mod dav svn configuration on change.
-## Example: `/etc/init.d/apache2 reload` or /home/USER/apache_reload.sh
-## Make sure user who runs RhodeCode process is allowed to reload Apache
-#svn.proxy.reload_cmd = /etc/init.d/apache2 reload
-## If the timeout expires before the reload command finishes, the command will
-## be killed. Setting it to zero means no timeout. Defaults to 10 seconds.
-#svn.proxy.reload_timeout = 10
-
-############################################################
-### SSH Support Settings                                 ###
-############################################################
-
-## Defines if a custom authorized_keys file should be created and written on
-## any change user ssh keys. Setting this to false also disables possibility
-## of adding SSH keys by users from web interface. Super admins can still
-## manage SSH Keys.
-ssh.generate_authorized_keyfile = false
-
-## Options for ssh, default is `no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding`
-# ssh.authorized_keys_ssh_opts =
-
-## Path to the authorized_keys file where the generate entries are placed.
-## It is possible to have multiple key files specified in `sshd_config` e.g.
-## AuthorizedKeysFile %h/.ssh/authorized_keys %h/.ssh/authorized_keys_rhodecode
-ssh.authorized_keys_file_path = ~/.ssh/authorized_keys_rhodecode
-
-## Command to execute the SSH wrapper. The binary is available in the
-## RhodeCode installation directory.
-## e.g ~/.rccontrol/community-1/profile/bin/rc-ssh-wrapper
-ssh.wrapper_cmd = ~/.rccontrol/community-1/rc-ssh-wrapper
-
-## Allow shell when executing the ssh-wrapper command
-ssh.wrapper_cmd_allow_shell = false
-
-## Enables logging, and detailed output send back to the client during SSH
-## operations. Useful for debugging, shouldn't be used in production.
-ssh.enable_debug_logging = true
-
-## Paths to binary executable, by default they are the names, but we can
-## override them if we want to use a custom one
-ssh.executable.hg = ~/.rccontrol/vcsserver-1/profile/bin/hg
-ssh.executable.git = ~/.rccontrol/vcsserver-1/profile/bin/git
-ssh.executable.svn = ~/.rccontrol/vcsserver-1/profile/bin/svnserve
-
-## Enables SSH key generator web interface. Disabling this still allows users
-## to add their own keys.
-ssh.enable_ui_key_generator = true
-
-
-## Dummy marker to add new entries after.
-## Add any custom entries below. Please don't remove.
+; Dummy marker to add new entries after.
+; Add any custom entries below. Please don't remove this marker.
 custom.conf = 1
 
 
-################################
-### LOGGING CONFIGURATION   ####
-################################
+; #####################
+; LOGGING CONFIGURATION
+; #####################
 [loggers]
 keys = root, sqlalchemy, beaker, celery, rhodecode, ssh_wrapper
 
 [handlers]
 keys = console, console_sql
 
 [formatters]
 keys = generic, color_formatter, color_formatter_sql
 
-#############
-## LOGGERS ##
-#############
+; #######
+; LOGGERS
+; #######
 [logger_root]
 level = NOTSET
 handlers = console
 
 [logger_sqlalchemy]
 level = INFO
 handlers = console_sql
 qualname = sqlalchemy.engine
 propagate = 0
 
 [logger_beaker]
 level = DEBUG
 handlers =
 qualname = beaker.container
 propagate = 1
 
 [logger_rhodecode]
 level = DEBUG
 handlers =
 qualname = rhodecode
 propagate = 1
 
 [logger_ssh_wrapper]
 level = DEBUG
 handlers =
 qualname = ssh_wrapper
 propagate = 1
 
 [logger_celery]
 level = DEBUG
 handlers =
 qualname = celery
 
 
-##############
-## HANDLERS ##
-##############
+; ########
+; HANDLERS
+; ########
 
 [handler_console]
 class = StreamHandler
 args = (sys.stderr, )
 level = DEBUG
 formatter = color_formatter
 
 [handler_console_sql]
-# "level = DEBUG" logs SQL queries and results.
-# "level = INFO" logs SQL queries.
-# "level = WARN" logs neither.  (Recommended for production systems.)
+; "level = DEBUG" logs SQL queries and results.
+; "level = INFO" logs SQL queries.
+; "level = WARN" logs neither.  (Recommended for production systems.)
 class = StreamHandler
 args = (sys.stderr, )
 level = WARN
 formatter = color_formatter_sql
 
-################
-## FORMATTERS ##
-################
+; ##########
+; FORMATTERS
+; ##########
 
 [formatter_generic]
 class = rhodecode.lib.logging_formatter.ExceptionAwareFormatter
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter]
 class = rhodecode.lib.logging_formatter.ColorFormatter
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter_sql]
 class = rhodecode.lib.logging_formatter.ColorFormatterSql
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S

configs/gunicorn_config.py

@@ -1,154 +1,265 b''
 """
-gunicorn config extension and hooks. Sets additional configuration that is
-available post the .ini config.
-
-- workers = ${cpu_number}
-- threads = 1
-- proc_name = ${gunicorn_proc_name}
-- worker_class = sync
-- worker_connections = 10
-- max_requests = 1000
-- max_requests_jitter = 30
-- timeout = 21600
-
+Gunicorn config extension and hooks. This config file adds some extra settings and memory management.
+Gunicorn configuration should be managed by .ini files entries of RhodeCode or VCSServer
 """
 
-import multiprocessing
+import gc
+import os
 import sys
+import math
 import time
-import datetime
 import threading
 import traceback
+import random
 from gunicorn.glogging import Logger
 
 
+def get_workers():
+    import multiprocessing
+    return multiprocessing.cpu_count() * 2 + 1
+
 # GLOBAL
 errorlog = '-'
 accesslog = '-'
-loglevel = 'debug'
-
-# SECURITY
-
-# The maximum size of HTTP request line in bytes.
-# 0 for unlimited
-limit_request_line = 0
-
-# Limit the number of HTTP headers fields in a request.
-# By default this value is 100 and can't be larger than 32768.
-limit_request_fields = 10240
-
-# Limit the allowed size of an HTTP request header field.
-# Value is a positive number or 0.
-# Setting it to 0 will allow unlimited header field sizes.
-limit_request_field_size = 0
-
-
-# Timeout for graceful workers restart.
-# After receiving a restart signal, workers have this much time to finish
-# serving requests. Workers still alive after the timeout (starting from the
-# receipt of the restart signal) are force killed.
-graceful_timeout = 30
-
-
-# The number of seconds to wait for requests on a Keep-Alive connection.
-# Generally set in the 1-5 seconds range.
-keepalive = 2
 
 
 # SERVER MECHANICS
 # None == system temp dir
 # worker_tmp_dir is recommended to be set to some tmpfs
 worker_tmp_dir = None
 tmp_upload_dir = None
 
 # Custom log format
 access_log_format = (
-    '%(t)s [%(p)-8s] GNCRN %(h)-15s rqt:%(L)s %(s)s %(b)-6s "%(m)s:%(U)s %(q)s" usr:%(u)s "%(f)s" "%(a)s"')
+    '%(t)s %(p)s INFO  [GNCRN] %(h)-15s rqt:%(L)s %(s)s %(b)-6s "%(m)s:%(U)s %(q)s" usr:%(u)s "%(f)s" "%(a)s"')
 
 # self adjust workers based on CPU count
-# workers = multiprocessing.cpu_count() * 2 + 1
+# workers = get_workers()
+
+
+def _get_process_rss(pid=None):
+    try:
+        import psutil
+        if pid:
+            proc = psutil.Process(pid)
+        else:
+            proc = psutil.Process()
+        return proc.memory_info().rss
+    except Exception:
+        return None
 
 
-def post_fork(server, worker):
-    server.log.info("[<%-10s>] WORKER spawned", worker.pid)
+def _get_config(ini_path):
+
+    try:
+        import configparser
+    except ImportError:
+        import ConfigParser as configparser
+    try:
+        config = configparser.RawConfigParser()
+        config.read(ini_path)
+        return config
+    except Exception:
+        return None
+
+
+def _time_with_offset(memory_usage_check_interval):
+    return time.time() - random.randint(0, memory_usage_check_interval/2.0)
 
 
 def pre_fork(server, worker):
     pass
 
 
+def post_fork(server, worker):
+
+    # memory spec defaults
+    _memory_max_usage = 0
+    _memory_usage_check_interval = 60
+    _memory_usage_recovery_threshold = 0.8
+
+    ini_path = os.path.abspath(server.cfg.paste)
+    conf = _get_config(ini_path)
+
+    section = 'server:main'
+    if conf and conf.has_section(section):
+
+        if conf.has_option(section, 'memory_max_usage'):
+            _memory_max_usage = conf.getint(section, 'memory_max_usage')
+
+        if conf.has_option(section, 'memory_usage_check_interval'):
+            _memory_usage_check_interval = conf.getint(section, 'memory_usage_check_interval')
+
+        if conf.has_option(section, 'memory_usage_recovery_threshold'):
+            _memory_usage_recovery_threshold = conf.getfloat(section, 'memory_usage_recovery_threshold')
+
+    worker._memory_max_usage = _memory_max_usage
+    worker._memory_usage_check_interval = _memory_usage_check_interval
+    worker._memory_usage_recovery_threshold = _memory_usage_recovery_threshold
+
+    # register memory last check time, with some random offset so we don't recycle all
+    # at once
+    worker._last_memory_check_time = _time_with_offset(_memory_usage_check_interval)
+
+    if _memory_max_usage:
+        server.log.info("[%-10s] WORKER spawned with max memory set at %s", worker.pid,
+                        _format_data_size(_memory_max_usage))
+    else:
+        server.log.info("[%-10s] WORKER spawned", worker.pid)
+
+
 def pre_exec(server):
     server.log.info("Forked child, re-executing.")
 
 
 def on_starting(server):
-    server.log.info("Server is starting.")
+    server_lbl = '{} {}'.format(server.proc_name, server.address)
+    server.log.info("Server %s is starting.", server_lbl)
 
 
 def when_ready(server):
-    server.log.info("Server is ready. Spawning workers")
+    server.log.info("Server %s is ready. Spawning workers", server)
 
 
 def on_reload(server):
     pass
 
 
+def _format_data_size(size, unit="B", precision=1, binary=True):
+    """Format a number using SI units (kilo, mega, etc.).
+
+    ``size``: The number as a float or int.
+
+    ``unit``: The unit name in plural form. Examples: "bytes", "B".
+
+    ``precision``: How many digits to the right of the decimal point. Default
+    is 1.  0 suppresses the decimal point.
+
+    ``binary``: If false, use base-10 decimal prefixes (kilo = K = 1000).
+    If true, use base-2 binary prefixes (kibi = Ki = 1024).
+
+    ``full_name``: If false (default), use the prefix abbreviation ("k" or
+    "Ki").  If true, use the full prefix ("kilo" or "kibi"). If false,
+    use abbreviation ("k" or "Ki").
+
+    """
+
+    if not binary:
+        base = 1000
+        multiples = ('', 'k', 'M', 'G', 'T', 'P', 'E', 'Z', 'Y')
+    else:
+        base = 1024
+        multiples = ('', 'Ki', 'Mi', 'Gi', 'Ti', 'Pi', 'Ei', 'Zi', 'Yi')
+
+    sign = ""
+    if size > 0:
+        m = int(math.log(size, base))
+    elif size < 0:
+        sign = "-"
+        size = -size
+        m = int(math.log(size, base))
+    else:
+        m = 0
+    if m > 8:
+        m = 8
+
+    if m == 0:
+        precision = '%.0f'
+    else:
+        precision = '%%.%df' % precision
+
+    size = precision % (size / math.pow(base, m))
+
+    return '%s%s %s%s' % (sign, size.strip(), multiples[m], unit)
+
+
+def _check_memory_usage(worker):
+    memory_max_usage = worker._memory_max_usage
+    if not memory_max_usage:
+        return
+
+    memory_usage_check_interval = worker._memory_usage_check_interval
+    memory_usage_recovery_threshold = memory_max_usage * worker._memory_usage_recovery_threshold
+
+    elapsed = time.time() - worker._last_memory_check_time
+    if elapsed > memory_usage_check_interval:
+        mem_usage = _get_process_rss()
+        if mem_usage and mem_usage > memory_max_usage:
+            worker.log.info(
+                "memory usage %s > %s, forcing gc",
+                _format_data_size(mem_usage), _format_data_size(memory_max_usage))
+            # Try to clean it up by forcing a full collection.
+            gc.collect()
+            mem_usage = _get_process_rss()
+            if mem_usage > memory_usage_recovery_threshold:
+                # Didn't clean up enough, we'll have to terminate.
+                worker.log.warning(
+                    "memory usage %s > %s after gc, quitting",
+                    _format_data_size(mem_usage), _format_data_size(memory_max_usage))
+                # This will cause worker to auto-restart itself
+                worker.alive = False
+        worker._last_memory_check_time = time.time()
+
+
 def worker_int(worker):
-    worker.log.info("[<%-10s>] worker received INT or QUIT signal", worker.pid)
+    worker.log.info("[%-10s] worker received INT or QUIT signal", worker.pid)
 
     # get traceback info, on worker crash
     id2name = dict([(th.ident, th.name) for th in threading.enumerate()])
     code = []
     for thread_id, stack in sys._current_frames().items():
         code.append(
             "\n# Thread: %s(%d)" % (id2name.get(thread_id, ""), thread_id))
         for fname, lineno, name, line in traceback.extract_stack(stack):
             code.append('File: "%s", line %d, in %s' % (fname, lineno, name))
             if line:
                 code.append("  %s" % (line.strip()))
     worker.log.debug("\n".join(code))
 
 
 def worker_abort(worker):
-    worker.log.info("[<%-10s>] worker received SIGABRT signal", worker.pid)
+    worker.log.info("[%-10s] worker received SIGABRT signal", worker.pid)
 
 
 def worker_exit(server, worker):
-    worker.log.info("[<%-10s>] worker exit", worker.pid)
+    worker.log.info("[%-10s] worker exit", worker.pid)
 
 
 def child_exit(server, worker):
-    worker.log.info("[<%-10s>] worker child exit", worker.pid)
+    worker.log.info("[%-10s] worker child exit", worker.pid)
 
 
 def pre_request(worker, req):
     worker.start_time = time.time()
     worker.log.debug(
         "GNCRN PRE  WORKER [cnt:%s]: %s %s", worker.nr, req.method, req.path)
 
 
 def post_request(worker, req, environ, resp):
     total_time = time.time() - worker.start_time
+    # Gunicorn sometimes has problems with reading the status_code
+    status_code = getattr(resp, 'status_code', '')
     worker.log.debug(
-        "GNCRN POST WORKER [cnt:%s]: %s %s resp: %s, Load Time: %.3fs",
-        worker.nr, req.method, req.path, resp.status_code, total_time)
+        "GNCRN POST WORKER [cnt:%s]: %s %s resp: %s, Load Time: %.4fs",
+        worker.nr, req.method, req.path, status_code, total_time)
+    _check_memory_usage(worker)
 
 
 class RhodeCodeLogger(Logger):
     """
     Custom Logger that allows some customization that gunicorn doesn't allow
     """
 
     datefmt = r"%Y-%m-%d %H:%M:%S"
 
     def __init__(self, cfg):
         Logger.__init__(self, cfg)
 
     def now(self):
         """ return date in RhodeCode Log format """
         now = time.time()
         msecs = int((now - long(now)) * 1000)
         return time.strftime(self.datefmt, time.localtime(now)) + '.{0:03d}'.format(msecs)
 
 
 logger_class = RhodeCodeLogger

configs/production.ini

@@ -1,714 +1,784 b''
-
+## -*- coding: utf-8 -*-
 
-################################################################################
-##                RHODECODE COMMUNITY  EDITION CONFIGURATION                  ##
-################################################################################
+; #########################################
+; RHODECODE COMMUNITY EDITION CONFIGURATION
+; #########################################
 
 [DEFAULT]
-## Debug flag sets all loggers to debug, and enables request tracking
+; Debug flag sets all loggers to debug, and enables request tracking
 debug = false
 
-################################################################################
-##                            EMAIL CONFIGURATION                             ##
-## Uncomment and replace with the email address which should receive          ##
-## any error reports after an application crash                               ##
-## Additionally these settings will be used by the RhodeCode mailing system   ##
-################################################################################
+; ########################################################################
+; EMAIL CONFIGURATION
+; These settings will be used by the RhodeCode mailing system
+; ########################################################################
 
-## prefix all emails subjects with given prefix, helps filtering out emails
+; prefix all emails subjects with given prefix, helps filtering out emails
 #email_prefix = [RhodeCode]
 
-## email FROM address all mails will be sent
+; email FROM address all mails will be sent
 #app_email_from = rhodecode-noreply@localhost
 
 #smtp_server = mail.server.com
 #smtp_username =
 #smtp_password =
 #smtp_port =
 #smtp_use_tls = false
 #smtp_use_ssl = true
 
 [server:main]
-## COMMON ##
+; COMMON HOST/IP CONFIG
 host = 127.0.0.1
 port = 5000
 
-###########################################################
-## WAITRESS WSGI SERVER - Recommended for Development  ####
-###########################################################
+
+; ###########################
+; GUNICORN APPLICATION SERVER
+; ###########################
+
+; run with gunicorn --log-config rhodecode.ini --paste rhodecode.ini
+
+; Module to use, this setting shouldn't be changed
+use = egg:gunicorn#main
+
+; Sets the number of process workers. More workers means more concurrent connections
+; RhodeCode can handle at the same time. Each additional worker also it increases
+; memory usage as each has it's own set of caches.
+; Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
+; than 8-10 unless for really big deployments .e.g 700-1000 users.
+; `instance_id = *` must be set in the [app:main] section below (which is the default)
+; when using more than 1 worker.
+workers = 2
+
+; Gunicorn access log level
+loglevel = info
+
+; Process name visible in process list
+proc_name = rhodecode
+
+; Type of worker class, one of `sync`, `gevent`
+; Recommended type is `gevent`
+worker_class = gevent
+
+; The maximum number of simultaneous clients per worker. Valid only for gevent
+worker_connections = 10
+
+; Max number of requests that worker will handle before being gracefully restarted.
+; Prevents memory leaks, jitter adds variability so not all workers are restarted at once.
+max_requests = 1000
+max_requests_jitter = 30
 
-#use = egg:waitress#main
-## number of worker threads
-#threads = 5
-## MAX BODY SIZE 100GB
-#max_request_body_size = 107374182400
-## Use poll instead of select, fixes file descriptors limits problems.
-## May not work on old windows systems.
-#asyncore_use_poll = true
+; Amount of time a worker can spend with handling a request before it
+; gets killed and restarted. By default set to 21600 (6hrs)
+; Examples: 1800 (30min), 3600 (1hr), 7200 (2hr), 43200 (12h)
+timeout = 21600
+
+; The maximum size of HTTP request line in bytes.
+; 0 for unlimited
+limit_request_line = 0
+
+; Limit the number of HTTP headers fields in a request.
+; By default this value is 100 and can't be larger than 32768.
+limit_request_fields = 32768
+
+; Limit the allowed size of an HTTP request header field.
+; Value is a positive number or 0.
+; Setting it to 0 will allow unlimited header field sizes.
+limit_request_field_size = 0
+
+; Timeout for graceful workers restart.
+; After receiving a restart signal, workers have this much time to finish
+; serving requests. Workers still alive after the timeout (starting from the
+; receipt of the restart signal) are force killed.
+; Examples: 1800 (30min), 3600 (1hr), 7200 (2hr), 43200 (12h)
+graceful_timeout = 3600
+
+# The number of seconds to wait for requests on a Keep-Alive connection.
+# Generally set in the 1-5 seconds range.
+keepalive = 2
+
+; Maximum memory usage that each worker can use before it will receive a
+; graceful restart signal 0 = memory monitoring is disabled
+; Examples: 268435456 (256MB), 536870912 (512MB)
+; 1073741824 (1GB), 2147483648 (2GB), 4294967296 (4GB)
+memory_max_usage = 0
+
+; How often in seconds to check for memory usage for each gunicorn worker
+memory_usage_check_interval = 60
+
+; Threshold value for which we don't recycle worker if GarbageCollection
+; frees up enough resources. Before each restart we try to run GC on worker
+; in case we get enough free memory after that, restart will not happen.
+memory_usage_recovery_threshold = 0.8
 
 
-##########################
-## GUNICORN WSGI SERVER ##
-##########################
-## run with gunicorn --log-config rhodecode.ini --paste rhodecode.ini
-
-use = egg:gunicorn#main
-## Sets the number of process workers. More workers means more concurrent connections
-## RhodeCode can handle at the same time. Each additional worker also it increases
-## memory usage as each has it's own set of caches.
-## Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
-## than 8-10 unless for really big deployments .e.g 700-1000 users.
-## `instance_id = *` must be set in the [app:main] section below (which is the default)
-## when using more than 1 worker.
-workers = 2
-## process name visible in process list
-proc_name = rhodecode
-## type of worker class, one of sync, gevent
-## recommended for bigger setup is using of of other than sync one
-worker_class = gevent
-## The maximum number of simultaneous clients. Valid only for Gevent
-worker_connections = 10
-## max number of requests that worker will handle before being gracefully
-## restarted, could prevent memory leaks
-max_requests = 1000
-max_requests_jitter = 30
-## amount of time a worker can spend with handling a request before it
-## gets killed and restarted. Set to 6hrs
-timeout = 21600
-
-
-## prefix middleware for RhodeCode.
-## recommended when using proxy setup.
-## allows to set RhodeCode under a prefix in server.
-## eg https://server.com/custom_prefix. Enable `filter-with =` option below as well.
-## And set your prefix like: `prefix = /custom_prefix`
-## be sure to also set beaker.session.cookie_path = /custom_prefix if you need
-## to make your cookies only work on prefix url
+; Prefix middleware for RhodeCode.
+; recommended when using proxy setup.
+; allows to set RhodeCode under a prefix in server.
+; eg https://server.com/custom_prefix. Enable `filter-with =` option below as well.
+; And set your prefix like: `prefix = /custom_prefix`
+; be sure to also set beaker.session.cookie_path = /custom_prefix if you need
+; to make your cookies only work on prefix url
 [filter:proxy-prefix]
 use = egg:PasteDeploy#prefix
 prefix = /
 
 [app:main]
-## The %(here)s variable will be replaced with the absolute path of parent directory
-## of this file
-## In addition ENVIRONMENT variables usage is possible, e.g
-## sqlalchemy.db1.url = {ENV_RC_DB_URL}
+; The %(here)s variable will be replaced with the absolute path of parent directory
+; of this file
+; In addition ENVIRONMENT variables usage is possible, e.g
+; sqlalchemy.db1.url = {ENV_RC_DB_URL}
 
 use = egg:rhodecode-enterprise-ce
 
-## enable proxy prefix middleware, defined above
+; enable proxy prefix middleware, defined above
 #filter-with = proxy-prefix
 
-## encryption key used to encrypt social plugin tokens,
-## remote_urls with credentials etc, if not set it defaults to
-## `beaker.session.secret`
+; encryption key used to encrypt social plugin tokens,
+; remote_urls with credentials etc, if not set it defaults to
+; `beaker.session.secret`
 #rhodecode.encrypted_values.secret =
 
-## decryption strict mode (enabled by default). It controls if decryption raises
-## `SignatureVerificationError` in case of wrong key, or damaged encryption data.
+; decryption strict mode (enabled by default). It controls if decryption raises
+; `SignatureVerificationError` in case of wrong key, or damaged encryption data.
 #rhodecode.encrypted_values.strict = false
 
-## Pick algorithm for encryption. Either fernet (more secure) or aes (default)
-## fernet is safer, and we strongly recommend switching to it.
-## Due to backward compatibility aes is used as default.
+; Pick algorithm for encryption. Either fernet (more secure) or aes (default)
+; fernet is safer, and we strongly recommend switching to it.
+; Due to backward compatibility aes is used as default.
 #rhodecode.encrypted_values.algorithm = fernet
 
-## return gzipped responses from RhodeCode (static files/application)
+; Return gzipped responses from RhodeCode (static files/application)
 gzip_responses = false
 
-## auto-generate javascript routes file on startup
+; Auto-generate javascript routes file on startup
 generate_js_files = false
 
-## System global default language.
-## All available languages: en(default), be, de, es, fr, it, ja, pl, pt, ru, zh
+; System global default language.
+; All available languages: en (default), be, de, es, fr, it, ja, pl, pt, ru, zh
 lang = en
 
-## Perform a full repository scan and import on each server start.
-## Settings this to true could lead to very long startup time.
+; Perform a full repository scan and import on each server start.
+; Settings this to true could lead to very long startup time.
 startup.import_repos = false
 
-## Uncomment and set this path to use archive download cache.
-## Once enabled, generated archives will be cached at this location
-## and served from the cache during subsequent requests for the same archive of
-## the repository.
+; Uncomment and set this path to use archive download cache.
+; Once enabled, generated archives will be cached at this location
+; and served from the cache during subsequent requests for the same archive of
+; the repository.
 #archive_cache_dir = /tmp/tarballcache
 
-## URL at which the application is running. This is used for Bootstrapping
-## requests in context when no web request is available. Used in ishell, or
-## SSH calls. Set this for events to receive proper url for SSH calls.
+; URL at which the application is running. This is used for Bootstrapping
+; requests in context when no web request is available. Used in ishell, or
+; SSH calls. Set this for events to receive proper url for SSH calls.
 app.base_url = http://rhodecode.local
 
-## Unique application ID. Should be a random unique string for security.
+; Unique application ID. Should be a random unique string for security.
 app_instance_uuid = rc-production
 
-## Cut off limit for large diffs (size in bytes). If overall diff size on
-## commit, or pull request exceeds this limit this diff will be displayed
-## partially. E.g 512000 == 512Kb
+; Cut off limit for large diffs (size in bytes). If overall diff size on
+; commit, or pull request exceeds this limit this diff will be displayed
+; partially. E.g 512000 == 512Kb
 cut_off_limit_diff = 512000
 
-## Cut off limit for large files inside diffs (size in bytes). Each individual
-## file inside diff which exceeds this limit will be displayed partially.
-##  E.g 128000 == 128Kb
+; Cut off limit for large files inside diffs (size in bytes). Each individual
+; file inside diff which exceeds this limit will be displayed partially.
+;  E.g 128000 == 128Kb
 cut_off_limit_file = 128000
 
-## use cached version of vcs repositories everywhere. Recommended to be `true`
+; Use cached version of vcs repositories everywhere. Recommended to be `true`
 vcs_full_cache = true
 
-## Force https in RhodeCode, fixes https redirects, assumes it's always https.
-## Normally this is controlled by proper http flags sent from http server
+; Force https in RhodeCode, fixes https redirects, assumes it's always https.
+; Normally this is controlled by proper flags sent from http server such as Nginx or Apache
 force_https = false
 
-## use Strict-Transport-Security headers
+; use Strict-Transport-Security headers
 use_htsts = false
 
-## git rev filter option, --all is the default filter, if you need to
-## hide all refs in changelog switch this to --branches --tags
-git_rev_filter = --branches --tags
-
-# Set to true if your repos are exposed using the dumb protocol
+; Set to true if your repos are exposed using the dumb protocol
 git_update_server_info = false
 
-## RSS/ATOM feed options
+; RSS/ATOM feed options
 rss_cut_off_limit = 256000
 rss_items_per_page = 10
 rss_include_diff = false
 
-## gist URL alias, used to create nicer urls for gist. This should be an
-## url that does rewrites to _admin/gists/{gistid}.
-## example: http://gist.rhodecode.org/{gistid}. Empty means use the internal
-## RhodeCode url, ie. http[s]://rhodecode.server/_admin/gists/{gistid}
+; gist URL alias, used to create nicer urls for gist. This should be an
+; url that does rewrites to _admin/gists/{gistid}.
+; example: http://gist.rhodecode.org/{gistid}. Empty means use the internal
+; RhodeCode url, ie. http[s]://rhodecode.server/_admin/gists/{gistid}
 gist_alias_url =
 
-## List of views (using glob pattern syntax) that AUTH TOKENS could be
-## used for access.
-## Adding ?auth_token=TOKEN_HASH to the url authenticates this request as if it
-## came from the the logged in user who own this authentication token.
-## Additionally @TOKEN syntax can be used to bound the view to specific
-## authentication token. Such view would be only accessible when used together
-## with this authentication token
-##
-## list of all views can be found under `/_admin/permissions/auth_token_access`
-## The list should be "," separated and on a single line.
-##
-## Most common views to enable:
+; List of views (using glob pattern syntax) that AUTH TOKENS could be
+; used for access.
+; Adding ?auth_token=TOKEN_HASH to the url authenticates this request as if it
+; came from the the logged in user who own this authentication token.
+; Additionally @TOKEN syntax can be used to bound the view to specific
+; authentication token. Such view would be only accessible when used together
+; with this authentication token
+; list of all views can be found under `/_admin/permissions/auth_token_access`
+; The list should be "," separated and on a single line.
+; Most common views to enable:
+
 #    RepoCommitsView:repo_commit_download
 #    RepoCommitsView:repo_commit_patch
 #    RepoCommitsView:repo_commit_raw
 #    RepoCommitsView:repo_commit_raw@TOKEN
 #    RepoFilesView:repo_files_diff
 #    RepoFilesView:repo_archivefile
 #    RepoFilesView:repo_file_raw
 #    GistView:*
 api_access_controllers_whitelist =
 
-## Default encoding used to convert from and to unicode
-## can be also a comma separated list of encoding in case of mixed encodings
+; Default encoding used to convert from and to unicode
+; can be also a comma separated list of encoding in case of mixed encodings
 default_encoding = UTF-8
 
-## instance-id prefix
-## a prefix key for this instance used for cache invalidation when running
-## multiple instances of RhodeCode, make sure it's globally unique for
-## all running RhodeCode instances. Leave empty if you don't use it
+; instance-id prefix
+; a prefix key for this instance used for cache invalidation when running
+; multiple instances of RhodeCode, make sure it's globally unique for
+; all running RhodeCode instances. Leave empty if you don't use it
 instance_id =
 
-## Fallback authentication plugin. Set this to a plugin ID to force the usage
-## of an authentication plugin also if it is disabled by it's settings.
-## This could be useful if you are unable to log in to the system due to broken
-## authentication settings. Then you can enable e.g. the internal RhodeCode auth
-## module to log in again and fix the settings.
-##
-## Available builtin plugin IDs (hash is part of the ID):
-## egg:rhodecode-enterprise-ce#rhodecode
-## egg:rhodecode-enterprise-ce#pam
-## egg:rhodecode-enterprise-ce#ldap
-## egg:rhodecode-enterprise-ce#jasig_cas
-## egg:rhodecode-enterprise-ce#headers
-## egg:rhodecode-enterprise-ce#crowd
+; Fallback authentication plugin. Set this to a plugin ID to force the usage
+; of an authentication plugin also if it is disabled by it's settings.
+; This could be useful if you are unable to log in to the system due to broken
+; authentication settings. Then you can enable e.g. the internal RhodeCode auth
+; module to log in again and fix the settings.
+; Available builtin plugin IDs (hash is part of the ID):
+; egg:rhodecode-enterprise-ce#rhodecode
+; egg:rhodecode-enterprise-ce#pam
+; egg:rhodecode-enterprise-ce#ldap
+; egg:rhodecode-enterprise-ce#jasig_cas
+; egg:rhodecode-enterprise-ce#headers
+; egg:rhodecode-enterprise-ce#crowd
+
 #rhodecode.auth_plugin_fallback = egg:rhodecode-enterprise-ce#rhodecode
 
-## alternative return HTTP header for failed authentication. Default HTTP
-## response is 401 HTTPUnauthorized. Currently HG clients have troubles with
-## handling that causing a series of failed authentication calls.
-## Set this variable to 403 to return HTTPForbidden, or any other HTTP code
-## This will be served instead of default 401 on bad authentication
+; Flag to control loading of legacy plugins in py:/path format
+auth_plugin.import_legacy_plugins = true
+
+; alternative return HTTP header for failed authentication. Default HTTP
+; response is 401 HTTPUnauthorized. Currently HG clients have troubles with
+; handling that causing a series of failed authentication calls.
+; Set this variable to 403 to return HTTPForbidden, or any other HTTP code
+; This will be served instead of default 401 on bad authentication
 auth_ret_code =
 
-## use special detection method when serving auth_ret_code, instead of serving
-## ret_code directly, use 401 initially (Which triggers credentials prompt)
-## and then serve auth_ret_code to clients
+; use special detection method when serving auth_ret_code, instead of serving
+; ret_code directly, use 401 initially (Which triggers credentials prompt)
+; and then serve auth_ret_code to clients
 auth_ret_code_detection = false
 
-## locking return code. When repository is locked return this HTTP code. 2XX
-## codes don't break the transactions while 4XX codes do
+; locking return code. When repository is locked return this HTTP code. 2XX
+; codes don't break the transactions while 4XX codes do
 lock_ret_code = 423
 
-## allows to change the repository location in settings page
+; allows to change the repository location in settings page
 allow_repo_location_change = true
 
-## allows to setup custom hooks in settings page
+; allows to setup custom hooks in settings page
 allow_custom_hooks_settings = true
 
-## Generated license token required for EE edition license.
-## New generated token value can be found in Admin > settings > license page.
+; Generated license token required for EE edition license.
+; New generated token value can be found in Admin > settings > license page.
 license_token =
 
-## supervisor connection uri, for managing supervisor and logs.
+; This flag hides sensitive information on the license page such as token, and license data
+license.hide_license_info = false
+
+; supervisor connection uri, for managing supervisor and logs.
 supervisor.uri =
-## supervisord group name/id we only want this RC instance to handle
+
+; supervisord group name/id we only want this RC instance to handle
 supervisor.group_id = prod
 
-## Display extended labs settings
+; Display extended labs settings
 labs_settings_active = true
 
-## Custom exception store path, defaults to TMPDIR
-## This is used to store exception from RhodeCode in shared directory
+; Custom exception store path, defaults to TMPDIR
+; This is used to store exception from RhodeCode in shared directory
 #exception_tracker.store_path =
 
-## File store configuration. This is used to store and serve uploaded files
+; File store configuration. This is used to store and serve uploaded files
 file_store.enabled = true
-## Storage backend, available options are: local
+
+; Storage backend, available options are: local
 file_store.backend = local
-## path to store the uploaded binaries
+
+; path to store the uploaded binaries
 file_store.storage_path = %(here)s/data/file_store
 
 
-####################################
-###        CELERY CONFIG        ####
-####################################
-## run: /path/to/celery worker \
-##     -E --beat --app rhodecode.lib.celerylib.loader \
-##     --scheduler rhodecode.lib.celerylib.scheduler.RcScheduler \
-##     --loglevel DEBUG --ini /path/to/rhodecode.ini
+; #############
+; CELERY CONFIG
+; #############
+
+; manually run celery: /path/to/celery worker -E --beat --app rhodecode.lib.celerylib.loader --scheduler rhodecode.lib.celerylib.scheduler.RcScheduler --loglevel DEBUG --ini /path/to/rhodecode.ini
 
 use_celery = false
 
-## connection url to the message broker (default rabbitmq)
-celery.broker_url = amqp://rabbitmq:qweqwe@localhost:5672/rabbitmqhost
+; connection url to the message broker (default redis)
+celery.broker_url = redis://localhost:6379/8
 
-## maximum tasks to execute before worker restart
+; rabbitmq example
+#celery.broker_url = amqp://rabbitmq:qweqwe@localhost:5672/rabbitmqhost
+
+; maximum tasks to execute before worker restart
 celery.max_tasks_per_child = 100
 
-## tasks will never be sent to the queue, but executed locally instead.
+; tasks will never be sent to the queue, but executed locally instead.
 celery.task_always_eager = false
 
-#####################################
-###         DOGPILE CACHE        ####
-#####################################
-## Default cache dir for caches.  Putting this into a ramdisk
-## can boost performance, eg. /tmpfs/data_ramdisk, however this directory might require
-## large amount of space
+; #############
+; DOGPILE CACHE
+; #############
+
+; Default cache dir for caches. Putting this into a ramdisk can boost performance.
+; eg. /tmpfs/data_ramdisk, however this directory might require large amount of space
 cache_dir = %(here)s/data
 
-## `cache_perms` cache settings for permission tree, auth TTL.
+; *********************************************
+; `sql_cache_short` cache for heavy SQL queries
+; Only supported backend is `memory_lru`
+; *********************************************
+rc_cache.sql_cache_short.backend = dogpile.cache.rc.memory_lru
+rc_cache.sql_cache_short.expiration_time = 30
+
+
+; *****************************************************
+; `cache_repo_longterm` cache for repo object instances
+; Only supported backend is `memory_lru`
+; *****************************************************
+rc_cache.cache_repo_longterm.backend = dogpile.cache.rc.memory_lru
+; by default we use 30 Days, cache is still invalidated on push
+rc_cache.cache_repo_longterm.expiration_time = 2592000
+; max items in LRU cache, set to smaller number to save memory, and expire last used caches
+rc_cache.cache_repo_longterm.max_size = 10000
+
+
+; *************************************************
+; `cache_perms` cache for permission tree, auth TTL
+; *************************************************
 rc_cache.cache_perms.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_perms.expiration_time = 300
+; file cache store path. Defaults to `cache_dir =` value or tempdir if both values are not set
+#rc_cache.cache_perms.arguments.filename = /tmp/cache_perms.db
 
-## alternative `cache_perms` redis backend with distributed lock
+; alternative `cache_perms` redis backend with distributed lock
 #rc_cache.cache_perms.backend = dogpile.cache.rc.redis
 #rc_cache.cache_perms.expiration_time = 300
-## redis_expiration_time needs to be greater then expiration_time
+
+; redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_perms.arguments.redis_expiration_time = 7200
-#rc_cache.cache_perms.arguments.socket_timeout = 30
+
 #rc_cache.cache_perms.arguments.host = localhost
 #rc_cache.cache_perms.arguments.port = 6379
 #rc_cache.cache_perms.arguments.db = 0
-## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
+#rc_cache.cache_perms.arguments.socket_timeout = 30
+; more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_perms.arguments.distributed_lock = true
 
-## `cache_repo` cache settings for FileTree, Readme, RSS FEEDS
+
+; ***************************************************
+; `cache_repo` cache for file tree, Readme, RSS FEEDS
+; ***************************************************
 rc_cache.cache_repo.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_repo.expiration_time = 2592000
+; file cache store path. Defaults to `cache_dir =` value or tempdir if both values are not set
+#rc_cache.cache_repo.arguments.filename = /tmp/cache_repo.db
 
-## alternative `cache_repo` redis backend with distributed lock
+; alternative `cache_repo` redis backend with distributed lock
 #rc_cache.cache_repo.backend = dogpile.cache.rc.redis
 #rc_cache.cache_repo.expiration_time = 2592000
-## redis_expiration_time needs to be greater then expiration_time
+
+; redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_repo.arguments.redis_expiration_time = 2678400
-#rc_cache.cache_repo.arguments.socket_timeout = 30
+
 #rc_cache.cache_repo.arguments.host = localhost
 #rc_cache.cache_repo.arguments.port = 6379
 #rc_cache.cache_repo.arguments.db = 1
-## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
+#rc_cache.cache_repo.arguments.socket_timeout = 30
+; more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_repo.arguments.distributed_lock = true
 
-## cache settings for SQL queries, this needs to use memory type backend
-rc_cache.sql_cache_short.backend = dogpile.cache.rc.memory_lru
-rc_cache.sql_cache_short.expiration_time = 30
 
-## `cache_repo_longterm` cache for repo object instances, this needs to use memory
-## type backend as the objects kept are not pickle serializable
-rc_cache.cache_repo_longterm.backend = dogpile.cache.rc.memory_lru
-## by default we use 96H, this is using invalidation on push anyway
-rc_cache.cache_repo_longterm.expiration_time = 345600
-## max items in LRU cache, reduce this number to save memory, and expire last used
-## cached objects
-rc_cache.cache_repo_longterm.max_size = 10000
+; ##############
+; BEAKER SESSION
+; ##############
 
-
-####################################
-###       BEAKER SESSION        ####
-####################################
-
-## .session.type is type of storage options for the session, current allowed
-## types are file, ext:memcached, ext:redis, ext:database, and memory (default).
+; beaker.session.type is type of storage options for the logged users sessions. Current allowed
+; types are file, ext:redis, ext:database, ext:memcached, and memory (default if not specified).
+; Fastest ones are Redis and ext:database
 beaker.session.type = file
 beaker.session.data_dir = %(here)s/data/sessions
 
-## redis sessions
+; Redis based sessions
 #beaker.session.type = ext:redis
 #beaker.session.url = redis://127.0.0.1:6379/2
 
-## db based session, fast, and allows easy management over logged in users
+; DB based session, fast, and allows easy management over logged in users
 #beaker.session.type = ext:database
 #beaker.session.table_name = db_session
 #beaker.session.sa.url = postgresql://postgres:secret@localhost/rhodecode
 #beaker.session.sa.url = mysql://root:secret@127.0.0.1/rhodecode
 #beaker.session.sa.pool_recycle = 3600
 #beaker.session.sa.echo = false
 
 beaker.session.key = rhodecode
 beaker.session.secret = production-rc-uytcxaz
 beaker.session.lock_dir = %(here)s/data/sessions/lock
 
-## Secure encrypted cookie. Requires AES and AES python libraries
-## you must disable beaker.session.secret to use this
+; Secure encrypted cookie. Requires AES and AES python libraries
+; you must disable beaker.session.secret to use this
 #beaker.session.encrypt_key = key_for_encryption
 #beaker.session.validate_key = validation_key
 
-## sets session as invalid(also logging out user) if it haven not been
-## accessed for given amount of time in seconds
+; Sets session as invalid (also logging out user) if it haven not been
+; accessed for given amount of time in seconds
 beaker.session.timeout = 2592000
 beaker.session.httponly = true
-## Path to use for the cookie. Set to prefix if you use prefix middleware
+
+; Path to use for the cookie. Set to prefix if you use prefix middleware
 #beaker.session.cookie_path = /custom_prefix
 
-## uncomment for https secure cookie
+; Set https secure cookie
 beaker.session.secure = false
 
-## auto save the session to not to use .save()
-beaker.session.auto = false
-
-## default cookie expiration time in seconds, set to `true` to set expire
-## at browser close
+; default cookie expiration time in seconds, set to `true` to set expire
+; at browser close
 #beaker.session.cookie_expires = 3600
 
-###################################
-## SEARCH INDEXING CONFIGURATION ##
-###################################
-## Full text search indexer is available in rhodecode-tools under
-## `rhodecode-tools index` command
+; #############################
+; SEARCH INDEXING CONFIGURATION
+; #############################
 
-## WHOOSH Backend, doesn't require additional services to run
-## it works good with few dozen repos
+; Full text search indexer is available in rhodecode-tools under
+; `rhodecode-tools index` command
+
+; WHOOSH Backend, doesn't require additional services to run
+; it works good with few dozen repos
 search.module = rhodecode.lib.index.whoosh
 search.location = %(here)s/data/index
 
-########################################
-###    CHANNELSTREAM CONFIG         ####
-########################################
-## channelstream enables persistent connections and live notification
-## in the system. It's also used by the chat system
+; ####################
+; CHANNELSTREAM CONFIG
+; ####################
+
+; channelstream enables persistent connections and live notification
+; in the system. It's also used by the chat system
 
 channelstream.enabled = false
 
-## server address for channelstream server on the backend
+; server address for channelstream server on the backend
 channelstream.server = 127.0.0.1:9800
 
-## location of the channelstream server from outside world
-## use ws:// for http or wss:// for https. This address needs to be handled
-## by external HTTP server such as Nginx or Apache
-## see Nginx/Apache configuration examples in our docs
+; location of the channelstream server from outside world
+; use ws:// for http or wss:// for https. This address needs to be handled
+; by external HTTP server such as Nginx or Apache
+; see Nginx/Apache configuration examples in our docs
 channelstream.ws_url = ws://rhodecode.yourserver.com/_channelstream
 channelstream.secret = secret
 channelstream.history.location = %(here)s/channelstream_history
 
-## Internal application path that Javascript uses to connect into.
-## If you use proxy-prefix the prefix should be added before /_channelstream
+; Internal application path that Javascript uses to connect into.
+; If you use proxy-prefix the prefix should be added before /_channelstream
 channelstream.proxy_path = /_channelstream
 
 
-###################################
-##       APPENLIGHT CONFIG       ##
-###################################
+; ##############################
+; MAIN RHODECODE DATABASE CONFIG
+; ##############################
+
+#sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
+#sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
+#sqlalchemy.db1.url = mysql://root:qweqwe@localhost/rhodecode?charset=utf8
+; pymysql is an alternative driver for MySQL, use in case of problems with default one
+#sqlalchemy.db1.url = mysql+pymysql://root:qweqwe@localhost/rhodecode
+
+sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
+
+; see sqlalchemy docs for other advanced settings
+; print the sql statements to output
+sqlalchemy.db1.echo = false
+
+; recycle the connections after this amount of seconds
+sqlalchemy.db1.pool_recycle = 3600
+sqlalchemy.db1.convert_unicode = true
+
+; the number of connections to keep open inside the connection pool.
+; 0 indicates no limit
+#sqlalchemy.db1.pool_size = 5
+
+; The number of connections to allow in connection pool "overflow", that is
+; connections that can be opened above and beyond the pool_size setting,
+; which defaults to five.
+#sqlalchemy.db1.max_overflow = 10
+
+; Connection check ping, used to detect broken database connections
+; could be enabled to better handle cases if MySQL has gone away errors
+#sqlalchemy.db1.ping_connection = true
+
+; ##########
+; VCS CONFIG
+; ##########
+vcs.server.enable = true
+vcs.server = localhost:9900
+
+; Web server connectivity protocol, responsible for web based VCS operations
+; Available protocols are:
+; `http` - use http-rpc backend (default)
+vcs.server.protocol = http
+
+; Push/Pull operations protocol, available options are:
+; `http` - use http-rpc backend (default)
+vcs.scm_app_implementation = http
+
+; Push/Pull operations hooks protocol, available options are:
+; `http` - use http-rpc backend (default)
+vcs.hooks.protocol = http
+
+; Host on which this instance is listening for hooks. If vcsserver is in other location
+; this should be adjusted.
+vcs.hooks.host = 127.0.0.1
+
+; Start VCSServer with this instance as a subprocess, useful for development
+vcs.start_server = false
+
+; List of enabled VCS backends, available options are:
+; `hg`  - mercurial
+; `git` - git
+; `svn` - subversion
+vcs.backends = hg, git, svn
+
+; Wait this number of seconds before killing connection to the vcsserver
+vcs.connection_timeout = 3600
+
+; Compatibility version when creating SVN repositories. Defaults to newest version when commented out.
+; Available options are: pre-1.4-compatible, pre-1.5-compatible, pre-1.6-compatible, pre-1.8-compatible, pre-1.9-compatible
+#vcs.svn.compatible_version = pre-1.8-compatible
+
 
-## Appenlight is tailored to work with RhodeCode, see
-## http://appenlight.com for details how to obtain an account
+; ####################################################
+; Subversion proxy support (mod_dav_svn)
+; Maps RhodeCode repo groups into SVN paths for Apache
+; ####################################################
+
+; Enable or disable the config file generation.
+svn.proxy.generate_config = false
+
+; Generate config file with `SVNListParentPath` set to `On`.
+svn.proxy.list_parent_path = true
+
+; Set location and file name of generated config file.
+svn.proxy.config_file_path = %(here)s/mod_dav_svn.conf
+
+; alternative mod_dav config template. This needs to be a valid mako template
+; Example template can be found in the source code:
+; rhodecode/apps/svn_support/templates/mod-dav-svn.conf.mako
+#svn.proxy.config_template = ~/.rccontrol/enterprise-1/custom_svn_conf.mako
+
+; Used as a prefix to the `Location` block in the generated config file.
+; In most cases it should be set to `/`.
+svn.proxy.location_root = /
+
+; Command to reload the mod dav svn configuration on change.
+; Example: `/etc/init.d/apache2 reload` or /home/USER/apache_reload.sh
+; Make sure user who runs RhodeCode process is allowed to reload Apache
+#svn.proxy.reload_cmd = /etc/init.d/apache2 reload
+
+; If the timeout expires before the reload command finishes, the command will
+; be killed. Setting it to zero means no timeout. Defaults to 10 seconds.
+#svn.proxy.reload_timeout = 10
+
+; ####################
+; SSH Support Settings
+; ####################
 
-## Appenlight integration enabled
+; Defines if a custom authorized_keys file should be created and written on
+; any change user ssh keys. Setting this to false also disables possibility
+; of adding SSH keys by users from web interface. Super admins can still
+; manage SSH Keys.
+ssh.generate_authorized_keyfile = false
+
+; Options for ssh, default is `no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding`
+# ssh.authorized_keys_ssh_opts =
+
+; Path to the authorized_keys file where the generate entries are placed.
+; It is possible to have multiple key files specified in `sshd_config` e.g.
+; AuthorizedKeysFile %h/.ssh/authorized_keys %h/.ssh/authorized_keys_rhodecode
+ssh.authorized_keys_file_path = ~/.ssh/authorized_keys_rhodecode
+
+; Command to execute the SSH wrapper. The binary is available in the
+; RhodeCode installation directory.
+; e.g ~/.rccontrol/community-1/profile/bin/rc-ssh-wrapper
+ssh.wrapper_cmd = ~/.rccontrol/community-1/rc-ssh-wrapper
+
+; Allow shell when executing the ssh-wrapper command
+ssh.wrapper_cmd_allow_shell = false
+
+; Enables logging, and detailed output send back to the client during SSH
+; operations. Useful for debugging, shouldn't be used in production.
+ssh.enable_debug_logging = false
+
+; Paths to binary executable, by default they are the names, but we can
+; override them if we want to use a custom one
+ssh.executable.hg = ~/.rccontrol/vcsserver-1/profile/bin/hg
+ssh.executable.git = ~/.rccontrol/vcsserver-1/profile/bin/git
+ssh.executable.svn = ~/.rccontrol/vcsserver-1/profile/bin/svnserve
+
+; Enables SSH key generator web interface. Disabling this still allows users
+; to add their own keys.
+ssh.enable_ui_key_generator = true
+
+
+; #################
+; APPENLIGHT CONFIG
+; #################
+
+; Appenlight is tailored to work with RhodeCode, see
+; http://appenlight.rhodecode.com for details how to obtain an account
+
+; Appenlight integration enabled
 appenlight = false
 
 appenlight.server_url = https://api.appenlight.com
 appenlight.api_key = YOUR_API_KEY
 #appenlight.transport_config = https://api.appenlight.com?threaded=1&timeout=5
 
-## used for JS client
+; used for JS client
 appenlight.api_public_key = YOUR_API_PUBLIC_KEY
 
-## TWEAK AMOUNT OF INFO SENT HERE
+; TWEAK AMOUNT OF INFO SENT HERE
 
-## enables 404 error logging (default False)
+; enables 404 error logging (default False)
 appenlight.report_404 = false
 
-## time in seconds after request is considered being slow (default 1)
+; time in seconds after request is considered being slow (default 1)
 appenlight.slow_request_time = 1
 
-## record slow requests in application
-## (needs to be enabled for slow datastore recording and time tracking)
+; record slow requests in application
+; (needs to be enabled for slow datastore recording and time tracking)
 appenlight.slow_requests = true
 
-## enable hooking to application loggers
+; enable hooking to application loggers
 appenlight.logging = true
 
-## minimum log level for log capture
+; minimum log level for log capture
 appenlight.logging.level = WARNING
 
-## send logs only from erroneous/slow requests
-## (saves API quota for intensive logging)
+; send logs only from erroneous/slow requests
+; (saves API quota for intensive logging)
 appenlight.logging_on_error = false
 
-## list of additional keywords that should be grabbed from environ object
-## can be string with comma separated list of words in lowercase
-## (by default client will always send following info:
-## 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
-## start with HTTP* this list be extended with additional keywords here
+; list of additional keywords that should be grabbed from environ object
+; can be string with comma separated list of words in lowercase
+; (by default client will always send following info:
+; 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
+; start with HTTP* this list be extended with additional keywords here
 appenlight.environ_keys_whitelist =
 
-## list of keywords that should be blanked from request object
-## can be string with comma separated list of words in lowercase
-## (by default client will always blank keys that contain following words
-## 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
-## this list be extended with additional keywords set here
+; list of keywords that should be blanked from request object
+; can be string with comma separated list of words in lowercase
+; (by default client will always blank keys that contain following words
+; 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
+; this list be extended with additional keywords set here
 appenlight.request_keys_blacklist =
 
-## list of namespaces that should be ignores when gathering log entries
-## can be string with comma separated list of namespaces
-## (by default the client ignores own entries: appenlight_client.client)
+; list of namespaces that should be ignores when gathering log entries
+; can be string with comma separated list of namespaces
+; (by default the client ignores own entries: appenlight_client.client)
 appenlight.log_namespace_blacklist =
 
-
-###########################################
-###   MAIN RHODECODE DATABASE CONFIG    ###
-###########################################
-#sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
-#sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
-#sqlalchemy.db1.url = mysql://root:qweqwe@localhost/rhodecode?charset=utf8
-# pymysql is an alternative driver for MySQL, use in case of problems with default one
-#sqlalchemy.db1.url = mysql+pymysql://root:qweqwe@localhost/rhodecode
-
-sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
-
-# see sqlalchemy docs for other advanced settings
-
-## print the sql statements to output
-sqlalchemy.db1.echo = false
-## recycle the connections after this amount of seconds
-sqlalchemy.db1.pool_recycle = 3600
-sqlalchemy.db1.convert_unicode = true
-
-## the number of connections to keep open inside the connection pool.
-## 0 indicates no limit
-#sqlalchemy.db1.pool_size = 5
-
-## the number of connections to allow in connection pool "overflow", that is
-## connections that can be opened above and beyond the pool_size setting,
-## which defaults to five.
-#sqlalchemy.db1.max_overflow = 10
-
-## Connection check ping, used to detect broken database connections
-## could be enabled to better handle cases if MySQL has gone away errors
-#sqlalchemy.db1.ping_connection = true
-
-##################
-### VCS CONFIG ###
-##################
-vcs.server.enable = true
-vcs.server = localhost:9900
-
-## Web server connectivity protocol, responsible for web based VCS operations
-## Available protocols are:
-## `http` - use http-rpc backend (default)
-vcs.server.protocol = http
-
-## Push/Pull operations protocol, available options are:
-## `http` - use http-rpc backend (default)
-vcs.scm_app_implementation = http
-
-## Push/Pull operations hooks protocol, available options are:
-## `http` - use http-rpc backend (default)
-vcs.hooks.protocol = http
-
-## Host on which this instance is listening for hooks. If vcsserver is in other location
-## this should be adjusted.
-vcs.hooks.host = 127.0.0.1
-
-vcs.server.log_level = info
-## Start VCSServer with this instance as a subprocess, useful for development
-vcs.start_server = false
-
-## List of enabled VCS backends, available options are:
-## `hg`  - mercurial
-## `git` - git
-## `svn` - subversion
-vcs.backends = hg, git, svn
-
-vcs.connection_timeout = 3600
-## Compatibility version when creating SVN repositories. Defaults to newest version when commented out.
-## Available options are: pre-1.4-compatible, pre-1.5-compatible, pre-1.6-compatible, pre-1.8-compatible, pre-1.9-compatible
-#vcs.svn.compatible_version = pre-1.8-compatible
-
-
-############################################################
-### Subversion proxy support (mod_dav_svn)               ###
-### Maps RhodeCode repo groups into SVN paths for Apache ###
-############################################################
-## Enable or disable the config file generation.
-svn.proxy.generate_config = false
-## Generate config file with `SVNListParentPath` set to `On`.
-svn.proxy.list_parent_path = true
-## Set location and file name of generated config file.
-svn.proxy.config_file_path = %(here)s/mod_dav_svn.conf
-## alternative mod_dav config template. This needs to be a mako template
-#svn.proxy.config_template = ~/.rccontrol/enterprise-1/custom_svn_conf.mako
-## Used as a prefix to the `Location` block in the generated config file.
-## In most cases it should be set to `/`.
-svn.proxy.location_root = /
-## Command to reload the mod dav svn configuration on change.
-## Example: `/etc/init.d/apache2 reload` or /home/USER/apache_reload.sh
-## Make sure user who runs RhodeCode process is allowed to reload Apache
-#svn.proxy.reload_cmd = /etc/init.d/apache2 reload
-## If the timeout expires before the reload command finishes, the command will
-## be killed. Setting it to zero means no timeout. Defaults to 10 seconds.
-#svn.proxy.reload_timeout = 10
-
-############################################################
-### SSH Support Settings                                 ###
-############################################################
-
-## Defines if a custom authorized_keys file should be created and written on
-## any change user ssh keys. Setting this to false also disables possibility
-## of adding SSH keys by users from web interface. Super admins can still
-## manage SSH Keys.
-ssh.generate_authorized_keyfile = false
-
-## Options for ssh, default is `no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding`
-# ssh.authorized_keys_ssh_opts =
-
-## Path to the authorized_keys file where the generate entries are placed.
-## It is possible to have multiple key files specified in `sshd_config` e.g.
-## AuthorizedKeysFile %h/.ssh/authorized_keys %h/.ssh/authorized_keys_rhodecode
-ssh.authorized_keys_file_path = ~/.ssh/authorized_keys_rhodecode
-
-## Command to execute the SSH wrapper. The binary is available in the
-## RhodeCode installation directory.
-## e.g ~/.rccontrol/community-1/profile/bin/rc-ssh-wrapper
-ssh.wrapper_cmd = ~/.rccontrol/community-1/rc-ssh-wrapper
-
-## Allow shell when executing the ssh-wrapper command
-ssh.wrapper_cmd_allow_shell = false
-
-## Enables logging, and detailed output send back to the client during SSH
-## operations. Useful for debugging, shouldn't be used in production.
-ssh.enable_debug_logging = false
-
-## Paths to binary executable, by default they are the names, but we can
-## override them if we want to use a custom one
-ssh.executable.hg = ~/.rccontrol/vcsserver-1/profile/bin/hg
-ssh.executable.git = ~/.rccontrol/vcsserver-1/profile/bin/git
-ssh.executable.svn = ~/.rccontrol/vcsserver-1/profile/bin/svnserve
-
-## Enables SSH key generator web interface. Disabling this still allows users
-## to add their own keys.
-ssh.enable_ui_key_generator = true
-
-
-## Dummy marker to add new entries after.
-## Add any custom entries below. Please don't remove.
+; Dummy marker to add new entries after.
+; Add any custom entries below. Please don't remove this marker.
 custom.conf = 1
 
 
-################################
-### LOGGING CONFIGURATION   ####
-################################
+; #####################
+; LOGGING CONFIGURATION
+; #####################
 [loggers]
 keys = root, sqlalchemy, beaker, celery, rhodecode, ssh_wrapper
 
 [handlers]
 keys = console, console_sql
 
 [formatters]
 keys = generic, color_formatter, color_formatter_sql
 
-#############
-## LOGGERS ##
-#############
+; #######
+; LOGGERS
+; #######
 [logger_root]
 level = NOTSET
 handlers = console
 
 [logger_sqlalchemy]
 level = INFO
 handlers = console_sql
 qualname = sqlalchemy.engine
 propagate = 0
 
 [logger_beaker]
 level = DEBUG
 handlers =
 qualname = beaker.container
 propagate = 1
 
 [logger_rhodecode]
 level = DEBUG
 handlers =
 qualname = rhodecode
 propagate = 1
 
 [logger_ssh_wrapper]
 level = DEBUG
 handlers =
 qualname = ssh_wrapper
 propagate = 1
 
 [logger_celery]
 level = DEBUG
 handlers =
 qualname = celery
 
 
-##############
-## HANDLERS ##
-##############
+; ########
+; HANDLERS
+; ########
 
 [handler_console]
 class = StreamHandler
 args = (sys.stderr, )
 level = INFO
 formatter = generic
 
 [handler_console_sql]
-# "level = DEBUG" logs SQL queries and results.
-# "level = INFO" logs SQL queries.
-# "level = WARN" logs neither.  (Recommended for production systems.)
+; "level = DEBUG" logs SQL queries and results.
+; "level = INFO" logs SQL queries.
+; "level = WARN" logs neither.  (Recommended for production systems.)
 class = StreamHandler
 args = (sys.stderr, )
 level = WARN
 formatter = generic
 
-################
-## FORMATTERS ##
-################
+; ##########
+; FORMATTERS
+; ##########
 
 [formatter_generic]
 class = rhodecode.lib.logging_formatter.ExceptionAwareFormatter
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter]
 class = rhodecode.lib.logging_formatter.ColorFormatter
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter_sql]
 class = rhodecode.lib.logging_formatter.ColorFormatterSql
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S

default.nix

@@ -1,296 +1,296 b''
 # Nix environment for the community edition
 #
 # This shall be as lean as possible, just producing the enterprise-ce
 # derivation. For advanced tweaks to pimp up the development environment we use
 # "shell.nix" so that it does not have to clutter this file.
 #
 # Configuration, set values in "~/.nixpkgs/config.nix".
 # example
 #  {
 #    # Thoughts on how to configure the dev environment
 #    rc = {
 #      codeInternalUrl = "https://usr:[e-mail]/internal";
 #      sources = {
 #        rhodecode-vcsserver = "/home/user/work/rhodecode-vcsserver";
 #        rhodecode-enterprise-ce = "/home/user/work/rhodecode-enterprise-ce";
 #        rhodecode-enterprise-ee = "/home/user/work/rhodecode-enterprise-ee";
 #      };
 #    };
 #  }
 
 args@
 { system ? builtins.currentSystem
 , pythonPackages ? "python27Packages"
 , pythonExternalOverrides ? self: super: {}
 , doCheck ? false
 , ...
 }:
 
 let
   pkgs_ = args.pkgs or (import <nixpkgs> { inherit system; });
 in
 
 let
   pkgs = import <nixpkgs> {
     overlays = [
       (import ./pkgs/overlays.nix)
     ];
     inherit
       (pkgs_)
       system;
   };
 
   # Works with the new python-packages, still can fallback to the old
   # variant.
   basePythonPackagesUnfix = basePythonPackages.__unfix__ or (
     self: basePythonPackages.override (a: { inherit self; }));
 
   # Evaluates to the last segment of a file system path.
   basename = path: with pkgs.lib; last (splitString "/" path);
 
   # source code filter used as arugment to builtins.filterSource.
   src-filter = path: type: with pkgs.lib;
     let
       ext = last (splitString "." path);
     in
       !builtins.elem (basename path) [
         ".git" ".hg" "__pycache__" ".eggs" ".idea" ".dev"
         "node_modules" "node_binaries"
         "build" "data" "result" "tmp"] &&
       !builtins.elem ext ["egg-info" "pyc"] &&
       # TODO: johbo: This check is wrong, since "path" contains an absolute path,
       # it would still be good to restore it since we want to ignore "result-*".
       !hasPrefix "result" path;
 
   sources =
     let
       inherit
         (pkgs.lib)
         all
         isString
         attrValues;
       sourcesConfig = pkgs.config.rc.sources or {};
     in
       # Ensure that sources are configured as strings. Using a path
       # would result in a copy into the nix store.
       assert all isString (attrValues sourcesConfig);
       sourcesConfig;
 
   version = builtins.readFile "${rhodecode-enterprise-ce-src}/rhodecode/VERSION";
   rhodecode-enterprise-ce-src = builtins.filterSource src-filter ./.;
 
   nodeEnv = import ./pkgs/node-default.nix {
     inherit
       pkgs
       system;
   };
   nodeDependencies = nodeEnv.shell.nodeDependencies;
 
   rhodecode-testdata-src = sources.rhodecode-testdata or (
     pkgs.fetchhg {
       url = "https://code.rhodecode.com/upstream/rc_testdata";
       rev = "v0.10.0";
       sha256 = "0zn9swwvx4vgw4qn8q3ri26vvzgrxn15x6xnjrysi1bwmz01qjl0";
   });
 
   rhodecode-testdata = import "${rhodecode-testdata-src}/default.nix" {
   inherit
     doCheck
     pkgs
     pythonPackages;
   };
 
   pythonLocalOverrides = self: super: {
     rhodecode-enterprise-ce =
       let
         linkNodePackages = ''
           export RHODECODE_CE_PATH=${rhodecode-enterprise-ce-src}
 
           echo "[BEGIN]: Link node packages and binaries"
           # johbo: Linking individual packages allows us to run "npm install"
           # inside of a shell to try things out. Re-entering the shell will
           # restore a clean environment.
           rm -fr node_modules
           mkdir node_modules
           ln -s ${nodeDependencies}/lib/node_modules/* node_modules/
           export NODE_PATH=./node_modules
 
           rm -fr node_binaries
           mkdir node_binaries
           ln -s ${nodeDependencies}/bin/* node_binaries/
           echo "[DONE ]: Link node packages and binaries"
         '';
 
         releaseName = "RhodeCodeEnterpriseCE-${version}";
       in super.rhodecode-enterprise-ce.override (attrs: {
       inherit
         doCheck
         version;
 
       name = "rhodecode-enterprise-ce-${version}";
       releaseName = releaseName;
       src = rhodecode-enterprise-ce-src;
       dontStrip = true; # prevent strip, we don't need it.
 
       # expose following attributed outside
       passthru = {
         inherit
           rhodecode-testdata
           linkNodePackages
           myPythonPackagesUnfix
           pythonLocalOverrides
           pythonCommunityOverrides;
 
         pythonPackages = self;
       };
 
       buildInputs =
         attrs.buildInputs or [] ++ [
           rhodecode-testdata
         ];
 
       #NOTE: option to inject additional propagatedBuildInputs
       propagatedBuildInputs =
         attrs.propagatedBuildInputs or [] ++ [
 
         ];
 
       LC_ALL = "en_US.UTF-8";
       LOCALE_ARCHIVE =
         if pkgs.stdenv.isLinux
         then "${pkgs.glibcLocales}/lib/locale/locale-archive"
         else "";
 
       # Add bin directory to path so that tests can find 'rhodecode'.
       preCheck = ''
         export PATH="$out/bin:$PATH"
       '';
 
       # custom check phase for testing
       checkPhase = ''
         runHook preCheck
         PYTHONHASHSEED=random py.test -vv -p no:sugar -r xw --cov-config=.coveragerc --cov=rhodecode --cov-report=term-missing rhodecode
         runHook postCheck
       '';
 
       postCheck = ''
         echo "Cleanup of rhodecode/tests"
         rm -rf $out/lib/${self.python.libPrefix}/site-packages/rhodecode/tests
       '';
 
       preBuild = ''
         echo "[BEGIN]: Building frontend assets"
         ${linkNodePackages}
         make web-build
         rm -fr node_modules
         rm -fr node_binaries
         echo "[DONE ]: Building frontend assets"
       '';
 
       postInstall = ''
         # check required files
         STATIC_CHECK="/robots.txt /502.html
-                      /js/scripts.js /js/rhodecode-components.js
+                      /js/scripts.min.js /js/rhodecode-components.js
                       /css/style.css /css/style-polymer.css /css/style-ipython.css"
 
         for file in $STATIC_CHECK;
         do
             if [ ! -f rhodecode/public/$file ]; then
               echo "Missing $file"
               exit 1
             fi
         done
 
         echo "Writing enterprise-ce meta information for rccontrol to nix-support/rccontrol"
         mkdir -p $out/nix-support/rccontrol
         cp -v rhodecode/VERSION $out/nix-support/rccontrol/version
         echo "[DONE ]: enterprise-ce meta information for rccontrol written"
 
         mkdir -p $out/etc
         cp configs/production.ini $out/etc
         echo "[DONE ]: saved enterprise-ce production.ini into $out/etc"
 
         cp -Rf rhodecode/config/rcextensions $out/etc/rcextensions.tmpl
         echo "[DONE ]: saved enterprise-ce rcextensions into $out/etc/rcextensions.tmpl"
 
         # python based programs need to be wrapped
         mkdir -p $out/bin
 
         # required binaries from dependencies
         ln -s ${self.supervisor}/bin/supervisorctl $out/bin/
         ln -s ${self.supervisor}/bin/supervisord $out/bin/
         ln -s ${self.pastescript}/bin/paster $out/bin/
         ln -s ${self.channelstream}/bin/channelstream $out/bin/
         ln -s ${self.celery}/bin/celery $out/bin/
         ln -s ${self.gunicorn}/bin/gunicorn $out/bin/
         ln -s ${self.pyramid}/bin/prequest $out/bin/
         ln -s ${self.pyramid}/bin/pserve $out/bin/
 
         echo "[DONE ]: created symlinks into $out/bin"
         DEPS="$out/bin/supervisorctl \
               $out/bin/supervisord \
               $out/bin/paster \
               $out/bin/channelstream \
               $out/bin/celery \
               $out/bin/gunicorn \
               $out/bin/prequest \
               $out/bin/pserve"
 
         # wrap only dependency scripts, they require to have full PYTHONPATH set
         # to be able to import all packages
         for file in $DEPS;
         do
           wrapProgram $file \
             --prefix PATH : $PATH \
             --prefix PYTHONPATH : $PYTHONPATH \
             --set PYTHONHASHSEED random
         done
 
         echo "[DONE ]: enterprise-ce binary wrapping"
 
         # rhodecode-tools don't need wrapping
         ln -s ${self.rhodecode-tools}/bin/rhodecode-* $out/bin/
 
         # expose sources of CE
         ln -s $out $out/etc/rhodecode_enterprise_ce_source
 
         # expose static files folder
         cp -Rf $out/lib/${self.python.libPrefix}/site-packages/rhodecode/public/ $out/etc/static
         chmod 755 -R $out/etc/static
 
       '';
     });
 
   };
 
   basePythonPackages = with builtins;
     if isAttrs pythonPackages then
       pythonPackages
     else
       getAttr pythonPackages pkgs;
 
   pythonGeneratedPackages = import ./pkgs/python-packages.nix {
     inherit
       pkgs;
     inherit
       (pkgs)
       fetchurl
       fetchgit
       fetchhg;
   };
 
   pythonCommunityOverrides = import ./pkgs/python-packages-overrides.nix {
     inherit pkgs basePythonPackages;
   };
 
   # Apply all overrides and fix the final package set
   myPythonPackagesUnfix = with pkgs.lib;
     (extends pythonExternalOverrides
     (extends pythonLocalOverrides
     (extends pythonCommunityOverrides
     (extends pythonGeneratedPackages
              basePythonPackagesUnfix))));
 
   myPythonPackages = (pkgs.lib.fix myPythonPackagesUnfix);
 
 in myPythonPackages.rhodecode-enterprise-ce

docs/admin/repo_admin/repo-admin-tasks.rst

@@ -1,24 +1,61 b''
 .. _repo-admin-tasks:
 
 Common Admin Tasks for Repositories
 -----------------------------------
 
 
 Manually Force Delete Repository
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 In case of attached forks or pull-requests repositories should be archived.
 Here is how to force delete a repository and remove all dependent objects
 
 
 .. code-block:: bash
+   :dedent: 1
 
     # starts the ishell interactive prompt
     $ rccontrol ishell enterprise-1
 
 .. code-block:: python
+   :dedent: 1
 
     In [4]: from rhodecode.model.repo import RepoModel
     In [3]: repo = Repository.get_by_repo_name('test_repos/repo_with_prs')
     In [5]: RepoModel().delete(repo, forks='detach', pull_requests='delete')
     In [6]: Session().commit()
+
+
+Below is a fully automated example to force delete repositories reading from a
+file where each line is a repository name. This can be executed via simple CLI command
+without entering the interactive shell.
+
+Save the below content as a file named `repo_delete_task.py`
+
+
+.. code-block:: python
+   :dedent: 1
+
+    from rhodecode.model.db import *
+    from rhodecode.model.repo import RepoModel
+    with open('delete_repos.txt', 'rb') as f:
+        # read all lines from file
+        repos = f.readlines()
+        for repo_name in repos:
+            repo_name = repo_name.strip()  # cleanup the name just in case
+            repo = Repository.get_by_repo_name(repo_name)
+            if not repo:
+                raise Exception('Repo with name {} not found'.format(repo_name))
+            RepoModel().delete(repo, forks='detach', pull_requests='delete')
+            Session().commit()
+            print('Removed repository {}'.format(repo_name))
+
+
+The code above will read the names of repositories from a file called `delete_repos.txt`
+Each lines should represent a single name e.g `repo_name_1` or `repo_group/repo_name_2`
+
+Run this line from CLI to execute the code from the `repo_delete_task.py` file and
+exit the ishell after the execution::
+
+    echo "%run repo_delete_task.py" | rccontrol ishell Enterprise-1
+

docs/admin/system_admin/nginx/nginx-config-example.rst

@@ -1,192 +1,193 b''
 Nginx Configuration Example
 ---------------------------
 
 Use the following example to configure Nginx as a your web server.
 
 
 .. code-block:: nginx
 
     ## Rate limiter for certain pages to prevent brute force attacks
     limit_req_zone  $binary_remote_addr  zone=req_limit:10m   rate=1r/s;
 
     ## cache zone
     proxy_cache_path /etc/nginx/nginx_cache levels=1:2 use_temp_path=off keys_zone=cache_zone:10m inactive=720h max_size=10g;
 
     ## Custom log format
     log_format log_custom '$remote_addr - $remote_user [$time_local] '
                           '"$request" $status $body_bytes_sent '
                           '"$http_referer" "$http_user_agent" '
                           '$request_time $upstream_response_time $pipe';
 
     ## Define one or more upstreams (local RhodeCode instance) to connect to
     upstream rc {
         # Url to running RhodeCode instance.
         # This is shown as `- URL: <host>` in output from rccontrol status.
         server 127.0.0.1:10002;
 
         # add more instances for load balancing
         # server 127.0.0.1:10003;
         # server 127.0.0.1:10004;
     }
 
     ## HTTP to HTTPS rewrite
     server {
         listen          80;
         server_name     rhodecode.myserver.com;
 
         if ($http_host = rhodecode.myserver.com) {
             rewrite  (.*)  https://rhodecode.myserver.com$1 permanent;
         }
     }
 
     ## Optional gist alias server, for serving nicer GIST urls.
     server {
         listen          443;
         server_name     gist.myserver.com;
         access_log      /var/log/nginx/gist.access.log log_custom;
         error_log       /var/log/nginx/gist.error.log;
 
         ssl on;
         ssl_certificate     gist.rhodecode.myserver.com.crt;
         ssl_certificate_key gist.rhodecode.myserver.com.key;
 
         ssl_session_timeout 5m;
 
         ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
         ssl_prefer_server_ciphers on;
         ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
 
         ## Strict http prevents from https -> http downgrade
         add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
 
         ## Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
         #ssl_dhparam /etc/nginx/ssl/dhparam.pem;
 
         rewrite ^/(.+)$ https://rhodecode.myserver.com/_admin/gists/$1;
         rewrite (.*)    https://rhodecode.myserver.com/_admin/gists;
     }
 
 
     ## MAIN SSL enabled server
     server {
         listen       443 ssl http2;
         server_name  rhodecode.myserver.com;
 
         access_log   /var/log/nginx/rhodecode.access.log log_custom;
         error_log    /var/log/nginx/rhodecode.error.log;
 
         ssl_certificate     rhodecode.myserver.com.crt;
         ssl_certificate_key rhodecode.myserver.com.key;
 
         # enable session resumption to improve https performance
         # http://vincent.bernat.im/en/blog/2011-ssl-session-reuse-rfc5077.html
         ssl_session_cache shared:SSL:50m;
         ssl_session_timeout 5m;
 
         ## Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
         #ssl_dhparam /etc/nginx/ssl/dhparam.pem;
 
         # enables server-side protection from BEAST attacks
         # http://blog.ivanristic.com/2013/09/is-beast-still-a-threat.html
         ssl_prefer_server_ciphers on;
 
         # disable SSLv3(enabled by default since nginx 0.8.19) since it's less secure then TLS http://en.wikipedia.org/wiki/Secure_Sockets_Layer#SSL_3.0
         ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
 
         # ciphers chosen for forward secrecy and compatibility
         # http://blog.ivanristic.com/2013/08/configuring-apache-nginx-and-openssl-for-forward-secrecy.html
         ssl_ciphers "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
 
         client_body_buffer_size     128k;
         # maximum number and size of buffers for large headers to read from client request
         large_client_header_buffers 16 256k;
 
         ## uncomment to serve static files by Nginx, recommended for performance
         # location /_static/rhodecode {
         #    gzip on;
         #    gzip_min_length  500;
         #    gzip_proxied     any;
         #    gzip_comp_level 4;
         #    gzip_types  text/css text/javascript text/xml text/plain text/x-component application/javascript application/json application/xml application/rss+xml font/truetype font/opentype application/vnd.ms-fontobject image/svg+xml;
         #    gzip_vary on;
         #    gzip_disable     "msie6";
+        #    expires 60d;
         #    alias /path/to/.rccontrol/community-1/static;
         #    alias /path/to/.rccontrol/enterprise-1/static;
         # }
 
         ## channelstream location handler, if channelstream live chat and notifications
         ## are enable this will proxy the requests to channelstream websocket server
         location /_channelstream {
             rewrite /_channelstream/(.*) /$1 break;
             gzip                         off;
             tcp_nodelay                  off;
 
             proxy_connect_timeout        10;
             proxy_send_timeout           10m;
             proxy_read_timeout           10m;
 
             proxy_set_header             Host $host;
             proxy_set_header             X-Real-IP $remote_addr;
             proxy_set_header             X-Url-Scheme $scheme;
             proxy_set_header             X-Forwarded-Proto $scheme;
             proxy_set_header             X-Forwarded-For $proxy_add_x_forwarded_for;
 
             proxy_http_version           1.1;
             proxy_set_header Upgrade     $http_upgrade;
             proxy_set_header Connection  "upgrade";
 
             proxy_pass                  http://127.0.0.1:9800;
         }
 
         ## rate limit this endpoint to prevent login page brute-force attacks
         location /_admin/login {
             limit_req  zone=req_limit  burst=10  nodelay;
             try_files $uri @rhodecode_http;
         }
 
         ## Special Cache for file store, make sure you enable this intentionally as
         ## it could bypass upload files permissions
         # location /_file_store/download {
         #
         #    proxy_cache cache_zone;
         #    # ignore Set-Cookie
         #    proxy_ignore_headers Set-Cookie;
         #    proxy_ignore_headers Cookie;
         #
         #    proxy_cache_key $host$uri$is_args$args;
         #    proxy_cache_methods GET;
         #
         #    proxy_cache_bypass  $http_cache_control;
         #    proxy_cache_valid 200 302 720h;
         #
         #    proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
         #
         #    # returns cache status in headers
         #    add_header X-Proxy-Cache $upstream_cache_status;
         #    add_header Cache-Control "public";
         #
         #    proxy_cache_lock on;
         #    proxy_cache_lock_age 5m;
         #
         #    proxy_pass  http://rc;
         #
         # }
 
         location / {
             try_files $uri @rhodecode_http;
         }
 
         location @rhodecode_http {
             # example of proxy.conf can be found in our docs.
             include     /etc/nginx/proxy.conf;
             proxy_pass  http://rc;
         }
 
         ## Custom 502 error page.
         ## Will be displayed while RhodeCode server is turned off
         error_page 502 /502.html;
         location = /502.html {
            #root  /path/to/.rccontrol/community-1/static;
            root  /path/to/.rccontrol/enterprise-1/static;
         }
     }
  No newline at end of file

docs/admin/system_admin/tuning-rhodecode.rst

@@ -1,21 +1,22 b''
 .. _rhodecode-tuning-ref:
 
 Tuning |RCE|
 ============
 
 To customize your |RCE| |version| installation for maximum performance you
 may find some of the following methods useful.
 
 .. toctree::
 
    tuning/tuning-gunicorn
-   tuning/tuning-vcs-memory-cache
+   tuning/tuning-vcs-server-memory-usage
+   tuning/tuning-rhodecode-memory-usage
    tuning/tuning-user-sessions-performance
    tuning/tuning-increase-db-performance
    tuning/tuning-scale-horizontally-cluster
    tuning/tuning-mount-cache-memory
    tuning/tuning-change-encoding
    tuning/tuning-change-large-file-dir
    tuning/tuning-change-lfs-dir
    tuning/tuning-hg-auth-loop
 

docs/admin/system_admin/tuning/tuning-gunicorn.rst

@@ -1,126 +1,118 b''
 .. _increase-gunicorn:
 
 Configure Gunicorn Workers
 --------------------------
 
 
 |RCE| comes with `Gunicorn`_ which is a Python WSGI HTTP Server for UNIX.
 
 To improve |RCE| performance you can increase the number of `Gunicorn`_  workers.
 This allows to handle more connections concurrently, and provide better
 responsiveness and performance.
 
 By default during installation |RCC|  tries to detect how many CPUs are
 available in the system, and set the number workers based on that information.
 However sometimes it's better to manually set the number of workers.
 
 To do this, use the following steps:
 
 1. Open the :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
 2. In the ``[server:main]`` section, change the number of Gunicorn
    ``workers`` using the following default formula :math:`(2 * Cores) + 1`.
    We however not recommend using more than 8-12 workers per server. It's better
    to start using the :ref:`scale-horizontal-cluster` in case that performance
    with 8-12 workers is not enough.
 
 .. code-block:: ini
 
-    use = egg:gunicorn#main
-    ## Sets the number of process workers. You must set `instance_id = *`
-    ## when this option is set to more than one worker, recommended
-    ## value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers
-    ## The `instance_id = *` must be set in the [app:main] section below
-    workers = 4
-    ## process name
-    proc_name = rhodecode
-    ## type of worker class, one of sync, gevent
-    ## recommended for bigger setup is using of of other than sync one
-    worker_class = sync
-    ## The maximum number of simultaneous clients. Valid only for Gevent
-    #worker_connections = 10
-    ## max number of requests that worker will handle before being gracefully
-    ## restarted, could prevent memory leaks
-    max_requests = 1000
-    max_requests_jitter = 30
-    ## amount of time a worker can spend with handling a request tuning-change-lfs-dir.before it
-    ## gets killed and restarted. Set to 6hrs
-    timeout = 21600
+    ; Sets the number of process workers. More workers means more concurrent connections
+    ; RhodeCode can handle at the same time. Each additional worker also it increases
+    ; memory usage as each has it's own set of caches.
+    ; Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
+    ; than 8-10 unless for really big deployments .e.g 700-1000 users.
+    ; `instance_id = *` must be set in the [app:main] section below (which is the default)
+    ; when using more than 1 worker.
+    workers = 6
+
+    ; Type of worker class, one of `sync`, `gevent`
+    ; Use `gevent` for rhodecode
+    worker_class = gevent
+
+    ; The maximum number of simultaneous clients per worker. Valid only for gevent
+    worker_connections = 10
+
 
 3. In the ``[app:main]`` section, set the ``instance_id`` property to ``*``.
 
 .. code-block:: ini
 
     # In the [app:main] section
     [app:main]
     # You must set `instance_id = *`
     instance_id = *
 
 4. Change the VCSServer workers too. Open the
    :file:`home/{user}/.rccontrol/{instance-id}/vcsserver.ini` file.
 
 5. In the ``[server:main]`` section, increase the number of Gunicorn
    ``workers`` using the following formula :math:`(2 * Cores) + 1`.
 
 .. code-block:: ini
 
-    ## run with gunicorn --log-config vcsserver.ini --paste vcsserver.ini
-    use = egg:gunicorn#main
-    ## Sets the number of process workers. Recommended
-    ## value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers
-    workers = 4
-    ## process name
-    proc_name = rhodecode_vcsserver
-    ## type of worker class, currently `sync` is the only option allowed.
+    ; Sets the number of process workers. More workers means more concurrent connections
+    ; RhodeCode can handle at the same time. Each additional worker also it increases
+    ; memory usage as each has it's own set of caches.
+    ; Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
+    ; than 8-10 unless for really big deployments .e.g 700-1000 users.
+    ; `instance_id = *` must be set in the [app:main] section below (which is the default)
+    ; when using more than 1 worker.
+    workers = 8
+
+    ; Type of worker class, one of `sync`, `gevent`
+    ; Use `sync` for vcsserver
     worker_class = sync
-    ## The maximum number of simultaneous clients. Valid only for Gevent
-    #worker_connections = 10
-    ## max number of requests that worker will handle before being gracefully
-    ## restarted, could prevent memory leaks
-    max_requests = 1000
-    max_requests_jitter = 30
-    ## amount of time a worker can spend with handling a request before it
-    ## gets killed and restarted. Set to 6hrs
-    timeout = 21600
+
 
 6. Save your changes.
 7. Restart your |RCE| instances, using the following command:
 
 .. code-block:: bash
 
     $ rccontrol restart '*'
 
 
 Gunicorn Gevent Backend
 -----------------------
 
 Gevent is an asynchronous worker type for Gunicorn. It allows accepting multiple
 connections on a single `Gunicorn`_  worker. This means you can handle 100s
 of concurrent clones, or API calls using just few workers. A setting called
 `worker_connections` defines on how many connections each worker can
 handle using `Gevent`.
 
 
 To enable `Gevent` on |RCE| do the following:
 
 
 1. Open the :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
 2. In the ``[server:main]`` section, change `worker_class` for Gunicorn.
 
 
 .. code-block:: ini
 
-    ## type of worker class, one of sync, gevent
-    ## recommended for bigger setup is using of of other than sync one
+    ; Type of worker class, one of `sync`, `gevent`
+    ; Use `gevent` for rhodecode
     worker_class = gevent
-    ## The maximum number of simultaneous clients. Valid only for Gevent
+
+    ; The maximum number of simultaneous clients per worker. Valid only for gevent
     worker_connections = 30
 
 
 .. note::
 
     `Gevent` is currently only supported for Enterprise/Community instances.
-    VCSServer doesn't yet support gevent.
+    VCSServer doesn't support gevent.
 
 
 
 .. _Gunicorn: http://gunicorn.org/

docs/admin/system_admin/tuning/tuning-scale-horizontally-cluster.rst

@@ -1,395 +1,396 b''
 .. _scale-horizontal-cluster:
 
 
 Scale Horizontally / RhodeCode Cluster
 --------------------------------------
 
 |RCE| is built in a way it support horizontal scaling across multiple machines.
 There are three main pre-requisites for that:
 
 - Shared storage that each machine can access. Using NFS or other shared storage system.
 - Shared DB connection across machines. Using `MySQL`/`PostgreSQL` that each node can access.
 - |RCE| user sessions and caches need to use a shared storage (e.g `Redis`_/`Memcached`)
 
 
 Horizontal scaling means adding more machines or workers into your pool of
 resources. Horizontally scaling |RCE| gives a huge performance increase,
 especially under large traffic scenarios with a high number of requests.
 This is very beneficial when |RCE| is serving many users simultaneously,
 or if continuous integration servers are automatically pulling and pushing code.
 It also adds High-Availability to your running system.
 
 
 Cluster Overview
 ^^^^^^^^^^^^^^^^
 
 Below we'll present a configuration example that will use two separate nodes to serve
 |RCE| in a load-balanced environment. The 3rd node will act as a shared storage/cache
 and handle load-balancing. In addition 3rd node will be used as shared database instance.
 
 This setup can be used both in Docker based configuration or with individual
 physical/virtual machines. Using the 3rd node for Storage/Redis/PostgreSQL/Nginx is
 optional. All those components can be installed on one of the two nodes used for |RCE|.
 We'll use following naming for our nodes:
 
  - `rc-node-1` (NFS, DB, Cache node)
  - `rc-node-2` (Worker node1)
  - `rc-node-3` (Worker node2)
 
 Our shares NFS storage in the example is located on `/home/rcdev/storage` and
 it's RW accessible on **each** node.
 
 In this example we used certain recommended components, however many
 of those can be replaced by other, in case your organization already uses them, for example:
 
 - `MySQL`/`PostgreSQL`: Aren't replaceable and are the two only supported databases.
 - `Nginx`_ on `rc-node-1` can be replaced by: `Hardware Load Balancer (F5)`, `Apache`_, `HA-Proxy` etc.
 - `Nginx`_ on rc-node-2/3 acts as a reverse proxy and can be replaced by other HTTP server
   acting as reverse proxy such as `Apache`_.
 - `Redis`_ on `rc-node-1` can be replaced by: `Memcached`
 
 
 Here's an overview what components should be installed/setup on each server in our example:
 
 - **rc-node-1**:
 
  - main storage acting as NFS host.
  - `nginx` acting as a load-balancer.
  - `postgresql-server` used for database and sessions.
  - `redis-server` used for storing shared caches.
- - optionally `rabbitmq-server` for `Celery` if used.
+ - optionally `rabbitmq-server` or `redis` for `Celery` if used.
  - optionally if `Celery` is used Enterprise/Community instance + VCSServer.
  - optionally mailserver that can be shared by other instances.
  - optionally channelstream server to handle live communication for all instances.
 
 
 - **rc-node-2/3**:
 
  - `nginx` acting as a reverse proxy to handle requests to |RCE|.
  - 1x RhodeCode Enterprise/Community instance.
  - 1x VCSServer instance.
  - optionally for testing connection: postgresql-client, redis-client (redis-tools).
 
 
 Before we start here are few assumptions that should be fulfilled:
 
 - make sure each node can access each other.
 - make sure `Redis`_/`MySQL`/`PostgreSQL`/`RabbitMQ`_ are running on `rc-node-1`
 - make sure both `rc-node-2`/`3` can access NFS storage with RW access
 - make sure rc-node-2/3 can access `Redis`_/`PostgreSQL`, `MySQL` database on `rc-node-1`.
 - make sure `Redis`_/Database/`RabbitMQ`_ are password protected and accessible only from rc-node-2/3.
 
 
 
 Setup rc-node-2/3
 ^^^^^^^^^^^^^^^^^
 
 Initially before `rc-node-1` we'll configure both nodes 2 and 3 to operate as standalone
 nodes with their own hostnames. Use a default installation settings, and use
 the default local addresses (127.0.0.1) to configure VCSServer and Community/Enterprise instances.
 All external connectivity will be handled by the reverse proxy (`Nginx`_ in our example).
 
 This way we can ensure each individual host works,
 accepts connections, or do some operations explicitly on chosen node.
 
 In addition this would allow use to explicitly direct certain traffic to a node, e.g
 CI server will only call directly `rc-node-3`. This should be done similar to normal
 installation so check out `Nginx`_/`Apache`_ configuration example to configure each host.
 Each one should already connect to shared database during installation.
 
 
 1) Assuming our final url will be http://rc-node-1, Configure `instances_id`, `app.base_url`
 
 a) On **rc-node-2** find the following settings and edit :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
 
 .. code-block:: ini
 
     ## required format is: *NAME-
     instance_id = *rc-node-2-
     app.base_url = http://rc-node-1
 
 
 b) On **rc-node-3** find the following settings and edit :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
 
 .. code-block:: ini
 
     ## required format is: *NAME-
     instance_id = *rc-node-3-
     app.base_url = http://rc-node-1
 
 
 
 2) Configure `User Session` to use a shared database. Example config that should be
    changed on both **rc-node-2** and **rc-node-3** .
    Edit :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
 
 .. code-block:: ini
 
     ####################################
     ###       BEAKER SESSION        ####
     ####################################
 
     ## Disable the default `file` sessions
     #beaker.session.type = file
     #beaker.session.data_dir = %(here)s/data/sessions
 
     ## use shared db based session, fast, and allows easy management over logged in users
     beaker.session.type = ext:database
     beaker.session.table_name = db_session
     # use our rc-node-1 here
     beaker.session.sa.url = postgresql://postgres:qweqwe@rc-node-1/rhodecode
     beaker.session.sa.pool_recycle = 3600
     beaker.session.sa.echo = false
 
 In addition make sure both instances use the same `session.secret` so users have
 persistent sessions across nodes. Please generate other one then in this example.
 
 .. code-block:: ini
 
     # use a unique generated long string
     beaker.session.secret = 70e116cae2274656ba7265fd860aebbd
 
 3) Configure stored cached/archive cache to our shared NFS `rc-node-1`
 
 .. code-block:: ini
 
     # note the `_` prefix that allows using a directory without
     # remap and rescan checking for vcs inside it.
     cache_dir = /home/rcdev/storage/_cache_dir/data
     # note archive cache dir is disabled by default, however if you enable
     # it also needs to be shared
     #archive_cache_dir = /home/rcdev/storage/_tarball_cache_dir
 
 
 4) Use shared exception store. Example config that should be
    changed on both **rc-node-2** and **rc-node-3**, and also for VCSServer.
    Edit :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini` and
    :file:`/home/{user}/.rccontrol/{vcsserver-instance-id}/vcsserver.ini`
    and add/change following setting.
 
 .. code-block:: ini
 
     exception_tracker.store_path = /home/rcdev/storage/_exception_store_data
 
 
 5) Change cache backends to use `Redis`_ based caches. Below full example config
    that replaces default file-based cache to shared `Redis`_ with Distributed Lock.
 
 
 .. code-block:: ini
 
     #####################################
     ###         DOGPILE CACHE        ####
     #####################################
 
     ## `cache_perms` cache settings for permission tree, auth TTL.
     #rc_cache.cache_perms.backend = dogpile.cache.rc.file_namespace
     #rc_cache.cache_perms.expiration_time = 300
 
     ## alternative `cache_perms` redis backend with distributed lock
     rc_cache.cache_perms.backend = dogpile.cache.rc.redis
     rc_cache.cache_perms.expiration_time = 300
     ## redis_expiration_time needs to be greater then expiration_time
     rc_cache.cache_perms.arguments.redis_expiration_time = 7200
     rc_cache.cache_perms.arguments.socket_timeout = 30
     rc_cache.cache_perms.arguments.host = rc-node-1
     rc_cache.cache_perms.arguments.password = qweqwe
     rc_cache.cache_perms.arguments.port = 6379
     rc_cache.cache_perms.arguments.db = 0
     rc_cache.cache_perms.arguments.distributed_lock = true
 
     ## `cache_repo` cache settings for FileTree, Readme, RSS FEEDS
     #rc_cache.cache_repo.backend = dogpile.cache.rc.file_namespace
     #rc_cache.cache_repo.expiration_time = 2592000
 
     ## alternative `cache_repo` redis backend with distributed lock
     rc_cache.cache_repo.backend = dogpile.cache.rc.redis
     rc_cache.cache_repo.expiration_time = 2592000
     ## redis_expiration_time needs to be greater then expiration_time
     rc_cache.cache_repo.arguments.redis_expiration_time = 2678400
     rc_cache.cache_repo.arguments.socket_timeout = 30
     rc_cache.cache_repo.arguments.host = rc-node-1
     rc_cache.cache_repo.arguments.password = qweqwe
     rc_cache.cache_repo.arguments.port = 6379
     rc_cache.cache_repo.arguments.db = 1
     rc_cache.cache_repo.arguments.distributed_lock = true
 
     ## cache settings for SQL queries, this needs to use memory type backend
     rc_cache.sql_cache_short.backend = dogpile.cache.rc.memory_lru
     rc_cache.sql_cache_short.expiration_time = 30
 
     ## `cache_repo_longterm` cache for repo object instances, this needs to use memory
     ## type backend as the objects kept are not pickle serializable
     rc_cache.cache_repo_longterm.backend = dogpile.cache.rc.memory_lru
     ## by default we use 96H, this is using invalidation on push anyway
     rc_cache.cache_repo_longterm.expiration_time = 345600
     ## max items in LRU cache, reduce this number to save memory, and expire last used
     ## cached objects
     rc_cache.cache_repo_longterm.max_size = 10000
 
 
 6) Configure `Nginx`_ as reverse proxy on `rc-node-2/3`:
    Minimal `Nginx`_ config used:
 
 
 .. code-block:: nginx
 
     ## rate limiter for certain pages to prevent brute force attacks
     limit_req_zone  $binary_remote_addr  zone=req_limit:10m   rate=1r/s;
 
     ## custom log format
     log_format log_custom '$remote_addr - $remote_user [$time_local] '
                           '"$request" $status $body_bytes_sent '
                           '"$http_referer" "$http_user_agent" '
                           '$request_time $upstream_response_time $pipe';
 
     server {
         listen          80;
         server_name     rc-node-2;
         #server_name     rc-node-3;
 
         access_log   /var/log/nginx/rhodecode.access.log log_custom;
         error_log    /var/log/nginx/rhodecode.error.log;
 
         # example of proxy.conf can be found in our docs.
         include     /etc/nginx/proxy.conf;
 
         ## serve static files by Nginx, recommended for performance
         location /_static/rhodecode {
             gzip on;
             gzip_min_length  500;
             gzip_proxied     any;
             gzip_comp_level 4;
             gzip_types  text/css text/javascript text/xml text/plain text/x-component application/javascript application/json application/xml application/rss+xml font/truetype font/opentype application/vnd.ms-fontobject image/svg+xml;
             gzip_vary on;
             gzip_disable     "msie6";
+            expires 60d;
             #alias /home/rcdev/.rccontrol/community-1/static;
             alias /home/rcdev/.rccontrol/enterprise-1/static;
         }
 
 
         location /_admin/login {
             limit_req  zone=req_limit  burst=10  nodelay;
             try_files $uri @rhode;
         }
 
         location / {
             try_files $uri @rhode;
         }
 
         location @rhode {
             # Url to running RhodeCode instance.
             # This is shown as `- URL: <host>` in output from rccontrol status.
             proxy_pass      http://127.0.0.1:10020;
         }
 
         ## custom 502 error page. Will be displayed while RhodeCode server
         ## is turned off
         error_page 502 /502.html;
         location = /502.html {
            #root  /home/rcdev/.rccontrol/community-1/static;
            root  /home/rcdev/.rccontrol/enterprise-1/static;
         }
     }
 
 
 7) Optional: Full text search, in case you use `Whoosh` full text search we also need a
    shared storage for the index. In our example our NFS is mounted at `/home/rcdev/storage`
    which represents out storage so we can use the following:
 
 .. code-block:: ini
 
     # note the `_` prefix that allows using a directory without
     # remap and rescan checking for vcs inside it.
     search.location = /home/rcdev/storage/_index_data/index
 
 
 .. note::
 
     If you use ElasticSearch it's by default shared, and simply running ES node is
     by default cluster compatible.
 
 
 8) Optional: If you intend to use mailing all instances need to use either a shared
    mailing node, or each will use individual local mail agent. Simply put node-1/2/3
    needs to use same mailing configuration.
 
 
 
 Setup rc-node-1
 ^^^^^^^^^^^^^^^
 
 
 Configure `Nginx`_ as Load Balancer to rc-node-2/3.
 Minimal `Nginx`_ example below:
 
 .. code-block:: nginx
 
     ## define rc-cluster which contains a pool of our instances to connect to
     upstream rc-cluster {
         # rc-node-2/3 are stored in /etc/hosts with correct IP addresses
         server rc-node-2:80;
         server rc-node-3:80;
     }
 
     server {
         listen          80;
         server_name     rc-node-1;
 
         location / {
             proxy_pass http://rc-cluster;
         }
     }
 
 
 .. note::
 
    You should configure your load balancing accordingly. We recommend writing
    load balancing rules that will separate regular user traffic from
    automated process traffic like continuous servers or build bots. Sticky sessions
    are not required.
 
 
 Show which instance handles a request
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 You can easily check if load-balancing is working as expected. Visit our main node
 `rc-node-1` URL which at that point should already handle incoming requests and balance
 it across node-2/3.
 
 Add a special GET param `?showrcid=1` to show current instance handling your request.
 
 For example: visiting url `http://rc-node-1/?showrcid=1` will show, in the bottom
 of the screen` cluster instance info.
 e.g: `RhodeCode instance id: rc-node-3-rc-node-3-3246`
 which is generated from::
 
     <NODE_HOSTNAME>-<INSTANCE_ID>-<WORKER_PID>
 
 
 Using Celery with cluster
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
 
 If `Celery` is used we recommend setting also an instance of Enterprise/Community+VCSserver
-on the node that is running `RabbitMQ`_. Those instances will be used to executed async
-tasks on the `rc-node-1`. This is the most efficient setup. `Celery` usually
-handles tasks such as sending emails, forking repositories, importing
+on the node that is running `RabbitMQ`_ or `Redis`_. Those instances will be used to
+executed async tasks on the `rc-node-1`. This is the most efficient setup.
+`Celery` usually handles tasks such as sending emails, forking repositories, importing
 repositories from external location etc. Using workers on instance that has
 the direct access to disks used by NFS as well as email server gives noticeable
 performance boost. Running local workers to the NFS storage results in faster
 execution of forking large repositories or sending lots of emails.
 
 Those instances need to be configured in the same way as for other nodes.
-The instance in rc-node-1 can be added to the cluser, but we don't recommend doing it.
+The instance in rc-node-1 can be added to the cluster, but we don't recommend doing it.
 For best results let it be isolated to only executing `Celery` tasks in the cluster setup.
 
 
 .. _Gunicorn: http://gunicorn.org/
 .. _Whoosh: https://pypi.python.org/pypi/Whoosh/
 .. _Elasticsearch: https://www.elastic.co/..
 .. _RabbitMQ: http://www.rabbitmq.com/
 .. _Nginx: http://nginx.io
 .. _Apache: http://nginx.io
 .. _Redis: http://redis.io
 

docs/admin/system_admin/tuning/tuning-vcs-server-memory-usage.rst

@@ -1,8 +1,26 b''
-.. _adjust-vcs-mem-cache:
+.. _adjust-vcs-server-mem:
 
-VCSServer Memory Cache
+VCSServer Memory Usage
 ----------------------
 
-The VCS Server mamory cache can be adjusted to work best with the resources
-available to your |RCE| instance. If you find that memory resources are under
-pressure, see the :ref:`vcs-server-maintain` section for details.
+Starting from Version 4.18.X RhodeCode has a builtin memory monitor for gunicorn workers.
+Enabling this can limit the maximum amount of memory system can use. Each worker
+for VCS Server is monitored independently.
+To enable Memory management make sure to have following settings inside `[app:main] section` of
+:file:`home/{user}/.rccontrol/{instance-id}/vcsserver.ini` file.
+
+
+
+    ; Maximum memory usage that each worker can use before it will receive a
+    ; graceful restart signal 0 = memory monitoring is disabled
+    ; Examples: 268435456 (256MB), 536870912 (512MB)
+    ; 1073741824 (1GB), 2147483648 (2GB), 4294967296 (4GB)
+    memory_max_usage = 1073741824
+
+    ; How often in seconds to check for memory usage for each gunicorn worker
+    memory_usage_check_interval = 60
+
+    ; Threshold value for which we don't recycle worker if GarbageCollection
+    ; frees up enough resources. Before each restart we try to run GC on worker
+    ; in case we get enough free memory after that, restart will not happen.
+    memory_usage_recovery_threshold = 0.8

docs/admin/system_admin/vcs-server.rst

@@ -1,324 +1,379 b''
 .. _vcs-server:
 
 VCS Server Management
 ---------------------
 
 The VCS Server handles |RCE| backend functionality. You need to configure
 a VCS Server to run with a |RCE| instance. If you do not, you will be missing
 the connection between |RCE| and its |repos|. This will cause error messages
 on the web interface. You can run your setup in the following configurations,
 currently the best performance is one of following:
 
 * One VCS Server per |RCE| instance.
 * One VCS Server handling multiple instances.
 
 .. important::
 
    If your server locale settings are not correctly configured,
    |RCE| and the VCS Server can run into issues. See this `Ask Ubuntu`_ post
    which explains the problem and gives a solution.
 
 For more information, see the following sections:
 
 * :ref:`install-vcs`
 * :ref:`config-vcs`
 * :ref:`vcs-server-options`
 * :ref:`vcs-server-versions`
 * :ref:`vcs-server-maintain`
 * :ref:`vcs-server-config-file`
 * :ref:`svn-http`
 
 .. _install-vcs:
 
 VCS Server Installation
 ^^^^^^^^^^^^^^^^^^^^^^^
 
 To install a VCS Server, see
 :ref:`Installing a VCS server <control:install-vcsserver>`.
 
 .. _config-vcs:
 
 Hooking |RCE| to its VCS Server
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To configure a |RCE| instance to use a VCS server, see
 :ref:`Configuring the VCS Server connection <control:manually-vcsserver-ini>`.
 
 .. _vcs-server-options:
 
 |RCE| VCS Server Options
 ^^^^^^^^^^^^^^^^^^^^^^^^
 
 The following list shows the available options on the |RCE| side of the
 connection to the VCS Server. The settings are configured per
 instance in the
 :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
 
 .. rst-class:: dl-horizontal
 
     \vcs.backends <available-vcs-systems>
         Set a comma-separated list of the |repo| options available from the
         web interface. The default is ``hg, git, svn``,
         which is all |repo| types available. The order of backends is also the
         order backend will try to detect requests type.
 
     \vcs.connection_timeout <seconds>
         Set the length of time in seconds that the VCS Server waits for
         requests to process. After the timeout expires,
         the request is closed. The default is ``3600``. Set to a higher
         number if you experience network latency, or timeout issues with very
         large push/pull requests.
 
     \vcs.server.enable <boolean>
         Enable or disable the VCS Server. The available options are ``true`` or
         ``false``. The default is ``true``.
 
     \vcs.server <host:port>
         Set the host, either hostname or IP Address, and port of the VCS server
         you wish to run with your |RCE| instance.
 
 .. code-block:: ini
 
     ##################
     ### VCS CONFIG ###
     ##################
     # set this line to match your VCS Server
     vcs.server = 127.0.0.1:10004
     # Set to False to disable the VCS Server
     vcs.server.enable = True
     vcs.backends = hg, git, svn
     vcs.connection_timeout = 3600
 
 
 .. _vcs-server-versions:
 
 VCS Server Versions
 ^^^^^^^^^^^^^^^^^^^
 
 An updated version of the VCS Server is released with each |RCE| version. Use
 the VCS Server number that matches with the |RCE| version to pair the
 appropriate ones together. For |RCE| versions pre 3.3.0,
 VCS Server 1.X.Y works with |RCE| 3.X.Y, for example:
 
 * VCS Server 1.0.0 works with |RCE| 3.0.0
 * VCS Server 1.2.2 works with |RCE| 3.2.2
 
 For |RCE| versions post 3.3.0, the VCS Server and |RCE| version numbers
 match, for example:
 
 * VCS Server |release| works with |RCE| |release|
 
 .. _vcs-server-maintain:
 
-VCS Server Memory Optimization
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+VCS Server Cache Optimization
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-To optimize the VCS server to manage the cache and memory usage efficiently, you need to
-configure the following options in the
-:file:`/home/{user}/.rccontrol/{vcsserver-id}/vcsserver.ini` file. Once
-configured, restart the VCS Server. By default we use an optimal settings, but in certain
-conditions tunning expiration_time and max_size can affect memory usage and performance
+To optimize the VCS server to manage the cache and memory usage efficiently, it's recommended to
+configure the Redis backend for VCSServer caches.
+Once configured, restart the VCS Server.
+
+Make sure Redis is installed and running.
+Open :file:`/home/{user}/.rccontrol/{vcsserver-id}/vcsserver.ini`
+file and ensure the below settings for `repo_object` type cache are set:
 
 .. code-block:: ini
 
-    ## cache region for storing repo_objects cache
-    rc_cache.repo_object.backend = dogpile.cache.rc.memory_lru
+    ; ensure the default file based cache is *commented out*
+    ##rc_cache.repo_object.backend = dogpile.cache.rc.file_namespace
+    ##rc_cache.repo_object.expiration_time = 2592000
 
-    ## cache auto-expires after N seconds, setting this to 0 disabled cache
-    rc_cache.repo_object.expiration_time = 300
+    ; `repo_object` cache settings for vcs methods for repositories
+    rc_cache.repo_object.backend = dogpile.cache.rc.redis_msgpack
 
-    ## max size of LRU, old values will be discarded if the size of cache reaches max_size
-    ## Sets the maximum number of items stored in the cache, before the cache
-    ## starts to be cleared.
+    ; cache auto-expires after N seconds
+    ; Examples: 86400 (1Day), 604800 (7Days), 1209600 (14Days), 2592000 (30days), 7776000 (90Days)
+    rc_cache.repo_object.expiration_time = 2592000
+
+    ; redis_expiration_time needs to be greater then expiration_time
+    rc_cache.repo_object.arguments.redis_expiration_time = 3592000
 
-    ## As a general rule of thumb, running this value at 120 resulted in a
-    ## 5GB cache. Running it at 240 resulted in a 9GB cache. Your results
-    ## will differ based on usage patterns and |repo| sizes.
-
-    ## Tweaking this value to run at a fairly constant memory load on your
-    ## server will help performance.
-
-    rc_cache.repo_object.max_size = 120
+    rc_cache.repo_object.arguments.host = localhost
+    rc_cache.repo_object.arguments.port = 6379
+    rc_cache.repo_object.arguments.db = 5
+    rc_cache.repo_object.arguments.socket_timeout = 30
+    ; more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
+    rc_cache.repo_object.arguments.distributed_lock = true
 
 
 To clear the cache completely, you can restart the VCS Server.
 
 .. important::
 
    While the VCS Server handles a restart gracefully on the web interface,
    it will drop connections during push/pull requests. So it is recommended
    you only perform this when there is very little traffic on the instance.
 
 Use the following example to restart your VCS Server,
 for full details see the :ref:`RhodeCode Control CLI <control:rcc-cli>`.
 
 .. code-block:: bash
 
     $ rccontrol status
 
 .. code-block:: vim
 
     - NAME: vcsserver-1
     - STATUS: RUNNING
       logs:/home/ubuntu/.rccontrol/vcsserver-1/vcsserver.log
     - VERSION: 4.7.2 VCSServer
     - URL: http://127.0.0.1:10008
     - CONFIG: /home/ubuntu/.rccontrol/vcsserver-1/vcsserver.ini
 
     $ rccontrol restart vcsserver-1
     Instance "vcsserver-1" successfully stopped.
     Instance "vcsserver-1" successfully started.
 
 .. _vcs-server-config-file:
 
 VCS Server Configuration
 ^^^^^^^^^^^^^^^^^^^^^^^^
 
 You can configure settings for multiple VCS Servers on your
 system using their individual configuration files. Use the following
 properties inside the configuration file to set up your system. The default
 location is :file:`home/{user}/.rccontrol/{vcsserver-id}/vcsserver.ini`.
 For a more detailed explanation of the logger levers, see :ref:`debug-mode`.
 
 .. rst-class:: dl-horizontal
 
     \host <ip-address>
         Set the host on which the VCS Server will run. VCSServer is not
         protected by any authentication, so we *highly* recommend running it
         under localhost ip that is `127.0.0.1`
 
     \port <int>
         Set the port number on which the VCS Server will be available.
 
-    \locale <locale_utf>
-        Set the locale the VCS Server expects.
-
-    \workers <int>
-        Set the number of process workers.Recommended
-        value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers
-
-    \max_requests <int>
-        The maximum number of requests a worker will process before restarting.
-        Any value greater than zero will limit the number of requests a work
-        will process before automatically restarting. This is a simple method
-        to help limit the damage of memory leaks.
-
-    \max_requests_jitter <int>
-        The maximum jitter to add to the max_requests setting.
-        The jitter causes the restart per worker to be randomized by
-        randint(0, max_requests_jitter). This is intended to stagger worker
-        restarts to avoid all workers restarting at the same time.
-
 
 .. note::
 
    After making changes, you need to restart your VCS Server to pick them up.
 
 .. code-block:: ini
 
-    ################################################################################
-    # RhodeCode VCSServer with HTTP Backend - configuration                        #
-    #                                                                              #
-    ################################################################################
-
+    ; #################################
+    ; RHODECODE VCSSERVER CONFIGURATION
+    ; #################################
 
     [server:main]
-    ## COMMON ##
+    ; COMMON HOST/IP CONFIG
     host = 127.0.0.1
     port = 10002
 
-    ##########################
-    ## GUNICORN WSGI SERVER ##
-    ##########################
-    ## run with gunicorn --log-config vcsserver.ini --paste vcsserver.ini
+    ; ###########################
+    ; GUNICORN APPLICATION SERVER
+    ; ###########################
+
+    ; run with gunicorn --log-config rhodecode.ini --paste rhodecode.ini
+
+    ; Module to use, this setting shouldn't be changed
     use = egg:gunicorn#main
-    ## Sets the number of process workers. Recommended
-    ## value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers
-    workers = 3
-    ## process name
+
+    ; Sets the number of process workers. More workers means more concurrent connections
+    ; RhodeCode can handle at the same time. Each additional worker also it increases
+    ; memory usage as each has it's own set of caches.
+    ; Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
+    ; than 8-10 unless for really big deployments .e.g 700-1000 users.
+    ; `instance_id = *` must be set in the [app:main] section below (which is the default)
+    ; when using more than 1 worker.
+    workers = 6
+
+    ; Gunicorn access log level
+    loglevel = info
+
+    ; Process name visible in process list
     proc_name = rhodecode_vcsserver
-    ## type of worker class, one of sync, gevent
-    ## recommended for bigger setup is using of of other than sync one
+
+    ; Type of worker class, one of sync, gevent
+    ; currently `sync` is the only option allowed.
     worker_class = sync
-    ## The maximum number of simultaneous clients. Valid only for Gevent
-    #worker_connections = 10
-    ## max number of requests that worker will handle before being gracefully
-    ## restarted, could prevent memory leaks
+
+    ; The maximum number of simultaneous clients. Valid only for gevent
+    worker_connections = 10
+
+    ; Max number of requests that worker will handle before being gracefully restarted.
+    ; Prevents memory leaks, jitter adds variability so not all workers are restarted at once.
     max_requests = 1000
     max_requests_jitter = 30
-    ## amount of time a worker can spend with handling a request before it
-    ## gets killed and restarted. Set to 6hrs
+
+    ; Amount of time a worker can spend with handling a request before it
+    ; gets killed and restarted. By default set to 21600 (6hrs)
+    ; Examples: 1800 (30min), 3600 (1hr), 7200 (2hr), 43200 (12h)
     timeout = 21600
 
+    ; The maximum size of HTTP request line in bytes.
+    ; 0 for unlimited
+    limit_request_line = 0
+
+    ; Limit the number of HTTP headers fields in a request.
+    ; By default this value is 100 and can't be larger than 32768.
+    limit_request_fields = 32768
+
+    ; Limit the allowed size of an HTTP request header field.
+    ; Value is a positive number or 0.
+    ; Setting it to 0 will allow unlimited header field sizes.
+    limit_request_field_size = 0
+
+    ; Timeout for graceful workers restart.
+    ; After receiving a restart signal, workers have this much time to finish
+    ; serving requests. Workers still alive after the timeout (starting from the
+    ; receipt of the restart signal) are force killed.
+    ; Examples: 1800 (30min), 3600 (1hr), 7200 (2hr), 43200 (12h)
+    graceful_timeout = 3600
+
+    # The number of seconds to wait for requests on a Keep-Alive connection.
+    # Generally set in the 1-5 seconds range.
+    keepalive = 2
+
+    ; Maximum memory usage that each worker can use before it will receive a
+    ; graceful restart signal 0 = memory monitoring is disabled
+    ; Examples: 268435456 (256MB), 536870912 (512MB)
+    ; 1073741824 (1GB), 2147483648 (2GB), 4294967296 (4GB)
+    memory_max_usage = 1073741824
+
+    ; How often in seconds to check for memory usage for each gunicorn worker
+    memory_usage_check_interval = 60
+
+    ; Threshold value for which we don't recycle worker if GarbageCollection
+    ; frees up enough resources. Before each restart we try to run GC on worker
+    ; in case we get enough free memory after that, restart will not happen.
+    memory_usage_recovery_threshold = 0.8
+
+
     [app:main]
     use = egg:rhodecode-vcsserver
 
     pyramid.default_locale_name = en
     pyramid.includes =
 
-    ## default locale used by VCS systems
+    ; default locale used by VCS systems
     locale = en_US.UTF-8
 
-    # cache regions, please don't change
-    beaker.cache.regions = repo_object
-    beaker.cache.repo_object.type = memorylru
-    beaker.cache.repo_object.max_items = 100
-    # cache auto-expires after N seconds
-    beaker.cache.repo_object.expire = 300
-    beaker.cache.repo_object.enabled = true
+    ; #############
+    ; DOGPILE CACHE
+    ; #############
+
+    ; Default cache dir for caches. Putting this into a ramdisk can boost performance.
+    ; eg. /tmpfs/data_ramdisk, however this directory might require large amount of space
+    cache_dir = %(here)s/data
+
+    ; **********************************************************
+    ; `repo_object` cache with redis backend
+    ; recommended for larger instance, or for better performance
+    ; **********************************************************
+
+    ; `repo_object` cache settings for vcs methods for repositories
+    rc_cache.repo_object.backend = dogpile.cache.rc.redis_msgpack
 
+    ; cache auto-expires after N seconds
+    ; Examples: 86400 (1Day), 604800 (7Days), 1209600 (14Days), 2592000 (30days), 7776000 (90Days)
+    rc_cache.repo_object.expiration_time = 2592000
 
-    ################################
-    ### LOGGING CONFIGURATION   ####
-    ################################
+    ; redis_expiration_time needs to be greater then expiration_time
+    rc_cache.repo_object.arguments.redis_expiration_time = 3592000
+
+    rc_cache.repo_object.arguments.host = localhost
+    rc_cache.repo_object.arguments.port = 6379
+    rc_cache.repo_object.arguments.db = 5
+    rc_cache.repo_object.arguments.socket_timeout = 30
+    ; more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
+    rc_cache.repo_object.arguments.distributed_lock = true
+
+    ; #####################
+    ; LOGGING CONFIGURATION
+    ; #####################
     [loggers]
-    keys = root, vcsserver, beaker
+    keys = root, vcsserver
 
     [handlers]
     keys = console
 
     [formatters]
     keys = generic
 
-    #############
-    ## LOGGERS ##
-    #############
+    ; #######
+    ; LOGGERS
+    ; #######
     [logger_root]
     level = NOTSET
     handlers = console
 
     [logger_vcsserver]
     level = DEBUG
     handlers =
     qualname = vcsserver
     propagate = 1
 
-    [logger_beaker]
-    level = DEBUG
-    handlers =
-    qualname = beaker
-    propagate = 1
 
-
-    ##############
-    ## HANDLERS ##
-    ##############
+    ; ########
+    ; HANDLERS
+    ; ########
 
     [handler_console]
     class = StreamHandler
-    args = (sys.stderr,)
-    level = DEBUG
+    args = (sys.stderr, )
+    level = INFO
     formatter = generic
 
-    ################
-    ## FORMATTERS ##
-    ################
+    ; ##########
+    ; FORMATTERS
+    ; ##########
 
     [formatter_generic]
-    format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
+    format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
     datefmt = %Y-%m-%d %H:%M:%S
 
 
 .. _Subversion Red Book: http://svnbook.red-bean.com/en/1.7/svn-book.html#svn.ref.svn
 
 .. _Ask Ubuntu: http://askubuntu.com/questions/162391/how-do-i-fix-my-locale-issue

docs/api/methods/pull-request-methods.rst

@@ -1,434 +1,442 b''
 .. _pull-request-methods-ref:
 
 pull_request methods
 ====================
 
 close_pull_request 
 ------------------
 
 .. py:function:: close_pull_request(apiuser, pullrequestid, repoid=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>, message=<Optional:''>)
 
    Close the pull request specified by `pullrequestid`.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Repository name or repository ID to which the pull
        request belongs.
    :type repoid: str or int
    :param pullrequestid: ID of the pull request to be closed.
    :type pullrequestid: int
    :param userid: Close the pull request as this user.
    :type userid: Optional(str or int)
    :param message: Optional message to close the Pull Request with. If not
        specified it will be generated automatically.
    :type message: Optional(str)
 
    Example output:
 
    .. code-block:: bash
 
        "id": <id_given_in_input>,
        "result": {
            "pull_request_id":  "<int>",
            "close_status":     "<str:status_lbl>,
            "closed":           "<bool>"
        },
        "error": null
 
 
 comment_pull_request 
 --------------------
 
-.. py:function:: comment_pull_request(apiuser, pullrequestid, repoid=<Optional:None>, message=<Optional:None>, commit_id=<Optional:None>, status=<Optional:None>, comment_type=<Optional:u'note'>, resolves_comment_id=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>)
+.. py:function:: comment_pull_request(apiuser, pullrequestid, repoid=<Optional:None>, message=<Optional:None>, commit_id=<Optional:None>, status=<Optional:None>, comment_type=<Optional:u'note'>, resolves_comment_id=<Optional:None>, extra_recipients=<Optional:[]>, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Comment on the pull request specified with the `pullrequestid`,
    in the |repo| specified by the `repoid`, and optionally change the
    review status.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Optional repository name or repository ID.
    :type repoid: str or int
    :param pullrequestid: The pull request ID.
    :type pullrequestid: int
    :param commit_id: Specify the commit_id for which to set a comment. If
        given commit_id is different than latest in the PR status
        change won't be performed.
    :type commit_id: str
    :param message: The text content of the comment.
    :type message: str
    :param status: (**Optional**) Set the approval status of the pull
        request. One of: 'not_reviewed', 'approved', 'rejected',
        'under_review'
    :type status: str
    :param comment_type: Comment type, one of: 'note', 'todo'
    :type comment_type: Optional(str), default: 'note'
+   :param resolves_comment_id: id of comment which this one will resolve
+   :type resolves_comment_id: Optional(int)
+   :param extra_recipients: list of user ids or usernames to add
+       notifications for this comment. Acts like a CC for notification
+   :type extra_recipients: Optional(list)
    :param userid: Comment on the pull request as this user
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result : {
            "pull_request_id":  "<Integer>",
            "comment_id":       "<Integer>",
            "status": {"given": <given_status>,
                       "was_changed": <bool status_was_actually_changed> },
        },
        error :  null
 
 
 create_pull_request 
 -------------------
 
 .. py:function:: create_pull_request(apiuser, source_repo, target_repo, source_ref, target_ref, owner=<Optional:<OptionalAttr:apiuser>>, title=<Optional:''>, description=<Optional:''>, description_renderer=<Optional:''>, reviewers=<Optional:None>)
 
    Creates a new pull request.
 
    Accepts refs in the following formats:
 
        * branch:<branch_name>:<sha>
        * branch:<branch_name>
        * bookmark:<bookmark_name>:<sha> (Mercurial only)
        * bookmark:<bookmark_name> (Mercurial only)
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param source_repo: Set the source repository name.
    :type source_repo: str
    :param target_repo: Set the target repository name.
    :type target_repo: str
    :param source_ref: Set the source ref name.
    :type source_ref: str
    :param target_ref: Set the target ref name.
    :type target_ref: str
    :param owner: user_id or username
    :type owner: Optional(str)
    :param title: Optionally Set the pull request title, it's generated otherwise
    :type title: str
    :param description: Set the pull request description.
    :type description: Optional(str)
    :type description_renderer: Optional(str)
    :param description_renderer: Set pull request renderer for the description.
        It should be 'rst', 'markdown' or 'plain'. If not give default
        system renderer will be used
    :param reviewers: Set the new pull request reviewers list.
        Reviewer defined by review rules will be added automatically to the
        defined list.
    :type reviewers: Optional(list)
        Accepts username strings or objects of the format:
 
            [{'username': 'nick', 'reasons': ['original author'], 'mandatory': <bool>}]
 
 
 get_pull_request 
 ----------------
 
-.. py:function:: get_pull_request(apiuser, pullrequestid, repoid=<Optional:None>)
+.. py:function:: get_pull_request(apiuser, pullrequestid, repoid=<Optional:None>, merge_state=<Optional:False>)
 
    Get a pull request based on the given ID.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Optional, repository name or repository ID from where
        the pull request was opened.
    :type repoid: str or int
    :param pullrequestid: ID of the requested pull request.
    :type pullrequestid: int
+   :param merge_state: Optional calculate merge state for each repository.
+       This could result in longer time to fetch the data
+   :type merge_state: bool
 
    Example output:
 
    .. code-block:: bash
 
      "id": <id_given_in_input>,
      "result":
        {
            "pull_request_id":   "<pull_request_id>",
            "url":               "<url>",
            "title":             "<title>",
            "description":       "<description>",
            "status" :           "<status>",
            "created_on":        "<date_time_created>",
            "updated_on":        "<date_time_updated>",
            "commit_ids":        [
                                     ...
                                     "<commit_id>",
                                     "<commit_id>",
                                     ...
                                 ],
            "review_status":    "<review_status>",
            "mergeable":         {
                                     "status":  "<bool>",
                                     "message": "<message>",
                                 },
            "source":            {
                                     "clone_url":     "<clone_url>",
                                     "repository":    "<repository_name>",
                                     "reference":
                                     {
                                         "name":      "<name>",
                                         "type":      "<type>",
                                         "commit_id": "<commit_id>",
                                     }
                                 },
            "target":            {
                                     "clone_url":   "<clone_url>",
                                     "repository":    "<repository_name>",
                                     "reference":
                                     {
                                         "name":      "<name>",
                                         "type":      "<type>",
                                         "commit_id": "<commit_id>",
                                     }
                                 },
            "merge":             {
                                     "clone_url":   "<clone_url>",
                                     "reference":
                                     {
                                         "name":      "<name>",
                                         "type":      "<type>",
                                         "commit_id": "<commit_id>",
                                     }
                                 },
           "author":             <user_obj>,
           "reviewers":          [
                                     ...
                                     {
                                        "user":          "<user_obj>",
                                        "review_status": "<review_status>",
                                     }
                                     ...
                                 ]
        },
       "error": null
 
 
 get_pull_request_comments 
 -------------------------
 
 .. py:function:: get_pull_request_comments(apiuser, pullrequestid, repoid=<Optional:None>)
 
    Get all comments of pull request specified with the `pullrequestid`
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Optional repository name or repository ID.
    :type repoid: str or int
    :param pullrequestid: The pull request ID.
    :type pullrequestid: int
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result : [
            {
              "comment_author": {
                "active": true,
                "full_name_or_username": "Tom Gore",
                "username": "admin"
              },
              "comment_created_on": "2017-01-02T18:43:45.533",
              "comment_f_path": null,
              "comment_id": 25,
              "comment_lineno": null,
              "comment_status": {
                "status": "under_review",
                "status_lbl": "Under Review"
              },
              "comment_text": "Example text",
              "comment_type": null,
              "pull_request_version": null
            }
        ],
        error :  null
 
 
 get_pull_requests 
 -----------------
 
-.. py:function:: get_pull_requests(apiuser, repoid, status=<Optional:'new'>, merge_state=<Optional:True>)
+.. py:function:: get_pull_requests(apiuser, repoid, status=<Optional:'new'>, merge_state=<Optional:False>)
 
    Get all pull requests from the repository specified in `repoid`.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Optional repository name or repository ID.
    :type repoid: str or int
    :param status: Only return pull requests with the specified status.
        Valid options are.
        * ``new`` (default)
        * ``open``
        * ``closed``
    :type status: str
    :param merge_state: Optional calculate merge state for each repository.
        This could result in longer time to fetch the data
    :type merge_state: bool
 
    Example output:
 
    .. code-block:: bash
 
      "id": <id_given_in_input>,
      "result":
        [
            ...
            {
                "pull_request_id":   "<pull_request_id>",
                "url":               "<url>",
                "title" :            "<title>",
                "description":       "<description>",
                "status":            "<status>",
                "created_on":        "<date_time_created>",
                "updated_on":        "<date_time_updated>",
                "commit_ids":        [
                                         ...
                                         "<commit_id>",
                                         "<commit_id>",
                                         ...
                                     ],
                "review_status":    "<review_status>",
                "mergeable":         {
                                        "status":      "<bool>",
                                        "message:      "<message>",
                                     },
                "source":            {
                                         "clone_url":     "<clone_url>",
                                         "reference":
                                         {
                                             "name":      "<name>",
                                             "type":      "<type>",
                                             "commit_id": "<commit_id>",
                                         }
                                     },
                "target":            {
                                         "clone_url":   "<clone_url>",
                                         "reference":
                                         {
                                             "name":      "<name>",
                                             "type":      "<type>",
                                             "commit_id": "<commit_id>",
                                         }
                                     },
                "merge":             {
                                         "clone_url":   "<clone_url>",
                                         "reference":
                                         {
                                             "name":      "<name>",
                                             "type":      "<type>",
                                             "commit_id": "<commit_id>",
                                         }
                                     },
               "author":             <user_obj>,
               "reviewers":          [
                                         ...
                                         {
                                            "user":          "<user_obj>",
                                            "review_status": "<review_status>",
                                         }
                                         ...
                                     ]
            }
            ...
        ],
      "error": null
 
 
 merge_pull_request 
 ------------------
 
 .. py:function:: merge_pull_request(apiuser, pullrequestid, repoid=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Merge the pull request specified by `pullrequestid` into its target
    repository.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Optional, repository name or repository ID of the
        target repository to which the |pr| is to be merged.
    :type repoid: str or int
    :param pullrequestid: ID of the pull request which shall be merged.
    :type pullrequestid: int
    :param userid: Merge the pull request as this user.
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
        "id": <id_given_in_input>,
        "result": {
            "executed":               "<bool>",
            "failure_reason":         "<int>",
            "merge_status_message":   "<str>",
            "merge_commit_id":        "<merge_commit_id>",
            "possible":               "<bool>",
            "merge_ref":        {
                                    "commit_id": "<commit_id>",
                                    "type":      "<type>",
                                    "name":      "<name>"
                                }
        },
        "error": null
 
 
 update_pull_request 
 -------------------
 
 .. py:function:: update_pull_request(apiuser, pullrequestid, repoid=<Optional:None>, title=<Optional:''>, description=<Optional:''>, description_renderer=<Optional:''>, reviewers=<Optional:None>, update_commits=<Optional:None>)
 
    Updates a pull request.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Optional repository name or repository ID.
    :type repoid: str or int
    :param pullrequestid: The pull request ID.
    :type pullrequestid: int
    :param title: Set the pull request title.
    :type title: str
    :param description: Update pull request description.
    :type description: Optional(str)
    :type description_renderer: Optional(str)
    :param description_renderer: Update pull request renderer for the description.
        It should be 'rst', 'markdown' or 'plain'
    :param reviewers: Update pull request reviewers list with new value.
    :type reviewers: Optional(list)
        Accepts username strings or objects of the format:
 
            [{'username': 'nick', 'reasons': ['original author'], 'mandatory': <bool>}]
 
    :param update_commits: Trigger update of commits for this pull request
    :type: update_commits: Optional(bool)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result : {
            "msg": "Updated pull request `63`",
            "pull_request": <pull_request_object>,
            "updated_reviewers": {
              "added": [
                "username"
              ],
              "removed": []
            },
            "updated_commits": {
              "added": [
                "<sha1_hash>"
              ],
              "common": [
                "<sha1_hash>",
                "<sha1_hash>",
              ],
              "removed": []
            }
        }
        error :  null
 
 

docs/api/methods/repo-methods.rst

@@ -1,1133 +1,1138 b''
 .. _repo-methods-ref:
 
 repo methods
 ============
 
 add_field_to_repo 
 -----------------
 
 .. py:function:: add_field_to_repo(apiuser, repoid, key, label=<Optional:''>, description=<Optional:''>)
 
    Adds an extra field to a repository.
 
    This command can only be run using an |authtoken| with at least
    write permissions to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository id.
    :type repoid: str or int
    :param key: Create a unique field key for this repository.
    :type key: str
    :param label:
    :type label: Optional(str)
    :param description:
    :type description: Optional(str)
 
 
 comment_commit 
 --------------
 
-.. py:function:: comment_commit(apiuser, repoid, commit_id, message, status=<Optional:None>, comment_type=<Optional:u'note'>, resolves_comment_id=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>)
+.. py:function:: comment_commit(apiuser, repoid, commit_id, message, status=<Optional:None>, comment_type=<Optional:u'note'>, resolves_comment_id=<Optional:None>, extra_recipients=<Optional:[]>, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Set a commit comment, and optionally change the status of the commit.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param commit_id: Specify the commit_id for which to set a comment.
    :type commit_id: str
    :param message: The comment text.
    :type message: str
    :param status: (**Optional**) status of commit, one of: 'not_reviewed',
        'approved', 'rejected', 'under_review'
    :type status: str
    :param comment_type: Comment type, one of: 'note', 'todo'
    :type comment_type: Optional(str), default: 'note'
+   :param resolves_comment_id: id of comment which this one will resolve
+   :type resolves_comment_id: Optional(int)
+   :param extra_recipients: list of user ids or usernames to add
+       notifications for this comment. Acts like a CC for notification
+   :type extra_recipients: Optional(list)
    :param userid: Set the user name of the comment creator.
    :type userid: Optional(str or int)
 
    Example error output:
 
    .. code-block:: bash
 
        {
            "id" : <id_given_in_input>,
            "result" : {
                "msg": "Commented on commit `<commit_id>` for repository `<repoid>`",
                "status_change": null or <status>,
                "success": true
            },
            "error" :  null
        }
 
 
 create_repo 
 -----------
 
-.. py:function:: create_repo(apiuser, repo_name, repo_type, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, push_uri=<Optional:None>, landing_rev=<Optional:'rev:tip'>, enable_statistics=<Optional:False>, enable_locking=<Optional:False>, enable_downloads=<Optional:False>, copy_permissions=<Optional:False>)
+.. py:function:: create_repo(apiuser, repo_name, repo_type, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, push_uri=<Optional:None>, landing_rev=<Optional:None>, enable_statistics=<Optional:False>, enable_locking=<Optional:False>, enable_downloads=<Optional:False>, copy_permissions=<Optional:False>)
 
    Creates a repository.
 
    * If the repository name contains "/", repository will be created inside
      a repository group or nested repository groups
 
      For example "foo/bar/repo1" will create |repo| called "repo1" inside
      group "foo/bar". You have to have permissions to access and write to
      the last repository group ("bar" in this example)
 
    This command can only be run using an |authtoken| with at least
    permissions to create repositories, or write permissions to
    parent repository groups.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repo_name: Set the repository name.
    :type repo_name: str
    :param repo_type: Set the repository type; 'hg','git', or 'svn'.
    :type repo_type: str
    :param owner: user_id or username
    :type owner: Optional(str)
    :param description: Set the repository description.
    :type description: Optional(str)
    :param private: set repository as private
    :type private: bool
    :param clone_uri: set clone_uri
    :type clone_uri: str
    :param push_uri: set push_uri
    :type push_uri: str
-   :param landing_rev: <rev_type>:<rev>
+   :param landing_rev: <rev_type>:<rev>, e.g branch:default, book:dev, rev:abcd
    :type landing_rev: str
    :param enable_locking:
    :type enable_locking: bool
    :param enable_downloads:
    :type enable_downloads: bool
    :param enable_statistics:
    :type enable_statistics: bool
    :param copy_permissions: Copy permission from group in which the
        repository is being created.
    :type copy_permissions: bool
 
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg": "Created new repository `<reponame>`",
                  "success": true,
                  "task": "<celery task id or None if done sync>"
                }
        error:  null
 
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
         'failed to create repository `<repo_name>`'
      }
 
 
 delete_repo 
 -----------
 
 .. py:function:: delete_repo(apiuser, repoid, forks=<Optional:''>)
 
    Deletes a repository.
 
    * When the `forks` parameter is set it's possible to detach or delete
      forks of deleted repository.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param forks: Set to `detach` or `delete` forks from the |repo|.
    :type forks: Optional(str)
 
    Example error output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg": "Deleted repository `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 fork_repo 
 ---------
 
-.. py:function:: fork_repo(apiuser, repoid, fork_name, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, landing_rev=<Optional:'rev:tip'>, copy_permissions=<Optional:False>)
+.. py:function:: fork_repo(apiuser, repoid, fork_name, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, landing_rev=<Optional:None>, copy_permissions=<Optional:False>)
 
    Creates a fork of the specified |repo|.
 
    * If the fork_name contains "/", fork will be created inside
      a repository group or nested repository groups
 
      For example "foo/bar/fork-repo" will create fork called "fork-repo"
      inside group "foo/bar". You have to have permissions to access and
      write to the last repository group ("bar" in this example)
 
    This command can only be run using an |authtoken| with minimum
    read permissions of the forked repo, create fork permissions for an user.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set repository name or repository ID.
    :type repoid: str or int
    :param fork_name: Set the fork name, including it's repository group membership.
    :type fork_name: str
    :param owner: Set the fork owner.
    :type owner: str
    :param description: Set the fork description.
    :type description: str
    :param copy_permissions: Copy permissions from parent |repo|. The
        default is False.
    :type copy_permissions: bool
    :param private: Make the fork private. The default is False.
    :type private: bool
-   :param landing_rev: Set the landing revision. The default is tip.
+   :param landing_rev: Set the landing revision. E.g branch:default, book:dev, rev:abcd
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_for_response>
        api_key : "<api_key>"
        args:     {
                    "repoid" :          "<reponame or repo_id>",
                    "fork_name":        "<forkname>",
                    "owner":            "<username or user_id = Optional(=apiuser)>",
                    "description":      "<description>",
                    "copy_permissions": "<bool>",
                    "private":          "<bool>",
                    "landing_rev":      "<landing_rev>"
                  }
 
    Example error output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg": "Created fork of `<reponame>` as `<forkname>`",
                  "success": true,
                  "task": "<celery task id or None if done sync>"
                }
        error:  null
 
 
 get_repo 
 --------
 
 .. py:function:: get_repo(apiuser, repoid, cache=<Optional:True>)
 
    Gets an existing repository by its name or repository_id.
 
    The members section so the output returns users groups or users
    associated with that repository.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository id.
    :type repoid: str or int
    :param cache: use the cached value for last changeset
    :type: cache: Optional(bool)
 
    Example output:
 
    .. code-block:: bash
 
        {
          "error": null,
          "id": <repo_id>,
          "result": {
            "clone_uri": null,
            "created_on": "timestamp",
            "description": "repo description",
            "enable_downloads": false,
            "enable_locking": false,
            "enable_statistics": false,
            "followers": [
              {
                "active": true,
                "admin": false,
                "api_key": "****************************************",
                "api_keys": [
                  "****************************************"
                ],
                "email": "[e-mail]",
                "emails": [
                  "[e-mail]"
                ],
                "extern_name": "rhodecode",
                "extern_type": "rhodecode",
                "firstname": "username",
                "ip_addresses": [],
                "language": null,
                "last_login": "2015-09-16T17:16:35.854",
                "lastname": "surname",
                "user_id": <user_id>,
                "username": "name"
              }
            ],
            "fork_of": "parent-repo",
            "landing_rev": [
              "rev",
              "tip"
            ],
            "last_changeset": {
              "author": "User <[e-mail]>",
              "branch": "default",
              "date": "timestamp",
              "message": "last commit message",
              "parents": [
                {
                  "raw_id": "commit-id"
                }
              ],
              "raw_id": "commit-id",
              "revision": <revision number>,
              "short_id": "short id"
            },
            "lock_reason": null,
            "locked_by": null,
            "locked_date": null,
            "owner": "owner-name",
            "permissions": [
              {
                "name": "super-admin-name",
                "origin": "super-admin",
                "permission": "repository.admin",
                "type": "user"
              },
              {
                "name": "owner-name",
                "origin": "owner",
                "permission": "repository.admin",
                "type": "user"
              },
              {
                "name": "user-group-name",
                "origin": "permission",
                "permission": "repository.write",
                "type": "user_group"
              }
            ],
            "private": true,
            "repo_id": 676,
            "repo_name": "user-group/repo-name",
            "repo_type": "hg"
          }
        }
 
 
 get_repo_changeset 
 ------------------
 
 .. py:function:: get_repo_changeset(apiuser, repoid, revision, details=<Optional:'basic'>)
 
    Returns information about a changeset.
 
    Additionally parameters define the amount of details returned by
    this function.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository id
    :type repoid: str or int
    :param revision: revision for which listing should be done
    :type revision: str
    :param details: details can be 'basic|extended|full' full gives diff
        info details like the diff itself, and number of changed files etc.
    :type details: Optional(str)
 
 
 get_repo_changesets 
 -------------------
 
 .. py:function:: get_repo_changesets(apiuser, repoid, start_rev, limit, details=<Optional:'basic'>)
 
    Returns a set of commits limited by the number starting
    from the `start_rev` option.
 
    Additional parameters define the amount of details returned by this
    function.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param start_rev: The starting revision from where to get changesets.
    :type start_rev: str
    :param limit: Limit the number of commits to this amount
    :type limit: str or int
    :param details: Set the level of detail returned. Valid option are:
        ``basic``, ``extended`` and ``full``.
    :type details: Optional(str)
 
    .. note::
 
       Setting the parameter `details` to the value ``full`` is extensive
       and returns details like the diff itself, and the number
       of changed files.
 
 
 get_repo_comments 
 -----------------
 
 .. py:function:: get_repo_comments(apiuser, repoid, commit_id=<Optional:None>, comment_type=<Optional:None>, userid=<Optional:None>)
 
    Get all comments for a repository
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param commit_id: Optionally filter the comments by the commit_id
    :type commit_id: Optional(str), default: None
    :param comment_type: Optionally filter the comments by the comment_type
        one of: 'note', 'todo'
    :type comment_type: Optional(str), default: None
    :param userid: Optionally filter the comments by the author of comment
    :type userid: Optional(str or int), Default: None
 
    Example error output:
 
    .. code-block:: bash
 
        {
            "id" : <id_given_in_input>,
            "result" : [
                {
                  "comment_author": <USER_DETAILS>,
                  "comment_created_on": "2017-02-01T14:38:16.309",
                  "comment_f_path": "file.txt",
                  "comment_id": 282,
                  "comment_lineno": "n1",
                  "comment_resolved_by": null,
                  "comment_status": [],
                  "comment_text": "This file needs a header",
                  "comment_type": "todo"
                }
            ],
            "error" :  null
        }
 
 
 get_repo_file 
 -------------
 
 .. py:function:: get_repo_file(apiuser, repoid, commit_id, file_path, max_file_bytes=<Optional:None>, details=<Optional:'basic'>, cache=<Optional:True>)
 
    Returns a single file from repository at given revision.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param commit_id: The revision for which listing should be done.
    :type commit_id: str
    :param file_path: The path from which to start displaying.
    :type file_path: str
    :param details: Returns different set of information about nodes.
        The valid options are ``minimal`` ``basic`` and ``full``.
    :type details: Optional(str)
    :param max_file_bytes: Only return file content under this file size bytes
    :type max_file_bytes: Optional(int)
    :param cache: Use internal caches for fetching files. If disabled fetching
        files is slower but more memory efficient
    :type cache: Optional(bool)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
            "binary": false,
            "extension": "py",
            "lines": 35,
            "content": "....",
            "md5": "76318336366b0f17ee249e11b0c99c41",
            "mimetype": "text/x-python",
            "name": "python.py",
            "size": 817,
            "type": "file",
        }
        error:  null
 
 
 get_repo_fts_tree 
 -----------------
 
 .. py:function:: get_repo_fts_tree(apiuser, repoid, commit_id, root_path)
 
    Returns a list of tree nodes for path at given revision. This api is built
    strictly for usage in full text search building, and shouldn't be consumed
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
 
 get_repo_nodes 
 --------------
 
 .. py:function:: get_repo_nodes(apiuser, repoid, revision, root_path, ret_type=<Optional:'all'>, details=<Optional:'basic'>, max_file_bytes=<Optional:None>)
 
    Returns a list of nodes and children in a flat list for a given
    path at given revision.
 
    It's possible to specify ret_type to show only `files` or `dirs`.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param revision: The revision for which listing should be done.
    :type revision: str
    :param root_path: The path from which to start displaying.
    :type root_path: str
    :param ret_type: Set the return type. Valid options are
        ``all`` (default), ``files`` and ``dirs``.
    :type ret_type: Optional(str)
    :param details: Returns extended information about nodes, such as
        md5, binary, and or content.
        The valid options are ``basic`` and ``full``.
    :type details: Optional(str)
    :param max_file_bytes: Only return file content under this file size bytes
    :type details: Optional(int)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: [
                    {
                      "binary": false,
                      "content": "File line",
                      "extension": "md",
                      "lines": 2,
                      "md5": "059fa5d29b19c0657e384749480f6422",
                      "mimetype": "text/x-minidsrc",
                      "name": "file.md",
                      "size": 580,
                      "type": "file"
                    },
                  ...
                ]
        error:  null
 
 
 get_repo_refs 
 -------------
 
 .. py:function:: get_repo_refs(apiuser, repoid)
 
    Returns a dictionary of current references. It returns
    bookmarks, branches, closed_branches, and tags for given repository
 
    It's possible to specify ret_type to show only `files` or `dirs`.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        "result": {
            "bookmarks": {
              "dev": "5611d30200f4040ba2ab4f3d64e5b06408a02188",
              "master": "367f590445081d8ec8c2ea0456e73ae1f1c3d6cf"
            },
            "branches": {
              "default": "5611d30200f4040ba2ab4f3d64e5b06408a02188",
              "stable": "367f590445081d8ec8c2ea0456e73ae1f1c3d6cf"
            },
            "branches_closed": {},
            "tags": {
              "tip": "5611d30200f4040ba2ab4f3d64e5b06408a02188",
              "v4.4.0": "1232313f9e6adac5ce5399c2a891dc1e72b79022",
              "v4.4.1": "cbb9f1d329ae5768379cdec55a62ebdd546c4e27",
              "v4.4.2": "24ffe44a27fcd1c5b6936144e176b9f6dd2f3a17",
            }
        }
        error:  null
 
 
 get_repo_settings 
 -----------------
 
 .. py:function:: get_repo_settings(apiuser, repoid, key=<Optional:None>)
 
    Returns all settings for a repository. If key is given it only returns the
    setting identified by the key or null.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository id.
    :type repoid: str or int
    :param key: Key of the setting to return.
    :type: key: Optional(str)
 
    Example output:
 
    .. code-block:: bash
 
        {
            "error": null,
            "id": 237,
            "result": {
                "extensions_largefiles": true,
                "extensions_evolve": true,
                "hooks_changegroup_push_logger": true,
                "hooks_changegroup_repo_size": false,
                "hooks_outgoing_pull_logger": true,
                "phases_publish": "True",
                "rhodecode_hg_use_rebase_for_merging": true,
                "rhodecode_pr_merge_enabled": true,
                "rhodecode_use_outdated_comments": true
            }
        }
 
 
 get_repos 
 ---------
 
 .. py:function:: get_repos(apiuser, root=<Optional:None>, traverse=<Optional:True>)
 
    Lists all existing repositories.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param root: specify root repository group to fetch repositories.
        filters the returned repositories to be members of given root group.
    :type root: Optional(None)
    :param traverse: traverse given root into subrepositories. With this flag
        set to False, it will only return top-level repositories from `root`.
        if root is empty it will return just top-level repositories.
    :type traverse: Optional(True)
 
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: [
                  {
                    "repo_id" :          "<repo_id>",
                    "repo_name" :        "<reponame>"
                    "repo_type" :        "<repo_type>",
                    "clone_uri" :        "<clone_uri>",
                    "private": :         "<bool>",
                    "created_on" :       "<datetimecreated>",
                    "description" :      "<description>",
                    "landing_rev":       "<landing_rev>",
                    "owner":             "<repo_owner>",
                    "fork_of":           "<name_of_fork_parent>",
                    "enable_downloads":  "<bool>",
                    "enable_locking":    "<bool>",
                    "enable_statistics": "<bool>",
                  },
                  ...
                ]
        error:  null
 
 
 grant_user_group_permission 
 ---------------------------
 
 .. py:function:: grant_user_group_permission(apiuser, repoid, usergroupid, perm)
 
    Grant permission for a user group on the specified repository,
    or update existing permissions.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param usergroupid: Specify the ID of the user group.
    :type usergroupid: str or int
    :param perm: Set the user group permissions using the following
        format: (repository.(none|read|write|admin))
    :type perm: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg" : "Granted perm: `<perm>` for group: `<usersgroupname>` in repo: `<reponame>`",
        "success": true
 
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to edit permission for user group: `<usergroup>` in repo `<repo>`'
      }
 
 
 grant_user_permission 
 ---------------------
 
 .. py:function:: grant_user_permission(apiuser, repoid, userid, perm)
 
    Grant permissions for the specified user on the given repository,
    or update existing permissions if found.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param userid: Set the user name.
    :type userid: str
    :param perm: Set the user permissions, using the following format
        ``(repository.(none|read|write|admin))``
    :type perm: str
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Granted perm: `<perm>` for user: `<username>` in repo: `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 invalidate_cache 
 ----------------
 
 .. py:function:: invalidate_cache(apiuser, repoid, delete_keys=<Optional:False>)
 
    Invalidates the cache for the specified repository.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Sets the repository name or repository ID.
    :type repoid: str or int
    :param delete_keys: This deletes the invalidated keys instead of
        just flagging them.
    :type delete_keys: Optional(``True`` | ``False``)
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'msg': Cache for repository `<repository name>` was invalidated,
        'repository': <repository name>
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error : {
         'Error occurred during cache invalidation action'
      }
 
 
 lock 
 ----
 
 .. py:function:: lock(apiuser, repoid, locked=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Sets the lock state of the specified |repo| by the given user.
    From more information, see :ref:`repo-locking`.
 
    * If the ``userid`` option is not set, the repository is locked to the
      user who called the method.
    * If the ``locked`` parameter is not set, the current lock state of the
      repository is displayed.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Sets the repository name or repository ID.
    :type repoid: str or int
    :param locked: Sets the lock state.
    :type locked: Optional(``True`` | ``False``)
    :param userid: Set the repository lock to this user.
    :type userid: Optional(str or int)
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'repo': '<reponame>',
        'locked': <bool: lock state>,
        'locked_since': <int: lock timestamp>,
        'locked_by': <username of person who made the lock>,
        'lock_reason': <str: reason for locking>,
        'lock_state_changed': <bool: True if lock state has been changed in this request>,
        'msg': 'Repo `<reponame>` locked by `<username>` on <timestamp>.'
        or
        'msg': 'Repo `<repository name>` not locked.'
        or
        'msg': 'User `<user name>` set lock state for repo `<repository name>` to `<new lock state>`'
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        'Error occurred locking repository `<reponame>`'
      }
 
 
 maintenance 
 -----------
 
 .. py:function:: maintenance(apiuser, repoid)
 
    Triggers a maintenance on the given repository.
 
    This command can only be run using an |authtoken| with admin
    rights to the specified repository. For more information,
    see :ref:`config-token-ref`.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "executed maintenance command",
        "executed_actions": [
           <action_message>, <action_message2>...
        ],
        "repository": "<repository name>"
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "Unable to execute maintenance on `<reponame>`"
      }
 
 
 pull 
 ----
 
 .. py:function:: pull(apiuser, repoid, remote_uri=<Optional:None>)
 
    Triggers a pull on the given repository from a remote location. You
    can use this to keep remote repositories up-to-date.
 
    This command can only be run using an |authtoken| with admin
    rights to the specified repository. For more information,
    see :ref:`config-token-ref`.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param remote_uri: Optional remote URI to pass in for pull
    :type remote_uri: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "Pulled from url `<remote_url>` on repo `<repository name>`"
        "repository": "<repository name>"
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "Unable to push changes from `<remote_url>`"
      }
 
 
 remove_field_from_repo 
 ----------------------
 
 .. py:function:: remove_field_from_repo(apiuser, repoid, key)
 
    Removes an extra field from a repository.
 
    This command can only be run using an |authtoken| with at least
    write permissions to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param key: Set the unique field key for this repository.
    :type key: str
 
 
 revoke_user_group_permission 
 ----------------------------
 
 .. py:function:: revoke_user_group_permission(apiuser, repoid, usergroupid)
 
    Revoke the permissions of a user group on a given repository.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param usergroupid: Specify the user group ID.
    :type usergroupid: str or int
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Revoked perm for group: `<usersgroupname>` in repo: `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 revoke_user_permission 
 ----------------------
 
 .. py:function:: revoke_user_permission(apiuser, repoid, userid)
 
    Revoke permission for a user on the specified repository.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param userid: Set the user name of revoked user.
    :type userid: str or int
 
    Example error output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Revoked perm for user: `<username>` in repo: `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 set_repo_settings 
 -----------------
 
 .. py:function:: set_repo_settings(apiuser, repoid, settings)
 
    Update repository settings. Returns true on success.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository id.
    :type repoid: str or int
    :param settings: The new settings for the repository.
    :type: settings: dict
 
    Example output:
 
    .. code-block:: bash
 
        {
            "error": null,
            "id": 237,
            "result": true
        }
 
 
 strip 
 -----
 
 .. py:function:: strip(apiuser, repoid, revision, branch)
 
    Strips the given revision from the specified repository.
 
    * This will remove the revision and all of its decendants.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param revision: The revision you wish to strip.
    :type revision: str
    :param branch: The branch from which to strip the revision.
    :type branch: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "'Stripped commit <commit_hash> from repo `<repository name>`'"
        "repository": "<repository name>"
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "Unable to strip commit <commit_hash> from repo `<repository name>`"
      }
 
 
 update_repo 
 -----------
 
-.. py:function:: update_repo(apiuser, repoid, repo_name=<Optional:None>, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, push_uri=<Optional:None>, landing_rev=<Optional:'rev:tip'>, fork_of=<Optional:None>, enable_statistics=<Optional:False>, enable_locking=<Optional:False>, enable_downloads=<Optional:False>, fields=<Optional:''>)
+.. py:function:: update_repo(apiuser, repoid, repo_name=<Optional:None>, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, push_uri=<Optional:None>, landing_rev=<Optional:None>, fork_of=<Optional:None>, enable_statistics=<Optional:False>, enable_locking=<Optional:False>, enable_downloads=<Optional:False>, fields=<Optional:''>)
 
    Updates a repository with the given information.
 
    This command can only be run using an |authtoken| with at least
    admin permissions to the |repo|.
 
    * If the repository name contains "/", repository will be updated
      accordingly with a repository group or nested repository groups
 
      For example repoid=repo-test name="foo/bar/repo-test" will update |repo|
      called "repo-test" and place it inside group "foo/bar".
      You have to have permissions to access and write to the last repository
      group ("bar" in this example)
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: repository name or repository ID.
    :type repoid: str or int
    :param repo_name: Update the |repo| name, including the
        repository group it's in.
    :type repo_name: str
    :param owner: Set the |repo| owner.
    :type owner: str
    :param fork_of: Set the |repo| as fork of another |repo|.
    :type fork_of: str
    :param description: Update the |repo| description.
    :type description: str
    :param private: Set the |repo| as private. (True | False)
    :type private: bool
    :param clone_uri: Update the |repo| clone URI.
    :type clone_uri: str
-   :param landing_rev: Set the |repo| landing revision. Default is ``rev:tip``.
+   :param landing_rev: Set the |repo| landing revision. e.g branch:default, book:dev, rev:abcd
    :type landing_rev: str
    :param enable_statistics: Enable statistics on the |repo|, (True | False).
    :type enable_statistics: bool
    :param enable_locking: Enable |repo| locking.
    :type enable_locking: bool
    :param enable_downloads: Enable downloads from the |repo|, (True | False).
    :type enable_downloads: bool
    :param fields: Add extra fields to the |repo|. Use the following
        example format: ``field_key=field_val,field_key2=fieldval2``.
        Escape ', ' with \,
    :type fields: str
 
 

docs/api/methods/search-methods.rst

@@ -1,35 +1,41 b''
 .. _search-methods-ref:
 
 search methods
 ==============
 
 search 
 ------
 
-.. py:function:: search(apiuser, search_query, search_type, page_limit=<Optional:10>, page=<Optional:1>, search_sort=<Optional:'newfirst'>, repo_name=<Optional:None>, repo_group_name=<Optional:None>)
+.. py:function:: search(apiuser, search_query, search_type, page_limit=<Optional:10>, page=<Optional:1>, search_sort=<Optional:'desc:date'>, repo_name=<Optional:None>, repo_group_name=<Optional:None>)
 
    Fetch Full Text Search results using API.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param search_query: Search query.
    :type search_query: str
    :param search_type: Search type. The following are valid options:
        * commit
        * content
        * path
    :type search_type: str
    :param page_limit: Page item limit, from 1 to 500. Default 10 items.
    :type page_limit: Optional(int)
    :param page: Page number. Default first page.
    :type page: Optional(int)
-   :param search_sort: Search sort order. Default newfirst. The following are valid options:
-       * newfirst
-       * oldfirst
+   :param search_sort: Search sort order.Must start with asc: or desc: Default desc:date.
+       The following are valid options:
+       * asc|desc:message.raw
+       * asc|desc:date
+       * asc|desc:author.email.raw
+       * asc|desc:message.raw
+       * newfirst (old legacy equal to desc:date)
+       * oldfirst (old legacy equal to asc:date)
+
    :type search_sort: Optional(str)
    :param repo_name: Filter by one repo. Default is all.
    :type repo_name: Optional(str)
    :param repo_group_name: Filter by one repo group. Default is all.
    :type repo_group_name: Optional(str)
 
 

docs/api/methods/store-methods.rst

@@ -1,37 +1,184 b''
 .. _store-methods-ref:
 
 store methods
 =============
 
 file_store_add (EE only)
 ------------------------
 
-.. py:function:: file_store_add(apiuser, filename, content)
+.. py:function:: file_store_add(apiuser, filename, content, description=<Optional:''>)
 
    Upload API for the file_store
 
    Example usage from CLI::
        rhodecode-api --instance-name=enterprise-1 upload_file "{"content": "$(cat image.jpg | base64)", "filename":"image.jpg"}"
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param filename: name of the file uploaded
    :type filename: str
+   :param description: Optional description for added file
+   :type description: str
    :param content: base64 encoded content of the uploaded file
    :type content: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result: {
        "access_path": "/_file_store/download/84d156f7-8323-4ad3-9fce-4a8e88e1deaf-0.jpg",
        "access_path_fqn": "http://server.domain.com/_file_store/download/84d156f7-8323-4ad3-9fce-4a8e88e1deaf-0.jpg",
        "store_fid": "84d156f7-8323-4ad3-9fce-4a8e88e1deaf-0.jpg"
      }
      error :  null
 
 
+file_store_add_with_acl (EE only)
+---------------------------------
+
+.. py:function:: file_store_add_with_acl(apiuser, filename, content, description=<Optional:''>, scope_user_id=<Optional:None>, scope_repo_id=<Optional:None>, scope_repo_group_id=<Optional:None>)
+
+   Upload API for the file_store
+
+   Example usage from CLI::
+       rhodecode-api --instance-name=enterprise-1 upload_file "{"content": "$(cat image.jpg | base64)", "filename":"image.jpg", "scope_repo_id":101}"
+
+   This command takes the following options:
+
+   :param apiuser: This is filled automatically from the |authtoken|.
+   :type apiuser: AuthUser
+   :param filename: name of the file uploaded
+   :type filename: str
+   :param description: Optional description for added file
+   :type description: str
+   :param content: base64 encoded content of the uploaded file
+   :type content: str
+
+   :param scope_user_id: Optionally bind this file to user.
+       This will check ACL in such way only this user can access the file.
+   :type scope_user_id: int
+   :param scope_repo_id: Optionally bind this file to repository.
+       This will check ACL in such way only user with proper access to such
+       repository can access the file.
+   :type scope_repo_id: int
+   :param scope_repo_group_id:  Optionally bind this file to repository group.
+       This will check ACL in such way only user with proper access to such
+       repository group can access the file.
+   :type scope_repo_group_id: int
+
+   Example output:
+
+   .. code-block:: bash
+
+     id : <id_given_in_input>
+     result: {
+       "access_path": "/_file_store/download/84d156f7-8323-4ad3-9fce-4a8e88e1deaf-0.jpg",
+       "access_path_fqn": "http://server.domain.com/_file_store/download/84d156f7-8323-4ad3-9fce-4a8e88e1deaf-0.jpg",
+       "store_fid": "84d156f7-8323-4ad3-9fce-4a8e88e1deaf-0.jpg"
+     }
+     error :  null
+
+
+file_store_get_info (EE only)
+-----------------------------
+
+.. py:function:: file_store_get_info(apiuser, store_fid)
+
+   Get artifact data.
+
+   Example output:
+
+   .. code-block:: bash
+
+     id : <id_given_in_input>
+     result: {
+         "artifact": {
+           "access_path_fqn": "https://rhodecode.example.com/_file_store/download/0-031c2aa0-0d56-49a7-9ba3-b570bdd342ab.jpg",
+           "created_on": "2019-10-15T16:25:35.491",
+           "description": "my upload",
+           "downloaded_times": 1,
+           "file_uid": "0-031c2aa0-0d56-49a7-9ba3-b570bdd342ab.jpg",
+           "filename": "example.jpg",
+           "filename_org": "0-031c2aa0-0d56-49a7-9ba3-b570bdd342ab.jpg",
+           "hidden": false,
+           "metadata": [
+             {
+               "artifact": "0-031c2aa0-0d56-49a7-9ba3-b570bdd342ab.jpg",
+               "key": "yellow",
+               "section": "tags",
+               "value": "bar"
+             }
+           ],
+           "sha256": "818dff0f44574dfb6814d38e6bf3c60c5943d1d13653398ecddaedf2f6a5b04d",
+           "size": 18599,
+           "uploaded_by": {
+             "email": "[e-mail]",
+             "emails": [
+               "[e-mail]"
+             ],
+             "firstname": "Admin",
+             "lastname": "LastName",
+             "user_id": 2,
+             "username": "admin"
+           }
+         }
+     }
+     error :  null
+
+
+file_store_add_metadata (EE only)
+---------------------------------
+
+.. py:function:: file_store_add_metadata(apiuser, store_fid, section, key, value, value_type=<Optional:'unicode'>)
+
+   Add metadata into artifact. The metadata consist of section, key, value. eg.
+   section='tags', 'key'='tag_name', value='1'
+
+   :param apiuser: This is filled automatically from the |authtoken|.
+   :type apiuser: AuthUser
+
+   :param store_fid: file uid, e.g 0-d054cb71-91ab-44e2-9e4b-23fe14b4d74a.mp4
+   :type store_fid: str
+
+   :param section: Section name to add metadata
+   :type section: str
+
+   :param key: Key to add as metadata
+   :type key: str
+
+   :param value: Value to add as metadata
+   :type value: str
+
+   :param value_type: Optional type, default is 'unicode' other types are:
+       int, list, bool, unicode, str
+
+   :type value_type: str
+
+   Example output:
+
+   .. code-block:: bash
+
+     id : <id_given_in_input>
+     result: {
+           "metadata": [
+             {
+               "artifact": "0-d054cb71-91ab-44e2-9e4b-23fe14b4d74a.mp4",
+               "key": "secret",
+               "section": "tags",
+               "value": "1"
+             },
+             {
+               "artifact": "0-d054cb71-91ab-44e2-9e4b-23fe14b4d74a.mp4",
+               "key": "video",
+               "section": "tags",
+               "value": "1"
+             }
+           ]
+     }
+     error :  null
+
+

docs/api/methods/user-group-methods.rst

@@ -1,418 +1,422 b''
 .. _user-group-methods-ref:
 
 user_group methods
 ==================
 
 add_user_to_user_group 
 ----------------------
 
 .. py:function:: add_user_to_user_group(apiuser, usergroupid, userid)
 
    Adds a user to a `user group`. If the user already exists in the group
    this command will return false.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified user group.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Set the name of the `user group` to which a
        user will be added.
    :type usergroupid: int
    :param userid: Set the `user_id` of the user to add to the group.
    :type userid: int
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
          "success": True|False # depends on if member is in group
          "msg": "added member `<username>` to user group `<groupname>` |
                  User is already in that group"
 
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to add member to user group `<user_group_name>`"
      }
 
 
 create_user_group 
 -----------------
 
 .. py:function:: create_user_group(apiuser, group_name, description=<Optional:''>, owner=<Optional:<OptionalAttr:apiuser>>, active=<Optional:True>, sync=<Optional:None>)
 
    Creates a new user group.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param group_name: Set the name of the new user group.
    :type group_name: str
    :param description: Give a description of the new user group.
    :type description: str
    :param owner: Set the owner of the new user group.
        If not set, the owner is the |authtoken| user.
    :type owner: Optional(str or int)
    :param active: Set this group as active.
    :type active: Optional(``True`` | ``False``)
    :param sync: Set enabled or disabled the automatically sync from
-       external authentication types like ldap.
+       external authentication types like ldap. If User Group will be named like
+       one from e.g ldap and sync flag is enabled members will be synced automatically.
+       Sync type when enabled via API is set to `manual_api`
    :type sync: Optional(``True`` | ``False``)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg": "created new user group `<groupname>`",
                  "user_group": <user_group_object>
                }
        error:  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "user group `<group name>` already exist"
        or
        "failed to create group `<group name>`"
      }
 
 
 delete_user_group 
 -----------------
 
 .. py:function:: delete_user_group(apiuser, usergroupid)
 
    Deletes the specified `user group`.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: filled automatically from apikey
    :type apiuser: AuthUser
    :param usergroupid:
    :type usergroupid: int
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "deleted user group ID:<user_group_id> <user_group_name>"
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to delete user group ID:<user_group_id> <user_group_name>"
        or
        "RepoGroup assigned to <repo_groups_list>"
      }
 
 
 get_user_group 
 --------------
 
 .. py:function:: get_user_group(apiuser, usergroupid)
 
    Returns the data of an existing user group.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Set the user group from which to return data.
    :type usergroupid: str or int
 
    Example error output:
 
    .. code-block:: bash
 
        {
          "error": null,
          "id": <id>,
          "result": {
            "active": true,
            "group_description": "group description",
            "group_name": "group name",
            "permissions": [
              {
                "name": "owner-name",
                "origin": "owner",
                "permission": "usergroup.admin",
                "type": "user"
              },
              {
              {
                "name": "user name",
                "origin": "permission",
                "permission": "usergroup.admin",
                "type": "user"
              },
              {
                "name": "user group name",
                "origin": "permission",
                "permission": "usergroup.write",
                "type": "user_group"
              }
            ],
            "permissions_summary": {
              "repositories": {
                "aa-root-level-repo-1": "repository.admin"
              },
              "repositories_groups": {}
            },
            "owner": "owner name",
            "users": [],
            "users_group_id": 2
          }
        }
 
 
 get_user_groups 
 ---------------
 
 .. py:function:: get_user_groups(apiuser)
 
    Lists all the existing user groups within RhodeCode.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
 
    Example error output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result : [<user_group_obj>,...]
        error : null
 
 
 grant_user_group_permission_to_user_group 
 -----------------------------------------
 
 .. py:function:: grant_user_group_permission_to_user_group(apiuser, usergroupid, sourceusergroupid, perm)
 
    Give one user group permissions to another user group.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Set the user group on which to edit permissions.
    :type usergroupid: str or int
    :param sourceusergroupid: Set the source user group to which
        access/permissions will be granted.
    :type sourceusergroupid: str or int
    :param perm: (usergroup.(none|read|write|admin))
    :type perm: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "Granted perm: `<perm_name>` for user group: `<source_user_group_name>` in user group: `<user_group_name>`",
        "success": true
      }
      error :  null
 
 
 grant_user_permission_to_user_group 
 -----------------------------------
 
 .. py:function:: grant_user_permission_to_user_group(apiuser, usergroupid, userid, perm)
 
    Set permissions for a user in a user group.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Set the user group to edit permissions on.
    :type usergroupid: str or int
    :param userid: Set the user from whom you wish to set permissions.
    :type userid: str
    :param perm: (usergroup.(none|read|write|admin))
    :type perm: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "Granted perm: `<perm_name>` for user: `<username>` in user group: `<user_group_name>`",
        "success": true
      }
      error :  null
 
 
 remove_user_from_user_group 
 ---------------------------
 
 .. py:function:: remove_user_from_user_group(apiuser, usergroupid, userid)
 
    Removes a user from a user group.
 
    * If the specified user is not in the group, this command will return
      `false`.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified user group.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Sets the user group name.
    :type usergroupid: str or int
    :param userid: The user you wish to remove from |RCE|.
    :type userid: str or int
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "success":  True|False,  # depends on if member is in group
                  "msg": "removed member <username> from user group <groupname> |
                          User wasn't in group"
                }
        error:  null
 
 
 revoke_user_group_permission_from_user_group 
 --------------------------------------------
 
 .. py:function:: revoke_user_group_permission_from_user_group(apiuser, usergroupid, sourceusergroupid)
 
    Revoke the permissions that one user group has to another.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Set the user group on which to edit permissions.
    :type usergroupid: str or int
    :param sourceusergroupid: Set the user group from which permissions
        are revoked.
    :type sourceusergroupid: str or int
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "Revoked perm for user group: `<user_group_name>` in user group: `<target_user_group_name>`",
        "success": true
      }
      error :  null
 
 
 revoke_user_permission_from_user_group 
 --------------------------------------
 
 .. py:function:: revoke_user_permission_from_user_group(apiuser, usergroupid, userid)
 
    Revoke a users permissions in a user group.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Set the user group from which to revoke the user
        permissions.
    :type: usergroupid: str or int
    :param userid: Set the userid of the user whose permissions will be
        revoked.
    :type userid: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "Revoked perm for user: `<username>` in user group: `<user_group_name>`",
        "success": true
      }
      error :  null
 
 
 update_user_group 
 -----------------
 
 .. py:function:: update_user_group(apiuser, usergroupid, group_name=<Optional:''>, description=<Optional:''>, owner=<Optional:None>, active=<Optional:True>, sync=<Optional:None>)
 
    Updates the specified `user group` with the details provided.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Set the id of the `user group` to update.
    :type usergroupid: str or int
    :param group_name: Set the new name the `user group`
    :type group_name: str
    :param description: Give a description for the `user group`
    :type description: str
    :param owner: Set the owner of the `user group`.
    :type owner: Optional(str or int)
    :param active: Set the group as active.
    :type active: Optional(``True`` | ``False``)
    :param sync: Set enabled or disabled the automatically sync from
-       external authentication types like ldap.
+       external authentication types like ldap. If User Group will be named like
+       one from e.g ldap and sync flag is enabled members will be synced automatically.
+       Sync type when enabled via API is set to `manual_api`
    :type sync: Optional(``True`` | ``False``)
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": 'updated user group ID:<user group id> <user group name>',
        "user_group": <user_group_object>
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to update user group `<user group name>`"
      }
 
 

docs/api/methods/user-methods.rst

@@ -1,326 +1,331 b''
 .. _user-methods-ref:
 
 user methods
 ============
 
 create_user 
 -----------
 
-.. py:function:: create_user(apiuser, username, email, password=<Optional:''>, firstname=<Optional:''>, lastname=<Optional:''>, active=<Optional:True>, admin=<Optional:False>, extern_name=<Optional:'rhodecode'>, extern_type=<Optional:'rhodecode'>, force_password_change=<Optional:False>, create_personal_repo_group=<Optional:None>)
+.. py:function:: create_user(apiuser, username, email, password=<Optional:''>, firstname=<Optional:''>, lastname=<Optional:''>, description=<Optional:''>, active=<Optional:True>, admin=<Optional:False>, extern_name=<Optional:'rhodecode'>, extern_type=<Optional:'rhodecode'>, force_password_change=<Optional:False>, create_personal_repo_group=<Optional:None>)
 
    Creates a new user and returns the new user object.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param username: Set the new username.
    :type username: str or int
    :param email: Set the user email address.
    :type email: str
    :param password: Set the new user password.
    :type password: Optional(str)
    :param firstname: Set the new user firstname.
    :type firstname: Optional(str)
    :param lastname: Set the new user surname.
    :type lastname: Optional(str)
+   :param description: Set user description, or short bio. Metatags are allowed.
+   :type description: Optional(str)
    :param active: Set the user as active.
    :type active: Optional(``True`` | ``False``)
    :param admin: Give the new user admin rights.
    :type admin: Optional(``True`` | ``False``)
    :param extern_name: Set the authentication plugin name.
        Using LDAP this is filled with LDAP UID.
    :type extern_name: Optional(str)
    :param extern_type: Set the new user authentication plugin.
    :type extern_type: Optional(str)
    :param force_password_change: Force the new user to change password
        on next login.
    :type force_password_change: Optional(``True`` | ``False``)
    :param create_personal_repo_group: Create personal repo group for this user
    :type create_personal_repo_group: Optional(``True`` | ``False``)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
            "msg" : "created new user `<username>`",
            "user": <user_obj>
        }
        error:  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "user `<username>` already exist"
        or
        "email `<email>` already exist"
        or
        "failed to create user `<username>`"
      }
 
 
 delete_user 
 -----------
 
 .. py:function:: delete_user(apiuser, userid)
 
    Deletes the specified user from the |RCE| user database.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    .. important::
 
       Ensure all open pull requests and open code review
       requests to this user are close.
 
       Also ensure all repositories, or repository groups owned by this
       user are reassigned before deletion.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param userid: Set the user to delete.
    :type userid: str or int
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
            "msg" : "deleted user ID:<userid> <username>",
            "user": null
        }
        error:  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to delete user ID:<userid> <username>"
      }
 
 
 get_user 
 --------
 
 .. py:function:: get_user(apiuser, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Returns the information associated with a username or userid.
 
    * If the ``userid`` is not set, this command returns the information
      for the ``userid`` calling the method.
 
    .. note::
 
       Normal users may only run this command against their ``userid``. For
       full privileges you must run this command using an |authtoken| with
       admin rights.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param userid: Sets the userid for which data will be returned.
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
        {
          "error": null,
          "id": <id>,
          "result": {
            "active": true,
            "admin": false,
            "api_keys": [ list of keys ],
            "auth_tokens": [ list of tokens with details ],
            "email": "[e-mail]",
            "emails": [
              "[e-mail]"
            ],
            "extern_name": "rhodecode",
            "extern_type": "rhodecode",
            "firstname": "username",
+           "description": "user description",
            "ip_addresses": [],
            "language": null,
            "last_login": "Timestamp",
            "last_activity": "Timestamp",
            "lastname": "surnae",
            "permissions": <deprecated>,
            "permissions_summary": {
              "global": [
                "hg.inherit_default_perms.true",
                "usergroup.read",
                "hg.repogroup.create.false",
                "hg.create.none",
                "hg.password_reset.enabled",
                "hg.extern_activate.manual",
                "hg.create.write_on_repogroup.false",
                "hg.usergroup.create.false",
                "group.none",
                "repository.none",
                "hg.register.none",
                "hg.fork.repository"
              ],
              "repositories": { "username/example": "repository.write"},
              "repositories_groups": { "user-group/repo": "group.none" },
              "user_groups": { "user_group_name": "usergroup.read" }
            }
            "user_id": 32,
            "username": "username"
          }
        }
 
 
 get_user_audit_logs 
 -------------------
 
 .. py:function:: get_user_audit_logs(apiuser, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Fetches all action logs made by the specified user.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param userid: Sets the userid whose list of locked |repos| will be
        displayed.
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result : {
            [action, action,...]
        }
        error :  null
 
 
 get_user_locks 
 --------------
 
 .. py:function:: get_user_locks(apiuser, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Displays all repositories locked by the specified user.
 
    * If this command is run by a non-admin user, it returns
      a list of |repos| locked by that user.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param userid: Sets the userid whose list of locked |repos| will be
        displayed.
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result : {
            [repo_object, repo_object,...]
        }
        error :  null
 
 
 get_users 
 ---------
 
 .. py:function:: get_users(apiuser)
 
    Lists all users in the |RCE| user database.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: [<user_object>, ...]
        error:  null
 
 
 update_user 
 -----------
 
-.. py:function:: update_user(apiuser, userid, username=<Optional:None>, email=<Optional:None>, password=<Optional:None>, firstname=<Optional:None>, lastname=<Optional:None>, active=<Optional:None>, admin=<Optional:None>, extern_type=<Optional:None>, extern_name=<Optional:None>)
+.. py:function:: update_user(apiuser, userid, username=<Optional:None>, email=<Optional:None>, password=<Optional:None>, firstname=<Optional:None>, lastname=<Optional:None>, description=<Optional:None>, active=<Optional:None>, admin=<Optional:None>, extern_type=<Optional:None>, extern_name=<Optional:None>)
 
    Updates the details for the specified user, if that user exists.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from |authtoken|.
    :type apiuser: AuthUser
    :param userid: Set the ``userid`` to update.
    :type userid: str or int
    :param username: Set the new username.
    :type username: str or int
    :param email: Set the new email.
    :type email: str
    :param password: Set the new password.
    :type password: Optional(str)
    :param firstname: Set the new first name.
    :type firstname: Optional(str)
    :param lastname: Set the new surname.
    :type lastname: Optional(str)
+   :param description: Set user description, or short bio. Metatags are allowed.
+   :type description: Optional(str)
    :param active: Set the new user as active.
    :type active: Optional(``True`` | ``False``)
    :param admin: Give the user admin rights.
    :type admin: Optional(``True`` | ``False``)
    :param extern_name: Set the authentication plugin user name.
        Using LDAP this is filled with LDAP UID.
    :type extern_name: Optional(str)
    :param extern_type: Set the authentication plugin type.
    :type extern_type: Optional(str)
 
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
            "msg" : "updated user ID:<userid> <username>",
            "user": <user_object>,
        }
        error:  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to update user `<username>`"
      }
 
 

docs/contributing/dev-setup.rst

@@ -1,243 +1,243 b''
 .. _dev-setup:
 
 ===================
  Development setup
 ===================
 
 
 RhodeCode Enterprise runs inside a Nix managed environment. This ensures build
 environment dependencies are correctly declared and installed during setup.
 It also enables atomic upgrades, rollbacks, and multiple instances of RhodeCode
 Enterprise running with isolation.
 
 To set up RhodeCode Enterprise inside the Nix environment, use the following steps:
 
 
 
 Setup Nix Package Manager
 -------------------------
 
 To install the Nix Package Manager, please run::
 
-   $ curl https://nixos.org/nix/install | sh
+   $ curl https://nixos.org/releases/nix/nix-2.0.4/install | sh
 
 or go to https://nixos.org/nix/ and follow the installation instructions.
 Once this is correctly set up on your system, you should be able to use the
 following commands:
 
 * `nix-env`
 
 * `nix-shell`
 
 
 .. tip::
 
    Update your channels frequently by running ``nix-channel --update``.
 
 .. note::
 
    To uninstall nix run the following:
 
     remove the . "$HOME/.nix-profile/etc/profile.d/nix.sh" line in your ~/.profile or ~/.bash_profile
     rm -rf $HOME/{.nix-channels,.nix-defexpr,.nix-profile,.config/nixpkgs}
     sudo rm -rf /nix
 
 Switch nix to the latest STABLE channel
 ---------------------------------------
 
 run::
 
    nix-channel --add https://nixos.org/channels/nixos-18.03 nixpkgs
 
 Followed by::
 
    nix-channel --update
    nix-env -i nix-2.0.4
 
 
 Install required binaries
 -------------------------
 
 We need some handy tools first.
 
 run::
 
     nix-env -i nix-prefetch-hg
     nix-env -i nix-prefetch-git
 
 
 Speed up JS build by installing PhantomJS
 -----------------------------------------
 
 PhantomJS will be downloaded each time nix-shell is invoked. To speed this by
 setting already downloaded version do this::
 
     nix-env -i phantomjs-2.1.1
 
     # and set nix bin path
     export PATH=$PATH:~/.nix-profile/bin
 
 
 Clone the required repositories
 -------------------------------
 
 After Nix is set up, clone the RhodeCode Enterprise Community Edition and
 RhodeCode VCSServer repositories into the same directory.
 RhodeCode currently is using Mercurial Version Control System, please make sure
 you have it installed before continuing.
 
 To obtain the required sources, use the following commands::
 
     mkdir rhodecode-develop && cd rhodecode-develop
     hg clone -u default https://code.rhodecode.com/rhodecode-enterprise-ce
     hg clone -u default https://code.rhodecode.com/rhodecode-vcsserver
 
 .. note::
 
    If you cannot clone the repository, please contact us via [e-mail]
 
 
 Install some required libraries
 -------------------------------
 
 There are some required drivers and dev libraries that we need to install to
 test RhodeCode under different types of databases. For example in Ubuntu we
 need to install the following.
 
 required libraries::
 
     # svn related
     sudo apt-get install libapr1-dev libaprutil1-dev
     sudo apt-get install libsvn-dev
     # libcurl required too
     sudo apt-get install libcurl4-openssl-dev
     # mysql/pg server for development, optional
     sudo apt-get install mysql-server libmysqlclient-dev
     sudo apt-get install postgresql postgresql-contrib libpq-dev
 
 
 
 Enter the Development Shell
 ---------------------------
 
 The final step is to start the development shells. To do this, run the
 following command from inside the cloned repository::
 
    # first, the vcsserver
    cd ~/rhodecode-vcsserver
    nix-shell
 
    # then enterprise sources
    cd ~/rhodecode-enterprise-ce
    nix-shell
 
 .. note::
 
    On the first run, this will take a while to download and optionally compile
    a few things. The following runs will be faster. The development shell works
    fine on both MacOS and Linux platforms.
 
 
 Create config.nix for development
 ---------------------------------
 
 In order to run proper tests and setup linking across projects, a config.nix
 file needs to be setup::
 
     # create config
     mkdir -p ~/.nixpkgs
     touch ~/.nixpkgs/config.nix
 
     # put the below content into the ~/.nixpkgs/config.nix file
     # adjusts, the path to where you cloned your repositories.
 
     {
       rc = {
        sources = {
         rhodecode-vcsserver = "/home/dev/rhodecode-vcsserver";
         rhodecode-enterprise-ce = "/home/dev/rhodecode-enterprise-ce";
         rhodecode-enterprise-ee = "/home/dev/rhodecode-enterprise-ee";
        };
       };
     }
 
 
 
 Creating a Development Configuration
 ------------------------------------
 
 To create a development environment for RhodeCode Enterprise,
 use the following steps:
 
 1. Create a copy of vcsserver config:
     `cp ~/rhodecode-vcsserver/configs/development.ini ~/rhodecode-vcsserver/configs/dev.ini`
 2. Create a copy of rhodocode config:
     `cp ~/rhodecode-enterprise-ce/configs/development.ini ~/rhodecode-enterprise-ce/configs/dev.ini`
 3. Adjust the configuration settings to your needs if needed.
 
 .. note::
 
   It is recommended to use the name `dev.ini` since it's included in .hgignore file.
 
 
 Setup the Development Database
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To create a development database, use the following example. This is a one
 time operation executed from the nix-shell of rhodecode-enterprise-ce sources ::
 
     rc-setup-app dev.ini \
         --user=admin --password=secret \
         --email=[e-mail] \
         --repos=~/my_dev_repos
 
 
 Compile CSS and JavaScript
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To use the application's frontend and prepare it for production deployment,
 you will need to compile the CSS and JavaScript with Grunt.
 This is easily done from within the nix-shell using the following command::
 
     make web-build
 
 When developing new features you will need to recompile following any
 changes made to the CSS or JavaScript files when developing the code::
 
     grunt watch
 
 This prepares the development (with comments/whitespace) versions of files.
 
 Start the Development Servers
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 From the rhodecode-vcsserver directory, start the development server in another
 nix-shell, using the following command::
 
       pserve configs/dev.ini
 
 In the adjacent nix-shell which you created for your development server, you may
 now start CE with the following command::
 
 
       pserve --reload configs/dev.ini
 
 .. note::
 
   `--reload` flag will automatically reload the server when source file changes.
 
 
 Run the Environment Tests
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Please make sure that the tests are passing to verify that your environment is
 set up correctly. RhodeCode uses py.test to run tests.
 While your instance is running, start a new nix-shell and simply run
 ``make test`` to run the basic test suite.
 
 
 Need Help?
 ^^^^^^^^^^
 
 Join us on Slack via https://rhodecode.com/join or post questions in our
 Community Portal at https://community.rhodecode.com

docs/index.rst

@@ -1,87 +1,93 b''
 |RCE|
 =====
 
 |RCE| is a high-performance source code management and collaboration system.
 It enables you to develop projects securely behind the firewall while
 providing collaboration tools that work with |git|, |hg|,
 and |svn| |repos|. The user interface allows you to create, edit,
 and commit files and |repos| while managing their security permissions.
 
 |RCE| provides the following features:
 
 * Source code management.
 * Extended permissions management.
 * Integrated code collaboration tools.
 * Integrated code review and notifications.
 * Scalability provided by multi-node setup.
 * Fully programmable automation API.
 * Web-based hook management.
 * Native |svn| support.
 * Migration from existing databases.
 * |RCE| SDK.
 * Built-in analytics
 * Built in integrations including: Slack, Webhooks (used for Jenkins/TeamCity and other CIs), Jira, Redmine, Hipchat
 * Pluggable authentication system.
 * Support for AD, |LDAP|, Crowd, CAS, PAM.
 * Support for external authentication via Oauth Google, Github, Bitbucket, Twitter.
 * Debug modes of operation.
 * Private and public gists.
 * Gists with limited lifetimes and within instance only sharing.
 * Fully integrated code search function.
 * Always on SSL connectivity.
 
 .. only:: html
 
    Table of Contents
    -----------------
 
 .. toctree::
    :maxdepth: 1
    :caption: Admin Documentation
 
    install/quick-start
    install/install-database
    install/install-steps
    admin/system-overview
    nix/default-env
    admin/system-admin
    admin/user-admin
    admin/repo-admin
    admin/security-tips
    auth/auth
    issue-trackers/issue-trackers
    admin/lab-settings
 
 .. toctree::
    :maxdepth: 1
    :caption: Feature Documentation
 
    collaboration/collaboration
    collaboration/review-notifications
    collaboration/pull-requests
    code-review/code-review
    integrations/integrations
 
 .. toctree::
    :maxdepth: 1
    :caption: User Documentation
 
    usage/basic-usage
    tutorials/tutorials
 
 .. toctree::
    :maxdepth: 1
    :caption: Developer Documentation
 
    api/api
    tools/rhodecode-tools
    extensions/extensions-hooks
    contributing/contributing
 
 .. toctree::
+   :maxdepth: 2
+   :caption: RhodeCode Control Documentation
+
+   RhodeCode Installer <https://docs.rhodecode.com/RhodeCode-Control/>
+
+.. toctree::
    :maxdepth: 1
    :caption: About
 
    known-issues/known-issues
    release-notes/release-notes
    admin/glossary

docs/install/configure-celery.rst

@@ -1,75 +1,85 b''
 
 .. _config-celery:
 
 Configure Celery
 ----------------
 
 Celery_ is an asynchronous task queue. It's a part of RhodeCode scheduler
 functionality. Celery_ makes certain heavy tasks perform more efficiently.
 Most important it allows sending notification emails, create repository forks,
 and import repositories in async way. It is also used for bi-directional
 repository sync in scheduler.
 
 If you decide to use Celery you also need a working message queue.
-The recommended and fully supported message broker is rabbitmq_.
+There are two fully supported message brokers is rabbitmq_ and redis_ (recommended).
+
+Since release 4.18.X we recommend using redis_ as a backend since it's generally
+easier to work with, and results in simpler stack as redis is generally recommended
+for caching purposes.
 
 
 In order to install and configure Celery, follow these steps:
 
-1. Install RabbitMQ, see the documentation on the Celery website for
-   `rabbitmq installation`_, or `rabbitmq website installation`_
+1. Install RabbitMQ or Redis for a message queue, see the documentation on the Celery website for
+   `redis installation`_ or `rabbitmq installation`_
 
 
-1a. As en example configuration after installation, you can run::
+1a. If you choose RabbitMQ example configuration after installation would look like that::
 
    sudo rabbitmqctl add_user rcuser secret_password
    sudo rabbitmqctl add_vhost rhodevhost
    sudo rabbitmqctl set_user_tags rcuser rhodecode
    sudo rabbitmqctl set_permissions -p rhodevhost rcuser ".*" ".*" ".*"
 
 
 2. Enable celery, and install `celery worker` process script using the `enable-module`::
 
     rccontrol enable-module celery {instance-id}
 
 .. note::
 
    In case when using multiple instances in one or multiple servers it's highly
    recommended that celery is running only once, for all servers connected to
    the same database. Having multiple celery instances running without special
    reconfiguration could cause scheduler issues.
 
 
 3. Configure Celery in the
    :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
    Set the broker_url as minimal settings required to enable operation.
    If used our example data from pt 1a, here is how the broker url should look like::
 
+        # for Redis
+        celery.broker_url = redis://localhost:6379/8
+
+        # for RabbitMQ
         celery.broker_url = amqp://rcuser:secret_password@localhost:5672/rhodevhost
 
    Full configuration example is below:
 
    .. code-block:: ini
 
         # Set this section of the ini file to match your Celery installation
         ####################################
         ###        CELERY CONFIG        ####
         ####################################
 
         use_celery = true
-        celery.broker_url = amqp://rcuser:secret@localhost:5672/rhodevhost
+        celery.broker_url = redis://localhost:6379/8
 
         # maximum tasks to execute before worker restart
         celery.max_tasks_per_child = 100
 
         ## tasks will never be sent to the queue, but executed locally instead.
         celery.task_always_eager = false
 
 
 .. _python: http://www.python.org/
 .. _mercurial: http://mercurial.selenic.com/
 .. _celery: http://celeryproject.org/
+.. _redis: http://redis.io
+.. _redis installation: https://redis.io/topics/quickstart
 .. _rabbitmq: http://www.rabbitmq.com/
 .. _rabbitmq installation: http://docs.celeryproject.org/en/latest/getting-started/brokers/rabbitmq.html
 .. _rabbitmq website installation: http://www.rabbitmq.com/download.html
 .. _Celery installation: http://docs.celeryproject.org/en/latest/getting-started/introduction.html#bundles

docs/install/database-string.rst

@@ -1,38 +1,37 b''
 .. _config-database:
 
 Make Database Changes
 ---------------------
 
 .. important::
 
    If you do change the |repo| database that |RCEE| uses, then you will need to
    upgrade the database, and also remap and rescan the |repos|. More detailed
    information is available in the
    :ref:`Alternative upgrade documentation <control:install-port>`.
 
 If you need to change database connection details for a |RCEE| instance,
 use the following steps:
 
 1. Open the :file:`rhodecode.ini` file for the instance you wish to edit. The
    default location is
    :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
 2. When you open the file, find the database configuration section,
    and use the below example to change the
    connection details:
 
 .. code-block:: ini
 
     #########################################################
     ### DB CONFIGS - EACH DB WILL HAVE IT'S OWN CONFIG    ###
     #########################################################
 
     # Default SQLite config
     sqlalchemy.db1.url = sqlite:////home/brian/.rccontrol/enterprise-1/rhodecode.db
 
     # Use this example for a PostgreSQL
     sqlalchemy.db1.url = postgresql://postgres:qwe@localhost/rhodecode
 
     # see sqlalchemy docs for other advanced settings
     sqlalchemy.db1.echo = false
     sqlalchemy.db1.pool_recycle = 3600
-    sqlalchemy.db1.convert_unicode = true

docs/issue-trackers/issue-trackers.rst

@@ -1,81 +1,76 b''
 .. _rhodecode-issue-trackers-ref:
 
 Issue Tracker Integration
 =========================
 
 You can set an issue tracker connection in two ways with |RCE|.
 
 * At the instance level, you can set a default issue tracker.
 * At the |repo| level, you can configure an integration with a different issue
   tracker.
 
 To integrate |RCE| with an issue tracker, you need to define a regular
 expression that will fetch the issue ID stored in commit messages, and replace
 it with a URL. This enables |RCE| to generate a link matching each issue to the
 target |repo|.
 
 Default Issue Tracker Configuration
 -----------------------------------
 
 To integrate your issue tracker, use the following steps:
 
 1. Open :menuselection:`Admin --> Settings --> Issue Tracker`.
 2. In the new entry field, enter the following information:
 
     * :guilabel:`Description`: A name for this set of rules.
     * :guilabel:`Pattern`: The regular expression that will match issues
       tagged in commit messages, or more see :ref:`issue-tr-eg-ref`.
     * :guilabel:`URL`: The URL to your issue tracker.
     * :guilabel:`Prefix`: The prefix with which you want to mark issues.
 
 3. Select **Add** so save the rule to your issue tracker configuration.
 
 Repository Issue Tracker Configuration
 --------------------------------------
 
 You can configure specific |repos| to use a different issue tracker than the
 default one. See the instructions in :ref:`repo-it`
 
 .. _issue-tr-eg-ref:
 
+
 Jira Integration
 ----------------
 
-* Regex = ``(?:^#|\s#)(\w+-\d+)``
-* URL = ``https://myissueserver.com/browse/${id}``
-* Issue Prefix = ``#``
+Please check examples in the view for configuration the issue trackers.
+
 
 Confluence (Wiki)
 -----------------
 
-* Regex = ``(?:conf-)([A-Z0-9]+)``
-* URL = ``https://example.atlassian.net/display/wiki/${id}/${repo_name}``
-* issue prefix = ``CONF-``
+Please check examples in the view for configuration the issue trackers.
+
 
 Redmine Integration
 -------------------
 
-* Regex = ``(issue-+\d+)``
-* URL = ``https://myissueserver.com/redmine/issue/${id}``
-* Issue Prefix = ``issue-``
+Please check examples in the view for configuration the issue trackers.
+
 
-Redmine (wiki)
---------------
+Redmine wiki Integration
+------------------------
 
-* Regex = ``(?:wiki-)([a-zA-Z0-9]+)``
-* URL = ``https://example.com/redmine/projects/wiki/${repo_name}``
-* Issue prefix = ``Issue-``
+Please check examples in the view for configuration the issue trackers.
+
 
 Pivotal Tracker
 ---------------
 
-* Regex = ``(?:pivot-)(?<project_id>\d+)-(?<story>\d+)``
-* URL = ``https://www.pivotaltracker.com/s/projects/${project_id}/stories/${story}``
-* Issue prefix = ``Piv-``
+Please check examples in the view for configuration the issue trackers.
+
 
 Trello
 ------
 
-* Regex = ``(?:trello-)(?<card_id>[a-zA-Z0-9]+)``
-* URL = ``https://trello.com/example.com/${card_id}``
-* Issue prefix = ``Trello-``
+Please check examples in the view for configuration the issue trackers.
+

docs/release-notes/release-notes-4.17.0.rst

@@ -1,85 +1,85 b''
 |RCE| 4.17.0 |RNS|
 ------------------
 
 Release Date
 ^^^^^^^^^^^^
 
 - 2019-07-04
 
 
 New Features
 ^^^^^^^^^^^^
 
 - New artifacts feature.
   Ability to store binary artifacts for repository with ACL
 - UI/UX refresh for most of the pages. This includes multiple fixes and improvements.
 - Diffs: store wide-diff mode in user sessions to store user preference for diff display.
 - Mercurial: added support for Mercurial 4.9
 - API: Added search API methods
 - Files: adding/editing allows previews for generated content.
 - Files: allowed multi file upload using UI.
 - Repository Groups: last change is now smartly calculated based on latest change
   from all it's children repositories.
 - Archives: it's now currently possible to download partial directories from files view.
 - SVN: allowed executing pre-commit code with rcextensions, also added example to
   validate SVN file size and paths on pre-commit level.
 
 
 General
 ^^^^^^^
 - Exception store: add filter for display and deletion.
 - Files: loading history doesn't display hidden and obsolete commits anymore.
 - Repositories: bring back missing watch action in summary view.
 - Admin: user groups is now using pure DB filtering to speed up display
   for large number of groups.
 - Mercurial: enabled full evolve+topic extensions when evolve is enabled.
 - Dependencies: bumped evolve to 8.5.1
 - Dependencies: bumped pyramid to 1.10.4
 - Dependencies: bumped psutil to 5.5.1
 - Dependencies: bumped pygments to 2.4.2
 - Dependencies: bumped pyramid to 1.10.4
 - Dependencies: bumped psycopg2 to 2.8.3
 - Dependencies [security]: updated colander to 1.7.0
 
 
 Security
 ^^^^^^^^
 
 - SSH: replaced pycrypto with cryptography to generate SSH keys as pycrypto isn't
   considered safe anymore.
 
 
 Performance
 ^^^^^^^^^^^
 
 - Config: updated header limits on gunicorn to prevent errors on large Mercurial repositories.
 - User sessions: added option to cleanup redis based sessions in user session interface.
 - Authentication: reduced usage of raw auth calls inside templates to speed up rendering.
 - Sessions: don't touch session for API calls. Before each API call created new session
   object which wasn't required.
 
 
 Fixes
 ^^^^^
 
-- hHooks: fixed more unicode problems with new pull-request link generator.
+- Hooks: fixed more unicode problems with new pull-request link generator.
 - Mercurial: fix ssh-server support for mercurial custom options.
 - Pull requests: updated metadata information for failed merges with multiple heads.
 - Pull requests: calculate ancestor in the same way as creation mode.
   Fixed problem with updates generating wrong diffs in case of merges.
 - Pull requests: fixed a bug in removal of multiple reviewers at once.
 - Summary: fix timeout issues loading summary page without styling.
 - SSH: fix invocation of custom hgrc.
 - SSH: call custom hooks via SSH backend
 - Markup: fix styling for check-lists.
 - Archives: allows downloading refs that have slashes and special refs. e.g f/feat1 branch names.
 - Files: ensure we generate archives with consistent hashing (except for .tar.gz which uses temp files names in header)
 - Files: fixed rendering of readme files under non-ascii paths.
 
 
 Upgrade notes
 ^^^^^^^^^^^^^
 
 - In this release we introduced new UI across the application.
   In case of problems with the display on your systems please send us info to [e-mail].
 

docs/release-notes/release-notes.rst

@@ -1,136 +1,137 b''
 .. _rhodecode-release-notes-ref:
 
 Release Notes
 =============
 
 |RCE| 4.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
+   release-notes-4.18.0.rst
    release-notes-4.17.4.rst
    release-notes-4.17.3.rst
    release-notes-4.17.2.rst
    release-notes-4.17.1.rst
    release-notes-4.17.0.rst
    release-notes-4.16.2.rst
    release-notes-4.16.1.rst
    release-notes-4.16.0.rst
    release-notes-4.15.2.rst
    release-notes-4.15.1.rst
    release-notes-4.15.0.rst
    release-notes-4.14.1.rst
    release-notes-4.14.0.rst
    release-notes-4.13.3.rst
    release-notes-4.13.2.rst
    release-notes-4.13.1.rst
    release-notes-4.13.0.rst
    release-notes-4.12.4.rst
    release-notes-4.12.3.rst
    release-notes-4.12.2.rst
    release-notes-4.12.1.rst
    release-notes-4.12.0.rst
    release-notes-4.11.6.rst
    release-notes-4.11.5.rst
    release-notes-4.11.4.rst
    release-notes-4.11.3.rst
    release-notes-4.11.2.rst
    release-notes-4.11.1.rst
    release-notes-4.11.0.rst
    release-notes-4.10.6.rst
    release-notes-4.10.5.rst
    release-notes-4.10.4.rst
    release-notes-4.10.3.rst
    release-notes-4.10.2.rst
    release-notes-4.10.1.rst
    release-notes-4.10.0.rst
    release-notes-4.9.1.rst
    release-notes-4.9.0.rst
    release-notes-4.8.0.rst
    release-notes-4.7.2.rst
    release-notes-4.7.1.rst
    release-notes-4.7.0.rst
    release-notes-4.6.1.rst
    release-notes-4.6.0.rst
    release-notes-4.5.2.rst
    release-notes-4.5.1.rst
    release-notes-4.5.0.rst
    release-notes-4.4.2.rst
    release-notes-4.4.1.rst
    release-notes-4.4.0.rst
    release-notes-4.3.1.rst
    release-notes-4.3.0.rst
    release-notes-4.2.1.rst
    release-notes-4.2.0.rst
    release-notes-4.1.2.rst
    release-notes-4.1.1.rst
    release-notes-4.1.0.rst
    release-notes-4.0.1.rst
    release-notes-4.0.0.rst
 
 |RCE| 3.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-3.8.4.rst
    release-notes-3.8.3.rst
    release-notes-3.8.2.rst
    release-notes-3.8.1.rst
    release-notes-3.8.0.rst
    release-notes-3.7.1.rst
    release-notes-3.7.0.rst
    release-notes-3.6.1.rst
    release-notes-3.6.0.rst
    release-notes-3.5.2.rst
    release-notes-3.5.1.rst
    release-notes-3.5.0.rst
    release-notes-3.4.1.rst
    release-notes-3.4.0.rst
    release-notes-3.3.4.rst
    release-notes-3.3.3.rst
    release-notes-3.3.2.rst
    release-notes-3.3.1.rst
    release-notes-3.3.0.rst
    release-notes-3.2.3.rst
    release-notes-3.2.2.rst
    release-notes-3.2.1.rst
    release-notes-3.2.0.rst
    release-notes-3.1.1.rst
    release-notes-3.1.0.rst
    release-notes-3.0.2.rst
    release-notes-3.0.1.rst
    release-notes-3.0.0.rst
 
 |RCE| 2.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-2.2.8.rst
    release-notes-2.2.7.rst
    release-notes-2.2.6.rst
    release-notes-2.2.5.rst
    release-notes-2.2.4.rst
    release-notes-2.2.3.rst
    release-notes-2.2.2.rst
    release-notes-2.2.1.rst
    release-notes-2.2.0.rst
    release-notes-2.1.0.rst
    release-notes-2.0.2.rst
    release-notes-2.0.1.rst
    release-notes-2.0.0.rst
 
 |RCE| 1.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-1.7.2.rst
    release-notes-1.7.1.rst
    release-notes-1.7.0.rst
    release-notes-1.6.0.rst

docs/tools/install-tools.rst

@@ -1,85 +1,88 b''
 .. _install-tools:
 
 |RCT| Installation
 ------------------
 
 As of |RCE| 3.4.1 |RCT| is installed automatically on the server with |RCE|. You
 do not need to install |RCT| on the server, but you will need to install them
 on machines that need remote access. The tools are linked to the instance
 folder, for example :file:`~/.rccontrol/{instance-id}/profile/bin`
 
 You can list the available tools using the following example, and the valid
 tools options are those which correspond with those in the :ref:`rc-tools`
 section.
 
 .. code-block:: bash
 
     $ ls ~/.rccontrol/enterprise-4/profile/bin/
 
     gen_js_i18n    rhodecode-cleanup-gists   rhodecode-tools  svnrdump
     gen_js_routes  rhodecode-cleanup-repos   supervisorctl    svnserve
     git            rhodecode-config          supervisord      svnsync
     gunicorn       rhodecode-extensions      svn              svnversion
     hg             rhodecode-gist            svnadmin         vcsserver
     paster         rhodecode-index           svndumpfilter
     rc-server      rhodecode-list-instances  svnlook
     rhodecode-api  rhodecode-setup-config    svnmucc
 
 You can then use the tools as described in the :ref:`rc-tools` section using the
 following example:
 
 .. code-block:: bash
 
     # Running the indexer
     $ ~/.rccontrol/enterprise-1/profile/bin/rhodecode-index \
         --instance-name=enterprise-1
 
     # Cleaning up gists
     $ ~/.rccontrol/enterprise-4/profile/bin/rhodecode-cleanup-gists \
         --instance-name=enterprise-4
 
     Scanning for gists in /home/brian/repos/.rc_gist_store...
     preparing to remove [1] found gists
 
 Installing |RCT|
 ^^^^^^^^^^^^^^^^
 
 |RCT| enable you to automate many of the most common |RCE| functions through
 the API. Installing them on a local machine lets you carry out maintenance on
 the server remotely. Once installed you can use them to index your |repos|
 to setup full-text search, strip commits, or install RhodeCode Extensions for
 additional functionality.
 
 For more detailed instructions about using |RCT| for indexing and full-text
 search, see :ref:`indexing-ref`
 
 To install |RCT|, use the following steps:
 
 1. Set up a ``virtualenv`` on your local machine, see virtualenv_ instructions
    here.
-2. Install |RCT| using pip. Full url with token is available at https://rhodecode.com/u/#rhodecode-tools
-   ``pip install -I https://dls.rhodecode.com/dls/<token>/rhodecode-tools/latest``
+2. Install |RCT| using pip. All downloadable versions of |RCT| are available at:
+   `https://code.rhodecode.com/rhodecode-tools-ce/artifacts`
 
+   Example installation::
+
+    pip install -I https://code.rhodecode.com/rhodecode-tools-ce/artifacts/download/0-10ac93f4-bb7d-4b97-baea-68110743dd5a.tar.gz
 
 Once |RCT| is installed using these steps there are a few extra
 configuration changes you can make. These are explained in more detail in the
 :ref:`indexing-ref` section, and the :ref:`rc-tools` section.
 
 .. code-block:: bash
 
     # Create a virtualenv
     brian@ubuntu:~$ virtualenv venv
     New python executable in venv/bin/python
     Installing setuptools, pip...done.
 
     # Activate the virtualenv
     brian@ubuntu:~$ . venv/bin/activate
 
     # Install RhodeCode Tools inside the virtualenv, full url with token is available at https://rhodecode.com/u/#rhodecode-tools
     $ pip install -I https://dls.rhodecode.com/dls/<token>/rhodecode-tools/latest
 
     # Check the installation
     $ rhodecode-tools --help
 
 .. _virtualenv: https://virtualenv.pypa.io/en/latest/index.html
 

docs/usage/keyboard-shortcuts.rst

@@ -1,59 +1,59 b''
 .. _keyboard-shortcuts:
 
 Keyboard Shortcuts
 ==================
 
 |RCE| has a number of user interface shortcut keys.
 
 System Wide Shortcuts
 ---------------------
 
 .. rst-class:: dl-horizontal
 
     \--:kbd:`/`
         Open the |repo| search box.
 
     \--:kbd:`?`
         Open the |RCE| keyboard shortcuts page.
 
     \--:kbd:`gh`
         Go to the home page.
 
     \--:kbd:`gg`
         Go to my private gists page.
 
     \--:kbd:`gG`
         Go to my public gists page.
 
     \--:kbd:`nr`
         Go to the create new repository page.
 
     \--:kbd:`ng`
         Go to the new gist page.
 
 Repository Shortcuts
 --------------------
 
 .. rst-class:: dl-horizontal
 
 
     \--:kbd:`gs`
         Go to the |repo| summary page.
 
     \--:kbd:`gc`
         Go to the changelog page.
 
     \--:kbd:`gf`
         Go to the files page.
 
     \--:kbd:`gF`
         Go to the files page with file search activated.
 
     \--:kbd:`gp`
         Go to the pull requests page.
 
     \--:kbd:`go`
         Go to the repository settings page.
 
     \--:kbd:`gO`
-        Go to the repository permissions settings.
+        Go to the repository access permissions settings.

grunt_config.json

@@ -1,177 +1,183 b''
 {
   "dirs": {
     "css": {
       "src": "rhodecode/public/css",
       "dest": "rhodecode/public/css"
     },
     "js": {
       "src": "rhodecode/public/js/src",
       "src_rc": "rhodecode/public/js/rhodecode",
       "dest": "rhodecode/public/js",
       "node_modules": "node_modules"
     }
   },
   "copy": {
     "main": {
       "files": [
         {
           "expand": true,
           "cwd": "node_modules/@webcomponents",
           "src": "webcomponentsjs/*.*",
           "dest": "<%= dirs.js.dest %>/vendors"
         },
         {
           "src": "<%= dirs.css.src %>/style-polymer.css",
           "dest": "<%= dirs.js.dest %>/src/components/style-polymer.css"
         }
       ]
     }
   },
   "concat": {
     "dist": {
       "src": [
         "<%= dirs.js.node_modules %>/jquery/dist/jquery.min.js",
         "<%= dirs.js.node_modules %>/mousetrap/mousetrap.min.js",
         "<%= dirs.js.node_modules %>/moment/min/moment.min.js",
         "<%= dirs.js.node_modules %>/clipboard/dist/clipboard.min.js",
         "<%= dirs.js.node_modules %>/favico.js/favico-0.3.10.min.js",
         "<%= dirs.js.node_modules %>/dropzone/dist/dropzone.js",
         "<%= dirs.js.node_modules %>/sticky-sidebar/dist/sticky-sidebar.min.js",
         "<%= dirs.js.node_modules %>/sticky-sidebar/dist/jquery.sticky-sidebar.min.js",
         "<%= dirs.js.node_modules %>/waypoints/lib/noframework.waypoints.min.js",
         "<%= dirs.js.node_modules %>/waypoints/lib/jquery.waypoints.min.js",
         "<%= dirs.js.node_modules %>/appenlight-client/appenlight-client.min.js",
         "<%= dirs.js.src %>/logging.js",
         "<%= dirs.js.src %>/bootstrap.js",
         "<%= dirs.js.src %>/i18n_utils.js",
         "<%= dirs.js.src %>/deform.js",
         "<%= dirs.js.src %>/ejs.js",
         "<%= dirs.js.src %>/ejs_templates/utils.js",
         "<%= dirs.js.src %>/plugins/jquery.pjax.js",
         "<%= dirs.js.src %>/plugins/jquery.dataTables.js",
         "<%= dirs.js.src %>/plugins/flavoured_checkbox.js",
         "<%= dirs.js.src %>/plugins/jquery.auto-grow-input.js",
         "<%= dirs.js.src %>/plugins/jquery.autocomplete.js",
         "<%= dirs.js.src %>/plugins/jquery.debounce.js",
         "<%= dirs.js.node_modules %>/mark.js/dist/jquery.mark.min.js",
         "<%= dirs.js.src %>/plugins/jquery.timeago.js",
         "<%= dirs.js.src %>/plugins/jquery.timeago-extension.js",
         "<%= dirs.js.src %>/select2/select2.js",
         "<%= dirs.js.src %>/codemirror/codemirror.js",
         "<%= dirs.js.src %>/codemirror/codemirror_loadmode.js",
         "<%= dirs.js.src %>/codemirror/codemirror_hint.js",
         "<%= dirs.js.src %>/codemirror/codemirror_overlay.js",
         "<%= dirs.js.src %>/codemirror/codemirror_placeholder.js",
         "<%= dirs.js.src %>/codemirror/codemirror_simplemode.js",
         "<%= dirs.js.dest %>/mode/meta.js",
         "<%= dirs.js.dest %>/mode/meta_ext.js",
         "<%= dirs.js.src_rc %>/i18n/select2/translations.js",
         "<%= dirs.js.src %>/rhodecode/utils/array.js",
         "<%= dirs.js.src %>/rhodecode/utils/string.js",
         "<%= dirs.js.src %>/rhodecode/utils/pyroutes.js",
         "<%= dirs.js.src %>/rhodecode/utils/ajax.js",
         "<%= dirs.js.src %>/rhodecode/utils/autocomplete.js",
         "<%= dirs.js.src %>/rhodecode/utils/colorgenerator.js",
         "<%= dirs.js.src %>/rhodecode/utils/ie.js",
         "<%= dirs.js.src %>/rhodecode/utils/os.js",
         "<%= dirs.js.src %>/rhodecode/utils/topics.js",
         "<%= dirs.js.src %>/rhodecode/init.js",
         "<%= dirs.js.src %>/rhodecode/changelog.js",
         "<%= dirs.js.src %>/rhodecode/codemirror.js",
         "<%= dirs.js.src %>/rhodecode/comments.js",
         "<%= dirs.js.src %>/rhodecode/constants.js",
         "<%= dirs.js.src %>/rhodecode/files.js",
         "<%= dirs.js.src %>/rhodecode/followers.js",
         "<%= dirs.js.src %>/rhodecode/menus.js",
         "<%= dirs.js.src %>/rhodecode/notifications.js",
         "<%= dirs.js.src %>/rhodecode/permissions.js",
         "<%= dirs.js.src %>/rhodecode/pjax.js",
         "<%= dirs.js.src %>/rhodecode/pullrequests.js",
         "<%= dirs.js.src %>/rhodecode/settings.js",
         "<%= dirs.js.src %>/rhodecode/select2_widgets.js",
         "<%= dirs.js.src %>/rhodecode/tooltips.js",
         "<%= dirs.js.src %>/rhodecode/users.js",
         "<%= dirs.js.src %>/rhodecode/appenlight.js",
         "<%= dirs.js.src %>/rhodecode.js",
         "<%= dirs.js.dest %>/rhodecode-components.js"
       ],
       "dest": "<%= dirs.js.dest %>/scripts.js",
       "nonull": true
     }
   },
+  "uglify": {
+    "dist": {
+      "src": "<%= dirs.js.dest %>/scripts.js",
+      "dest": "<%= dirs.js.dest %>/scripts.min.js"
+    }
+  },
   "less": {
     "development": {
       "options": {
         "compress": false,
         "yuicompress": false,
         "optimization": 0
       },
       "files": {
         "<%= dirs.css.dest %>/style.css": "<%= dirs.css.src %>/main.less",
         "<%= dirs.css.dest %>/style-polymer.css": "<%= dirs.css.src %>/polymer.less",
         "<%= dirs.css.dest %>/style-ipython.css": "<%= dirs.css.src %>/ipython.less"
       }
     },
     "production": {
       "options": {
         "compress": true,
         "yuicompress": true,
         "optimization": 2
       },
       "files": {
         "<%= dirs.css.dest %>/style.css": "<%= dirs.css.src %>/main.less",
         "<%= dirs.css.dest %>/style-polymer.css": "<%= dirs.css.src %>/polymer.less",
         "<%= dirs.css.dest %>/style-ipython.css": "<%= dirs.css.src %>/ipython.less"
       }
     },
     "components": {
       "files": [
         {
           "cwd": "<%= dirs.js.src %>/components/",
           "dest": "<%= dirs.js.src %>/components/",
           "src": [
             "**/*.less"
           ],
           "expand": true,
           "ext": ".css"
         }
       ]
     }
   },
   "watch": {
     "less": {
       "files": [
         "<%= dirs.css.src %>/**/*.less",
         "<%= dirs.js.src %>/components/**/*.less"
       ],
       "tasks": [
         "less:development",
         "less:components",
         "concat:polymercss",
         "webpack",
         "concat:dist"
       ]
     },
     "js": {
       "files": [
         "!<%= dirs.js.src %>/components/root-styles.gen.html",
         "<%= dirs.js.src %>/**/*.js",
         "<%= dirs.js.src %>/components/**/*.html"
       ],
       "tasks": [
         "less:components",
         "concat:polymercss",
         "webpack",
         "concat:dist"
       ]
     }
   },
   "jshint": {
     "rhodecode": {
       "src": "<%= dirs.js.src %>/rhodecode/**/*.js",
       "options": {
         "jshintrc": ".jshintrc"
       }
     }
   }
 }

package.json

@@ -1,60 +1,61 b''
 {
   "name": "rhodecode-enterprise",
   "version": "2.0.0",
   "private": true,
   "description" : "RhodeCode JS packaged",
   "license": "SEE LICENSE IN LICENSE.txt",
   "repository" : {
     "type" : "hg",
     "url" : "https://code.rhodecode.com/rhodecode-enterprise-ce"
   },
   "devDependencies": {
     "appenlight-client": "git+https://[e-mail]/AppEnlight/appenlight-client-js.git#0.5.1",
     "clipboard": "^2.0.1",
     "exports-loader": "^0.6.4",
     "favico.js": "^0.3.10",
     "dropzone": "^5.5.0",
     "grunt": "^0.4.5",
     "grunt-cli": "^1.3.1",
     "grunt-contrib-concat": "^0.5.1",
     "grunt-contrib-copy": "^1.0.0",
     "grunt-contrib-jshint": "^0.12.0",
     "grunt-contrib-less": "^1.1.0",
     "grunt-contrib-watch": "^0.6.1",
     "grunt-webpack": "^3.1.3",
+    "grunt-contrib-uglify": "^4.0.1",
     "jquery": "1.11.3",
     "mark.js": "8.11.1",
     "jshint": "^2.9.1-rc3",
     "moment": "^2.18.1",
     "mousetrap": "^1.6.1",
     "qrious": "^4.0.2",
     "sticky-sidebar": "3.3.1",
     "waypoints": "4.0.1",
     "webpack": "4.23.1",
     "webpack-cli": "3.1.2",
     "babel-core": "^6.26.3",
     "babel-loader": "^7.1.2",
     "babel-plugin-transform-object-rest-spread": "^6.26.0",
     "babel-preset-env": "^1.6.0",
     "copy-webpack-plugin": "^4.4.2",
     "css-loader": "^0.28.11",
     "html-loader": "^0.4.4",
     "html-webpack-plugin": "^3.2.0",
     "imports-loader": "^0.7.1",
     "polymer-webpack-loader": "^2.0.1",
     "style-loader": "^0.21.0",
     "webpack-uglify-js-plugin": "^1.1.9",
     "raw-loader": "1.0.0-beta.0",
     "ts-loader": "^1.3.3",
     "@webcomponents/webcomponentsjs": "^2.0.0",
     "@polymer/polymer": "^3.0.0",
     "@polymer/paper-button": "^3.0.0",
     "@polymer/paper-spinner": "^3.0.0",
     "@polymer/paper-tooltip": "^3.0.0",
     "@polymer/paper-toast": "^3.0.0",
     "@polymer/paper-toggle-button": "^3.0.0",
     "@polymer/iron-ajax": "^3.0.0",
     "@polymer/iron-autogrow-textarea": "^3.0.0",
     "@polymer/iron-a11y-keys": "^3.0.0"
   }
 }