release: Merge default into stable for release preparation

docs/admin/system_admin/tuning/tuning-rhodecode-memory-usage.rst

@@ -0,0 +1,26 b''
+.. _adjust-rhodecode-mem:
+
+RhodeCode Memory Usage
+----------------------
+
+Starting from Version 4.18.X RhodeCode has a builtin memory monitor for gunicorn workers.
+Enabling this can limit the maximum amount of memory system can use. Each worker
+for RhodeCode is monitored independently.
+To enable Memory management make sure to have following settings inside `[app:main] section` of
+:file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
+
+
+
+    ; Maximum memory usage that each worker can use before it will receive a
+    ; graceful restart signal 0 = memory monitoring is disabled
+    ; Examples: 268435456 (256MB), 536870912 (512MB)
+    ; 1073741824 (1GB), 2147483648 (2GB), 4294967296 (4GB)
+    memory_max_usage = 1073741824
+
+    ; How often in seconds to check for memory usage for each gunicorn worker
+    memory_usage_check_interval = 60
+
+    ; Threshold value for which we don't recycle worker if GarbageCollection
+    ; frees up enough resources. Before each restart we try to run GC on worker
+    ; in case we get enough free memory after that, restart will not happen.
+    memory_usage_recovery_threshold = 0.8

docs/release-notes/release-notes-4.18.0.rst

@@ -0,0 +1,230 b''
+|RCE| 4.18.0 |RNS|
+------------------
+
+Release Date
+^^^^^^^^^^^^
+
+- 2020-01-05
+
+
+New Features
+^^^^^^^^^^^^
+
+- Artifacts: are no longer in BETA. New info page is available for uploaded artifacts
+  which exposes some useful information like sha256, various access urls etc, and also
+  allows deletion of artifacts, and updating their description.
+- Artifacts: support new download url based on access to artifacts using new auth-token types.
+- Artifacts: added ability to store artifacts using API, and internal cli upload.
+  This allows uploading of artifacts that can have 100s of GBs in size efficiently.
+- Artifacts: added metadata logic to store various extra custom data for artifacts.
+- Comments: added support for adding comment attachments using the artifacts logic.
+  Logged in users can now pick or drag and drop attachments into comment forms.
+- Comments: enable linkification of certain patterns on comments in repo/pull request scopes.
+  This will render now active links to commits, pull-requests mentioned in comments body.
+- Jira: new update integration plugin.
+  Plugin now fetches possible transitions from tickets and show them to users in the interface.
+  Allow sending extra attributes during a transition like `resolution` message.
+- Navigation: Added new consistent and contextual way of creating new objects
+  likes gists, repositories, and repository groups using dedicated action (with a `+` sign)
+  available in the top navigation.
+- Hovercards: added new tooltips and hovercards to expose certain information for objects shown in UI.
+  RhodeCode usernames, issues, pull-requests will have active hovercard logic that will
+  load extra information about them and exposing them to users.
+- Files: all readme files found in repository file browser will be now rendered, allowing having readme per directory.
+- Search: expose line counts in search files information.
+- Audit-logs: expose download user audit logs as JSON file.
+- Users: added description field for users.
+  Allows users to write a short BIO, or description of their role in the organization.
+- Users: allow super-admins to change bound authentication type for users.
+  E.g internal rhodecode accounts can be changed to ldap easily from user settings page.
+- Pull requests: simplified the UI for display view, hide less important information and expose the most important ones.
+- Pull requests: add merge check that detects WIP marker in title.
+  Usually WIP in title means unfinished task that needs still some work, such marker will prevent accidental merges.
+- Pull requests: TODO comments have now a dedicated box below reviewers to keep track
+  of important TODOs that still need attention before review process is finalized.
+- Pull requests: participants of pull request will receive an email about update of a
+  pull requests with a small summary of changes made.
+- Pull requests: change the naming from #NUM into !NUM.
+  !NUM format is now parsed and linkified in comments and commit messages.
+- Pull requests: pull requests which state is changing can now be viewed with a limited view.
+- Pull requests: re-organize merge/close buttons and merge checks according to the new UI.
+- Pull requests: update commits button allows a force-refresh update now using dropdown option.
+- Pull requests: added quick filter to grid view to filter/search pull requests in a repository.
+- Pull requests: closing a pull-request without a merge requires additional confirmation now.
+- Pull requests: merge checks will now show which files caused conflicts and are blocking the merge.
+- Emails: updated all generated emails design and cleanup the data fields they expose.
+  a) More consistent UI for all types of emails. b) Improved formatting of plaintext emails
+  c) Added reply link to comment type emails for quicker response action.
+
+
+General
+^^^^^^^
+
+- Artifacts: don't show hidden artifacts, allow showing them via a GET ?hidden=1 flag.
+  Hidden artifacts are for example comment attachments.
+- UI: new commits page, according to the new design, which started on 4.17.X release lines
+- UI: use explicit named actions like "create user" instead of generic "save" which is bad UX.
+- UI: fixed problems with generating last change in repository groups.
+  There's now a new logic that checks all objects inside group for latest update time.
+- API: add artifact `get_info`, and `store_metadata` methods.
+- API: allowed to specify extra recipients for pr/commit comments api methods.
+- Vcsserver: set file based cache as default for vcsserver which can be shared
+  across multiple workers saving memory usage.
+- Vcsserver: added redis as possible cache backend for even greater performance.
+- Dependencies: bumped GIT version to 2.23.0
+- Dependencies: bumped SVN version to 1.12.2
+- Dependencies: bumped Mercurial version to 5.1.1 and hg-evolve to 9.1.0
+- Search: added logic for sorting ElasticSearch6 backend search results.
+- User bookmarks: make it easier to re-organize existing entries.
+- Data grids: hide pagination for single pages in grids.
+- Gists: UX, removed private/public gist buttons and replaced them with radio group.
+- Gunicorn: moved all configuration of gunicorn workers to .ini files.
+- Gunicorn: added worker memory management allowing setting maximum per-worker memory usage.
+- Automation: moved update groups task into celery task
+- Cache commits: add option to refresh caches manually from advanced pages.
+- Pull requests: add indication of state change in list of pull-requests and actually show them in the list.
+- Cache keys: register and self cleanup cache keys used for invalidation to prevent leaking lot of them into DB on worker recycle
+- Repo groups: removed locking inheritance flag from repo-groups. We'll deprecate this soon and this only brings in confusion
+- System snapshot: improved formatting for better readability
+- System info: expose data about vcsserver.
+- Packages: updated celery to 4.3.0 and switch default backend to redis instead of RabbitMQ.
+  Redis is stable enough and easier to install. Having Redis simplifies the stack as it's used in other parts of RhodeCode.
+- Dependencies: bumped alembic to 1.2.1
+- Dependencies: bumped amqp==2.5.2 and kombu==4.6.6
+- Dependencies: bumped atomicwrites==1.3.0
+- Dependencies: bumped cffi==1.12.3
+- Dependencies: bumped configparser==4.0.2
+- Dependencies: bumped deform==2.0.8
+- Dependencies: bumped dogpile.cache==0.9.0
+- Dependencies: bumped hupper==1.8.1
+- Dependencies: bumped mako to 1.1.0
+- Dependencies: bumped markupsafe to 1.1.1
+- Dependencies: bumped packaging==19.2
+- Dependencies: bumped paste==3.2.1
+- Dependencies: bumped pastescript==3.2.0
+- Dependencies: bumped pathlib2 to 2.3.4
+- Dependencies: bumped pluggy==0.13.0
+- Dependencies: bumped psutil to 5.6.3
+- Dependencies: bumped psutil==5.6.5
+- Dependencies: bumped psycopg2==2.8.4
+- Dependencies: bumped pycurl to 7.43.0.3
+- Dependencies: bumped pyotp==2.3.0
+- Dependencies: bumped pyparsing to 2.4.2
+- Dependencies: bumped pyramid-debugtoolbar==4.5.1
+- Dependencies: bumped pyramid-mako to 1.1.0
+- Dependencies: bumped redis to 3.3.8
+- Dependencies: bumped sqlalchemy to 1.3.8
+- Dependencies: bumped sqlalchemy==1.3.11
+- Dependencies: bumped test libraries.
+- Dependencies: freeze alembic==1.3.1
+- Dependencies: freeze python-dateutil
+- Dependencies: freeze redis==3.3.11
+- Dependencies: freeze supervisor==4.1.0
+
+
+Security
+^^^^^^^^
+
+- Security: fixed issues with exposing wrong http status (403) indicating repository with
+  given name exists and we don't have permissions to it. This was exposed in the redirection
+  logic of the global pull-request page. In case of redirection we also exposed
+  repository name in the URL.
+
+
+Performance
+^^^^^^^^^^^
+
+- Core: many various small improvements and optimizations to make rhodecode faster then before.
+- VCSServer: new cache implementation for remote functions.
+  Single worker shared caches that can use redis/file-cache.
+  This greatly improves performance on larger instances, and doesn't trigger cache
+  re-calculation on worker restarts.
+- GIT: switched internal git operations from Dulwich to libgit2 in order to obtain better performance and scalability.
+- SSH: skip loading unneeded application parts for SSH to make execution of ssh commands faster.
+- Main page: main page will now load repositories and repositories groups using partial DB calls instead of big JSON files.
+  In case of many repositories in root this could lead to very slow page rendering.
+- Admin pages: made all grids use same DB based partial loading logic. We'll no longer fetch
+  all objects into JSON for display purposes. This significantly improves speed of those pages in case
+  of many objects shown in them.
+- Summary page: use non-memory cache for readme, and cleanup cache for repo stats.
+  This change won't re-cache after worker restarts and can be shared across all workers
+- Files: only check for git_lfs/hg_largefiles if they are enabled.
+  This speeds up fetching of files if they are not LF and very big.
+- Vcsserver: added support for streaming data from the remote methods. This allows
+  to stream very large files without taking up memory, mostly for usage in SVN when
+  downloading large binaries from vcs system.
+- Files: added streaming remote attributes for vcsserver.
+  This change enables streaming raw content or raw downloads of large files without
+  transferring them over to enterprise for pack & repack using msgpack.
+  Msgpack has a limit of 2gb and generally pack+repack for ~2gb is very slow.
+- Files: ensure over size limit files never do any content fetching when viewing such files.
+- VCSServer: skip host verification to speed up pycurl calls.
+- User-bookmarks: cache fetching of bookmarks since this is quite expensive query to
+  make with joinedload on repos/repo groups.
+- Goto-switcher: reduce query data to only required attributes for speedups.
+- My account: owner/watched repos are now loaded only using DB queries.
+
+
+Fixes
+^^^^^
+
+- Mercurial: move imports from top-level to prevent from loading mercurial code on hook execution for svn/git.
+- GIT: limit sync-fetch logic to only retrieve tags/ and heads/ with default execution arguments.
+- GIT: fixed issue with git submodules detection.
+- SVN: fix checkout url for ssh+svn backend not having special prefix resulting in incorrect command shown.
+- SVN: fixed problem with showing empty directories.
+- OAuth: use a vendored version of `authomatic` library, and switch Bitbucket authentication to use oauth2.
+- Diffs: handle paths with quotes in diffs.
+- Diffs: fixed outdated files in pull-requests re-using the filediff raw_id for anchor generation. Fixes #5567
+- Diffs: toggle race condition on sticky vs wide-diff-mode that caused some display problems on larger diffs.
+- Pull requests: handle exceptions in state change and improve logging.
+- Pull requests: fixed title/description generation for single commits which are numbers.
+- Pull requests: changed the source of changes to be using shadow repos if it exists.
+  In case of `git push -f` and rebase we lost commits in the repo resulting in
+  problems of displaying versions of pull-requests.
+- Pull requests: handle case when removing existing files from a repository in compare versions diff.
+- Files: don't expose copy content helper in case of binary files.
+- Registration: properly expose first_name/last_name into email on user registration.
+- Markup renderers: fixed broken code highlight for rst files.
+- Ui: make super admin be named consistently across ui.
+- Audit logs: fixed search cases with special chars such as `-`.
+
+
+Upgrade notes
+^^^^^^^^^^^^^
+
+- New Automation task. We've changed the logic for updating latest change inside repository group.
+  New logic includes scanning for changes in all nested objects. Since this is a heavy task
+  a new dedicated scheduler task has been created to update it automatically on a scheduled base.
+  Please review in `admin > settings > automation` to enable this task.
+
+- New safer encryption algorithm. Some setting values are encrypted before storing it inside the database.
+  To keep full backward compatibility old AES algorithm is used.
+  If you wish to enable a safer option set fernet encryption instead inside rhodecode.ini
+  `rhodecode.encrypted_values.algorithm = fernet`
+
+- Pull requests UI changes. We've simplified the UI on pull requests page.
+  Please review the new UI to prevent surprises. All actions from old UI should be still possible with the new one.
+
+- Redis is now a default recommended backend for Celery and replaces previous rabbitmq.
+  Redis is generally easier to manage and install, and it's also very stable for usage
+  in the scheduler/celery async tasks. Since we also recommend Redis for caches the application
+  stack can be simplified by removing rabbitmq and replacing it with single Redis instance.
+
+- Recommendation for using Redis as the new cache backend on vcsserver.
+  Since Version 4.18.0 VCSServer has a new cache implementation for VCS data.
+  By default, for simplicity the cache type is file based. We strongly recommend using
+  Redis instead for better Performance and scalability
+  Please review vcsserver.ini settings under:
+  `rc_cache.repo_object.backend = dogpile.cache.rc.redis_msgpack`
+
+- New memory monitoring for Gunicorn workers. Starting from 4.18 release a option was added
+  to limit the maximum amount of memory used by a worker.
+  Please review new settings in `[server:main]` section for memory management in both
+  rhodecode.ini and vcsserver.ini::
+
+    ; Maximum memory usage that each worker can use before it will receive a
+    ; graceful restart signal 0 = memory monitoring is disabled
+    ; Examples: 268435456 (256MB), 536870912 (512MB)
+    ; 1073741824 (1GB), 2147483648 (2GB), 4294967296 (4GB)
+    memory_max_usage = 0

rhodecode/apps/file_store/backends/__init__.py

@@ -0,0 +1,19 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2016-2019 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/

rhodecode/apps/hovercards/__init__.py

@@ -0,0 +1,41 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2018-2019 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+
+def includeme(config):
+
+    config.add_route(
+        name='hovercard_user',
+        pattern='/_hovercard/user/{user_id}')
+
+    config.add_route(
+        name='hovercard_user_group',
+        pattern='/_hovercard/user_group/{user_group_id}')
+
+    config.add_route(
+        name='hovercard_pull_request',
+        pattern='/_hovercard/pull_request/{pull_request_id}')
+
+    config.add_route(
+        name='hovercard_repo_commit',
+        pattern='/_hovercard/commit/{repo_name:.*?[^/]}/{commit_id}', repo_route=True)
+
+    # Scan module for configuration decorators.
+    config.scan('.views', ignore='.tests')

rhodecode/apps/hovercards/views.py

@@ -0,0 +1,110 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2016-2019 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+import re
+import logging
+import collections
+
+from pyramid.httpexceptions import HTTPNotFound
+from pyramid.view import view_config
+
+from rhodecode.apps._base import BaseAppView, RepoAppView
+from rhodecode.lib import helpers as h
+from rhodecode.lib.auth import (
+    LoginRequired, NotAnonymous, HasRepoGroupPermissionAnyDecorator, CSRFRequired,
+    HasRepoPermissionAnyDecorator)
+from rhodecode.lib.codeblocks import filenode_as_lines_tokens
+from rhodecode.lib.index import searcher_from_config
+from rhodecode.lib.utils2 import safe_unicode, str2bool, safe_int
+from rhodecode.lib.ext_json import json
+from rhodecode.lib.vcs.exceptions import CommitDoesNotExistError, EmptyRepositoryError
+from rhodecode.lib.vcs.nodes import FileNode
+from rhodecode.model.db import (
+    func, true, or_, case, in_filter_generator, Repository, RepoGroup, User, UserGroup, PullRequest)
+from rhodecode.model.repo import RepoModel
+from rhodecode.model.repo_group import RepoGroupModel
+from rhodecode.model.scm import RepoGroupList, RepoList
+from rhodecode.model.user import UserModel
+from rhodecode.model.user_group import UserGroupModel
+
+log = logging.getLogger(__name__)
+
+
+class HoverCardsView(BaseAppView):
+
+    def load_default_context(self):
+        c = self._get_local_tmpl_context()
+        return c
+
+    @LoginRequired()
+    @view_config(
+        route_name='hovercard_user', request_method='GET', xhr=True,
+        renderer='rhodecode:templates/hovercards/hovercard_user.mako')
+    def hovercard_user(self):
+        c = self.load_default_context()
+        user_id = self.request.matchdict['user_id']
+        c.user = User.get_or_404(user_id)
+        return self._get_template_context(c)
+
+    @LoginRequired()
+    @view_config(
+        route_name='hovercard_user_group', request_method='GET', xhr=True,
+        renderer='rhodecode:templates/hovercards/hovercard_user_group.mako')
+    def hovercard_user_group(self):
+        c = self.load_default_context()
+        user_group_id = self.request.matchdict['user_group_id']
+        c.user_group = UserGroup.get_or_404(user_group_id)
+        return self._get_template_context(c)
+
+    @LoginRequired()
+    @view_config(
+        route_name='hovercard_pull_request', request_method='GET', xhr=True,
+        renderer='rhodecode:templates/hovercards/hovercard_pull_request.mako')
+    def hovercard_pull_request(self):
+        c = self.load_default_context()
+        c.pull_request = PullRequest.get_or_404(
+            self.request.matchdict['pull_request_id'])
+        perms = ['repository.read', 'repository.write', 'repository.admin']
+        c.can_view_pr = h.HasRepoPermissionAny(*perms)(
+            c.pull_request.target_repo.repo_name)
+        return self._get_template_context(c)
+
+
+class HoverCardsRepoView(RepoAppView):
+    def load_default_context(self):
+        c = self._get_local_tmpl_context()
+        return c
+
+    @LoginRequired()
+    @HasRepoPermissionAnyDecorator('repository.read', 'repository.write', 'repository.admin')
+    @view_config(
+        route_name='hovercard_repo_commit', request_method='GET', xhr=True,
+        renderer='rhodecode:templates/hovercards/hovercard_repo_commit.mako')
+    def hovercard_repo_commit(self):
+        c = self.load_default_context()
+        commit_id = self.request.matchdict['commit_id']
+        pre_load = ['author', 'branch', 'date', 'message']
+        try:
+            c.commit = self.rhodecode_vcs_repo.get_commit(
+                commit_id=commit_id, pre_load=pre_load)
+        except (CommitDoesNotExistError, EmptyRepositoryError):
+            raise HTTPNotFound()
+
+        return self._get_template_context(c)

.bumpversion.cfg

@@ -1,5 +1,5 b''
 [bumpversion]
-current_version = 4.17.4
+current_version = 4.18.0
 message = release: Bump version {current_version} to {new_version}
 
 [bumpversion:file:rhodecode/VERSION]

.coveragerc

@@ -8,6 +8,7 b' include ='
 omit =
     rhodecode/lib/dbmigrate/*
     rhodecode/lib/paster_commands/*
+    rhodecode/lib/_vendor/*
 
 [report]
 

.hgignore

@@ -45,6 +45,7 b' syntax: regexp'
 ^rhodecode/public/js/rhodecode-components.html$
 ^rhodecode/public/js/rhodecode-components.js$
 ^rhodecode/public/js/scripts.js$
+^rhodecode/public/js/scripts.min.js$
 ^rhodecode/public/js/src/components/root-styles.gen.html$
 ^rhodecode/public/js/vendors/webcomponentsjs/
 ^rhodecode\.db$

.release.cfg

@@ -5,25 +5,20 b' done = false'
 done = true
 
 [task:rc_tools_pinned]
-done = true
 
 [task:fixes_on_stable]
-done = true
 
 [task:pip2nix_generated]
-done = true
 
 [task:changelog_updated]
-done = true
 
 [task:generate_api_docs]
-done = true
+
+[task:updated_translation]
 
 [release]
-state = prepared
-version = 4.17.4
-
-[task:updated_translation]
+state = in_progress
+version = 4.18.0
 
 [task:generate_js_routes]
 

Gruntfile.js

@@ -13,9 +13,10 b' module.exports = function(grunt) {'
 
   grunt.loadNpmTasks('grunt-contrib-less');
   grunt.loadNpmTasks('grunt-contrib-concat');
+  grunt.loadNpmTasks('grunt-contrib-uglify');
   grunt.loadNpmTasks('grunt-contrib-watch');
   grunt.loadNpmTasks('grunt-contrib-jshint');
   grunt.loadNpmTasks('grunt-contrib-copy');
   grunt.loadNpmTasks('grunt-webpack');
-  grunt.registerTask('default', ['less:production', 'less:components', 'copy', 'webpack', 'concat:dist']);
+  grunt.registerTask('default', ['less:production', 'less:components', 'copy', 'webpack', 'concat:dist', 'uglify:dist']);
 };

Makefile

@@ -17,6 +17,7 b' test:'
 test-clean:
 	rm -rf coverage.xml htmlcov junit.xml pylint.log result
 	find . -type d -name "__pycache__" -prune -exec rm -rf '{}' ';'
+	find . -type f \( -iname '.coverage.*' \) -exec rm '{}' ';'
 
 test-only:
 	PYTHONHASHSEED=random \
@@ -28,7 +29,7 b' test-only-mysql:'
 	PYTHONHASHSEED=random \
 	py.test -x -vv -r xw -p no:sugar --cov=rhodecode \
     --cov-report=term-missing --cov-report=html \
-    --ini-config-override='{"app:main": {"sqlalchemy.db1.url": "mysql://root:qweqwe@localhost/rhodecode_test"}}' \
+    --ini-config-override='{"app:main": {"sqlalchemy.db1.url": "mysql://root:qweqwe@localhost/rhodecode_test?charset=utf8"}}' \
     rhodecode
 
 test-only-postgres:

README.rst

@@ -5,20 +5,24 b' RhodeCode'
 About
 -----
 
-``RhodeCode`` is a fast and powerful management tool for Mercurial_ and GIT_
-and Subversion_ with a built in push/pull server, full text search,
-pull requests and powerful code-review system. It works on http/https, SSH and
-has a few unique features like:
+``RhodeCode`` is a fast and powerful source code management tool for
+Mercurial_, GIT_ and Subversion_. It's main features are:
+
 
-- plugable architecture from Pyramid web-framework.
-- advanced permission system with IP restrictions, inheritation, and user-groups.
+- built in push/pull server
+- SSH with key management support
+- full text search.
+- plugable authentication.
+- pull requests and powerful code-review system.
+- advanced permission system with IP restrictions, permission inheritation, and user-groups.
 - rich set of authentication plugins including LDAP, ActiveDirectory, SAML 2.0,
   Atlassian Crowd, Http-Headers, Pam, Token-Auth, OAuth.
 - live code-review chat, and reviewer rules.
 - full web based file editing.
 - unified multi vcs support.
 - snippets (gist) system.
-- integration framework for Slack, CI systems, Webhooks.
+- artfacts store for binaries.
+- integration framework for Slack, CI systems, Webhooks, Jira, Redmine etc.
 - integration with all 3rd party issue trackers.
 
 
@@ -41,7 +45,8 b' Source code'
 -----------
 
 The latest sources can be obtained from official RhodeCode instance
-https://code.rhodecode.com
+https://code.rhodecode.com/rhodecode-enterprise-ce
+https://code.rhodecode.com/rhodecode-vcsserver
 
 
 Contributions

configs/development.ini

@@ -1,24 +1,22 b''
-
+## -*- coding: utf-8 -*-
 
-################################################################################
-##                RHODECODE COMMUNITY  EDITION CONFIGURATION                  ##
-################################################################################
+; #########################################
+; RHODECODE COMMUNITY EDITION CONFIGURATION
+; #########################################
 
 [DEFAULT]
-## Debug flag sets all loggers to debug, and enables request tracking
+; Debug flag sets all loggers to debug, and enables request tracking
 debug = true
 
-################################################################################
-##                            EMAIL CONFIGURATION                             ##
-## Uncomment and replace with the email address which should receive          ##
-## any error reports after an application crash                               ##
-## Additionally these settings will be used by the RhodeCode mailing system   ##
-################################################################################
+; ########################################################################
+; EMAIL CONFIGURATION
+; These settings will be used by the RhodeCode mailing system
+; ########################################################################
 
-## prefix all emails subjects with given prefix, helps filtering out emails
+; prefix all emails subjects with given prefix, helps filtering out emails
 #email_prefix = [RhodeCode]
 
-## email FROM address all mails will be sent
+; email FROM address all mails will be sent
 #app_email_from = rhodecode-noreply@localhost
 
 #smtp_server = mail.server.com
@@ -29,82 +27,139 b' debug = true'
 #smtp_use_ssl = true
 
 [server:main]
-## COMMON ##
+; COMMON HOST/IP CONFIG
 host = 127.0.0.1
 port = 5000
 
-###########################################################
-## WAITRESS WSGI SERVER - Recommended for Development  ####
-###########################################################
+; ##################################################
+; WAITRESS WSGI SERVER - Recommended for Development
+; ##################################################
 
+; use server type
 use = egg:waitress#main
-## number of worker threads
+
+; number of worker threads
 threads = 5
-## MAX BODY SIZE 100GB
+
+; MAX BODY SIZE 100GB
 max_request_body_size = 107374182400
-## Use poll instead of select, fixes file descriptors limits problems.
-## May not work on old windows systems.
+
+; Use poll instead of select, fixes file descriptors limits problems.
+; May not work on old windows systems.
 asyncore_use_poll = true
 
 
-##########################
-## GUNICORN WSGI SERVER ##
-##########################
-## run with gunicorn --log-config rhodecode.ini --paste rhodecode.ini
+; ###########################
+; GUNICORN APPLICATION SERVER
+; ###########################
 
+; run with gunicorn --log-config rhodecode.ini --paste rhodecode.ini
+
+; Module to use, this setting shouldn't be changed
 #use = egg:gunicorn#main
-## Sets the number of process workers. More workers means more concurrent connections
-## RhodeCode can handle at the same time. Each additional worker also it increases
-## memory usage as each has it's own set of caches.
-## Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
-## than 8-10 unless for really big deployments .e.g 700-1000 users.
-## `instance_id = *` must be set in the [app:main] section below (which is the default)
-## when using more than 1 worker.
+
+; Sets the number of process workers. More workers means more concurrent connections
+; RhodeCode can handle at the same time. Each additional worker also it increases
+; memory usage as each has it's own set of caches.
+; Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
+; than 8-10 unless for really big deployments .e.g 700-1000 users.
+; `instance_id = *` must be set in the [app:main] section below (which is the default)
+; when using more than 1 worker.
 #workers = 2
-## process name visible in process list
+
+; Gunicorn access log level
+#loglevel = info
+
+; Process name visible in process list
 #proc_name = rhodecode
-## type of worker class, one of sync, gevent
-## recommended for bigger setup is using of of other than sync one
+
+; Type of worker class, one of `sync`, `gevent`
+; Recommended type is `gevent`
 #worker_class = gevent
-## The maximum number of simultaneous clients. Valid only for Gevent
+
+; The maximum number of simultaneous clients. Valid only for gevent
 #worker_connections = 10
-## max number of requests that worker will handle before being gracefully
-## restarted, could prevent memory leaks
+
+; Max number of requests that worker will handle before being gracefully restarted.
+; Prevents memory leaks, jitter adds variability so not all workers are restarted at once.
 #max_requests = 1000
 #max_requests_jitter = 30
-## amount of time a worker can spend with handling a request before it
-## gets killed and restarted. Set to 6hrs
+
+; Amount of time a worker can spend with handling a request before it
+; gets killed and restarted. By default set to 21600 (6hrs)
+; Examples: 1800 (30min), 3600 (1hr), 7200 (2hr), 43200 (12h)
 #timeout = 21600
 
+; The maximum size of HTTP request line in bytes.
+; 0 for unlimited
+#limit_request_line = 0
+
+; Limit the number of HTTP headers fields in a request.
+; By default this value is 100 and can't be larger than 32768.
+#limit_request_fields = 32768
+
+; Limit the allowed size of an HTTP request header field.
+; Value is a positive number or 0.
+; Setting it to 0 will allow unlimited header field sizes.
+#limit_request_field_size = 0
+
+; Timeout for graceful workers restart.
+; After receiving a restart signal, workers have this much time to finish
+; serving requests. Workers still alive after the timeout (starting from the
+; receipt of the restart signal) are force killed.
+; Examples: 1800 (30min), 3600 (1hr), 7200 (2hr), 43200 (12h)
+#graceful_timeout = 3600
+
+# The number of seconds to wait for requests on a Keep-Alive connection.
+# Generally set in the 1-5 seconds range.
+#keepalive = 2
+
+; Maximum memory usage that each worker can use before it will receive a
+; graceful restart signal 0 = memory monitoring is disabled
+; Examples: 268435456 (256MB), 536870912 (512MB)
+; 1073741824 (1GB), 2147483648 (2GB), 4294967296 (4GB)
+#memory_max_usage = 0
+
+; How often in seconds to check for memory usage for each gunicorn worker
+#memory_usage_check_interval = 60
+
+; Threshold value for which we don't recycle worker if GarbageCollection
+; frees up enough resources. Before each restart we try to run GC on worker
+; in case we get enough free memory after that, restart will not happen.
+#memory_usage_recovery_threshold = 0.8
+
 
-## prefix middleware for RhodeCode.
-## recommended when using proxy setup.
-## allows to set RhodeCode under a prefix in server.
-## eg https://server.com/custom_prefix. Enable `filter-with =` option below as well.
-## And set your prefix like: `prefix = /custom_prefix`
-## be sure to also set beaker.session.cookie_path = /custom_prefix if you need
-## to make your cookies only work on prefix url
+; Prefix middleware for RhodeCode.
+; recommended when using proxy setup.
+; allows to set RhodeCode under a prefix in server.
+; eg https://server.com/custom_prefix. Enable `filter-with =` option below as well.
+; And set your prefix like: `prefix = /custom_prefix`
+; be sure to also set beaker.session.cookie_path = /custom_prefix if you need
+; to make your cookies only work on prefix url
 [filter:proxy-prefix]
 use = egg:PasteDeploy#prefix
 prefix = /
 
 [app:main]
-## The %(here)s variable will be replaced with the absolute path of parent directory
-## of this file
-## In addition ENVIRONMENT variables usage is possible, e.g
-## sqlalchemy.db1.url = {ENV_RC_DB_URL}
+; The %(here)s variable will be replaced with the absolute path of parent directory
+; of this file
+; In addition ENVIRONMENT variables usage is possible, e.g
+; sqlalchemy.db1.url = {ENV_RC_DB_URL}
 
 use = egg:rhodecode-enterprise-ce
 
-## enable proxy prefix middleware, defined above
+; enable proxy prefix middleware, defined above
 #filter-with = proxy-prefix
 
+; #############
+; DEBUG OPTIONS
+; #############
+
+pyramid.reload_templates = true
+
 # During development the we want to have the debug toolbar enabled
 pyramid.includes =
     pyramid_debugtoolbar
-    rhodecode.lib.middleware.request_wrapper
-
-pyramid.reload_templates = true
 
 debugtoolbar.hosts = 0.0.0.0/0
 debugtoolbar.exclude_prefixes =
@@ -121,101 +176,100 b' rhodecode.includes ='
 # api prefix url
 rhodecode.api.url = /_admin/api
 
-
-## END RHODECODE PLUGINS ##
+; enable debug style page
+debug_style = true
 
-## encryption key used to encrypt social plugin tokens,
-## remote_urls with credentials etc, if not set it defaults to
-## `beaker.session.secret`
+; #################
+; END DEBUG OPTIONS
+; #################
+
+; encryption key used to encrypt social plugin tokens,
+; remote_urls with credentials etc, if not set it defaults to
+; `beaker.session.secret`
 #rhodecode.encrypted_values.secret =
 
-## decryption strict mode (enabled by default). It controls if decryption raises
-## `SignatureVerificationError` in case of wrong key, or damaged encryption data.
+; decryption strict mode (enabled by default). It controls if decryption raises
+; `SignatureVerificationError` in case of wrong key, or damaged encryption data.
 #rhodecode.encrypted_values.strict = false
 
-## Pick algorithm for encryption. Either fernet (more secure) or aes (default)
-## fernet is safer, and we strongly recommend switching to it.
-## Due to backward compatibility aes is used as default.
+; Pick algorithm for encryption. Either fernet (more secure) or aes (default)
+; fernet is safer, and we strongly recommend switching to it.
+; Due to backward compatibility aes is used as default.
 #rhodecode.encrypted_values.algorithm = fernet
 
-## return gzipped responses from RhodeCode (static files/application)
+; Return gzipped responses from RhodeCode (static files/application)
 gzip_responses = false
 
-## auto-generate javascript routes file on startup
+; Auto-generate javascript routes file on startup
 generate_js_files = false
 
-## System global default language.
-## All available languages: en(default), be, de, es, fr, it, ja, pl, pt, ru, zh
+; System global default language.
+; All available languages: en (default), be, de, es, fr, it, ja, pl, pt, ru, zh
 lang = en
 
-## Perform a full repository scan and import on each server start.
-## Settings this to true could lead to very long startup time.
+; Perform a full repository scan and import on each server start.
+; Settings this to true could lead to very long startup time.
 startup.import_repos = false
 
-## Uncomment and set this path to use archive download cache.
-## Once enabled, generated archives will be cached at this location
-## and served from the cache during subsequent requests for the same archive of
-## the repository.
+; Uncomment and set this path to use archive download cache.
+; Once enabled, generated archives will be cached at this location
+; and served from the cache during subsequent requests for the same archive of
+; the repository.
 #archive_cache_dir = /tmp/tarballcache
 
-## URL at which the application is running. This is used for Bootstrapping
-## requests in context when no web request is available. Used in ishell, or
-## SSH calls. Set this for events to receive proper url for SSH calls.
+; URL at which the application is running. This is used for Bootstrapping
+; requests in context when no web request is available. Used in ishell, or
+; SSH calls. Set this for events to receive proper url for SSH calls.
 app.base_url = http://rhodecode.local
 
-## Unique application ID. Should be a random unique string for security.
+; Unique application ID. Should be a random unique string for security.
 app_instance_uuid = rc-production
 
-## Cut off limit for large diffs (size in bytes). If overall diff size on
-## commit, or pull request exceeds this limit this diff will be displayed
-## partially. E.g 512000 == 512Kb
+; Cut off limit for large diffs (size in bytes). If overall diff size on
+; commit, or pull request exceeds this limit this diff will be displayed
+; partially. E.g 512000 == 512Kb
 cut_off_limit_diff = 512000
 
-## Cut off limit for large files inside diffs (size in bytes). Each individual
-## file inside diff which exceeds this limit will be displayed partially.
-##  E.g 128000 == 128Kb
+; Cut off limit for large files inside diffs (size in bytes). Each individual
+; file inside diff which exceeds this limit will be displayed partially.
+;  E.g 128000 == 128Kb
 cut_off_limit_file = 128000
 
-## use cached version of vcs repositories everywhere. Recommended to be `true`
+; Use cached version of vcs repositories everywhere. Recommended to be `true`
 vcs_full_cache = true
 
-## Force https in RhodeCode, fixes https redirects, assumes it's always https.
-## Normally this is controlled by proper http flags sent from http server
+; Force https in RhodeCode, fixes https redirects, assumes it's always https.
+; Normally this is controlled by proper flags sent from http server such as Nginx or Apache
 force_https = false
 
-## use Strict-Transport-Security headers
+; use Strict-Transport-Security headers
 use_htsts = false
 
-## git rev filter option, --all is the default filter, if you need to
-## hide all refs in changelog switch this to --branches --tags
-git_rev_filter = --branches --tags
-
-# Set to true if your repos are exposed using the dumb protocol
+; Set to true if your repos are exposed using the dumb protocol
 git_update_server_info = false
 
-## RSS/ATOM feed options
+; RSS/ATOM feed options
 rss_cut_off_limit = 256000
 rss_items_per_page = 10
 rss_include_diff = false
 
-## gist URL alias, used to create nicer urls for gist. This should be an
-## url that does rewrites to _admin/gists/{gistid}.
-## example: http://gist.rhodecode.org/{gistid}. Empty means use the internal
-## RhodeCode url, ie. http[s]://rhodecode.server/_admin/gists/{gistid}
+; gist URL alias, used to create nicer urls for gist. This should be an
+; url that does rewrites to _admin/gists/{gistid}.
+; example: http://gist.rhodecode.org/{gistid}. Empty means use the internal
+; RhodeCode url, ie. http[s]://rhodecode.server/_admin/gists/{gistid}
 gist_alias_url =
 
-## List of views (using glob pattern syntax) that AUTH TOKENS could be
-## used for access.
-## Adding ?auth_token=TOKEN_HASH to the url authenticates this request as if it
-## came from the the logged in user who own this authentication token.
-## Additionally @TOKEN syntax can be used to bound the view to specific
-## authentication token. Such view would be only accessible when used together
-## with this authentication token
-##
-## list of all views can be found under `/_admin/permissions/auth_token_access`
-## The list should be "," separated and on a single line.
-##
-## Most common views to enable:
+; List of views (using glob pattern syntax) that AUTH TOKENS could be
+; used for access.
+; Adding ?auth_token=TOKEN_HASH to the url authenticates this request as if it
+; came from the the logged in user who own this authentication token.
+; Additionally @TOKEN syntax can be used to bound the view to specific
+; authentication token. Such view would be only accessible when used together
+; with this authentication token
+; list of all views can be found under `/_admin/permissions/auth_token_access`
+; The list should be "," separated and on a single line.
+; Most common views to enable:
+
 #    RepoCommitsView:repo_commit_download
 #    RepoCommitsView:repo_commit_patch
 #    RepoCommitsView:repo_commit_raw
@@ -226,164 +280,194 b' gist_alias_url ='
 #    GistView:*
 api_access_controllers_whitelist =
 
-## Default encoding used to convert from and to unicode
-## can be also a comma separated list of encoding in case of mixed encodings
+; Default encoding used to convert from and to unicode
+; can be also a comma separated list of encoding in case of mixed encodings
 default_encoding = UTF-8
 
-## instance-id prefix
-## a prefix key for this instance used for cache invalidation when running
-## multiple instances of RhodeCode, make sure it's globally unique for
-## all running RhodeCode instances. Leave empty if you don't use it
+; instance-id prefix
+; a prefix key for this instance used for cache invalidation when running
+; multiple instances of RhodeCode, make sure it's globally unique for
+; all running RhodeCode instances. Leave empty if you don't use it
 instance_id =
 
-## Fallback authentication plugin. Set this to a plugin ID to force the usage
-## of an authentication plugin also if it is disabled by it's settings.
-## This could be useful if you are unable to log in to the system due to broken
-## authentication settings. Then you can enable e.g. the internal RhodeCode auth
-## module to log in again and fix the settings.
-##
-## Available builtin plugin IDs (hash is part of the ID):
-## egg:rhodecode-enterprise-ce#rhodecode
-## egg:rhodecode-enterprise-ce#pam
-## egg:rhodecode-enterprise-ce#ldap
-## egg:rhodecode-enterprise-ce#jasig_cas
-## egg:rhodecode-enterprise-ce#headers
-## egg:rhodecode-enterprise-ce#crowd
+; Fallback authentication plugin. Set this to a plugin ID to force the usage
+; of an authentication plugin also if it is disabled by it's settings.
+; This could be useful if you are unable to log in to the system due to broken
+; authentication settings. Then you can enable e.g. the internal RhodeCode auth
+; module to log in again and fix the settings.
+; Available builtin plugin IDs (hash is part of the ID):
+; egg:rhodecode-enterprise-ce#rhodecode
+; egg:rhodecode-enterprise-ce#pam
+; egg:rhodecode-enterprise-ce#ldap
+; egg:rhodecode-enterprise-ce#jasig_cas
+; egg:rhodecode-enterprise-ce#headers
+; egg:rhodecode-enterprise-ce#crowd
+
 #rhodecode.auth_plugin_fallback = egg:rhodecode-enterprise-ce#rhodecode
 
-## alternative return HTTP header for failed authentication. Default HTTP
-## response is 401 HTTPUnauthorized. Currently HG clients have troubles with
-## handling that causing a series of failed authentication calls.
-## Set this variable to 403 to return HTTPForbidden, or any other HTTP code
-## This will be served instead of default 401 on bad authentication
+; Flag to control loading of legacy plugins in py:/path format
+auth_plugin.import_legacy_plugins = true
+
+; alternative return HTTP header for failed authentication. Default HTTP
+; response is 401 HTTPUnauthorized. Currently HG clients have troubles with
+; handling that causing a series of failed authentication calls.
+; Set this variable to 403 to return HTTPForbidden, or any other HTTP code
+; This will be served instead of default 401 on bad authentication
 auth_ret_code =
 
-## use special detection method when serving auth_ret_code, instead of serving
-## ret_code directly, use 401 initially (Which triggers credentials prompt)
-## and then serve auth_ret_code to clients
+; use special detection method when serving auth_ret_code, instead of serving
+; ret_code directly, use 401 initially (Which triggers credentials prompt)
+; and then serve auth_ret_code to clients
 auth_ret_code_detection = false
 
-## locking return code. When repository is locked return this HTTP code. 2XX
-## codes don't break the transactions while 4XX codes do
+; locking return code. When repository is locked return this HTTP code. 2XX
+; codes don't break the transactions while 4XX codes do
 lock_ret_code = 423
 
-## allows to change the repository location in settings page
+; allows to change the repository location in settings page
 allow_repo_location_change = true
 
-## allows to setup custom hooks in settings page
+; allows to setup custom hooks in settings page
 allow_custom_hooks_settings = true
 
-## Generated license token required for EE edition license.
-## New generated token value can be found in Admin > settings > license page.
+; Generated license token required for EE edition license.
+; New generated token value can be found in Admin > settings > license page.
 license_token =
 
-## supervisor connection uri, for managing supervisor and logs.
+; This flag hides sensitive information on the license page such as token, and license data
+license.hide_license_info = false
+
+; supervisor connection uri, for managing supervisor and logs.
 supervisor.uri =
-## supervisord group name/id we only want this RC instance to handle
+
+; supervisord group name/id we only want this RC instance to handle
 supervisor.group_id = dev
 
-## Display extended labs settings
+; Display extended labs settings
 labs_settings_active = true
 
-## Custom exception store path, defaults to TMPDIR
-## This is used to store exception from RhodeCode in shared directory
+; Custom exception store path, defaults to TMPDIR
+; This is used to store exception from RhodeCode in shared directory
 #exception_tracker.store_path =
 
-## File store configuration. This is used to store and serve uploaded files
+; File store configuration. This is used to store and serve uploaded files
 file_store.enabled = true
-## Storage backend, available options are: local
+
+; Storage backend, available options are: local
 file_store.backend = local
-## path to store the uploaded binaries
+
+; path to store the uploaded binaries
 file_store.storage_path = %(here)s/data/file_store
 
 
-####################################
-###        CELERY CONFIG        ####
-####################################
-## run: /path/to/celery worker \
-##     -E --beat --app rhodecode.lib.celerylib.loader \
-##     --scheduler rhodecode.lib.celerylib.scheduler.RcScheduler \
-##     --loglevel DEBUG --ini /path/to/rhodecode.ini
+; #############
+; CELERY CONFIG
+; #############
+
+; manually run celery: /path/to/celery worker -E --beat --app rhodecode.lib.celerylib.loader --scheduler rhodecode.lib.celerylib.scheduler.RcScheduler --loglevel DEBUG --ini /path/to/rhodecode.ini
 
 use_celery = false
 
-## connection url to the message broker (default rabbitmq)
-celery.broker_url = amqp://rabbitmq:qweqwe@localhost:5672/rabbitmqhost
+; connection url to the message broker (default redis)
+celery.broker_url = redis://localhost:6379/8
 
-## maximum tasks to execute before worker restart
+; rabbitmq example
+#celery.broker_url = amqp://rabbitmq:qweqwe@localhost:5672/rabbitmqhost
+
+; maximum tasks to execute before worker restart
 celery.max_tasks_per_child = 100
 
-## tasks will never be sent to the queue, but executed locally instead.
+; tasks will never be sent to the queue, but executed locally instead.
 celery.task_always_eager = false
 
-#####################################
-###         DOGPILE CACHE        ####
-#####################################
-## Default cache dir for caches.  Putting this into a ramdisk
-## can boost performance, eg. /tmpfs/data_ramdisk, however this directory might require
-## large amount of space
+; #############
+; DOGPILE CACHE
+; #############
+
+; Default cache dir for caches. Putting this into a ramdisk can boost performance.
+; eg. /tmpfs/data_ramdisk, however this directory might require large amount of space
 cache_dir = %(here)s/data
 
-## `cache_perms` cache settings for permission tree, auth TTL.
+; *********************************************
+; `sql_cache_short` cache for heavy SQL queries
+; Only supported backend is `memory_lru`
+; *********************************************
+rc_cache.sql_cache_short.backend = dogpile.cache.rc.memory_lru
+rc_cache.sql_cache_short.expiration_time = 30
+
+
+; *****************************************************
+; `cache_repo_longterm` cache for repo object instances
+; Only supported backend is `memory_lru`
+; *****************************************************
+rc_cache.cache_repo_longterm.backend = dogpile.cache.rc.memory_lru
+; by default we use 30 Days, cache is still invalidated on push
+rc_cache.cache_repo_longterm.expiration_time = 2592000
+; max items in LRU cache, set to smaller number to save memory, and expire last used caches
+rc_cache.cache_repo_longterm.max_size = 10000
+
+
+; *************************************************
+; `cache_perms` cache for permission tree, auth TTL
+; *************************************************
 rc_cache.cache_perms.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_perms.expiration_time = 300
+; file cache store path. Defaults to `cache_dir =` value or tempdir if both values are not set
+#rc_cache.cache_perms.arguments.filename = /tmp/cache_perms.db
 
-## alternative `cache_perms` redis backend with distributed lock
+; alternative `cache_perms` redis backend with distributed lock
 #rc_cache.cache_perms.backend = dogpile.cache.rc.redis
 #rc_cache.cache_perms.expiration_time = 300
-## redis_expiration_time needs to be greater then expiration_time
+
+; redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_perms.arguments.redis_expiration_time = 7200
-#rc_cache.cache_perms.arguments.socket_timeout = 30
+
 #rc_cache.cache_perms.arguments.host = localhost
 #rc_cache.cache_perms.arguments.port = 6379
 #rc_cache.cache_perms.arguments.db = 0
-## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
+#rc_cache.cache_perms.arguments.socket_timeout = 30
+; more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_perms.arguments.distributed_lock = true
 
-## `cache_repo` cache settings for FileTree, Readme, RSS FEEDS
+
+; ***************************************************
+; `cache_repo` cache for file tree, Readme, RSS FEEDS
+; ***************************************************
 rc_cache.cache_repo.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_repo.expiration_time = 2592000
+; file cache store path. Defaults to `cache_dir =` value or tempdir if both values are not set
+#rc_cache.cache_repo.arguments.filename = /tmp/cache_repo.db
 
-## alternative `cache_repo` redis backend with distributed lock
+; alternative `cache_repo` redis backend with distributed lock
 #rc_cache.cache_repo.backend = dogpile.cache.rc.redis
 #rc_cache.cache_repo.expiration_time = 2592000
-## redis_expiration_time needs to be greater then expiration_time
+
+; redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_repo.arguments.redis_expiration_time = 2678400
-#rc_cache.cache_repo.arguments.socket_timeout = 30
+
 #rc_cache.cache_repo.arguments.host = localhost
 #rc_cache.cache_repo.arguments.port = 6379
 #rc_cache.cache_repo.arguments.db = 1
-## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
+#rc_cache.cache_repo.arguments.socket_timeout = 30
+; more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_repo.arguments.distributed_lock = true
 
-## cache settings for SQL queries, this needs to use memory type backend
-rc_cache.sql_cache_short.backend = dogpile.cache.rc.memory_lru
-rc_cache.sql_cache_short.expiration_time = 30
 
-## `cache_repo_longterm` cache for repo object instances, this needs to use memory
-## type backend as the objects kept are not pickle serializable
-rc_cache.cache_repo_longterm.backend = dogpile.cache.rc.memory_lru
-## by default we use 96H, this is using invalidation on push anyway
-rc_cache.cache_repo_longterm.expiration_time = 345600
-## max items in LRU cache, reduce this number to save memory, and expire last used
-## cached objects
-rc_cache.cache_repo_longterm.max_size = 10000
+; ##############
+; BEAKER SESSION
+; ##############
 
-
-####################################
-###       BEAKER SESSION        ####
-####################################
-
-## .session.type is type of storage options for the session, current allowed
-## types are file, ext:memcached, ext:redis, ext:database, and memory (default).
+; beaker.session.type is type of storage options for the logged users sessions. Current allowed
+; types are file, ext:redis, ext:database, ext:memcached, and memory (default if not specified).
+; Fastest ones are Redis and ext:database
 beaker.session.type = file
 beaker.session.data_dir = %(here)s/data/sessions
 
-## redis sessions
+; Redis based sessions
 #beaker.session.type = ext:redis
 #beaker.session.url = redis://127.0.0.1:6379/2
 
-## db based session, fast, and allows easy management over logged in users
+; DB based session, fast, and allows easy management over logged in users
 #beaker.session.type = ext:database
 #beaker.session.table_name = db_session
 #beaker.session.sa.url = postgresql://postgres:secret@localhost/rhodecode
@@ -395,267 +479,275 b' beaker.session.key = rhodecode'
 beaker.session.secret = develop-rc-uytcxaz
 beaker.session.lock_dir = %(here)s/data/sessions/lock
 
-## Secure encrypted cookie. Requires AES and AES python libraries
-## you must disable beaker.session.secret to use this
+; Secure encrypted cookie. Requires AES and AES python libraries
+; you must disable beaker.session.secret to use this
 #beaker.session.encrypt_key = key_for_encryption
 #beaker.session.validate_key = validation_key
 
-## sets session as invalid(also logging out user) if it haven not been
-## accessed for given amount of time in seconds
+; Sets session as invalid (also logging out user) if it haven not been
+; accessed for given amount of time in seconds
 beaker.session.timeout = 2592000
 beaker.session.httponly = true
-## Path to use for the cookie. Set to prefix if you use prefix middleware
+
+; Path to use for the cookie. Set to prefix if you use prefix middleware
 #beaker.session.cookie_path = /custom_prefix
 
-## uncomment for https secure cookie
+; Set https secure cookie
 beaker.session.secure = false
 
-## auto save the session to not to use .save()
-beaker.session.auto = false
-
-## default cookie expiration time in seconds, set to `true` to set expire
-## at browser close
+; default cookie expiration time in seconds, set to `true` to set expire
+; at browser close
 #beaker.session.cookie_expires = 3600
 
-###################################
-## SEARCH INDEXING CONFIGURATION ##
-###################################
-## Full text search indexer is available in rhodecode-tools under
-## `rhodecode-tools index` command
+; #############################
+; SEARCH INDEXING CONFIGURATION
+; #############################
 
-## WHOOSH Backend, doesn't require additional services to run
-## it works good with few dozen repos
+; Full text search indexer is available in rhodecode-tools under
+; `rhodecode-tools index` command
+
+; WHOOSH Backend, doesn't require additional services to run
+; it works good with few dozen repos
 search.module = rhodecode.lib.index.whoosh
 search.location = %(here)s/data/index
 
-########################################
-###    CHANNELSTREAM CONFIG         ####
-########################################
-## channelstream enables persistent connections and live notification
-## in the system. It's also used by the chat system
+; ####################
+; CHANNELSTREAM CONFIG
+; ####################
+
+; channelstream enables persistent connections and live notification
+; in the system. It's also used by the chat system
 
 channelstream.enabled = false
 
-## server address for channelstream server on the backend
+; server address for channelstream server on the backend
 channelstream.server = 127.0.0.1:9800
 
-## location of the channelstream server from outside world
-## use ws:// for http or wss:// for https. This address needs to be handled
-## by external HTTP server such as Nginx or Apache
-## see Nginx/Apache configuration examples in our docs
+; location of the channelstream server from outside world
+; use ws:// for http or wss:// for https. This address needs to be handled
+; by external HTTP server such as Nginx or Apache
+; see Nginx/Apache configuration examples in our docs
 channelstream.ws_url = ws://rhodecode.yourserver.com/_channelstream
 channelstream.secret = secret
 channelstream.history.location = %(here)s/channelstream_history
 
-## Internal application path that Javascript uses to connect into.
-## If you use proxy-prefix the prefix should be added before /_channelstream
+; Internal application path that Javascript uses to connect into.
+; If you use proxy-prefix the prefix should be added before /_channelstream
 channelstream.proxy_path = /_channelstream
 
 
-###################################
-##       APPENLIGHT CONFIG       ##
-###################################
+; ##############################
+; MAIN RHODECODE DATABASE CONFIG
+; ##############################
+
+#sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
+#sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
+#sqlalchemy.db1.url = mysql://root:qweqwe@localhost/rhodecode?charset=utf8
+; pymysql is an alternative driver for MySQL, use in case of problems with default one
+#sqlalchemy.db1.url = mysql+pymysql://root:qweqwe@localhost/rhodecode
+
+sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
+
+; see sqlalchemy docs for other advanced settings
+; print the sql statements to output
+sqlalchemy.db1.echo = false
+
+; recycle the connections after this amount of seconds
+sqlalchemy.db1.pool_recycle = 3600
+sqlalchemy.db1.convert_unicode = true
+
+; the number of connections to keep open inside the connection pool.
+; 0 indicates no limit
+#sqlalchemy.db1.pool_size = 5
+
+; The number of connections to allow in connection pool "overflow", that is
+; connections that can be opened above and beyond the pool_size setting,
+; which defaults to five.
+#sqlalchemy.db1.max_overflow = 10
+
+; Connection check ping, used to detect broken database connections
+; could be enabled to better handle cases if MySQL has gone away errors
+#sqlalchemy.db1.ping_connection = true
+
+; ##########
+; VCS CONFIG
+; ##########
+vcs.server.enable = true
+vcs.server = localhost:9900
+
+; Web server connectivity protocol, responsible for web based VCS operations
+; Available protocols are:
+; `http` - use http-rpc backend (default)
+vcs.server.protocol = http
+
+; Push/Pull operations protocol, available options are:
+; `http` - use http-rpc backend (default)
+vcs.scm_app_implementation = http
+
+; Push/Pull operations hooks protocol, available options are:
+; `http` - use http-rpc backend (default)
+vcs.hooks.protocol = http
+
+; Host on which this instance is listening for hooks. If vcsserver is in other location
+; this should be adjusted.
+vcs.hooks.host = 127.0.0.1
+
+; Start VCSServer with this instance as a subprocess, useful for development
+vcs.start_server = false
+
+; List of enabled VCS backends, available options are:
+; `hg`  - mercurial
+; `git` - git
+; `svn` - subversion
+vcs.backends = hg, git, svn
+
+; Wait this number of seconds before killing connection to the vcsserver
+vcs.connection_timeout = 3600
+
+; Compatibility version when creating SVN repositories. Defaults to newest version when commented out.
+; Available options are: pre-1.4-compatible, pre-1.5-compatible, pre-1.6-compatible, pre-1.8-compatible, pre-1.9-compatible
+#vcs.svn.compatible_version = pre-1.8-compatible
+
 
-## Appenlight is tailored to work with RhodeCode, see
-## http://appenlight.com for details how to obtain an account
+; ####################################################
+; Subversion proxy support (mod_dav_svn)
+; Maps RhodeCode repo groups into SVN paths for Apache
+; ####################################################
+
+; Enable or disable the config file generation.
+svn.proxy.generate_config = false
+
+; Generate config file with `SVNListParentPath` set to `On`.
+svn.proxy.list_parent_path = true
+
+; Set location and file name of generated config file.
+svn.proxy.config_file_path = %(here)s/mod_dav_svn.conf
+
+; alternative mod_dav config template. This needs to be a valid mako template
+; Example template can be found in the source code:
+; rhodecode/apps/svn_support/templates/mod-dav-svn.conf.mako
+#svn.proxy.config_template = ~/.rccontrol/enterprise-1/custom_svn_conf.mako
+
+; Used as a prefix to the `Location` block in the generated config file.
+; In most cases it should be set to `/`.
+svn.proxy.location_root = /
+
+; Command to reload the mod dav svn configuration on change.
+; Example: `/etc/init.d/apache2 reload` or /home/USER/apache_reload.sh
+; Make sure user who runs RhodeCode process is allowed to reload Apache
+#svn.proxy.reload_cmd = /etc/init.d/apache2 reload
+
+; If the timeout expires before the reload command finishes, the command will
+; be killed. Setting it to zero means no timeout. Defaults to 10 seconds.
+#svn.proxy.reload_timeout = 10
+
+; ####################
+; SSH Support Settings
+; ####################
 
-## Appenlight integration enabled
+; Defines if a custom authorized_keys file should be created and written on
+; any change user ssh keys. Setting this to false also disables possibility
+; of adding SSH keys by users from web interface. Super admins can still
+; manage SSH Keys.
+ssh.generate_authorized_keyfile = false
+
+; Options for ssh, default is `no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding`
+# ssh.authorized_keys_ssh_opts =
+
+; Path to the authorized_keys file where the generate entries are placed.
+; It is possible to have multiple key files specified in `sshd_config` e.g.
+; AuthorizedKeysFile %h/.ssh/authorized_keys %h/.ssh/authorized_keys_rhodecode
+ssh.authorized_keys_file_path = ~/.ssh/authorized_keys_rhodecode
+
+; Command to execute the SSH wrapper. The binary is available in the
+; RhodeCode installation directory.
+; e.g ~/.rccontrol/community-1/profile/bin/rc-ssh-wrapper
+ssh.wrapper_cmd = ~/.rccontrol/community-1/rc-ssh-wrapper
+
+; Allow shell when executing the ssh-wrapper command
+ssh.wrapper_cmd_allow_shell = false
+
+; Enables logging, and detailed output send back to the client during SSH
+; operations. Useful for debugging, shouldn't be used in production.
+ssh.enable_debug_logging = true
+
+; Paths to binary executable, by default they are the names, but we can
+; override them if we want to use a custom one
+ssh.executable.hg = ~/.rccontrol/vcsserver-1/profile/bin/hg
+ssh.executable.git = ~/.rccontrol/vcsserver-1/profile/bin/git
+ssh.executable.svn = ~/.rccontrol/vcsserver-1/profile/bin/svnserve
+
+; Enables SSH key generator web interface. Disabling this still allows users
+; to add their own keys.
+ssh.enable_ui_key_generator = true
+
+
+; #################
+; APPENLIGHT CONFIG
+; #################
+
+; Appenlight is tailored to work with RhodeCode, see
+; http://appenlight.rhodecode.com for details how to obtain an account
+
+; Appenlight integration enabled
 appenlight = false
 
 appenlight.server_url = https://api.appenlight.com
 appenlight.api_key = YOUR_API_KEY
 #appenlight.transport_config = https://api.appenlight.com?threaded=1&timeout=5
 
-## used for JS client
+; used for JS client
 appenlight.api_public_key = YOUR_API_PUBLIC_KEY
 
-## TWEAK AMOUNT OF INFO SENT HERE
+; TWEAK AMOUNT OF INFO SENT HERE
 
-## enables 404 error logging (default False)
+; enables 404 error logging (default False)
 appenlight.report_404 = false
 
-## time in seconds after request is considered being slow (default 1)
+; time in seconds after request is considered being slow (default 1)
 appenlight.slow_request_time = 1
 
-## record slow requests in application
-## (needs to be enabled for slow datastore recording and time tracking)
+; record slow requests in application
+; (needs to be enabled for slow datastore recording and time tracking)
 appenlight.slow_requests = true
 
-## enable hooking to application loggers
+; enable hooking to application loggers
 appenlight.logging = true
 
-## minimum log level for log capture
+; minimum log level for log capture
 appenlight.logging.level = WARNING
 
-## send logs only from erroneous/slow requests
-## (saves API quota for intensive logging)
+; send logs only from erroneous/slow requests
+; (saves API quota for intensive logging)
 appenlight.logging_on_error = false
 
-## list of additional keywords that should be grabbed from environ object
-## can be string with comma separated list of words in lowercase
-## (by default client will always send following info:
-## 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
-## start with HTTP* this list be extended with additional keywords here
+; list of additional keywords that should be grabbed from environ object
+; can be string with comma separated list of words in lowercase
+; (by default client will always send following info:
+; 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
+; start with HTTP* this list be extended with additional keywords here
 appenlight.environ_keys_whitelist =
 
-## list of keywords that should be blanked from request object
-## can be string with comma separated list of words in lowercase
-## (by default client will always blank keys that contain following words
-## 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
-## this list be extended with additional keywords set here
+; list of keywords that should be blanked from request object
+; can be string with comma separated list of words in lowercase
+; (by default client will always blank keys that contain following words
+; 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
+; this list be extended with additional keywords set here
 appenlight.request_keys_blacklist =
 
-## list of namespaces that should be ignores when gathering log entries
-## can be string with comma separated list of namespaces
-## (by default the client ignores own entries: appenlight_client.client)
+; list of namespaces that should be ignores when gathering log entries
+; can be string with comma separated list of namespaces
+; (by default the client ignores own entries: appenlight_client.client)
 appenlight.log_namespace_blacklist =
 
-# enable debug style page
-debug_style = true
-
-###########################################
-###   MAIN RHODECODE DATABASE CONFIG    ###
-###########################################
-#sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
-#sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
-#sqlalchemy.db1.url = mysql://root:qweqwe@localhost/rhodecode?charset=utf8
-# pymysql is an alternative driver for MySQL, use in case of problems with default one
-#sqlalchemy.db1.url = mysql+pymysql://root:qweqwe@localhost/rhodecode
-
-sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
-
-# see sqlalchemy docs for other advanced settings
-
-## print the sql statements to output
-sqlalchemy.db1.echo = false
-## recycle the connections after this amount of seconds
-sqlalchemy.db1.pool_recycle = 3600
-sqlalchemy.db1.convert_unicode = true
-
-## the number of connections to keep open inside the connection pool.
-## 0 indicates no limit
-#sqlalchemy.db1.pool_size = 5
-
-## the number of connections to allow in connection pool "overflow", that is
-## connections that can be opened above and beyond the pool_size setting,
-## which defaults to five.
-#sqlalchemy.db1.max_overflow = 10
-
-## Connection check ping, used to detect broken database connections
-## could be enabled to better handle cases if MySQL has gone away errors
-#sqlalchemy.db1.ping_connection = true
-
-##################
-### VCS CONFIG ###
-##################
-vcs.server.enable = true
-vcs.server = localhost:9900
-
-## Web server connectivity protocol, responsible for web based VCS operations
-## Available protocols are:
-## `http` - use http-rpc backend (default)
-vcs.server.protocol = http
-
-## Push/Pull operations protocol, available options are:
-## `http` - use http-rpc backend (default)
-vcs.scm_app_implementation = http
-
-## Push/Pull operations hooks protocol, available options are:
-## `http` - use http-rpc backend (default)
-vcs.hooks.protocol = http
-
-## Host on which this instance is listening for hooks. If vcsserver is in other location
-## this should be adjusted.
-vcs.hooks.host = 127.0.0.1
-
-vcs.server.log_level = debug
-## Start VCSServer with this instance as a subprocess, useful for development
-vcs.start_server = false
-
-## List of enabled VCS backends, available options are:
-## `hg`  - mercurial
-## `git` - git
-## `svn` - subversion
-vcs.backends = hg, git, svn
-
-vcs.connection_timeout = 3600
-## Compatibility version when creating SVN repositories. Defaults to newest version when commented out.
-## Available options are: pre-1.4-compatible, pre-1.5-compatible, pre-1.6-compatible, pre-1.8-compatible, pre-1.9-compatible
-#vcs.svn.compatible_version = pre-1.8-compatible
-
-
-############################################################
-### Subversion proxy support (mod_dav_svn)               ###
-### Maps RhodeCode repo groups into SVN paths for Apache ###
-############################################################
-## Enable or disable the config file generation.
-svn.proxy.generate_config = false
-## Generate config file with `SVNListParentPath` set to `On`.
-svn.proxy.list_parent_path = true
-## Set location and file name of generated config file.
-svn.proxy.config_file_path = %(here)s/mod_dav_svn.conf
-## alternative mod_dav config template. This needs to be a mako template
-#svn.proxy.config_template = ~/.rccontrol/enterprise-1/custom_svn_conf.mako
-## Used as a prefix to the `Location` block in the generated config file.
-## In most cases it should be set to `/`.
-svn.proxy.location_root = /
-## Command to reload the mod dav svn configuration on change.
-## Example: `/etc/init.d/apache2 reload` or /home/USER/apache_reload.sh
-## Make sure user who runs RhodeCode process is allowed to reload Apache
-#svn.proxy.reload_cmd = /etc/init.d/apache2 reload
-## If the timeout expires before the reload command finishes, the command will
-## be killed. Setting it to zero means no timeout. Defaults to 10 seconds.
-#svn.proxy.reload_timeout = 10
-
-############################################################
-### SSH Support Settings                                 ###
-############################################################
-
-## Defines if a custom authorized_keys file should be created and written on
-## any change user ssh keys. Setting this to false also disables possibility
-## of adding SSH keys by users from web interface. Super admins can still
-## manage SSH Keys.
-ssh.generate_authorized_keyfile = false
-
-## Options for ssh, default is `no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding`
-# ssh.authorized_keys_ssh_opts =
-
-## Path to the authorized_keys file where the generate entries are placed.
-## It is possible to have multiple key files specified in `sshd_config` e.g.
-## AuthorizedKeysFile %h/.ssh/authorized_keys %h/.ssh/authorized_keys_rhodecode
-ssh.authorized_keys_file_path = ~/.ssh/authorized_keys_rhodecode
-
-## Command to execute the SSH wrapper. The binary is available in the
-## RhodeCode installation directory.
-## e.g ~/.rccontrol/community-1/profile/bin/rc-ssh-wrapper
-ssh.wrapper_cmd = ~/.rccontrol/community-1/rc-ssh-wrapper
-
-## Allow shell when executing the ssh-wrapper command
-ssh.wrapper_cmd_allow_shell = false
-
-## Enables logging, and detailed output send back to the client during SSH
-## operations. Useful for debugging, shouldn't be used in production.
-ssh.enable_debug_logging = true
-
-## Paths to binary executable, by default they are the names, but we can
-## override them if we want to use a custom one
-ssh.executable.hg = ~/.rccontrol/vcsserver-1/profile/bin/hg
-ssh.executable.git = ~/.rccontrol/vcsserver-1/profile/bin/git
-ssh.executable.svn = ~/.rccontrol/vcsserver-1/profile/bin/svnserve
-
-## Enables SSH key generator web interface. Disabling this still allows users
-## to add their own keys.
-ssh.enable_ui_key_generator = true
-
-
-## Dummy marker to add new entries after.
-## Add any custom entries below. Please don't remove.
+; Dummy marker to add new entries after.
+; Add any custom entries below. Please don't remove this marker.
 custom.conf = 1
 
 
-################################
-### LOGGING CONFIGURATION   ####
-################################
+; #####################
+; LOGGING CONFIGURATION
+; #####################
 [loggers]
 keys = root, sqlalchemy, beaker, celery, rhodecode, ssh_wrapper
 
@@ -665,9 +757,9 b' keys = console, console_sql'
 [formatters]
 keys = generic, color_formatter, color_formatter_sql
 
-#############
-## LOGGERS ##
-#############
+; #######
+; LOGGERS
+; #######
 [logger_root]
 level = NOTSET
 handlers = console
@@ -702,9 +794,9 b' handlers ='
 qualname = celery
 
 
-##############
-## HANDLERS ##
-##############
+; ########
+; HANDLERS
+; ########
 
 [handler_console]
 class = StreamHandler
@@ -713,17 +805,17 b' level = DEBUG'
 formatter = color_formatter
 
 [handler_console_sql]
-# "level = DEBUG" logs SQL queries and results.
-# "level = INFO" logs SQL queries.
-# "level = WARN" logs neither.  (Recommended for production systems.)
+; "level = DEBUG" logs SQL queries and results.
+; "level = INFO" logs SQL queries.
+; "level = WARN" logs neither.  (Recommended for production systems.)
 class = StreamHandler
 args = (sys.stderr, )
 level = WARN
 formatter = color_formatter_sql
 
-################
-## FORMATTERS ##
-################
+; ##########
+; FORMATTERS
+; ##########
 
 [formatter_generic]
 class = rhodecode.lib.logging_formatter.ExceptionAwareFormatter

configs/gunicorn_config.py

@@ -1,58 +1,26 b''
 """
-gunicorn config extension and hooks. Sets additional configuration that is
-available post the .ini config.
-
-- workers = ${cpu_number}
-- threads = 1
-- proc_name = ${gunicorn_proc_name}
-- worker_class = sync
-- worker_connections = 10
-- max_requests = 1000
-- max_requests_jitter = 30
-- timeout = 21600
-
+Gunicorn config extension and hooks. This config file adds some extra settings and memory management.
+Gunicorn configuration should be managed by .ini files entries of RhodeCode or VCSServer
 """
 
-import multiprocessing
+import gc
+import os
 import sys
+import math
 import time
-import datetime
 import threading
 import traceback
+import random
 from gunicorn.glogging import Logger
 
 
+def get_workers():
+    import multiprocessing
+    return multiprocessing.cpu_count() * 2 + 1
+
 # GLOBAL
 errorlog = '-'
 accesslog = '-'
-loglevel = 'debug'
-
-# SECURITY
-
-# The maximum size of HTTP request line in bytes.
-# 0 for unlimited
-limit_request_line = 0
-
-# Limit the number of HTTP headers fields in a request.
-# By default this value is 100 and can't be larger than 32768.
-limit_request_fields = 10240
-
-# Limit the allowed size of an HTTP request header field.
-# Value is a positive number or 0.
-# Setting it to 0 will allow unlimited header field sizes.
-limit_request_field_size = 0
-
-
-# Timeout for graceful workers restart.
-# After receiving a restart signal, workers have this much time to finish
-# serving requests. Workers still alive after the timeout (starting from the
-# receipt of the restart signal) are force killed.
-graceful_timeout = 30
-
-
-# The number of seconds to wait for requests on a Keep-Alive connection.
-# Generally set in the 1-5 seconds range.
-keepalive = 2
 
 
 # SERVER MECHANICS
@@ -63,38 +31,178 b' tmp_upload_dir = None'
 
 # Custom log format
 access_log_format = (
-    '%(t)s [%(p)-8s] GNCRN %(h)-15s rqt:%(L)s %(s)s %(b)-6s "%(m)s:%(U)s %(q)s" usr:%(u)s "%(f)s" "%(a)s"')
+    '%(t)s %(p)s INFO  [GNCRN] %(h)-15s rqt:%(L)s %(s)s %(b)-6s "%(m)s:%(U)s %(q)s" usr:%(u)s "%(f)s" "%(a)s"')
 
 # self adjust workers based on CPU count
-# workers = multiprocessing.cpu_count() * 2 + 1
+# workers = get_workers()
+
+
+def _get_process_rss(pid=None):
+    try:
+        import psutil
+        if pid:
+            proc = psutil.Process(pid)
+        else:
+            proc = psutil.Process()
+        return proc.memory_info().rss
+    except Exception:
+        return None
 
 
-def post_fork(server, worker):
-    server.log.info("[<%-10s>] WORKER spawned", worker.pid)
+def _get_config(ini_path):
+
+    try:
+        import configparser
+    except ImportError:
+        import ConfigParser as configparser
+    try:
+        config = configparser.RawConfigParser()
+        config.read(ini_path)
+        return config
+    except Exception:
+        return None
+
+
+def _time_with_offset(memory_usage_check_interval):
+    return time.time() - random.randint(0, memory_usage_check_interval/2.0)
 
 
 def pre_fork(server, worker):
     pass
 
 
+def post_fork(server, worker):
+
+    # memory spec defaults
+    _memory_max_usage = 0
+    _memory_usage_check_interval = 60
+    _memory_usage_recovery_threshold = 0.8
+
+    ini_path = os.path.abspath(server.cfg.paste)
+    conf = _get_config(ini_path)
+
+    section = 'server:main'
+    if conf and conf.has_section(section):
+
+        if conf.has_option(section, 'memory_max_usage'):
+            _memory_max_usage = conf.getint(section, 'memory_max_usage')
+
+        if conf.has_option(section, 'memory_usage_check_interval'):
+            _memory_usage_check_interval = conf.getint(section, 'memory_usage_check_interval')
+
+        if conf.has_option(section, 'memory_usage_recovery_threshold'):
+            _memory_usage_recovery_threshold = conf.getfloat(section, 'memory_usage_recovery_threshold')
+
+    worker._memory_max_usage = _memory_max_usage
+    worker._memory_usage_check_interval = _memory_usage_check_interval
+    worker._memory_usage_recovery_threshold = _memory_usage_recovery_threshold
+
+    # register memory last check time, with some random offset so we don't recycle all
+    # at once
+    worker._last_memory_check_time = _time_with_offset(_memory_usage_check_interval)
+
+    if _memory_max_usage:
+        server.log.info("[%-10s] WORKER spawned with max memory set at %s", worker.pid,
+                        _format_data_size(_memory_max_usage))
+    else:
+        server.log.info("[%-10s] WORKER spawned", worker.pid)
+
+
 def pre_exec(server):
     server.log.info("Forked child, re-executing.")
 
 
 def on_starting(server):
-    server.log.info("Server is starting.")
+    server_lbl = '{} {}'.format(server.proc_name, server.address)
+    server.log.info("Server %s is starting.", server_lbl)
 
 
 def when_ready(server):
-    server.log.info("Server is ready. Spawning workers")
+    server.log.info("Server %s is ready. Spawning workers", server)
 
 
 def on_reload(server):
     pass
 
 
+def _format_data_size(size, unit="B", precision=1, binary=True):
+    """Format a number using SI units (kilo, mega, etc.).
+
+    ``size``: The number as a float or int.
+
+    ``unit``: The unit name in plural form. Examples: "bytes", "B".
+
+    ``precision``: How many digits to the right of the decimal point. Default
+    is 1.  0 suppresses the decimal point.
+
+    ``binary``: If false, use base-10 decimal prefixes (kilo = K = 1000).
+    If true, use base-2 binary prefixes (kibi = Ki = 1024).
+
+    ``full_name``: If false (default), use the prefix abbreviation ("k" or
+    "Ki").  If true, use the full prefix ("kilo" or "kibi"). If false,
+    use abbreviation ("k" or "Ki").
+
+    """
+
+    if not binary:
+        base = 1000
+        multiples = ('', 'k', 'M', 'G', 'T', 'P', 'E', 'Z', 'Y')
+    else:
+        base = 1024
+        multiples = ('', 'Ki', 'Mi', 'Gi', 'Ti', 'Pi', 'Ei', 'Zi', 'Yi')
+
+    sign = ""
+    if size > 0:
+        m = int(math.log(size, base))
+    elif size < 0:
+        sign = "-"
+        size = -size
+        m = int(math.log(size, base))
+    else:
+        m = 0
+    if m > 8:
+        m = 8
+
+    if m == 0:
+        precision = '%.0f'
+    else:
+        precision = '%%.%df' % precision
+
+    size = precision % (size / math.pow(base, m))
+
+    return '%s%s %s%s' % (sign, size.strip(), multiples[m], unit)
+
+
+def _check_memory_usage(worker):
+    memory_max_usage = worker._memory_max_usage
+    if not memory_max_usage:
+        return
+
+    memory_usage_check_interval = worker._memory_usage_check_interval
+    memory_usage_recovery_threshold = memory_max_usage * worker._memory_usage_recovery_threshold
+
+    elapsed = time.time() - worker._last_memory_check_time
+    if elapsed > memory_usage_check_interval:
+        mem_usage = _get_process_rss()
+        if mem_usage and mem_usage > memory_max_usage:
+            worker.log.info(
+                "memory usage %s > %s, forcing gc",
+                _format_data_size(mem_usage), _format_data_size(memory_max_usage))
+            # Try to clean it up by forcing a full collection.
+            gc.collect()
+            mem_usage = _get_process_rss()
+            if mem_usage > memory_usage_recovery_threshold:
+                # Didn't clean up enough, we'll have to terminate.
+                worker.log.warning(
+                    "memory usage %s > %s after gc, quitting",
+                    _format_data_size(mem_usage), _format_data_size(memory_max_usage))
+                # This will cause worker to auto-restart itself
+                worker.alive = False
+        worker._last_memory_check_time = time.time()
+
+
 def worker_int(worker):
-    worker.log.info("[<%-10s>] worker received INT or QUIT signal", worker.pid)
+    worker.log.info("[%-10s] worker received INT or QUIT signal", worker.pid)
 
     # get traceback info, on worker crash
     id2name = dict([(th.ident, th.name) for th in threading.enumerate()])
@@ -110,15 +218,15 b' def worker_int(worker):'
 
 
 def worker_abort(worker):
-    worker.log.info("[<%-10s>] worker received SIGABRT signal", worker.pid)
+    worker.log.info("[%-10s] worker received SIGABRT signal", worker.pid)
 
 
 def worker_exit(server, worker):
-    worker.log.info("[<%-10s>] worker exit", worker.pid)
+    worker.log.info("[%-10s] worker exit", worker.pid)
 
 
 def child_exit(server, worker):
-    worker.log.info("[<%-10s>] worker child exit", worker.pid)
+    worker.log.info("[%-10s] worker child exit", worker.pid)
 
 
 def pre_request(worker, req):
@@ -129,9 +237,12 b' def pre_request(worker, req):'
 
 def post_request(worker, req, environ, resp):
     total_time = time.time() - worker.start_time
+    # Gunicorn sometimes has problems with reading the status_code
+    status_code = getattr(resp, 'status_code', '')
     worker.log.debug(
-        "GNCRN POST WORKER [cnt:%s]: %s %s resp: %s, Load Time: %.3fs",
-        worker.nr, req.method, req.path, resp.status_code, total_time)
+        "GNCRN POST WORKER [cnt:%s]: %s %s resp: %s, Load Time: %.4fs",
+        worker.nr, req.method, req.path, status_code, total_time)
+    _check_memory_usage(worker)
 
 
 class RhodeCodeLogger(Logger):

configs/production.ini

@@ -1,24 +1,22 b''
-
+## -*- coding: utf-8 -*-
 
-################################################################################
-##                RHODECODE COMMUNITY  EDITION CONFIGURATION                  ##
-################################################################################
+; #########################################
+; RHODECODE COMMUNITY EDITION CONFIGURATION
+; #########################################
 
 [DEFAULT]
-## Debug flag sets all loggers to debug, and enables request tracking
+; Debug flag sets all loggers to debug, and enables request tracking
 debug = false
 
-################################################################################
-##                            EMAIL CONFIGURATION                             ##
-## Uncomment and replace with the email address which should receive          ##
-## any error reports after an application crash                               ##
-## Additionally these settings will be used by the RhodeCode mailing system   ##
-################################################################################
+; ########################################################################
+; EMAIL CONFIGURATION
+; These settings will be used by the RhodeCode mailing system
+; ########################################################################
 
-## prefix all emails subjects with given prefix, helps filtering out emails
+; prefix all emails subjects with given prefix, helps filtering out emails
 #email_prefix = [RhodeCode]
 
-## email FROM address all mails will be sent
+; email FROM address all mails will be sent
 #app_email_from = rhodecode-noreply@localhost
 
 #smtp_server = mail.server.com
@@ -29,168 +27,200 b' debug = false'
 #smtp_use_ssl = true
 
 [server:main]
-## COMMON ##
+; COMMON HOST/IP CONFIG
 host = 127.0.0.1
 port = 5000
 
-###########################################################
-## WAITRESS WSGI SERVER - Recommended for Development  ####
-###########################################################
+
+; ###########################
+; GUNICORN APPLICATION SERVER
+; ###########################
+
+; run with gunicorn --log-config rhodecode.ini --paste rhodecode.ini
+
+; Module to use, this setting shouldn't be changed
+use = egg:gunicorn#main
+
+; Sets the number of process workers. More workers means more concurrent connections
+; RhodeCode can handle at the same time. Each additional worker also it increases
+; memory usage as each has it's own set of caches.
+; Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
+; than 8-10 unless for really big deployments .e.g 700-1000 users.
+; `instance_id = *` must be set in the [app:main] section below (which is the default)
+; when using more than 1 worker.
+workers = 2
+
+; Gunicorn access log level
+loglevel = info
+
+; Process name visible in process list
+proc_name = rhodecode
+
+; Type of worker class, one of `sync`, `gevent`
+; Recommended type is `gevent`
+worker_class = gevent
+
+; The maximum number of simultaneous clients per worker. Valid only for gevent
+worker_connections = 10
+
+; Max number of requests that worker will handle before being gracefully restarted.
+; Prevents memory leaks, jitter adds variability so not all workers are restarted at once.
+max_requests = 1000
+max_requests_jitter = 30
 
-#use = egg:waitress#main
-## number of worker threads
-#threads = 5
-## MAX BODY SIZE 100GB
-#max_request_body_size = 107374182400
-## Use poll instead of select, fixes file descriptors limits problems.
-## May not work on old windows systems.
-#asyncore_use_poll = true
+; Amount of time a worker can spend with handling a request before it
+; gets killed and restarted. By default set to 21600 (6hrs)
+; Examples: 1800 (30min), 3600 (1hr), 7200 (2hr), 43200 (12h)
+timeout = 21600
+
+; The maximum size of HTTP request line in bytes.
+; 0 for unlimited
+limit_request_line = 0
+
+; Limit the number of HTTP headers fields in a request.
+; By default this value is 100 and can't be larger than 32768.
+limit_request_fields = 32768
+
+; Limit the allowed size of an HTTP request header field.
+; Value is a positive number or 0.
+; Setting it to 0 will allow unlimited header field sizes.
+limit_request_field_size = 0
+
+; Timeout for graceful workers restart.
+; After receiving a restart signal, workers have this much time to finish
+; serving requests. Workers still alive after the timeout (starting from the
+; receipt of the restart signal) are force killed.
+; Examples: 1800 (30min), 3600 (1hr), 7200 (2hr), 43200 (12h)
+graceful_timeout = 3600
+
+# The number of seconds to wait for requests on a Keep-Alive connection.
+# Generally set in the 1-5 seconds range.
+keepalive = 2
+
+; Maximum memory usage that each worker can use before it will receive a
+; graceful restart signal 0 = memory monitoring is disabled
+; Examples: 268435456 (256MB), 536870912 (512MB)
+; 1073741824 (1GB), 2147483648 (2GB), 4294967296 (4GB)
+memory_max_usage = 0
+
+; How often in seconds to check for memory usage for each gunicorn worker
+memory_usage_check_interval = 60
+
+; Threshold value for which we don't recycle worker if GarbageCollection
+; frees up enough resources. Before each restart we try to run GC on worker
+; in case we get enough free memory after that, restart will not happen.
+memory_usage_recovery_threshold = 0.8
 
 
-##########################
-## GUNICORN WSGI SERVER ##
-##########################
-## run with gunicorn --log-config rhodecode.ini --paste rhodecode.ini
-
-use = egg:gunicorn#main
-## Sets the number of process workers. More workers means more concurrent connections
-## RhodeCode can handle at the same time. Each additional worker also it increases
-## memory usage as each has it's own set of caches.
-## Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
-## than 8-10 unless for really big deployments .e.g 700-1000 users.
-## `instance_id = *` must be set in the [app:main] section below (which is the default)
-## when using more than 1 worker.
-workers = 2
-## process name visible in process list
-proc_name = rhodecode
-## type of worker class, one of sync, gevent
-## recommended for bigger setup is using of of other than sync one
-worker_class = gevent
-## The maximum number of simultaneous clients. Valid only for Gevent
-worker_connections = 10
-## max number of requests that worker will handle before being gracefully
-## restarted, could prevent memory leaks
-max_requests = 1000
-max_requests_jitter = 30
-## amount of time a worker can spend with handling a request before it
-## gets killed and restarted. Set to 6hrs
-timeout = 21600
-
-
-## prefix middleware for RhodeCode.
-## recommended when using proxy setup.
-## allows to set RhodeCode under a prefix in server.
-## eg https://server.com/custom_prefix. Enable `filter-with =` option below as well.
-## And set your prefix like: `prefix = /custom_prefix`
-## be sure to also set beaker.session.cookie_path = /custom_prefix if you need
-## to make your cookies only work on prefix url
+; Prefix middleware for RhodeCode.
+; recommended when using proxy setup.
+; allows to set RhodeCode under a prefix in server.
+; eg https://server.com/custom_prefix. Enable `filter-with =` option below as well.
+; And set your prefix like: `prefix = /custom_prefix`
+; be sure to also set beaker.session.cookie_path = /custom_prefix if you need
+; to make your cookies only work on prefix url
 [filter:proxy-prefix]
 use = egg:PasteDeploy#prefix
 prefix = /
 
 [app:main]
-## The %(here)s variable will be replaced with the absolute path of parent directory
-## of this file
-## In addition ENVIRONMENT variables usage is possible, e.g
-## sqlalchemy.db1.url = {ENV_RC_DB_URL}
+; The %(here)s variable will be replaced with the absolute path of parent directory
+; of this file
+; In addition ENVIRONMENT variables usage is possible, e.g
+; sqlalchemy.db1.url = {ENV_RC_DB_URL}
 
 use = egg:rhodecode-enterprise-ce
 
-## enable proxy prefix middleware, defined above
+; enable proxy prefix middleware, defined above
 #filter-with = proxy-prefix
 
-## encryption key used to encrypt social plugin tokens,
-## remote_urls with credentials etc, if not set it defaults to
-## `beaker.session.secret`
+; encryption key used to encrypt social plugin tokens,
+; remote_urls with credentials etc, if not set it defaults to
+; `beaker.session.secret`
 #rhodecode.encrypted_values.secret =
 
-## decryption strict mode (enabled by default). It controls if decryption raises
-## `SignatureVerificationError` in case of wrong key, or damaged encryption data.
+; decryption strict mode (enabled by default). It controls if decryption raises
+; `SignatureVerificationError` in case of wrong key, or damaged encryption data.
 #rhodecode.encrypted_values.strict = false
 
-## Pick algorithm for encryption. Either fernet (more secure) or aes (default)
-## fernet is safer, and we strongly recommend switching to it.
-## Due to backward compatibility aes is used as default.
+; Pick algorithm for encryption. Either fernet (more secure) or aes (default)
+; fernet is safer, and we strongly recommend switching to it.
+; Due to backward compatibility aes is used as default.
 #rhodecode.encrypted_values.algorithm = fernet
 
-## return gzipped responses from RhodeCode (static files/application)
+; Return gzipped responses from RhodeCode (static files/application)
 gzip_responses = false
 
-## auto-generate javascript routes file on startup
+; Auto-generate javascript routes file on startup
 generate_js_files = false
 
-## System global default language.
-## All available languages: en(default), be, de, es, fr, it, ja, pl, pt, ru, zh
+; System global default language.
+; All available languages: en (default), be, de, es, fr, it, ja, pl, pt, ru, zh
 lang = en
 
-## Perform a full repository scan and import on each server start.
-## Settings this to true could lead to very long startup time.
+; Perform a full repository scan and import on each server start.
+; Settings this to true could lead to very long startup time.
 startup.import_repos = false
 
-## Uncomment and set this path to use archive download cache.
-## Once enabled, generated archives will be cached at this location
-## and served from the cache during subsequent requests for the same archive of
-## the repository.
+; Uncomment and set this path to use archive download cache.
+; Once enabled, generated archives will be cached at this location
+; and served from the cache during subsequent requests for the same archive of
+; the repository.
 #archive_cache_dir = /tmp/tarballcache
 
-## URL at which the application is running. This is used for Bootstrapping
-## requests in context when no web request is available. Used in ishell, or
-## SSH calls. Set this for events to receive proper url for SSH calls.
+; URL at which the application is running. This is used for Bootstrapping
+; requests in context when no web request is available. Used in ishell, or
+; SSH calls. Set this for events to receive proper url for SSH calls.
 app.base_url = http://rhodecode.local
 
-## Unique application ID. Should be a random unique string for security.
+; Unique application ID. Should be a random unique string for security.
 app_instance_uuid = rc-production
 
-## Cut off limit for large diffs (size in bytes). If overall diff size on
-## commit, or pull request exceeds this limit this diff will be displayed
-## partially. E.g 512000 == 512Kb
+; Cut off limit for large diffs (size in bytes). If overall diff size on
+; commit, or pull request exceeds this limit this diff will be displayed
+; partially. E.g 512000 == 512Kb
 cut_off_limit_diff = 512000
 
-## Cut off limit for large files inside diffs (size in bytes). Each individual
-## file inside diff which exceeds this limit will be displayed partially.
-##  E.g 128000 == 128Kb
+; Cut off limit for large files inside diffs (size in bytes). Each individual
+; file inside diff which exceeds this limit will be displayed partially.
+;  E.g 128000 == 128Kb
 cut_off_limit_file = 128000
 
-## use cached version of vcs repositories everywhere. Recommended to be `true`
+; Use cached version of vcs repositories everywhere. Recommended to be `true`
 vcs_full_cache = true
 
-## Force https in RhodeCode, fixes https redirects, assumes it's always https.
-## Normally this is controlled by proper http flags sent from http server
+; Force https in RhodeCode, fixes https redirects, assumes it's always https.
+; Normally this is controlled by proper flags sent from http server such as Nginx or Apache
 force_https = false
 
-## use Strict-Transport-Security headers
+; use Strict-Transport-Security headers
 use_htsts = false
 
-## git rev filter option, --all is the default filter, if you need to
-## hide all refs in changelog switch this to --branches --tags
-git_rev_filter = --branches --tags
-
-# Set to true if your repos are exposed using the dumb protocol
+; Set to true if your repos are exposed using the dumb protocol
 git_update_server_info = false
 
-## RSS/ATOM feed options
+; RSS/ATOM feed options
 rss_cut_off_limit = 256000
 rss_items_per_page = 10
 rss_include_diff = false
 
-## gist URL alias, used to create nicer urls for gist. This should be an
-## url that does rewrites to _admin/gists/{gistid}.
-## example: http://gist.rhodecode.org/{gistid}. Empty means use the internal
-## RhodeCode url, ie. http[s]://rhodecode.server/_admin/gists/{gistid}
+; gist URL alias, used to create nicer urls for gist. This should be an
+; url that does rewrites to _admin/gists/{gistid}.
+; example: http://gist.rhodecode.org/{gistid}. Empty means use the internal
+; RhodeCode url, ie. http[s]://rhodecode.server/_admin/gists/{gistid}
 gist_alias_url =
 
-## List of views (using glob pattern syntax) that AUTH TOKENS could be
-## used for access.
-## Adding ?auth_token=TOKEN_HASH to the url authenticates this request as if it
-## came from the the logged in user who own this authentication token.
-## Additionally @TOKEN syntax can be used to bound the view to specific
-## authentication token. Such view would be only accessible when used together
-## with this authentication token
-##
-## list of all views can be found under `/_admin/permissions/auth_token_access`
-## The list should be "," separated and on a single line.
-##
-## Most common views to enable:
+; List of views (using glob pattern syntax) that AUTH TOKENS could be
+; used for access.
+; Adding ?auth_token=TOKEN_HASH to the url authenticates this request as if it
+; came from the the logged in user who own this authentication token.
+; Additionally @TOKEN syntax can be used to bound the view to specific
+; authentication token. Such view would be only accessible when used together
+; with this authentication token
+; list of all views can be found under `/_admin/permissions/auth_token_access`
+; The list should be "," separated and on a single line.
+; Most common views to enable:
+
 #    RepoCommitsView:repo_commit_download
 #    RepoCommitsView:repo_commit_patch
 #    RepoCommitsView:repo_commit_raw
@@ -201,164 +231,194 b' gist_alias_url ='
 #    GistView:*
 api_access_controllers_whitelist =
 
-## Default encoding used to convert from and to unicode
-## can be also a comma separated list of encoding in case of mixed encodings
+; Default encoding used to convert from and to unicode
+; can be also a comma separated list of encoding in case of mixed encodings
 default_encoding = UTF-8
 
-## instance-id prefix
-## a prefix key for this instance used for cache invalidation when running
-## multiple instances of RhodeCode, make sure it's globally unique for
-## all running RhodeCode instances. Leave empty if you don't use it
+; instance-id prefix
+; a prefix key for this instance used for cache invalidation when running
+; multiple instances of RhodeCode, make sure it's globally unique for
+; all running RhodeCode instances. Leave empty if you don't use it
 instance_id =
 
-## Fallback authentication plugin. Set this to a plugin ID to force the usage
-## of an authentication plugin also if it is disabled by it's settings.
-## This could be useful if you are unable to log in to the system due to broken
-## authentication settings. Then you can enable e.g. the internal RhodeCode auth
-## module to log in again and fix the settings.
-##
-## Available builtin plugin IDs (hash is part of the ID):
-## egg:rhodecode-enterprise-ce#rhodecode
-## egg:rhodecode-enterprise-ce#pam
-## egg:rhodecode-enterprise-ce#ldap
-## egg:rhodecode-enterprise-ce#jasig_cas
-## egg:rhodecode-enterprise-ce#headers
-## egg:rhodecode-enterprise-ce#crowd
+; Fallback authentication plugin. Set this to a plugin ID to force the usage
+; of an authentication plugin also if it is disabled by it's settings.
+; This could be useful if you are unable to log in to the system due to broken
+; authentication settings. Then you can enable e.g. the internal RhodeCode auth
+; module to log in again and fix the settings.
+; Available builtin plugin IDs (hash is part of the ID):
+; egg:rhodecode-enterprise-ce#rhodecode
+; egg:rhodecode-enterprise-ce#pam
+; egg:rhodecode-enterprise-ce#ldap
+; egg:rhodecode-enterprise-ce#jasig_cas
+; egg:rhodecode-enterprise-ce#headers
+; egg:rhodecode-enterprise-ce#crowd
+
 #rhodecode.auth_plugin_fallback = egg:rhodecode-enterprise-ce#rhodecode
 
-## alternative return HTTP header for failed authentication. Default HTTP
-## response is 401 HTTPUnauthorized. Currently HG clients have troubles with
-## handling that causing a series of failed authentication calls.
-## Set this variable to 403 to return HTTPForbidden, or any other HTTP code
-## This will be served instead of default 401 on bad authentication
+; Flag to control loading of legacy plugins in py:/path format
+auth_plugin.import_legacy_plugins = true
+
+; alternative return HTTP header for failed authentication. Default HTTP
+; response is 401 HTTPUnauthorized. Currently HG clients have troubles with
+; handling that causing a series of failed authentication calls.
+; Set this variable to 403 to return HTTPForbidden, or any other HTTP code
+; This will be served instead of default 401 on bad authentication
 auth_ret_code =
 
-## use special detection method when serving auth_ret_code, instead of serving
-## ret_code directly, use 401 initially (Which triggers credentials prompt)
-## and then serve auth_ret_code to clients
+; use special detection method when serving auth_ret_code, instead of serving
+; ret_code directly, use 401 initially (Which triggers credentials prompt)
+; and then serve auth_ret_code to clients
 auth_ret_code_detection = false
 
-## locking return code. When repository is locked return this HTTP code. 2XX
-## codes don't break the transactions while 4XX codes do
+; locking return code. When repository is locked return this HTTP code. 2XX
+; codes don't break the transactions while 4XX codes do
 lock_ret_code = 423
 
-## allows to change the repository location in settings page
+; allows to change the repository location in settings page
 allow_repo_location_change = true
 
-## allows to setup custom hooks in settings page
+; allows to setup custom hooks in settings page
 allow_custom_hooks_settings = true
 
-## Generated license token required for EE edition license.
-## New generated token value can be found in Admin > settings > license page.
+; Generated license token required for EE edition license.
+; New generated token value can be found in Admin > settings > license page.
 license_token =
 
-## supervisor connection uri, for managing supervisor and logs.
+; This flag hides sensitive information on the license page such as token, and license data
+license.hide_license_info = false
+
+; supervisor connection uri, for managing supervisor and logs.
 supervisor.uri =
-## supervisord group name/id we only want this RC instance to handle
+
+; supervisord group name/id we only want this RC instance to handle
 supervisor.group_id = prod
 
-## Display extended labs settings
+; Display extended labs settings
 labs_settings_active = true
 
-## Custom exception store path, defaults to TMPDIR
-## This is used to store exception from RhodeCode in shared directory
+; Custom exception store path, defaults to TMPDIR
+; This is used to store exception from RhodeCode in shared directory
 #exception_tracker.store_path =
 
-## File store configuration. This is used to store and serve uploaded files
+; File store configuration. This is used to store and serve uploaded files
 file_store.enabled = true
-## Storage backend, available options are: local
+
+; Storage backend, available options are: local
 file_store.backend = local
-## path to store the uploaded binaries
+
+; path to store the uploaded binaries
 file_store.storage_path = %(here)s/data/file_store
 
 
-####################################
-###        CELERY CONFIG        ####
-####################################
-## run: /path/to/celery worker \
-##     -E --beat --app rhodecode.lib.celerylib.loader \
-##     --scheduler rhodecode.lib.celerylib.scheduler.RcScheduler \
-##     --loglevel DEBUG --ini /path/to/rhodecode.ini
+; #############
+; CELERY CONFIG
+; #############
+
+; manually run celery: /path/to/celery worker -E --beat --app rhodecode.lib.celerylib.loader --scheduler rhodecode.lib.celerylib.scheduler.RcScheduler --loglevel DEBUG --ini /path/to/rhodecode.ini
 
 use_celery = false
 
-## connection url to the message broker (default rabbitmq)
-celery.broker_url = amqp://rabbitmq:qweqwe@localhost:5672/rabbitmqhost
+; connection url to the message broker (default redis)
+celery.broker_url = redis://localhost:6379/8
 
-## maximum tasks to execute before worker restart
+; rabbitmq example
+#celery.broker_url = amqp://rabbitmq:qweqwe@localhost:5672/rabbitmqhost
+
+; maximum tasks to execute before worker restart
 celery.max_tasks_per_child = 100
 
-## tasks will never be sent to the queue, but executed locally instead.
+; tasks will never be sent to the queue, but executed locally instead.
 celery.task_always_eager = false
 
-#####################################
-###         DOGPILE CACHE        ####
-#####################################
-## Default cache dir for caches.  Putting this into a ramdisk
-## can boost performance, eg. /tmpfs/data_ramdisk, however this directory might require
-## large amount of space
+; #############
+; DOGPILE CACHE
+; #############
+
+; Default cache dir for caches. Putting this into a ramdisk can boost performance.
+; eg. /tmpfs/data_ramdisk, however this directory might require large amount of space
 cache_dir = %(here)s/data
 
-## `cache_perms` cache settings for permission tree, auth TTL.
+; *********************************************
+; `sql_cache_short` cache for heavy SQL queries
+; Only supported backend is `memory_lru`
+; *********************************************
+rc_cache.sql_cache_short.backend = dogpile.cache.rc.memory_lru
+rc_cache.sql_cache_short.expiration_time = 30
+
+
+; *****************************************************
+; `cache_repo_longterm` cache for repo object instances
+; Only supported backend is `memory_lru`
+; *****************************************************
+rc_cache.cache_repo_longterm.backend = dogpile.cache.rc.memory_lru
+; by default we use 30 Days, cache is still invalidated on push
+rc_cache.cache_repo_longterm.expiration_time = 2592000
+; max items in LRU cache, set to smaller number to save memory, and expire last used caches
+rc_cache.cache_repo_longterm.max_size = 10000
+
+
+; *************************************************
+; `cache_perms` cache for permission tree, auth TTL
+; *************************************************
 rc_cache.cache_perms.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_perms.expiration_time = 300
+; file cache store path. Defaults to `cache_dir =` value or tempdir if both values are not set
+#rc_cache.cache_perms.arguments.filename = /tmp/cache_perms.db
 
-## alternative `cache_perms` redis backend with distributed lock
+; alternative `cache_perms` redis backend with distributed lock
 #rc_cache.cache_perms.backend = dogpile.cache.rc.redis
 #rc_cache.cache_perms.expiration_time = 300
-## redis_expiration_time needs to be greater then expiration_time
+
+; redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_perms.arguments.redis_expiration_time = 7200
-#rc_cache.cache_perms.arguments.socket_timeout = 30
+
 #rc_cache.cache_perms.arguments.host = localhost
 #rc_cache.cache_perms.arguments.port = 6379
 #rc_cache.cache_perms.arguments.db = 0
-## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
+#rc_cache.cache_perms.arguments.socket_timeout = 30
+; more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_perms.arguments.distributed_lock = true
 
-## `cache_repo` cache settings for FileTree, Readme, RSS FEEDS
+
+; ***************************************************
+; `cache_repo` cache for file tree, Readme, RSS FEEDS
+; ***************************************************
 rc_cache.cache_repo.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_repo.expiration_time = 2592000
+; file cache store path. Defaults to `cache_dir =` value or tempdir if both values are not set
+#rc_cache.cache_repo.arguments.filename = /tmp/cache_repo.db
 
-## alternative `cache_repo` redis backend with distributed lock
+; alternative `cache_repo` redis backend with distributed lock
 #rc_cache.cache_repo.backend = dogpile.cache.rc.redis
 #rc_cache.cache_repo.expiration_time = 2592000
-## redis_expiration_time needs to be greater then expiration_time
+
+; redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_repo.arguments.redis_expiration_time = 2678400
-#rc_cache.cache_repo.arguments.socket_timeout = 30
+
 #rc_cache.cache_repo.arguments.host = localhost
 #rc_cache.cache_repo.arguments.port = 6379
 #rc_cache.cache_repo.arguments.db = 1
-## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
+#rc_cache.cache_repo.arguments.socket_timeout = 30
+; more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_repo.arguments.distributed_lock = true
 
-## cache settings for SQL queries, this needs to use memory type backend
-rc_cache.sql_cache_short.backend = dogpile.cache.rc.memory_lru
-rc_cache.sql_cache_short.expiration_time = 30
 
-## `cache_repo_longterm` cache for repo object instances, this needs to use memory
-## type backend as the objects kept are not pickle serializable
-rc_cache.cache_repo_longterm.backend = dogpile.cache.rc.memory_lru
-## by default we use 96H, this is using invalidation on push anyway
-rc_cache.cache_repo_longterm.expiration_time = 345600
-## max items in LRU cache, reduce this number to save memory, and expire last used
-## cached objects
-rc_cache.cache_repo_longterm.max_size = 10000
+; ##############
+; BEAKER SESSION
+; ##############
 
-
-####################################
-###       BEAKER SESSION        ####
-####################################
-
-## .session.type is type of storage options for the session, current allowed
-## types are file, ext:memcached, ext:redis, ext:database, and memory (default).
+; beaker.session.type is type of storage options for the logged users sessions. Current allowed
+; types are file, ext:redis, ext:database, ext:memcached, and memory (default if not specified).
+; Fastest ones are Redis and ext:database
 beaker.session.type = file
 beaker.session.data_dir = %(here)s/data/sessions
 
-## redis sessions
+; Redis based sessions
 #beaker.session.type = ext:redis
 #beaker.session.url = redis://127.0.0.1:6379/2
 
-## db based session, fast, and allows easy management over logged in users
+; DB based session, fast, and allows easy management over logged in users
 #beaker.session.type = ext:database
 #beaker.session.table_name = db_session
 #beaker.session.sa.url = postgresql://postgres:secret@localhost/rhodecode
@@ -370,265 +430,275 b' beaker.session.key = rhodecode'
 beaker.session.secret = production-rc-uytcxaz
 beaker.session.lock_dir = %(here)s/data/sessions/lock
 
-## Secure encrypted cookie. Requires AES and AES python libraries
-## you must disable beaker.session.secret to use this
+; Secure encrypted cookie. Requires AES and AES python libraries
+; you must disable beaker.session.secret to use this
 #beaker.session.encrypt_key = key_for_encryption
 #beaker.session.validate_key = validation_key
 
-## sets session as invalid(also logging out user) if it haven not been
-## accessed for given amount of time in seconds
+; Sets session as invalid (also logging out user) if it haven not been
+; accessed for given amount of time in seconds
 beaker.session.timeout = 2592000
 beaker.session.httponly = true
-## Path to use for the cookie. Set to prefix if you use prefix middleware
+
+; Path to use for the cookie. Set to prefix if you use prefix middleware
 #beaker.session.cookie_path = /custom_prefix
 
-## uncomment for https secure cookie
+; Set https secure cookie
 beaker.session.secure = false
 
-## auto save the session to not to use .save()
-beaker.session.auto = false
-
-## default cookie expiration time in seconds, set to `true` to set expire
-## at browser close
+; default cookie expiration time in seconds, set to `true` to set expire
+; at browser close
 #beaker.session.cookie_expires = 3600
 
-###################################
-## SEARCH INDEXING CONFIGURATION ##
-###################################
-## Full text search indexer is available in rhodecode-tools under
-## `rhodecode-tools index` command
+; #############################
+; SEARCH INDEXING CONFIGURATION
+; #############################
 
-## WHOOSH Backend, doesn't require additional services to run
-## it works good with few dozen repos
+; Full text search indexer is available in rhodecode-tools under
+; `rhodecode-tools index` command
+
+; WHOOSH Backend, doesn't require additional services to run
+; it works good with few dozen repos
 search.module = rhodecode.lib.index.whoosh
 search.location = %(here)s/data/index
 
-########################################
-###    CHANNELSTREAM CONFIG         ####
-########################################
-## channelstream enables persistent connections and live notification
-## in the system. It's also used by the chat system
+; ####################
+; CHANNELSTREAM CONFIG
+; ####################
+
+; channelstream enables persistent connections and live notification
+; in the system. It's also used by the chat system
 
 channelstream.enabled = false
 
-## server address for channelstream server on the backend
+; server address for channelstream server on the backend
 channelstream.server = 127.0.0.1:9800
 
-## location of the channelstream server from outside world
-## use ws:// for http or wss:// for https. This address needs to be handled
-## by external HTTP server such as Nginx or Apache
-## see Nginx/Apache configuration examples in our docs
+; location of the channelstream server from outside world
+; use ws:// for http or wss:// for https. This address needs to be handled
+; by external HTTP server such as Nginx or Apache
+; see Nginx/Apache configuration examples in our docs
 channelstream.ws_url = ws://rhodecode.yourserver.com/_channelstream
 channelstream.secret = secret
 channelstream.history.location = %(here)s/channelstream_history
 
-## Internal application path that Javascript uses to connect into.
-## If you use proxy-prefix the prefix should be added before /_channelstream
+; Internal application path that Javascript uses to connect into.
+; If you use proxy-prefix the prefix should be added before /_channelstream
 channelstream.proxy_path = /_channelstream
 
 
-###################################
-##       APPENLIGHT CONFIG       ##
-###################################
+; ##############################
+; MAIN RHODECODE DATABASE CONFIG
+; ##############################
+
+#sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
+#sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
+#sqlalchemy.db1.url = mysql://root:qweqwe@localhost/rhodecode?charset=utf8
+; pymysql is an alternative driver for MySQL, use in case of problems with default one
+#sqlalchemy.db1.url = mysql+pymysql://root:qweqwe@localhost/rhodecode
+
+sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
+
+; see sqlalchemy docs for other advanced settings
+; print the sql statements to output
+sqlalchemy.db1.echo = false
+
+; recycle the connections after this amount of seconds
+sqlalchemy.db1.pool_recycle = 3600
+sqlalchemy.db1.convert_unicode = true
+
+; the number of connections to keep open inside the connection pool.
+; 0 indicates no limit
+#sqlalchemy.db1.pool_size = 5
+
+; The number of connections to allow in connection pool "overflow", that is
+; connections that can be opened above and beyond the pool_size setting,
+; which defaults to five.
+#sqlalchemy.db1.max_overflow = 10
+
+; Connection check ping, used to detect broken database connections
+; could be enabled to better handle cases if MySQL has gone away errors
+#sqlalchemy.db1.ping_connection = true
+
+; ##########
+; VCS CONFIG
+; ##########
+vcs.server.enable = true
+vcs.server = localhost:9900
+
+; Web server connectivity protocol, responsible for web based VCS operations
+; Available protocols are:
+; `http` - use http-rpc backend (default)
+vcs.server.protocol = http
+
+; Push/Pull operations protocol, available options are:
+; `http` - use http-rpc backend (default)
+vcs.scm_app_implementation = http
+
+; Push/Pull operations hooks protocol, available options are:
+; `http` - use http-rpc backend (default)
+vcs.hooks.protocol = http
+
+; Host on which this instance is listening for hooks. If vcsserver is in other location
+; this should be adjusted.
+vcs.hooks.host = 127.0.0.1
+
+; Start VCSServer with this instance as a subprocess, useful for development
+vcs.start_server = false
+
+; List of enabled VCS backends, available options are:
+; `hg`  - mercurial
+; `git` - git
+; `svn` - subversion
+vcs.backends = hg, git, svn
+
+; Wait this number of seconds before killing connection to the vcsserver
+vcs.connection_timeout = 3600
+
+; Compatibility version when creating SVN repositories. Defaults to newest version when commented out.
+; Available options are: pre-1.4-compatible, pre-1.5-compatible, pre-1.6-compatible, pre-1.8-compatible, pre-1.9-compatible
+#vcs.svn.compatible_version = pre-1.8-compatible
+
 
-## Appenlight is tailored to work with RhodeCode, see
-## http://appenlight.com for details how to obtain an account
+; ####################################################
+; Subversion proxy support (mod_dav_svn)
+; Maps RhodeCode repo groups into SVN paths for Apache
+; ####################################################
+
+; Enable or disable the config file generation.
+svn.proxy.generate_config = false
+
+; Generate config file with `SVNListParentPath` set to `On`.
+svn.proxy.list_parent_path = true
+
+; Set location and file name of generated config file.
+svn.proxy.config_file_path = %(here)s/mod_dav_svn.conf
+
+; alternative mod_dav config template. This needs to be a valid mako template
+; Example template can be found in the source code:
+; rhodecode/apps/svn_support/templates/mod-dav-svn.conf.mako
+#svn.proxy.config_template = ~/.rccontrol/enterprise-1/custom_svn_conf.mako
+
+; Used as a prefix to the `Location` block in the generated config file.
+; In most cases it should be set to `/`.
+svn.proxy.location_root = /
+
+; Command to reload the mod dav svn configuration on change.
+; Example: `/etc/init.d/apache2 reload` or /home/USER/apache_reload.sh
+; Make sure user who runs RhodeCode process is allowed to reload Apache
+#svn.proxy.reload_cmd = /etc/init.d/apache2 reload
+
+; If the timeout expires before the reload command finishes, the command will
+; be killed. Setting it to zero means no timeout. Defaults to 10 seconds.
+#svn.proxy.reload_timeout = 10
+
+; ####################
+; SSH Support Settings
+; ####################
 
-## Appenlight integration enabled
+; Defines if a custom authorized_keys file should be created and written on
+; any change user ssh keys. Setting this to false also disables possibility
+; of adding SSH keys by users from web interface. Super admins can still
+; manage SSH Keys.
+ssh.generate_authorized_keyfile = false
+
+; Options for ssh, default is `no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding`
+# ssh.authorized_keys_ssh_opts =
+
+; Path to the authorized_keys file where the generate entries are placed.
+; It is possible to have multiple key files specified in `sshd_config` e.g.
+; AuthorizedKeysFile %h/.ssh/authorized_keys %h/.ssh/authorized_keys_rhodecode
+ssh.authorized_keys_file_path = ~/.ssh/authorized_keys_rhodecode
+
+; Command to execute the SSH wrapper. The binary is available in the
+; RhodeCode installation directory.
+; e.g ~/.rccontrol/community-1/profile/bin/rc-ssh-wrapper
+ssh.wrapper_cmd = ~/.rccontrol/community-1/rc-ssh-wrapper
+
+; Allow shell when executing the ssh-wrapper command
+ssh.wrapper_cmd_allow_shell = false
+
+; Enables logging, and detailed output send back to the client during SSH
+; operations. Useful for debugging, shouldn't be used in production.
+ssh.enable_debug_logging = false
+
+; Paths to binary executable, by default they are the names, but we can
+; override them if we want to use a custom one
+ssh.executable.hg = ~/.rccontrol/vcsserver-1/profile/bin/hg
+ssh.executable.git = ~/.rccontrol/vcsserver-1/profile/bin/git
+ssh.executable.svn = ~/.rccontrol/vcsserver-1/profile/bin/svnserve
+
+; Enables SSH key generator web interface. Disabling this still allows users
+; to add their own keys.
+ssh.enable_ui_key_generator = true
+
+
+; #################
+; APPENLIGHT CONFIG
+; #################
+
+; Appenlight is tailored to work with RhodeCode, see
+; http://appenlight.rhodecode.com for details how to obtain an account
+
+; Appenlight integration enabled
 appenlight = false
 
 appenlight.server_url = https://api.appenlight.com
 appenlight.api_key = YOUR_API_KEY
 #appenlight.transport_config = https://api.appenlight.com?threaded=1&timeout=5
 
-## used for JS client
+; used for JS client
 appenlight.api_public_key = YOUR_API_PUBLIC_KEY
 
-## TWEAK AMOUNT OF INFO SENT HERE
+; TWEAK AMOUNT OF INFO SENT HERE
 
-## enables 404 error logging (default False)
+; enables 404 error logging (default False)
 appenlight.report_404 = false
 
-## time in seconds after request is considered being slow (default 1)
+; time in seconds after request is considered being slow (default 1)
 appenlight.slow_request_time = 1
 
-## record slow requests in application
-## (needs to be enabled for slow datastore recording and time tracking)
+; record slow requests in application
+; (needs to be enabled for slow datastore recording and time tracking)
 appenlight.slow_requests = true
 
-## enable hooking to application loggers
+; enable hooking to application loggers
 appenlight.logging = true
 
-## minimum log level for log capture
+; minimum log level for log capture
 appenlight.logging.level = WARNING
 
-## send logs only from erroneous/slow requests
-## (saves API quota for intensive logging)
+; send logs only from erroneous/slow requests
+; (saves API quota for intensive logging)
 appenlight.logging_on_error = false
 
-## list of additional keywords that should be grabbed from environ object
-## can be string with comma separated list of words in lowercase
-## (by default client will always send following info:
-## 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
-## start with HTTP* this list be extended with additional keywords here
+; list of additional keywords that should be grabbed from environ object
+; can be string with comma separated list of words in lowercase
+; (by default client will always send following info:
+; 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
+; start with HTTP* this list be extended with additional keywords here
 appenlight.environ_keys_whitelist =
 
-## list of keywords that should be blanked from request object
-## can be string with comma separated list of words in lowercase
-## (by default client will always blank keys that contain following words
-## 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
-## this list be extended with additional keywords set here
+; list of keywords that should be blanked from request object
+; can be string with comma separated list of words in lowercase
+; (by default client will always blank keys that contain following words
+; 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
+; this list be extended with additional keywords set here
 appenlight.request_keys_blacklist =
 
-## list of namespaces that should be ignores when gathering log entries
-## can be string with comma separated list of namespaces
-## (by default the client ignores own entries: appenlight_client.client)
+; list of namespaces that should be ignores when gathering log entries
+; can be string with comma separated list of namespaces
+; (by default the client ignores own entries: appenlight_client.client)
 appenlight.log_namespace_blacklist =
 
-
-###########################################
-###   MAIN RHODECODE DATABASE CONFIG    ###
-###########################################
-#sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
-#sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
-#sqlalchemy.db1.url = mysql://root:qweqwe@localhost/rhodecode?charset=utf8
-# pymysql is an alternative driver for MySQL, use in case of problems with default one
-#sqlalchemy.db1.url = mysql+pymysql://root:qweqwe@localhost/rhodecode
-
-sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
-
-# see sqlalchemy docs for other advanced settings
-
-## print the sql statements to output
-sqlalchemy.db1.echo = false
-## recycle the connections after this amount of seconds
-sqlalchemy.db1.pool_recycle = 3600
-sqlalchemy.db1.convert_unicode = true
-
-## the number of connections to keep open inside the connection pool.
-## 0 indicates no limit
-#sqlalchemy.db1.pool_size = 5
-
-## the number of connections to allow in connection pool "overflow", that is
-## connections that can be opened above and beyond the pool_size setting,
-## which defaults to five.
-#sqlalchemy.db1.max_overflow = 10
-
-## Connection check ping, used to detect broken database connections
-## could be enabled to better handle cases if MySQL has gone away errors
-#sqlalchemy.db1.ping_connection = true
-
-##################
-### VCS CONFIG ###
-##################
-vcs.server.enable = true
-vcs.server = localhost:9900
-
-## Web server connectivity protocol, responsible for web based VCS operations
-## Available protocols are:
-## `http` - use http-rpc backend (default)
-vcs.server.protocol = http
-
-## Push/Pull operations protocol, available options are:
-## `http` - use http-rpc backend (default)
-vcs.scm_app_implementation = http
-
-## Push/Pull operations hooks protocol, available options are:
-## `http` - use http-rpc backend (default)
-vcs.hooks.protocol = http
-
-## Host on which this instance is listening for hooks. If vcsserver is in other location
-## this should be adjusted.
-vcs.hooks.host = 127.0.0.1
-
-vcs.server.log_level = info
-## Start VCSServer with this instance as a subprocess, useful for development
-vcs.start_server = false
-
-## List of enabled VCS backends, available options are:
-## `hg`  - mercurial
-## `git` - git
-## `svn` - subversion
-vcs.backends = hg, git, svn
-
-vcs.connection_timeout = 3600
-## Compatibility version when creating SVN repositories. Defaults to newest version when commented out.
-## Available options are: pre-1.4-compatible, pre-1.5-compatible, pre-1.6-compatible, pre-1.8-compatible, pre-1.9-compatible
-#vcs.svn.compatible_version = pre-1.8-compatible
-
-
-############################################################
-### Subversion proxy support (mod_dav_svn)               ###
-### Maps RhodeCode repo groups into SVN paths for Apache ###
-############################################################
-## Enable or disable the config file generation.
-svn.proxy.generate_config = false
-## Generate config file with `SVNListParentPath` set to `On`.
-svn.proxy.list_parent_path = true
-## Set location and file name of generated config file.
-svn.proxy.config_file_path = %(here)s/mod_dav_svn.conf
-## alternative mod_dav config template. This needs to be a mako template
-#svn.proxy.config_template = ~/.rccontrol/enterprise-1/custom_svn_conf.mako
-## Used as a prefix to the `Location` block in the generated config file.
-## In most cases it should be set to `/`.
-svn.proxy.location_root = /
-## Command to reload the mod dav svn configuration on change.
-## Example: `/etc/init.d/apache2 reload` or /home/USER/apache_reload.sh
-## Make sure user who runs RhodeCode process is allowed to reload Apache
-#svn.proxy.reload_cmd = /etc/init.d/apache2 reload
-## If the timeout expires before the reload command finishes, the command will
-## be killed. Setting it to zero means no timeout. Defaults to 10 seconds.
-#svn.proxy.reload_timeout = 10
-
-############################################################
-### SSH Support Settings                                 ###
-############################################################
-
-## Defines if a custom authorized_keys file should be created and written on
-## any change user ssh keys. Setting this to false also disables possibility
-## of adding SSH keys by users from web interface. Super admins can still
-## manage SSH Keys.
-ssh.generate_authorized_keyfile = false
-
-## Options for ssh, default is `no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding`
-# ssh.authorized_keys_ssh_opts =
-
-## Path to the authorized_keys file where the generate entries are placed.
-## It is possible to have multiple key files specified in `sshd_config` e.g.
-## AuthorizedKeysFile %h/.ssh/authorized_keys %h/.ssh/authorized_keys_rhodecode
-ssh.authorized_keys_file_path = ~/.ssh/authorized_keys_rhodecode
-
-## Command to execute the SSH wrapper. The binary is available in the
-## RhodeCode installation directory.
-## e.g ~/.rccontrol/community-1/profile/bin/rc-ssh-wrapper
-ssh.wrapper_cmd = ~/.rccontrol/community-1/rc-ssh-wrapper
-
-## Allow shell when executing the ssh-wrapper command
-ssh.wrapper_cmd_allow_shell = false
-
-## Enables logging, and detailed output send back to the client during SSH
-## operations. Useful for debugging, shouldn't be used in production.
-ssh.enable_debug_logging = false
-
-## Paths to binary executable, by default they are the names, but we can
-## override them if we want to use a custom one
-ssh.executable.hg = ~/.rccontrol/vcsserver-1/profile/bin/hg
-ssh.executable.git = ~/.rccontrol/vcsserver-1/profile/bin/git
-ssh.executable.svn = ~/.rccontrol/vcsserver-1/profile/bin/svnserve
-
-## Enables SSH key generator web interface. Disabling this still allows users
-## to add their own keys.
-ssh.enable_ui_key_generator = true
-
-
-## Dummy marker to add new entries after.
-## Add any custom entries below. Please don't remove.
+; Dummy marker to add new entries after.
+; Add any custom entries below. Please don't remove this marker.
 custom.conf = 1
 
 
-################################
-### LOGGING CONFIGURATION   ####
-################################
+; #####################
+; LOGGING CONFIGURATION
+; #####################
 [loggers]
 keys = root, sqlalchemy, beaker, celery, rhodecode, ssh_wrapper
 
@@ -638,9 +708,9 b' keys = console, console_sql'
 [formatters]
 keys = generic, color_formatter, color_formatter_sql
 
-#############
-## LOGGERS ##
-#############
+; #######
+; LOGGERS
+; #######
 [logger_root]
 level = NOTSET
 handlers = console
@@ -675,9 +745,9 b' handlers ='
 qualname = celery
 
 
-##############
-## HANDLERS ##
-##############
+; ########
+; HANDLERS
+; ########
 
 [handler_console]
 class = StreamHandler
@@ -686,17 +756,17 b' level = INFO'
 formatter = generic
 
 [handler_console_sql]
-# "level = DEBUG" logs SQL queries and results.
-# "level = INFO" logs SQL queries.
-# "level = WARN" logs neither.  (Recommended for production systems.)
+; "level = DEBUG" logs SQL queries and results.
+; "level = INFO" logs SQL queries.
+; "level = WARN" logs neither.  (Recommended for production systems.)
 class = StreamHandler
 args = (sys.stderr, )
 level = WARN
 formatter = generic
 
-################
-## FORMATTERS ##
-################
+; ##########
+; FORMATTERS
+; ##########
 
 [formatter_generic]
 class = rhodecode.lib.logging_formatter.ExceptionAwareFormatter

default.nix

@@ -190,7 +190,7 b' let'
       postInstall = ''
         # check required files
         STATIC_CHECK="/robots.txt /502.html
-                      /js/scripts.js /js/rhodecode-components.js
+                      /js/scripts.min.js /js/rhodecode-components.js
                       /css/style.css /css/style-polymer.css /css/style-ipython.css"
 
         for file in $STATIC_CHECK;

docs/admin/repo_admin/repo-admin-tasks.rst

@@ -12,13 +12,50 b' Here is how to force delete a repository'
 
 
 .. code-block:: bash
+   :dedent: 1
 
     # starts the ishell interactive prompt
     $ rccontrol ishell enterprise-1
 
 .. code-block:: python
+   :dedent: 1
 
     In [4]: from rhodecode.model.repo import RepoModel
     In [3]: repo = Repository.get_by_repo_name('test_repos/repo_with_prs')
     In [5]: RepoModel().delete(repo, forks='detach', pull_requests='delete')
     In [6]: Session().commit()
+
+
+Below is a fully automated example to force delete repositories reading from a
+file where each line is a repository name. This can be executed via simple CLI command
+without entering the interactive shell.
+
+Save the below content as a file named `repo_delete_task.py`
+
+
+.. code-block:: python
+   :dedent: 1
+
+    from rhodecode.model.db import *
+    from rhodecode.model.repo import RepoModel
+    with open('delete_repos.txt', 'rb') as f:
+        # read all lines from file
+        repos = f.readlines()
+        for repo_name in repos:
+            repo_name = repo_name.strip()  # cleanup the name just in case
+            repo = Repository.get_by_repo_name(repo_name)
+            if not repo:
+                raise Exception('Repo with name {} not found'.format(repo_name))
+            RepoModel().delete(repo, forks='detach', pull_requests='delete')
+            Session().commit()
+            print('Removed repository {}'.format(repo_name))
+
+
+The code above will read the names of repositories from a file called `delete_repos.txt`
+Each lines should represent a single name e.g `repo_name_1` or `repo_group/repo_name_2`
+
+Run this line from CLI to execute the code from the `repo_delete_task.py` file and
+exit the ishell after the execution::
+
+    echo "%run repo_delete_task.py" | rccontrol ishell Enterprise-1
+

docs/admin/system_admin/nginx/nginx-config-example.rst

@@ -110,6 +110,7 b' Use the following example to configure N'
         #    gzip_types  text/css text/javascript text/xml text/plain text/x-component application/javascript application/json application/xml application/rss+xml font/truetype font/opentype application/vnd.ms-fontobject image/svg+xml;
         #    gzip_vary on;
         #    gzip_disable     "msie6";
+        #    expires 60d;
         #    alias /path/to/.rccontrol/community-1/static;
         #    alias /path/to/.rccontrol/enterprise-1/static;
         # }

docs/admin/system_admin/tuning-rhodecode.rst

@@ -9,7 +9,8 b' may find some of the following methods u'
 .. toctree::
 
    tuning/tuning-gunicorn
-   tuning/tuning-vcs-memory-cache
+   tuning/tuning-vcs-server-memory-usage
+   tuning/tuning-rhodecode-memory-usage
    tuning/tuning-user-sessions-performance
    tuning/tuning-increase-db-performance
    tuning/tuning-scale-horizontally-cluster

docs/admin/system_admin/tuning/tuning-gunicorn.rst

@@ -25,26 +25,22 b' 2. In the ``[server:main]`` section, cha'
 
 .. code-block:: ini
 
-    use = egg:gunicorn#main
-    ## Sets the number of process workers. You must set `instance_id = *`
-    ## when this option is set to more than one worker, recommended
-    ## value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers
-    ## The `instance_id = *` must be set in the [app:main] section below
-    workers = 4
-    ## process name
-    proc_name = rhodecode
-    ## type of worker class, one of sync, gevent
-    ## recommended for bigger setup is using of of other than sync one
-    worker_class = sync
-    ## The maximum number of simultaneous clients. Valid only for Gevent
-    #worker_connections = 10
-    ## max number of requests that worker will handle before being gracefully
-    ## restarted, could prevent memory leaks
-    max_requests = 1000
-    max_requests_jitter = 30
-    ## amount of time a worker can spend with handling a request tuning-change-lfs-dir.before it
-    ## gets killed and restarted. Set to 6hrs
-    timeout = 21600
+    ; Sets the number of process workers. More workers means more concurrent connections
+    ; RhodeCode can handle at the same time. Each additional worker also it increases
+    ; memory usage as each has it's own set of caches.
+    ; Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
+    ; than 8-10 unless for really big deployments .e.g 700-1000 users.
+    ; `instance_id = *` must be set in the [app:main] section below (which is the default)
+    ; when using more than 1 worker.
+    workers = 6
+
+    ; Type of worker class, one of `sync`, `gevent`
+    ; Use `gevent` for rhodecode
+    worker_class = gevent
+
+    ; The maximum number of simultaneous clients per worker. Valid only for gevent
+    worker_connections = 10
+
 
 3. In the ``[app:main]`` section, set the ``instance_id`` property to ``*``.
 
@@ -63,24 +59,19 b' 5. In the ``[server:main]`` section, inc'
 
 .. code-block:: ini
 
-    ## run with gunicorn --log-config vcsserver.ini --paste vcsserver.ini
-    use = egg:gunicorn#main
-    ## Sets the number of process workers. Recommended
-    ## value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers
-    workers = 4
-    ## process name
-    proc_name = rhodecode_vcsserver
-    ## type of worker class, currently `sync` is the only option allowed.
+    ; Sets the number of process workers. More workers means more concurrent connections
+    ; RhodeCode can handle at the same time. Each additional worker also it increases
+    ; memory usage as each has it's own set of caches.
+    ; Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
+    ; than 8-10 unless for really big deployments .e.g 700-1000 users.
+    ; `instance_id = *` must be set in the [app:main] section below (which is the default)
+    ; when using more than 1 worker.
+    workers = 8
+
+    ; Type of worker class, one of `sync`, `gevent`
+    ; Use `sync` for vcsserver
     worker_class = sync
-    ## The maximum number of simultaneous clients. Valid only for Gevent
-    #worker_connections = 10
-    ## max number of requests that worker will handle before being gracefully
-    ## restarted, could prevent memory leaks
-    max_requests = 1000
-    max_requests_jitter = 30
-    ## amount of time a worker can spend with handling a request before it
-    ## gets killed and restarted. Set to 6hrs
-    timeout = 21600
+
 
 6. Save your changes.
 7. Restart your |RCE| instances, using the following command:
@@ -109,17 +100,18 b' 2. In the ``[server:main]`` section, cha'
 
 .. code-block:: ini
 
-    ## type of worker class, one of sync, gevent
-    ## recommended for bigger setup is using of of other than sync one
+    ; Type of worker class, one of `sync`, `gevent`
+    ; Use `gevent` for rhodecode
     worker_class = gevent
-    ## The maximum number of simultaneous clients. Valid only for Gevent
+
+    ; The maximum number of simultaneous clients per worker. Valid only for gevent
     worker_connections = 30
 
 
 .. note::
 
     `Gevent` is currently only supported for Enterprise/Community instances.
-    VCSServer doesn't yet support gevent.
+    VCSServer doesn't support gevent.
 
 
 

docs/admin/system_admin/tuning/tuning-scale-horizontally-cluster.rst

@@ -57,7 +57,7 b" Here's an overview what components shoul"
  - `nginx` acting as a load-balancer.
  - `postgresql-server` used for database and sessions.
  - `redis-server` used for storing shared caches.
- - optionally `rabbitmq-server` for `Celery` if used.
+ - optionally `rabbitmq-server` or `redis` for `Celery` if used.
  - optionally if `Celery` is used Enterprise/Community instance + VCSServer.
  - optionally mailserver that can be shared by other instances.
  - optionally channelstream server to handle live communication for all instances.
@@ -263,6 +263,7 b' 6) Configure `Nginx`_ as reverse proxy o'
             gzip_types  text/css text/javascript text/xml text/plain text/x-component application/javascript application/json application/xml application/rss+xml font/truetype font/opentype application/vnd.ms-fontobject image/svg+xml;
             gzip_vary on;
             gzip_disable     "msie6";
+            expires 60d;
             #alias /home/rcdev/.rccontrol/community-1/static;
             alias /home/rcdev/.rccontrol/enterprise-1/static;
         }
@@ -372,16 +373,16 b' Using Celery with cluster'
 
 
 If `Celery` is used we recommend setting also an instance of Enterprise/Community+VCSserver
-on the node that is running `RabbitMQ`_. Those instances will be used to executed async
-tasks on the `rc-node-1`. This is the most efficient setup. `Celery` usually
-handles tasks such as sending emails, forking repositories, importing
+on the node that is running `RabbitMQ`_ or `Redis`_. Those instances will be used to
+executed async tasks on the `rc-node-1`. This is the most efficient setup.
+`Celery` usually handles tasks such as sending emails, forking repositories, importing
 repositories from external location etc. Using workers on instance that has
 the direct access to disks used by NFS as well as email server gives noticeable
 performance boost. Running local workers to the NFS storage results in faster
 execution of forking large repositories or sending lots of emails.
 
 Those instances need to be configured in the same way as for other nodes.
-The instance in rc-node-1 can be added to the cluser, but we don't recommend doing it.
+The instance in rc-node-1 can be added to the cluster, but we don't recommend doing it.
 For best results let it be isolated to only executing `Celery` tasks in the cluster setup.
 
 

docs/admin/system_admin/tuning/tuning-vcs-server-memory-usage.rst

@@ -1,8 +1,26 b''
-.. _adjust-vcs-mem-cache:
+.. _adjust-vcs-server-mem:
 
-VCSServer Memory Cache
+VCSServer Memory Usage
 ----------------------
 
-The VCS Server mamory cache can be adjusted to work best with the resources
-available to your |RCE| instance. If you find that memory resources are under
-pressure, see the :ref:`vcs-server-maintain` section for details.
+Starting from Version 4.18.X RhodeCode has a builtin memory monitor for gunicorn workers.
+Enabling this can limit the maximum amount of memory system can use. Each worker
+for VCS Server is monitored independently.
+To enable Memory management make sure to have following settings inside `[app:main] section` of
+:file:`home/{user}/.rccontrol/{instance-id}/vcsserver.ini` file.
+
+
+
+    ; Maximum memory usage that each worker can use before it will receive a
+    ; graceful restart signal 0 = memory monitoring is disabled
+    ; Examples: 268435456 (256MB), 536870912 (512MB)
+    ; 1073741824 (1GB), 2147483648 (2GB), 4294967296 (4GB)
+    memory_max_usage = 1073741824
+
+    ; How often in seconds to check for memory usage for each gunicorn worker
+    memory_usage_check_interval = 60
+
+    ; Threshold value for which we don't recycle worker if GarbageCollection
+    ; frees up enough resources. Before each restart we try to run GC on worker
+    ; in case we get enough free memory after that, restart will not happen.
+    memory_usage_recovery_threshold = 0.8

docs/admin/system_admin/vcs-server.rst

@@ -110,35 +110,39 b' match, for example:'
 
 .. _vcs-server-maintain:
 
-VCS Server Memory Optimization
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+VCS Server Cache Optimization
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-To optimize the VCS server to manage the cache and memory usage efficiently, you need to
-configure the following options in the
-:file:`/home/{user}/.rccontrol/{vcsserver-id}/vcsserver.ini` file. Once
-configured, restart the VCS Server. By default we use an optimal settings, but in certain
-conditions tunning expiration_time and max_size can affect memory usage and performance
+To optimize the VCS server to manage the cache and memory usage efficiently, it's recommended to
+configure the Redis backend for VCSServer caches.
+Once configured, restart the VCS Server.
+
+Make sure Redis is installed and running.
+Open :file:`/home/{user}/.rccontrol/{vcsserver-id}/vcsserver.ini`
+file and ensure the below settings for `repo_object` type cache are set:
 
 .. code-block:: ini
 
-    ## cache region for storing repo_objects cache
-    rc_cache.repo_object.backend = dogpile.cache.rc.memory_lru
+    ; ensure the default file based cache is *commented out*
+    ##rc_cache.repo_object.backend = dogpile.cache.rc.file_namespace
+    ##rc_cache.repo_object.expiration_time = 2592000
 
-    ## cache auto-expires after N seconds, setting this to 0 disabled cache
-    rc_cache.repo_object.expiration_time = 300
+    ; `repo_object` cache settings for vcs methods for repositories
+    rc_cache.repo_object.backend = dogpile.cache.rc.redis_msgpack
 
-    ## max size of LRU, old values will be discarded if the size of cache reaches max_size
-    ## Sets the maximum number of items stored in the cache, before the cache
-    ## starts to be cleared.
+    ; cache auto-expires after N seconds
+    ; Examples: 86400 (1Day), 604800 (7Days), 1209600 (14Days), 2592000 (30days), 7776000 (90Days)
+    rc_cache.repo_object.expiration_time = 2592000
+
+    ; redis_expiration_time needs to be greater then expiration_time
+    rc_cache.repo_object.arguments.redis_expiration_time = 3592000
 
-    ## As a general rule of thumb, running this value at 120 resulted in a
-    ## 5GB cache. Running it at 240 resulted in a 9GB cache. Your results
-    ## will differ based on usage patterns and |repo| sizes.
-
-    ## Tweaking this value to run at a fairly constant memory load on your
-    ## server will help performance.
-
-    rc_cache.repo_object.max_size = 120
+    rc_cache.repo_object.arguments.host = localhost
+    rc_cache.repo_object.arguments.port = 6379
+    rc_cache.repo_object.arguments.db = 5
+    rc_cache.repo_object.arguments.socket_timeout = 30
+    ; more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
+    rc_cache.repo_object.arguments.distributed_lock = true
 
 
 To clear the cache completely, you can restart the VCS Server.
@@ -190,25 +194,6 b' For a more detailed explanation of the l'
     \port <int>
         Set the port number on which the VCS Server will be available.
 
-    \locale <locale_utf>
-        Set the locale the VCS Server expects.
-
-    \workers <int>
-        Set the number of process workers.Recommended
-        value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers
-
-    \max_requests <int>
-        The maximum number of requests a worker will process before restarting.
-        Any value greater than zero will limit the number of requests a work
-        will process before automatically restarting. This is a simple method
-        to help limit the damage of memory leaks.
-
-    \max_requests_jitter <int>
-        The maximum jitter to add to the max_requests setting.
-        The jitter causes the restart per worker to be randomized by
-        randint(0, max_requests_jitter). This is intended to stagger worker
-        restarts to avoid all workers restarting at the same time.
-
 
 .. note::
 
@@ -216,63 +201,139 b' For a more detailed explanation of the l'
 
 .. code-block:: ini
 
-    ################################################################################
-    # RhodeCode VCSServer with HTTP Backend - configuration                        #
-    #                                                                              #
-    ################################################################################
-
+    ; #################################
+    ; RHODECODE VCSSERVER CONFIGURATION
+    ; #################################
 
     [server:main]
-    ## COMMON ##
+    ; COMMON HOST/IP CONFIG
     host = 127.0.0.1
     port = 10002
 
-    ##########################
-    ## GUNICORN WSGI SERVER ##
-    ##########################
-    ## run with gunicorn --log-config vcsserver.ini --paste vcsserver.ini
+    ; ###########################
+    ; GUNICORN APPLICATION SERVER
+    ; ###########################
+
+    ; run with gunicorn --log-config rhodecode.ini --paste rhodecode.ini
+
+    ; Module to use, this setting shouldn't be changed
     use = egg:gunicorn#main
-    ## Sets the number of process workers. Recommended
-    ## value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers
-    workers = 3
-    ## process name
+
+    ; Sets the number of process workers. More workers means more concurrent connections
+    ; RhodeCode can handle at the same time. Each additional worker also it increases
+    ; memory usage as each has it's own set of caches.
+    ; Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
+    ; than 8-10 unless for really big deployments .e.g 700-1000 users.
+    ; `instance_id = *` must be set in the [app:main] section below (which is the default)
+    ; when using more than 1 worker.
+    workers = 6
+
+    ; Gunicorn access log level
+    loglevel = info
+
+    ; Process name visible in process list
     proc_name = rhodecode_vcsserver
-    ## type of worker class, one of sync, gevent
-    ## recommended for bigger setup is using of of other than sync one
+
+    ; Type of worker class, one of sync, gevent
+    ; currently `sync` is the only option allowed.
     worker_class = sync
-    ## The maximum number of simultaneous clients. Valid only for Gevent
-    #worker_connections = 10
-    ## max number of requests that worker will handle before being gracefully
-    ## restarted, could prevent memory leaks
+
+    ; The maximum number of simultaneous clients. Valid only for gevent
+    worker_connections = 10
+
+    ; Max number of requests that worker will handle before being gracefully restarted.
+    ; Prevents memory leaks, jitter adds variability so not all workers are restarted at once.
     max_requests = 1000
     max_requests_jitter = 30
-    ## amount of time a worker can spend with handling a request before it
-    ## gets killed and restarted. Set to 6hrs
+
+    ; Amount of time a worker can spend with handling a request before it
+    ; gets killed and restarted. By default set to 21600 (6hrs)
+    ; Examples: 1800 (30min), 3600 (1hr), 7200 (2hr), 43200 (12h)
     timeout = 21600
 
+    ; The maximum size of HTTP request line in bytes.
+    ; 0 for unlimited
+    limit_request_line = 0
+
+    ; Limit the number of HTTP headers fields in a request.
+    ; By default this value is 100 and can't be larger than 32768.
+    limit_request_fields = 32768
+
+    ; Limit the allowed size of an HTTP request header field.
+    ; Value is a positive number or 0.
+    ; Setting it to 0 will allow unlimited header field sizes.
+    limit_request_field_size = 0
+
+    ; Timeout for graceful workers restart.
+    ; After receiving a restart signal, workers have this much time to finish
+    ; serving requests. Workers still alive after the timeout (starting from the
+    ; receipt of the restart signal) are force killed.
+    ; Examples: 1800 (30min), 3600 (1hr), 7200 (2hr), 43200 (12h)
+    graceful_timeout = 3600
+
+    # The number of seconds to wait for requests on a Keep-Alive connection.
+    # Generally set in the 1-5 seconds range.
+    keepalive = 2
+
+    ; Maximum memory usage that each worker can use before it will receive a
+    ; graceful restart signal 0 = memory monitoring is disabled
+    ; Examples: 268435456 (256MB), 536870912 (512MB)
+    ; 1073741824 (1GB), 2147483648 (2GB), 4294967296 (4GB)
+    memory_max_usage = 1073741824
+
+    ; How often in seconds to check for memory usage for each gunicorn worker
+    memory_usage_check_interval = 60
+
+    ; Threshold value for which we don't recycle worker if GarbageCollection
+    ; frees up enough resources. Before each restart we try to run GC on worker
+    ; in case we get enough free memory after that, restart will not happen.
+    memory_usage_recovery_threshold = 0.8
+
+
     [app:main]
     use = egg:rhodecode-vcsserver
 
     pyramid.default_locale_name = en
     pyramid.includes =
 
-    ## default locale used by VCS systems
+    ; default locale used by VCS systems
     locale = en_US.UTF-8
 
-    # cache regions, please don't change
-    beaker.cache.regions = repo_object
-    beaker.cache.repo_object.type = memorylru
-    beaker.cache.repo_object.max_items = 100
-    # cache auto-expires after N seconds
-    beaker.cache.repo_object.expire = 300
-    beaker.cache.repo_object.enabled = true
+    ; #############
+    ; DOGPILE CACHE
+    ; #############
+
+    ; Default cache dir for caches. Putting this into a ramdisk can boost performance.
+    ; eg. /tmpfs/data_ramdisk, however this directory might require large amount of space
+    cache_dir = %(here)s/data
+
+    ; **********************************************************
+    ; `repo_object` cache with redis backend
+    ; recommended for larger instance, or for better performance
+    ; **********************************************************
+
+    ; `repo_object` cache settings for vcs methods for repositories
+    rc_cache.repo_object.backend = dogpile.cache.rc.redis_msgpack
 
+    ; cache auto-expires after N seconds
+    ; Examples: 86400 (1Day), 604800 (7Days), 1209600 (14Days), 2592000 (30days), 7776000 (90Days)
+    rc_cache.repo_object.expiration_time = 2592000
 
-    ################################
-    ### LOGGING CONFIGURATION   ####
-    ################################
+    ; redis_expiration_time needs to be greater then expiration_time
+    rc_cache.repo_object.arguments.redis_expiration_time = 3592000
+
+    rc_cache.repo_object.arguments.host = localhost
+    rc_cache.repo_object.arguments.port = 6379
+    rc_cache.repo_object.arguments.db = 5
+    rc_cache.repo_object.arguments.socket_timeout = 30
+    ; more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
+    rc_cache.repo_object.arguments.distributed_lock = true
+
+    ; #####################
+    ; LOGGING CONFIGURATION
+    ; #####################
     [loggers]
-    keys = root, vcsserver, beaker
+    keys = root, vcsserver
 
     [handlers]
     keys = console
@@ -280,9 +341,9 b' For a more detailed explanation of the l'
     [formatters]
     keys = generic
 
-    #############
-    ## LOGGERS ##
-    #############
+    ; #######
+    ; LOGGERS
+    ; #######
     [logger_root]
     level = NOTSET
     handlers = console
@@ -293,29 +354,23 b' For a more detailed explanation of the l'
     qualname = vcsserver
     propagate = 1
 
-    [logger_beaker]
-    level = DEBUG
-    handlers =
-    qualname = beaker
-    propagate = 1
 
-
-    ##############
-    ## HANDLERS ##
-    ##############
+    ; ########
+    ; HANDLERS
+    ; ########
 
     [handler_console]
     class = StreamHandler
     args = (sys.stderr,)
-    level = DEBUG
+    level = INFO
     formatter = generic
 
-    ################
-    ## FORMATTERS ##
-    ################
+    ; ##########
+    ; FORMATTERS
+    ; ##########
 
     [formatter_generic]
-    format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
+    format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
     datefmt = %Y-%m-%d %H:%M:%S
 
 

docs/api/methods/pull-request-methods.rst

@@ -39,7 +39,7 b' close_pull_request'
 comment_pull_request 
 --------------------
 
-.. py:function:: comment_pull_request(apiuser, pullrequestid, repoid=<Optional:None>, message=<Optional:None>, commit_id=<Optional:None>, status=<Optional:None>, comment_type=<Optional:u'note'>, resolves_comment_id=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>)
+.. py:function:: comment_pull_request(apiuser, pullrequestid, repoid=<Optional:None>, message=<Optional:None>, commit_id=<Optional:None>, status=<Optional:None>, comment_type=<Optional:u'note'>, resolves_comment_id=<Optional:None>, extra_recipients=<Optional:[]>, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Comment on the pull request specified with the `pullrequestid`,
    in the |repo| specified by the `repoid`, and optionally change the
@@ -63,6 +63,11 b' comment_pull_request'
    :type status: str
    :param comment_type: Comment type, one of: 'note', 'todo'
    :type comment_type: Optional(str), default: 'note'
+   :param resolves_comment_id: id of comment which this one will resolve
+   :type resolves_comment_id: Optional(int)
+   :param extra_recipients: list of user ids or usernames to add
+       notifications for this comment. Acts like a CC for notification
+   :type extra_recipients: Optional(list)
    :param userid: Comment on the pull request as this user
    :type userid: Optional(str or int)
 
@@ -126,7 +131,7 b' create_pull_request'
 get_pull_request 
 ----------------
 
-.. py:function:: get_pull_request(apiuser, pullrequestid, repoid=<Optional:None>)
+.. py:function:: get_pull_request(apiuser, pullrequestid, repoid=<Optional:None>, merge_state=<Optional:False>)
 
    Get a pull request based on the given ID.
 
@@ -137,6 +142,9 b' get_pull_request'
    :type repoid: str or int
    :param pullrequestid: ID of the requested pull request.
    :type pullrequestid: int
+   :param merge_state: Optional calculate merge state for each repository.
+       This could result in longer time to fetch the data
+   :type merge_state: bool
 
    Example output:
 
@@ -250,7 +258,7 b' get_pull_request_comments'
 get_pull_requests 
 -----------------
 
-.. py:function:: get_pull_requests(apiuser, repoid, status=<Optional:'new'>, merge_state=<Optional:True>)
+.. py:function:: get_pull_requests(apiuser, repoid, status=<Optional:'new'>, merge_state=<Optional:False>)
 
    Get all pull requests from the repository specified in `repoid`.
 

docs/api/methods/repo-methods.rst

@@ -28,7 +28,7 b' add_field_to_repo'
 comment_commit 
 --------------
 
-.. py:function:: comment_commit(apiuser, repoid, commit_id, message, status=<Optional:None>, comment_type=<Optional:u'note'>, resolves_comment_id=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>)
+.. py:function:: comment_commit(apiuser, repoid, commit_id, message, status=<Optional:None>, comment_type=<Optional:u'note'>, resolves_comment_id=<Optional:None>, extra_recipients=<Optional:[]>, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Set a commit comment, and optionally change the status of the commit.
 
@@ -45,6 +45,11 b' comment_commit'
    :type status: str
    :param comment_type: Comment type, one of: 'note', 'todo'
    :type comment_type: Optional(str), default: 'note'
+   :param resolves_comment_id: id of comment which this one will resolve
+   :type resolves_comment_id: Optional(int)
+   :param extra_recipients: list of user ids or usernames to add
+       notifications for this comment. Acts like a CC for notification
+   :type extra_recipients: Optional(list)
    :param userid: Set the user name of the comment creator.
    :type userid: Optional(str or int)
 
@@ -66,7 +71,7 b' comment_commit'
 create_repo 
 -----------
 
-.. py:function:: create_repo(apiuser, repo_name, repo_type, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, push_uri=<Optional:None>, landing_rev=<Optional:'rev:tip'>, enable_statistics=<Optional:False>, enable_locking=<Optional:False>, enable_downloads=<Optional:False>, copy_permissions=<Optional:False>)
+.. py:function:: create_repo(apiuser, repo_name, repo_type, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, push_uri=<Optional:None>, landing_rev=<Optional:None>, enable_statistics=<Optional:False>, enable_locking=<Optional:False>, enable_downloads=<Optional:False>, copy_permissions=<Optional:False>)
 
    Creates a repository.
 
@@ -97,7 +102,7 b' create_repo'
    :type clone_uri: str
    :param push_uri: set push_uri
    :type push_uri: str
-   :param landing_rev: <rev_type>:<rev>
+   :param landing_rev: <rev_type>:<rev>, e.g branch:default, book:dev, rev:abcd
    :type landing_rev: str
    :param enable_locking:
    :type enable_locking: bool
@@ -169,7 +174,7 b' delete_repo'
 fork_repo 
 ---------
 
-.. py:function:: fork_repo(apiuser, repoid, fork_name, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, landing_rev=<Optional:'rev:tip'>, copy_permissions=<Optional:False>)
+.. py:function:: fork_repo(apiuser, repoid, fork_name, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, landing_rev=<Optional:None>, copy_permissions=<Optional:False>)
 
    Creates a fork of the specified |repo|.
 
@@ -198,7 +203,7 b' fork_repo'
    :type copy_permissions: bool
    :param private: Make the fork private. The default is False.
    :type private: bool
-   :param landing_rev: Set the landing revision. The default is tip.
+   :param landing_rev: Set the landing revision. E.g branch:default, book:dev, rev:abcd
 
    Example output:
 
@@ -1085,7 +1090,7 b' strip'
 update_repo 
 -----------
 
-.. py:function:: update_repo(apiuser, repoid, repo_name=<Optional:None>, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, push_uri=<Optional:None>, landing_rev=<Optional:'rev:tip'>, fork_of=<Optional:None>, enable_statistics=<Optional:False>, enable_locking=<Optional:False>, enable_downloads=<Optional:False>, fields=<Optional:''>)
+.. py:function:: update_repo(apiuser, repoid, repo_name=<Optional:None>, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, push_uri=<Optional:None>, landing_rev=<Optional:None>, fork_of=<Optional:None>, enable_statistics=<Optional:False>, enable_locking=<Optional:False>, enable_downloads=<Optional:False>, fields=<Optional:''>)
 
    Updates a repository with the given information.
 
@@ -1117,7 +1122,7 b' update_repo'
    :type private: bool
    :param clone_uri: Update the |repo| clone URI.
    :type clone_uri: str
-   :param landing_rev: Set the |repo| landing revision. Default is ``rev:tip``.
+   :param landing_rev: Set the |repo| landing revision. e.g branch:default, book:dev, rev:abcd
    :type landing_rev: str
    :param enable_statistics: Enable statistics on the |repo|, (True | False).
    :type enable_statistics: bool

docs/api/methods/search-methods.rst

@@ -6,7 +6,7 b' search methods'
 search 
 ------
 
-.. py:function:: search(apiuser, search_query, search_type, page_limit=<Optional:10>, page=<Optional:1>, search_sort=<Optional:'newfirst'>, repo_name=<Optional:None>, repo_group_name=<Optional:None>)
+.. py:function:: search(apiuser, search_query, search_type, page_limit=<Optional:10>, page=<Optional:1>, search_sort=<Optional:'desc:date'>, repo_name=<Optional:None>, repo_group_name=<Optional:None>)
 
    Fetch Full Text Search results using API.
 
@@ -23,9 +23,15 b' search'
    :type page_limit: Optional(int)
    :param page: Page number. Default first page.
    :type page: Optional(int)
-   :param search_sort: Search sort order. Default newfirst. The following are valid options:
-       * newfirst
-       * oldfirst
+   :param search_sort: Search sort order.Must start with asc: or desc: Default desc:date.
+       The following are valid options:
+       * asc|desc:message.raw
+       * asc|desc:date
+       * asc|desc:author.email.raw
+       * asc|desc:message.raw
+       * newfirst (old legacy equal to desc:date)
+       * oldfirst (old legacy equal to asc:date)
+
    :type search_sort: Optional(str)
    :param repo_name: Filter by one repo. Default is all.
    :type repo_name: Optional(str)

docs/api/methods/store-methods.rst

@@ -6,7 +6,7 b' store methods'
 file_store_add (EE only)
 ------------------------
 
-.. py:function:: file_store_add(apiuser, filename, content)
+.. py:function:: file_store_add(apiuser, filename, content, description=<Optional:''>)
 
    Upload API for the file_store
 
@@ -19,6 +19,8 b' file_store_add (EE only)'
    :type apiuser: AuthUser
    :param filename: name of the file uploaded
    :type filename: str
+   :param description: Optional description for added file
+   :type description: str
    :param content: base64 encoded content of the uploaded file
    :type content: str
 
@@ -35,3 +37,148 b' file_store_add (EE only)'
      error :  null
 
 
+file_store_add_with_acl (EE only)
+---------------------------------
+
+.. py:function:: file_store_add_with_acl(apiuser, filename, content, description=<Optional:''>, scope_user_id=<Optional:None>, scope_repo_id=<Optional:None>, scope_repo_group_id=<Optional:None>)
+
+   Upload API for the file_store
+
+   Example usage from CLI::
+       rhodecode-api --instance-name=enterprise-1 upload_file "{"content": "$(cat image.jpg | base64)", "filename":"image.jpg", "scope_repo_id":101}"
+
+   This command takes the following options:
+
+   :param apiuser: This is filled automatically from the |authtoken|.
+   :type apiuser: AuthUser
+   :param filename: name of the file uploaded
+   :type filename: str
+   :param description: Optional description for added file
+   :type description: str
+   :param content: base64 encoded content of the uploaded file
+   :type content: str
+
+   :param scope_user_id: Optionally bind this file to user.
+       This will check ACL in such way only this user can access the file.
+   :type scope_user_id: int
+   :param scope_repo_id: Optionally bind this file to repository.
+       This will check ACL in such way only user with proper access to such
+       repository can access the file.
+   :type scope_repo_id: int
+   :param scope_repo_group_id:  Optionally bind this file to repository group.
+       This will check ACL in such way only user with proper access to such
+       repository group can access the file.
+   :type scope_repo_group_id: int
+
+   Example output:
+
+   .. code-block:: bash
+
+     id : <id_given_in_input>
+     result: {
+       "access_path": "/_file_store/download/84d156f7-8323-4ad3-9fce-4a8e88e1deaf-0.jpg",
+       "access_path_fqn": "http://server.domain.com/_file_store/download/84d156f7-8323-4ad3-9fce-4a8e88e1deaf-0.jpg",
+       "store_fid": "84d156f7-8323-4ad3-9fce-4a8e88e1deaf-0.jpg"
+     }
+     error :  null
+
+
+file_store_get_info (EE only)
+-----------------------------
+
+.. py:function:: file_store_get_info(apiuser, store_fid)
+
+   Get artifact data.
+
+   Example output:
+
+   .. code-block:: bash
+
+     id : <id_given_in_input>
+     result: {
+         "artifact": {
+           "access_path_fqn": "https://rhodecode.example.com/_file_store/download/0-031c2aa0-0d56-49a7-9ba3-b570bdd342ab.jpg",
+           "created_on": "2019-10-15T16:25:35.491",
+           "description": "my upload",
+           "downloaded_times": 1,
+           "file_uid": "0-031c2aa0-0d56-49a7-9ba3-b570bdd342ab.jpg",
+           "filename": "example.jpg",
+           "filename_org": "0-031c2aa0-0d56-49a7-9ba3-b570bdd342ab.jpg",
+           "hidden": false,
+           "metadata": [
+             {
+               "artifact": "0-031c2aa0-0d56-49a7-9ba3-b570bdd342ab.jpg",
+               "key": "yellow",
+               "section": "tags",
+               "value": "bar"
+             }
+           ],
+           "sha256": "818dff0f44574dfb6814d38e6bf3c60c5943d1d13653398ecddaedf2f6a5b04d",
+           "size": 18599,
+           "uploaded_by": {
+             "email": "[e-mail]",
+             "emails": [
+               "[e-mail]"
+             ],
+             "firstname": "Admin",
+             "lastname": "LastName",
+             "user_id": 2,
+             "username": "admin"
+           }
+         }
+     }
+     error :  null
+
+
+file_store_add_metadata (EE only)
+---------------------------------
+
+.. py:function:: file_store_add_metadata(apiuser, store_fid, section, key, value, value_type=<Optional:'unicode'>)
+
+   Add metadata into artifact. The metadata consist of section, key, value. eg.
+   section='tags', 'key'='tag_name', value='1'
+
+   :param apiuser: This is filled automatically from the |authtoken|.
+   :type apiuser: AuthUser
+
+   :param store_fid: file uid, e.g 0-d054cb71-91ab-44e2-9e4b-23fe14b4d74a.mp4
+   :type store_fid: str
+
+   :param section: Section name to add metadata
+   :type section: str
+
+   :param key: Key to add as metadata
+   :type key: str
+
+   :param value: Value to add as metadata
+   :type value: str
+
+   :param value_type: Optional type, default is 'unicode' other types are:
+       int, list, bool, unicode, str
+
+   :type value_type: str
+
+   Example output:
+
+   .. code-block:: bash
+
+     id : <id_given_in_input>
+     result: {
+           "metadata": [
+             {
+               "artifact": "0-d054cb71-91ab-44e2-9e4b-23fe14b4d74a.mp4",
+               "key": "secret",
+               "section": "tags",
+               "value": "1"
+             },
+             {
+               "artifact": "0-d054cb71-91ab-44e2-9e4b-23fe14b4d74a.mp4",
+               "key": "video",
+               "section": "tags",
+               "value": "1"
+             }
+           ]
+     }
+     error :  null
+
+

docs/api/methods/user-group-methods.rst

@@ -72,7 +72,9 b' create_user_group'
    :param active: Set this group as active.
    :type active: Optional(``True`` | ``False``)
    :param sync: Set enabled or disabled the automatically sync from
-       external authentication types like ldap.
+       external authentication types like ldap. If User Group will be named like
+       one from e.g ldap and sync flag is enabled members will be synced automatically.
+       Sync type when enabled via API is set to `manual_api`
    :type sync: Optional(``True`` | ``False``)
 
    Example output:
@@ -391,7 +393,9 b' update_user_group'
    :param active: Set the group as active.
    :type active: Optional(``True`` | ``False``)
    :param sync: Set enabled or disabled the automatically sync from
-       external authentication types like ldap.
+       external authentication types like ldap. If User Group will be named like
+       one from e.g ldap and sync flag is enabled members will be synced automatically.
+       Sync type when enabled via API is set to `manual_api`
    :type sync: Optional(``True`` | ``False``)
 
    Example output:

docs/api/methods/user-methods.rst

@@ -6,7 +6,7 b' user methods'
 create_user 
 -----------
 
-.. py:function:: create_user(apiuser, username, email, password=<Optional:''>, firstname=<Optional:''>, lastname=<Optional:''>, active=<Optional:True>, admin=<Optional:False>, extern_name=<Optional:'rhodecode'>, extern_type=<Optional:'rhodecode'>, force_password_change=<Optional:False>, create_personal_repo_group=<Optional:None>)
+.. py:function:: create_user(apiuser, username, email, password=<Optional:''>, firstname=<Optional:''>, lastname=<Optional:''>, description=<Optional:''>, active=<Optional:True>, admin=<Optional:False>, extern_name=<Optional:'rhodecode'>, extern_type=<Optional:'rhodecode'>, force_password_change=<Optional:False>, create_personal_repo_group=<Optional:None>)
 
    Creates a new user and returns the new user object.
 
@@ -27,6 +27,8 b' create_user'
    :type firstname: Optional(str)
    :param lastname: Set the new user surname.
    :type lastname: Optional(str)
+   :param description: Set user description, or short bio. Metatags are allowed.
+   :type description: Optional(str)
    :param active: Set the user as active.
    :type active: Optional(``True`` | ``False``)
    :param admin: Give the new user admin rights.
@@ -155,6 +157,7 b' get_user'
            "extern_name": "rhodecode",
            "extern_type": "rhodecode",
            "firstname": "username",
+           "description": "user description",
            "ip_addresses": [],
            "language": null,
            "last_login": "Timestamp",
@@ -268,7 +271,7 b' get_users'
 update_user 
 -----------
 
-.. py:function:: update_user(apiuser, userid, username=<Optional:None>, email=<Optional:None>, password=<Optional:None>, firstname=<Optional:None>, lastname=<Optional:None>, active=<Optional:None>, admin=<Optional:None>, extern_type=<Optional:None>, extern_name=<Optional:None>)
+.. py:function:: update_user(apiuser, userid, username=<Optional:None>, email=<Optional:None>, password=<Optional:None>, firstname=<Optional:None>, lastname=<Optional:None>, description=<Optional:None>, active=<Optional:None>, admin=<Optional:None>, extern_type=<Optional:None>, extern_name=<Optional:None>)
 
    Updates the details for the specified user, if that user exists.
 
@@ -291,6 +294,8 b' update_user'
    :type firstname: Optional(str)
    :param lastname: Set the new surname.
    :type lastname: Optional(str)
+   :param description: Set user description, or short bio. Metatags are allowed.
+   :type description: Optional(str)
    :param active: Set the new user as active.
    :type active: Optional(``True`` | ``False``)
    :param admin: Give the user admin rights.

docs/contributing/dev-setup.rst

@@ -19,7 +19,7 b' Setup Nix Package Manager'
 
 To install the Nix Package Manager, please run::
 
-   $ curl https://nixos.org/nix/install | sh
+   $ curl https://nixos.org/releases/nix/nix-2.0.4/install | sh
 
 or go to https://nixos.org/nix/ and follow the installation instructions.
 Once this is correctly set up on your system, you should be able to use the

docs/index.rst

@@ -79,6 +79,12 b' and commit files and |repos| while manag'
    contributing/contributing
 
 .. toctree::
+   :maxdepth: 2
+   :caption: RhodeCode Control Documentation
+
+   RhodeCode Installer <https://docs.rhodecode.com/RhodeCode-Control/>
+
+.. toctree::
    :maxdepth: 1
    :caption: About
 

docs/install/configure-celery.rst

@@ -11,16 +11,20 b' and import repositories in async way. It'
 repository sync in scheduler.
 
 If you decide to use Celery you also need a working message queue.
-The recommended and fully supported message broker is rabbitmq_.
+There are two fully supported message brokers is rabbitmq_ and redis_ (recommended).
+
+Since release 4.18.X we recommend using redis_ as a backend since it's generally
+easier to work with, and results in simpler stack as redis is generally recommended
+for caching purposes.
 
 
 In order to install and configure Celery, follow these steps:
 
-1. Install RabbitMQ, see the documentation on the Celery website for
-   `rabbitmq installation`_, or `rabbitmq website installation`_
+1. Install RabbitMQ or Redis for a message queue, see the documentation on the Celery website for
+   `redis installation`_ or `rabbitmq installation`_
 
 
-1a. As en example configuration after installation, you can run::
+1a. If you choose RabbitMQ example configuration after installation would look like that::
 
    sudo rabbitmqctl add_user rcuser secret_password
    sudo rabbitmqctl add_vhost rhodevhost
@@ -45,6 +49,10 b' 3. Configure Celery in the'
    Set the broker_url as minimal settings required to enable operation.
    If used our example data from pt 1a, here is how the broker url should look like::
 
+        # for Redis
+        celery.broker_url = redis://localhost:6379/8
+
+        # for RabbitMQ
         celery.broker_url = amqp://rcuser:secret_password@localhost:5672/rhodevhost
 
    Full configuration example is below:
@@ -57,7 +65,7 b' 3. Configure Celery in the'
         ####################################
 
         use_celery = true
-        celery.broker_url = amqp://rcuser:secret@localhost:5672/rhodevhost
+        celery.broker_url = redis://localhost:6379/8
 
         # maximum tasks to execute before worker restart
         celery.max_tasks_per_child = 100
@@ -69,6 +77,8 b' 3. Configure Celery in the'
 .. _python: http://www.python.org/
 .. _mercurial: http://mercurial.selenic.com/
 .. _celery: http://celeryproject.org/
+.. _redis: http://redis.io
+.. _redis installation: https://redis.io/topics/quickstart
 .. _rabbitmq: http://www.rabbitmq.com/
 .. _rabbitmq installation: http://docs.celeryproject.org/en/latest/getting-started/brokers/rabbitmq.html
 .. _rabbitmq website installation: http://www.rabbitmq.com/download.html

docs/install/database-string.rst

@@ -35,4 +35,3 b' 2. When you open the file, find the data'
     # see sqlalchemy docs for other advanced settings
     sqlalchemy.db1.echo = false
     sqlalchemy.db1.pool_recycle = 3600
-    sqlalchemy.db1.convert_unicode = true

docs/issue-trackers/issue-trackers.rst

@@ -38,44 +38,39 b' default one. See the instructions in :re'
 
 .. _issue-tr-eg-ref:
 
+
 Jira Integration
 ----------------
 
-* Regex = ``(?:^#|\s#)(\w+-\d+)``
-* URL = ``https://myissueserver.com/browse/${id}``
-* Issue Prefix = ``#``
+Please check examples in the view for configuration the issue trackers.
+
 
 Confluence (Wiki)
 -----------------
 
-* Regex = ``(?:conf-)([A-Z0-9]+)``
-* URL = ``https://example.atlassian.net/display/wiki/${id}/${repo_name}``
-* issue prefix = ``CONF-``
+Please check examples in the view for configuration the issue trackers.
+
 
 Redmine Integration
 -------------------
 
-* Regex = ``(issue-+\d+)``
-* URL = ``https://myissueserver.com/redmine/issue/${id}``
-* Issue Prefix = ``issue-``
+Please check examples in the view for configuration the issue trackers.
+
 
-Redmine (wiki)
---------------
+Redmine wiki Integration
+------------------------
 
-* Regex = ``(?:wiki-)([a-zA-Z0-9]+)``
-* URL = ``https://example.com/redmine/projects/wiki/${repo_name}``
-* Issue prefix = ``Issue-``
+Please check examples in the view for configuration the issue trackers.
+
 
 Pivotal Tracker
 ---------------
 
-* Regex = ``(?:pivot-)(?<project_id>\d+)-(?<story>\d+)``
-* URL = ``https://www.pivotaltracker.com/s/projects/${project_id}/stories/${story}``
-* Issue prefix = ``Piv-``
+Please check examples in the view for configuration the issue trackers.
+
 
 Trello
 ------
 
-* Regex = ``(?:trello-)(?<card_id>[a-zA-Z0-9]+)``
-* URL = ``https://trello.com/example.com/${card_id}``
-* Issue prefix = ``Trello-``
+Please check examples in the view for configuration the issue trackers.
+

docs/release-notes/release-notes-4.17.0.rst

@@ -62,7 +62,7 b' Performance'
 Fixes
 ^^^^^
 
-- hHooks: fixed more unicode problems with new pull-request link generator.
+- Hooks: fixed more unicode problems with new pull-request link generator.
 - Mercurial: fix ssh-server support for mercurial custom options.
 - Pull requests: updated metadata information for failed merges with multiple heads.
 - Pull requests: calculate ancestor in the same way as creation mode.

docs/release-notes/release-notes.rst

@@ -9,6 +9,7 b' Release Notes'
 .. toctree::
    :maxdepth: 1
 
+   release-notes-4.18.0.rst
    release-notes-4.17.4.rst
    release-notes-4.17.3.rst
    release-notes-4.17.2.rst

docs/tools/install-tools.rst

@@ -57,9 +57,12 b' To install |RCT|, use the following step'
 
 1. Set up a ``virtualenv`` on your local machine, see virtualenv_ instructions
    here.
-2. Install |RCT| using pip. Full url with token is available at https://rhodecode.com/u/#rhodecode-tools
-   ``pip install -I https://dls.rhodecode.com/dls/<token>/rhodecode-tools/latest``
+2. Install |RCT| using pip. All downloadable versions of |RCT| are available at:
+   `https://code.rhodecode.com/rhodecode-tools-ce/artifacts`
 
+   Example installation::
+
+    pip install -I https://code.rhodecode.com/rhodecode-tools-ce/artifacts/download/0-10ac93f4-bb7d-4b97-baea-68110743dd5a.tar.gz
 
 Once |RCT| is installed using these steps there are a few extra
 configuration changes you can make. These are explained in more detail in the

docs/usage/keyboard-shortcuts.rst

@@ -56,4 +56,4 b' Repository Shortcuts'
         Go to the repository settings page.
 
     \--:kbd:`gO`
-        Go to the repository permissions settings.
+        Go to the repository access permissions settings.

grunt_config.json

@@ -99,6 +99,12 b''
       "nonull": true
     }
   },
+  "uglify": {
+    "dist": {
+      "src": "<%= dirs.js.dest %>/scripts.js",
+      "dest": "<%= dirs.js.dest %>/scripts.min.js"
+    }
+  },
   "less": {
     "development": {
       "options": {

package.json

@@ -22,6 +22,7 b''
     "grunt-contrib-less": "^1.1.0",
     "grunt-contrib-watch": "^0.6.1",
     "grunt-webpack": "^3.1.3",
+    "grunt-contrib-uglify": "^4.0.1",
     "jquery": "1.11.3",
     "mark.js": "8.11.1",
     "jshint": "^2.9.1-rc3",

pkgs/node-packages.nix

@@ -211,13 +211,13 b' let'
         sha512 = "yiUk09opTEnE1lK+tb501ENb+yQBi4p++Ep0eGJAHesVYKVMPNgPphVKkIizkDaU+n0SE+zXfTsRbYyOMDYXSg==";
       };
     };
-    "@polymer/polymer-3.2.0" = {
+    "@polymer/polymer-3.3.0" = {
       name = "_at_polymer_slash_polymer";
       packageName = "@polymer/polymer";
-      version = "3.2.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/@polymer/polymer/-/polymer-3.2.0.tgz";
-        sha512 = "L6uV1oM6T6xbwbVx6t3biG5T2VSSB03LxnIrUd9M2pr6RkHVPFHJ37pC5MUwBAEhkGFJif7eks7fdMMSGZTeEQ==";
+      version = "3.3.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/@polymer/polymer/-/polymer-3.3.0.tgz";
+        sha512 = "rij7suomS7DxdBamnwr/Xa0V5hpypf7I9oYKseF2FWz5Xh2a3wJNpVjgJy1adXVCxqIyPhghsrthnfCt7EblsQ==";
       };
     };
     "@types/clone-0.1.30" = {
@@ -229,13 +229,13 b' let'
         sha1 = "e7365648c1b42136a59c7d5040637b3b5c83b614";
       };
     };
-    "@types/node-6.14.6" = {
+    "@types/node-6.14.9" = {
       name = "_at_types_slash_node";
       packageName = "@types/node";
-      version = "6.14.6";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/@types/node/-/node-6.14.6.tgz";
-        sha512 = "rFs9zCFtSHuseiNXxYxFlun8ibu+jtZPgRM+2ILCmeLiGeGLiIGxuOzD+cNyHegI1GD+da3R/cIbs9+xCLp13w==";
+      version = "6.14.9";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/@types/node/-/node-6.14.9.tgz";
+        sha512 = "leP/gxHunuazPdZaCvsCefPQxinqUDsCxCR5xaDUrY2MkYxQRFZZwU5e7GojyYsGB7QVtCi7iVEl/hoFXQYc+w==";
       };
     };
     "@types/parse5-2.2.34" = {
@@ -409,22 +409,22 b' let'
         sha512 = "mJ3QKWtCchL1vhU/kZlJnLPuQZnlDOdZsyP0bbLWPGdYsQDnSBvyTLhzwBA3QAMlzEL9V4JHygEmK6/OTEyytA==";
       };
     };
-    "@webcomponents/shadycss-1.9.1" = {
+    "@webcomponents/shadycss-1.9.2" = {
       name = "_at_webcomponents_slash_shadycss";
       packageName = "@webcomponents/shadycss";
-      version = "1.9.1";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/@webcomponents/shadycss/-/shadycss-1.9.1.tgz";
-        sha512 = "IaZOnWOKXHghqk/WfPNDRIgDBi3RsVPY2IFAw6tYiL9UBGvQRy5R6uC+Fk7qTZsReTJ0xh5MTT8yAcb3MUR4mQ==";
-      };
-    };
-    "@webcomponents/webcomponentsjs-2.2.10" = {
+      version = "1.9.2";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/@webcomponents/shadycss/-/shadycss-1.9.2.tgz";
+        sha512 = "GsD7RpDVrVdgC6e+D8zQia8RGNmEGQ9/qotnVPQYPrIXhGS5xSt6ZED9YmuHz3HbLqY+E54tE1EK3tjLzSCGrw==";
+      };
+    };
+    "@webcomponents/webcomponentsjs-2.3.0" = {
       name = "_at_webcomponents_slash_webcomponentsjs";
       packageName = "@webcomponents/webcomponentsjs";
-      version = "2.2.10";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/@webcomponents/webcomponentsjs/-/webcomponentsjs-2.2.10.tgz";
-        sha512 = "5dzhUhP+h0qMiK0IWb7VNb0OGBoXO3AuI6Qi8t9PoKT50s5L1jv0xnwnLq+cFgPuTB8FLTNP8xIDmyoOsKBy9Q==";
+      version = "2.3.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/@webcomponents/webcomponentsjs/-/webcomponentsjs-2.3.0.tgz";
+        sha512 = "sR6FOrNnnncRuoJDqq9QxtRsJMbIvASw4vnJwIYKVlKO3AMc+NAr/bIQNnUiTTE9pBDTJkFpVaUdjJaRdsjmyA==";
       };
     };
     "@xtuc/ieee754-1.2.0" = {
@@ -499,22 +499,22 b' let'
         sha1 = "82ffb02b29e662ae53bdc20af15947706739c536";
       };
     };
-    "ajv-6.10.0" = {
+    "ajv-6.10.2" = {
       name = "ajv";
       packageName = "ajv";
-      version = "6.10.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/ajv/-/ajv-6.10.0.tgz";
-        sha512 = "nffhOpkymDECQyR0mnsUtoCE8RlX38G0rYP+wgLWFyZuUyuuojSSvi/+euOiQBIn63whYwYVIIH1TvE3tu4OEg==";
-      };
-    };
-    "ajv-keywords-3.4.0" = {
+      version = "6.10.2";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/ajv/-/ajv-6.10.2.tgz";
+        sha512 = "TXtUUEYHuaTEbLZWIKUr5pmBuhDLy+8KYtPYdcV8qC+pOZL+NKqYwvWSRrVXHn+ZmRRAu8vJTAznH7Oag6RVRw==";
+      };
+    };
+    "ajv-keywords-3.4.1" = {
       name = "ajv-keywords";
       packageName = "ajv-keywords";
-      version = "3.4.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/ajv-keywords/-/ajv-keywords-3.4.0.tgz";
-        sha512 = "aUjdRFISbuFOl0EIZc+9e4FfZp0bDZgAdOOf30bJmw8VM9v84SHyVyxDfbWxpGYbdZD/9XoKxfHVNmxPkhwyGw==";
+      version = "3.4.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/ajv-keywords/-/ajv-keywords-3.4.1.tgz";
+        sha512 = "RO1ibKvd27e6FEShVFfPALuHI3WjSVNeK5FIsmme/LYRNxjKuNj+Dt7bucLa6NdSv3JcVTyMlm9kGR84z1XpaQ==";
       };
     };
     "align-text-0.1.4" = {
@@ -806,13 +806,13 b' let'
         sha1 = "b6bbe0b0674b9d719708ca38de8c237cb526c3d1";
       };
     };
-    "async-2.6.2" = {
+    "async-2.6.3" = {
       name = "async";
       packageName = "async";
-      version = "2.6.2";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/async/-/async-2.6.2.tgz";
-        sha512 = "H1qVYh1MYhEEFLsP97cVKqCGo7KfCyTt6uEWqsTBr9SO84oK9Uwbyd/yCW+6rKJLHksBNUVWZDAjfS+Ccx0Bbg==";
+      version = "2.6.3";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/async/-/async-2.6.3.tgz";
+        sha512 = "zflvls11DCy+dQWzTW2dzuilv8Z5X/pjfmZOWba6TNIVDm+2UDaJmXSOXlasHKfNBs8oo3M0aT50fDEWfKZjXg==";
       };
     };
     "async-each-1.0.3" = {
@@ -1400,13 +1400,13 b' let'
         sha512 = "5T6P4xPgpp0YDFvSWwEZ4NoE3aM4QBQXDzmVbraCkFj8zHM+mba8SyqB5DbZWyR7mYHo6Y7BdQo3MoA4m0TeQg==";
       };
     };
-    "base64-js-1.3.0" = {
+    "base64-js-1.3.1" = {
       name = "base64-js";
       packageName = "base64-js";
-      version = "1.3.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/base64-js/-/base64-js-1.3.0.tgz";
-        sha512 = "ccav/yGvoa80BQDljCxsmmQ3Xvx60/UpBIij5QN21W3wBi/hhIC9OoO+KLpu9IJTS9j4DRVJ3aDDF9cMSoa2lw==";
+      version = "1.3.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/base64-js/-/base64-js-1.3.1.tgz";
+        sha512 = "mLQ4i2QO1ytvGWFWmcngKO//JXAQueZvwEKtjgQFM4jIK0kU+ytMfplL8j+n5mspOfjHwoAg+9yhb7BwAHm36g==";
       };
     };
     "bcrypt-pbkdf-1.0.2" = {
@@ -1445,13 +1445,13 b' let'
         sha512 = "Un7MIEDdUC5gNpcGDV97op1Ywk748MpHcFTHoYs6qnj1Z3j7I53VG3nwZhKzoBZmbdRNnb6WRdFlwl7tSDuZGw==";
       };
     };
-    "bluebird-3.5.4" = {
+    "bluebird-3.7.1" = {
       name = "bluebird";
       packageName = "bluebird";
-      version = "3.5.4";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/bluebird/-/bluebird-3.5.4.tgz";
-        sha512 = "FG+nFEZChJrbQ9tIccIfZJBz3J7mLrAhxakAbnrJWn8d7aKOC+LWifa0G+p4ZqKp4y13T7juYvdhq9NzKdsrjw==";
+      version = "3.7.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/bluebird/-/bluebird-3.7.1.tgz";
+        sha512 = "DdmyoGCleJnkbp3nkbxTLJ18rjDsE4yCggEwKNXkeV123sPNfOCYeDoeuOY+F2FrSjO1YXcTU+dsy96KMy+gcg==";
       };
     };
     "bn.js-4.11.8" = {
@@ -1670,22 +1670,22 b' let'
         sha1 = "b534e7c734c4f81ec5fbe8aca2ad24354b962c6c";
       };
     };
-    "caniuse-db-1.0.30000967" = {
+    "caniuse-db-1.0.30001006" = {
       name = "caniuse-db";
       packageName = "caniuse-db";
-      version = "1.0.30000967";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/caniuse-db/-/caniuse-db-1.0.30000967.tgz";
-        sha512 = "70gk6cLSD5rItxnZ7WUxyCpM9LAjEb1tVzlENQfXQXZS/IiGnfAC6u32G5cZFlDBKjNPBIta/QSx5CZLZepxRA==";
-      };
-    };
-    "caniuse-lite-1.0.30000967" = {
+      version = "1.0.30001006";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/caniuse-db/-/caniuse-db-1.0.30001006.tgz";
+        sha512 = "Xn25grc0GXATFnnEX+KP3IwEv6ZdHs4CALyLKvK8pBeeBe+hSpqy3/GyKBgEp4hn6o+bI+GNeNeQBf9PBOK0EQ==";
+      };
+    };
+    "caniuse-lite-1.0.30001006" = {
       name = "caniuse-lite";
       packageName = "caniuse-lite";
-      version = "1.0.30000967";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30000967.tgz";
-        sha512 = "rUBIbap+VJfxTzrM4akJ00lkvVb5/n5v3EGXfWzSH5zT8aJmGzjA8HWhJ4U6kCpzxozUSnB+yvAYDRPY6mRpgQ==";
+      version = "1.0.30001006";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001006.tgz";
+        sha512 = "MXnUVX27aGs/QINz+QG1sWSLDr3P1A3Hq5EUWoIt0T7K24DuvMxZEnh3Y5aHlJW6Bz2aApJdSewdYLd8zQnUuw==";
       };
     };
     "caseless-0.12.0" = {
@@ -1733,31 +1733,31 b' let'
         sha512 = "Mti+f9lpJNcwF4tWV8/OrTTtF1gZi+f8FqlyAdouralcFWFQWF2+NgCHShjkCb+IFBLq9buZwE1xckQU4peSuQ==";
       };
     };
-    "chokidar-2.1.5" = {
+    "chokidar-2.1.8" = {
       name = "chokidar";
       packageName = "chokidar";
-      version = "2.1.5";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/chokidar/-/chokidar-2.1.5.tgz";
-        sha512 = "i0TprVWp+Kj4WRPtInjexJ8Q+BqTE909VpH8xVhXrJkoc5QC8VO9TryGOqTr+2hljzc1sC62t22h5tZePodM/A==";
-      };
-    };
-    "chownr-1.1.1" = {
+      version = "2.1.8";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/chokidar/-/chokidar-2.1.8.tgz";
+        sha512 = "ZmZUazfOzf0Nve7duiCKD23PFSCs4JPoYyccjUFF3aQkQadqBhfzhjkwBH2mNOG9cTBwhamM37EIsIkZw3nRgg==";
+      };
+    };
+    "chownr-1.1.3" = {
       name = "chownr";
       packageName = "chownr";
-      version = "1.1.1";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/chownr/-/chownr-1.1.1.tgz";
-        sha512 = "j38EvO5+LHX84jlo6h4UzmOwi0UgW61WRyPtJz4qaadK5eY3BTS5TY/S1Stc3Uk2lIM6TPevAlULiEJwie860g==";
-      };
-    };
-    "chrome-trace-event-1.0.0" = {
+      version = "1.1.3";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/chownr/-/chownr-1.1.3.tgz";
+        sha512 = "i70fVHhmV3DtTl6nqvZOnIjbY0Pe4kAUjwHj8z0zAdgBtYrJyYwLKCCuRBQ5ppkyL0AkN7HKRnETdmdp1zqNXw==";
+      };
+    };
+    "chrome-trace-event-1.0.2" = {
       name = "chrome-trace-event";
       packageName = "chrome-trace-event";
-      version = "1.0.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/chrome-trace-event/-/chrome-trace-event-1.0.0.tgz";
-        sha512 = "xDbVgyfDTT2piup/h8dK/y4QZfJRSa73bw1WZ8b4XM1o7fsFubUVGYcE+1ANtOzJJELGpYoG2961z0Z6OAld9A==";
+      version = "1.0.2";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/chrome-trace-event/-/chrome-trace-event-1.0.2.tgz";
+        sha512 = "9e/zx1jw7B4CO+c/RXoCsfg/x1AfUBioy4owYH0bJprEYAx5hRFLRhWBqHAG57D0ZM4H7vxbP7bPe0VwhQRYDQ==";
       };
     };
     "cipher-base-1.0.4" = {
@@ -1958,13 +1958,13 b' let'
         sha1 = "168a4701756b6a7f51a12ce0c97bfa28c084ed63";
       };
     };
-    "colors-1.3.3" = {
+    "colors-1.4.0" = {
       name = "colors";
       packageName = "colors";
-      version = "1.3.3";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/colors/-/colors-1.3.3.tgz";
-        sha512 = "mmGt/1pZqYRjMxB1axhTo16/snVZ5krrKkcmMeVKxzECMMXoCgnvTPp10QgHfcbQZw8Dq2jMNG6je4JlWU0gWg==";
+      version = "1.4.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/colors/-/colors-1.4.0.tgz";
+        sha512 = "a+UqTh4kgZg/SlGvfbzDHpgRu7AAQOmmqRHJnxhRZICKFUT91brVhNNt58CMWU9PsBbv3PDCZUHbVxuDiH2mtA==";
       };
     };
     "combined-stream-1.0.8" = {
@@ -2003,6 +2003,15 b' let'
         sha512 = "6tvAOO+D6OENvRAh524Dh9jcfKTYDQAqvqezbCW82xj5X0pSrcpxtvRKHLG0yBY6SD7PSDrJaj+0AiOcKVd1Xg==";
       };
     };
+    "commander-2.20.3" = {
+      name = "commander";
+      packageName = "commander";
+      version = "2.20.3";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/commander/-/commander-2.20.3.tgz";
+        sha512 = "GpVkmM8vF2vQUkj2LvZmD35JxeJOLCwJ9cUkugyk2nuhbv3+mJvpLYYt+0+USMxE+oj+ey/lJEnhZw75x/OMcQ==";
+      };
+    };
     "commondir-1.0.1" = {
       name = "commondir";
       packageName = "commondir";
@@ -2093,13 +2102,13 b' let'
         sha512 = "Y+SQCF+0NoWQryez2zXn5J5knmr9z/9qSQt7fbL78u83rxmigOy8X5+BFn8CFSuX+nKT8gpYwJX68ekqtQt6ZA==";
       };
     };
-    "core-js-2.6.5" = {
+    "core-js-2.6.10" = {
       name = "core-js";
       packageName = "core-js";
-      version = "2.6.5";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/core-js/-/core-js-2.6.5.tgz";
-        sha512 = "klh/kDpwX8hryYL14M9w/xei6vrv6sE8gTHDG7/T/+SEovB/G4ejwcfE/CBzO6Edsu+OETZMZ3wcX/EjUkrl5A==";
+      version = "2.6.10";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/core-js/-/core-js-2.6.10.tgz";
+        sha512 = "I39t74+4t+zau64EN1fE5v2W31Adtc/REhzWN+gWRRXg6WH5qAsZm62DHpQ1+Yhe4047T55jvzz7MUqF/dBBlA==";
       };
     };
     "core-util-is-1.0.2" = {
@@ -2237,13 +2246,13 b' let'
         sha1 = "ddd52c587033f49e94b71fc55569f252e8ff5f85";
       };
     };
-    "cyclist-0.2.2" = {
+    "cyclist-1.0.1" = {
       name = "cyclist";
       packageName = "cyclist";
-      version = "0.2.2";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/cyclist/-/cyclist-0.2.2.tgz";
-        sha1 = "1b33792e11e914a2fd6d6ed6447464444e5fa640";
+      version = "1.0.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/cyclist/-/cyclist-1.0.1.tgz";
+        sha1 = "596e9698fd0c80e12038c2b82d6eb1b35b6224d9";
       };
     };
     "dashdash-1.14.1" = {
@@ -2435,13 +2444,13 b' let'
         sha512 = "gd3ypIPfOMr9h5jIKq8E3sHOTCjeirnl0WK5ZdS1AW0Odt0b1PaWaHdJ4Qk4klv+YB9aJBS7mESXjFoDQPu6DA==";
       };
     };
-    "dom-serializer-0.1.1" = {
+    "dom-serializer-0.2.1" = {
       name = "dom-serializer";
       packageName = "dom-serializer";
-      version = "0.1.1";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/dom-serializer/-/dom-serializer-0.1.1.tgz";
-        sha512 = "l0IU0pPzLWSHBcieZbpOKgkIn3ts3vAh7ZuFyXNwJxJXk/c4Gwj9xaTJwIDVQCXawWD0qb3IzMGH5rglQaO0XA==";
+      version = "0.2.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/dom-serializer/-/dom-serializer-0.2.1.tgz";
+        sha512 = "sK3ujri04WyjwQXVoK4PU3y8ula1stq10GJZpqHIUgoGZdsGzAGu65BnU3d08aTVSvO7mGPZUc0wTEDL+qGE0Q==";
       };
     };
     "dom5-2.3.0" = {
@@ -2471,6 +2480,15 b' let'
         sha512 = "BSKB+TSpMpFI/HOxCNr1O8aMOTZ8hT3pM3GQ0w/mWRmkhEDSFJkkyzz4XQsBV44BChwGkrDfMyjVD0eA2aFV3w==";
       };
     };
+    "domelementtype-2.0.1" = {
+      name = "domelementtype";
+      packageName = "domelementtype";
+      version = "2.0.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/domelementtype/-/domelementtype-2.0.1.tgz";
+        sha512 = "5HOHUDsYZWV8FGWN0Njbr/Rn7f/eWSQi1v7+HsUVwXgn8nWWlL64zKDkS0n8ZmQ3mlWOMuXOnR+7Nx/5tMO5AQ==";
+      };
+    };
     "domhandler-2.3.0" = {
       name = "domhandler";
       packageName = "domhandler";
@@ -2498,6 +2516,15 b' let'
         sha512 = "3VduRWLxx9hbVr42QieQN25mx/I61/mRdUSuxAmDGdDqZIN8qtP7tcKMa3KfpJjuGjOJGYYUzzeq6eGDnkzesA==";
       };
     };
+    "duplexer-0.1.1" = {
+      name = "duplexer";
+      packageName = "duplexer";
+      version = "0.1.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/duplexer/-/duplexer-0.1.1.tgz";
+        sha1 = "ace6ff808c1ce66b57d1ebf97977acb02334cfc1";
+      };
+    };
     "duplexify-3.7.1" = {
       name = "duplexify";
       packageName = "duplexify";
@@ -2516,22 +2543,22 b' let'
         sha1 = "3a83a904e54353287874c564b7549386849a98c9";
       };
     };
-    "electron-to-chromium-1.3.133" = {
+    "electron-to-chromium-1.3.302" = {
       name = "electron-to-chromium";
       packageName = "electron-to-chromium";
-      version = "1.3.133";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.3.133.tgz";
-        sha512 = "lyoC8aoqbbDqsprb6aPdt9n3DpOZZzdz/T4IZKsR0/dkZIxnJVUjjcpOSwA66jPRIOyDAamCTAUqweU05kKNSg==";
-      };
-    };
-    "elliptic-6.4.1" = {
+      version = "1.3.302";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.3.302.tgz";
+        sha512 = "1qConyiVEbj4xZRBXqtGR003+9tV0rJF0PS6aeO0Ln/UL637js9hdwweCl07meh/kJoI2N4W8q3R3g3F5z46ww==";
+      };
+    };
+    "elliptic-6.5.1" = {
       name = "elliptic";
       packageName = "elliptic";
-      version = "6.4.1";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/elliptic/-/elliptic-6.4.1.tgz";
-        sha512 = "BsXLz5sqX8OHcsh7CqBMztyXARmGQ3LWPtGjJi6DiJHq5C/qvi9P3OqgswKSDftbu8+IoI/QDTAm2fFnQ9SZSQ==";
+      version = "6.5.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/elliptic/-/elliptic-6.5.1.tgz";
+        sha512 = "xvJINNLbTeWQjrl6X+7eQCrIy/YPv5XCpKW6kB5mKvtnGILoLDcySuwomfdzt0BMdLNVnuRNTuzKNHj0bva1Cg==";
       };
     };
     "emojis-list-2.1.0" = {
@@ -2543,13 +2570,13 b' let'
         sha1 = "4daa4d9db00f9819880c79fa457ae5b09a1fd389";
       };
     };
-    "end-of-stream-1.4.1" = {
+    "end-of-stream-1.4.4" = {
       name = "end-of-stream";
       packageName = "end-of-stream";
-      version = "1.4.1";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.1.tgz";
-        sha512 = "1MkrZNvWTKCaigbn+W15elq2BB/L22nqrSY5DKlo3X6+vclJm8Bb5djXJBmEX6fS3+zCh/F4VBK5Z2KxJt4s2Q==";
+      version = "1.4.4";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.4.tgz";
+        sha512 = "+uw1inIHVPQoaVuHzRyXd21icM+cnt4CzD5rW+NC1wjOUSTOs+Te7FOv7AhN7vS9x/oIyhLP5PR1H+phQAHu5Q==";
       };
     };
     "enhanced-resolve-3.4.1" = {
@@ -2561,13 +2588,13 b' let'
         sha1 = "0421e339fd71419b3da13d129b3979040230476e";
       };
     };
-    "enhanced-resolve-4.1.0" = {
+    "enhanced-resolve-4.1.1" = {
       name = "enhanced-resolve";
       packageName = "enhanced-resolve";
-      version = "4.1.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-4.1.0.tgz";
-        sha512 = "F/7vkyTtyc/llOIn8oWclcB25KdRaiPBpZYDgJHgh/UHtpgT2p2eldQgtQnLtUvfMKPKxbRaQM/hHkvLHt1Vng==";
+      version = "4.1.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-4.1.1.tgz";
+        sha512 = "98p2zE+rL7/g/DzMHMTF4zZlCgeVdJ7yr6xzEpJRYwFYrGi9ANdn5DnJURg6RpBkyk60XYDnWIv51VfIhfNGuA==";
       };
     };
     "entities-1.0.0" = {
@@ -2579,13 +2606,13 b' let'
         sha1 = "b2987aa3821347fcde642b24fdfc9e4fb712bf26";
       };
     };
-    "entities-1.1.2" = {
+    "entities-2.0.0" = {
       name = "entities";
       packageName = "entities";
-      version = "1.1.2";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/entities/-/entities-1.1.2.tgz";
-        sha512 = "f2LZMYl1Fzu7YSBKg+RoROelpOaNrcGmE9AZubeDfrCEia483oW4MI4VyFd5VNHIgQ/7qm1I0wUHK1eJnn2y2w==";
+      version = "2.0.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/entities/-/entities-2.0.0.tgz";
+        sha512 = "D9f7V0JSRwIxlRI2mjMqufDrRDnx8p+eEOz7aUM9SuvF8gsBzra0/6tbjl1m8eQHrZlYj6PxqE00hZ1SAIKPLw==";
       };
     };
     "errno-0.1.7" = {
@@ -2597,13 +2624,13 b' let'
         sha512 = "MfrRBDWzIWifgq6tJj60gkAwtLNb6sQPlcFrSOflcP1aFmmruKQ2wRnze/8V6kgyz7H3FF8Npzv78mZ7XLLflg==";
       };
     };
-    "es-abstract-1.13.0" = {
+    "es-abstract-1.16.0" = {
       name = "es-abstract";
       packageName = "es-abstract";
-      version = "1.13.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/es-abstract/-/es-abstract-1.13.0.tgz";
-        sha512 = "vDZfg/ykNxQVwup/8E1BZhVzFfBxs9NqMzGcvIJrqg5k2/5Za2bWo40dK2J1pgLngZ7c+Shh8lwYtLGyrwPutg==";
+      version = "1.16.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/es-abstract/-/es-abstract-1.16.0.tgz";
+        sha512 = "xdQnfykZ9JMEiasTAJZJdMWCQ1Vm00NBw79/AWi7ELfZuuPCSOMDZbT9mkOfSctVtfhb+sAAzrm+j//GjjLHLg==";
       };
     };
     "es-to-primitive-1.2.0" = {
@@ -2687,22 +2714,22 b' let'
         sha512 = "64RBB++fIOAXPw3P9cy89qfMlvZEXZkqqJkjqqXIvzP5ezRZjW+lPWjw35UX/3EhUPFYbg5ER4JYgDw4007/DQ==";
       };
     };
-    "estraverse-4.2.0" = {
+    "estraverse-4.3.0" = {
       name = "estraverse";
       packageName = "estraverse";
-      version = "4.2.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/estraverse/-/estraverse-4.2.0.tgz";
-        sha1 = "0dee3fed31fcd469618ce7342099fc1afa0bdb13";
-      };
-    };
-    "esutils-2.0.2" = {
+      version = "4.3.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/estraverse/-/estraverse-4.3.0.tgz";
+        sha512 = "39nnKffWz8xN1BU/2c79n9nB9HDzo0niYUqx6xyqUnyoAnQyyWpOTdZEeiCch8BBu515t4wp9ZmgVfVhn9EBpw==";
+      };
+    };
+    "esutils-2.0.3" = {
       name = "esutils";
       packageName = "esutils";
-      version = "2.0.2";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/esutils/-/esutils-2.0.2.tgz";
-        sha1 = "0abf4f1caa5bcb1f7a9d8acc6dea4faaa04bac9b";
+      version = "2.0.3";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/esutils/-/esutils-2.0.3.tgz";
+        sha512 = "kVscqXk4OCp68SZ0dkgEKVi6/8ij300KBWTJq32P/dYeWTSwK41WyTxalN1eRmA5Z9UU/LX9D7FWSmV9SAYx6g==";
       };
     };
     "eventemitter2-0.4.14" = {
@@ -2867,6 +2894,15 b' let'
         sha1 = "c14c5b3bf14d7417ffbfd990c0a7495cd9f337bc";
       };
     };
+    "figures-1.7.0" = {
+      name = "figures";
+      packageName = "figures";
+      version = "1.7.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/figures/-/figures-1.7.0.tgz";
+        sha1 = "cbe1e3affcf1cd44b80cadfed28dc793a9701d2e";
+      };
+    };
     "file-sync-cmp-0.1.1" = {
       name = "file-sync-cmp";
       packageName = "file-sync-cmp";
@@ -2948,13 +2984,13 b' let'
         sha512 = "lNaHNVymajmk0OJMBn8fVUAU1BtDeKIqKoVhk4xAALB57aALg6b4W0MfJ/cUE0g9YBXy5XhSlPIpYIJ7HaY/3Q==";
       };
     };
-    "flatten-1.0.2" = {
+    "flatten-1.0.3" = {
       name = "flatten";
       packageName = "flatten";
-      version = "1.0.2";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/flatten/-/flatten-1.0.2.tgz";
-        sha1 = "dae46a9d78fbe25292258cc1e780a41d95c03782";
+      version = "1.0.3";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/flatten/-/flatten-1.0.3.tgz";
+        sha512 = "dVsPA/UwQ8+2uoFe5GHtiBMu48dWLTdsuEd7CKGlZlD78r1TTWBvDuFaFGKCo/ZfEr95Uk56vZoX86OsHkUeIg==";
       };
     };
     "flush-write-stream-1.1.1" = {
@@ -3128,13 +3164,13 b' let'
         sha1 = "4a973f635b9190f715d10987d5c00fd2815ebe3d";
       };
     };
-    "glob-7.1.4" = {
+    "glob-7.1.5" = {
       name = "glob";
       packageName = "glob";
-      version = "7.1.4";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/glob/-/glob-7.1.4.tgz";
-        sha512 = "hkLPepehmnKk41pUGm3sYxoFs/umurYfYJCerbXEyFIWcAzvpipAgVkBqqT9RBKMGjnq6kMuyYwha6csxbiM1A==";
+      version = "7.1.5";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/glob/-/glob-7.1.5.tgz";
+        sha512 = "J9dlskqUXK1OeTOYBEn5s8aMukWMwWfs+rPTn/jn50Ux4MNXVhubL1wu/j2t+H4NVI+cXEcCaYellqaPVGXNqQ==";
       };
     };
     "glob-parent-3.1.0" = {
@@ -3218,13 +3254,13 b' let'
         sha1 = "15a4806a57547cb2d2dbf27f42e89a8c3451b364";
       };
     };
-    "graceful-fs-4.1.15" = {
+    "graceful-fs-4.2.3" = {
       name = "graceful-fs";
       packageName = "graceful-fs";
-      version = "4.1.15";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/graceful-fs/-/graceful-fs-4.1.15.tgz";
-        sha512 = "6uHUhOPEBgQ24HM+r6b/QwWfZq+yiFcipKFrOFiBEnWdy5sdzYoi+pJeQaPI5qOLRFqWmAXUPQNsielzdLoecA==";
+      version = "4.2.3";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/graceful-fs/-/graceful-fs-4.2.3.tgz";
+        sha512 = "a30VEBm4PEdx1dRB7MFK7BejejvCvBronbLjht+sHuGYj8PHs7M/5Z+rt5lw551vZ7yfTCj4Vuyy3mSJytDWRQ==";
       };
     };
     "grunt-0.4.5" = {
@@ -3281,6 +3317,15 b' let'
         sha1 = "3bbdec0b75d12ceaa55d62943625c0b0861cdf6f";
       };
     };
+    "grunt-contrib-uglify-4.0.1" = {
+      name = "grunt-contrib-uglify";
+      packageName = "grunt-contrib-uglify";
+      version = "4.0.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/grunt-contrib-uglify/-/grunt-contrib-uglify-4.0.1.tgz";
+        sha512 = "dwf8/+4uW1+7pH72WButOEnzErPGmtUvc8p08B0eQS/6ON0WdeQu0+WFeafaPTbbY1GqtS25lsHWaDeiTQNWPg==";
+      };
+    };
     "grunt-contrib-watch-0.6.1" = {
       name = "grunt-contrib-watch";
       packageName = "grunt-contrib-watch";
@@ -3335,6 +3380,15 b' let'
         sha512 = "SaZ8K8lG4iTxs7ClZxOWCf3kxqS2y+Eel8SbaEGgBKwhAp6e45beIu+vhBZRLX3vonKML2kjemKsQ21REaqNFQ==";
       };
     };
+    "gzip-size-3.0.0" = {
+      name = "gzip-size";
+      packageName = "gzip-size";
+      version = "3.0.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/gzip-size/-/gzip-size-3.0.0.tgz";
+        sha1 = "546188e9bdc337f673772f81660464b389dce520";
+      };
+    };
     "har-schema-1.0.5" = {
       name = "har-schema";
       packageName = "har-schema";
@@ -3695,15 +3749,6 b' let'
         sha1 = "f30f716c8e2bd346c7b67d3df3915566a7c05607";
       };
     };
-    "indexof-0.0.1" = {
-      name = "indexof";
-      packageName = "indexof";
-      version = "0.0.1";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/indexof/-/indexof-0.0.1.tgz";
-        sha1 = "82dc336d232b9062179d05ab3293a66059fd435d";
-      };
-    };
     "inflight-1.0.6" = {
       name = "inflight";
       packageName = "inflight";
@@ -3740,6 +3785,15 b' let'
         sha1 = "633c2c83e3da42a502f52466022480f4208261de";
       };
     };
+    "inherits-2.0.4" = {
+      name = "inherits";
+      packageName = "inherits";
+      version = "2.0.4";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz";
+        sha512 = "k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==";
+      };
+    };
     "ini-1.3.5" = {
       name = "ini";
       packageName = "ini";
@@ -4424,13 +4478,13 b' let'
         sha1 = "fadd834b9683073da179b3eae6d9c0d15053f73e";
       };
     };
-    "lodash-4.17.11" = {
+    "lodash-4.17.15" = {
       name = "lodash";
       packageName = "lodash";
-      version = "4.17.11";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/lodash/-/lodash-4.17.11.tgz";
-        sha512 = "cQKh8igo5QUhZ7lg38DYWAxMvjSAKG0A8wGSVimP07SIUEK2UO+arSRKbRZWtelMtN5V0Hkwh5ryOto/SshYIg==";
+      version = "4.17.15";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/lodash/-/lodash-4.17.15.tgz";
+        sha512 = "8xOcRHvCjnocdS5cpwXQXVzmmh5e5+saE2QGoeQmbKmRS6J3VQppPOIt0MnmE+4xlZoumy0GPG0D0MVIQbNA1A==";
       };
     };
     "lodash.camelcase-4.3.0" = {
@@ -4577,6 +4631,15 b' let'
         sha1 = "de819fdbcd84dccd8fae59c6aeb79615b9d266ac";
       };
     };
+    "maxmin-2.1.0" = {
+      name = "maxmin";
+      packageName = "maxmin";
+      version = "2.1.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/maxmin/-/maxmin-2.1.0.tgz";
+        sha1 = "4d3b220903d95eee7eb7ac7fa864e72dc09a3166";
+      };
+    };
     "md5.js-1.3.5" = {
       name = "md5.js";
       packageName = "md5.js";
@@ -4604,6 +4667,15 b' let'
         sha1 = "3a9a20b8462523e447cfbc7e8bb80ed667bfc552";
       };
     };
+    "memory-fs-0.5.0" = {
+      name = "memory-fs";
+      packageName = "memory-fs";
+      version = "0.5.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/memory-fs/-/memory-fs-0.5.0.tgz";
+        sha512 = "jA0rdU5KoQMC0e6ppoNRtpp6vjFq6+NY7r8hywnC7V+1Xj/MtHwGIbB1QaK/dunyjWteJzmkpd7ooeWg10T7GA==";
+      };
+    };
     "micromatch-3.1.10" = {
       name = "micromatch";
       packageName = "micromatch";
@@ -4730,13 +4802,13 b' let'
         sha512 = "zHo8v+otD1J10j/tC+VNoGK9keCuByhKovAvdn74dmxJl9+mWHnx6EMsDN4lgRoMI/eYo2nchAxniIbUPb5onw==";
       };
     };
-    "mixin-deep-1.3.1" = {
+    "mixin-deep-1.3.2" = {
       name = "mixin-deep";
       packageName = "mixin-deep";
-      version = "1.3.1";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/mixin-deep/-/mixin-deep-1.3.1.tgz";
-        sha512 = "8ZItLHeEgaqEvd5lYBXfm4EZSFCX29Jb9K+lAHhDKzReKBQKj3R+7NOF6tjqYi9t4oI8VUfaWITJQm86wnXGNQ==";
+      version = "1.3.2";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/mixin-deep/-/mixin-deep-1.3.2.tgz";
+        sha512 = "WRoDn//mXBiJ1H40rqa3vH0toePwSsGb45iInWlTySa+Uu4k3tYUSxa2v1KqAiLtvlrSzaExqS1gtk96A9zvEA==";
       };
     };
     "mkdirp-0.5.1" = {
@@ -4784,13 +4856,13 b' let'
         sha1 = "5608aeadfc00be6c2901df5f9861788de0d597c8";
       };
     };
-    "nan-2.13.2" = {
+    "nan-2.14.0" = {
       name = "nan";
       packageName = "nan";
-      version = "2.13.2";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/nan/-/nan-2.13.2.tgz";
-        sha512 = "TghvYc72wlMGMVMluVo9WRJc0mB8KxxF/gZ4YYFy7V2ZQX9l7rgbPg7vjS9mt6U5HXODVFVI2bOduCzwOMv/lw==";
+      version = "2.14.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/nan/-/nan-2.14.0.tgz";
+        sha512 = "INOFj37C7k3AfaNTtX8RhsTw7qRy7eLET14cROi9+5HAVbbHuIWUHEauBv5qT4Av2tWasiTY1Jw6puUNqRJXQg==";
       };
     };
     "nanomatch-1.2.13" = {
@@ -4829,13 +4901,13 b' let'
         sha512 = "rmTZ9kz+f3rCvK2TD1Ue/oZlns7OGoIWP4fc3llxxRXlOkHKoWPPWJOfFYpITabSow43QJbRIoHQXtt10VldyQ==";
       };
     };
-    "node-libs-browser-2.2.0" = {
+    "node-libs-browser-2.2.1" = {
       name = "node-libs-browser";
       packageName = "node-libs-browser";
-      version = "2.2.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/node-libs-browser/-/node-libs-browser-2.2.0.tgz";
-        sha512 = "5MQunG/oyOaBdttrL40dA7bUfPORLRWMUJLQtMg7nluxUvk5XwnLdL9twQHFAjRx/y7mIMkLKT9++qPbbk6BZA==";
+      version = "2.2.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/node-libs-browser/-/node-libs-browser-2.2.1.tgz";
+        sha512 = "h/zcD8H9kaDZ9ALUWwlBUDo6TKF8a7qBSCSEGfjTVIYeqsioSKaAX+BN7NgiMGp6iSIXZ3PxgCu8KS3b71YK5Q==";
       };
     };
     "nopt-1.0.10" = {
@@ -4973,6 +5045,15 b' let'
         sha1 = "7e7d858b781bd7c991a41ba975ed3812754e998c";
       };
     };
+    "object-inspect-1.6.0" = {
+      name = "object-inspect";
+      packageName = "object-inspect";
+      version = "1.6.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/object-inspect/-/object-inspect-1.6.0.tgz";
+        sha512 = "GJzfBZ6DgDAmnuaM3104jR4s1Myxr3Y3zfIyN4z3UdqN69oSRacNK8UhnobDdC+7J2AHCjGwxQubNJfE70SXXQ==";
+      };
+    };
     "object-keys-1.1.1" = {
       name = "object-keys";
       packageName = "object-keys";
@@ -5117,13 +5198,13 b' let'
         sha512 = "vvcXsLAJ9Dr5rQOPk7toZQZJApBl2K4J6dANSsEuh6QI41JYcsS/qhTGa9ErIUUgK3WNQoJYvylxvjqmiqEA9Q==";
       };
     };
-    "p-limit-2.2.0" = {
+    "p-limit-2.2.1" = {
       name = "p-limit";
       packageName = "p-limit";
-      version = "2.2.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/p-limit/-/p-limit-2.2.0.tgz";
-        sha512 = "pZbTJpoUsCzV48Mc9Nh51VbwO0X9cuPFE8gYwx9BTCt9SF8/b7Zljd2fVgOxhIF/HDTKgpVzs+GPhyKfjLLFRQ==";
+      version = "2.2.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/p-limit/-/p-limit-2.2.1.tgz";
+        sha512 = "85Tk+90UCVWvbDavCLKPOLC9vvY8OwEX/RtKF+/1OADJMVlFfEHOiMTPVyxg7mk/dKa+ipdHm0OUkTvCpMTuwg==";
       };
     };
     "p-locate-2.0.0" = {
@@ -5171,13 +5252,13 b' let'
         sha512 = "0DTvPVU3ed8+HNXOu5Bs+o//Mbdj9VNQMUOe9oKCwh8l0GNwpTDMKCWbRjgtD291AWnkAgkqA/LOnQS8AmS1tw==";
       };
     };
-    "parallel-transform-1.1.0" = {
+    "parallel-transform-1.2.0" = {
       name = "parallel-transform";
       packageName = "parallel-transform";
-      version = "1.1.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/parallel-transform/-/parallel-transform-1.1.0.tgz";
-        sha1 = "d410f065b05da23081fcd10f28854c29bda33b06";
+      version = "1.2.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/parallel-transform/-/parallel-transform-1.2.0.tgz";
+        sha512 = "P2vSmIu38uIlvdcU7fDkyrxj33gTUy/ABO5ZUbGowxNCopBq/OoD42bP4UmMrJoPyk4Uqf0mu3mtWBhHCZD8yg==";
       };
     };
     "param-case-2.1.1" = {
@@ -5189,13 +5270,13 b' let'
         sha1 = "df94fd8cf6531ecf75e6bef9a0858fbc72be2247";
       };
     };
-    "parse-asn1-5.1.4" = {
+    "parse-asn1-5.1.5" = {
       name = "parse-asn1";
       packageName = "parse-asn1";
-      version = "5.1.4";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/parse-asn1/-/parse-asn1-5.1.4.tgz";
-        sha512 = "Qs5duJcuvNExRfFZ99HDD3z4mAi3r9Wl/FOjEOijlxwCZs7E7mW2vjTpgQ4J8LpTF8x5v+1Vn5UQFejmWT11aw==";
+      version = "5.1.5";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/parse-asn1/-/parse-asn1-5.1.5.tgz";
+        sha512 = "jkMYn1dcJqF6d5CpU689bq7w/b5ALS9ROVSpQDPrZsqqesUJii9qutvoT5ltGedNXMO2e16YUWIghG9KxaViTQ==";
       };
     };
     "parse-filepath-1.0.2" = {
@@ -5252,13 +5333,13 b' let'
         sha1 = "b363e55e8006ca6fe21784d2db22bd15d7917f14";
       };
     };
-    "path-browserify-0.0.0" = {
+    "path-browserify-0.0.1" = {
       name = "path-browserify";
       packageName = "path-browserify";
-      version = "0.0.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/path-browserify/-/path-browserify-0.0.0.tgz";
-        sha1 = "a0b870729aae214005b7d5032ec2cbbb0fb4451a";
+      version = "0.0.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/path-browserify/-/path-browserify-0.0.1.tgz";
+        sha512 = "BapA40NHICOS+USX9SN4tyhq+A2RrN/Ws5F0Z5aMHDp98Fl86lX8Oti8B7uN93L4Ifv4fHOEA+pQw87gmMO/lQ==";
       };
     };
     "path-dirname-1.0.2" = {
@@ -5711,6 +5792,15 b' let'
         sha1 = "d4f4562b0ce3696e41ac52d0e002e57a635dc6dc";
       };
     };
+    "pretty-bytes-3.0.1" = {
+      name = "pretty-bytes";
+      packageName = "pretty-bytes";
+      version = "3.0.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/pretty-bytes/-/pretty-bytes-3.0.1.tgz";
+        sha1 = "27d0008d778063a0b4811bb35c79f1bd5d5fbccf";
+      };
+    };
     "pretty-error-2.1.1" = {
       name = "pretty-error";
       packageName = "pretty-error";
@@ -5738,13 +5828,13 b' let'
         sha1 = "7332300e840161bda3e69a1d1d91a7d4bc16f182";
       };
     };
-    "process-nextick-args-2.0.0" = {
+    "process-nextick-args-2.0.1" = {
       name = "process-nextick-args";
       packageName = "process-nextick-args";
-      version = "2.0.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/process-nextick-args/-/process-nextick-args-2.0.0.tgz";
-        sha512 = "MtEC1TqN0EU5nephaJ4rAtThHtC86dNN9qCuEhtshvpVBkAW5ZO7BASN9REnF9eoXGcRub+pFuKEpOHE+HbEMw==";
+      version = "2.0.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/process-nextick-args/-/process-nextick-args-2.0.1.tgz";
+        sha512 = "3ouUOpQhtgrbOa17J7+uxOTpITYWaGP7/AhoR3+A+/1e9skrzelGi/dXzEYyvbxubEF6Wn2ypscTKiKJFFn1ag==";
       };
     };
     "promise-7.3.1" = {
@@ -5990,13 +6080,13 b' let'
         sha1 = "747c914e049614a4c9cfbba629871ad1d2927716";
       };
     };
-    "reduce-function-call-1.0.2" = {
+    "reduce-function-call-1.0.3" = {
       name = "reduce-function-call";
       packageName = "reduce-function-call";
-      version = "1.0.2";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/reduce-function-call/-/reduce-function-call-1.0.2.tgz";
-        sha1 = "5a200bf92e0e37751752fe45b0ab330fd4b6be99";
+      version = "1.0.3";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/reduce-function-call/-/reduce-function-call-1.0.3.tgz";
+        sha512 = "Hl/tuV2VDgWgCSEeWMLwxLZqX7OK59eU1guxXsRKTAyeYimivsKdtcV4fu3r710tpG5GmDKDhQ0HSZLExnNmyQ==";
       };
     };
     "regenerate-1.4.0" = {
@@ -6152,13 +6242,13 b' let'
         sha1 = "97f717b69d48784f5f526a6c5aa8ffdda055a4d1";
       };
     };
-    "resolve-1.10.1" = {
+    "resolve-1.12.0" = {
       name = "resolve";
       packageName = "resolve";
-      version = "1.10.1";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/resolve/-/resolve-1.10.1.tgz";
-        sha512 = "KuIe4mf++td/eFb6wkaPbMDnP6kObCaEtIDuHOUED6MNUo4K670KZUHuuvYPZDxNF0WVLw49n06M2m2dXphEzA==";
+      version = "1.12.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/resolve/-/resolve-1.12.0.tgz";
+        sha512 = "B/dOmuoAik5bKcD6s6nXDCjzUKnaDvdkRyAk6rsmsKLipWj4797iothd7jmmUhWTfinVMU+wc56rYKsit2Qy4w==";
       };
     };
     "resolve-cwd-2.0.0" = {
@@ -6224,13 +6314,13 b' let'
         sha1 = "e439be2aaee327321952730f99a8929e4fc50582";
       };
     };
-    "rimraf-2.6.3" = {
+    "rimraf-2.7.1" = {
       name = "rimraf";
       packageName = "rimraf";
-      version = "2.6.3";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/rimraf/-/rimraf-2.6.3.tgz";
-        sha512 = "mwqeW5XsA2qAejG46gYdENaxXjx9onRNCfn7L0duuP4hCuTIi/QO7PDK07KJfp1d+izWPrzEJDcSqBa0OZQriA==";
+      version = "2.7.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/rimraf/-/rimraf-2.7.1.tgz";
+        sha512 = "uWjbaKIK3T1OSVptzX7Nl6PvQ3qAGtKEtVRjRuazjfL3Bx5eI409VZSqgND+4UNnmzLVdPj9FqFJNPqBZFve4w==";
       };
     };
     "ripemd160-2.0.2" = {
@@ -6260,6 +6350,15 b' let'
         sha512 = "Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g==";
       };
     };
+    "safe-buffer-5.2.0" = {
+      name = "safe-buffer";
+      packageName = "safe-buffer";
+      version = "5.2.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.0.tgz";
+        sha512 = "fZEwUGbVl7kouZs1jCdMLdt95hdIv0ZeHg6L7qPeciMZhZ+/gdesW4wgTARkrFWEpspjEATAzUGPG8N2jJiwbg==";
+      };
+    };
     "safe-regex-1.1.0" = {
       name = "safe-regex";
       packageName = "safe-regex";
@@ -6305,22 +6404,22 b' let'
         sha1 = "0e7350acdec80b1108528786ec1d4418d11b396d";
       };
     };
-    "semver-5.7.0" = {
+    "semver-5.7.1" = {
       name = "semver";
       packageName = "semver";
-      version = "5.7.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/semver/-/semver-5.7.0.tgz";
-        sha512 = "Ya52jSX2u7QKghxeoFGpLwCtGlt7j0oY9DYb5apt9nPlJ42ID+ulTXESnt/qAQcoSERyZ5sl3LDIOw0nAn/5DA==";
-      };
-    };
-    "serialize-javascript-1.7.0" = {
+      version = "5.7.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/semver/-/semver-5.7.1.tgz";
+        sha512 = "sauaDf/PZdVgrLTNYHRtpXa1iRiKcaebiKQ1BJdpQlWH2lCvexQdX55snPFyK7QzpudqbCI0qXFfOasHdyNDGQ==";
+      };
+    };
+    "serialize-javascript-1.9.1" = {
       name = "serialize-javascript";
       packageName = "serialize-javascript";
-      version = "1.7.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/serialize-javascript/-/serialize-javascript-1.7.0.tgz";
-        sha512 = "ke8UG8ulpFOxO8f8gRYabHQe/ZntKlcig2Mp+8+URDP1D8vJZ0KUt7LYo07q25Z/+JVSgpr/cui9PIp5H6/+nA==";
+      version = "1.9.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/serialize-javascript/-/serialize-javascript-1.9.1.tgz";
+        sha512 = "0Vb/54WJ6k5v8sSWN09S0ora+Hnr+cX40r9F170nT+mSkaxltoE/7R3OrIdBSUv1OoiobH1QoWQbCnAO+e8J1A==";
       };
     };
     "set-blocking-2.0.0" = {
@@ -6332,22 +6431,13 b' let'
         sha1 = "045f9782d011ae9a6803ddd382b24392b3d890f7";
       };
     };
-    "set-value-0.4.3" = {
+    "set-value-2.0.1" = {
       name = "set-value";
       packageName = "set-value";
-      version = "0.4.3";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/set-value/-/set-value-0.4.3.tgz";
-        sha1 = "7db08f9d3d22dc7f78e53af3c3bf4666ecdfccf1";
-      };
-    };
-    "set-value-2.0.0" = {
-      name = "set-value";
-      packageName = "set-value";
-      version = "2.0.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/set-value/-/set-value-2.0.0.tgz";
-        sha512 = "hw0yxk9GT/Hr5yJEYnHNKYXkIA8mVJgd9ditYZCe16ZczcaELYYcfvaXesNACk2O8O0nTiPQcQhGUQj8JLzeeg==";
+      version = "2.0.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/set-value/-/set-value-2.0.1.tgz";
+        sha512 = "JxHc1weCN68wRY0fhCoXpyK55m/XPHafOmK4UWD7m2CI14GMcFypt4w/0+NV5f/ZMby2F6S2wwA7fgynh9gWSw==";
       };
     };
     "setimmediate-1.0.5" = {
@@ -6665,6 +6755,24 b' let'
         sha512 = "nOqH59deCq9SRHlxq1Aw85Jnt4w6KvLKqWVik6oA9ZklXLNIOlqg4F2yrT1MVaTjAqvVwdfeZ7w7aCvJD7ugkw==";
       };
     };
+    "string.prototype.trimleft-2.1.0" = {
+      name = "string.prototype.trimleft";
+      packageName = "string.prototype.trimleft";
+      version = "2.1.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/string.prototype.trimleft/-/string.prototype.trimleft-2.1.0.tgz";
+        sha512 = "FJ6b7EgdKxxbDxc79cOlok6Afd++TTs5szo+zJTUyow3ycrRfJVE2pq3vcN53XexvKZu/DJMDfeI/qMiZTrjTw==";
+      };
+    };
+    "string.prototype.trimright-2.1.0" = {
+      name = "string.prototype.trimright";
+      packageName = "string.prototype.trimright";
+      version = "2.1.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/string.prototype.trimright/-/string.prototype.trimright-2.1.0.tgz";
+        sha512 = "fXZTSV55dNBwv16uw+hh5jkghxSnc5oHq+5K/gXgizHwAvMetdAJlHqqoFC1FSDVPYWLkAKl2cxpUT41sV7nSg==";
+      };
+    };
     "string_decoder-0.10.31" = {
       name = "string_decoder";
       packageName = "string_decoder";
@@ -6683,13 +6791,13 b' let'
         sha512 = "n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg==";
       };
     };
-    "string_decoder-1.2.0" = {
+    "string_decoder-1.3.0" = {
       name = "string_decoder";
       packageName = "string_decoder";
-      version = "1.2.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/string_decoder/-/string_decoder-1.2.0.tgz";
-        sha512 = "6YqyX6ZWEYguAxgZzHGL7SsCeGx3V2TtOTqZz1xSTSWnqsbWwbptafNyvf/ACquZUXV3DANr5BDIwNYe1mN42w==";
+      version = "1.3.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz";
+        sha512 = "hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==";
       };
     };
     "stringstream-0.0.6" = {
@@ -6836,13 +6944,13 b' let'
         sha512 = "/mrRod8xqpA+IHSLyGCQ2s8SPHiCDEeQJSep1jqLYeEUClOFG2Qsh+4FU6G9VeqpZnGW/Su8LQGc4YKni5rYSQ==";
       };
     };
-    "timers-browserify-2.0.10" = {
+    "timers-browserify-2.0.11" = {
       name = "timers-browserify";
       packageName = "timers-browserify";
-      version = "2.0.10";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/timers-browserify/-/timers-browserify-2.0.10.tgz";
-        sha512 = "YvC1SV1XdOUaL6gx5CoGroT3Gu49pK9+TZ38ErPldOWW4j49GI1HKs9DV+KGq/w6y+LZ72W1c8cKz2vzY+qpzg==";
+      version = "2.0.11";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/timers-browserify/-/timers-browserify-2.0.11.tgz";
+        sha512 = "60aV6sgJ5YEbzUdn9c8kYGIqOubPoUdqQCul3SBAsRCZ40s6Y5cMcrW4dt3/k/EsbLVJNl9n6Vz3fTc+k2GeKQ==";
       };
     };
     "tiny-emitter-2.1.0" = {
@@ -6944,13 +7052,13 b' let'
         sha1 = "30c6203e1e66b841a88701ed8858f1725d94b026";
       };
     };
-    "tslib-1.9.3" = {
+    "tslib-1.10.0" = {
       name = "tslib";
       packageName = "tslib";
-      version = "1.9.3";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/tslib/-/tslib-1.9.3.tgz";
-        sha512 = "4krF8scpejhaOgqzBEcGM7yDIEfi0/8+8zDRZhNZZ2kjmHJ4hv3zCbQWxoJGz1iw5U0Jl0nma13xzHXcncMavQ==";
+      version = "1.10.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/tslib/-/tslib-1.10.0.tgz";
+        sha512 = "qOebF53frne81cf0S9B41ByenJ3/IuH8yJKngAX35CmiZySA0khhkovshKK+jGCaMnVomla7gVlIcc3EvKPbTQ==";
       };
     };
     "tty-browserify-0.0.0" = {
@@ -7016,6 +7124,15 b' let'
         sha512 = "Y2VsbPVs0FIshJztycsO2SfPk7/KAF/T72qzv9u5EpQ4kB2hQoHlhNQTsNyy6ul7lQtqJN/AoWeS23OzEiEFxw==";
       };
     };
+    "uglify-js-3.6.7" = {
+      name = "uglify-js";
+      packageName = "uglify-js";
+      version = "3.6.7";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/uglify-js/-/uglify-js-3.6.7.tgz";
+        sha512 = "4sXQDzmdnoXiO+xvmTzQsfIiwrjUCSA95rSP4SEd8tDb51W2TiDOlL76Hl+Kw0Ie42PSItCW8/t6pBNCF2R48A==";
+      };
+    };
     "uglify-to-browserify-1.0.2" = {
       name = "uglify-to-browserify";
       packageName = "uglify-to-browserify";
@@ -7079,13 +7196,13 b' let'
         sha1 = "8cdd8fbac4e2d2ea1e7e2e8097c42f442280f85b";
       };
     };
-    "union-value-1.0.0" = {
+    "union-value-1.0.1" = {
       name = "union-value";
       packageName = "union-value";
-      version = "1.0.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/union-value/-/union-value-1.0.0.tgz";
-        sha1 = "5c71c34cb5bad5dcebe3ea0cd08207ba5aa1aea4";
+      version = "1.0.1";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/union-value/-/union-value-1.0.1.tgz";
+        sha512 = "tJfXmxMeWYnczCVs7XAEvIV7ieppALdyepWMkHkwciRpZraG/xwT+s2JN8+pr1+8jCRf80FFzvr+MpQeeoF4Xg==";
       };
     };
     "uniq-1.0.1" = {
@@ -7115,13 +7232,13 b' let'
         sha512 = "Vmp0jIp2ln35UTXuryvjzkjGdRyf9b2lTXuSYUiPmzRcl3FDtYqAwOnTJkAngD9SWhnoJzDbTKwaOrZ+STtxNQ==";
       };
     };
-    "unique-slug-2.0.1" = {
+    "unique-slug-2.0.2" = {
       name = "unique-slug";
       packageName = "unique-slug";
-      version = "2.0.1";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/unique-slug/-/unique-slug-2.0.1.tgz";
-        sha512 = "n9cU6+gITaVu7VGj1Z8feKMmfAjEAQGhwD9fE3zvpRRa0wEIx8ODYkVGfSc94M2OX00tUFV8wH3zYbm1I8mxFg==";
+      version = "2.0.2";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/unique-slug/-/unique-slug-2.0.2.tgz";
+        sha512 = "zoWr9ObaxALD3DOPfjPSqxt4fnZiWblxHIgeWqW8x7UqDzEtHEQLzji2cuJYQFCU6KmoJikOYAZlrTHHebjx2w==";
       };
     };
     "unset-value-1.0.0" = {
@@ -7133,13 +7250,13 b' let'
         sha1 = "8376873f7d2335179ffb1e6fc3a8ed0dfc8ab559";
       };
     };
-    "upath-1.1.2" = {
+    "upath-1.2.0" = {
       name = "upath";
       packageName = "upath";
-      version = "1.1.2";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/upath/-/upath-1.1.2.tgz";
-        sha512 = "kXpym8nmDmlCBr7nKdIx8P2jNBa+pBpIUFRnKJ4dr8htyYGJFokkr2ZvERRtUN+9SY+JqXouNgUPtv6JQva/2Q==";
+      version = "1.2.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/upath/-/upath-1.2.0.tgz";
+        sha512 = "aZwGpamFO61g3OlfT7OQCHqhGnW43ieH9WZeP7QxN/G/jS4jfqUkZxoryvJgVPEcrl5NL/ggHsSmLMHuH64Lhg==";
       };
     };
     "upper-case-1.1.3" = {
@@ -7160,6 +7277,15 b' let'
         sha512 = "KY9Frmirql91X2Qgjry0Wd4Y+YTdrdZheS8TFwvkbLWf/G5KNJDCh6pKL5OZctEW4+0Baa5idK2ZQuELRwPznQ==";
       };
     };
+    "uri-path-1.0.0" = {
+      name = "uri-path";
+      packageName = "uri-path";
+      version = "1.0.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/uri-path/-/uri-path-1.0.0.tgz";
+        sha1 = "9747f018358933c31de0fccfd82d138e67262e32";
+      };
+    };
     "urix-0.1.0" = {
       name = "urix";
       packageName = "urix";
@@ -7232,22 +7358,22 b' let'
         sha1 = "8a16a05d445657a3aea5eecc5b12a4fa5379772c";
       };
     };
-    "uuid-3.3.2" = {
+    "uuid-3.3.3" = {
       name = "uuid";
       packageName = "uuid";
-      version = "3.3.2";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/uuid/-/uuid-3.3.2.tgz";
-        sha512 = "yXJmeNaw3DnnKAOKJE51sL/ZaYfWJRl1pK9dr19YFCu0ObS231AB1/LbqTKRAQ5kw8A90rA6fr4riOUpTZvQZA==";
-      };
-    };
-    "v8-compile-cache-2.0.3" = {
+      version = "3.3.3";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/uuid/-/uuid-3.3.3.tgz";
+        sha512 = "pW0No1RGHgzlpHJO1nsVrHKpOEIxkGg1xB+v0ZmdNH5OAeAwzAVrCnI2/6Mtx+Uys6iaylxa+D3g4j63IKKjSQ==";
+      };
+    };
+    "v8-compile-cache-2.1.0" = {
       name = "v8-compile-cache";
       packageName = "v8-compile-cache";
-      version = "2.0.3";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/v8-compile-cache/-/v8-compile-cache-2.0.3.tgz";
-        sha512 = "CNmdbwQMBjwr9Gsmohvm0pbL954tJrNzf6gWL3K+QMQf00PF7ERGrEiLgjuU3mKreLC2MeGhUsNV9ybTbLgd3w==";
+      version = "2.1.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/v8-compile-cache/-/v8-compile-cache-2.1.0.tgz";
+        sha512 = "usZBT3PW+LOjM25wbqIlZwPeJV+3OSz3M1k1Ws8snlW39dZyYL9lOGC5FgPVHfk0jKmjiDV8Z0mIbVQPiwFs7g==";
       };
     };
     "v8flags-3.1.3" = {
@@ -7277,13 +7403,13 b' let'
         sha1 = "3a105ca17053af55d6e270c1f8288682e18da400";
       };
     };
-    "vm-browserify-0.0.4" = {
+    "vm-browserify-1.1.0" = {
       name = "vm-browserify";
       packageName = "vm-browserify";
-      version = "0.0.4";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/vm-browserify/-/vm-browserify-0.0.4.tgz";
-        sha1 = "5d7ea45bbef9e4a6ff65f95438e0a87c357d5a73";
+      version = "1.1.0";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/vm-browserify/-/vm-browserify-1.1.0.tgz";
+        sha512 = "iq+S7vZJE60yejDYM0ek6zg308+UZsdtPExWP9VZoCFCz1zkJoXFnAX7aZfd/ZwrkidzdUZL0C/ryW+JwAiIGw==";
       };
     };
     "watchpack-1.6.0" = {
@@ -7331,13 +7457,13 b' let'
         sha1 = "fc571588c8558da77be9efb6debdc5a3b172bdc2";
       };
     };
-    "webpack-sources-1.3.0" = {
+    "webpack-sources-1.4.3" = {
       name = "webpack-sources";
       packageName = "webpack-sources";
-      version = "1.3.0";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/webpack-sources/-/webpack-sources-1.3.0.tgz";
-        sha512 = "OiVgSrbGu7NEnEvQJJgdSFPl2qWKkWq5lHMhgiToIiN9w34EBnjYzSYs+VbL5KoYiLNtFFa7BZIKxRED3I32pA==";
+      version = "1.4.3";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/webpack-sources/-/webpack-sources-1.4.3.tgz";
+        sha512 = "lgTS3Xhv1lCOKo7SA5TjKXMjpSM4sBjNV5+q2bqesbSPs5FjGmU6jjtBSkX9b4qW87vDIsCIlUPOEhbZrMdjeQ==";
       };
     };
     "webpack-uglify-js-plugin-1.1.9" = {
@@ -7430,13 +7556,13 b' let'
         sha1 = "b5243d8f3ec1aa35f1364605bc0d1036e30ab69f";
       };
     };
-    "xtend-4.0.1" = {
+    "xtend-4.0.2" = {
       name = "xtend";
       packageName = "xtend";
-      version = "4.0.1";
-      src = fetchurl {
-        url = "https://registry.npmjs.org/xtend/-/xtend-4.0.1.tgz";
-        sha1 = "a5c6d532be656e23db820efb943a1f04998d63af";
+      version = "4.0.2";
+      src = fetchurl {
+        url = "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz";
+        sha512 = "LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==";
       };
     };
     "y18n-4.0.0" = {
@@ -7514,9 +7640,9 b' let'
       sources."@polymer/paper-toast-3.0.1"
       sources."@polymer/paper-toggle-button-3.0.1"
       sources."@polymer/paper-tooltip-3.0.1"
-      sources."@polymer/polymer-3.2.0"
+      sources."@polymer/polymer-3.3.0"
       sources."@types/clone-0.1.30"
-      sources."@types/node-6.14.6"
+      sources."@types/node-6.14.9"
       sources."@types/parse5-2.2.34"
       sources."@webassemblyjs/ast-1.7.10"
       sources."@webassemblyjs/floating-point-hex-parser-1.7.10"
@@ -7536,8 +7662,8 b' let'
       sources."@webassemblyjs/wasm-parser-1.7.10"
       sources."@webassemblyjs/wast-parser-1.7.10"
       sources."@webassemblyjs/wast-printer-1.7.10"
-      sources."@webcomponents/shadycss-1.9.1"
-      sources."@webcomponents/webcomponentsjs-2.2.10"
+      sources."@webcomponents/shadycss-1.9.2"
+      sources."@webcomponents/webcomponentsjs-2.3.0"
       sources."@xtuc/ieee754-1.2.0"
       sources."@xtuc/long-4.2.1"
       sources."abbrev-1.1.1"
@@ -7549,7 +7675,7 b' let'
         ];
       })
       sources."ajv-4.11.8"
-      sources."ajv-keywords-3.4.0"
+      sources."ajv-keywords-3.4.1"
       (sources."align-text-0.1.4" // {
         dependencies = [
           sources."kind-of-3.2.2"
@@ -7615,20 +7741,20 b' let'
       (sources."babel-core-6.26.3" // {
         dependencies = [
           sources."json5-0.5.1"
-          sources."lodash-4.17.11"
+          sources."lodash-4.17.15"
           sources."minimatch-3.0.4"
         ];
       })
       (sources."babel-generator-6.26.1" // {
         dependencies = [
-          sources."lodash-4.17.11"
+          sources."lodash-4.17.15"
         ];
       })
       sources."babel-helper-builder-binary-assignment-operator-visitor-6.24.1"
       sources."babel-helper-call-delegate-6.24.1"
       (sources."babel-helper-define-map-6.26.0" // {
         dependencies = [
-          sources."lodash-4.17.11"
+          sources."lodash-4.17.15"
         ];
       })
       sources."babel-helper-explode-assignable-expression-6.24.1"
@@ -7638,7 +7764,7 b' let'
       sources."babel-helper-optimise-call-expression-6.24.1"
       (sources."babel-helper-regex-6.26.0" // {
         dependencies = [
-          sources."lodash-4.17.11"
+          sources."lodash-4.17.15"
         ];
       })
       sources."babel-helper-remap-async-to-generator-6.24.1"
@@ -7656,7 +7782,7 b' let'
       sources."babel-plugin-transform-es2015-block-scoped-functions-6.22.0"
       (sources."babel-plugin-transform-es2015-block-scoping-6.26.0" // {
         dependencies = [
-          sources."lodash-4.17.11"
+          sources."lodash-4.17.15"
         ];
       })
       sources."babel-plugin-transform-es2015-classes-6.24.1"
@@ -7685,23 +7811,23 b' let'
       sources."babel-preset-env-1.7.0"
       (sources."babel-register-6.26.0" // {
         dependencies = [
-          sources."lodash-4.17.11"
+          sources."lodash-4.17.15"
         ];
       })
       sources."babel-runtime-6.26.0"
       (sources."babel-template-6.26.0" // {
         dependencies = [
-          sources."lodash-4.17.11"
+          sources."lodash-4.17.15"
         ];
       })
       (sources."babel-traverse-6.26.0" // {
         dependencies = [
-          sources."lodash-4.17.11"
+          sources."lodash-4.17.15"
         ];
       })
       (sources."babel-types-6.26.0" // {
         dependencies = [
-          sources."lodash-4.17.11"
+          sources."lodash-4.17.15"
         ];
       })
       sources."babylon-6.18.0"
@@ -7711,11 +7837,11 b' let'
           sources."define-property-1.0.0"
         ];
       })
-      sources."base64-js-1.3.0"
+      sources."base64-js-1.3.1"
       sources."bcrypt-pbkdf-1.0.2"
       sources."big.js-5.2.2"
       sources."binary-extensions-1.13.1"
-      sources."bluebird-3.5.4"
+      sources."bluebird-3.7.1"
       sources."bn.js-4.11.8"
       sources."boolbase-1.0.0"
       sources."boom-2.10.1"
@@ -7739,11 +7865,11 b' let'
       sources."builtin-status-codes-3.0.0"
       (sources."cacache-10.0.4" // {
         dependencies = [
-          sources."glob-7.1.4"
-          sources."graceful-fs-4.1.15"
+          sources."glob-7.1.5"
+          sources."graceful-fs-4.2.3"
           sources."lru-cache-4.1.5"
           sources."minimatch-3.0.4"
-          sources."rimraf-2.6.3"
+          sources."rimraf-2.7.1"
         ];
       })
       sources."cache-base-1.0.1"
@@ -7754,18 +7880,18 b' let'
           sources."browserslist-1.7.7"
         ];
       })
-      sources."caniuse-db-1.0.30000967"
-      sources."caniuse-lite-1.0.30000967"
+      sources."caniuse-db-1.0.30001006"
+      sources."caniuse-lite-1.0.30001006"
       sources."caseless-0.12.0"
       sources."center-align-0.1.3"
       sources."chalk-0.5.1"
-      (sources."chokidar-2.1.5" // {
+      (sources."chokidar-2.1.8" // {
         dependencies = [
           sources."is-glob-4.0.1"
         ];
       })
-      sources."chownr-1.1.1"
-      sources."chrome-trace-event-1.0.0"
+      sources."chownr-1.1.3"
+      sources."chrome-trace-event-1.0.2"
       sources."cipher-base-1.0.4"
       (sources."clap-1.2.3" // {
         dependencies = [
@@ -7801,7 +7927,7 b' let'
       })
       (sources."cli-1.0.1" // {
         dependencies = [
-          sources."glob-7.1.4"
+          sources."glob-7.1.5"
           sources."minimatch-3.0.4"
         ];
       })
@@ -7825,24 +7951,29 b' let'
       sources."colormin-1.1.2"
       sources."colors-0.6.2"
       sources."combined-stream-1.0.8"
-      sources."commander-2.14.1"
+      sources."commander-2.20.3"
       sources."commondir-1.0.1"
       sources."component-emitter-1.3.0"
       sources."concat-map-0.0.1"
       (sources."concat-stream-1.6.2" // {
         dependencies = [
           sources."readable-stream-2.3.6"
+          sources."safe-buffer-5.1.2"
           sources."string_decoder-1.1.1"
         ];
       })
       sources."console-browserify-1.1.0"
       sources."constants-browserify-1.0.0"
-      sources."convert-source-map-1.6.0"
+      (sources."convert-source-map-1.6.0" // {
+        dependencies = [
+          sources."safe-buffer-5.1.2"
+        ];
+      })
       (sources."copy-concurrently-1.0.5" // {
         dependencies = [
-          sources."glob-7.1.4"
+          sources."glob-7.1.5"
           sources."minimatch-3.0.4"
-          sources."rimraf-2.6.3"
+          sources."rimraf-2.7.1"
         ];
       })
       sources."copy-descriptor-0.1.1"
@@ -7852,7 +7983,7 b' let'
           sources."minimatch-3.0.4"
         ];
       })
-      sources."core-js-2.6.5"
+      sources."core-js-2.6.10"
       sources."core-util-is-1.0.2"
       sources."create-ecdh-4.0.3"
       sources."create-hash-1.2.0"
@@ -7876,7 +8007,7 b' let'
       sources."cssesc-0.1.0"
       sources."cssnano-3.10.0"
       sources."csso-2.3.2"
-      sources."cyclist-0.2.2"
+      sources."cyclist-1.0.1"
       (sources."dashdash-1.14.1" // {
         dependencies = [
           sources."assert-plus-1.0.0"
@@ -7899,9 +8030,10 b' let'
       sources."diffie-hellman-5.0.3"
       sources."dir-glob-2.2.2"
       sources."dom-converter-0.2.0"
-      (sources."dom-serializer-0.1.1" // {
-        dependencies = [
-          sources."entities-1.1.2"
+      (sources."dom-serializer-0.2.1" // {
+        dependencies = [
+          sources."domelementtype-2.0.1"
+          sources."entities-2.0.0"
         ];
       })
       (sources."dom5-2.3.0" // {
@@ -7915,25 +8047,31 b' let'
       sources."domhandler-2.3.0"
       sources."domutils-1.5.1"
       sources."dropzone-5.5.1"
+      sources."duplexer-0.1.1"
       (sources."duplexify-3.7.1" // {
         dependencies = [
           sources."readable-stream-2.3.6"
+          sources."safe-buffer-5.1.2"
           sources."string_decoder-1.1.1"
         ];
       })
       sources."ecc-jsbn-0.1.2"
-      sources."electron-to-chromium-1.3.133"
-      sources."elliptic-6.4.1"
+      sources."electron-to-chromium-1.3.302"
+      sources."elliptic-6.5.1"
       sources."emojis-list-2.1.0"
-      sources."end-of-stream-1.4.1"
-      (sources."enhanced-resolve-4.1.0" // {
-        dependencies = [
-          sources."graceful-fs-4.1.15"
+      sources."end-of-stream-1.4.4"
+      (sources."enhanced-resolve-4.1.1" // {
+        dependencies = [
+          sources."graceful-fs-4.2.3"
+          sources."memory-fs-0.5.0"
+          sources."readable-stream-2.3.6"
+          sources."safe-buffer-5.1.2"
+          sources."string_decoder-1.1.1"
         ];
       })
       sources."entities-1.0.0"
       sources."errno-0.1.7"
-      sources."es-abstract-1.13.0"
+      sources."es-abstract-1.16.0"
       sources."es-to-primitive-1.2.0"
       sources."es6-templates-0.2.3"
       sources."escape-string-regexp-1.0.5"
@@ -7941,8 +8079,8 b' let'
       sources."espree-3.5.4"
       sources."esprima-1.0.4"
       sources."esrecurse-4.2.1"
-      sources."estraverse-4.2.0"
-      sources."esutils-2.0.2"
+      sources."estraverse-4.3.0"
+      sources."esutils-2.0.3"
       sources."eventemitter2-0.4.14"
       sources."events-3.0.0"
       sources."evp_bytestokey-1.0.3"
@@ -7986,6 +8124,7 b' let'
       sources."fastparse-1.1.2"
       sources."favico.js-0.3.10"
       sources."faye-websocket-0.4.4"
+      sources."figures-1.7.0"
       sources."file-sync-cmp-0.1.1"
       (sources."fill-range-4.0.0" // {
         dependencies = [
@@ -8003,10 +8142,11 b' let'
       })
       sources."fined-1.2.0"
       sources."flagged-respawn-1.0.1"
-      sources."flatten-1.0.2"
+      sources."flatten-1.0.3"
       (sources."flush-write-stream-1.1.1" // {
         dependencies = [
           sources."readable-stream-2.3.6"
+          sources."safe-buffer-5.1.2"
           sources."string_decoder-1.1.1"
         ];
       })
@@ -8018,12 +8158,13 b' let'
       (sources."from2-2.3.0" // {
         dependencies = [
           sources."readable-stream-2.3.6"
+          sources."safe-buffer-5.1.2"
           sources."string_decoder-1.1.1"
         ];
       })
       (sources."fs-write-stream-atomic-1.0.10" // {
         dependencies = [
-          sources."graceful-fs-4.1.15"
+          sources."graceful-fs-4.2.3"
         ];
       })
       sources."fs.realpath-1.0.0"
@@ -8059,7 +8200,7 b' let'
       sources."globals-9.18.0"
       (sources."globby-7.1.1" // {
         dependencies = [
-          sources."glob-7.1.4"
+          sources."glob-7.1.5"
           sources."minimatch-3.0.4"
         ];
       })
@@ -8094,7 +8235,7 b' let'
       (sources."grunt-contrib-jshint-0.12.0" // {
         dependencies = [
           sources."jshint-2.9.7"
-          sources."lodash-4.17.11"
+          sources."lodash-4.17.15"
           sources."minimatch-3.0.4"
         ];
       })
@@ -8102,14 +8243,21 b' let'
         dependencies = [
           sources."ansi-regex-2.1.1"
           sources."ansi-styles-2.2.1"
-          sources."async-2.6.2"
+          sources."async-2.6.3"
           sources."chalk-1.1.3"
           sources."has-ansi-2.0.0"
-          sources."lodash-4.17.11"
+          sources."lodash-4.17.15"
           sources."strip-ansi-3.0.1"
           sources."supports-color-2.0.0"
         ];
       })
+      (sources."grunt-contrib-uglify-4.0.1" // {
+        dependencies = [
+          sources."ansi-styles-3.2.1"
+          sources."chalk-2.4.2"
+          sources."supports-color-5.5.0"
+        ];
+      })
       (sources."grunt-contrib-watch-0.6.1" // {
         dependencies = [
           sources."async-0.2.10"
@@ -8132,9 +8280,10 b' let'
       sources."grunt-legacy-util-0.2.0"
       (sources."grunt-webpack-3.1.3" // {
         dependencies = [
-          sources."lodash-4.17.11"
-        ];
-      })
+          sources."lodash-4.17.15"
+        ];
+      })
+      sources."gzip-size-3.0.0"
       sources."har-schema-1.0.5"
       sources."har-validator-4.2.1"
       sources."has-1.0.3"
@@ -8161,6 +8310,12 b' let'
       (sources."html-minifier-3.5.21" // {
         dependencies = [
           sources."commander-2.17.1"
+          sources."source-map-0.6.1"
+          (sources."uglify-js-3.4.10" // {
+            dependencies = [
+              sources."commander-2.19.0"
+            ];
+          })
         ];
       })
       (sources."html-webpack-plugin-3.2.0" // {
@@ -8168,7 +8323,7 b' let'
           sources."big.js-3.2.0"
           sources."json5-0.5.1"
           sources."loader-utils-0.2.17"
-          sources."lodash-4.17.11"
+          sources."lodash-4.17.15"
         ];
       })
       sources."htmlparser2-3.8.3"
@@ -8193,7 +8348,7 b' let'
         dependencies = [
           sources."find-up-3.0.0"
           sources."locate-path-3.0.0"
-          sources."p-limit-2.2.0"
+          sources."p-limit-2.2.1"
           sources."p-locate-3.0.0"
           sources."p-try-2.2.0"
           sources."pkg-dir-3.0.0"
@@ -8202,9 +8357,8 b' let'
       sources."imports-loader-0.7.1"
       sources."imurmurhash-0.1.4"
       sources."indexes-of-1.0.1"
-      sources."indexof-0.0.1"
       sources."inflight-1.0.6"
-      sources."inherits-2.0.3"
+      sources."inherits-2.0.4"
       sources."ini-1.3.5"
       sources."interpret-1.1.0"
       sources."invariant-2.2.4"
@@ -8250,7 +8404,7 b' let'
       sources."jsesc-1.3.0"
       (sources."jshint-2.10.2" // {
         dependencies = [
-          sources."lodash-4.17.11"
+          sources."lodash-4.17.15"
           sources."minimatch-3.0.4"
         ];
       })
@@ -8271,7 +8425,7 b' let'
       sources."lcid-2.0.0"
       (sources."less-2.7.3" // {
         dependencies = [
-          sources."graceful-fs-4.1.15"
+          sources."graceful-fs-4.2.3"
         ];
       })
       (sources."liftoff-2.5.0" // {
@@ -8298,11 +8452,22 b' let'
       sources."map-visit-1.0.0"
       sources."mark.js-8.11.1"
       sources."math-expression-evaluator-1.2.17"
+      (sources."maxmin-2.1.0" // {
+        dependencies = [
+          sources."ansi-regex-2.1.1"
+          sources."ansi-styles-2.2.1"
+          sources."chalk-1.1.3"
+          sources."has-ansi-2.0.0"
+          sources."strip-ansi-3.0.1"
+          sources."supports-color-2.0.0"
+        ];
+      })
       sources."md5.js-1.3.5"
       sources."mem-4.3.0"
       (sources."memory-fs-0.4.1" // {
         dependencies = [
           sources."readable-stream-2.3.6"
+          sources."safe-buffer-5.1.2"
           sources."string_decoder-1.1.1"
         ];
       })
@@ -8317,7 +8482,7 b' let'
       sources."minimatch-0.2.14"
       sources."minimist-1.2.0"
       sources."mississippi-2.0.0"
-      (sources."mixin-deep-1.3.1" // {
+      (sources."mixin-deep-1.3.2" // {
         dependencies = [
           sources."is-extendable-1.0.1"
         ];
@@ -8331,25 +8496,30 b' let'
       sources."mousetrap-1.6.3"
       (sources."move-concurrently-1.0.1" // {
         dependencies = [
-          sources."glob-7.1.4"
+          sources."glob-7.1.5"
           sources."minimatch-3.0.4"
-          sources."rimraf-2.6.3"
+          sources."rimraf-2.7.1"
         ];
       })
       sources."ms-2.0.0"
-      sources."nan-2.13.2"
+      sources."nan-2.14.0"
       sources."nanomatch-1.2.13"
       sources."neo-async-2.6.1"
       sources."nice-try-1.0.5"
       sources."no-case-2.3.2"
-      (sources."node-libs-browser-2.2.0" // {
+      (sources."node-libs-browser-2.2.1" // {
         dependencies = [
           (sources."readable-stream-2.3.6" // {
             dependencies = [
               sources."string_decoder-1.1.1"
             ];
           })
-          sources."string_decoder-1.2.0"
+          sources."safe-buffer-5.1.2"
+          (sources."string_decoder-1.3.0" // {
+            dependencies = [
+              sources."safe-buffer-5.2.0"
+            ];
+          })
         ];
       })
       sources."nopt-1.0.10"
@@ -8380,6 +8550,7 b' let'
           sources."kind-of-3.2.2"
         ];
       })
+      sources."object-inspect-1.6.0"
       sources."object-keys-1.1.1"
       sources."object-visit-1.0.1"
       sources."object.defaults-1.1.0"
@@ -8399,14 +8570,15 b' let'
       sources."p-locate-2.0.0"
       sources."p-try-1.0.0"
       sources."pako-1.0.10"
-      (sources."parallel-transform-1.1.0" // {
+      (sources."parallel-transform-1.2.0" // {
         dependencies = [
           sources."readable-stream-2.3.6"
+          sources."safe-buffer-5.1.2"
           sources."string_decoder-1.1.1"
         ];
       })
       sources."param-case-2.1.1"
-      sources."parse-asn1-5.1.4"
+      sources."parse-asn1-5.1.5"
       sources."parse-filepath-1.0.2"
       sources."parse-passwd-1.0.0"
       sources."parse5-3.0.3"
@@ -8416,7 +8588,7 b' let'
         ];
       })
       sources."pascalcase-0.1.1"
-      sources."path-browserify-0.0.0"
+      sources."path-browserify-0.0.1"
       sources."path-dirname-1.0.2"
       sources."path-exists-3.0.0"
       sources."path-is-absolute-1.0.1"
@@ -8527,10 +8699,11 b' let'
       sources."postcss-value-parser-3.3.1"
       sources."postcss-zindex-2.2.0"
       sources."prepend-http-1.0.4"
+      sources."pretty-bytes-3.0.1"
       sources."pretty-error-2.1.1"
       sources."private-0.1.8"
       sources."process-0.11.10"
-      sources."process-nextick-args-2.0.0"
+      sources."process-nextick-args-2.0.1"
       sources."promise-7.3.1"
       sources."promise-inflight-1.0.1"
       sources."prr-1.0.1"
@@ -8555,8 +8728,9 b' let'
       })
       (sources."readdirp-2.2.1" // {
         dependencies = [
-          sources."graceful-fs-4.1.15"
+          sources."graceful-fs-4.2.3"
           sources."readable-stream-2.3.6"
+          sources."safe-buffer-5.1.2"
           sources."string_decoder-1.1.1"
         ];
       })
@@ -8571,11 +8745,7 b' let'
           sources."balanced-match-0.4.2"
         ];
       })
-      (sources."reduce-function-call-1.0.2" // {
-        dependencies = [
-          sources."balanced-match-0.4.2"
-        ];
-      })
+      sources."reduce-function-call-1.0.3"
       sources."regenerate-1.4.0"
       sources."regenerator-runtime-0.11.1"
       sources."regenerator-transform-0.10.1"
@@ -8601,7 +8771,7 b' let'
       sources."request-2.81.0"
       sources."require-directory-2.1.1"
       sources."require-main-filename-1.0.1"
-      sources."resolve-1.10.1"
+      sources."resolve-1.12.0"
       sources."resolve-cwd-2.0.0"
       sources."resolve-dir-1.0.1"
       sources."resolve-from-3.0.0"
@@ -8611,20 +8781,20 b' let'
       sources."rimraf-2.2.8"
       sources."ripemd160-2.0.2"
       sources."run-queue-1.0.3"
-      sources."safe-buffer-5.1.2"
+      sources."safe-buffer-5.2.0"
       sources."safe-regex-1.1.0"
       sources."safer-buffer-2.1.2"
       sources."sax-1.2.4"
       (sources."schema-utils-0.4.7" // {
         dependencies = [
-          sources."ajv-6.10.0"
+          sources."ajv-6.10.2"
         ];
       })
       sources."select-1.1.2"
-      sources."semver-5.7.0"
-      sources."serialize-javascript-1.7.0"
+      sources."semver-5.7.1"
+      sources."serialize-javascript-1.9.1"
       sources."set-blocking-2.0.0"
-      (sources."set-value-2.0.0" // {
+      (sources."set-value-2.0.1" // {
         dependencies = [
           sources."extend-shallow-2.0.1"
         ];
@@ -8701,6 +8871,7 b' let'
       (sources."stream-browserify-2.0.2" // {
         dependencies = [
           sources."readable-stream-2.3.6"
+          sources."safe-buffer-5.1.2"
           sources."string_decoder-1.1.1"
         ];
       })
@@ -8708,6 +8879,7 b' let'
       (sources."stream-http-2.8.3" // {
         dependencies = [
           sources."readable-stream-2.3.6"
+          sources."safe-buffer-5.1.2"
           sources."string_decoder-1.1.1"
         ];
       })
@@ -8720,6 +8892,8 b' let'
           sources."strip-ansi-4.0.0"
         ];
       })
+      sources."string.prototype.trimleft-2.1.0"
+      sources."string.prototype.trimright-2.1.0"
       sources."string_decoder-0.10.31"
       sources."stringstream-0.0.6"
       sources."strip-ansi-0.3.0"
@@ -8740,10 +8914,11 b' let'
       (sources."through2-2.0.5" // {
         dependencies = [
           sources."readable-stream-2.3.6"
+          sources."safe-buffer-5.1.2"
           sources."string_decoder-1.1.1"
         ];
       })
-      sources."timers-browserify-2.0.10"
+      sources."timers-browserify-2.0.11"
       sources."tiny-emitter-2.1.0"
       (sources."tiny-lr-fork-0.0.5" // {
         dependencies = [
@@ -8766,27 +8941,27 b' let'
       (sources."ts-loader-1.3.3" // {
         dependencies = [
           sources."big.js-3.2.0"
-          sources."colors-1.3.3"
+          sources."colors-1.4.0"
           sources."enhanced-resolve-3.4.1"
-          sources."graceful-fs-4.1.15"
+          sources."graceful-fs-4.2.3"
           sources."json5-0.5.1"
           sources."loader-utils-0.2.17"
           sources."tapable-0.2.9"
         ];
       })
-      sources."tslib-1.9.3"
+      sources."tslib-1.10.0"
       sources."tty-browserify-0.0.0"
       sources."tunnel-agent-0.6.0"
       sources."tweetnacl-0.14.5"
       sources."typedarray-0.0.6"
       (sources."uglify-es-3.3.10" // {
         dependencies = [
+          sources."commander-2.14.1"
           sources."source-map-0.6.1"
         ];
       })
-      (sources."uglify-js-3.4.10" // {
-        dependencies = [
-          sources."commander-2.19.0"
+      (sources."uglify-js-3.6.7" // {
+        dependencies = [
           sources."source-map-0.6.1"
         ];
       })
@@ -8799,16 +8974,11 b' let'
       sources."unc-path-regex-0.1.2"
       sources."underscore-1.7.0"
       sources."underscore.string-2.2.1"
-      (sources."union-value-1.0.0" // {
-        dependencies = [
-          sources."extend-shallow-2.0.1"
-          sources."set-value-0.4.3"
-        ];
-      })
+      sources."union-value-1.0.1"
       sources."uniq-1.0.1"
       sources."uniqs-2.0.0"
       sources."unique-filename-1.1.1"
-      sources."unique-slug-2.0.1"
+      sources."unique-slug-2.0.2"
       (sources."unset-value-1.0.0" // {
         dependencies = [
           (sources."has-value-0.3.1" // {
@@ -8819,13 +8989,14 b' let'
           sources."has-values-0.1.4"
         ];
       })
-      sources."upath-1.1.2"
+      sources."upath-1.2.0"
       sources."upper-case-1.1.3"
       (sources."uri-js-4.2.2" // {
         dependencies = [
           sources."punycode-2.1.1"
         ];
       })
+      sources."uri-path-1.0.0"
       sources."urix-0.1.0"
       (sources."url-0.11.0" // {
         dependencies = [
@@ -8833,12 +9004,16 b' let'
         ];
       })
       sources."use-3.1.1"
-      sources."util-0.11.1"
+      (sources."util-0.11.1" // {
+        dependencies = [
+          sources."inherits-2.0.3"
+        ];
+      })
       sources."util-deprecate-1.0.2"
       sources."util.promisify-1.0.0"
       sources."utila-0.4.0"
-      sources."uuid-3.3.2"
-      sources."v8-compile-cache-2.0.3"
+      sources."uuid-3.3.3"
+      sources."v8-compile-cache-2.1.0"
       sources."v8flags-3.1.3"
       sources."vendors-1.0.3"
       (sources."verror-1.10.0" // {
@@ -8846,16 +9021,16 b' let'
           sources."assert-plus-1.0.0"
         ];
       })
-      sources."vm-browserify-0.0.4"
+      sources."vm-browserify-1.1.0"
       (sources."watchpack-1.6.0" // {
         dependencies = [
-          sources."graceful-fs-4.1.15"
+          sources."graceful-fs-4.2.3"
         ];
       })
       sources."waypoints-4.0.1"
       (sources."webpack-4.23.1" // {
         dependencies = [
-          sources."ajv-6.10.0"
+          sources."ajv-6.10.2"
         ];
       })
       (sources."webpack-cli-3.1.2" // {
@@ -8871,7 +9046,7 b' let'
           sources."source-map-0.4.4"
         ];
       })
-      (sources."webpack-sources-1.3.0" // {
+      (sources."webpack-sources-1.4.3" // {
         dependencies = [
           sources."source-map-0.6.1"
         ];
@@ -8904,14 +9079,14 b' let'
         ];
       })
       sources."wrappy-1.0.2"
-      sources."xtend-4.0.1"
+      sources."xtend-4.0.2"
       sources."y18n-4.0.0"
       sources."yallist-2.1.2"
       (sources."yargs-12.0.5" // {
         dependencies = [
           sources."find-up-3.0.0"
           sources."locate-path-3.0.0"
-          sources."p-limit-2.2.0"
+          sources."p-limit-2.2.1"
           sources."p-locate-3.0.0"
           sources."p-try-2.2.0"
         ];

pkgs/python-packages.nix

@@ -5,7 +5,7 b''
 
 self: super: {
   "alembic" = super.buildPythonPackage {
-    name = "alembic-1.0.10";
+    name = "alembic-1.3.1";
     doCheck = false;
     propagatedBuildInputs = [
       self."sqlalchemy"
@@ -14,22 +14,22 b' self: super: {'
       self."python-dateutil"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/6e/8b/fa3bd058cccd5e9177fea4efa26bfb769228fdd3178436ad5e05830ef6ef/alembic-1.0.10.tar.gz";
-      sha256 = "1dwl0264r6ri2jyrjr68am04x538ab26xwy4crqjnnhm4alwm3c2";
+      url = "https://files.pythonhosted.org/packages/84/64/493c45119dce700a4b9eeecc436ef9e8835ab67bae6414f040cdc7b58f4b/alembic-1.3.1.tar.gz";
+      sha256 = "1cl2chk5jx0rf4hmsd5lljic7iifw17yv3y5xawvp4i14jvpn9s9";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
     };
   };
   "amqp" = super.buildPythonPackage {
-    name = "amqp-2.3.1";
+    name = "amqp-2.5.2";
     doCheck = false;
     propagatedBuildInputs = [
       self."vine"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/1b/32/242ff76cd802766f11c89c72f3389b5c8de4bdfbab406137b90c5fae8b05/amqp-2.3.1.tar.gz";
-      sha256 = "0wlfnvhmfrn7c8qif2jyvsm63ibdxp02ss564qwrvqfhz0di72s0";
+      url = "https://files.pythonhosted.org/packages/92/1d/433541994a5a69f4ad2fff39746ddbb0bdedb0ea0d85673eb0db68a7edd9/amqp-2.5.2.tar.gz";
+      sha256 = "13dhhfxjrqcjybnq4zahg92mydhpg2l76nxcmq7d560687wsxwbp";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal ];
@@ -63,33 +63,22 b' self: super: {'
     };
   };
   "atomicwrites" = super.buildPythonPackage {
-    name = "atomicwrites-1.2.1";
+    name = "atomicwrites-1.3.0";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/ac/ed/a311712ef6b4355035489f665e63e1a73f9eb371929e3c98e5efd451069e/atomicwrites-1.2.1.tar.gz";
-      sha256 = "1vmkbw9j0qammwxbxycrs39gvdg4lc2d4lk98kwf8ag2manyi6pc";
+      url = "https://files.pythonhosted.org/packages/ec/0f/cd484ac8820fed363b374af30049adc8fd13065720fd4f4c6be8a2309da7/atomicwrites-1.3.0.tar.gz";
+      sha256 = "19ngcscdf3jsqmpcxn6zl5b6anmsajb6izp1smcd1n02midl9abm";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
     };
   };
   "attrs" = super.buildPythonPackage {
-    name = "attrs-18.2.0";
+    name = "attrs-19.3.0";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/0f/9e/26b1d194aab960063b266170e53c39f73ea0d0d3f5ce23313e0ec8ee9bdf/attrs-18.2.0.tar.gz";
-      sha256 = "0s9ydh058wmmf5v391pym877x4ahxg45dw6a0w4c7s5wgpigdjqh";
-    };
-    meta = {
-      license = [ pkgs.lib.licenses.mit ];
-    };
-  };
-  "authomatic" = super.buildPythonPackage {
-    name = "authomatic-0.1.0.post1";
-    doCheck = false;
-    src = fetchurl {
-      url = "https://code.rhodecode.com/upstream/authomatic/artifacts/download/0-4fe9c041-a567-4f84-be4c-7efa2a606d3c.tar.gz?md5=f6bdc3c769688212db68233e8d2b0383";
-      sha256 = "0pc716mva0ym6xd8jwzjbjp8dqxy9069wwwv2aqwb8lyhl4757ab";
+      url = "https://files.pythonhosted.org/packages/98/c3/2c227e66b5e896e15ccdae2e00bbc69aa46e9a8ce8869cc5fa96310bf612/attrs-19.3.0.tar.gz";
+      sha256 = "0wky4h28n7xnr6xv69p9z6kv8bzn50d10c3drmd9ds8gawbcxdzp";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
@@ -146,11 +135,11 b' self: super: {'
     };
   };
   "billiard" = super.buildPythonPackage {
-    name = "billiard-3.5.0.3";
+    name = "billiard-3.6.1.0";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/39/ac/f5571210cca2e4f4532e38aaff242f26c8654c5e2436bee966c230647ccc/billiard-3.5.0.3.tar.gz";
-      sha256 = "1riwiiwgb141151md4ykx49qrz749akj5k8g290ji9bsqjyj4yqx";
+      url = "https://files.pythonhosted.org/packages/68/1d/2aea8fbb0b1e1260a8a2e77352de2983d36d7ac01207cf14c2b9c6cc860e/billiard-3.6.1.0.tar.gz";
+      sha256 = "09hzy3aqi7visy4vmf4xiish61n0rq5nd3iwjydydps8yrs9r05q";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal ];
@@ -183,30 +172,42 b' self: super: {'
     };
   };
   "celery" = super.buildPythonPackage {
-    name = "celery-4.1.1";
+    name = "celery-4.3.0";
     doCheck = false;
     propagatedBuildInputs = [
       self."pytz"
       self."billiard"
       self."kombu"
+      self."vine"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/e9/cf/a4c0597effca20c57eb586324e41d1180bc8f13a933da41e0646cff69f02/celery-4.1.1.tar.gz";
-      sha256 = "1xbir4vw42n2ir9lanhwl7w69zpmj7lbi66fxm2b7pyvkcss7wni";
+      url = "https://files.pythonhosted.org/packages/a2/4b/d020836f751617e907e84753a41c92231cd4b673ff991b8ee9da52361323/celery-4.3.0.tar.gz";
+      sha256 = "1y8y0gbgkwimpxqnxq2rm5qz2vy01fvjiybnpm00y5rzd2m34iac";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal ];
     };
   };
+  "certifi" = super.buildPythonPackage {
+    name = "certifi-2019.11.28";
+    doCheck = false;
+    src = fetchurl {
+      url = "https://files.pythonhosted.org/packages/41/bf/9d214a5af07debc6acf7f3f257265618f1db242a3f8e49a9b516f24523a6/certifi-2019.11.28.tar.gz";
+      sha256 = "07qg6864bk4qxa8akr967amlmsq9v310hp039mcpjx6dliylrdi5";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.mpl20 { fullName = "Mozilla Public License 2.0 (MPL 2.0)"; } ];
+    };
+  };
   "cffi" = super.buildPythonPackage {
-    name = "cffi-1.12.2";
+    name = "cffi-1.12.3";
     doCheck = false;
     propagatedBuildInputs = [
       self."pycparser"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/64/7c/27367b38e6cc3e1f49f193deb761fe75cda9f95da37b67b422e62281fcac/cffi-1.12.2.tar.gz";
-      sha256 = "19qfks2djya8vix95bmg3xzipjb8w9b8mbj4j5k2hqkc8j58f4z1";
+      url = "https://files.pythonhosted.org/packages/93/1a/ab8c62b5838722f29f3daffcc8d4bd61844aa9b5f437341cc890ceee483b/cffi-1.12.3.tar.gz";
+      sha256 = "0x075521fxwv0mfp4cqzk7lvmw4n94bjw601qkcv314z5s182704";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
@@ -243,6 +244,17 b' self: super: {'
       license = [ pkgs.lib.licenses.bsdOriginal ];
     };
   };
+  "chardet" = super.buildPythonPackage {
+    name = "chardet-3.0.4";
+    doCheck = false;
+    src = fetchurl {
+      url = "https://files.pythonhosted.org/packages/fc/bb/a5768c230f9ddb03acc9ef3f0d4a3cf93462473795d18e9535498c8f929d/chardet-3.0.4.tar.gz";
+      sha256 = "1bpalpia6r5x1kknbk11p1fzph56fmmnp405ds8icksd3knr5aw4";
+    };
+    meta = {
+      license = [ { fullName = "LGPL"; } { fullName = "GNU Library or Lesser General Public License (LGPL)"; } ];
+    };
+  };
   "click" = super.buildPythonPackage {
     name = "click-7.0";
     doCheck = false;
@@ -285,16 +297,27 b' self: super: {'
     };
   };
   "configparser" = super.buildPythonPackage {
-    name = "configparser-3.7.4";
+    name = "configparser-4.0.2";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/e2/1c/83fd53748d8245cb9a3399f705c251d3fc0ce7df04450aac1cfc49dd6a0f/configparser-3.7.4.tar.gz";
-      sha256 = "0xac32886ihs2xg7w1gppcq2sgin5qsm8lqwijs5xifq9w0x0q6s";
+      url = "https://files.pythonhosted.org/packages/16/4f/48975536bd488d3a272549eb795ac4a13a5f7fcdc8995def77fbef3532ee/configparser-4.0.2.tar.gz";
+      sha256 = "1priacxym85yjcf68hh38w55nqswaxp71ryjyfdk222kg9l85ln7";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
     };
   };
+  "contextlib2" = super.buildPythonPackage {
+    name = "contextlib2-0.6.0.post1";
+    doCheck = false;
+    src = fetchurl {
+      url = "https://files.pythonhosted.org/packages/02/54/669207eb72e3d8ae8b38aa1f0703ee87a0e9f88f30d3c0a47bebdb6de242/contextlib2-0.6.0.post1.tar.gz";
+      sha256 = "0bhnr2ac7wy5l85ji909gyljyk85n92w8pdvslmrvc8qih4r1x01";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.psfl ];
+    };
+  };
   "cov-core" = super.buildPythonPackage {
     name = "cov-core-1.15.0";
     doCheck = false;
@@ -310,11 +333,11 b' self: super: {'
     };
   };
   "coverage" = super.buildPythonPackage {
-    name = "coverage-4.5.3";
+    name = "coverage-4.5.4";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/82/70/2280b5b29a0352519bb95ab0ef1ea942d40466ca71c53a2085bdeff7b0eb/coverage-4.5.3.tar.gz";
-      sha256 = "02f6m073qdispn96rc616hg0rnmw1pgqzw3bgxwiwza4zf9hirlx";
+      url = "https://files.pythonhosted.org/packages/85/d5/818d0e603685c4a613d56f065a721013e942088047ff1027a632948bdae6/coverage-4.5.4.tar.gz";
+      sha256 = "0p0j4di6h8k6ica7jwwj09azdcg4ycxq60i9qsskmsg94cd9yzg0";
     };
     meta = {
       license = [ pkgs.lib.licenses.asl20 ];
@@ -361,7 +384,7 b' self: super: {'
     };
   };
   "deform" = super.buildPythonPackage {
-    name = "deform-2.0.7";
+    name = "deform-2.0.8";
     doCheck = false;
     propagatedBuildInputs = [
       self."chameleon"
@@ -372,8 +395,8 b' self: super: {'
       self."zope.deprecation"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/cf/a1/bc234527b8f181de9acd80e796483c00007658d1e32b7de78f1c2e004d9a/deform-2.0.7.tar.gz";
-      sha256 = "0jnpi0zr2hjvbmiz6nm33yqv976dn9lf51vhlzqc0i75xcr9rwig";
+      url = "https://files.pythonhosted.org/packages/21/d0/45fdf891a82722c02fc2da319cf2d1ae6b5abf9e470ad3762135a895a868/deform-2.0.8.tar.gz";
+      sha256 = "0wbjv98sib96649aqaygzxnrkclyy50qij2rha6fn1i4c86bfdl9";
     };
     meta = {
       license = [ { fullName = "Repoze Public License"; } { fullName = "BSD-derived (http://www.repoze.org/LICENSE.txt)"; } ];
@@ -417,14 +440,14 b' self: super: {'
     };
   };
   "dogpile.cache" = super.buildPythonPackage {
-    name = "dogpile.cache-0.7.1";
+    name = "dogpile.cache-0.9.0";
     doCheck = false;
     propagatedBuildInputs = [
       self."decorator"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/84/3e/dbf1cfc5228f1d3dca80ef714db2c5aaec5cd9efaf54d7e3daef6bc48b19/dogpile.cache-0.7.1.tar.gz";
-      sha256 = "0caazmrzhnfqb5yrp8myhw61ny637jj69wcngrpbvi31jlcpy6v9";
+      url = "https://files.pythonhosted.org/packages/ac/6a/9ac405686a94b7f009a20a50070a5786b0e1aedc707b88d40d0c4b51a82e/dogpile.cache-0.9.0.tar.gz";
+      sha256 = "0sr1fn6b4k5bh0cscd9yi8csqxvj4ngzildav58x5p694mc86j5k";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal ];
@@ -514,14 +537,14 b' self: super: {'
     };
   };
   "elasticsearch2" = super.buildPythonPackage {
-    name = "elasticsearch2-2.5.0";
+    name = "elasticsearch2-2.5.1";
     doCheck = false;
     propagatedBuildInputs = [
       self."urllib3"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/84/77/63cf63d4ba11d913b5278406f2a37b0712bec6fc85edfb6151a33eaeba25/elasticsearch2-2.5.0.tar.gz";
-      sha256 = "0ky0q16lbvz022yv6q3pix7aamf026p1y994537ccjf0p0dxnbxr";
+      url = "https://files.pythonhosted.org/packages/f6/09/f9b24aa6b1120bea371cd57ef6f57c7694cf16660469456a8be6c2bdbe22/elasticsearch2-2.5.1.tar.gz";
+      sha256 = "19k2znpjfyp0hrq73cz7pjyj289040xpsxsm0xhh4jfh6y551g7k";
     };
     meta = {
       license = [ pkgs.lib.licenses.asl20 ];
@@ -666,16 +689,44 b' self: super: {'
     };
   };
   "hupper" = super.buildPythonPackage {
-    name = "hupper-1.6.1";
+    name = "hupper-1.9.1";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/85/d9/e005d357b11249c5d70ddf5b7adab2e4c0da4e8b0531ff146917a04fe6c0/hupper-1.6.1.tar.gz";
-      sha256 = "0d3cvkc8ssgwk54wvhbifj56ry97qi10pfzwfk8vwzzcikbfp3zy";
+      url = "https://files.pythonhosted.org/packages/09/3a/4f215659f31eeffe364a984dba486bfa3907bfcc54b7013bdfe825cebb5f/hupper-1.9.1.tar.gz";
+      sha256 = "0pyg879fv9mbwlnbzw2a3234qqycqs9l97h5mpkmk0bvxhi2471v";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
     };
   };
+  "idna" = super.buildPythonPackage {
+    name = "idna-2.8";
+    doCheck = false;
+    src = fetchurl {
+      url = "https://files.pythonhosted.org/packages/ad/13/eb56951b6f7950cadb579ca166e448ba77f9d24efc03edd7e55fa57d04b7/idna-2.8.tar.gz";
+      sha256 = "01rlkigdxg17sf9yar1jl8n18ls59367wqh59hnawlyg53vb6my3";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.bsdOriginal { fullName = "BSD-like"; } ];
+    };
+  };
+  "importlib-metadata" = super.buildPythonPackage {
+    name = "importlib-metadata-0.23";
+    doCheck = false;
+    propagatedBuildInputs = [
+      self."zipp"
+      self."contextlib2"
+      self."configparser"
+      self."pathlib2"
+    ];
+    src = fetchurl {
+      url = "https://files.pythonhosted.org/packages/5d/44/636bcd15697791943e2dedda0dbe098d8530a38d113b202817133e0b06c0/importlib_metadata-0.23.tar.gz";
+      sha256 = "09mdqdfv5rdrwz80jh9m379gxmvk2vhjfz0fg53hid00icvxf65a";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.asl20 ];
+    };
+  };
   "infrae.cache" = super.buildPythonPackage {
     name = "infrae.cache-1.0.1";
     doCheck = false;
@@ -703,11 +754,11 b' self: super: {'
     };
   };
   "ipaddress" = super.buildPythonPackage {
-    name = "ipaddress-1.0.22";
+    name = "ipaddress-1.0.23";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/97/8d/77b8cedcfbf93676148518036c6b1ce7f8e14bf07e95d7fd4ddcb8cc052f/ipaddress-1.0.22.tar.gz";
-      sha256 = "0b570bm6xqpjwqis15pvdy6lyvvzfndjvkynilcddjj5x98wfimi";
+      url = "https://files.pythonhosted.org/packages/b9/9a/3e9da40ea28b8210dd6504d3fe9fe7e013b62bf45902b458d1cdc3c34ed9/ipaddress-1.0.23.tar.gz";
+      sha256 = "1qp743h30s04m3cg3yk3fycad930jv17q7dsslj4mfw0jlvf1y5p";
     };
     meta = {
       license = [ pkgs.lib.licenses.psfl ];
@@ -859,14 +910,15 b' self: super: {'
     };
   };
   "kombu" = super.buildPythonPackage {
-    name = "kombu-4.2.1";
+    name = "kombu-4.6.6";
     doCheck = false;
     propagatedBuildInputs = [
       self."amqp"
+      self."importlib-metadata"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/39/9f/556b988833abede4a80dbd18b2bdf4e8ff4486dd482ed45da961347e8ed2/kombu-4.2.1.tar.gz";
-      sha256 = "10lh3hncvw67fz0k5vgbx3yh9gjfpqdlia1f13i28cgnc1nfrbc6";
+      url = "https://files.pythonhosted.org/packages/20/e6/bc2d9affba6138a1dc143f77fef253e9e08e238fa7c0688d917c09005e96/kombu-4.6.6.tar.gz";
+      sha256 = "11mxpcy8mg1l35bgbhba70v29bydr2hrhdbdlb4lg98m3m5vaq0p";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal ];
@@ -884,14 +936,14 b' self: super: {'
     };
   };
   "mako" = super.buildPythonPackage {
-    name = "mako-1.0.7";
+    name = "mako-1.1.0";
     doCheck = false;
     propagatedBuildInputs = [
       self."markupsafe"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/eb/f3/67579bb486517c0d49547f9697e36582cd19dafb5df9e687ed8e22de57fa/Mako-1.0.7.tar.gz";
-      sha256 = "1bi5gnr8r8dva06qpyx4kgjc6spm2k1y908183nbbaylggjzs0jf";
+      url = "https://files.pythonhosted.org/packages/b0/3c/8dcd6883d009f7cae0f3157fb53e9afb05a0d3d33b3db1268ec2e6f4a56b/Mako-1.1.0.tar.gz";
+      sha256 = "0jqa3qfpykyn4fmkn0kh6043sfls7br8i2bsdbccazcvk9cijsd3";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
@@ -909,25 +961,14 b' self: super: {'
     };
   };
   "markupsafe" = super.buildPythonPackage {
-    name = "markupsafe-1.1.0";
+    name = "markupsafe-1.1.1";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/ac/7e/1b4c2e05809a4414ebce0892fe1e32c14ace86ca7d50c70f00979ca9b3a3/MarkupSafe-1.1.0.tar.gz";
-      sha256 = "1lxirjypbdd3l9jl4vliilhfnhy7c7f2vlldqg1b0i74khn375sf";
+      url = "https://files.pythonhosted.org/packages/b9/2e/64db92e53b86efccfaea71321f597fa2e1b2bd3853d8ce658568f7a13094/MarkupSafe-1.1.1.tar.gz";
+      sha256 = "0sqipg4fk7xbixqd8kq6rlkxj664d157bdwbh93farcphf92x1r9";
     };
     meta = {
-      license = [ pkgs.lib.licenses.bsdOriginal ];
-    };
-  };
-  "meld3" = super.buildPythonPackage {
-    name = "meld3-1.0.2";
-    doCheck = false;
-    src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/45/a0/317c6422b26c12fe0161e936fc35f36552069ba8e6f7ecbd99bbffe32a5f/meld3-1.0.2.tar.gz";
-      sha256 = "0n4mkwlpsqnmn0dm0wm5hn9nkda0nafl0jdy5sdl5977znh59dzp";
-    };
-    meta = {
-      license = [ { fullName = "BSD-derived (http://www.repoze.org/LICENSE.txt)"; } ];
+      license = [ pkgs.lib.licenses.bsdOriginal pkgs.lib.licenses.bsd3 ];
     };
   };
   "mistune" = super.buildPythonPackage {
@@ -942,14 +983,18 b' self: super: {'
     };
   };
   "mock" = super.buildPythonPackage {
-    name = "mock-1.0.1";
+    name = "mock-3.0.5";
     doCheck = false;
+    propagatedBuildInputs = [
+      self."six"
+      self."funcsigs"
+    ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/a2/52/7edcd94f0afb721a2d559a5b9aae8af4f8f2c79bc63fdbe8a8a6c9b23bbe/mock-1.0.1.tar.gz";
-      sha256 = "0kzlsbki6q0awf89rc287f3aj8x431lrajf160a70z0ikhnxsfdq";
+      url = "https://files.pythonhosted.org/packages/2e/ab/4fe657d78b270aa6a32f027849513b829b41b0f28d9d8d7f8c3d29ea559a/mock-3.0.5.tar.gz";
+      sha256 = "1hrp6j0yrx2xzylfv02qa8kph661m6yq4p0mc8fnimch9j4psrc3";
     };
     meta = {
-      license = [ pkgs.lib.licenses.bsdOriginal ];
+      license = [ pkgs.lib.licenses.bsdOriginal { fullName = "OSI Approved :: BSD License"; } ];
     };
   };
   "more-itertools" = super.buildPythonPackage {
@@ -1029,14 +1074,18 b' self: super: {'
     };
   };
   "packaging" = super.buildPythonPackage {
-    name = "packaging-15.2";
+    name = "packaging-19.2";
     doCheck = false;
+    propagatedBuildInputs = [
+      self."pyparsing"
+      self."six"
+    ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/24/c4/185da1304f07047dc9e0c46c31db75c0351bd73458ac3efad7da3dbcfbe1/packaging-15.2.tar.gz";
-      sha256 = "1zn60w84bxvw6wypffka18ca66pa1k2cfrq3cq8fnsfja5m3k4ng";
+      url = "https://files.pythonhosted.org/packages/5a/2f/449ded84226d0e2fda8da9252e5ee7731bdf14cd338f622dfcd9934e0377/packaging-19.2.tar.gz";
+      sha256 = "0izwlz9h0bw171a1chr311g2y7n657zjaf4mq4rgm8pp9lbj9f98";
     };
     meta = {
-      license = [ pkgs.lib.licenses.asl20 ];
+      license = [ pkgs.lib.licenses.bsdOriginal { fullName = "BSD or Apache License, Version 2.0"; } pkgs.lib.licenses.asl20 ];
     };
   };
   "pandocfilters" = super.buildPythonPackage {
@@ -1051,14 +1100,14 b' self: super: {'
     };
   };
   "paste" = super.buildPythonPackage {
-    name = "paste-3.0.8";
+    name = "paste-3.2.1";
     doCheck = false;
     propagatedBuildInputs = [
       self."six"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/66/65/e3acf1663438483c1f6ced0b6c6f3b90da9f0faacb0a6e2aa0f3f9f4b235/Paste-3.0.8.tar.gz";
-      sha256 = "05w1sh6ky4d7pmdb8nv82n13w22jcn3qsagg5ih3hjmbws9kkwf4";
+      url = "https://files.pythonhosted.org/packages/0d/86/7008b5563594e8a63763f05212a3eb84c85f0b2eff834e5697716e56bca9/Paste-3.2.1.tar.gz";
+      sha256 = "1vjxr8n1p31c9x9rh8g0f34yisa9028cxpvn36q7g1s0m2b9x71x";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
@@ -1076,7 +1125,7 b' self: super: {'
     };
   };
   "pastescript" = super.buildPythonPackage {
-    name = "pastescript-3.1.0";
+    name = "pastescript-3.2.0";
     doCheck = false;
     propagatedBuildInputs = [
       self."paste"
@@ -1084,23 +1133,23 b' self: super: {'
       self."six"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/9e/1d/14db1c283eb21a5d36b6ba1114c13b709629711e64acab653d9994fe346f/PasteScript-3.1.0.tar.gz";
-      sha256 = "02qcxjjr32ks7a6d4f533wl34ysc7yhwlrfcyqwqbzr52250v4fs";
+      url = "https://files.pythonhosted.org/packages/ff/47/45c6f5a3cb8f5abf786fea98dbb8d02400a55768a9b623afb7df12346c61/PasteScript-3.2.0.tar.gz";
+      sha256 = "1b3jq7xh383nvrrlblk05m37345bv97xrhx77wshllba3h7mq3wv";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
     };
   };
   "pathlib2" = super.buildPythonPackage {
-    name = "pathlib2-2.3.4";
+    name = "pathlib2-2.3.5";
     doCheck = false;
     propagatedBuildInputs = [
       self."six"
       self."scandir"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/b5/f4/9c7cc726ece2498b6c8b62d3262aa43f59039b953fe23c9964ac5e18d40b/pathlib2-2.3.4.tar.gz";
-      sha256 = "1y0f9rkm1924zrc5dn4bwxlhgdkbml82lkcc28l5rgmr7d918q24";
+      url = "https://files.pythonhosted.org/packages/94/d8/65c86584e7e97ef824a1845c72bbe95d79f5b306364fa778a3c3e401b309/pathlib2-2.3.5.tar.gz";
+      sha256 = "0s4qa8c082fdkb17izh4mfgwrjd1n5pya18wvrbwqdvvb5xs9nbc";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
@@ -1175,48 +1224,51 b' self: super: {'
     };
   };
   "pluggy" = super.buildPythonPackage {
-    name = "pluggy-0.11.0";
+    name = "pluggy-0.13.1";
     doCheck = false;
+    propagatedBuildInputs = [
+      self."importlib-metadata"
+    ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/0d/a1/862ab336e8128fde20981d2c1aa8506693412daf5083b1911d539412676b/pluggy-0.11.0.tar.gz";
-      sha256 = "10511a54dvafw1jrk75mrhml53c7b7w4yaw7241696lc2hfvr895";
+      url = "https://files.pythonhosted.org/packages/f8/04/7a8542bed4b16a65c2714bf76cf5a0b026157da7f75e87cc88774aa10b14/pluggy-0.13.1.tar.gz";
+      sha256 = "1c35qyhvy27q9ih9n899f3h4sdnpgq027dbiilly2qb5cvgarchm";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
     };
   };
   "prompt-toolkit" = super.buildPythonPackage {
-    name = "prompt-toolkit-1.0.16";
+    name = "prompt-toolkit-1.0.18";
     doCheck = false;
     propagatedBuildInputs = [
       self."six"
       self."wcwidth"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/f1/03/bb36771dc9fa7553ac4bdc639a9ecdf6fda0ff4176faf940d97e3c16e41d/prompt_toolkit-1.0.16.tar.gz";
-      sha256 = "1d65hm6nf0cbq0q0121m60zzy4s1fpg9fn761s1yxf08dridvkn1";
+      url = "https://files.pythonhosted.org/packages/c5/64/c170e5b1913b540bf0c8ab7676b21fdd1d25b65ddeb10025c6ca43cccd4c/prompt_toolkit-1.0.18.tar.gz";
+      sha256 = "09h1153wgr5x2ny7ds0w2m81n3bb9j8hjb8sjfnrg506r01clkyx";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal ];
     };
   };
   "psutil" = super.buildPythonPackage {
-    name = "psutil-5.5.1";
+    name = "psutil-5.6.5";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/c7/01/7c30b247cdc5ba29623faa5c8cf1f1bbf7e041783c340414b0ed7e067c64/psutil-5.5.1.tar.gz";
-      sha256 = "045qaqvn6k90bj5bcy259yrwcd2afgznaav3sfhphy9b8ambzkkj";
+      url = "https://files.pythonhosted.org/packages/03/9a/95c4b3d0424426e5fd94b5302ff74cea44d5d4f53466e1228ac8e73e14b4/psutil-5.6.5.tar.gz";
+      sha256 = "0isil5jxwwd8awz54qk28rpgjg43i5l6yl70g40vxwa4r4m56lfh";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal ];
     };
   };
   "psycopg2" = super.buildPythonPackage {
-    name = "psycopg2-2.8.3";
+    name = "psycopg2-2.8.4";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/5c/1c/6997288da181277a0c29bc39a5f9143ff20b8c99f2a7d059cfb55163e165/psycopg2-2.8.3.tar.gz";
-      sha256 = "0ms4kx0p5n281l89awccix4d05ybmdngnjjpi9jbzd0rhf1nwyl9";
+      url = "https://files.pythonhosted.org/packages/84/d7/6a93c99b5ba4d4d22daa3928b983cec66df4536ca50b22ce5dcac65e4e71/psycopg2-2.8.4.tar.gz";
+      sha256 = "1djvh98pi4hjd8rxbq8qzc63bg8v78k33yg6pl99wak61b6fb67q";
     };
     meta = {
       license = [ pkgs.lib.licenses.zpl21 { fullName = "GNU Library or Lesser General Public License (LGPL)"; } { fullName = "LGPL with exceptions or ZPL"; } ];
@@ -1234,11 +1286,11 b' self: super: {'
     };
   };
   "py" = super.buildPythonPackage {
-    name = "py-1.6.0";
+    name = "py-1.8.0";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/4f/38/5f427d1eedae73063ce4da680d2bae72014995f9fdeaa57809df61c968cd/py-1.6.0.tar.gz";
-      sha256 = "1wcs3zv9wl5m5x7p16avqj2gsrviyb23yvc3pr330isqs0sh98q6";
+      url = "https://files.pythonhosted.org/packages/f1/5a/87ca5909f400a2de1561f1648883af74345fe96349f34f737cdfc94eba8c/py-1.8.0.tar.gz";
+      sha256 = "0lsy1gajva083pzc7csj1cvbmminb7b4l6a0prdzyb3fd829nqyw";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
@@ -1271,28 +1323,28 b' self: super: {'
     };
   };
   "pyasn1" = super.buildPythonPackage {
-    name = "pyasn1-0.4.5";
+    name = "pyasn1-0.4.8";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/46/60/b7e32f6ff481b8a1f6c8f02b0fd9b693d1c92ddd2efb038ec050d99a7245/pyasn1-0.4.5.tar.gz";
-      sha256 = "1xqh3jh2nfi2bflk5a0vn59y3pp1vn54f3ksx652sid92gz2096s";
+      url = "https://files.pythonhosted.org/packages/a4/db/fffec68299e6d7bad3d504147f9094830b704527a7fc098b721d38cc7fa7/pyasn1-0.4.8.tar.gz";
+      sha256 = "1fnhbi3rmk47l9851gbik0flfr64vs5j0hbqx24cafjap6gprxxf";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal ];
     };
   };
   "pyasn1-modules" = super.buildPythonPackage {
-    name = "pyasn1-modules-0.2.5";
+    name = "pyasn1-modules-0.2.6";
     doCheck = false;
     propagatedBuildInputs = [
       self."pyasn1"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/ec/0b/69620cb04a016e4a1e8e352e8a42717862129b574b3479adb2358a1f12f7/pyasn1-modules-0.2.5.tar.gz";
-      sha256 = "15nvfx0vnl8akdlv3k6s0n80vqvryj82bm040jdsn7wmyxl1ywpg";
+      url = "https://files.pythonhosted.org/packages/f1/a9/a1ef72a0e43feff643cf0130a08123dea76205e7a0dda37e3efb5f054a31/pyasn1-modules-0.2.6.tar.gz";
+      sha256 = "08hph9j1r018drnrny29l7dl2q0cin78csswrhwrh8jmq61pmha3";
     };
     meta = {
-      license = [ pkgs.lib.licenses.bsdOriginal ];
+      license = [ pkgs.lib.licenses.bsdOriginal pkgs.lib.licenses.bsd2 ];
     };
   };
   "pycparser" = super.buildPythonPackage {
@@ -1318,11 +1370,11 b' self: super: {'
     };
   };
   "pycurl" = super.buildPythonPackage {
-    name = "pycurl-7.43.0.2";
+    name = "pycurl-7.43.0.3";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/e8/e4/0dbb8735407189f00b33d84122b9be52c790c7c3b25286826f4e1bdb7bde/pycurl-7.43.0.2.tar.gz";
-      sha256 = "1915kb04k1j4y6k1dx1sgnbddxrl9r1n4q928if2lkrdm73xy30g";
+      url = "https://files.pythonhosted.org/packages/ac/b3/0f3979633b7890bab6098d84c84467030b807a1e2b31f5d30103af5a71ca/pycurl-7.43.0.3.tar.gz";
+      sha256 = "13nsvqhvnmnvfk75s8iynqsgszyv06cjp4drd3psi7zpbh63623g";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit { fullName = "LGPL/MIT"; } { fullName = "GNU Library or Lesser General Public License (LGPL)"; } ];
@@ -1351,22 +1403,22 b' self: super: {'
     };
   };
   "pyotp" = super.buildPythonPackage {
-    name = "pyotp-2.2.7";
+    name = "pyotp-2.3.0";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/b1/ab/477cda97b6ca7baced5106471cb1ac1fe698d1b035983b9f8ee3422989eb/pyotp-2.2.7.tar.gz";
-      sha256 = "00p69nw431f0s2ilg0hnd77p1l22m06p9rq4f8zfapmavnmzw3xy";
+      url = "https://files.pythonhosted.org/packages/f7/15/395c4945ea6bc37e8811280bb675615cb4c2b2c1cd70bdc43329da91a386/pyotp-2.3.0.tar.gz";
+      sha256 = "18d13ikra1iq0xyfqfm72zhgwxi2qi9ps6z1a6zmqp4qrn57wlzw";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
     };
   };
   "pyparsing" = super.buildPythonPackage {
-    name = "pyparsing-2.3.0";
+    name = "pyparsing-2.4.5";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/d0/09/3e6a5eeb6e04467b737d55f8bba15247ac0876f98fae659e58cd744430c6/pyparsing-2.3.0.tar.gz";
-      sha256 = "14k5v7n3xqw8kzf42x06bzp184spnlkya2dpjyflax6l3yrallzk";
+      url = "https://files.pythonhosted.org/packages/00/32/8076fa13e832bb4dcff379f18f228e5a53412be0631808b9ca2610c0f566/pyparsing-2.4.5.tar.gz";
+      sha256 = "0fk8gsybiw1gm146mkjdjvaajwh20xwvpv4j7syh2zrnpq0j19jc";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
@@ -1396,7 +1448,7 b' self: super: {'
     };
   };
   "pyramid-debugtoolbar" = super.buildPythonPackage {
-    name = "pyramid-debugtoolbar-4.5";
+    name = "pyramid-debugtoolbar-4.5.1";
     doCheck = false;
     propagatedBuildInputs = [
       self."pyramid"
@@ -1406,8 +1458,8 b' self: super: {'
       self."ipaddress"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/14/28/1f240239af340d19ee271ac62958158c79edb01a44ad8c9885508dd003d2/pyramid_debugtoolbar-4.5.tar.gz";
-      sha256 = "0x2p3409pnx66n6dx5vc0mk2r1cp1ydr8mp120w44r9pwcngbibl";
+      url = "https://files.pythonhosted.org/packages/88/21/74e7fa52edc74667e29403bd0cb4f2bb74dc4014711de313868001bf639f/pyramid_debugtoolbar-4.5.1.tar.gz";
+      sha256 = "0hgf6i1fzvq43m9vjdmb24nnv8fwp7sdzrx9bcwrgpy24n07am9a";
     };
     meta = {
       license = [ { fullName = "Repoze Public License"; } pkgs.lib.licenses.bsdOriginal ];
@@ -1447,15 +1499,15 b' self: super: {'
     };
   };
   "pyramid-mako" = super.buildPythonPackage {
-    name = "pyramid-mako-1.0.2";
+    name = "pyramid-mako-1.1.0";
     doCheck = false;
     propagatedBuildInputs = [
       self."pyramid"
       self."mako"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/f1/92/7e69bcf09676d286a71cb3bbb887b16595b96f9ba7adbdc239ffdd4b1eb9/pyramid_mako-1.0.2.tar.gz";
-      sha256 = "18gk2vliq8z4acblsl6yzgbvnr9rlxjlcqir47km7kvlk1xri83d";
+      url = "https://files.pythonhosted.org/packages/63/7b/5e2af68f675071a6bad148c1c393928f0ef5fcd94e95cbf53b89d6471a83/pyramid_mako-1.1.0.tar.gz";
+      sha256 = "1qj0m091mnii86j2q1d82yir22nha361rvhclvg3s70z8iiwhrh0";
     };
     meta = {
       license = [ { fullName = "Repoze Public License"; } { fullName = "BSD-derived (http://www.repoze.org/LICENSE.txt)"; } ];
@@ -1473,44 +1525,46 b' self: super: {'
     };
   };
   "pytest" = super.buildPythonPackage {
-    name = "pytest-3.8.2";
+    name = "pytest-4.6.5";
     doCheck = false;
     propagatedBuildInputs = [
       self."py"
       self."six"
-      self."setuptools"
+      self."packaging"
       self."attrs"
-      self."more-itertools"
       self."atomicwrites"
       self."pluggy"
+      self."importlib-metadata"
+      self."wcwidth"
       self."funcsigs"
       self."pathlib2"
+      self."more-itertools"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/5f/d2/7f77f406ac505abda02ab4afb50d06ebf304f6ea42fca34f8f37529106b2/pytest-3.8.2.tar.gz";
-      sha256 = "18nrwzn61kph2y6gxwfz9ms68rfvr9d4vcffsxng9p7jk9z18clk";
+      url = "https://files.pythonhosted.org/packages/2a/c6/1d1f32f6a5009900521b12e6560fb6b7245b0d4bc3fb771acd63d10e30e1/pytest-4.6.5.tar.gz";
+      sha256 = "0iykwwfp4h181nd7rsihh2120b0rkawlw7rvbl19sgfspncr3hwg";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
     };
   };
   "pytest-cov" = super.buildPythonPackage {
-    name = "pytest-cov-2.6.0";
+    name = "pytest-cov-2.7.1";
     doCheck = false;
     propagatedBuildInputs = [
       self."pytest"
       self."coverage"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/d9/e2/58f90a316fbd94dd50bf5c826a23f3f5d079fb3cc448c1e9f0e3c33a3d2a/pytest-cov-2.6.0.tar.gz";
-      sha256 = "0qnpp9y3ygx4jk4pf5ad71fh2skbvnr6gl54m7rg5qysnx4g0q73";
+      url = "https://files.pythonhosted.org/packages/bb/0f/3db7ff86801883b21d5353b258c994b1b8e2abbc804e2273b8d0fd19004b/pytest-cov-2.7.1.tar.gz";
+      sha256 = "0filvmmyqm715azsl09ql8hy2x7h286n6d8z5x42a1wpvvys83p0";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal pkgs.lib.licenses.mit ];
     };
   };
   "pytest-profiling" = super.buildPythonPackage {
-    name = "pytest-profiling-1.3.0";
+    name = "pytest-profiling-1.7.0";
     doCheck = false;
     propagatedBuildInputs = [
       self."six"
@@ -1518,62 +1572,63 b' self: super: {'
       self."gprof2dot"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/f5/34/4626126e041a51ef50a80d0619519b18d20aef249aac25b0d0fdd47e57ee/pytest-profiling-1.3.0.tar.gz";
-      sha256 = "08r5afx5z22yvpmsnl91l4amsy1yxn8qsmm61mhp06mz8zjs51kb";
+      url = "https://files.pythonhosted.org/packages/39/70/22a4b33739f07f1732a63e33bbfbf68e0fa58cfba9d200e76d01921eddbf/pytest-profiling-1.7.0.tar.gz";
+      sha256 = "0abz9gi26jpcfdzgsvwad91555lpgdc8kbymicmms8k2fqa8z4wk";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
     };
   };
   "pytest-runner" = super.buildPythonPackage {
-    name = "pytest-runner-4.2";
+    name = "pytest-runner-5.1";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/9e/b7/fe6e8f87f9a756fd06722216f1b6698ccba4d269eac6329d9f0c441d0f93/pytest-runner-4.2.tar.gz";
-      sha256 = "1gkpyphawxz38ni1gdq1fmwyqcg02m7ypzqvv46z06crwdxi2gyj";
+      url = "https://files.pythonhosted.org/packages/d9/6d/4b41a74b31720e25abd4799be72d54811da4b4d0233e38b75864dcc1f7ad/pytest-runner-5.1.tar.gz";
+      sha256 = "0ykfcnpp8c22winj63qzc07l5axwlc9ikl8vn05sc32gv3417815";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
     };
   };
   "pytest-sugar" = super.buildPythonPackage {
-    name = "pytest-sugar-0.9.1";
+    name = "pytest-sugar-0.9.2";
     doCheck = false;
     propagatedBuildInputs = [
       self."pytest"
       self."termcolor"
+      self."packaging"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/3e/6a/a3f909083079d03bde11d06ab23088886bbe25f2c97fbe4bb865e2bf05bc/pytest-sugar-0.9.1.tar.gz";
-      sha256 = "0b4av40dv30727m54v211r0nzwjp2ajkjgxix6j484qjmwpw935b";
+      url = "https://files.pythonhosted.org/packages/55/59/f02f78d1c80f7e03e23177f60624c8106d4f23d124c921df103f65692464/pytest-sugar-0.9.2.tar.gz";
+      sha256 = "1asq7yc4g8bx2sn7yy974mhc9ywvaihasjab4inkirdwn9s7mn7w";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal ];
     };
   };
   "pytest-timeout" = super.buildPythonPackage {
-    name = "pytest-timeout-1.3.2";
+    name = "pytest-timeout-1.3.3";
     doCheck = false;
     propagatedBuildInputs = [
       self."pytest"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/8c/3e/1b6a319d12ae7baa3acb7c18ff2c8630a09471a0319d43535c683b4d03eb/pytest-timeout-1.3.2.tar.gz";
-      sha256 = "09wnmzvnls2mnsdz7x3c3sk2zdp6jl4dryvyj5i8hqz16q2zq5qi";
+      url = "https://files.pythonhosted.org/packages/13/48/7a166eaa29c1dca6cc253e3ba5773ff2e4aa4f567c1ea3905808e95ac5c1/pytest-timeout-1.3.3.tar.gz";
+      sha256 = "1cczcjhw4xx5sjkhxlhc5c1bkr7x6fcyx12wrnvwfckshdvblc2a";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit { fullName = "DFSG approved"; } ];
     };
   };
   "python-dateutil" = super.buildPythonPackage {
-    name = "python-dateutil-2.8.0";
+    name = "python-dateutil-2.8.1";
     doCheck = false;
     propagatedBuildInputs = [
       self."six"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/ad/99/5b2e99737edeb28c71bcbec5b5dda19d0d9ef3ca3e92e3e925e7c0bb364c/python-dateutil-2.8.0.tar.gz";
-      sha256 = "17nsfhy4xdz1khrfxa61vd7pmvd5z0wa3zb6v4gb4kfnykv0b668";
+      url = "https://files.pythonhosted.org/packages/be/ed/5bbc91f03fa4c839c4c7360375da77f9659af5f7086b7a7bdda65771c8e0/python-dateutil-2.8.1.tar.gz";
+      sha256 = "0g42w7k5007iv9dam6gnja2ry8ydwirh99mgdll35s12pyfzxsvk";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal pkgs.lib.licenses.asl20 { fullName = "Dual License"; } ];
@@ -1647,11 +1702,11 b' self: super: {'
     };
   };
   "pytz" = super.buildPythonPackage {
-    name = "pytz-2018.4";
+    name = "pytz-2019.2";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/10/76/52efda4ef98e7544321fd8d5d512e11739c1df18b0649551aeccfb1c8376/pytz-2018.4.tar.gz";
-      sha256 = "0jgpqx3kk2rhv81j1izjxvmx8d0x7hzs1857pgqnixic5wq2ar60";
+      url = "https://files.pythonhosted.org/packages/27/c0/fbd352ca76050952a03db776d241959d5a2ee1abddfeb9e2a53fdb489be4/pytz-2019.2.tar.gz";
+      sha256 = "0ckb27hhjc8i8gcdvk4d9avld62b7k52yjijc60s2m3y8cpb7h16";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
@@ -1669,11 +1724,11 b' self: super: {'
     };
   };
   "redis" = super.buildPythonPackage {
-    name = "redis-2.10.6";
+    name = "redis-3.3.11";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/09/8d/6d34b75326bf96d4139a2ddd8e74b80840f800a0a79f9294399e212cb9a7/redis-2.10.6.tar.gz";
-      sha256 = "03vcgklykny0g0wpvqmy8p6azi2s078317wgb2xjv5m2rs9sjb52";
+      url = "https://files.pythonhosted.org/packages/06/ca/00557c74279d2f256d3c42cabf237631355f3a132e4c74c2000e6647ad98/redis-3.3.11.tar.gz";
+      sha256 = "1hicqbi5xl92hhml82awrr2rxl9jar5fp8nbcycj9qgmsdwc43wd";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
@@ -1707,18 +1762,24 b' self: super: {'
     };
   };
   "requests" = super.buildPythonPackage {
-    name = "requests-2.9.1";
+    name = "requests-2.22.0";
     doCheck = false;
+    propagatedBuildInputs = [
+      self."chardet"
+      self."idna"
+      self."urllib3"
+      self."certifi"
+    ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/f9/6d/07c44fb1ebe04d069459a189e7dab9e4abfe9432adcd4477367c25332748/requests-2.9.1.tar.gz";
-      sha256 = "0zsqrzlybf25xscgi7ja4s48y2abf9wvjkn47wh984qgs1fq2xy5";
+      url = "https://files.pythonhosted.org/packages/01/62/ddcf76d1d19885e8579acb1b1df26a852b03472c0e46d2b959a714c90608/requests-2.22.0.tar.gz";
+      sha256 = "1d5ybh11jr5sm7xp6mz8fyc7vrp4syifds91m7sj60xalal0gq0i";
     };
     meta = {
       license = [ pkgs.lib.licenses.asl20 ];
     };
   };
   "rhodecode-enterprise-ce" = super.buildPythonPackage {
-    name = "rhodecode-enterprise-ce-4.17.4";
+    name = "rhodecode-enterprise-ce-4.18.0";
     buildInputs = [
       self."pytest"
       self."py"
@@ -1738,7 +1799,6 b' self: super: {'
     doCheck = true;
     propagatedBuildInputs = [
       self."amqp"
-      self."authomatic"
       self."babel"
       self."beaker"
       self."bleach"
@@ -1808,7 +1868,6 b' self: super: {'
       self."venusian"
       self."weberror"
       self."webhelpers2"
-      self."webhelpers"
       self."webob"
       self."whoosh"
       self."wsgiref"
@@ -1823,6 +1882,7 b' self: super: {'
       self."nbconvert"
       self."nbformat"
       self."jupyter-client"
+      self."jupyter-core"
       self."alembic"
       self."invoke"
       self."bumpversion"
@@ -1854,7 +1914,7 b' self: super: {'
     };
   };
   "rhodecode-tools" = super.buildPythonPackage {
-    name = "rhodecode-tools-1.2.1";
+    name = "rhodecode-tools-1.4.0";
     doCheck = false;
     propagatedBuildInputs = [
       self."click"
@@ -1871,8 +1931,8 b' self: super: {'
       self."elasticsearch1-dsl"
     ];
     src = fetchurl {
-      url = "https://code.rhodecode.com/rhodecode-tools-ce/artifacts/download/0-10ac93f4-bb7d-4b97-baea-68110743dd5a.tar.gz?md5=962dc77c06aceee62282b98d33149661";
-      sha256 = "1vfhgf46inbx7jvlfx4fdzh3vz7lh37r291gzb5hx447pfm3qllg";
+      url = "https://code.rhodecode.com/rhodecode-tools-ce/artifacts/download/0-ed54e749-2ef5-4bc7-ae7f-7900e3c2aa15.tar.gz?sha256=76f024bad3a1e55fdb3d64f13f5b77ff21a12fee699918de2110fe21effd5a3a";
+      sha256 = "0fjszppj3zhh47g1i6b9xqps28gzfxdkzwb47pdmzrd1sfx29w3n";
     };
     meta = {
       license = [ { fullName = "Apache 2.0 and Proprietary"; } ];
@@ -1916,11 +1976,11 b' self: super: {'
     };
   };
   "setuptools" = super.buildPythonPackage {
-    name = "setuptools-41.0.1";
+    name = "setuptools-44.0.0";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/1d/64/a18a487b4391a05b9c7f938b94a16d80305bf0369c6b0b9509e86165e1d3/setuptools-41.0.1.zip";
-      sha256 = "04sns22y2hhsrwfy1mha2lgslvpjsjsz8xws7h2rh5a7ylkd28m2";
+      url = "https://files.pythonhosted.org/packages/b0/f3/44da7482ac6da3f36f68e253cb04de37365b3dba9036a3c70773b778b485/setuptools-44.0.0.zip";
+      sha256 = "025h5cnxcmda1893l6i12hrwdvs1n8r31qs6q4pkif2v7rrggfp5";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
@@ -1960,11 +2020,11 b' self: super: {'
     };
   };
   "sqlalchemy" = super.buildPythonPackage {
-    name = "sqlalchemy-1.1.18";
+    name = "sqlalchemy-1.3.11";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/cc/4d/96d93ff77cd67aca7618e402191eee3490d8f5f245d6ab7622d35fe504f4/SQLAlchemy-1.1.18.tar.gz";
-      sha256 = "1ab4ysip6irajfbxl9wy27kv76miaz8h6759hfx92499z4dcf3lb";
+      url = "https://files.pythonhosted.org/packages/34/5c/0e1d7ad0ca52544bb12f9cb8d5cc454af45821c92160ffedd38db0a317f6/SQLAlchemy-1.3.11.tar.gz";
+      sha256 = "12izpqqgy738ndn7qqn962qxi8qw2xb9vg2i880x12paklg599dg";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
@@ -1997,14 +2057,11 b' self: super: {'
     };
   };
   "supervisor" = super.buildPythonPackage {
-    name = "supervisor-4.0.3";
+    name = "supervisor-4.1.0";
     doCheck = false;
-    propagatedBuildInputs = [
-      self."meld3"
-    ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/97/48/f38bf70bd9282d1a18d591616557cc1a77a1c627d57dff66ead65c891dc8/supervisor-4.0.3.tar.gz";
-      sha256 = "17hla7mx6w5m5jzkkjxgqa8wpswqmfhbhf49f692hw78fg0ans7p";
+      url = "https://files.pythonhosted.org/packages/de/87/ee1ad8fa533a4b5f2c7623f4a2b585d3c1947af7bed8e65bc7772274320e/supervisor-4.1.0.tar.gz";
+      sha256 = "10q36sa1jqljyyyl7cif52akpygl5kmlqq9x91hmx53f8zh6zj1d";
     };
     meta = {
       license = [ { fullName = "BSD-derived (http://www.repoze.org/LICENSE.txt)"; } ];
@@ -2033,18 +2090,18 b' self: super: {'
     };
   };
   "testpath" = super.buildPythonPackage {
-    name = "testpath-0.4.2";
+    name = "testpath-0.4.4";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/06/30/9a7e917066d851d8b4117e85794b5f14516419ea714a8a2681ec6aa8a981/testpath-0.4.2.tar.gz";
-      sha256 = "1y40hywscnnyb734pnzm55nd8r8kp1072bjxbil83gcd53cv755n";
+      url = "https://files.pythonhosted.org/packages/2c/b3/5d57205e896d8998d77ad12aa42ebce75cd97d8b9a97d00ba078c4c9ffeb/testpath-0.4.4.tar.gz";
+      sha256 = "0zpcmq22dz79ipvvsfnw1ykpjcaj6xyzy7ws77s5b5ql3hka7q30";
     };
     meta = {
       license = [  ];
     };
   };
   "traitlets" = super.buildPythonPackage {
-    name = "traitlets-4.3.2";
+    name = "traitlets-4.3.3";
     doCheck = false;
     propagatedBuildInputs = [
       self."ipython-genutils"
@@ -2053,8 +2110,8 b' self: super: {'
       self."enum34"
     ];
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/a5/98/7f5ef2fe9e9e071813aaf9cb91d1a732e0a68b6c44a32b38cb8e14c3f069/traitlets-4.3.2.tar.gz";
-      sha256 = "0dbq7sx26xqz5ixs711k5nc88p8a0nqyz6162pwks5dpcz9d4jww";
+      url = "https://files.pythonhosted.org/packages/75/b0/43deb021bc943f18f07cbe3dac1d681626a48997b7ffa1e7fb14ef922b21/traitlets-4.3.3.tar.gz";
+      sha256 = "1xsrwgivpkxlbr4dfndfsi098s29yqgswgjc1qqn69yxklvfw8yh";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal ];
@@ -2100,11 +2157,11 b' self: super: {'
     };
   };
   "urllib3" = super.buildPythonPackage {
-    name = "urllib3-1.24.1";
+    name = "urllib3-1.25.2";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/b1/53/37d82ab391393565f2f831b8eedbffd57db5a718216f82f1a8b4d381a1c1/urllib3-1.24.1.tar.gz";
-      sha256 = "08lwd9f3hqznyf32vnzwvp87pchx062nkbgyrf67rwlkgj0jk5fy";
+      url = "https://files.pythonhosted.org/packages/9a/8b/ea6d2beb2da6e331e9857d0a60b79ed4f72dcbc4e2c7f2d2521b0480fda2/urllib3-1.25.2.tar.gz";
+      sha256 = "1nq2k4pss1ihsjh02r41sqpjpm5rfqkjfysyq7g7n2i1p7c66c55";
     };
     meta = {
       license = [ pkgs.lib.licenses.mit ];
@@ -2144,11 +2201,11 b' self: super: {'
     };
   };
   "waitress" = super.buildPythonPackage {
-    name = "waitress-1.3.0";
+    name = "waitress-1.3.1";
     doCheck = false;
     src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/43/50/9890471320d5ad22761ae46661cf745f487b1c8c4ec49352b99e1078b970/waitress-1.3.0.tar.gz";
-      sha256 = "09j5dzbbcxib7vdskhx39s1qsydlr4n2p2png71d7mjnr9pnwajf";
+      url = "https://files.pythonhosted.org/packages/a6/e6/708da7bba65898e5d759ade8391b1077e49d07be0b0223c39f5be04def56/waitress-1.3.1.tar.gz";
+      sha256 = "1iysl8ka3l4cdrr0r19fh1cv28q41mwpvgsb81ji7k4shkb0k3i7";
     };
     meta = {
       license = [ pkgs.lib.licenses.zpl21 ];
@@ -2193,20 +2250,6 b' self: super: {'
       license = [ pkgs.lib.licenses.mit ];
     };
   };
-  "webhelpers" = super.buildPythonPackage {
-    name = "webhelpers-1.3";
-    doCheck = false;
-    propagatedBuildInputs = [
-      self."markupsafe"
-    ];
-    src = fetchurl {
-      url = "https://files.pythonhosted.org/packages/ee/68/4d07672821d514184357f1552f2dad923324f597e722de3b016ca4f7844f/WebHelpers-1.3.tar.gz";
-      sha256 = "10x5i82qdkrvyw18gsybwggfhfpl869siaab89vnndi9x62g51pa";
-    };
-    meta = {
-      license = [ pkgs.lib.licenses.bsdOriginal ];
-    };
-  };
   "webhelpers2" = super.buildPythonPackage {
     name = "webhelpers2-2.0";
     doCheck = false;
@@ -2283,6 +2326,20 b' self: super: {'
       license = [ { fullName = "PSF or ZPL"; } ];
     };
   };
+  "zipp" = super.buildPythonPackage {
+    name = "zipp-0.6.0";
+    doCheck = false;
+    propagatedBuildInputs = [
+      self."more-itertools"
+    ];
+    src = fetchurl {
+      url = "https://files.pythonhosted.org/packages/57/dd/585d728479d97d25aeeb9aa470d36a4ad8d0ba5610f84e14770128ce6ff7/zipp-0.6.0.tar.gz";
+      sha256 = "13ndkf7vklw978a4gdl1yfvn8hch28429a0iam67sg4nrp5v261p";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.mit ];
+    };
+  };
   "zope.cachedescriptors" = super.buildPythonPackage {
     name = "zope.cachedescriptors-4.3.1";
     doCheck = false;

pytest.ini

@@ -9,8 +9,11 b' vcsserver_config_http = rhodecode/tests/'
 
 addopts =
   --pdbcls=IPython.terminal.debugger:TerminalPdb
+  --strict-markers
 
 markers =
   vcs_operations: Mark tests depending on a running RhodeCode instance.
   xfail_backends: Mark tests as xfail for given backends.
   skip_backends: Mark tests as skipped for given backends.
+  backends: Mark backends
+  dbs: database markers for running tests for given DB

requirements.txt

@@ -1,13 +1,10 b''
 ## dependencies
 
-amqp==2.3.1
-# not released authomatic that has updated some oauth providers
-https://code.rhodecode.com/upstream/authomatic/artifacts/download/0-4fe9c041-a567-4f84-be4c-7efa2a606d3c.tar.gz?md5=f6bdc3c769688212db68233e8d2b0383#egg=authomatic==0.1.0.post1
-
+amqp==2.5.2
 babel==1.3
 beaker==1.9.1
 bleach==3.1.0
-celery==4.1.1
+celery==4.3.0
 channelstream==0.5.2
 click==7.0
 colander==1.7.0
@@ -16,9 +13,9 b' https://code.rhodecode.com/upstream/conf'
 cssselect==1.0.3
 cryptography==2.6.1
 decorator==4.1.2
-deform==2.0.7
+deform==2.0.8
 docutils==0.14.0
-dogpile.cache==0.7.1
+dogpile.cache==0.9.0
 dogpile.core==0.4.1
 formencode==1.2.4
 future==0.14.3
@@ -26,55 +23,54 b' futures==3.0.2'
 infrae.cache==1.0.1
 iso8601==0.1.12
 itsdangerous==0.24
-kombu==4.2.1
+kombu==4.6.6
 lxml==4.2.5
-mako==1.0.7
+mako==1.1.0
 markdown==2.6.11
-markupsafe==1.1.0
+markupsafe==1.1.1
 msgpack-python==0.5.6
-pyotp==2.2.7
-packaging==15.2
-pathlib2==2.3.4
-paste==3.0.8
+pyotp==2.3.0
+packaging==19.2
+pathlib2==2.3.5
+paste==3.2.1
 pastedeploy==2.0.1
-pastescript==3.1.0
+pastescript==3.2.0
 peppercorn==0.6
-psutil==5.5.1
+psutil==5.6.5
 py-bcrypt==0.4
-pycurl==7.43.0.2
+pycurl==7.43.0.3
 pycrypto==2.6.1
 pygments==2.4.2
-pyparsing==2.3.0
-pyramid-debugtoolbar==4.5.0
-pyramid-mako==1.0.2
+pyparsing==2.4.5
+pyramid-debugtoolbar==4.5.1
+pyramid-mako==1.1.0
 pyramid==1.10.4
 pyramid_mailer==0.15.1
-python-dateutil
+python-dateutil==2.8.1
 python-ldap==3.1.0
 python-memcached==1.59
 python-pam==1.8.4
 python-saml==2.4.2
-pytz==2018.4
+pytz==2019.2
 tzlocal==1.5.1
 pyzmq==14.6.0
 py-gfm==0.1.4
-redis==2.10.6
+redis==3.3.11
 repoze.lru==0.7
-requests==2.9.1
+requests==2.22.0
 routes==2.4.1
 simplejson==3.16.0
 six==1.11.0
-sqlalchemy==1.1.18
+sqlalchemy==1.3.11
 sshpubkeys==3.1.0
 subprocess32==3.5.4
-supervisor==4.0.3
+supervisor==4.1.0
 translationstring==1.3
-urllib3==1.24.1
+urllib3==1.25.2
 urlobject==2.4.3
 venusian==1.2.0
 weberror==0.10.3
 webhelpers2==2.0
-webhelpers==1.3
 webob==1.8.5
 whoosh==2.7.4
 wsgiref==0.1.2
@@ -87,17 +83,18 b' zope.interface==4.6.0'
 mysql-python==1.2.5
 pymysql==0.8.1
 pysqlite==2.8.3
-psycopg2==2.8.3
+psycopg2==2.8.4
 
 # IPYTHON RENDERING
 # entrypoints backport, pypi version doesn't support egg installs
 https://code.rhodecode.com/upstream/entrypoints/artifacts/download/0-8e9ee9e4-c4db-409c-b07e-81568fd1832d.tar.gz?md5=3a027b8ff1d257b91fe257de6c43357d#egg=entrypoints==0.2.2.rhodecode-upstream1
 nbconvert==5.3.1
 nbformat==4.4.0
-jupyter_client==5.0.0
+jupyter-client==5.0.0
+jupyter-core==4.5.0
 
 ## cli tools
-alembic==1.0.10
+alembic==1.3.1
 invoke==0.13.0
 bumpversion==0.5.3
 
@@ -105,14 +102,15 b' bumpversion==0.5.3'
 gevent==1.4.0
 greenlet==0.4.15
 gunicorn==19.9.0
-waitress==1.3.0
+waitress==1.3.1
 
 ## debug
 ipdb==0.12.0
 ipython==5.1.0
 
-## rhodecode-tools, special case
-https://code.rhodecode.com/rhodecode-tools-ce/artifacts/download/0-10ac93f4-bb7d-4b97-baea-68110743dd5a.tar.gz?md5=962dc77c06aceee62282b98d33149661#egg=rhodecode-tools==1.2.1
+## rhodecode-tools, special case, use file://PATH.tar.gz#egg=rhodecode-tools==X.Y.Z, to test local version
+https://code.rhodecode.com/rhodecode-tools-ce/artifacts/download/0-ed54e749-2ef5-4bc7-ae7f-7900e3c2aa15.tar.gz?sha256=76f024bad3a1e55fdb3d64f13f5b77ff21a12fee699918de2110fe21effd5a3a#egg=rhodecode-tools==1.4.0
+
 
 ## appenlight
 appenlight-client==0.6.26

requirements_pinned.txt

@@ -1,19 +1,27 b''
 # contains not directly required libraries we want to pin the version.
 
-atomicwrites==1.2.1
-attrs==18.2.0
-billiard==3.5.0.3
+atomicwrites==1.3.0
+attrs==19.3.0
+asn1crypto==0.24.0
+billiard==3.6.1.0
+cffi==1.12.3
 chameleon==2.24
-cffi==1.12.2
+configparser==4.0.2
+contextlib2==0.6.0.post1
 ecdsa==0.13.2
-hupper==1.6.1
 gnureadline==6.3.8
+hupper==1.9.1
+ipaddress==1.0.23
+importlib-metadata==0.23
 jinja2==2.9.6
 jsonschema==2.6.0
+pluggy==0.13.1
+pyasn1-modules==0.2.6
 pyramid-jinja2==2.7
-pluggy==0.11.0
-setproctitle==1.1.10
 scandir==1.10.0
+setproctitle==1.1.10
 tempita==0.5.2
+testpath==0.4.4
+transaction==2.4.0
 vine==1.3.0
-configparser==3.7.4
+wcwidth==0.1.7

requirements_test.txt

@@ -1,16 +1,16 b''
 # test related requirements
-pytest==3.8.2
-py==1.6.0
-pytest-cov==2.6.0
-pytest-sugar==0.9.1
-pytest-runner==4.2.0
-pytest-profiling==1.3.0
-pytest-timeout==1.3.2
+pytest==4.6.5
+py==1.8.0
+pytest-cov==2.7.1
+pytest-sugar==0.9.2
+pytest-runner==5.1.0
+pytest-profiling==1.7.0
+pytest-timeout==1.3.3
 gprof2dot==2017.9.19
 
-mock==1.0.1
+mock==3.0.5
 cov-core==1.15.0
-coverage==4.5.3
+coverage==4.5.4
 
 webtest==2.0.33
 beautifulsoup4==4.6.3

rhodecode/VERSION

@@ -1,1 +1,1 b''
-4.17.4
  No newline at end of file
+4.18.0
  No newline at end of file

rhodecode/__init__.py

@@ -45,7 +45,7 b' PYRAMID_SETTINGS = {}'
 EXTENSIONS = {}
 
 __version__ = ('.'.join((str(each) for each in VERSION[:3])))
-__dbversion__ = 98  # defines current db version for migrations
+__dbversion__ = 103  # defines current db version for migrations
 __platform__ = platform.system()
 __license__ = 'AGPLv3, and Commercial License'
 __author__ = 'RhodeCode GmbH'

rhodecode/api/__init__.py

@@ -122,7 +122,7 b' def jsonrpc_response(request, result):'
     return response
 
 
-def jsonrpc_error(request, message, retid=None, code=None):
+def jsonrpc_error(request, message, retid=None, code=None, headers=None):
     """
     Generate a Response object with a JSON-RPC error body
 
@@ -132,10 +132,12 b' def jsonrpc_error(request, message, reti'
     """
     err_dict = {'id': retid, 'result': None, 'error': message}
     body = render(DEFAULT_RENDERER, err_dict, request=request).encode('utf-8')
+
     return Response(
         body=body,
         status=code,
-        content_type='application/json'
+        content_type='application/json',
+        headerlist=headers
     )
 
 
@@ -287,8 +289,7 b' def request_view(request):'
     })
 
     # register some common functions for usage
-    attach_context_attributes(
-        TemplateArgs(), request, request.rpc_user.user_id)
+    attach_context_attributes(TemplateArgs(), request, request.rpc_user.user_id)
 
     try:
         ret_value = func(**call_params)
@@ -298,9 +299,13 b' def request_view(request):'
     except Exception:
         log.exception('Unhandled exception occurred on api call: %s', func)
         exc_info = sys.exc_info()
-        store_exception(id(exc_info), exc_info, prefix='rhodecode-api')
+        exc_id, exc_type_name = store_exception(
+            id(exc_info), exc_info, prefix='rhodecode-api')
+        error_headers = [('RhodeCode-Exception-Id', str(exc_id)),
+                         ('RhodeCode-Exception-Type', str(exc_type_name))]
         return jsonrpc_error(
-            request, retid=request.rpc_id, message='Internal server error')
+            request, retid=request.rpc_id, message='Internal server error',
+            headers=error_headers)
 
 
 def setup_request(request):
@@ -333,6 +338,7 b' def setup_request(request):'
             raise JSONRPCError("Content-Length is 0")
 
     raw_body = request.body
+    log.debug("Loading JSON body now")
     try:
         json_body = json.loads(raw_body)
     except ValueError as e:
@@ -359,7 +365,7 b' def setup_request(request):'
         request.rpc_params = json_body['args'] \
             if isinstance(json_body['args'], dict) else {}
 
-        log.debug('method: %s, params: %s', request.rpc_method, request.rpc_params)
+        log.debug('method: %s, params: %.10240r', request.rpc_method, request.rpc_params)
     except KeyError as e:
         raise JSONRPCError('Incorrect JSON data. Missing %s' % e)
 

rhodecode/api/tests/test_close_pull_request.py

@@ -49,7 +49,7 b' class TestClosePullRequest(object):'
         assert_ok(id_, expected, response.body)
         journal = UserLog.query()\
             .filter(UserLog.user_id == author) \
-            .order_by('user_log_id') \
+            .order_by(UserLog.user_log_id.asc()) \
             .filter(UserLog.repository_id == repo)\
             .all()
         assert journal[-1].action == 'repo.pull_request.close'

rhodecode/api/tests/test_comment_commit.py

@@ -20,7 +20,7 b''
 
 import pytest
 
-from rhodecode.model.db import ChangesetStatus
+from rhodecode.model.db import ChangesetStatus, User
 from rhodecode.api.tests.utils import (
     build_data, api_call, assert_error, assert_ok)
 
@@ -79,3 +79,38 b' class TestCommentCommit(object):'
             'success': True
         }
         assert_ok(id_, expected, given=response.body)
+
+    def test_api_comment_commit_with_extra_recipients(self, backend, user_util):
+
+        commit_id = backend.repo.scm_instance().get_commit('tip').raw_id
+
+        user1 = user_util.create_user()
+        user1_id = user1.user_id
+        user2 = user_util.create_user()
+        user2_id = user2.user_id
+
+        id_, params = build_data(
+            self.apikey, 'comment_commit', repoid=backend.repo_name,
+            commit_id=commit_id,
+            message='abracadabra',
+            extra_recipients=[user1.user_id, user2.username])
+
+        response = api_call(self.app, params)
+        repo = backend.repo.scm_instance()
+
+        expected = {
+            'msg': 'Commented on commit `%s` for repository `%s`' % (
+                repo.get_commit().raw_id, backend.repo_name),
+            'status_change': None,
+            'success': True
+        }
+
+        assert_ok(id_, expected, given=response.body)
+        # check user1/user2 inbox for notification
+        user1 = User.get(user1_id)
+        assert 1 == len(user1.notifications)
+        assert 'abracadabra' in user1.notifications[0].notification.body
+
+        user2 = User.get(user2_id)
+        assert 1 == len(user2.notifications)
+        assert 'abracadabra' in user2.notifications[0].notification.body

rhodecode/api/tests/test_comment_pull_request.py

@@ -21,7 +21,7 b''
 import pytest
 
 from rhodecode.model.comment import CommentsModel
-from rhodecode.model.db import UserLog
+from rhodecode.model.db import UserLog, User
 from rhodecode.model.pull_request import PullRequestModel
 from rhodecode.tests import TEST_USER_ADMIN_LOGIN
 from rhodecode.api.tests.utils import (
@@ -65,11 +65,48 b' class TestCommentPullRequest(object):'
         journal = UserLog.query()\
             .filter(UserLog.user_id == author)\
             .filter(UserLog.repository_id == repo) \
-            .order_by('user_log_id') \
+            .order_by(UserLog.user_log_id.asc()) \
             .all()
         assert journal[-1].action == 'repo.pull_request.comment.create'
 
     @pytest.mark.backends("git", "hg")
+    def test_api_comment_pull_request_with_extra_recipients(self, pr_util, user_util):
+        pull_request = pr_util.create_pull_request()
+
+        user1 = user_util.create_user()
+        user1_id = user1.user_id
+        user2 = user_util.create_user()
+        user2_id = user2.user_id
+
+        id_, params = build_data(
+            self.apikey, 'comment_pull_request',
+            repoid=pull_request.target_repo.repo_name,
+            pullrequestid=pull_request.pull_request_id,
+            message='test message',
+            extra_recipients=[user1.user_id, user2.username]
+        )
+        response = api_call(self.app, params)
+        pull_request = PullRequestModel().get(pull_request.pull_request_id)
+
+        comments = CommentsModel().get_comments(
+            pull_request.target_repo.repo_id, pull_request=pull_request)
+
+        expected = {
+            'pull_request_id': pull_request.pull_request_id,
+            'comment_id': comments[-1].comment_id,
+            'status': {'given': None, 'was_changed': None}
+        }
+        assert_ok(id_, expected, response.body)
+        # check user1/user2 inbox for notification
+        user1 = User.get(user1_id)
+        assert 1 == len(user1.notifications)
+        assert 'test message' in user1.notifications[0].notification.body
+
+        user2 = User.get(user2_id)
+        assert 1 == len(user2.notifications)
+        assert 'test message' in user2.notifications[0].notification.body
+
+    @pytest.mark.backends("git", "hg")
     def test_api_comment_pull_request_change_status(
             self, pr_util, no_notifications):
         pull_request = pr_util.create_pull_request()

rhodecode/api/tests/test_create_user.py

@@ -82,6 +82,7 b' class TestCreateUser(object):'
             self.apikey, 'create_user',
             username=username,
             email=email,
+            description='CTO of Things',
             password='example')
         response = api_call(self.app, params)
 

rhodecode/api/tests/test_fts_search.py

@@ -27,6 +27,16 b' from rhodecode.api.tests.utils import ('
 @pytest.mark.usefixtures("testuser_api", "app")
 class TestApiSearch(object):
 
+    @pytest.mark.parametrize("sort_dir", [
+        "asc",
+        "desc",
+    ])
+    @pytest.mark.parametrize("sort", [
+        "xxx",
+        "author_email",
+        "date",
+        "message",
+    ])
     @pytest.mark.parametrize("query, expected_hits, expected_paths", [
         ('todo', 23, [
             'vcs/backends/hg/inmemory.py',
@@ -55,10 +65,11 b' class TestApiSearch(object):'
             'vcs/tests/test_cli.py']),
         ('owner:michał test', 0, []),
     ])
-    def test_search_content_results(self, query, expected_hits, expected_paths):
+    def test_search_content_results(self, sort_dir, sort, query, expected_hits, expected_paths):
         id_, params = build_data(
             self.apikey_regular, 'search',
             search_query=query,
+            search_sort='{}:{}'.format(sort_dir, sort),
             search_type='content')
 
         response = api_call(self.app, params)
@@ -70,6 +81,16 b' class TestApiSearch(object):'
         for expected_path in expected_paths:
             assert expected_path in paths
 
+    @pytest.mark.parametrize("sort_dir", [
+        "asc",
+        "desc",
+    ])
+    @pytest.mark.parametrize("sort", [
+        "xxx",
+        "date",
+        "file",
+        "size",
+    ])
     @pytest.mark.parametrize("query, expected_hits, expected_paths", [
         ('readme.rst', 3, []),
         ('test*', 75, []),
@@ -77,10 +98,11 b' class TestApiSearch(object):'
         ('extension:rst', 48, []),
         ('extension:rst api', 24, []),
     ])
-    def test_search_file_paths(self, query, expected_hits, expected_paths):
+    def test_search_file_paths(self, sort_dir, sort, query, expected_hits, expected_paths):
         id_, params = build_data(
             self.apikey_regular, 'search',
             search_query=query,
+            search_sort='{}:{}'.format(sort_dir, sort),
             search_type='path')
 
         response = api_call(self.app, params)

rhodecode/api/tests/test_get_method.py

@@ -50,6 +50,7 b' class TestGetMethod(object):'
                     {'apiuser': '<RequiredType>',
                      'comment_type': "<Optional:u'note'>",
                      'commit_id': '<RequiredType>',
+                     'extra_recipients': '<Optional:[]>',
                      'message': '<RequiredType>',
                      'repoid': '<RequiredType>',
                      'request': '<RequiredType>',

rhodecode/api/tests/test_get_repo_changeset.py

@@ -54,7 +54,7 b' class TestGetRepoChangeset(object):'
             details=details,
         )
         response = api_call(self.app, params)
-        expected = 'commit_id must be a string value'
+        expected = "commit_id must be a string value got <type 'int'> instead"
         assert_error(id_, expected, given=response.body)
 
     @pytest.mark.parametrize("details", ['basic', 'extended', 'full'])
@@ -137,5 +137,5 b' class TestGetRepoChangeset(object):'
             details=details,
         )
         response = api_call(self.app, params)
-        expected = 'commit_id must be a string value'
+        expected = "commit_id must be a string value got <type 'int'> instead"
         assert_error(id_, expected, given=response.body)

rhodecode/api/tests/test_get_repo_comments.py

@@ -31,7 +31,9 b' from rhodecode.api.tests.utils import ('
 @pytest.fixture()
 def make_repo_comments_factory(request):
 
-    def maker(repo):
+    class Make(object):
+
+        def make_comments(self, repo):
         user = User.get_first_super_admin()
         commit = repo.scm_instance()[0]
 
@@ -60,7 +62,7 b' def make_repo_comments_factory(request):'
         def cleanup():
             for comment in comments:
                 Session().delete(comment)
-    return maker
+    return Make()
 
 
 @pytest.mark.usefixtures("testuser_api", "app")
@@ -76,7 +78,7 b' class TestGetRepo(object):'
                                    make_repo_comments_factory, filters, expected_count):
         commits = [{'message': 'A'}, {'message': 'B'}]
         repo = backend.create_repo(commits=commits)
-        make_repo_comments_factory(repo)
+        make_repo_comments_factory.make_comments(repo)
 
         api_call_params = {'repoid': repo.repo_name,}
         api_call_params.update(filters)
@@ -92,12 +94,13 b' class TestGetRepo(object):'
 
         assert len(result) == expected_count
 
-    def test_api_get_repo_comments_wrong_comment_typ(self, backend_hg):
+    def test_api_get_repo_comments_wrong_comment_type(
+            self, make_repo_comments_factory, backend_hg):
+        commits = [{'message': 'A'}, {'message': 'B'}]
+        repo = backend_hg.create_repo(commits=commits)
+        make_repo_comments_factory.make_comments(repo)
 
-        repo = backend_hg.create_repo()
-        make_repo_comments_factory(repo)
-
-        api_call_params = {'repoid': repo.repo_name,}
+        api_call_params = {'repoid': repo.repo_name}
         api_call_params.update({'comment_type': 'bogus'})
 
         expected = 'comment_type must be one of `{}` got {}'.format(

rhodecode/api/tests/test_get_server_info.py

@@ -25,7 +25,7 b' from rhodecode.model.scm import ScmModel'
 from rhodecode.api.tests.utils import build_data, api_call, assert_ok
 
 
[e-mail]
[e-mail]()
 def http_host_stub():
     """
     To ensure that we can get an IP address, this test shall run with a

rhodecode/api/tests/test_merge_pull_request.py

@@ -120,7 +120,7 b' class TestMergePullRequest(object):'
         journal = UserLog.query()\
             .filter(UserLog.user_id == author)\
             .filter(UserLog.repository_id == repo) \
-            .order_by('user_log_id') \
+            .order_by(UserLog.user_log_id.asc()) \
             .all()
         assert journal[-2].action == 'repo.pull_request.merge'
         assert journal[-1].action == 'repo.pull_request.close'
@@ -221,7 +221,7 b' class TestMergePullRequest(object):'
         journal = UserLog.query() \
             .filter(UserLog.user_id == merge_user_id) \
             .filter(UserLog.repository_id == repo) \
-            .order_by('user_log_id') \
+            .order_by(UserLog.user_log_id.asc()) \
             .all()
         assert journal[-2].action == 'repo.pull_request.merge'
         assert journal[-1].action == 'repo.pull_request.close'

rhodecode/api/tests/test_update_user.py

@@ -42,7 +42,8 b' class TestUpdateUser(object):'
         ('extern_name', None),
         ('active', False),
         ('active', True),
-        ('password', 'newpass')
+        ('password', 'newpass'),
+        ('description', 'CTO 4 Life')
     ])
     def test_api_update_user(self, name, expected, user_util):
         usr = user_util.create_user()

rhodecode/api/tests/utils.py

@@ -20,11 +20,16 b''
 
 
 import random
+import pytest
 
 from rhodecode.api.utils import get_origin
 from rhodecode.lib.ext_json import json
 
 
+def jsonify(obj):
+    return json.loads(json.dumps(obj))
+
+
 API_URL = '/_admin/api'
 
 
@@ -42,12 +47,16 b' def assert_call_ok(id_, given):'
 
 
 def assert_ok(id_, expected, given):
+    given = json.loads(given)
+    if given.get('error'):
+        pytest.fail("Unexpected ERROR in success response: {}".format(given['error']))
+
     expected = jsonify({
         'id': id_,
         'error': None,
         'result': expected
     })
-    given = json.loads(given)
+
     assert expected == given
 
 
@@ -61,10 +70,6 b' def assert_error(id_, expected, given):'
     assert expected == given
 
 
-def jsonify(obj):
-    return json.loads(json.dumps(obj))
-
-
 def build_data(apikey, method, **kw):
     """
     Builds API data with given random ID

rhodecode/api/views/pull_request_api.py

@@ -451,7 +451,7 b' def comment_pull_request('
         request, apiuser, pullrequestid, repoid=Optional(None),
         message=Optional(None), commit_id=Optional(None), status=Optional(None),
         comment_type=Optional(ChangesetComment.COMMENT_TYPE_NOTE),
-        resolves_comment_id=Optional(None),
+        resolves_comment_id=Optional(None), extra_recipients=Optional([]),
         userid=Optional(OAttr('apiuser'))):
     """
     Comment on the pull request specified with the `pullrequestid`,
@@ -476,6 +476,11 b' def comment_pull_request('
     :type status: str
     :param comment_type: Comment type, one of: 'note', 'todo'
     :type comment_type: Optional(str), default: 'note'
+    :param resolves_comment_id: id of comment which this one will resolve
+    :type resolves_comment_id: Optional(int)
+    :param extra_recipients: list of user ids or usernames to add
+        notifications for this comment. Acts like a CC for notification
+    :type extra_recipients: Optional(list)
     :param userid: Comment on the pull request as this user
     :type userid: Optional(str or int)
 
@@ -521,6 +526,7 b' def comment_pull_request('
     commit_id = Optional.extract(commit_id)
     comment_type = Optional.extract(comment_type)
     resolves_comment_id = Optional.extract(resolves_comment_id)
+    extra_recipients = Optional.extract(extra_recipients)
 
     if not message and not status:
         raise JSONRPCError(
@@ -580,7 +586,8 b' def comment_pull_request('
         renderer=renderer,
         comment_type=comment_type,
         resolves_comment_id=resolves_comment_id,
-        auth_user=auth_user
+        auth_user=auth_user,
+        extra_recipients=extra_recipients
     )
 
     if allowed_to_change_status and status:
@@ -888,7 +895,9 b' def update_pull_request('
 
         with pull_request.set_state(PullRequest.STATE_UPDATING):
             if PullRequestModel().has_valid_update_type(pull_request):
-                update_response = PullRequestModel().update_commits(pull_request)
+                db_user = apiuser.get_instance()
+                update_response = PullRequestModel().update_commits(
+                    pull_request, db_user)
                 commit_changes = update_response.changes or commit_changes
             Session().commit()
 

rhodecode/api/views/repo_api.py

@@ -617,9 +617,7 b' def get_repo_fts_tree(request, apiuser, '
     cache_namespace_uid = 'cache_repo.{}'.format(repo_id)
     region = rc_cache.get_or_create_region('cache_repo', cache_namespace_uid)
 
-    @region.conditional_cache_on_arguments(namespace=cache_namespace_uid,
-                                           condition=cache_on)
-    def compute_fts_tree(repo_id, commit_id, root_path, cache_ver):
+    def compute_fts_tree(cache_ver, repo_id, commit_id, root_path):
         return ScmModel().get_fts_data(repo_id, commit_id, root_path)
 
     try:
@@ -640,7 +638,7 b' def get_repo_fts_tree(request, apiuser, '
             'with caching: %s[TTL: %ss]' % (
                 repo_id, commit_id, cache_on, cache_seconds or 0))
 
-        tree_files = compute_fts_tree(repo_id, commit_id, root_path, 'v1')
+        tree_files = compute_fts_tree(rc_cache.FILE_TREE_CACHE_VER, repo_id, commit_id, root_path)
         return tree_files
 
     except Exception:
@@ -714,7 +712,7 b' def create_repo('
         private=Optional(False),
         clone_uri=Optional(None),
         push_uri=Optional(None),
-        landing_rev=Optional('rev:tip'),
+        landing_rev=Optional(None),
         enable_statistics=Optional(False),
         enable_locking=Optional(False),
         enable_downloads=Optional(False),
@@ -749,7 +747,7 b' def create_repo('
     :type clone_uri: str
     :param push_uri: set push_uri
     :type push_uri: str
-    :param landing_rev: <rev_type>:<rev>
+    :param landing_rev: <rev_type>:<rev>, e.g branch:default, book:dev, rev:abcd
     :type landing_rev: str
     :param enable_locking:
     :type enable_locking: bool
@@ -793,7 +791,6 b' def create_repo('
     copy_permissions = Optional.extract(copy_permissions)
     clone_uri = Optional.extract(clone_uri)
     push_uri = Optional.extract(push_uri)
-    landing_commit_ref = Optional.extract(landing_rev)
 
     defs = SettingsModel().get_default_repo_settings(strip_prefix=True)
     if isinstance(private, Optional):
@@ -807,8 +804,15 b' def create_repo('
     if isinstance(enable_downloads, Optional):
         enable_downloads = defs.get('repo_enable_downloads')
 
+    landing_ref, _label = ScmModel.backend_landing_ref(repo_type)
+    ref_choices, _labels = ScmModel().get_repo_landing_revs(request.translate)
+    ref_choices = list(set(ref_choices + [landing_ref]))
+
+    landing_commit_ref = Optional.extract(landing_rev) or landing_ref
+
     schema = repo_schema.RepoSchema().bind(
         repo_type_options=rhodecode.BACKENDS.keys(),
+        repo_ref_options=ref_choices,
         repo_type=repo_type,
         # user caller
         user=apiuser)
@@ -958,7 +962,7 b' def update_repo('
         owner=Optional(OAttr('apiuser')), description=Optional(''),
         private=Optional(False),
         clone_uri=Optional(None), push_uri=Optional(None),
-        landing_rev=Optional('rev:tip'), fork_of=Optional(None),
+        landing_rev=Optional(None), fork_of=Optional(None),
         enable_statistics=Optional(False),
         enable_locking=Optional(False),
         enable_downloads=Optional(False), fields=Optional('')):
@@ -993,7 +997,7 b' def update_repo('
     :type private: bool
     :param clone_uri: Update the |repo| clone URI.
     :type clone_uri: str
-    :param landing_rev: Set the |repo| landing revision. Default is ``rev:tip``.
+    :param landing_rev: Set the |repo| landing revision. e.g branch:default, book:dev, rev:abcd
     :type landing_rev: str
     :param enable_statistics: Enable statistics on the |repo|, (True | False).
     :type enable_statistics: bool
@@ -1049,8 +1053,10 b' def update_repo('
         repo_enable_downloads=enable_downloads
         if not isinstance(enable_downloads, Optional) else repo.enable_downloads)
 
+    landing_ref, _label = ScmModel.backend_landing_ref(repo.repo_type)
     ref_choices, _labels = ScmModel().get_repo_landing_revs(
         request.translate, repo=repo)
+    ref_choices = list(set(ref_choices + [landing_ref]))
 
     old_values = repo.get_api_data()
     repo_type = repo.repo_type
@@ -1128,7 +1134,7 b' def fork_repo(request, apiuser, repoid, '
               description=Optional(''),
               private=Optional(False),
               clone_uri=Optional(None),
-              landing_rev=Optional('rev:tip'),
+              landing_rev=Optional(None),
               copy_permissions=Optional(False)):
     """
     Creates a fork of the specified |repo|.
@@ -1158,7 +1164,7 b' def fork_repo(request, apiuser, repoid, '
     :type copy_permissions: bool
     :param private: Make the fork private. The default is False.
     :type private: bool
-    :param landing_rev: Set the landing revision. The default is tip.
+    :param landing_rev: Set the landing revision. E.g branch:default, book:dev, rev:abcd
 
     Example output:
 
@@ -1210,11 +1216,17 b' def fork_repo(request, apiuser, repoid, '
     description = Optional.extract(description)
     copy_permissions = Optional.extract(copy_permissions)
     clone_uri = Optional.extract(clone_uri)
-    landing_commit_ref = Optional.extract(landing_rev)
+
+    landing_ref, _label = ScmModel.backend_landing_ref(repo.repo_type)
+    ref_choices, _labels = ScmModel().get_repo_landing_revs(request.translate)
+    ref_choices = list(set(ref_choices + [landing_ref]))
+    landing_commit_ref = Optional.extract(landing_rev) or landing_ref
+
     private = Optional.extract(private)
 
     schema = repo_schema.RepoSchema().bind(
         repo_type_options=rhodecode.BACKENDS.keys(),
+        repo_ref_options=ref_choices,
         repo_type=repo.repo_type,
         # user caller
         user=apiuser)
@@ -1538,7 +1550,7 b' def lock(request, apiuser, repoid, locke'
 def comment_commit(
         request, apiuser, repoid, commit_id, message, status=Optional(None),
         comment_type=Optional(ChangesetComment.COMMENT_TYPE_NOTE),
-        resolves_comment_id=Optional(None),
+        resolves_comment_id=Optional(None), extra_recipients=Optional([]),
         userid=Optional(OAttr('apiuser'))):
     """
     Set a commit comment, and optionally change the status of the commit.
@@ -1556,6 +1568,11 b' def comment_commit('
     :type status: str
     :param comment_type: Comment type, one of: 'note', 'todo'
     :type comment_type: Optional(str), default: 'note'
+    :param resolves_comment_id: id of comment which this one will resolve
+    :type resolves_comment_id: Optional(int)
+    :param extra_recipients: list of user ids or usernames to add
+        notifications for this comment. Acts like a CC for notification
+    :type extra_recipients: Optional(list)
     :param userid: Set the user name of the comment creator.
     :type userid: Optional(str or int)
 
@@ -1592,6 +1609,7 b' def comment_commit('
     status = Optional.extract(status)
     comment_type = Optional.extract(comment_type)
     resolves_comment_id = Optional.extract(resolves_comment_id)
+    extra_recipients = Optional.extract(extra_recipients)
 
     allowed_statuses = [x[0] for x in ChangesetStatus.STATUSES]
     if status and status not in allowed_statuses:
@@ -1620,7 +1638,8 b' def comment_commit('
             renderer=renderer,
             comment_type=comment_type,
             resolves_comment_id=resolves_comment_id,
-            auth_user=apiuser
+            auth_user=apiuser,
+            extra_recipients=extra_recipients
         )
         if status:
             # also do a status change

rhodecode/api/views/search_api.py

@@ -33,7 +33,7 b' log = logging.getLogger(__name__)'
 
 @jsonrpc_method()
 def search(request, apiuser, search_query, search_type, page_limit=Optional(10),
-           page=Optional(1), search_sort=Optional('newfirst'),
+           page=Optional(1), search_sort=Optional('desc:date'),
            repo_name=Optional(None), repo_group_name=Optional(None)):
     """
     Fetch Full Text Search results using API.
@@ -51,9 +51,15 b' def search(request, apiuser, search_quer'
     :type page_limit: Optional(int)
     :param page: Page number. Default first page.
     :type page: Optional(int)
-    :param search_sort: Search sort order. Default newfirst. The following are valid options:
-        * newfirst
-        * oldfirst
+    :param search_sort: Search sort order.Must start with asc: or desc: Default desc:date.
+        The following are valid options:
+        * asc|desc:message.raw
+        * asc|desc:date
+        * asc|desc:author.email.raw
+        * asc|desc:message.raw
+        * newfirst (old legacy equal to desc:date)
+        * oldfirst (old legacy equal to asc:date)
+
     :type search_sort: Optional(str)
     :param repo_name: Filter by one repo. Default is all.
     :type repo_name: Optional(str)
@@ -101,7 +107,7 b' def search(request, apiuser, search_quer'
             searcher.cleanup()
 
         if not search_result['error']:
-            data['execution_time'] = '%s results (%.3f seconds)' % (
+            data['execution_time'] = '%s results (%.4f seconds)' % (
                 search_result['count'],
                 search_result['runtime'])
         else:

rhodecode/api/views/user_api.py

@@ -75,6 +75,7 b' def get_user(request, apiuser, userid=Op'
             "extern_name": "rhodecode",
             "extern_type": "rhodecode",
             "firstname": "username",
+            "description": "user description",
             "ip_addresses": [],
             "language": null,
             "last_login": "Timestamp",
@@ -159,7 +160,7 b' def get_users(request, apiuser):'
 
 @jsonrpc_method()
 def create_user(request, apiuser, username, email, password=Optional(''),
-                firstname=Optional(''), lastname=Optional(''),
+                firstname=Optional(''), lastname=Optional(''), description=Optional(''),
                 active=Optional(True), admin=Optional(False),
                 extern_name=Optional('rhodecode'),
                 extern_type=Optional('rhodecode'),
@@ -185,6 +186,8 b' def create_user(request, apiuser, userna'
     :type firstname: Optional(str)
     :param lastname: Set the new user surname.
     :type lastname: Optional(str)
+    :param description: Set user description, or short bio. Metatags are allowed.
+    :type description: Optional(str)
     :param active: Set the user as active.
     :type active: Optional(``True`` | ``False``)
     :param admin: Give the new user admin rights.
@@ -250,6 +253,7 b' def create_user(request, apiuser, userna'
     email = Optional.extract(email)
     first_name = Optional.extract(firstname)
     last_name = Optional.extract(lastname)
+    description = Optional.extract(description)
     active = Optional.extract(active)
     admin = Optional.extract(admin)
     extern_type = Optional.extract(extern_type)
@@ -267,6 +271,7 b' def create_user(request, apiuser, userna'
             last_name=last_name,
             active=active,
             admin=admin,
+            description=description,
             extern_type=extern_type,
             extern_name=extern_name,
             ))
@@ -280,6 +285,7 b' def create_user(request, apiuser, userna'
             email=schema_data['email'],
             firstname=schema_data['first_name'],
             lastname=schema_data['last_name'],
+            description=schema_data['description'],
             active=schema_data['active'],
             admin=schema_data['admin'],
             extern_type=schema_data['extern_type'],
@@ -307,7 +313,7 b' def create_user(request, apiuser, userna'
 def update_user(request, apiuser, userid, username=Optional(None),
                 email=Optional(None), password=Optional(None),
                 firstname=Optional(None), lastname=Optional(None),
-                active=Optional(None), admin=Optional(None),
+                description=Optional(None), active=Optional(None), admin=Optional(None),
                 extern_type=Optional(None), extern_name=Optional(None), ):
     """
     Updates the details for the specified user, if that user exists.
@@ -331,6 +337,8 b' def update_user(request, apiuser, userid'
     :type firstname: Optional(str)
     :param lastname: Set the new surname.
     :type lastname: Optional(str)
+    :param description: Set user description, or short bio. Metatags are allowed.
+    :type description: Optional(str)
     :param active: Set the new user as active.
     :type active: Optional(``True`` | ``False``)
     :param admin: Give the user admin rights.
@@ -379,6 +387,7 b' def update_user(request, apiuser, userid'
         store_update(updates, email, 'email')
         store_update(updates, firstname, 'name')
         store_update(updates, lastname, 'lastname')
+        store_update(updates, description, 'description')
         store_update(updates, active, 'active')
         store_update(updates, admin, 'admin')
         store_update(updates, extern_name, 'extern_name')

rhodecode/api/views/user_group_api.py

@@ -191,7 +191,9 b' def create_user_group('
     :param active: Set this group as active.
     :type active: Optional(``True`` | ``False``)
     :param sync: Set enabled or disabled the automatically sync from
-        external authentication types like ldap.
+        external authentication types like ldap. If User Group will be named like
+        one from e.g ldap and sync flag is enabled members will be synced automatically.
+        Sync type when enabled via API is set to `manual_api`
     :type sync: Optional(``True`` | ``False``)
 
     Example output:
@@ -303,7 +305,9 b' def update_user_group(request, apiuser, '
     :param active: Set the group as active.
     :type active: Optional(``True`` | ``False``)
     :param sync: Set enabled or disabled the automatically sync from
-        external authentication types like ldap.
+        external authentication types like ldap. If User Group will be named like
+        one from e.g ldap and sync flag is enabled members will be synced automatically.
+        Sync type when enabled via API is set to `manual_api`
     :type sync: Optional(``True`` | ``False``)
 
     Example output:

rhodecode/apps/_base/__init__.py

@@ -25,9 +25,11 b' import operator'
 from pyramid import compat
 from pyramid.httpexceptions import HTTPFound, HTTPForbidden, HTTPBadRequest
 
-from rhodecode.lib import helpers as h, diffs
+from rhodecode.lib import helpers as h, diffs, rc_cache
 from rhodecode.lib.utils2 import (
     StrictAttributeDict, str2bool, safe_int, datetime_to_time, safe_unicode)
+from rhodecode.lib.markup_renderer import MarkupRenderer, relative_links
+from rhodecode.lib.vcs.backends.base import EmptyCommit
 from rhodecode.lib.vcs.exceptions import RepositoryRequirementError
 from rhodecode.model import repo
 from rhodecode.model import repo_group
@@ -36,6 +38,7 b' from rhodecode.model import user'
 from rhodecode.model.db import User
 from rhodecode.model.scm import ScmModel
 from rhodecode.model.settings import VcsSettingsModel
+from rhodecode.model.repo import ReadmeFinder
 
 log = logging.getLogger(__name__)
 
@@ -222,6 +225,7 b' class RepoAppView(BaseAppView):'
         self.db_repo = request.db_repo
         self.db_repo_name = self.db_repo.repo_name
         self.db_repo_pull_requests = ScmModel().get_pull_requests(self.db_repo)
+        self.db_repo_artifacts = ScmModel().get_artifacts(self.db_repo)
 
     def _handle_missing_requirements(self, error):
         log.error(
@@ -237,6 +241,7 b' class RepoAppView(BaseAppView):'
         c.rhodecode_db_repo = self.db_repo
         c.repo_name = self.db_repo_name
         c.repository_pull_requests = self.db_repo_pull_requests
+        c.repository_artifacts = self.db_repo_artifacts
         c.repository_is_user_following = ScmModel().is_following_repo(
             self.db_repo_name, self._rhodecode_user.user_id)
         self.path_filter = PathFilter(None)
@@ -305,6 +310,69 b' class RepoAppView(BaseAppView):'
         settings = settings_model.get_general_settings()
         return settings.get(settings_key, default)
 
+    def _get_repo_setting(self, target_repo, settings_key, default=False):
+        settings_model = VcsSettingsModel(repo=target_repo)
+        settings = settings_model.get_repo_settings_inherited()
+        return settings.get(settings_key, default)
+
+    def _get_readme_data(self, db_repo, renderer_type, commit_id=None, path='/'):
+        log.debug('Looking for README file at path %s', path)
+        if commit_id:
+            landing_commit_id = commit_id
+        else:
+            landing_commit = db_repo.get_landing_commit()
+            if isinstance(landing_commit, EmptyCommit):
+                return None, None
+            landing_commit_id = landing_commit.raw_id
+
+        cache_namespace_uid = 'cache_repo.{}'.format(db_repo.repo_id)
+        region = rc_cache.get_or_create_region('cache_repo', cache_namespace_uid)
+        start = time.time()
+
+        @region.conditional_cache_on_arguments(namespace=cache_namespace_uid)
+        def generate_repo_readme(repo_id, _commit_id, _repo_name, _readme_search_path, _renderer_type):
+            readme_data = None
+            readme_filename = None
+
+            commit = db_repo.get_commit(_commit_id)
+            log.debug("Searching for a README file at commit %s.", _commit_id)
+            readme_node = ReadmeFinder(_renderer_type).search(commit, path=_readme_search_path)
+
+            if readme_node:
+                log.debug('Found README node: %s', readme_node)
+                relative_urls = {
+                    'raw': h.route_path(
+                        'repo_file_raw', repo_name=_repo_name,
+                        commit_id=commit.raw_id, f_path=readme_node.path),
+                    'standard': h.route_path(
+                        'repo_files', repo_name=_repo_name,
+                        commit_id=commit.raw_id, f_path=readme_node.path),
+                }
+                readme_data = self._render_readme_or_none(commit, readme_node, relative_urls)
+                readme_filename = readme_node.unicode_path
+
+            return readme_data, readme_filename
+
+        readme_data, readme_filename = generate_repo_readme(
+            db_repo.repo_id, landing_commit_id, db_repo.repo_name, path, renderer_type,)
+        compute_time = time.time() - start
+        log.debug('Repo README for path %s generated and computed in %.4fs',
+                  path, compute_time)
+        return readme_data, readme_filename
+
+    def _render_readme_or_none(self, commit, readme_node, relative_urls):
+        log.debug('Found README file `%s` rendering...', readme_node.path)
+        renderer = MarkupRenderer()
+        try:
+            html_source = renderer.render(
+                readme_node.content, filename=readme_node.path)
+            if relative_urls:
+                return relative_links(html_source, relative_urls)
+            return html_source
+        except Exception:
+            log.exception(
+                "Exception while trying to render the README")
+
     def get_recache_flag(self):
         for flag_name in ['force_recache', 'force-recache', 'no-cache']:
             flag_val = self.request.GET.get(flag_name)
@@ -464,7 +532,7 b' class BaseReferencesView(RepoAppView):'
 
     def load_refs_context(self, ref_items, partials_template):
         _render = self.request.get_partial_renderer(partials_template)
-        pre_load = ["author", "date", "message"]
+        pre_load = ["author", "date", "message", "parents"]
 
         is_svn = h.is_svn(self.rhodecode_vcs_repo)
         is_hg = h.is_hg(self.rhodecode_vcs_repo)

rhodecode/apps/admin/__init__.py

@@ -140,7 +140,6 b' def admin_routes(config):'
         name='admin_settings_visual_update',
         pattern='/settings/visual/update')
 
-
     config.add_route(
         name='admin_settings_issuetracker',
         pattern='/settings/issue-tracker')
@@ -378,6 +377,10 b' def admin_routes(config):'
         name='edit_user_audit_logs',
         pattern='/users/{user_id:\d+}/edit/audit', user_route=True)
 
+    config.add_route(
+        name='edit_user_audit_logs_download',
+        pattern='/users/{user_id:\d+}/edit/audit/download', user_route=True)
+
     # user caches
     config.add_route(
         name='edit_user_caches',
@@ -411,6 +414,10 b' def admin_routes(config):'
         pattern='/repos')
 
     config.add_route(
+        name='repos_data',
+        pattern='/repos_data')
+
+    config.add_route(
         name='repo_new',
         pattern='/repos/new')
 

rhodecode/apps/admin/tests/test_admin_auth_settings.py

@@ -155,7 +155,7 b' class TestAuthSettingsView(object):'
         response = self._post_ldap_settings(params, override={
             'port': invalid_port_value,
         })
-        assertr = AssertResponse(response)
+        assertr = response.assert_response()
         assertr.element_contains(
             '.form .field #port ~ .error-message',
             invalid_port_value)

rhodecode/apps/admin/tests/test_admin_repos.py

@@ -47,6 +47,7 b' def route_path(name, params=None, **kwar'
 
     base_url = {
         'repos': ADMIN_PREFIX + '/repos',
+        'repos_data': ADMIN_PREFIX + '/repos_data',
         'repo_new': ADMIN_PREFIX + '/repos/new',
         'repo_create': ADMIN_PREFIX + '/repos/create',
 
@@ -70,24 +71,25 b' def _get_permission_for_user(user, repo)'
 @pytest.mark.usefixtures("app")
 class TestAdminRepos(object):
 
-    def test_repo_list(self, autologin_user, user_util):
+    def test_repo_list(self, autologin_user, user_util, xhr_header):
         repo = user_util.create_repo()
         repo_name = repo.repo_name
         response = self.app.get(
-            route_path('repos'), status=200)
+            route_path('repos_data'), status=200,
+            extra_environ=xhr_header)
 
         response.mustcontain(repo_name)
 
     def test_create_page_restricted_to_single_backend(self, autologin_user, backend):
         with mock.patch('rhodecode.BACKENDS', {'git': 'git'}):
             response = self.app.get(route_path('repo_new'), status=200)
-        assert_response = AssertResponse(response)
+        assert_response = response.assert_response()
         element = assert_response.get_element('#repo_type')
         assert element.text_content() == '\ngit\n'
 
     def test_create_page_non_restricted_backends(self, autologin_user, backend):
         response = self.app.get(route_path('repo_new'), status=200)
-        assert_response = AssertResponse(response)
+        assert_response = response.assert_response()
         assert_response.element_contains('#repo_type', 'git')
         assert_response.element_contains('#repo_type', 'svn')
         assert_response.element_contains('#repo_type', 'hg')

rhodecode/apps/admin/tests/test_admin_repository_groups.py

@@ -84,7 +84,7 b' class TestAdminRepositoryGroups(object):'
         fixture.create_repo_group('test_repo_group')
         response = self.app.get(route_path(
             'repo_groups_data'), extra_environ=xhr_header)
-        response.mustcontain('"name_raw": "test_repo_group"')
+        response.mustcontain('<a href=\\"/{}/_edit\\" title=\\"Edit\\">Edit</a>'.format('test_repo_group'))
         fixture.destroy_repo_group('test_repo_group')
 
     def test_new(self, autologin_user):

rhodecode/apps/admin/tests/test_admin_settings.py

@@ -367,7 +367,7 b' class TestAdminSettingsVcs(object):'
 
     def test_has_an_input_for_invalidation_of_inline_comments(self):
         response = self.app.get(route_path('admin_settings_vcs'))
-        assert_response = AssertResponse(response)
+        assert_response = response.assert_response()
         assert_response.one_element_exists(
             '[name=rhodecode_use_outdated_comments]')
 
@@ -412,14 +412,14 b' class TestAdminSettingsVcs(object):'
         setting = SettingsModel().get_setting_by_name(setting_key)
         assert setting.app_settings_value is new_value
 
-    @pytest.fixture
+    @pytest.fixture()
     def disable_sql_cache(self, request):
         patcher = mock.patch(
             'rhodecode.lib.caching_query.FromCache.process_query')
         request.addfinalizer(patcher.stop)
         patcher.start()
 
-    @pytest.fixture
+    @pytest.fixture()
     def form_defaults(self):
         from rhodecode.apps.admin.views.settings import AdminSettingsView
         return AdminSettingsView._form_defaults()
@@ -518,7 +518,7 b' class TestOpenSourceLicenses(object):'
             response = self.app.get(
                 route_path('admin_settings_open_source'), status=200)
 
-        assert_response = AssertResponse(response)
+        assert_response = response.assert_response()
         assert_response.element_contains(
             '.panel-heading', 'Licenses of Third Party Packages')
         for license_data in sample_licenses:
@@ -528,7 +528,7 b' class TestOpenSourceLicenses(object):'
     def test_records_can_be_read(self, autologin_user):
         response = self.app.get(
             route_path('admin_settings_open_source'), status=200)
-        assert_response = AssertResponse(response)
+        assert_response = response.assert_response()
         assert_response.element_contains(
             '.panel-heading', 'Licenses of Third Party Packages')
 
@@ -726,7 +726,7 b' class TestAdminSettingsIssueTracker(obje'
             IssueTrackerSettingsModel().delete_entries(self.uid)
 
     def test_delete_issuetracker_pattern(
-            self, autologin_user, backend, csrf_token, settings_util):
+            self, autologin_user, backend, csrf_token, settings_util, xhr_header):
         pattern = 'issuetracker_pat'
         uid = md5(pattern)
         settings_util.create_rhodecode_setting(
@@ -734,10 +734,9 b' class TestAdminSettingsIssueTracker(obje'
 
         post_url = route_path('admin_settings_issuetracker_delete')
         post_data = {
-            '_method': 'delete',
             'uid': uid,
             'csrf_token': csrf_token
         }
-        self.app.post(post_url, post_data, status=302)
+        self.app.post(post_url, post_data, extra_environ=xhr_header, status=200)
         settings = SettingsModel().get_all_settings()
         assert 'rhodecode_%s%s' % (self.SHORT_PATTERN_KEY, uid) not in settings

rhodecode/apps/admin/tests/test_admin_users.py

@@ -91,6 +91,9 b' def route_path(name, params=None, **kwar'
         'edit_user_audit_logs':
             ADMIN_PREFIX + '/users/{user_id}/edit/audit',
 
+        'edit_user_audit_logs_download':
+            ADMIN_PREFIX + '/users/{user_id}/edit/audit/download',
+
     }[name].format(**kwargs)
 
     if params:
@@ -318,7 +321,6 b' class TestAdminUsersView(TestController)'
             route_path('edit_user_emails', user_id=user_id))
         response.mustcontain(no=['[e-mail]'])
 
-
     def test_create(self, request, xhr_header):
         self.log_user()
         username = 'newtestuser'
@@ -333,6 +335,7 b' class TestAdminUsersView(TestController)'
         response = self.app.post(route_path('users_create'), params={
             'username': username,
             'password': password,
+            'description': 'mr CTO',
             'password_confirmation': password_confirmation,
             'firstname': name,
             'active': True,
@@ -381,6 +384,7 b' class TestAdminUsersView(TestController)'
             'name': name,
             'active': False,
             'lastname': lastname,
+            'description': 'mr CTO',
             'email': email,
             'csrf_token': self.csrf_token,
         })
@@ -418,6 +422,7 b' class TestAdminUsersView(TestController)'
         ('email', {'email': '[e-mail]'}),
         ('language', {'language': 'de'}),
         ('language', {'language': 'en'}),
+        ('description', {'description': 'hello CTO'}),
         # ('new_password', {'new_password': 'foobar123',
         #                   'password_confirmation': 'foobar123'})
         ])
@@ -515,7 +520,7 b' class TestAdminUsersView(TestController)'
             route_path('user_delete', user_id=new_user.user_id),
             params={'csrf_token': self.csrf_token})
 
-        assert_session_flash(response, 'Successfully deleted user')
+        assert_session_flash(response, 'Successfully deleted user `{}`'.format(username))
 
     def test_delete_owner_of_repository(self, request, user_util):
         self.log_user()
@@ -531,8 +536,7 b' class TestAdminUsersView(TestController)'
             params={'csrf_token': self.csrf_token})
 
         msg = 'user "%s" still owns 1 repositories and cannot be removed. ' \
-              'Switch owners or remove those repositories:%s' % (username,
-                                                                 obj_name)
+              'Switch owners or remove those repositories:%s' % (username, obj_name)
         assert_session_flash(response, msg)
         fixture.destroy_repo(obj_name)
 
@@ -542,6 +546,7 b' class TestAdminUsersView(TestController)'
         usr = user_util.create_user(auto_cleanup=False)
         username = usr.username
         fixture.create_repo(obj_name, cur_user=usr.username)
+        Session().commit()
 
         new_user = Session().query(User)\
             .filter(User.username == username).one()
@@ -583,8 +588,7 b' class TestAdminUsersView(TestController)'
             params={'csrf_token': self.csrf_token})
 
         msg = 'user "%s" still owns 1 repository groups and cannot be removed. ' \
-              'Switch owners or remove those repository groups:%s' % (username,
-                                                                      obj_name)
+              'Switch owners or remove those repository groups:%s' % (username, obj_name)
         assert_session_flash(response, msg)
         fixture.destroy_repo_group(obj_name)
 
@@ -635,8 +639,7 b' class TestAdminUsersView(TestController)'
             params={'csrf_token': self.csrf_token})
 
         msg = 'user "%s" still owns 1 user groups and cannot be removed. ' \
-              'Switch owners or remove those user groups:%s' % (username,
-                                                                obj_name)
+              'Switch owners or remove those user groups:%s' % (username, obj_name)
         assert_session_flash(response, msg)
         fixture.destroy_user_group(obj_name)
 
@@ -779,3 +782,13 b' class TestAdminUsersView(TestController)'
         user = self.log_user()
         self.app.get(
             route_path('edit_user_audit_logs', user_id=user['user_id']))
+
+    def test_audit_log_page_download(self):
+        user = self.log_user()
+        user_id = user['user_id']
+        response = self.app.get(
+            route_path('edit_user_audit_logs_download', user_id=user_id))
+
+        assert response.content_disposition == \
+               'attachment; filename=user_{}_audit_logs.json'.format(user_id)
+        assert response.content_type == "application/json"

rhodecode/apps/admin/views/audit_logs.py

@@ -28,7 +28,7 b' from rhodecode.model.db import joinedloa'
 from rhodecode.lib.user_log_filter import user_log_filter
 from rhodecode.lib.auth import LoginRequired, HasPermissionAllDecorator
 from rhodecode.lib.utils2 import safe_int
-from rhodecode.lib.helpers import Page
+from rhodecode.lib.helpers import SqlPage
 
 log = logging.getLogger(__name__)
 
@@ -62,13 +62,16 b' class AdminAuditLogsView(BaseAppView):'
 
         p = safe_int(self.request.GET.get('page', 1), 1)
 
-        def url_generator(**kw):
+        def url_generator(page_num):
+            query_params = {
+                'page': page_num
+            }
             if c.search_term:
-                kw['filter'] = c.search_term
-            return self.request.current_route_path(_query=kw)
+                query_params['filter'] = c.search_term
+            return self.request.current_route_path(_query=query_params)
 
-        c.audit_logs = Page(users_log, page=p, items_per_page=10,
-                            url=url_generator)
+        c.audit_logs = SqlPage(users_log, page=p, items_per_page=10,
+                               url_maker=url_generator)
         return self._get_template_context(c)
 
     @LoginRequired()

rhodecode/apps/admin/views/main_views.py

@@ -25,7 +25,7 b' from pyramid.view import view_config'
 
 from rhodecode.apps._base import BaseAppView
 from rhodecode.lib import helpers as h
-from rhodecode.lib.auth import (LoginRequired, NotAnonymous)
+from rhodecode.lib.auth import (LoginRequired, NotAnonymous, HasRepoPermissionAny)
 from rhodecode.model.db import PullRequest
 
 
@@ -66,6 +66,13 b' class AdminMainView(BaseAppView):'
         pull_request_id = pull_request.pull_request_id
 
         repo_name = pull_request.target_repo.repo_name
+        # NOTE(marcink):
+        # check permissions so we don't redirect to repo that we don't have access to
+        # exposing it's name
+        target_repo_perm = HasRepoPermissionAny(
+            'repository.read', 'repository.write', 'repository.admin')(repo_name)
+        if not target_repo_perm:
+            raise HTTPNotFound()
 
         raise HTTPFound(
             h.route_path('pullrequest_show', repo_name=repo_name,

rhodecode/apps/admin/views/permissions.py

@@ -68,7 +68,8 b' class AdminPermissionsView(BaseAppView, '
 
         c.user = User.get_default_user(refresh=True)
 
-        app_settings = SettingsModel().get_all_settings()
+        app_settings = c.rc_config
+
         defaults = {
             'anonymous': c.user.active,
             'default_register_message': app_settings.get(

rhodecode/apps/admin/views/process_management.py

@@ -47,7 +47,7 b' class AdminProcessManagementView(BaseApp'
                 'name': proc.name(),
                 'mem_rss': mem.rss,
                 'mem_vms': mem.vms,
-                'cpu_percent': proc.cpu_percent(),
+                'cpu_percent': proc.cpu_percent(interval=0.1),
                 'create_time': proc.create_time(),
                 'cmd': ' '.join(proc.cmdline()),
             })

rhodecode/apps/admin/views/repo_groups.py

@@ -19,6 +19,8 b''
 # and proprietary license terms, please see https://rhodecode.com/licenses/
 import datetime
 import logging
+import time
+
 import formencode
 import formencode.htmlfill
 
@@ -63,7 +65,7 b' class AdminRepoGroupsView(BaseAppView, D'
         # and display only those we have ADMIN right
         groups_with_admin_rights = RepoGroupList(
             RepoGroup.query().all(),
-            perm_set=['group.admin'])
+            perm_set=['group.admin'], extra_kwargs=dict(user=self._rhodecode_user))
         c.repo_groups = RepoGroup.groups_choices(
             groups=groups_with_admin_rights,
             show_empty_group=allow_empty_group)
@@ -109,9 +111,9 b' class AdminRepoGroupsView(BaseAppView, D'
     def repo_group_list_data(self):
         self.load_default_context()
         column_map = {
-            'name_raw': 'group_name_hash',
+            'name': 'group_name_hash',
             'desc': 'group_description',
-            'last_change_raw': 'updated_on',
+            'last_change': 'updated_on',
             'top_level_repos': 'repos_total',
             'owner': 'user_username',
         }
@@ -131,9 +133,10 b' class AdminRepoGroupsView(BaseAppView, D'
 
         def last_change(last_change):
             if isinstance(last_change, datetime.datetime) and not last_change.tzinfo:
-                delta = datetime.timedelta(
-                    seconds=(datetime.datetime.now() - datetime.datetime.utcnow()).seconds)
-                last_change = last_change + delta
+                ts = time.time()
+                utc_offset = (datetime.datetime.fromtimestamp(ts)
+                              - datetime.datetime.utcfromtimestamp(ts)).total_seconds()
+                last_change = last_change + datetime.timedelta(seconds=utc_offset)
             return _render("last_change", last_change)
 
         def desc(desc, personal):
@@ -147,12 +150,8 b' class AdminRepoGroupsView(BaseAppView, D'
         def user_profile(username):
             return _render('user_profile', username)
 
-        auth_repo_group_list = RepoGroupList(
-            RepoGroup.query().all(), perm_set=['group.admin'])
-
-        allowed_ids = [-1]
-        for repo_group in auth_repo_group_list:
-                allowed_ids.append(repo_group.group_id)
+        _perms = ['group.admin']
+        allowed_ids = [-1] + self._rhodecode_user.repo_group_acl_ids_from_stack(_perms)
 
         repo_groups_data_total_count = RepoGroup.query()\
             .filter(or_(
@@ -180,7 +179,7 b' class AdminRepoGroupsView(BaseAppView, D'
                 # generate multiple IN to fix limitation problems
                 *in_filter_generator(RepoGroup.group_id, allowed_ids)
             )) \
-            .outerjoin(Repository) \
+            .outerjoin(Repository,  Repository.group_id == RepoGroup.group_id) \
             .join(User, User.user_id == RepoGroup.user_id) \
             .group_by(RepoGroup, User)
 
@@ -224,9 +223,8 b' class AdminRepoGroupsView(BaseAppView, D'
             row = {
                 "menu": quick_menu(repo_gr.group_name),
                 "name": repo_group_lnk(repo_gr.group_name),
-                "name_raw": repo_gr.group_name,
+
                 "last_change": last_change(repo_gr.updated_on),
-                "last_change_raw": datetime_to_time(repo_gr.updated_on),
 
                 "last_changeset": "",
                 "last_changeset_raw": "",

rhodecode/apps/admin/views/repositories.py

@@ -31,7 +31,6 b' from rhodecode import events'
 from rhodecode.apps._base import BaseAppView, DataGridAppView
 from rhodecode.lib.celerylib.utils import get_task_id
 
-from rhodecode.lib.ext_json import json
 from rhodecode.lib.auth import (
     LoginRequired, CSRFRequired, NotAnonymous,
     HasPermissionAny, HasRepoGroupPermissionAny)
@@ -43,7 +42,8 b' from rhodecode.model.permission import P'
 from rhodecode.model.repo import RepoModel
 from rhodecode.model.scm import RepoList, RepoGroupList, ScmModel
 from rhodecode.model.settings import SettingsModel
-from rhodecode.model.db import Repository, RepoGroup
+from rhodecode.model.db import (
+    in_filter_generator, or_, func, Session, Repository, RepoGroup, User)
 
 log = logging.getLogger(__name__)
 
@@ -60,8 +60,6 b' class AdminReposView(BaseAppView, DataGr'
                                    perm_set=['group.write', 'group.admin'])
         c.repo_groups = RepoGroup.groups_choices(groups=acl_groups)
         c.repo_groups_choices = map(lambda k: safe_unicode(k[0]), c.repo_groups)
-        c.landing_revs_choices, c.landing_revs = \
-            ScmModel().get_repo_landing_revs(self.request.translate)
         c.personal_repo_group = self._rhodecode_user.personal_repo_group
 
     @LoginRequired()
@@ -72,15 +70,94 b' class AdminReposView(BaseAppView, DataGr'
         renderer='rhodecode:templates/admin/repos/repos.mako')
     def repository_list(self):
         c = self.load_default_context()
+        return self._get_template_context(c)
 
-        repo_list = Repository.get_all_repos()
-        c.repo_list = RepoList(repo_list, perm_set=['repository.admin'])
+    @LoginRequired()
+    @NotAnonymous()
+    # perms check inside
+    @view_config(
+        route_name='repos_data', request_method='GET',
+        renderer='json_ext', xhr=True)
+    def repository_list_data(self):
+        self.load_default_context()
+        column_map = {
+            'name': 'repo_name',
+            'desc': 'description',
+            'last_change': 'updated_on',
+            'owner': 'user_username',
+        }
+        draw, start, limit = self._extract_chunk(self.request)
+        search_q, order_by, order_dir = self._extract_ordering(
+            self.request, column_map=column_map)
+
+        _perms = ['repository.admin']
+        allowed_ids = [-1] + self._rhodecode_user.repo_acl_ids_from_stack(_perms)
+
+        repos_data_total_count = Repository.query() \
+            .filter(or_(
+                # generate multiple IN to fix limitation problems
+                *in_filter_generator(Repository.repo_id, allowed_ids))
+            ) \
+            .count()
+
+        base_q = Session.query(
+            Repository.repo_id,
+            Repository.repo_name,
+            Repository.description,
+            Repository.repo_type,
+            Repository.repo_state,
+            Repository.private,
+            Repository.archived,
+            Repository.fork,
+            Repository.updated_on,
+            Repository._changeset_cache,
+            User,
+            ) \
+            .filter(or_(
+                # generate multiple IN to fix limitation problems
+                *in_filter_generator(Repository.repo_id, allowed_ids))
+            ) \
+            .join(User, User.user_id == Repository.user_id) \
+            .group_by(Repository, User)
+
+        if search_q:
+            like_expression = u'%{}%'.format(safe_unicode(search_q))
+            base_q = base_q.filter(or_(
+                Repository.repo_name.ilike(like_expression),
+            ))
+
+        repos_data_total_filtered_count = base_q.count()
+
+        sort_defined = False
+        if order_by == 'repo_name':
+            sort_col = func.lower(Repository.repo_name)
+            sort_defined = True
+        elif order_by == 'user_username':
+            sort_col = User.username
+        else:
+            sort_col = getattr(Repository, order_by, None)
+
+        if sort_defined or sort_col:
+            if order_dir == 'asc':
+                sort_col = sort_col.asc()
+            else:
+                sort_col = sort_col.desc()
+
+        base_q = base_q.order_by(sort_col)
+        base_q = base_q.offset(start).limit(limit)
+
+        repos_list = base_q.all()
+
         repos_data = RepoModel().get_repos_as_dict(
-            repo_list=c.repo_list, admin=True, super_user_actions=True)
-        # json used to render the grid
-        c.data = json.dumps(repos_data)
+            repo_list=repos_list, admin=True, super_user_actions=True)
 
-        return self._get_template_context(c)
+        data = ({
+            'draw': draw,
+            'data': repos_data,
+            'recordsTotal': repos_data_total_count,
+            'recordsFiltered': repos_data_total_filtered_count,
+        })
+        return data
 
     @LoginRequired()
     @NotAnonymous()
@@ -151,8 +228,7 b' class AdminReposView(BaseAppView, DataGr'
         try:
             # CanWriteToGroup validators checks permissions of this POST
             form = RepoForm(
-                self.request.translate, repo_groups=c.repo_groups_choices,
-                landing_revs=c.landing_revs_choices)()
+                self.request.translate, repo_groups=c.repo_groups_choices)()
             form_result = form.to_python(dict(self.request.POST))
             copy_permissions = form_result.get('repo_copy_permissions')
             # create is done sometimes async on celery, db transaction

rhodecode/apps/admin/views/settings.py

@@ -445,7 +445,7 b' class AdminSettingsView(BaseAppView):'
     def settings_issuetracker(self):
         c = self.load_default_context()
         c.active = 'issuetracker'
-        defaults = SettingsModel().get_all_settings()
+        defaults = c.rc_config
 
         entry_key = 'rhodecode_issuetracker_pat_'
 
@@ -518,7 +518,7 b' class AdminSettingsView(BaseAppView):'
     @CSRFRequired()
     @view_config(
         route_name='admin_settings_issuetracker_delete', request_method='POST',
-        renderer='rhodecode:templates/admin/settings/settings.mako')
+        renderer='json_ext', xhr=True)
     def settings_issuetracker_delete(self):
         _ = self.request.translate
         self.load_default_context()
@@ -528,8 +528,11 b' class AdminSettingsView(BaseAppView):'
         except Exception:
             log.exception('Failed to delete issue tracker setting %s', uid)
             raise HTTPNotFound()
-        h.flash(_('Removed issue tracker entry'), category='success')
-        raise HTTPFound(h.route_path('admin_settings_issuetracker'))
+
+        SettingsModel().invalidate_settings_cache()
+        h.flash(_('Removed issue tracker entry.'), category='success')
+
+        return {'deleted': uid}
 
     @LoginRequired()
     @HasPermissionAllDecorator('hg.admin')
@@ -570,8 +573,7 b' class AdminSettingsView(BaseAppView):'
 
         email_kwargs = {
             'date': datetime.datetime.now(),
-            'user': c.rhodecode_user,
-            'rhodecode_version': c.rhodecode_version
+            'user': c.rhodecode_user
         }
 
         (subject, headers, email_body,

rhodecode/apps/admin/views/system_info.py

@@ -155,6 +155,10 b' class AdminSystemInfoSettingsView(BaseAp'
 
         ]
 
+        c.vcsserver_data_items = [
+            (k, v) for k,v in (val('vcs_server_config') or {}).items()
+        ]
+
         if snapshot:
             if c.allowed_to_snapshot:
                 c.data_items.pop(0)  # remove server info

rhodecode/apps/admin/views/user_groups.py

@@ -99,12 +99,8 b' class AdminUserGroupsView(BaseAppView, D'
         def user_profile(username):
             return _render('user_profile', username)
 
-        auth_user_group_list = UserGroupList(
-            UserGroup.query().all(), perm_set=['usergroup.admin'])
-
-        allowed_ids = [-1]
-        for user_group in auth_user_group_list:
-                allowed_ids.append(user_group.users_group_id)
+        _perms = ['usergroup.admin']
+        allowed_ids = [-1] + self._rhodecode_user.user_group_acl_ids_from_stack(_perms)
 
         user_groups_data_total_count = UserGroup.query()\
             .filter(or_(
@@ -134,7 +130,7 b' class AdminUserGroupsView(BaseAppView, D'
                 # generate multiple IN to fix limitation problems
                 *in_filter_generator(UserGroup.users_group_id, allowed_ids)
             )) \
-            .outerjoin(UserGroupMember) \
+            .outerjoin(UserGroupMember, UserGroupMember.users_group_id == UserGroup.users_group_id) \
             .join(User, User.user_id == UserGroup.user_id) \
             .group_by(UserGroup, User)
 
@@ -175,7 +171,6 b' class AdminUserGroupsView(BaseAppView, D'
         for user_gr in auth_user_group_list:
             row = {
                 "users_group_name": user_group_name(user_gr.users_group_name),
-                "name_raw": h.escape(user_gr.users_group_name),
                 "description": h.escape(user_gr.user_group_description),
                 "members": user_gr.member_count,
                 # NOTE(marcink): because of advanced query we

rhodecode/apps/admin/views/users.py

@@ -31,6 +31,7 b' from pyramid.response import Response'
 from rhodecode import events
 from rhodecode.apps._base import BaseAppView, DataGridAppView, UserAppView
 from rhodecode.apps.ssh_support import SshKeyFileChangeEvent
+from rhodecode.authentication.base import get_authn_registry, RhodeCodeExternalAuthPlugin
 from rhodecode.authentication.plugins import auth_rhodecode
 from rhodecode.events import trigger
 from rhodecode.model.db import true
@@ -43,6 +44,7 b' from rhodecode.lib.ext_json import json'
 from rhodecode.lib.auth import (
     LoginRequired, HasPermissionAllDecorator, CSRFRequired)
 from rhodecode.lib import helpers as h
+from rhodecode.lib.helpers import SqlPage
 from rhodecode.lib.utils2 import safe_int, safe_unicode, AttributeDict
 from rhodecode.model.auth_token import AuthTokenModel
 from rhodecode.model.forms import (
@@ -249,7 +251,32 b' class UsersView(UserAppView):'
     in there as well.
     """
 
+    def get_auth_plugins(self):
+        valid_plugins = []
+        authn_registry = get_authn_registry(self.request.registry)
+        for plugin in authn_registry.get_plugins_for_authentication():
+            if isinstance(plugin, RhodeCodeExternalAuthPlugin):
+                valid_plugins.append(plugin)
+            elif plugin.name == 'rhodecode':
+                valid_plugins.append(plugin)
+
+        # extend our choices if user has set a bound plugin which isn't enabled at the
+        # moment
+        extern_type = self.db_user.extern_type
+        if extern_type not in [x.uid for x in valid_plugins]:
+            try:
+                plugin = authn_registry.get_plugin_by_uid(extern_type)
+                if plugin:
+                    valid_plugins.append(plugin)
+
+            except Exception:
+                log.exception(
+                    'Could not extend user plugins with `{}`'.format(extern_type))
+        return valid_plugins
+
     def load_default_context(self):
+        req = self.request
+
         c = self._get_local_tmpl_context()
         c.allow_scoped_tokens = self.ALLOW_SCOPED_TOKENS
         c.allowed_languages = [
@@ -263,7 +290,10 b' class UsersView(UserAppView):'
             ('ru', 'Russian (ru)'),
             ('zh', 'Chinese (zh)'),
         ]
-        req = self.request
+
+        c.allowed_extern_types = [
+            (x.uid, x.get_display_name()) for x in self.get_auth_plugins()
+        ]
 
         c.available_permissions = req.registry.settings['available_permissions']
         PermissionModel().set_global_permission_choices(
@@ -297,7 +327,7 b' class UsersView(UserAppView):'
         old_values = c.user.get_api_data()
         try:
             form_result = _form.to_python(dict(self.request.POST))
-            skip_attrs = ['extern_type', 'extern_name']
+            skip_attrs = ['extern_name']
             # TODO: plugin should define if username can be updated
             if c.extern_type != "rhodecode":
                 # forbid updating username for external accounts
@@ -347,59 +377,69 b' class UsersView(UserAppView):'
         _repos = c.user.repositories
         _repo_groups = c.user.repository_groups
         _user_groups = c.user.user_groups
+        _artifacts = c.user.artifacts
 
         handle_repos = None
         handle_repo_groups = None
         handle_user_groups = None
-        # dummy call for flash of handle
-        set_handle_flash_repos = lambda: None
-        set_handle_flash_repo_groups = lambda: None
-        set_handle_flash_user_groups = lambda: None
+        handle_artifacts = None
+
+        # calls for flash of handle based on handle case detach or delete
+        def set_handle_flash_repos():
+            handle = handle_repos
+            if handle == 'detach':
+                h.flash(_('Detached %s repositories') % len(_repos),
+                        category='success')
+            elif handle == 'delete':
+                h.flash(_('Deleted %s repositories') % len(_repos),
+                        category='success')
+
+        def set_handle_flash_repo_groups():
+            handle = handle_repo_groups
+            if handle == 'detach':
+                h.flash(_('Detached %s repository groups') % len(_repo_groups),
+                        category='success')
+            elif handle == 'delete':
+                h.flash(_('Deleted %s repository groups') % len(_repo_groups),
+                        category='success')
+
+        def set_handle_flash_user_groups():
+            handle = handle_user_groups
+            if handle == 'detach':
+                h.flash(_('Detached %s user groups') % len(_user_groups),
+                        category='success')
+            elif handle == 'delete':
+                h.flash(_('Deleted %s user groups') % len(_user_groups),
+                        category='success')
+
+        def set_handle_flash_artifacts():
+            handle = handle_artifacts
+            if handle == 'detach':
+                h.flash(_('Detached %s artifacts') % len(_artifacts),
+                        category='success')
+            elif handle == 'delete':
+                h.flash(_('Deleted %s artifacts') % len(_artifacts),
+                        category='success')
 
         if _repos and self.request.POST.get('user_repos'):
-            do = self.request.POST['user_repos']
-            if do == 'detach':
-                handle_repos = 'detach'
-                set_handle_flash_repos = lambda: h.flash(
-                    _('Detached %s repositories') % len(_repos),
-                    category='success')
-            elif do == 'delete':
-                handle_repos = 'delete'
-                set_handle_flash_repos = lambda: h.flash(
-                    _('Deleted %s repositories') % len(_repos),
-                    category='success')
+            handle_repos = self.request.POST['user_repos']
 
         if _repo_groups and self.request.POST.get('user_repo_groups'):
-            do = self.request.POST['user_repo_groups']
-            if do == 'detach':
-                handle_repo_groups = 'detach'
-                set_handle_flash_repo_groups = lambda: h.flash(
-                    _('Detached %s repository groups') % len(_repo_groups),
-                    category='success')
-            elif do == 'delete':
-                handle_repo_groups = 'delete'
-                set_handle_flash_repo_groups = lambda: h.flash(
-                    _('Deleted %s repository groups') % len(_repo_groups),
-                    category='success')
+            handle_repo_groups = self.request.POST['user_repo_groups']
 
         if _user_groups and self.request.POST.get('user_user_groups'):
-            do = self.request.POST['user_user_groups']
-            if do == 'detach':
-                handle_user_groups = 'detach'
-                set_handle_flash_user_groups = lambda: h.flash(
-                    _('Detached %s user groups') % len(_user_groups),
-                    category='success')
-            elif do == 'delete':
-                handle_user_groups = 'delete'
-                set_handle_flash_user_groups = lambda: h.flash(
-                    _('Deleted %s user groups') % len(_user_groups),
-                    category='success')
+            handle_user_groups = self.request.POST['user_user_groups']
+
+        if _artifacts and self.request.POST.get('user_artifacts'):
+            handle_artifacts = self.request.POST['user_artifacts']
 
         old_values = c.user.get_api_data()
+
         try:
             UserModel().delete(c.user, handle_repos=handle_repos,
                                handle_repo_groups=handle_repo_groups,
-                               handle_user_groups=handle_user_groups)
+                               handle_user_groups=handle_user_groups,
+                               handle_artifacts=handle_artifacts)
 
             audit_logger.store_web(
                 'user.delete', action_data={'old_data': old_values},
@@ -409,7 +449,9 b' class UsersView(UserAppView):'
             set_handle_flash_repos()
             set_handle_flash_repo_groups()
             set_handle_flash_user_groups()
-            h.flash(_('Successfully deleted user'), category='success')
+            set_handle_flash_artifacts()
+            username = h.escape(old_values['username'])
+            h.flash(_('Successfully deleted user `{}`').format(username), category='success')
         except (UserOwnsReposException, UserOwnsRepoGroupsException,
                 UserOwnsUserGroupsException, DefaultUserException) as e:
             h.flash(e, category='warning')
@@ -1187,19 +1229,45 b' class UsersView(UserAppView):'
         filter_term = self.request.GET.get('filter')
         user_log = UserModel().get_user_log(c.user, filter_term)
 
-        def url_generator(**kw):
+        def url_generator(page_num):
+            query_params = {
+                'page': page_num
+            }
             if filter_term:
-                kw['filter'] = filter_term
-            return self.request.current_route_path(_query=kw)
+                query_params['filter'] = filter_term
+            return self.request.current_route_path(_query=query_params)
 
-        c.audit_logs = h.Page(
-            user_log, page=p, items_per_page=10, url=url_generator)
+        c.audit_logs = SqlPage(
+            user_log, page=p, items_per_page=10, url_maker=url_generator)
         c.filter_term = filter_term
         return self._get_template_context(c)
 
     @LoginRequired()
     @HasPermissionAllDecorator('hg.admin')
     @view_config(
+        route_name='edit_user_audit_logs_download', request_method='GET',
+        renderer='string')
+    def user_audit_logs_download(self):
+        _ = self.request.translate
+        c = self.load_default_context()
+        c.user = self.db_user
+
+        user_log = UserModel().get_user_log(c.user, filter_term=None)
+
+        audit_log_data = {}
+        for entry in user_log:
+            audit_log_data[entry.user_log_id] = entry.get_dict()
+
+        response = Response(json.dumps(audit_log_data, indent=4))
+        response.content_disposition = str(
+            'attachment; filename=%s' % 'user_{}_audit_logs.json'.format(c.user.user_id))
+        response.content_type = 'application/json'
+
+        return response
+
+    @LoginRequired()
+    @HasPermissionAllDecorator('hg.admin')
+    @view_config(
         route_name='edit_user_perms_summary', request_method='GET',
         renderer='rhodecode:templates/admin/users/user_edit.mako')
     def user_perms_summary(self):

rhodecode/apps/debug_style/__init__.py

@@ -43,6 +43,14 b' def includeme(config):'
         pattern=ADMIN_PREFIX + '/debug_style',
         debug_style=True)
     config.add_route(
+        name='debug_style_email',
+        pattern=ADMIN_PREFIX + '/debug_style/email/{email_id}',
+        debug_style=True)
+    config.add_route(
+        name='debug_style_email_plain_rendered',
+        pattern=ADMIN_PREFIX + '/debug_style/email-rendered/{email_id}',
+        debug_style=True)
+    config.add_route(
         name='debug_style_template',
         pattern=ADMIN_PREFIX + '/debug_style/t/{t_path}',
         debug_style=True)

rhodecode/apps/debug_style/views.py

@@ -20,10 +20,15 b''
 
 import os
 import logging
+import datetime
 
 from pyramid.view import view_config
 from pyramid.renderers import render_to_response
 from rhodecode.apps._base import BaseAppView
+from rhodecode.lib.celerylib import run_task, tasks
+from rhodecode.lib.utils2 import AttributeDict
+from rhodecode.model.db import User
+from rhodecode.model.notification import EmailNotificationModel
 
 log = logging.getLogger(__name__)
 
@@ -46,6 +51,317 b' class DebugStyleView(BaseAppView):'
             request=self.request)
 
     @view_config(
+        route_name='debug_style_email', request_method='GET',
+        renderer=None)
+    @view_config(
+        route_name='debug_style_email_plain_rendered', request_method='GET',
+        renderer=None)
+    def render_email(self):
+        c = self.load_default_context()
+        email_id = self.request.matchdict['email_id']
+        c.active = 'emails'
+
+        pr = AttributeDict(
+            pull_request_id=123,
+            title='digital_ocean: fix redis, elastic search start on boot, '
+                  'fix fd limits on supervisor, set postgres 11 version',
+            description='''
+Check if we should use full-topic or mini-topic.
+
+- full topic produces some problems with merge states etc
+- server-mini-topic needs probably tweeks.            
+            ''',
+            repo_name='foobar',
+            source_ref_parts=AttributeDict(type='branch', name='fix-ticket-2000'),
+            target_ref_parts=AttributeDict(type='branch', name='master'),
+        )
+        target_repo = AttributeDict(repo_name='repo_group/target_repo')
+        source_repo = AttributeDict(repo_name='repo_group/source_repo')
+        user = User.get_by_username(self.request.GET.get('user')) or self._rhodecode_db_user
+        # file/commit changes for PR update
+        commit_changes = AttributeDict({
+            'added': ['aaaaaaabbbbb', 'cccccccddddddd'],
+            'removed': ['eeeeeeeeeee'],
+        })
+        file_changes = AttributeDict({
+            'added': ['a/file1.md', 'file2.py'],
+            'modified': ['b/modified_file.rst'],
+            'removed': ['.idea'],
+        })
+        email_kwargs = {
+            'test': {},
+            'message': {
+                'body': 'message body !'
+            },
+            'email_test': {
+                'user': user,
+                'date': datetime.datetime.now(),
+            },
+            'password_reset': {
+                'password_reset_url': 'http://example.com/reset-rhodecode-password/token',
+
+                'user': user,
+                'date': datetime.datetime.now(),
+                'email': '[e-mail]',
+                'first_admin_email': User.get_first_super_admin().email
+            },
+            'password_reset_confirmation': {
+                'new_password': 'new-password-example',
+                'user': user,
+                'date': datetime.datetime.now(),
+                'email': '[e-mail]',
+                'first_admin_email': User.get_first_super_admin().email
+            },
+            'registration': {
+                'user': user,
+                'date': datetime.datetime.now(),
+            },
+
+            'pull_request_comment': {
+                'user': user,
+
+                'status_change': None,
+                'status_change_type': None,
+
+                'pull_request': pr,
+                'pull_request_commits': [],
+
+                'pull_request_target_repo': target_repo,
+                'pull_request_target_repo_url': 'http://target-repo/url',
+
+                'pull_request_source_repo': source_repo,
+                'pull_request_source_repo_url': 'http://source-repo/url',
+
+                'pull_request_url': 'http://localhost/pr1',
+                'pr_comment_url': 'http://comment-url',
+                'pr_comment_reply_url': 'http://comment-url#reply',
+
+                'comment_file': None,
+                'comment_line': None,
+                'comment_type': 'note',
+                'comment_body': 'This is my comment body. *I like !*',
+                'comment_id': 2048,
+                'renderer_type': 'markdown',
+                'mention': True,
+
+            },
+            'pull_request_comment+status': {
+                'user': user,
+
+                'status_change': 'approved',
+                'status_change_type': 'approved',
+
+                'pull_request': pr,
+                'pull_request_commits': [],
+
+                'pull_request_target_repo': target_repo,
+                'pull_request_target_repo_url': 'http://target-repo/url',
+
+                'pull_request_source_repo': source_repo,
+                'pull_request_source_repo_url': 'http://source-repo/url',
+
+                'pull_request_url': 'http://localhost/pr1',
+                'pr_comment_url': 'http://comment-url',
+                'pr_comment_reply_url': 'http://comment-url#reply',
+
+                'comment_type': 'todo',
+                'comment_file': None,
+                'comment_line': None,
+                'comment_body': '''
+I think something like this would be better
+
+```py
+
+def db():
+   global connection
+   return connection
+
+```
+                
+                ''',
+                'comment_id': 2048,
+                'renderer_type': 'markdown',
+                'mention': True,
+
+            },
+            'pull_request_comment+file': {
+                'user': user,
+
+                'status_change': None,
+                'status_change_type': None,
+
+                'pull_request': pr,
+                'pull_request_commits': [],
+
+                'pull_request_target_repo': target_repo,
+                'pull_request_target_repo_url': 'http://target-repo/url',
+
+                'pull_request_source_repo': source_repo,
+                'pull_request_source_repo_url': 'http://source-repo/url',
+
+                'pull_request_url': 'http://localhost/pr1',
+
+                'pr_comment_url': 'http://comment-url',
+                'pr_comment_reply_url': 'http://comment-url#reply',
+
+                'comment_file': 'rhodecode/model/db.py',
+                'comment_line': 'o1210',
+                'comment_type': 'todo',
+                'comment_body': '''
+I like this !
+
+But please check this code::
+    
+    def main():
+        print 'ok'
+
+This should work better !
+                ''',
+                'comment_id': 2048,
+                'renderer_type': 'rst',
+                'mention': True,
+
+            },
+
+            'pull_request_update': {
+                'updating_user': user,
+
+                'status_change': None,
+                'status_change_type': None,
+
+                'pull_request': pr,
+                'pull_request_commits': [],
+
+                'pull_request_target_repo': target_repo,
+                'pull_request_target_repo_url': 'http://target-repo/url',
+
+                'pull_request_source_repo': source_repo,
+                'pull_request_source_repo_url': 'http://source-repo/url',
+
+                'pull_request_url': 'http://localhost/pr1',
+
+                # update comment links
+                'pr_comment_url': 'http://comment-url',
+                'pr_comment_reply_url': 'http://comment-url#reply',
+                'ancestor_commit_id': 'f39bd443',
+                'added_commits': commit_changes.added,
+                'removed_commits': commit_changes.removed,
+                'changed_files': (file_changes.added + file_changes.modified + file_changes.removed),
+                'added_files': file_changes.added,
+                'modified_files': file_changes.modified,
+                'removed_files': file_changes.removed,
+            },
+
+            'cs_comment': {
+                'user': user,
+                'commit': AttributeDict(idx=123, raw_id='a'*40, message='Commit message'),
+                'status_change': None,
+                'status_change_type': None,
+
+                'commit_target_repo_url': 'http://foo.example.com/#comment1',
+                'repo_name': 'test-repo',
+                'comment_type': 'note',
+                'comment_file': None,
+                'comment_line': None,
+                'commit_comment_url': 'http://comment-url',
+                'commit_comment_reply_url': 'http://comment-url#reply',
+                'comment_body': 'This is my comment body. *I like !*',
+                'comment_id': 2048,
+                'renderer_type': 'markdown',
+                'mention': True,
+            },
+            'cs_comment+status': {
+                'user': user,
+                'commit': AttributeDict(idx=123, raw_id='a' * 40, message='Commit message'),
+                'status_change': 'approved',
+                'status_change_type': 'approved',
+
+                'commit_target_repo_url': 'http://foo.example.com/#comment1',
+                'repo_name': 'test-repo',
+                'comment_type': 'note',
+                'comment_file': None,
+                'comment_line': None,
+                'commit_comment_url': 'http://comment-url',
+                'commit_comment_reply_url': 'http://comment-url#reply',
+                'comment_body': '''
+Hello **world**
+
+This is a multiline comment :)
+
+- list
+- list2
+                ''',
+                'comment_id': 2048,
+                'renderer_type': 'markdown',
+                'mention': True,
+            },
+            'cs_comment+file': {
+                'user': user,
+                'commit': AttributeDict(idx=123, raw_id='a' * 40, message='Commit message'),
+                'status_change': None,
+                'status_change_type': None,
+
+                'commit_target_repo_url': 'http://foo.example.com/#comment1',
+                'repo_name': 'test-repo',
+
+                'comment_type': 'note',
+                'comment_file': 'test-file.py',
+                'comment_line': 'n100',
+
+                'commit_comment_url': 'http://comment-url',
+                'commit_comment_reply_url': 'http://comment-url#reply',
+                'comment_body': 'This is my comment body. *I like !*',
+                'comment_id': 2048,
+                'renderer_type': 'markdown',
+                'mention': True,
+            },
+            
+            'pull_request': {
+                'user': user,
+                'pull_request': pr,
+                'pull_request_commits': [
+                    ('472d1df03bf7206e278fcedc6ac92b46b01c4e21', '''\
+my-account: moved email closer to profile as it's similar data just moved outside.                    
+                    '''),
+                    ('cbfa3061b6de2696c7161ed15ba5c6a0045f90a7', '''\
+users: description edit fixes
+
+- tests
+- added metatags info                    
+                    '''),
+                ],
+
+                'pull_request_target_repo': target_repo,
+                'pull_request_target_repo_url': 'http://target-repo/url',
+
+                'pull_request_source_repo': source_repo,
+                'pull_request_source_repo_url': 'http://source-repo/url',
+
+                'pull_request_url': 'http://code.rhodecode.com/_pull-request/123',
+            }
+
+        }
+
+        template_type = email_id.split('+')[0]
+        (c.subject, c.headers, c.email_body,
+         c.email_body_plaintext) = EmailNotificationModel().render_email(
+            template_type, **email_kwargs.get(email_id, {}))
+
+        test_email = self.request.GET.get('email')
+        if test_email:
+            recipients = [test_email]
+            run_task(tasks.send_email, recipients, c.subject,
+                     c.email_body_plaintext, c.email_body)
+
+        if self.request.matched_route.name == 'debug_style_email_plain_rendered':
+            template = 'debug_style/email_plain_rendered.mako'
+        else:
+            template = 'debug_style/email.mako'
+        return render_to_response(
+            template, self._get_template_context(c),
+            request=self.request)
+
+    @view_config(
         route_name='debug_style_template', request_method='GET',
         renderer=None)
     def template(self):
@@ -53,7 +369,18 b' class DebugStyleView(BaseAppView):'
         c = self.load_default_context()
         c.active = os.path.splitext(t_path)[0]
         c.came_from = ''
+        c.email_types = {
+            'cs_comment+file': {},
+            'cs_comment+status': {},
+
+            'pull_request_comment+file': {},
+            'pull_request_comment+status': {},
+
+            'pull_request_update': {},
+        }
+        c.email_types.update(EmailNotificationModel.email_types)
 
         return render_to_response(
             'debug_style/' + t_path, self._get_template_context(c),
-            request=self.request)
  No newline at end of file
+            request=self.request)
+

rhodecode/apps/file_store/__init__.py

@@ -44,6 +44,9 b' def includeme(config):'
     config.add_route(
         name='download_file',
         pattern='/_file_store/download/{fid}')
+    config.add_route(
+        name='download_file_by_token',
+        pattern='/_file_store/token-download/{_auth_token}/{fid}')
 
     # Scan module for configuration decorators.
     config.scan('.views', ignore='.tests')

rhodecode/apps/file_store/backends/local_store.py

@@ -26,7 +26,8 b' import hashlib'
 from rhodecode.lib.ext_json import json
 from rhodecode.apps.file_store import utils
 from rhodecode.apps.file_store.extensions import resolve_extensions
-from rhodecode.apps.file_store.exceptions import FileNotAllowedException
+from rhodecode.apps.file_store.exceptions import (
+    FileNotAllowedException, FileOverSizeException)
 
 METADATA_VER = 'v1'
 
@@ -91,6 +92,9 b' class LocalFileStorage(object):'
         self.base_path = base_path
         self.extensions = resolve_extensions([], groups=extension_groups)
 
+    def __repr__(self):
+        return '{}@{}'.format(self.__class__, self.base_path)
+
     def store_path(self, filename):
         """
         Returns absolute file path of the filename, joined to the
@@ -140,16 +144,21 b' class LocalFileStorage(object):'
         :param ext: extension to check
         :param extensions: iterable of extensions to validate against (or self.extensions)
         """
+        def normalize_ext(_ext):
+            if _ext.startswith('.'):
+                _ext = _ext[1:]
+            return _ext.lower()
 
         extensions = extensions or self.extensions
         if not extensions:
             return True
-        if ext.startswith('.'):
-            ext = ext[1:]
-        return ext.lower() in extensions
+
+        ext = normalize_ext(ext)
+
+        return ext in [normalize_ext(x) for x in extensions]
 
     def save_file(self, file_obj, filename, directory=None, extensions=None,
-                  extra_metadata=None, **kwargs):
+                  extra_metadata=None, max_filesize=None, **kwargs):
         """
         Saves a file object to the uploads location.
         Returns the resolved filename, i.e. the directory +
@@ -159,7 +168,9 b' class LocalFileStorage(object):'
         :param filename: original filename
         :param directory: relative path of sub-directory
         :param extensions: iterable of allowed extensions, if not default
+        :param max_filesize: maximum size of file that should be allowed
         :param extra_metadata: extra JSON metadata to store next to the file with .meta suffix
+
         """
 
         extensions = extensions or self.extensions
@@ -191,6 +202,12 b' class LocalFileStorage(object):'
             metadata = extra_metadata
 
         size = os.stat(path).st_size
+
+        if max_filesize and size > max_filesize:
+            # free up the copied file, and raise exc
+            os.remove(path)
+            raise FileOverSizeException()
+
         file_hash = self.calculate_path_hash(path)
 
         metadata.update(

rhodecode/apps/file_store/tests/test_upload_file.py

@@ -21,7 +21,8 b' import os'
 import pytest
 
 from rhodecode.lib.ext_json import json
-from rhodecode.model.db import Session, FileStore
+from rhodecode.model.auth_token import AuthTokenModel
+from rhodecode.model.db import Session, FileStore, Repository, User
 from rhodecode.tests import TestController
 from rhodecode.apps.file_store import utils, config_keys
 
@@ -32,6 +33,7 b' def route_path(name, params=None, **kwar'
     base_url = {
         'upload_file': '/_file_store/upload',
         'download_file': '/_file_store/download/{fid}',
+        'download_file_by_token': '/_file_store/token-download/{_auth_token}/{fid}'
 
     }[name].format(**kwargs)
 
@@ -124,3 +126,136 b' class TestFileStoreViews(TestController)'
             status=200)
 
         assert response.json['store_fid']
+
+    @pytest.fixture()
+    def create_artifact_factory(self, tmpdir):
+        def factory(user_id, content):
+            store_path = self.app._pyramid_settings[config_keys.store_path]
+            store = utils.get_file_storage({config_keys.store_path: store_path})
+            fid = 'example.txt'
+
+            filesystem_file = os.path.join(str(tmpdir), fid)
+            with open(filesystem_file, 'wb') as f:
+                f.write(content)
+
+            with open(filesystem_file, 'rb') as f:
+                store_uid, metadata = store.save_file(f, fid, extra_metadata={'filename': fid})
+
+            entry = FileStore.create(
+                file_uid=store_uid, filename=metadata["filename"],
+                file_hash=metadata["sha256"], file_size=metadata["size"],
+                file_display_name='file_display_name',
+                file_description='repo artifact `{}`'.format(metadata["filename"]),
+                check_acl=True, user_id=user_id,
+            )
+            Session().add(entry)
+            Session().commit()
+            return entry
+        return factory
+
+    def test_download_file_non_scoped(self, user_util, create_artifact_factory):
+        user = self.log_user()
+        user_id = user['user_id']
+        content = 'HELLO MY NAME IS ARTIFACT !'
+
+        artifact = create_artifact_factory(user_id, content)
+        file_uid = artifact.file_uid
+        response = self.app.get(route_path('download_file', fid=file_uid), status=200)
+        assert response.text == content
+
+        # log-in to new user and test download again
+        user = user_util.create_user(password='qweqwe')
+        self.log_user(user.username, 'qweqwe')
+        response = self.app.get(route_path('download_file', fid=file_uid), status=200)
+        assert response.text == content
+
+    def test_download_file_scoped_to_repo(self, user_util, create_artifact_factory):
+        user = self.log_user()
+        user_id = user['user_id']
+        content = 'HELLO MY NAME IS ARTIFACT !'
+
+        artifact = create_artifact_factory(user_id, content)
+        # bind to repo
+        repo = user_util.create_repo()
+        repo_id = repo.repo_id
+        artifact.scope_repo_id = repo_id
+        Session().add(artifact)
+        Session().commit()
+
+        file_uid = artifact.file_uid
+        response = self.app.get(route_path('download_file', fid=file_uid), status=200)
+        assert response.text == content
+
+        # log-in to new user and test download again
+        user = user_util.create_user(password='qweqwe')
+        self.log_user(user.username, 'qweqwe')
+        response = self.app.get(route_path('download_file', fid=file_uid), status=200)
+        assert response.text == content
+
+        # forbid user the rights to repo
+        repo = Repository.get(repo_id)
+        user_util.grant_user_permission_to_repo(repo, user, 'repository.none')
+        self.app.get(route_path('download_file', fid=file_uid), status=404)
+
+    def test_download_file_scoped_to_user(self, user_util, create_artifact_factory):
+        user = self.log_user()
+        user_id = user['user_id']
+        content = 'HELLO MY NAME IS ARTIFACT !'
+
+        artifact = create_artifact_factory(user_id, content)
+        # bind to user
+        user = user_util.create_user(password='qweqwe')
+
+        artifact.scope_user_id = user.user_id
+        Session().add(artifact)
+        Session().commit()
+
+        # artifact creator doesn't have access since it's bind to another user
+        file_uid = artifact.file_uid
+        self.app.get(route_path('download_file', fid=file_uid), status=404)
+
+        # log-in to new user and test download again, should be ok since we're bind to this artifact
+        self.log_user(user.username, 'qweqwe')
+        response = self.app.get(route_path('download_file', fid=file_uid), status=200)
+        assert response.text == content
+
+    def test_download_file_scoped_to_repo_with_bad_token(self, user_util, create_artifact_factory):
+        user_id = User.get_first_super_admin().user_id
+        content = 'HELLO MY NAME IS ARTIFACT !'
+
+        artifact = create_artifact_factory(user_id, content)
+        # bind to repo
+        repo = user_util.create_repo()
+        repo_id = repo.repo_id
+        artifact.scope_repo_id = repo_id
+        Session().add(artifact)
+        Session().commit()
+
+        file_uid = artifact.file_uid
+        self.app.get(route_path('download_file_by_token',
+                                _auth_token='bogus', fid=file_uid), status=302)
+
+    def test_download_file_scoped_to_repo_with_token(self, user_util, create_artifact_factory):
+        user = User.get_first_super_admin()
+        AuthTokenModel().create(user, 'test artifact token',
+                                role=AuthTokenModel.cls.ROLE_ARTIFACT_DOWNLOAD)
+
+        user = User.get_first_super_admin()
+        artifact_token = user.artifact_token
+
+        user_id = User.get_first_super_admin().user_id
+        content = 'HELLO MY NAME IS ARTIFACT !'
+
+        artifact = create_artifact_factory(user_id, content)
+        # bind to repo
+        repo = user_util.create_repo()
+        repo_id = repo.repo_id
+        artifact.scope_repo_id = repo_id
+        Session().add(artifact)
+        Session().commit()
+
+        file_uid = artifact.file_uid
+        response = self.app.get(
+            route_path('download_file_by_token',
+                       _auth_token=artifact_token, fid=file_uid), status=200)
+        assert response.text == content

rhodecode/apps/file_store/utils.py

@@ -25,7 +25,7 b' import pathlib2'
 
 
 def get_file_storage(settings):
-    from rhodecode.apps.file_store.local_store import LocalFileStorage
+    from rhodecode.apps.file_store.backends.local_store import LocalFileStorage
     from rhodecode.apps.file_store import config_keys
     store_path = settings.get(config_keys.store_path)
     return LocalFileStorage(base_path=store_path)

rhodecode/apps/file_store/views.py

@@ -30,8 +30,10 b' from rhodecode.apps.file_store.exception'
 
 from rhodecode.lib import helpers as h
 from rhodecode.lib import audit_logger
-from rhodecode.lib.auth import (CSRFRequired, NotAnonymous, HasRepoPermissionAny, HasRepoGroupPermissionAny)
-from rhodecode.model.db import Session, FileStore
+from rhodecode.lib.auth import (
+    CSRFRequired, NotAnonymous, HasRepoPermissionAny, HasRepoGroupPermissionAny,
+    LoginRequired)
+from rhodecode.model.db import Session, FileStore, UserApiKeys
 
 log = logging.getLogger(__name__)
 
@@ -44,6 +46,55 b' class FileStoreView(BaseAppView):'
         self.storage = utils.get_file_storage(self.request.registry.settings)
         return c
 
+    def _serve_file(self, file_uid):
+
+        if not self.storage.exists(file_uid):
+            store_path = self.storage.store_path(file_uid)
+            log.debug('File with FID:%s not found in the store under `%s`',
+                      file_uid, store_path)
+            raise HTTPNotFound()
+
+        db_obj = FileStore().query().filter(FileStore.file_uid == file_uid).scalar()
+        if not db_obj:
+            raise HTTPNotFound()
+
+        # private upload for user
+        if db_obj.check_acl and db_obj.scope_user_id:
+            log.debug('Artifact: checking scope access for bound artifact user: `%s`',
+                      db_obj.scope_user_id)
+            user = db_obj.user
+            if self._rhodecode_db_user.user_id != user.user_id:
+                log.warning('Access to file store object forbidden')
+                raise HTTPNotFound()
+
+        # scoped to repository permissions
+        if db_obj.check_acl and db_obj.scope_repo_id:
+            log.debug('Artifact: checking scope access for bound artifact repo: `%s`',
+                      db_obj.scope_repo_id)
+            repo = db_obj.repo
+            perm_set = ['repository.read', 'repository.write', 'repository.admin']
+            has_perm = HasRepoPermissionAny(*perm_set)(repo.repo_name, 'FileStore check')
+            if not has_perm:
+                log.warning('Access to file store object `%s` forbidden', file_uid)
+                raise HTTPNotFound()
+
+        # scoped to repository group permissions
+        if db_obj.check_acl and db_obj.scope_repo_group_id:
+            log.debug('Artifact: checking scope access for bound artifact repo group: `%s`',
+                      db_obj.scope_repo_group_id)
+            repo_group = db_obj.repo_group
+            perm_set = ['group.read', 'group.write', 'group.admin']
+            has_perm = HasRepoGroupPermissionAny(*perm_set)(repo_group.group_name, 'FileStore check')
+            if not has_perm:
+                log.warning('Access to file store object `%s` forbidden', file_uid)
+                raise HTTPNotFound()
+
+        FileStore.bump_access_counter(file_uid)
+
+        file_path = self.storage.store_path(file_uid)
+        return FileResponse(file_path)
+
+    @LoginRequired()
     @NotAnonymous()
     @CSRFRequired()
     @view_config(route_name='upload_file', request_method='POST', renderer='json_ext')
@@ -84,7 +135,7 b' class FileStoreView(BaseAppView):'
             entry = FileStore.create(
                 file_uid=store_uid, filename=metadata["filename"],
                 file_hash=metadata["sha256"], file_size=metadata["size"],
-                file_description='upload attachment',
+                file_description=u'upload attachment',
                 check_acl=False, user_id=self._rhodecode_user.user_id
             )
             Session().add(entry)
@@ -99,46 +150,25 b' class FileStoreView(BaseAppView):'
         return {'store_fid': store_uid,
                 'access_path': h.route_path('download_file', fid=store_uid)}
 
+    # ACL is checked by scopes, if no scope the file is accessible to all
     @view_config(route_name='download_file')
     def download_file(self):
         self.load_default_context()
         file_uid = self.request.matchdict['fid']
         log.debug('Requesting FID:%s from store %s', file_uid, self.storage)
-
-        if not self.storage.exists(file_uid):
-            log.debug('File with FID:%s not found in the store', file_uid)
-            raise HTTPNotFound()
-
-        db_obj = FileStore().query().filter(FileStore.file_uid == file_uid).scalar()
-        if not db_obj:
-            raise HTTPNotFound()
-
-        # private upload for user
-        if db_obj.check_acl and db_obj.scope_user_id:
-            user = db_obj.user
-            if self._rhodecode_db_user.user_id != user.user_id:
-                log.warning('Access to file store object forbidden')
-                raise HTTPNotFound()
+        return self._serve_file(file_uid)
 
-        # scoped to repository permissions
-        if db_obj.check_acl and db_obj.scope_repo_id:
-            repo = db_obj.repo
-            perm_set = ['repository.read', 'repository.write', 'repository.admin']
-            has_perm = HasRepoPermissionAny(*perm_set)(repo.repo_name, 'FileStore check')
-            if not has_perm:
-                log.warning('Access to file store object forbidden')
-                raise HTTPNotFound()
+    # in addition to @LoginRequired ACL is checked by scopes
+    @LoginRequired(auth_token_access=[UserApiKeys.ROLE_ARTIFACT_DOWNLOAD])
+    @NotAnonymous()
+    @view_config(route_name='download_file_by_token')
+    def download_file_by_token(self):
+        """
+        Special view that allows to access the download file by special URL that
+        is stored inside the URL.
 
-        # scoped to repository group permissions
-        if db_obj.check_acl and db_obj.scope_repo_group_id:
-            repo_group = db_obj.repo_group
-            perm_set = ['group.read', 'group.write', 'group.admin']
-            has_perm = HasRepoGroupPermissionAny(*perm_set)(repo_group.group_name, 'FileStore check')
-            if not has_perm:
-                log.warning('Access to file store object forbidden')
-                raise HTTPNotFound()
-
-        FileStore.bump_access_counter(file_uid)
-
-        file_path = self.storage.store_path(file_uid)
-        return FileResponse(file_path)
+        http://example.com/_file_store/token-download/TOKEN/FILE_UID
+        """
+        self.load_default_context()
+        file_uid = self.request.matchdict['fid']
+        return self._serve_file(file_uid)

rhodecode/apps/gist/tests/test_admin_gists.py

@@ -83,7 +83,7 b' class GistUtility(object):'
         Session().commit()
 
 
[e-mail]
[e-mail]()
 def create_gist(request):
     gist_utility = GistUtility()
     request.addfinalizer(gist_utility.cleanup)
@@ -159,7 +159,7 b' class TestGistsController(TestController'
             params={'lifetime': -1,
                     'content': 'gist test',
                     'filename': 'foo',
-                    'public': 'public',
+                    'gist_type': 'public',
                     'gist_acl_level': Gist.ACL_LEVEL_PUBLIC,
                     'csrf_token': self.csrf_token},
             status=302)
@@ -174,7 +174,7 b' class TestGistsController(TestController'
             params={'lifetime': -1,
                     'content': 'gist test',
                     'filename': '/home/foo',
-                    'public': 'public',
+                    'gist_type': 'public',
                     'gist_acl_level': Gist.ACL_LEVEL_PUBLIC,
                     'csrf_token': self.csrf_token},
             status=200)
@@ -197,7 +197,7 b' class TestGistsController(TestController'
             params={'lifetime': -1,
                     'content': 'private gist test',
                     'filename': 'private-foo',
-                    'private': 'private',
+                    'gist_type': 'private',
                     'gist_acl_level': Gist.ACL_LEVEL_PUBLIC,
                     'csrf_token': self.csrf_token},
             status=302)
@@ -216,7 +216,7 b' class TestGistsController(TestController'
             params={'lifetime': -1,
                     'content': 'private gist test',
                     'filename': 'private-foo',
-                    'private': 'private',
+                    'gist_type': 'private',
                     'gist_acl_level': Gist.ACL_LEVEL_PRIVATE,
                     'csrf_token': self.csrf_token},
             status=302)
@@ -236,7 +236,7 b' class TestGistsController(TestController'
                     'content': 'gist test',
                     'filename': 'foo-desc',
                     'description': 'gist-desc',
-                    'public': 'public',
+                    'gist_type': 'public',
                     'gist_acl_level': Gist.ACL_LEVEL_PUBLIC,
                     'csrf_token': self.csrf_token},
              status=302)
@@ -252,7 +252,7 b' class TestGistsController(TestController'
             'content': 'gist test',
             'filename': 'foo-desc',
             'description': 'gist-desc',
-            'public': 'public',
+            'gist_type': 'public',
             'gist_acl_level': Gist.ACL_LEVEL_PUBLIC,
             'csrf_token': self.csrf_token
         }

rhodecode/apps/gist/views.py

@@ -72,7 +72,7 b' class GistView(BaseAppView):'
     @LoginRequired()
     @view_config(
         route_name='gists_show', request_method='GET',
-        renderer='rhodecode:templates/admin/gists/index.mako')
+        renderer='rhodecode:templates/admin/gists/gist_index.mako')
     def gist_show_all(self):
         c = self.load_default_context()
 
@@ -136,7 +136,7 b' class GistView(BaseAppView):'
     @NotAnonymous()
     @view_config(
         route_name='gists_new', request_method='GET',
-        renderer='rhodecode:templates/admin/gists/new.mako')
+        renderer='rhodecode:templates/admin/gists/gist_new.mako')
     def gist_new(self):
         c = self.load_default_context()
         return self._get_template_context(c)
@@ -146,21 +146,26 b' class GistView(BaseAppView):'
     @CSRFRequired()
     @view_config(
         route_name='gists_create', request_method='POST',
-        renderer='rhodecode:templates/admin/gists/new.mako')
+        renderer='rhodecode:templates/admin/gists/gist_new.mako')
     def gist_create(self):
         _ = self.request.translate
         c = self.load_default_context()
 
         data = dict(self.request.POST)
         data['filename'] = data.get('filename') or Gist.DEFAULT_FILENAME
+
         data['nodes'] = [{
             'filename': data['filename'],
             'content': data.get('content'),
             'mimetype': data.get('mimetype')  # None is autodetect
         }]
+        gist_type = {
+            'public': Gist.GIST_PUBLIC,
+            'private': Gist.GIST_PRIVATE
+        }.get(data.get('gist_type')) or Gist.GIST_PRIVATE
 
-        data['gist_type'] = (
-            Gist.GIST_PUBLIC if data.get('public') else Gist.GIST_PRIVATE)
+        data['gist_type'] = gist_type
+
         data['gist_acl_level'] = (
             data.get('gist_acl_level') or Gist.ACL_LEVEL_PRIVATE)
 
@@ -196,7 +201,7 b' class GistView(BaseAppView):'
                 errors['filename'] = errors['nodes.0.filename']
                 del errors['nodes.0.filename']
 
-            data = render('rhodecode:templates/admin/gists/new.mako',
+            data = render('rhodecode:templates/admin/gists/gist_new.mako',
                           self._get_template_context(c), self.request)
             html = formencode.htmlfill.render(
                 data,
@@ -260,10 +265,10 b' class GistView(BaseAppView):'
     @LoginRequired()
     @view_config(
         route_name='gist_show', request_method='GET',
-        renderer='rhodecode:templates/admin/gists/show.mako')
+        renderer='rhodecode:templates/admin/gists/gist_show.mako')
     @view_config(
         route_name='gist_show_rev', request_method='GET',
-        renderer='rhodecode:templates/admin/gists/show.mako')
+        renderer='rhodecode:templates/admin/gists/gist_show.mako')
     @view_config(
         route_name='gist_show_formatted', request_method='GET',
         renderer=None)
@@ -304,7 +309,7 b' class GistView(BaseAppView):'
     @NotAnonymous()
     @view_config(
         route_name='gist_edit', request_method='GET',
-        renderer='rhodecode:templates/admin/gists/edit.mako')
+        renderer='rhodecode:templates/admin/gists/gist_edit.mako')
     def gist_edit(self):
         _ = self.request.translate
         gist_id = self.request.matchdict['gist_id']
@@ -338,7 +343,7 b' class GistView(BaseAppView):'
     @CSRFRequired()
     @view_config(
         route_name='gist_update', request_method='POST',
-        renderer='rhodecode:templates/admin/gists/edit.mako')
+        renderer='rhodecode:templates/admin/gists/gist_edit.mako')
     def gist_update(self):
         _ = self.request.translate
         gist_id = self.request.matchdict['gist_id']

rhodecode/apps/home/__init__.py

@@ -44,6 +44,14 b' def includeme(config):'
         pattern='/')
 
     config.add_route(
+        name='main_page_repos_data',
+        pattern='/_home_repos')
+
+    config.add_route(
+        name='main_page_repo_groups_data',
+        pattern='/_home_repo_groups')
+
+    config.add_route(
         name='user_autocomplete_data',
         pattern='/_users')
 

rhodecode/apps/home/tests/test_home.py

@@ -22,7 +22,7 b''
 import pytest
 
 import rhodecode
-from rhodecode.model.db import Repository
+from rhodecode.model.db import Repository, RepoGroup, User
 from rhodecode.model.meta import Session
 from rhodecode.model.repo import RepoModel
 from rhodecode.model.repo_group import RepoGroupModel
@@ -37,6 +37,8 b' fixture = Fixture()'
 def route_path(name, **kwargs):
     return {
         'home': '/',
+        'main_page_repos_data': '/_home_repos',
+        'main_page_repo_groups_data': '/_home_repo_groups',
         'repo_group_home': '/{repo_group_name}'
     }[name].format(**kwargs)
 
@@ -47,11 +49,42 b' class TestHomeController(TestController)'
         self.log_user()
         response = self.app.get(route_path('home'))
         # if global permission is set
-        response.mustcontain('Add Repository')
+        response.mustcontain('New Repository')
+
+    def test_index_grid_repos(self, xhr_header):
+        self.log_user()
+        response = self.app.get(route_path('main_page_repos_data'), extra_environ=xhr_header)
+        # search for objects inside the JavaScript JSON
+        for obj in Repository.getAll():
+            response.mustcontain('<a href=\\"/{}\\">'.format(obj.repo_name))
+
+    def test_index_grid_repo_groups(self, xhr_header):
+        self.log_user()
+        response = self.app.get(route_path('main_page_repo_groups_data'),
+                                extra_environ=xhr_header,)
 
         # search for objects inside the JavaScript JSON
-        for repo in Repository.getAll():
-            response.mustcontain('"name_raw": "%s"' % repo.repo_name)
+        for obj in RepoGroup.getAll():
+            response.mustcontain('<a href=\\"/{}\\">'.format(obj.group_name))
+
+    def test_index_grid_repo_groups_without_access(self, xhr_header, user_util):
+        user = user_util.create_user(password='qweqwe')
+        group_ok = user_util.create_repo_group(owner=user)
+        group_id_ok = group_ok.group_id
+
+        group_forbidden = user_util.create_repo_group(owner=User.get_first_super_admin())
+        group_id_forbidden = group_forbidden.group_id
+
+        user_util.grant_user_permission_to_repo_group(group_forbidden, user, 'group.none')
+        self.log_user(user.username, 'qweqwe')
+
+        self.app.get(route_path('main_page_repo_groups_data'),
+                     extra_environ=xhr_header,
+                     params={'repo_group_id': group_id_ok}, status=200)
+
+        self.app.get(route_path('main_page_repo_groups_data'),
+                     extra_environ=xhr_header,
+                     params={'repo_group_id': group_id_forbidden}, status=404)
 
     def test_index_contains_statics_with_ver(self):
         from rhodecode.lib.base import calculate_version_hash
@@ -62,11 +95,11 b' class TestHomeController(TestController)'
         rhodecode_version_hash = calculate_version_hash(
             {'beaker.session.secret': 'test-rc-uytcxaz'})
         response.mustcontain('style.css?ver={0}'.format(rhodecode_version_hash))
-        response.mustcontain('scripts.js?ver={0}'.format(rhodecode_version_hash))
+        response.mustcontain('scripts.min.js?ver={0}'.format(rhodecode_version_hash))
 
-    def test_index_contains_backend_specific_details(self, backend):
+    def test_index_contains_backend_specific_details(self, backend, xhr_header):
         self.log_user()
-        response = self.app.get(route_path('home'))
+        response = self.app.get(route_path('main_page_repos_data'), extra_environ=xhr_header)
         tip = backend.repo.get_commit().raw_id
 
         # html in javascript variable:
@@ -81,39 +114,44 b' class TestHomeController(TestController)'
             response = self.app.get(route_path('home'), status=302)
             assert 'login' in response.location
 
-    def test_index_page_on_groups(self, autologin_user, repo_group):
-        response = self.app.get(route_path('repo_group_home', repo_group_name='gr1'))
-        response.mustcontain("gr1/repo_in_group")
+    def test_index_page_on_groups_with_wrong_group_id(self, autologin_user, xhr_header):
+        group_id = 918123
+        self.app.get(
+            route_path('main_page_repo_groups_data'),
+            params={'repo_group_id': group_id},
+            status=404, extra_environ=xhr_header)
 
-    def test_index_page_on_group_with_trailing_slash(
-            self, autologin_user, repo_group):
-        response = self.app.get(route_path('repo_group_home', repo_group_name='gr1') + '/')
-        response.mustcontain("gr1/repo_in_group")
+    def test_index_page_on_groups(self, autologin_user, user_util, xhr_header):
+        gr = user_util.create_repo_group()
+        repo = user_util.create_repo(parent=gr)
+        repo_name = repo.repo_name
+        group_id = gr.group_id
 
-    @pytest.fixture(scope='class')
-    def repo_group(self, request):
-        gr = fixture.create_repo_group('gr1')
-        fixture.create_repo(name='gr1/repo_in_group', repo_group=gr)
+        response = self.app.get(route_path(
+            'repo_group_home', repo_group_name=gr.group_name))
+        response.mustcontain('d.repo_group_id = {}'.format(group_id))
 
-        @request.addfinalizer
-        def cleanup():
-            RepoModel().delete('gr1/repo_in_group')
-            RepoGroupModel().delete(repo_group='gr1', force_delete=True)
-            Session().commit()
+        response = self.app.get(
+            route_path('main_page_repos_data'),
+            params={'repo_group_id': group_id},
+            extra_environ=xhr_header,)
+        response.mustcontain(repo_name)
 
-    def test_index_with_name_with_tags(self, user_util, autologin_user):
-        user = user_util.create_user()
-        username = user.username
-        user.name = '<img src="/image1" onload="alert(\'Hello, World!\');">'
-        user.lastname = '#"><img src=x onerror=prompt(document.cookie);>'
+    def test_index_page_on_group_with_trailing_slash(self, autologin_user, user_util, xhr_header):
+        gr = user_util.create_repo_group()
+        repo = user_util.create_repo(parent=gr)
+        repo_name = repo.repo_name
+        group_id = gr.group_id
 
-        Session().add(user)
-        Session().commit()
-        user_util.create_repo(owner=username)
+        response = self.app.get(route_path(
+            'repo_group_home', repo_group_name=gr.group_name+'/'))
+        response.mustcontain('d.repo_group_id = {}'.format(group_id))
 
-        response = self.app.get(route_path('home'))
-        response.mustcontain(h.html_escape(user.first_name))
-        response.mustcontain(h.html_escape(user.last_name))
+        response = self.app.get(
+            route_path('main_page_repos_data'),
+            params={'repo_group_id': group_id},
+            extra_environ=xhr_header, )
+        response.mustcontain(repo_name)
 
     @pytest.mark.parametrize("name, state", [
         ('Disabled', False),
@@ -137,5 +175,5 b' class TestHomeController(TestController)'
     def test_logout_form_contains_csrf(self, autologin_user, csrf_token):
         response = self.app.get(route_path('home'))
         assert_response = response.assert_response()
-        element = assert_response.get_element('.logout #csrf_token')
+        element = assert_response.get_element('.logout [name=csrf_token]')
         assert element.value == csrf_token

rhodecode/apps/home/views.py

@@ -22,29 +22,30 b' import re'
 import logging
 import collections
 
+from pyramid.httpexceptions import HTTPNotFound
 from pyramid.view import view_config
 
-from rhodecode.apps._base import BaseAppView
+from rhodecode.apps._base import BaseAppView, DataGridAppView
 from rhodecode.lib import helpers as h
 from rhodecode.lib.auth import (
-    LoginRequired, NotAnonymous, HasRepoGroupPermissionAnyDecorator, CSRFRequired)
+    LoginRequired, NotAnonymous, HasRepoGroupPermissionAnyDecorator, CSRFRequired,
+    HasRepoGroupPermissionAny, AuthUser)
 from rhodecode.lib.codeblocks import filenode_as_lines_tokens
 from rhodecode.lib.index import searcher_from_config
 from rhodecode.lib.utils2 import safe_unicode, str2bool, safe_int
-from rhodecode.lib.ext_json import json
 from rhodecode.lib.vcs.nodes import FileNode
 from rhodecode.model.db import (
-    func, true, or_, case, in_filter_generator, Repository, RepoGroup, User, UserGroup)
+    func, true, or_, case, in_filter_generator, Session,
+    Repository, RepoGroup, User, UserGroup)
 from rhodecode.model.repo import RepoModel
 from rhodecode.model.repo_group import RepoGroupModel
-from rhodecode.model.scm import RepoGroupList, RepoList
 from rhodecode.model.user import UserModel
 from rhodecode.model.user_group import UserGroupModel
 
 log = logging.getLogger(__name__)
 
 
-class HomeView(BaseAppView):
+class HomeView(BaseAppView, DataGridAppView):
 
     def load_default_context(self):
         c = self._get_local_tmpl_context()
@@ -112,7 +113,12 b' class HomeView(BaseAppView):'
             ['repository.read', 'repository.write', 'repository.admin'],
             cache=False, name_filter=name_contains) or [-1]
 
-        query = Repository.query()\
+        query = Session().query(
+                Repository.repo_name,
+                Repository.repo_id,
+                Repository.repo_type,
+                Repository.private,
+            )\
             .filter(Repository.archived.isnot(true()))\
             .filter(or_(
                 # generate multiple IN to fix limitation problems
@@ -158,7 +164,10 b' class HomeView(BaseAppView):'
             ['group.read', 'group.write', 'group.admin'],
             cache=False, name_filter=name_contains) or [-1]
 
-        query = RepoGroup.query()\
+        query = Session().query(
+                RepoGroup.group_id,
+                RepoGroup.group_name,
+            )\
             .filter(or_(
                 # generate multiple IN to fix limitation problems
                 *in_filter_generator(RepoGroup.group_id, allowed_ids)
@@ -449,6 +458,7 b' class HomeView(BaseAppView):'
                 'id': -10,
                 'value': query,
                 'value_display': label,
+                'value_icon': '<i class="icon-code"></i>',
                 'type': 'search',
                 'subtype': 'repo',
                 'url': h.route_path('search_repo',
@@ -466,6 +476,7 b' class HomeView(BaseAppView):'
                 'id': -20,
                 'value': query,
                 'value_display': label,
+                'value_icon': '<i class="icon-history"></i>',
                 'type': 'search',
                 'subtype': 'repo',
                 'url': h.route_path('search_repo',
@@ -491,6 +502,7 b' class HomeView(BaseAppView):'
                 'id': -30,
                 'value': query,
                 'value_display': label,
+                'value_icon': '<i class="icon-code"></i>',
                 'type': 'search',
                 'subtype': 'repo_group',
                 'url': h.route_path('search_repo_group',
@@ -508,6 +520,7 b' class HomeView(BaseAppView):'
                 'id': -40,
                 'value': query,
                 'value_display': label,
+                'value_icon': '<i class="icon-history"></i>',
                 'type': 'search',
                 'subtype': 'repo_group',
                 'url': h.route_path('search_repo_group',
@@ -529,6 +542,7 b' class HomeView(BaseAppView):'
                     'id': -1,
                     'value': query,
                     'value_display': u'File search for: `{}`'.format(query),
+                    'value_icon': '<i class="icon-code"></i>',
                     'type': 'search',
                     'subtype': 'global',
                     'url': h.route_path('search',
@@ -539,6 +553,7 b' class HomeView(BaseAppView):'
                     'id': -2,
                     'value': query,
                     'value_display': u'Commit search for: `{}`'.format(query),
+                    'value_icon': '<i class="icon-history"></i>',
                     'type': 'search',
                     'subtype': 'global',
                     'url': h.route_path('search',
@@ -667,23 +682,6 b' class HomeView(BaseAppView):'
 
         return {'suggestions': res}
 
-    def _get_groups_and_repos(self, repo_group_id=None):
-        # repo groups groups
-        repo_group_list = RepoGroup.get_all_repo_groups(group_id=repo_group_id)
-        _perms = ['group.read', 'group.write', 'group.admin']
-        repo_group_list_acl = RepoGroupList(repo_group_list, perm_set=_perms)
-        repo_group_data = RepoGroupModel().get_repo_groups_as_dict(
-            repo_group_list=repo_group_list_acl, admin=False)
-
-        # repositories
-        repo_list = Repository.get_all_repos(group_id=repo_group_id)
-        _perms = ['repository.read', 'repository.write', 'repository.admin']
-        repo_list_acl = RepoList(repo_list, perm_set=_perms)
-        repo_data = RepoModel().get_repos_as_dict(
-            repo_list=repo_list_acl, admin=False)
-
-        return repo_data, repo_group_data
-
     @LoginRequired()
     @view_config(
         route_name='home', request_method='GET',
@@ -691,17 +689,74 b' class HomeView(BaseAppView):'
     def main_page(self):
         c = self.load_default_context()
         c.repo_group = None
-
-        repo_data, repo_group_data = self._get_groups_and_repos()
-        # json used to render the grids
-        c.repos_data = json.dumps(repo_data)
-        c.repo_groups_data = json.dumps(repo_group_data)
-
         return self._get_template_context(c)
 
+    def _main_page_repo_groups_data(self, repo_group_id):
+        column_map = {
+            'name': 'group_name_hash',
+            'desc': 'group_description',
+            'last_change': 'updated_on',
+            'owner': 'user_username',
+        }
+        draw, start, limit = self._extract_chunk(self.request)
+        search_q, order_by, order_dir = self._extract_ordering(
+            self.request, column_map=column_map)
+        return RepoGroupModel().get_repo_groups_data_table(
+            draw, start, limit,
+            search_q, order_by, order_dir,
+            self._rhodecode_user, repo_group_id)
+
+    def _main_page_repos_data(self, repo_group_id):
+        column_map = {
+            'name': 'repo_name',
+            'desc': 'description',
+            'last_change': 'updated_on',
+            'owner': 'user_username',
+        }
+        draw, start, limit = self._extract_chunk(self.request)
+        search_q, order_by, order_dir = self._extract_ordering(
+            self.request, column_map=column_map)
+        return RepoModel().get_repos_data_table(
+            draw, start, limit,
+            search_q, order_by, order_dir,
+            self._rhodecode_user, repo_group_id)
+
     @LoginRequired()
-    @HasRepoGroupPermissionAnyDecorator(
-        'group.read', 'group.write', 'group.admin')
+    @view_config(
+        route_name='main_page_repo_groups_data',
+        request_method='GET', renderer='json_ext', xhr=True)
+    def main_page_repo_groups_data(self):
+        self.load_default_context()
+        repo_group_id = safe_int(self.request.GET.get('repo_group_id'))
+
+        if repo_group_id:
+            group = RepoGroup.get_or_404(repo_group_id)
+            _perms = AuthUser.repo_group_read_perms
+            if not HasRepoGroupPermissionAny(*_perms)(
+                    group.group_name, 'user is allowed to list repo group children'):
+                raise HTTPNotFound()
+
+        return self._main_page_repo_groups_data(repo_group_id)
+
+    @LoginRequired()
+    @view_config(
+        route_name='main_page_repos_data',
+        request_method='GET', renderer='json_ext', xhr=True)
+    def main_page_repos_data(self):
+        self.load_default_context()
+        repo_group_id = safe_int(self.request.GET.get('repo_group_id'))
+
+        if repo_group_id:
+            group = RepoGroup.get_or_404(repo_group_id)
+            _perms = AuthUser.repo_group_read_perms
+            if not HasRepoGroupPermissionAny(*_perms)(
+                    group.group_name, 'user is allowed to list repo group children'):
+                raise HTTPNotFound()
+
+        return self._main_page_repos_data(repo_group_id)
+
+    @LoginRequired()
+    @HasRepoGroupPermissionAnyDecorator(*AuthUser.repo_group_read_perms)
     @view_config(
         route_name='repo_group_home', request_method='GET',
         renderer='rhodecode:templates/index_repo_group.mako')
@@ -711,16 +766,6 b' class HomeView(BaseAppView):'
     def repo_group_main_page(self):
         c = self.load_default_context()
         c.repo_group = self.request.db_repo_group
-        repo_data, repo_group_data = self._get_groups_and_repos(c.repo_group.group_id)
-
-        # update every 5 min
-        if self.request.db_repo_group.last_commit_cache_update_diff > 60 * 5:
-            self.request.db_repo_group.update_commit_cache()
-
-        # json used to render the grids
-        c.repos_data = json.dumps(repo_data)
-        c.repo_groups_data = json.dumps(repo_group_data)
-
         return self._get_template_context(c)
 
     @LoginRequired()

rhodecode/apps/journal/views.py

@@ -22,7 +22,7 b''
 import logging
 import itertools
 
-from webhelpers.feedgenerator import Atom1Feed, Rss201rev2Feed
+
 
 from pyramid.view import view_config
 from pyramid.httpexceptions import HTTPBadRequest
@@ -34,10 +34,11 b' from rhodecode.model.db import ('
     or_, joinedload, Repository, UserLog, UserFollowing, User, UserApiKeys)
 from rhodecode.model.meta import Session
 import rhodecode.lib.helpers as h
-from rhodecode.lib.helpers import Page
+from rhodecode.lib.helpers import SqlPage
 from rhodecode.lib.user_log_filter import user_log_filter
 from rhodecode.lib.auth import LoginRequired, NotAnonymous, CSRFRequired, HasRepoPermissionAny
 from rhodecode.lib.utils2 import safe_int, AttributeDict, md5_safe
+from rhodecode.lib.feedgenerator.feedgenerator import Atom1Feed, Rss201rev2Feed
 from rhodecode.model.scm import ScmModel
 
 log = logging.getLogger(__name__)
@@ -166,7 +167,7 b' class JournalView(BaseAppView):'
                 description=desc)
 
         response = Response(feed.writeString('utf-8'))
-        response.content_type = feed.mime_type
+        response.content_type = feed.content_type
         return response
 
     def _rss_feed(self, repos, search_term, public=True):
@@ -212,7 +213,7 b' class JournalView(BaseAppView):'
                 description=desc)
 
         response = Response(feed.writeString('utf-8'))
-        response.content_type = feed.mime_type
+        response.content_type = feed.content_type
         return response
 
     @LoginRequired()
@@ -232,15 +233,15 b' class JournalView(BaseAppView):'
 
         journal = self._get_journal_data(following, c.search_term)
 
-        def url_generator(**kw):
+        def url_generator(page_num):
             query_params = {
+                'page': page_num,
                 'filter': c.search_term
             }
-            query_params.update(kw)
             return self.request.current_route_path(_query=query_params)
 
-        c.journal_pager = Page(
-            journal, page=p, items_per_page=20, url=url_generator)
+        c.journal_pager = SqlPage(
+            journal, page=p, items_per_page=20, url_maker=url_generator)
         c.journal_day_aggreagate = self._get_daily_aggregate(c.journal_pager)
 
         c.journal_data = render(
@@ -333,13 +334,14 b' class JournalView(BaseAppView):'
 
         journal = self._get_journal_data(c.following, c.search_term)
 
-        def url_generator(**kw):
-            query_params = {}
-            query_params.update(kw)
+        def url_generator(page_num):
+            query_params = {
+                'page': page_num
+            }
             return self.request.current_route_path(_query=query_params)
 
-        c.journal_pager = Page(
-            journal, page=p, items_per_page=20, url=url_generator)
+        c.journal_pager = SqlPage(
+            journal, page=p, items_per_page=20, url_maker=url_generator)
         c.journal_day_aggreagate = self._get_daily_aggregate(c.journal_pager)
 
         c.journal_data = render(

rhodecode/apps/login/tests/test_login.py

@@ -93,7 +93,7 b' class TestLoginController(object):'
         session = response.get_session_from_response()
         username = session['rhodecode_user'].get('username')
         assert username == 'test_admin'
-        response.mustcontain('/%s' % HG_REPO)
+        response.mustcontain('logout')
 
     def test_login_regular_ok(self):
         response = self.app.post(route_path('login'),
@@ -104,8 +104,7 b' class TestLoginController(object):'
         session = response.get_session_from_response()
         username = session['rhodecode_user'].get('username')
         assert username == 'test_regular'
-
-        response.mustcontain('/%s' % HG_REPO)
+        response.mustcontain('logout')
 
     def test_login_regular_forbidden_when_super_admin_restriction(self):
         from rhodecode.authentication.plugins.auth_rhodecode import RhodeCodeAuthPlugin
@@ -225,7 +224,7 b' class TestLoginController(object):'
         session = response.get_session_from_response()
         username = session['rhodecode_user'].get('username')
         assert username == temp_user
-        response.mustcontain('/%s' % HG_REPO)
+        response.mustcontain('logout')
 
         # new password should be bcrypted, after log-in and transfer
         user = User.get_by_username(temp_user)
@@ -401,7 +400,7 b' class TestLoginController(object):'
         )  # This should be overridden
 
         assert_session_flash(
-            response, 'You have successfully registered with RhodeCode')
+            response, 'You have successfully registered with RhodeCode. You can log-in now.')
 
         ret = Session().query(User).filter(
             User.username == 'test_regular4').one()

rhodecode/apps/login/tests/test_password_reset.py

@@ -88,7 +88,7 b' class TestPasswordReset(TestController):'
 
         response = self.app.get(route_path('reset_password'))
         
-        assert_response = AssertResponse(response)
+        assert_response = response.assert_response()
         if show_reset:
             response.mustcontain('Send password reset email')
             assert_response.one_element_exists('#email')

rhodecode/apps/login/tests/test_register_captcha.py

@@ -90,7 +90,7 b' class TestRegisterCaptcha(object):'
 
         response = app.get(ADMIN_PREFIX + '/register')
 
-        assertr = AssertResponse(response)
+        assertr = response.assert_response()
         if active:
             assertr.one_element_exists('#recaptcha_field')
         else:
@@ -128,6 +128,6 b' class TestRegisterCaptcha(object):'
         else:
             # If captche input is invalid we expect to stay on the registration
             # page with an error message displayed.
-            assertr = AssertResponse(response)
+            assertr = response.assert_response()
             assert response.status_int == 200
             assertr.one_element_exists('#recaptcha_field ~ span.error-message')

rhodecode/apps/login/views.py

@@ -312,8 +312,6 b' class LoginView(BaseAppView):'
             action_data = {'data': new_user.get_api_data(),
                            'user_agent': self.request.user_agent}
 
-
-
             if external_identity:
                 action_data['external_identity'] = external_identity
 
@@ -329,7 +327,12 b' class LoginView(BaseAppView):'
             event = UserRegistered(user=new_user, session=self.session)
             trigger(event)
             h.flash(
-                _('You have successfully registered with RhodeCode'),
+                _('You have successfully registered with RhodeCode. You can log-in now.'),
+                category='success')
+            if external_identity:
+                h.flash(
+                    _('Please use the {identity} button to log-in').format(
+                        identity=external_identity),
                 category='success')
             Session().commit()
 

rhodecode/apps/my_account/views/my_account.py

@@ -33,22 +33,21 b' from rhodecode import forms'
 from rhodecode.lib import helpers as h
 from rhodecode.lib import audit_logger
 from rhodecode.lib.ext_json import json
-from rhodecode.lib.auth import LoginRequired, NotAnonymous, CSRFRequired, \
-    HasRepoPermissionAny, HasRepoGroupPermissionAny
+from rhodecode.lib.auth import (
+    LoginRequired, NotAnonymous, CSRFRequired,
+    HasRepoPermissionAny, HasRepoGroupPermissionAny, AuthUser)
 from rhodecode.lib.channelstream import (
     channelstream_request, ChannelstreamException)
 from rhodecode.lib.utils2 import safe_int, md5, str2bool
 from rhodecode.model.auth_token import AuthTokenModel
 from rhodecode.model.comment import CommentsModel
 from rhodecode.model.db import (
-    IntegrityError, joinedload,
+    IntegrityError, or_, in_filter_generator,
     Repository, UserEmailMap, UserApiKeys, UserFollowing,
     PullRequest, UserBookmark, RepoGroup)
 from rhodecode.model.meta import Session
 from rhodecode.model.pull_request import PullRequestModel
-from rhodecode.model.scm import RepoList
 from rhodecode.model.user import UserModel
-from rhodecode.model.repo import RepoModel
 from rhodecode.model.user_group import UserGroupModel
 from rhodecode.model.validation_schema.schemas import user_schema
 
@@ -345,22 +344,59 b' class MyAccountView(BaseAppView, DataGri'
                             'You should see a new live message now.'}
 
     def _load_my_repos_data(self, watched=False):
+
+        allowed_ids = [-1] + self._rhodecode_user.repo_acl_ids_from_stack(AuthUser.repo_read_perms)
+
         if watched:
-            admin = False
-            follows_repos = Session().query(UserFollowing)\
-                .filter(UserFollowing.user_id == self._rhodecode_user.user_id)\
-                .options(joinedload(UserFollowing.follows_repository))\
+            # repos user watch
+            repo_list = Session().query(
+                    Repository
+                ) \
+                .join(
+                    (UserFollowing, UserFollowing.follows_repo_id == Repository.repo_id)
+                ) \
+                .filter(
+                    UserFollowing.user_id == self._rhodecode_user.user_id
+                ) \
+                .filter(or_(
+                    # generate multiple IN to fix limitation problems
+                    *in_filter_generator(Repository.repo_id, allowed_ids))
+                ) \
+                .order_by(Repository.repo_name) \
                 .all()
-            repo_list = [x.follows_repository for x in follows_repos]
+
         else:
-            admin = True
-            repo_list = Repository.get_all_repos(
-                user_id=self._rhodecode_user.user_id)
-            repo_list = RepoList(repo_list, perm_set=[
-                'repository.read', 'repository.write', 'repository.admin'])
+            # repos user is owner of
+            repo_list = Session().query(
+                    Repository
+                ) \
+                .filter(
+                    Repository.user_id == self._rhodecode_user.user_id
+                ) \
+                .filter(or_(
+                    # generate multiple IN to fix limitation problems
+                    *in_filter_generator(Repository.repo_id, allowed_ids))
+                ) \
+                .order_by(Repository.repo_name) \
+                .all()
 
-        repos_data = RepoModel().get_repos_as_dict(
-            repo_list=repo_list, admin=admin, short_name=False)
+        _render = self.request.get_partial_renderer(
+            'rhodecode:templates/data_table/_dt_elements.mako')
+
+        def repo_lnk(name, rtype, rstate, private, archived, fork_of):
+            return _render('repo_name', name, rtype, rstate, private, archived, fork_of,
+                           short_name=False, admin=False)
+
+        repos_data = []
+        for repo in repo_list:
+            row = {
+                "name": repo_lnk(repo.repo_name, repo.repo_type, repo.repo_state,
+                                 repo.private, repo.archived, repo.fork),
+                "name_raw": repo.repo_name.lower(),
+            }
+
+            repos_data.append(row)
+
         # json used to render the grid
         return json.dumps(repos_data)
 
@@ -398,16 +434,19 b' class MyAccountView(BaseAppView, DataGri'
     def my_account_bookmarks(self):
         c = self.load_default_context()
         c.active = 'bookmarks'
+        c.bookmark_items = UserBookmark.get_bookmarks_for_user(
+            self._rhodecode_db_user.user_id, cache=False)
         return self._get_template_context(c)
 
-    def _process_entry(self, entry, user_id):
+    def _process_bookmark_entry(self, entry, user_id):
         position = safe_int(entry.get('position'))
+        cur_position = safe_int(entry.get('cur_position'))
         if position is None:
             return
 
         # check if this is an existing entry
         is_new = False
-        db_entry = UserBookmark().get_by_position_for_user(position, user_id)
+        db_entry = UserBookmark().get_by_position_for_user(cur_position, user_id)
 
         if db_entry and str2bool(entry.get('remove')):
             log.debug('Marked bookmark %s for deletion', db_entry)
@@ -446,12 +485,12 b' class MyAccountView(BaseAppView, DataGri'
             should_save = True
 
         if should_save:
-            log.debug('Saving bookmark %s, new:%s', db_entry, is_new)
             # mark user and position
             db_entry.user_id = user_id
             db_entry.position = position
             db_entry.title = entry.get('title')
             db_entry.redirect_url = entry.get('redirect_url') or default_redirect_url
+            log.debug('Saving bookmark %s, new:%s', db_entry, is_new)
 
             Session().add(db_entry)
 
@@ -468,15 +507,31 b' class MyAccountView(BaseAppView, DataGri'
         controls = peppercorn.parse(self.request.POST.items())
         user_id = c.user.user_id
 
-        try:
+        # validate positions
+        positions = {}
             for entry in controls.get('bookmarks', []):
-                self._process_entry(entry, user_id)
+            position = safe_int(entry['position'])
+            if position is None:
+                continue
+
+            if position in positions:
+                h.flash(_("Position {} is defined twice. "
+                          "Please correct this error.").format(position), category='error')
+                return HTTPFound(h.route_path('my_account_bookmarks'))
+
+            entry['position'] = position
+            entry['cur_position'] = safe_int(entry.get('cur_position'))
+            positions[position] = entry
+
+        try:
+            for entry in positions.values():
+                self._process_bookmark_entry(entry, user_id)
 
             Session().commit()
             h.flash(_("Update Bookmarks"), category='success')
         except IntegrityError:
             h.flash(_("Failed to update bookmarks. "
-                      "Make sure an unique position is used"), category='error')
+                      "Make sure an unique position is used."), category='error')
 
         return HTTPFound(h.route_path('my_account_bookmarks'))
 
@@ -582,6 +637,7 b' class MyAccountView(BaseAppView, DataGri'
             'email': c.user.email,
             'firstname': c.user.firstname,
             'lastname': c.user.lastname,
+            'description': c.user.description,
         }
         c.form = forms.RcForm(
             schema, appstruct=appstruct,
@@ -664,7 +720,8 b' class MyAccountView(BaseAppView, DataGri'
                 'target_repo': _render('pullrequest_target_repo',
                                        pr.target_repo.repo_name),
                 'name': _render('pullrequest_name',
-                                pr.pull_request_id, pr.target_repo.repo_name,
+                                pr.pull_request_id, pr.pull_request_state,
+                                pr.work_in_progress, pr.target_repo.repo_name,
                                 short=True),
                 'name_raw': pr.pull_request_id,
                 'status': _render('pullrequest_status',

rhodecode/apps/my_account/views/my_account_notifications.py

@@ -28,7 +28,7 b' from rhodecode.apps._base import BaseApp'
 from rhodecode.lib.auth import LoginRequired, NotAnonymous, CSRFRequired
 
 from rhodecode.lib import helpers as h
-from rhodecode.lib.helpers import Page
+from rhodecode.lib.helpers import SqlPage
 from rhodecode.lib.utils2 import safe_int
 from rhodecode.model.db import Notification
 from rhodecode.model.notification import NotificationModel
@@ -74,13 +74,16 b' class MyAccountNotificationsView(BaseApp'
 
         p = safe_int(self.request.GET.get('page', 1), 1)
 
-        def url_generator(**kw):
+        def url_generator(page_num):
+            query_params = {
+                'page': page_num
+            }
             _query = self.request.GET.mixed()
-            _query.update(kw)
-            return self.request.current_route_path(_query=_query)
+            query_params.update(_query)
+            return self.request.current_route_path(_query=query_params)
 
-        c.notifications = Page(notifications, page=p, items_per_page=10,
-                               url=url_generator)
+        c.notifications = SqlPage(notifications, page=p, items_per_page=10,
+                                  url_maker=url_generator)
 
         c.unread_type = 'unread'
         c.all_type = 'all'

rhodecode/apps/repo_group/views/repo_group_advanced.py

@@ -46,9 +46,16 b' class RepoGroupSettingsView(RepoGroupApp'
         route_name='edit_repo_group_advanced', request_method='GET',
         renderer='rhodecode:templates/admin/repo_groups/repo_group_edit.mako')
     def edit_repo_group_advanced(self):
+        _ = self.request.translate
         c = self.load_default_context()
         c.active = 'advanced'
         c.repo_group = self.db_repo_group
+
+        # update commit cache if GET flag is present
+        if self.request.GET.get('update_commit_cache'):
+            self.db_repo_group.update_commit_cache()
+            h.flash(_('updated commit cache'), category='success')
+
         return self._get_template_context(c)
 
     @LoginRequired()

rhodecode/apps/repository/__init__.py

@@ -79,6 +79,10 b' def includeme(config):'
         pattern='/{repo_name:.*?[^/]}/changeset/{commit_id}/comment/preview', repo_route=True)
 
     config.add_route(
+        name='repo_commit_comment_attachment_upload',
+        pattern='/{repo_name:.*?[^/]}/changeset/{commit_id}/comment/attachment_upload', repo_route=True)
+
+    config.add_route(
         name='repo_commit_comment_delete',
         pattern='/{repo_name:.*?[^/]}/changeset/{commit_id}/comment/{comment_id}/delete', repo_route=True)
 

rhodecode/apps/repository/tests/test_repo_commit_comments.py

@@ -98,7 +98,7 b' class TestRepoCommitCommentsView(TestCon'
         assert notification.type_ == Notification.TYPE_CHANGESET_COMMENT
 
         author = notification.created_by_user.username_and_name
-        sbj = '{0} left a {1} on commit `{2}` in the {3} repository'.format(
+        sbj = '@{0} left a {1} on commit `{2}` in the `{3}` repository'.format(
             author, comment_type, h.show_id(commit), backend.repo_name)
         assert sbj == notification.subject
 
@@ -159,7 +159,7 b' class TestRepoCommitCommentsView(TestCon'
         assert comment.revision == commit_id
 
         author = notification.created_by_user.username_and_name
-        sbj = '{0} left a {1} on file `{2}` in commit `{3}` in the {4} repository'.format(
+        sbj = '@{0} left a {1} on file `{2}` in commit `{3}` in the `{4}` repository'.format(
             author, comment_type, f_path, h.show_id(commit), backend.repo_name)
 
         assert sbj == notification.subject
@@ -230,7 +230,7 b' class TestRepoCommitCommentsView(TestCon'
         assert notification.type_ == Notification.TYPE_CHANGESET_COMMENT
 
         author = notification.created_by_user.username_and_name
-        sbj = '[status: Approved] {0} left a note on commit `{1}` in the {2} repository'.format(
+        sbj = '[status: Approved] @{0} left a note on commit `{1}` in the `{2}` repository'.format(
             author, h.show_id(commit), backend.repo_name)
         assert sbj == notification.subject
 
@@ -299,14 +299,14 b' class TestRepoCommitCommentsView(TestCon'
 
 def assert_comment_links(response, comments, inline_comments):
     if comments == 1:
-        comments_text = "%d Commit comment" % comments
+        comments_text = "%d General" % comments
     else:
-        comments_text = "%d Commit comments" % comments
+        comments_text = "%d General" % comments
 
     if inline_comments == 1:
-        inline_comments_text = "%d Inline Comment" % inline_comments
+        inline_comments_text = "%d Inline" % inline_comments
     else:
-        inline_comments_text = "%d Inline Comments" % inline_comments
+        inline_comments_text = "%d Inline" % inline_comments
 
     if comments:
         response.mustcontain('<a href="#comments">%s</a>,' % comments_text)
@@ -315,6 +315,6 b' def assert_comment_links(response, comme'
 
     if inline_comments:
         response.mustcontain(
-            'id="inline-comments-counter">%s</' % inline_comments_text)
+            'id="inline-comments-counter">%s' % inline_comments_text)
     else:
         response.mustcontain(inline_comments_text)

rhodecode/apps/repository/tests/test_repo_commits.py

@@ -20,6 +20,7 b''
 
 import pytest
 
+from rhodecode.apps.repository.tests.test_repo_compare import ComparePage
 from rhodecode.lib.helpers import _shorten_commit_id
 
 
@@ -79,13 +80,22 b' class TestRepoCommitView(object):'
             'git': '03fa803d7e9fb14daa9a3089e0d1494eda75d986',
             'svn': '337',
         }
+        diff_stat = {
+            'hg': (21, 943, 288),
+            'git': (20, 941, 286),
+            'svn': (21, 943, 288),
+        }
+
         commit_id = commit_id[backend.alias]
         response = self.app.get(route_path(
             'repo_commit',
             repo_name=backend.repo_name, commit_id=commit_id))
 
         response.mustcontain(_shorten_commit_id(commit_id))
-        response.mustcontain('21 files changed: 943 inserted, 288 deleted')
+
+        compare_page = ComparePage(response)
+        file_changes = diff_stat[backend.alias]
+        compare_page.contains_change_summary(*file_changes)
 
         # files op files
         response.mustcontain('File not present at commit: %s' %
@@ -121,20 +131,24 b' class TestRepoCommitView(object):'
         response.mustcontain(_shorten_commit_id(commit_ids[0]))
         response.mustcontain(_shorten_commit_id(commit_ids[1]))
 
+        compare_page = ComparePage(response)
+
         # svn is special
         if backend.alias == 'svn':
             response.mustcontain('new file 10644')
-            response.mustcontain('1 file changed: 5 inserted, 1 deleted')
-            response.mustcontain('12 files changed: 236 inserted, 22 deleted')
-            response.mustcontain('21 files changed: 943 inserted, 288 deleted')
+            for file_changes in [(1, 5, 1), (12, 236, 22), (21, 943, 288)]:
+                compare_page.contains_change_summary(*file_changes)
+        elif backend.alias == 'git':
+            response.mustcontain('new file 100644')
+            for file_changes in [(12, 222, 20), (20, 941, 286)]:
+                compare_page.contains_change_summary(*file_changes)
         else:
             response.mustcontain('new file 100644')
-            response.mustcontain('12 files changed: 222 inserted, 20 deleted')
-            response.mustcontain('21 files changed: 943 inserted, 288 deleted')
+            for file_changes in [(12, 222, 20), (21, 943, 288)]:
+                compare_page.contains_change_summary(*file_changes)
 
         # files op files
-        response.mustcontain('File not present at commit: %s' %
-                             _shorten_commit_id(commit_ids[1]))
+        response.mustcontain('File not present at commit: %s' % _shorten_commit_id(commit_ids[1]))
         response.mustcontain('Added docstrings to vcs.cli')  # commit msg
         response.mustcontain('Changed theme to ADC theme')  # commit msg
 
@@ -166,13 +180,21 b' class TestRepoCommitView(object):'
         response.mustcontain('File not present at commit: %s' %
                              _shorten_commit_id(commit_ids[1]))
 
+        compare_page = ComparePage(response)
+
         # svn is special
         if backend.alias == 'svn':
             response.mustcontain('new file 10644')
-            response.mustcontain('32 files changed: 1179 inserted, 310 deleted')
+            file_changes = (32, 1179, 310)
+            compare_page.contains_change_summary(*file_changes)
+        elif backend.alias == 'git':
+            response.mustcontain('new file 100644')
+            file_changes = (31, 1163, 306)
+            compare_page.contains_change_summary(*file_changes)
         else:
             response.mustcontain('new file 100644')
-            response.mustcontain('32 files changed: 1165 inserted, 308 deleted')
+            file_changes = (32, 1165, 308)
+            compare_page.contains_change_summary(*file_changes)
 
         response.mustcontain('Added docstrings to vcs.cli')  # commit msg
         response.mustcontain('Changed theme to ADC theme')  # commit msg
@@ -246,7 +268,7 b' new file mode 120000'
 """,
         'git': r"""diff --git a/README b/README
 new file mode 120000
-index 0000000000000000000000000000000000000000..92cacd285355271487b7e379dba6ca60f9a554a4
+index 0000000..92cacd2
 --- /dev/null
 +++ b/README
 @@ -0,0 +1 @@
@@ -300,6 +322,6 b' Added a symlink'
 
         # right pane diff menus
         if right_menu:
-            for elem in ['Hide whitespace changes', 'Toggle Wide Mode diff',
+            for elem in ['Hide whitespace changes', 'Toggle wide diff',
                          'Show full context diff']:
                 response.mustcontain(elem)

rhodecode/apps/repository/tests/test_repo_compare.py

@@ -623,8 +623,8 b' class ComparePage(AssertResponse):'
 
     def contains_change_summary(self, files_changed, inserted, deleted):
         template = (
-            "{files_changed} file{plural} changed: "
-            "{inserted} inserted, {deleted} deleted")
+            '{files_changed} file{plural} changed: '
+            '<span class="op-added">{inserted} inserted</span>, <span class="op-deleted">{deleted} deleted</span>')
         self.response.mustcontain(template.format(
             files_changed=files_changed,
             plural="s" if files_changed > 1 else "",

rhodecode/apps/repository/tests/test_repo_compare_local.py

@@ -102,7 +102,7 b' class TestCompareView(object):'
             'git': {
                 'tag': 'v0.2.2',
                 'branch': 'master',
-                'response': (71, 2269, 3416)
+                'response': (70, 1855, 3002)
             },
         }
 

rhodecode/apps/repository/tests/test_repo_compare_on_single_file.py

@@ -20,6 +20,7 b''
 
 import pytest
 
+from rhodecode.apps.repository.tests.test_repo_compare import ComparePage
 from rhodecode.lib.vcs import nodes
 from rhodecode.lib.vcs.backends.base import EmptyCommit
 from rhodecode.tests.fixture import Fixture
@@ -49,18 +50,18 b' class TestSideBySideDiff(object):'
             'hg': {
                 'commits': ['25d7e49c18b159446cadfa506a5cf8ad1cb04067',
                             '603d6c72c46d953420c89d36372f08d9f305f5dd'],
-                'changes': '21 files changed: 943 inserted, 288 deleted'
+                'changes': (21, 943, 288),
             },
             'git': {
                 'commits': ['6fc9270775aaf5544c1deb014f4ddd60c952fcbb',
                             '03fa803d7e9fb14daa9a3089e0d1494eda75d986'],
-                'changes': '21 files changed: 943 inserted, 288 deleted'
+                'changes': (20, 941, 286),
             },
 
             'svn': {
                 'commits': ['336',
                             '337'],
-                'changes': '21 files changed: 943 inserted, 288 deleted'
+                'changes': (21, 943, 288),
             },
         }
 
@@ -79,26 +80,27 b' class TestSideBySideDiff(object):'
             params=dict(target_repo=backend.repo_name, diffmode='sidebyside')
         ))
 
-        response.mustcontain(file_changes)
-        response.mustcontain('Expand 1 commit')
+        compare_page = ComparePage(response)
+        compare_page.contains_change_summary(*file_changes)
+        response.mustcontain('Collapse 1 commit')
 
     def test_diff_sidebyside_two_commits(self, app, backend):
         commit_id_range = {
             'hg': {
                 'commits': ['4fdd71e9427417b2e904e0464c634fdee85ec5a7',
                             '603d6c72c46d953420c89d36372f08d9f305f5dd'],
-                'changes': '32 files changed: 1165 inserted, 308 deleted'
+                'changes': (32, 1165, 308),
             },
             'git': {
                 'commits': ['f5fbf9cfd5f1f1be146f6d3b38bcd791a7480c13',
                             '03fa803d7e9fb14daa9a3089e0d1494eda75d986'],
-                'changes': '32 files changed: 1165 inserted, 308 deleted'
+                'changes': (31, 1163, 306),
             },
 
             'svn': {
                 'commits': ['335',
                             '337'],
-                'changes': '32 files changed: 1179 inserted, 310 deleted'
+                'changes': (32, 1179, 310),
             },
         }
 
@@ -117,8 +119,36 b' class TestSideBySideDiff(object):'
             params=dict(target_repo=backend.repo_name, diffmode='sidebyside')
         ))
 
-        response.mustcontain(file_changes)
-        response.mustcontain('Expand 2 commits')
+        compare_page = ComparePage(response)
+        compare_page.contains_change_summary(*file_changes)
+
+        response.mustcontain('Collapse 2 commits')
+
+    def test_diff_sidebyside_collapsed_commits(self, app, backend_svn):
+        commit_id_range = {
+
+            'svn': {
+                'commits': ['330',
+                            '337'],
+
+            },
+        }
+
+        commit_info = commit_id_range['svn']
+        commit2, commit1 = commit_info['commits']
+
+        response = self.app.get(route_path(
+            'repo_compare',
+            repo_name=backend_svn.repo_name,
+            source_ref_type='rev',
+            source_ref=commit2,
+            target_repo=backend_svn.repo_name,
+            target_ref_type='rev',
+            target_ref=commit1,
+            params=dict(target_repo=backend_svn.repo_name, diffmode='sidebyside')
+        ))
+
+        response.mustcontain('Expand 7 commits')
 
     @pytest.mark.xfail(reason='GIT does not handle empty commit compare correct (missing 1 commit)')
     def test_diff_side_by_side_from_0_commit(self, app, backend, backend_stub):
@@ -145,14 +175,14 b' class TestSideBySideDiff(object):'
             params=dict(diffmode='sidebyside')
         ))
 
-        response.mustcontain('Expand 2 commits')
+        response.mustcontain('Collapse 2 commits')
         response.mustcontain('123 file changed')
 
         response.mustcontain(
             'r%s:%s...r%s:%s' % (
                 commit1.idx, commit1.short_id, commit2.idx, commit2.short_id))
 
-        response.mustcontain('<strong>{}</strong>'.format(f_path))
+        response.mustcontain(f_path)
 
     @pytest.mark.xfail(reason='GIT does not handle empty commit compare correct (missing 1 commit)')
     def test_diff_side_by_side_from_0_commit_with_file_filter(self, app, backend, backend_stub):
@@ -179,14 +209,14 b' class TestSideBySideDiff(object):'
             params=dict(f_path=f_path, target_repo=repo.repo_name, diffmode='sidebyside')
         ))
 
-        response.mustcontain('Expand 2 commits')
+        response.mustcontain('Collapse 2 commits')
         response.mustcontain('1 file changed')
 
         response.mustcontain(
             'r%s:%s...r%s:%s' % (
                 commit1.idx, commit1.short_id, commit2.idx, commit2.short_id))
 
-        response.mustcontain('<strong>{}</strong>'.format(f_path))
+        response.mustcontain(f_path)
 
     def test_diff_side_by_side_with_empty_file(self, app, backend, backend_stub):
         commits = [
@@ -211,32 +241,32 b' class TestSideBySideDiff(object):'
             params=dict(f_path=f_path, target_repo=repo.repo_name, diffmode='sidebyside')
         ))
 
-        response.mustcontain('Expand 2 commits')
+        response.mustcontain('Collapse 2 commits')
         response.mustcontain('1 file changed')
 
         response.mustcontain(
             'r%s:%s...r%s:%s' % (
                 commit2.idx, commit2.short_id, commit3.idx, commit3.short_id))
 
-        response.mustcontain('<strong>{}</strong>'.format(f_path))
+        response.mustcontain(f_path)
 
     def test_diff_sidebyside_two_commits_with_file_filter(self, app, backend):
         commit_id_range = {
             'hg': {
                 'commits': ['4fdd71e9427417b2e904e0464c634fdee85ec5a7',
                             '603d6c72c46d953420c89d36372f08d9f305f5dd'],
-                'changes': '1 file changed: 3 inserted, 3 deleted'
+                'changes': (1, 3, 3)
             },
             'git': {
                 'commits': ['f5fbf9cfd5f1f1be146f6d3b38bcd791a7480c13',
                             '03fa803d7e9fb14daa9a3089e0d1494eda75d986'],
-                'changes': '1 file changed: 3 inserted, 3 deleted'
+                'changes': (1, 3, 3)
             },
 
             'svn': {
                 'commits': ['335',
                             '337'],
-                'changes': '1 file changed: 3 inserted, 3 deleted'
+                'changes': (1, 3, 3)
             },
         }
         f_path = 'docs/conf.py'
@@ -255,5 +285,7 b' class TestSideBySideDiff(object):'
             params=dict(f_path=f_path, target_repo=backend.repo_name, diffmode='sidebyside')
         ))
 
-        response.mustcontain('Expand 2 commits')
-        response.mustcontain(file_changes)
+        response.mustcontain('Collapse 2 commits')
+
+        compare_page = ComparePage(response)
+        compare_page.contains_change_summary(*file_changes)

rhodecode/apps/repository/tests/test_repo_feed.py

@@ -41,7 +41,7 b' def route_path(name, params=None, **kwar'
 class TestFeedView(TestController):
 
     @pytest.mark.parametrize("feed_type,response_types,content_type",[
-        ('rss', ['<rss version="2.0">'],
+        ('rss', ['<rss version="2.0"'],
          "application/rss+xml"),
         ('atom', ['xmlns="http://www.w3.org/2005/Atom"', 'xml:lang="en-us"'],
          "application/atom+xml"),
@@ -116,7 +116,7 b' class TestFeedView(TestController):'
             self, backend, user_util, feed_type):
         user = user_util.create_user()
         auth_token = AuthTokenModel().create(
-            user.user_id, 'test-token', -1, AuthTokenModel.cls.ROLE_API)
+            user.user_id, u'test-token', -1, AuthTokenModel.cls.ROLE_API)
         auth_token = auth_token.api_key
 
         self.app.get(
@@ -127,7 +127,7 b' class TestFeedView(TestController):'
             status=302)
 
         auth_token = AuthTokenModel().create(
-            user.user_id, 'test-token', -1, AuthTokenModel.cls.ROLE_FEED)
+            user.user_id, u'test-token', -1, AuthTokenModel.cls.ROLE_FEED)
         auth_token = auth_token.api_key
         self.app.get(
             route_path(

rhodecode/apps/repository/tests/test_repo_files.py

@@ -23,6 +23,7 b' import os'
 import mock
 import pytest
 
+from rhodecode.apps.repository.tests.test_repo_compare import ComparePage
 from rhodecode.apps.repository.views.repo_files import RepoFilesView
 from rhodecode.lib import helpers as h
 from rhodecode.lib.compat import OrderedDict
@@ -616,8 +617,11 b' class TestFilesDiff(object):'
             })
         # use redirect since this is OLD view redirecting to compare page
         response = response.follow()
-        response.mustcontain('Expand 1 commit')
-        response.mustcontain('1 file changed: 0 inserted, 0 deleted')
+        response.mustcontain('Collapse 1 commit')
+        file_changes = (1, 0, 0)
+
+        compare_page = ComparePage(response)
+        compare_page.contains_change_summary(*file_changes)
 
         if backend.alias == 'svn':
             response.mustcontain('new file 10644')

rhodecode/apps/repository/tests/test_repo_forks.py

@@ -130,7 +130,6 b' class TestRepoForkViewTests(TestControll'
             'repo_type': backend.alias,
             'description': description,
             'private': 'False',
-            'landing_rev': 'rev:tip',
             'csrf_token': csrf_token,
         }
 
@@ -159,7 +158,6 b' class TestRepoForkViewTests(TestControll'
             'repo_type': backend.alias,
             'description': description,
             'private': 'False',
-            'landing_rev': 'rev:tip',
             'csrf_token': csrf_token,
         }
         self.app.post(
@@ -172,8 +170,8 b' class TestRepoForkViewTests(TestControll'
             route_path('repo_creating_check', repo_name=fork_name))
         # test if we have a message that fork is ok
         assert_session_flash(response,
-                'Forked repository %s as <a href="/%s">%s</a>'
-                % (repo_name, fork_name, fork_name))
+                'Forked repository %s as <a href="/%s">%s</a>' % (
+                                 repo_name, fork_name, fork_name))
 
         # test if the fork was created in the database
         fork_repo = Session().query(Repository)\
@@ -205,7 +203,6 b' class TestRepoForkViewTests(TestControll'
             'repo_type': backend.alias,
             'description': description,
             'private': 'False',
-            'landing_rev': 'rev:tip',
             'csrf_token': csrf_token,
         }
         self.app.post(
@@ -218,8 +215,8 b' class TestRepoForkViewTests(TestControll'
             route_path('repo_creating_check', repo_name=fork_name_full))
         # test if we have a message that fork is ok
         assert_session_flash(response,
-                'Forked repository %s as <a href="/%s">%s</a>'
-                % (repo_name, fork_name_full, fork_name_full))
+                             'Forked repository %s as <a href="/%s">%s</a>' % (
+                                 repo_name, fork_name_full, fork_name_full))
 
         # test if the fork was created in the database
         fork_repo = Session().query(Repository)\

rhodecode/apps/repository/tests/test_repo_issue_tracker.py

@@ -84,7 +84,7 b' class TestRepoIssueTracker(object):'
             extra_environ=xhr_header, params=data)
 
         assert response.body == \
-               'example of <a class="issue-tracker-link" href="http://url">prefix</a> replacement'
+               'example of <a class="tooltip issue-tracker-link" href="http://url" title="description">prefix</a> replacement'
 
         @request.addfinalizer
         def cleanup():
@@ -125,7 +125,7 b' class TestRepoIssueTracker(object):'
             self.settings_model.delete_entries(self.uid)
 
     def test_delete_issuetracker_pattern(
-            self, autologin_user, backend, csrf_token, settings_util):
+            self, autologin_user, backend, csrf_token, settings_util, xhr_header):
         repo = backend.create_repo()
         repo_name = repo.repo_name
         entry_key = 'issuetracker_pat_'
@@ -141,8 +141,9 b' class TestRepoIssueTracker(object):'
                 repo_name=backend.repo.repo_name),
             {
                 'uid': uid,
-                'csrf_token': csrf_token
-            }, status=302)
+                'csrf_token': csrf_token,
+                '': ''
+            }, extra_environ=xhr_header, status=200)
         settings = IssueTrackerSettingsModel(
             repo=Repository.get_by_repo_name(repo_name)).get_repo_settings()
         assert 'rhodecode_%s%s' % (entry_key, uid) not in settings

rhodecode/apps/repository/tests/test_repo_permissions.py

@@ -62,7 +62,7 b' class TestRepoPermissionsView(object):'
             route_path('edit_repo_perms',
                        repo_name=repo_name), form_data).follow()
 
-        assert 'Repository permissions updated' in response
+        assert 'Repository access permissions updated' in response
 
         # revoke given
         form_data = permission_update_data_generator(
@@ -74,4 +74,4 b' class TestRepoPermissionsView(object):'
             route_path('edit_repo_perms',
                        repo_name=repo_name), form_data).follow()
 
-        assert 'Repository permissions updated' in response
+        assert 'Repository access permissions updated' in response

rhodecode/apps/repository/tests/test_repo_pullrequests.py

@@ -101,12 +101,11 b' class TestPullrequestsView(object):'
         for commit_id in pull_request.revisions:
             response.mustcontain(commit_id)
 
-        assert pull_request.target_ref_parts.type in response
-        assert pull_request.target_ref_parts.name in response
-        target_clone_url = pull_request.target_repo.clone_url()
-        assert target_clone_url in response
+        response.mustcontain(pull_request.target_ref_parts.type)
+        response.mustcontain(pull_request.target_ref_parts.name)
 
-        assert 'class="pull-request-merge"' in response
+        response.mustcontain('class="pull-request-merge"')
+
         if pr_merge_enabled:
             response.mustcontain('Pull request reviewer approval is pending')
         else:
@@ -261,7 +260,7 b' class TestPullrequestsView(object):'
             True, True, '', MergeFailureReason.MISSING_TARGET_REF,
             metadata={'target_ref': PullRequest.unicode_to_reference(unicode_reference)})
         response.assert_response().element_contains(
-            'span[data-role="merge-message"]', merge_resp.merge_status_message)
+            'div[data-role="merge-message"]', merge_resp.merge_status_message)
 
     def test_comment_and_close_pull_request_custom_message_approved(
             self, pr_util, csrf_token, xhr_header):
@@ -284,7 +283,7 b' class TestPullrequestsView(object):'
         journal = UserLog.query()\
             .filter(UserLog.user_id == author)\
             .filter(UserLog.repository_id == repo) \
-            .order_by('user_log_id') \
+            .order_by(UserLog.user_log_id.asc()) \
             .all()
         assert journal[-1].action == 'repo.pull_request.close'
 
@@ -323,7 +322,7 b' class TestPullrequestsView(object):'
 
         journal = UserLog.query()\
             .filter(UserLog.user_id == author, UserLog.repository_id == repo) \
-            .order_by('user_log_id') \
+            .order_by(UserLog.user_log_id.asc()) \
             .all()
         assert journal[-1].action == 'repo.pull_request.close'
 
@@ -467,7 +466,7 b' class TestPullrequestsView(object):'
             .filter(Notification.created_by == pull_request.author.user_id,
                     Notification.type_ == Notification.TYPE_PULL_REQUEST,
                     Notification.subject.contains(
-                        "wants you to review pull request #%s" % pull_request_id))
+                        "requested a pull request review. !%s" % pull_request_id))
         assert len(notifications.all()) == 1
 
         # Change reviewers and check that a notification was made
@@ -536,9 +535,9 b' class TestPullrequestsView(object):'
 
         # Check generated diff contents
         response = response.follow()
-        assert 'content_of_ancestor' not in response.body
-        assert 'content_of_ancestor-child' not in response.body
-        assert 'content_of_change' in response.body
+        response.mustcontain(no=['content_of_ancestor'])
+        response.mustcontain(no=['content_of_ancestor-child'])
+        response.mustcontain('content_of_change')
 
     def test_merge_pull_request_enabled(self, pr_util, csrf_token):
         # Clear any previous calls to rcextensions
@@ -549,11 +548,10 b' class TestPullrequestsView(object):'
         pull_request_id = pull_request.pull_request_id
         repo_name = pull_request.target_repo.scm_instance().name,
 
-        response = self.app.post(
-            route_path('pullrequest_merge',
+        url = route_path('pullrequest_merge',
                 repo_name=str(repo_name[0]),
-                pull_request_id=pull_request_id),
-            params={'csrf_token': csrf_token}).follow()
+                         pull_request_id=pull_request_id)
+        response = self.app.post(url, params={'csrf_token': csrf_token}).follow()
 
         pull_request = PullRequest.get(pull_request_id)
 
@@ -563,7 +561,7 b' class TestPullrequestsView(object):'
             pull_request, ChangesetStatus.STATUS_APPROVED)
 
         # Check the relevant log entries were added
-        user_logs = UserLog.query().order_by('-user_log_id').limit(3)
+        user_logs = UserLog.query().order_by(UserLog.user_log_id.desc()).limit(3)
         actions = [log.action for log in user_logs]
         pr_commit_ids = PullRequestModel()._get_commit_ids(pull_request)
         expected_actions = [
@@ -573,7 +571,7 b' class TestPullrequestsView(object):'
         ]
         assert actions == expected_actions
 
-        user_logs = UserLog.query().order_by('-user_log_id').limit(4)
+        user_logs = UserLog.query().order_by(UserLog.user_log_id.desc()).limit(4)
         actions = [log for log in user_logs]
         assert actions[-1].action == 'user.push'
         assert actions[-1].action_data['commit_ids'] == pr_commit_ids
@@ -690,8 +688,8 b' class TestPullrequestsView(object):'
                        pull_request_id=pull_request.pull_request_id))
 
         assert response.status_int == 200
-        assert 'Pull request updated to' in response.body
-        assert 'with 1 added, 0 removed commits.' in response.body
+        response.mustcontain('Pull request updated to')
+        response.mustcontain('with 1 added, 0 removed commits.')
 
         # check that we have now both revisions
         pull_request = PullRequest.get(pull_request_id)
@@ -735,10 +733,10 b' class TestPullrequestsView(object):'
         backend.pull_heads(source, heads=['change-rebased'])
 
         # update PR
-        self.app.post(
-            route_path('pullrequest_update',
+        url = route_path('pullrequest_update',
                        repo_name=target.repo_name,
-                       pull_request_id=pull_request_id),
+                         pull_request_id=pull_request_id)
+        self.app.post(url,
             params={'update_commits': 'true', 'csrf_token': csrf_token},
             status=200)
 
@@ -753,8 +751,8 b' class TestPullrequestsView(object):'
                        repo_name=target.repo_name,
                        pull_request_id=pull_request.pull_request_id))
         assert response.status_int == 200
-        assert 'Pull request updated to' in response.body
-        assert 'with 1 added, 1 removed commits.' in response.body
+        response.mustcontain('Pull request updated to')
+        response.mustcontain('with 1 added, 1 removed commits.')
 
     def test_update_target_revision_with_removal_of_1_commit_git(self, backend_git, csrf_token):
         backend = backend_git
@@ -801,10 +799,10 b' class TestPullrequestsView(object):'
         vcsrepo.run_git_command(['reset', '--soft', 'HEAD~2'])
 
         # update PR
-        self.app.post(
-            route_path('pullrequest_update',
+        url = route_path('pullrequest_update',
                 repo_name=target.repo_name,
-                pull_request_id=pull_request_id),
+                         pull_request_id=pull_request_id)
+        self.app.post(url,
             params={'update_commits': 'true', 'csrf_token': csrf_token},
             status=200)
 
@@ -871,6 +869,7 b' class TestPullrequestsView(object):'
             {'message': 'new-feature', 'branch': branch_name},
         ]
         repo = backend_git.create_repo(commits)
+        repo_name = repo.repo_name
         commit_ids = backend_git.commit_ids
 
         pull_request = PullRequest()
@@ -888,13 +887,15 b' class TestPullrequestsView(object):'
         Session().add(pull_request)
         Session().commit()
 
+        pull_request_id = pull_request.pull_request_id
+
         vcs = repo.scm_instance()
         vcs.remove_ref('refs/heads/{}'.format(branch_name))
 
         response = self.app.get(route_path(
             'pullrequest_show',
-            repo_name=repo.repo_name,
-            pull_request_id=pull_request.pull_request_id))
+            repo_name=repo_name,
+            pull_request_id=pull_request_id))
 
         assert response.status_int == 200
 
@@ -958,15 +959,15 b' class TestPullrequestsView(object):'
         else:
             vcs.strip(pr_util.commit_ids['new-feature'])
 
-        response = self.app.post(
-            route_path('pullrequest_update',
+        url = route_path('pullrequest_update',
                 repo_name=pull_request.target_repo.repo_name,
-                pull_request_id=pull_request.pull_request_id),
+                         pull_request_id=pull_request.pull_request_id)
+        response = self.app.post(url,
             params={'update_commits': 'true',
                     'csrf_token': csrf_token})
 
         assert response.status_int == 200
-        assert response.body == 'true'
+        assert response.body == '{"response": true, "redirect_url": null}'
 
         # Make sure that after update, it won't raise 500 errors
         response = self.app.get(route_path(
@@ -992,12 +993,13 b' class TestPullrequestsView(object):'
             pull_request_id=pull_request.pull_request_id))
         assert response.status_int == 200
 
-        origin = response.assert_response().get_element('.pr-origininfo .tag')
-        origin_children = origin.getchildren()
-        assert len(origin_children) == 1
-        target = response.assert_response().get_element('.pr-targetinfo .tag')
-        target_children = target.getchildren()
-        assert len(target_children) == 1
+        source = response.assert_response().get_element('.pr-source-info')
+        source_parent = source.getparent()
+        assert len(source_parent) == 1
+
+        target = response.assert_response().get_element('.pr-target-info')
+        target_parent = target.getparent()
+        assert len(target_parent) == 1
 
         expected_origin_link = route_path(
             'repo_commits',
@@ -1007,10 +1009,8 b' class TestPullrequestsView(object):'
             'repo_commits',
             repo_name=pull_request.target_repo.scm_instance().name,
             params=dict(branch='target'))
-        assert origin_children[0].attrib['href'] == expected_origin_link
-        assert origin_children[0].text == 'branch: origin'
-        assert target_children[0].attrib['href'] == expected_target_link
-        assert target_children[0].text == 'branch: target'
+        assert source_parent.attrib['href'] == expected_origin_link
+        assert target_parent.attrib['href'] == expected_target_link
 
     def test_bookmark_is_not_a_link(self, pr_util):
         pull_request = pr_util.create_pull_request()
@@ -1025,13 +1025,13 b' class TestPullrequestsView(object):'
             pull_request_id=pull_request.pull_request_id))
         assert response.status_int == 200
 
-        origin = response.assert_response().get_element('.pr-origininfo .tag')
-        assert origin.text.strip() == 'bookmark: origin'
-        assert origin.getchildren() == []
+        source = response.assert_response().get_element('.pr-source-info')
+        assert source.text.strip() == 'bookmark:origin'
+        assert source.getparent().attrib.get('href') is None
 
-        target = response.assert_response().get_element('.pr-targetinfo .tag')
+        target = response.assert_response().get_element('.pr-target-info')
         assert target.text.strip() == 'bookmark: target'
-        assert target.getchildren() == []
+        assert target.getparent().attrib.get('href') is None
 
     def test_tag_is_not_a_link(self, pr_util):
         pull_request = pr_util.create_pull_request()
@@ -1046,13 +1046,13 b' class TestPullrequestsView(object):'
             pull_request_id=pull_request.pull_request_id))
         assert response.status_int == 200
 
-        origin = response.assert_response().get_element('.pr-origininfo .tag')
-        assert origin.text.strip() == 'tag: origin'
-        assert origin.getchildren() == []
+        source = response.assert_response().get_element('.pr-source-info')
+        assert source.text.strip() == 'tag:origin'
+        assert source.getparent().attrib.get('href') is None
 
-        target = response.assert_response().get_element('.pr-targetinfo .tag')
+        target = response.assert_response().get_element('.pr-target-info')
         assert target.text.strip() == 'tag: target'
-        assert target.getchildren() == []
+        assert target.getparent().attrib.get('href') is None
 
     @pytest.mark.parametrize('mergeable', [True, False])
     def test_shadow_repository_link(
@@ -1205,14 +1205,11 b' class TestPullrequestsControllerDelete(o'
 
 
 def assert_pull_request_status(pull_request, expected_status):
-    status = ChangesetStatusModel().calculated_review_status(
-        pull_request=pull_request)
+    status = ChangesetStatusModel().calculated_review_status(pull_request=pull_request)
     assert status == expected_status
 
 
 @pytest.mark.parametrize('route', ['pullrequest_new', 'pullrequest_create'])
 @pytest.mark.usefixtures("autologin_user")
 def test_forbidde_to_repo_summary_for_svn_repositories(backend_svn, app, route):
-    response = app.get(
-        route_path(route, repo_name=backend_svn.repo_name), status=404)
-
+    app.get(route_path(route, repo_name=backend_svn.repo_name), status=404)

rhodecode/apps/repository/tests/test_repo_summary.py

@@ -236,7 +236,7 b' class TestSummaryView(object):'
         with scm_patcher:
             response = self.app.get(
                 route_path('repo_summary', repo_name=repo_name))
-        assert_response = AssertResponse(response)
+        assert_response = response.assert_response()
         assert_response.element_contains(
             '.main .alert-warning strong', 'Missing requirements')
         assert_response.element_contains(
@@ -327,7 +327,7 b' def summary_view(context_stub, request_s'
 @pytest.mark.usefixtures('app')
 class TestCreateReferenceData(object):
 
-    @pytest.fixture
+    @pytest.fixture()
     def example_refs(self):
         section_1_refs = OrderedDict((('a', 'a_id'), ('b', 'b_id')))
         example_refs = [

rhodecode/apps/repository/tests/test_repo_vcs_settings.py

@@ -98,7 +98,7 b' class TestVcsSettings(object):'
         repo_name = backend.repo_name
         response = self.app.get(route_path('edit_repo_vcs', repo_name=repo_name))
 
-        assert_response = AssertResponse(response)
+        assert_response = response.assert_response()
         element = assert_response.get_element('#inherit_global_settings')
         assert element.checked
 
@@ -111,7 +111,7 b' class TestVcsSettings(object):'
             repo, 'inherit_vcs_settings', checked_value, 'bool')
         response = self.app.get(route_path('edit_repo_vcs', repo_name=repo_name))
 
-        assert_response = AssertResponse(response)
+        assert_response = response.assert_response()
         element = assert_response.get_element('#inherit_global_settings')
         assert element.checked == checked_value
 
@@ -491,7 +491,7 b' class TestVcsSettings(object):'
 
         response = self.app.get(
             route_path('edit_repo_vcs', repo_name=repo_name), status=200)
-        assert_response = AssertResponse(response)
+        assert_response = response.assert_response()
         for branch in branches:
             css_selector = '[name=branch_value_{}]'.format(branch.ui_id)
             element = assert_response.get_element(css_selector)
@@ -668,7 +668,7 b' class TestVcsSettings(object):'
         Session().commit()
 
     def assert_repo_value_equals_global_value(self, response, setting):
-        assert_response = AssertResponse(response)
+        assert_response = response.assert_response()
         global_css_selector = '[name={}_inherited]'.format(setting)
         repo_css_selector = '[name={}]'.format(setting)
         repo_element = assert_response.get_element(repo_css_selector)

rhodecode/apps/repository/tests/test_vcs_settings.py

@@ -59,7 +59,7 b' class TestAdminRepoVcsSettings(object):'
                 rhodecode.CONFIG, {'labs_settings_active': 'true'}):
             response = self.app.get(vcs_settings_url)
 
-        assertr = AssertResponse(response)
+        assertr = response.assert_response()
         assertr.one_element_exists('#rhodecode_{}'.format(setting_name))
 
     @pytest.mark.parametrize('setting_name, setting_backends', [

rhodecode/apps/repository/views/repo_audit_logs.py

@@ -22,6 +22,7 b' import logging'
 from pyramid.view import view_config
 
 from rhodecode.apps._base import RepoAppView
+from rhodecode.lib.helpers import SqlPage
 from rhodecode.lib import helpers as h
 from rhodecode.lib.auth import LoginRequired, HasRepoPermissionAnyDecorator
 from rhodecode.lib.utils2 import safe_int
@@ -33,8 +34,6 b' log = logging.getLogger(__name__)'
 class AuditLogsView(RepoAppView):
     def load_default_context(self):
         c = self._get_local_tmpl_context()
-
-
         return c
 
     @LoginRequired()
@@ -54,12 +53,15 b' class AuditLogsView(RepoAppView):'
         filter_term = self.request.GET.get('filter')
         user_log = RepoModel().get_repo_log(c.db_repo, filter_term)
 
-        def url_generator(**kw):
+        def url_generator(page_num):
+            query_params = {
+                'page': page_num
+            }
             if filter_term:
-                kw['filter'] = filter_term
-            return self.request.current_route_path(_query=kw)
+                query_params['filter'] = filter_term
+            return self.request.current_route_path(_query=query_params)
 
-        c.audit_logs = h.Page(
-            user_log, page=p, items_per_page=10, url=url_generator)
+        c.audit_logs = SqlPage(
+            user_log, page=p, items_per_page=10, url_maker=url_generator)
         c.filter_term = filter_term
         return self._get_template_context(c)