release: Merge default into stable for release preparation

docs/admin/tuning-change-lfs-dir.rst

@@ -0,0 +1,14 b''
+.. _git-lfs-loc:
+
+Change the |git| LFS storage Location
+-------------------------------------
+
+|RCE| manages |git| LFS files from the following default location
+:file:`/home/{user}/repos/.cache/lfs_store`. If you wish to change this, use
+the following steps:
+
+1. Open :menuselection:`Admin --> Settings --> VCS` as super-admin.
+
+In section called `Git Settings` you can change where the LFS
+objects should be stored.
+

docs/release-notes/release-notes-4.7.0.rst

@@ -0,0 +1,162 b''
+|RCE| 4.7.0 |RNS|
+-----------------
+
+Release Date
+^^^^^^^^^^^^
+
+- 2017-04-08
+
+
+New Features
+^^^^^^^^^^^^
+
+- Git: added support for Git LFS v2 protocol. RhodeCode now supports both
+  Mercurial Largefiles, and Git LFS for storing large binaries.
+- Largefiles: detect Git LFS or Mercurial Largefiles objects in UI.
+  Those are now available for downloading together with showing their size.
+- Files: Jupyter notebooks will be now rendered inside the file view. Including
+  MatJax support, and relative images.
+- Files: render images inside the file view.
+  Instead of displaying binary message, render images icons and gifs
+  inside the file view page.
+- Files: relative ULR support inside rendered files. It's now possible to
+  write Markup files and relative links will be handled from the RhodeCode
+  instance itself. Adds basic wiki functionality.
+- Files: allow to show inline pdf in browser using embedded files from source code.
+- Annotation: added shortcut links to browse the annotation view with previous
+  commits. Allows browsing history for each line from annotation view.
+- Pull Requests: add explicit close action instead of close with status from
+  status selector. This allows closing of approved or rejected
+  pull requests, without performing a merge action.
+- Authentication: LDAP now has an option to sync LDA groups using two
+  distinct ways. Either using rfc2307 or rfc2307bis. Increases compatibility
+  with different OpenLDAP and AD servers.
+- Slack: updated slack integration to use the attachments for nicer formatting.
+  Added number of commits inside the message, changed UI for all Slack events.
+- Authentication (EE edition only): added repository scope for VCS type auth
+  tokens. Each token can be now bound to particular repository for added security.
+- User administration: added audit page to allow showing single user actions.
+- API: implemented `get_user_audit_logs` method to fetch audit logs via API endpoint.
+- User administration: It's now possible to edit user group membership from
+  user view.
+- User groups administration: added managing and showing the group
+  synchronization in UI. It's now possible to enable manual group syncing on
+  already existing user groups from external sources such as LDAP/AD.
+- Repositories: added new strip view allowing removing commits from repositories
+  via web interface for repository administrators.
+- System Info: added info about workers and worker type.
+  Added more details about CPU. Expose workers of VCSServer in system info data.
+  Detect database migration errors.
+
+
+General
+^^^^^^^
+
+- Core: ported many views into pure pyramid code with python3.6 compatibility.
+- Core: removed deprecated Pyro4 backend from Enterprise code.
+- Maintenance: implemented maintenance view for Mercurial and GIT repositories.
+  For HG it will run `hg verify`, and for GIT a `git gc` command.
+- Notifications: different approach with fixed/standard container. Floating
+  notifications no longer hide the menu when browsed on top of the page.
+  Also added option to remove single elements from stacked notifications.
+- VCS server: exception-handling: better handling of remote exception and logging.
+- VCS server: propagate hooks tracebacks to VCS server for easier debugging.
+- Core: prevent `httplib3` logs to spam internal RhodeCode logs.
+  It often confuses people looking at those entries, misleading during debug.
+- Mercurial: allow editing Largefiles store location from web interface.
+- Git: allow editing GIT LFS store location from web interface.
+- API: add get_method API call. This allows showing the method and it's parameter
+  from the CLI without reading the documentation.
+  In addition use it's mechanics to propose users other methods with close names
+  if the calling method is not found.
+- UI: add timezone info into tooltips.
+- Dependencies: bumped pyramid to 1.7.4
+- Dependencies: bumped Mercurial version to 4.1.2
+
+
+Security
+^^^^^^^^
+
+- Hooks: added changes to propagate commit metadata on pre-push.
+  This allows easier implementation of checking hooks such as branch protection.
+- Hooks: added new pretx hook to allow mercurial checks such as protected
+  branches, or force push.
+- Auth: give owner of user group proper admin permissions to the user group.
+  This makes the behaviour consistent with repositories and repository groups.
+  And allows delegation of administration of those to other users.
+- Password reset: strengthen security on password reset logic.
+  Generate token that has special password reset role.
+  Set 10 minute expiration for the token.
+  Add some logic to prevent brute forcing attacks.
+  Use more implicit messages to prevent user email discovery attacks.
+- Core: added checks for password change for authenticated users in pure
+  Pyramid views. 2 views were still available and not forcing users to change
+  their passwords.
+- Auth tokens: removed builtin auth-token for users.
+  Builtin token were non-removable, and always generated for new users. This
+  wasn't best practice for security as some users are strictly not allowed to
+  use tokens. From now on new users needs a new token generation in case they
+  want to use token based authentication.
+- Auth tokens: don't generate builtin token for new users.
+  Also don't change them when password reset is made.
+- Api: added last-activity into returned data of get_user api.
+
+
+Performance
+^^^^^^^^^^^
+
+- Mercurial: enabled new `Zstandard` compression algorithm available with
+  Mercurial 4.1.X. This allows faster, more CPU efficient clones when used
+  with new Mercurial clients.
+
+- Users Admin: moved user admin to pyramid, and made it load users in chunks.
+  Fixed loading data to be lazy fetched, drastically improves speed of user
+  administration page in case of large amount of users.
+
+
+Fixes
+^^^^^
+
+- Search: goto commit search will now use a safe search option and never
+  throw any exceptions even if search is misconfigured
+  e.g. Elastic Search cluster is down.
+- Events: fix a case for events called from API that couldn't fetch
+  registered user object.
+- Comments: unlock submit if we use slash commands to set status.
+- UI: fixed an issue with date of last change was not displayed correctly.
+- Emails: added comment types (TODO/NOTE) into emails.
+- Events: fix wrongly returned author data.
+- Error middleware: read the instance title from cached object.
+  Reading from settings inside error handler can cause error hiding when
+  error_handler was caused by database errors.
+- Pull requests: show version age component should use local dates instead of UTC.
+- Pull requests: lock button when updating reviewers to forbid multi-submit
+  problems. Additionally fixed some small UI issues found in that view.
+- Pull requests: forbid browsing versions on closed pull request.
+- Pull requests: allow super-admins to delete pull requests instead of only owners.
+- Diffs: support mercurial copy operation in diffs details.
+- SVN: escape special chars to allow interactions with non-standard svn paths.
+  Path with special characters such as '#' will no longer trigger 404 errors.
+- Data grids: fix some styling and processing text display.
+- API: use consistent way to extract users, repos, repo groups and user groups
+  by id or name. Makes usage of Number vs String to differentiate if we pick
+  object ID or it's name this will allow editing of objects by either id or
+  it's name, including numeric string names.
+- API: validate commit_id when using commit_comment API
+- API: cleanup sessions enforce older_then must be a valid INT.
+
+
+Upgrade notes
+^^^^^^^^^^^^^
+
+- Auth-tokens: a builtin token will be migrated for all users into a custom
+  external token. We advise to inform users that the current builtin tokens
+  will now show as external ones. Builtin tokens were removed to allow expiring
+  ,or removing them. It's now possible to create users without any tokens.
+
+  From now on new users needs a new token generation in case they want to use
+  token based authentication.
+
+- Hooks: we added via migration a pre transaction hook for Mercurial. If you're
+  using a custom code inside pre-push function of rcextensions make sure it
+  will not block your pushes.

docs/tutorials/git-lfs-ext.rst

@@ -0,0 +1,93 b''
+.. _git-lfs-files:
+
+|git| LFS Extension
+===================
+
+
+Git Large File Storage (or LFS) is a new, open-source extension to Git that
+aims to improve handling of large files. It does this by replacing large files
+in your repository—such as graphics and videos—with simple text pointers.
+|RC| Server includes an embedded LFS object store server, allowing storage of
+large files without the need for an external object store.
+Git LFS is disabled by default, globally, and for each individual repository.
+
+.. note::
+
+    |RC| implements V2 API of Git LFS. Please make sure your git client is
+    using the latest version (2.0.X recommended) to leverage full feature set
+    of the V2 API.
+
+
+
+Enabling Git LFS
+++++++++++++++++
+
+Git LFS is disabled by default within |RC| Server.
+
+To enable Git LFS Globally:
+
+    - Go to :menuselection:`Admin --> Settings --> VCS`
+
+    - Scroll down into `Git settings`
+
+    - Tick `Enable lfs extension`
+
+    - Save your settings.
+
+Those settings apply globally to each repository that inherits from the defaults
+You can leave `lfs extension` disabled globally, and only enable it per
+repository that would use the lfs.
+
+
+.. note::
+
+    You might want to adjust the global storage location at that point, however
+    we recommend leaving the default one created.
+
+
+Installing and using the Git LFS command line client
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+Git LFS aims to integrate with the standard Git workflow as seamlessly
+as possible. To push your first Git LFS files to an existing repository
+Download and install the git-lfs command line client
+Install the Git LFS filters::
+
+    git lfs install
+
+This adds the following lines to the .gitconfig file located in your home directory::
+
+    [filter "lfs"]
+        clean = git-lfs clean %f
+        smudge = git-lfs smudge %f
+        required = true
+
+The above change applies globally, so it is not necessary to run this for
+each repository you work with. Choose the file types you would like LFS to
+handle by executing the git lfs track command. The git lfs track command
+creates or updates the .gitattributes file in your repository.
+Change to your cloned repository, then execute git add to ensure updates
+to the .gitattributes are later committed::
+
+    git lfs track "*.jpg"
+    git add .gitattributes
+
+Add, commit, and push your changes as you normally would::
+
+    git add image.jpg
+    git commit -m "Added an image"
+    git push
+
+When pushed, the Git tree updates to include a pointer to the file actual
+file content. This pointer will include the SHA256 hash of the object and its
+size in bytes. For example::
+
+    oid sha256:4fa32d6f9b1461c4a53618a47324ee43e36ce7ceaea2ad440cc811a7e6881be1
+    size 2580390
+
+
+The object itself will be uploaded to a separate location via the Git LFS Batch API.
+The transfer is validated and authorized by |RC| server itself.
+
+If give repository has Git LFS disabled, a proper message will be sent back to
+the client and upload of LFS objects will be forbidden.

pkgs/patch-celery-dateutil.diff

@@ -0,0 +1,12 b''
+diff -rup celery-2.2.10-orig/setup.py celery-2.2.10/setup.py
+--- celery-2.2.10-orig/setup.py	2017-02-25 15:30:34.000000000 +0100
++++ celery-2.2.10/setup.py	2017-02-25 15:30:34.000000000 +0100
+@@ -48,7 +48,7 @@ try:
+ except ImportError:
+     install_requires.append("importlib")
+ install_requires.extend([
+-    "python-dateutil>=1.5.0,<2.0.0",
++    "python-dateutil>=1.5.0,<2.2.0",
+     "anyjson>=0.3.1",
+     "kombu>=1.1.2,<2.0.0",
+     "pyparsing>=1.5.0,<2.0.0",

rhodecode/api/tests/test_get_method.py

@@ -0,0 +1,58 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2010-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+
+import pytest
+
+from rhodecode.api.tests.utils import build_data, api_call, assert_ok
+
+
[e-mail]("testuser_api", "app")
+class TestGetMethod(object):
+    def test_get_methods_no_matches(self):
+        id_, params = build_data(self.apikey, 'get_method', pattern='hello')
+        response = api_call(self.app, params)
+
+        expected = []
+        assert_ok(id_, expected, given=response.body)
+
+    def test_get_methods(self):
+        id_, params = build_data(self.apikey, 'get_method', pattern='*comment*')
+        response = api_call(self.app, params)
+
+        expected = ['changeset_comment', 'comment_pull_request',
+                    'comment_commit']
+        assert_ok(id_, expected, given=response.body)
+
+    def test_get_methods_on_single_match(self):
+        id_, params = build_data(self.apikey, 'get_method', pattern='*comment_commit*')
+        response = api_call(self.app, params)
+
+        expected = ['comment_commit',
+                    {'apiuser': '<RequiredType>',
+                     'comment_type': "<Optional:u'note'>",
+                     'commit_id': '<RequiredType>',
+                     'message': '<RequiredType>',
+                     'repoid': '<RequiredType>',
+                     'request': '<RequiredType>',
+                     'resolves_comment_id': '<Optional:None>',
+                     'status': '<Optional:None>',
+                     'userid': '<Optional:<OptionalAttr:apiuser>>'}]
+        assert_ok(id_, expected, given=response.body)

rhodecode/apps/_base/__init__.py

@@ -0,0 +1,163 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2016-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+import time
+import logging
+from pylons import tmpl_context as c
+from pyramid.httpexceptions import HTTPFound
+
+from rhodecode.lib import helpers as h
+from rhodecode.lib.utils2 import StrictAttributeDict, safe_int
+from rhodecode.model import repo
+from rhodecode.model.db import User
+from rhodecode.model.scm import ScmModel
+
+log = logging.getLogger(__name__)
+
+
+ADMIN_PREFIX = '/_admin'
+STATIC_FILE_PREFIX = '/_static'
+
+
+class TemplateArgs(StrictAttributeDict):
+    pass
+
+
+class BaseAppView(object):
+
+    def __init__(self, context, request):
+        self.request = request
+        self.context = context
+        self.session = request.session
+        self._rhodecode_user = request.user  # auth user
+        self._rhodecode_db_user = self._rhodecode_user.get_instance()
+        self._maybe_needs_password_change(
+            request.matched_route.name, self._rhodecode_db_user)
+
+    def _maybe_needs_password_change(self, view_name, user_obj):
+        log.debug('Checking if user %s needs password change on view %s',
+                  user_obj, view_name)
+        skip_user_views = [
+            'logout', 'login',
+            'my_account_password', 'my_account_password_update'
+        ]
+
+        if not user_obj:
+            return
+
+        if user_obj.username == User.DEFAULT_USER:
+            return
+
+        now = time.time()
+        should_change = user_obj.user_data.get('force_password_change')
+        change_after = safe_int(should_change) or 0
+        if should_change and now > change_after:
+            log.debug('User %s requires password change', user_obj)
+            h.flash('You are required to change your password', 'warning',
+                    ignore_duplicate=True)
+
+            if view_name not in skip_user_views:
+                raise HTTPFound(
+                    self.request.route_path('my_account_password'))
+
+    def _get_local_tmpl_context(self):
+        c = TemplateArgs()
+        c.auth_user = self.request.user
+        return c
+
+    def _register_global_c(self, tmpl_args):
+        """
+        Registers attributes to pylons global `c`
+        """
+        # TODO(marcink): remove once pyramid migration is finished
+        for k, v in tmpl_args.items():
+            setattr(c, k, v)
+
+    def _get_template_context(self, tmpl_args):
+        self._register_global_c(tmpl_args)
+
+        local_tmpl_args = {
+            'defaults': {},
+            'errors': {},
+        }
+        local_tmpl_args.update(tmpl_args)
+        return local_tmpl_args
+
+    def load_default_context(self):
+        """
+        example:
+
+        def load_default_context(self):
+            c = self._get_local_tmpl_context()
+            c.custom_var = 'foobar'
+            self._register_global_c(c)
+            return c
+        """
+        raise NotImplementedError('Needs implementation in view class')
+
+
+class RepoAppView(BaseAppView):
+
+    def __init__(self, context, request):
+        super(RepoAppView, self).__init__(context, request)
+        self.db_repo = request.db_repo
+        self.db_repo_name = self.db_repo.repo_name
+        self.db_repo_pull_requests = ScmModel().get_pull_requests(self.db_repo)
+
+    def _get_local_tmpl_context(self):
+        c = super(RepoAppView, self)._get_local_tmpl_context()
+        # register common vars for this type of view
+        c.rhodecode_db_repo = self.db_repo
+        c.repo_name = self.db_repo_name
+        c.repository_pull_requests = self.db_repo_pull_requests
+        return c
+
+
+class RepoRoutePredicate(object):
+    def __init__(self, val, config):
+        self.val = val
+
+    def text(self):
+        return 'repo_route = %s' % self.val
+
+    phash = text
+
+    def __call__(self, info, request):
+        repo_name = info['match']['repo_name']
+        repo_model = repo.RepoModel()
+        by_name_match = repo_model.get_by_repo_name(repo_name, cache=True)
+        # if we match quickly from database, short circuit the operation,
+        # and validate repo based on the type.
+        if by_name_match:
+            # register this as request object we can re-use later
+            request.db_repo = by_name_match
+            return True
+
+        by_id_match = repo_model.get_repo_by_id(repo_name)
+        if by_id_match:
+            request.db_repo = by_id_match
+            return True
+
+        return False
+
+
+def includeme(config):
+    config.add_route_predicate(
+        'repo_route', RepoRoutePredicate)

rhodecode/apps/admin/tests/test_admin_users.py

@@ -0,0 +1,142 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2010-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+import pytest
+
+from rhodecode.model.db import User, UserApiKeys
+
+from rhodecode.tests import (
+    TestController, TEST_USER_REGULAR_LOGIN, assert_session_flash)
+from rhodecode.tests.fixture import Fixture
+
+fixture = Fixture()
+
+
+def route_path(name, params=None, **kwargs):
+    import urllib
+    from rhodecode.apps._base import ADMIN_PREFIX
+
+    base_url = {
+        'users':
+            ADMIN_PREFIX + '/users',
+        'users_data':
+            ADMIN_PREFIX + '/users_data',
+        'edit_user_auth_tokens':
+            ADMIN_PREFIX + '/users/{user_id}/edit/auth_tokens',
+        'edit_user_auth_tokens_add':
+            ADMIN_PREFIX + '/users/{user_id}/edit/auth_tokens/new',
+        'edit_user_auth_tokens_delete':
+            ADMIN_PREFIX + '/users/{user_id}/edit/auth_tokens/delete',
+    }[name].format(**kwargs)
+
+    if params:
+        base_url = '{}?{}'.format(base_url, urllib.urlencode(params))
+    return base_url
+
+
+class TestAdminUsersView(TestController):
+
+    def test_show_users(self):
+        self.log_user()
+        self.app.get(route_path('users'))
+
+    def test_show_users_data(self, xhr_header):
+        self.log_user()
+        response = self.app.get(route_path(
+            'users_data'), extra_environ=xhr_header)
+
+        all_users = User.query().filter(
+            User.username != User.DEFAULT_USER).count()
+        assert response.json['recordsTotal'] == all_users
+
+    def test_show_users_data_filtered(self, xhr_header):
+        self.log_user()
+        response = self.app.get(route_path(
+            'users_data', params={'search[value]': 'empty_search'}),
+            extra_environ=xhr_header)
+
+        all_users = User.query().filter(
+            User.username != User.DEFAULT_USER).count()
+        assert response.json['recordsTotal'] == all_users
+        assert response.json['recordsFiltered'] == 0
+
+    def test_auth_tokens_default_user(self):
+        self.log_user()
+        user = User.get_default_user()
+        response = self.app.get(
+            route_path('edit_user_auth_tokens', user_id=user.user_id),
+            status=302)
+
+    def test_auth_tokens(self):
+        self.log_user()
+
+        user = User.get_by_username(TEST_USER_REGULAR_LOGIN)
+        response = self.app.get(
+            route_path('edit_user_auth_tokens', user_id=user.user_id))
+        for token in user.auth_tokens:
+            response.mustcontain(token)
+            response.mustcontain('never')
+
+    @pytest.mark.parametrize("desc, lifetime", [
+        ('forever', -1),
+        ('5mins', 60*5),
+        ('30days', 60*60*24*30),
+    ])
+    def test_add_auth_token(self, desc, lifetime, user_util):
+        self.log_user()
+        user = user_util.create_user()
+        user_id = user.user_id
+
+        response = self.app.post(
+            route_path('edit_user_auth_tokens_add', user_id=user_id),
+            {'description': desc, 'lifetime': lifetime,
+             'csrf_token': self.csrf_token})
+        assert_session_flash(response, 'Auth token successfully created')
+
+        response = response.follow()
+        user = User.get(user_id)
+        for auth_token in user.auth_tokens:
+            response.mustcontain(auth_token)
+
+    def test_delete_auth_token(self, user_util):
+        self.log_user()
+        user = user_util.create_user()
+        user_id = user.user_id
+        keys = user.extra_auth_tokens
+        assert 2 == len(keys)
+
+        response = self.app.post(
+            route_path('edit_user_auth_tokens_add', user_id=user_id),
+            {'description': 'desc', 'lifetime': -1,
+             'csrf_token': self.csrf_token})
+        assert_session_flash(response, 'Auth token successfully created')
+        response.follow()
+
+        # now delete our key
+        keys = UserApiKeys.query().filter(UserApiKeys.user_id == user_id).all()
+        assert 3 == len(keys)
+
+        response = self.app.post(
+            route_path('edit_user_auth_tokens_delete', user_id=user_id),
+            {'del_auth_token': keys[0].api_key, 'csrf_token': self.csrf_token})
+
+        assert_session_flash(response, 'Auth token successfully deleted')
+        keys = UserApiKeys.query().filter(UserApiKeys.user_id == user_id).all()
+        assert 2 == len(keys)

rhodecode/apps/admin/views/users.py

@@ -0,0 +1,317 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2016-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+import logging
+
+from pyramid.httpexceptions import HTTPFound
+from pyramid.view import view_config
+
+from rhodecode.lib.helpers import Page
+from rhodecode_tools.lib.ext_json import json
+
+from rhodecode.apps._base import BaseAppView
+from rhodecode.lib.auth import (
+    LoginRequired, HasPermissionAllDecorator, CSRFRequired)
+from rhodecode.lib import helpers as h
+from rhodecode.lib.utils import PartialRenderer
+from rhodecode.lib.utils2 import safe_int, safe_unicode
+from rhodecode.model.auth_token import AuthTokenModel
+from rhodecode.model.user import UserModel
+from rhodecode.model.user_group import UserGroupModel
+from rhodecode.model.db import User, or_
+from rhodecode.model.meta import Session
+
+log = logging.getLogger(__name__)
+
+
+class AdminUsersView(BaseAppView):
+    ALLOW_SCOPED_TOKENS = False
+    """
+    This view has alternative version inside EE, if modified please take a look
+    in there as well.
+    """
+
+    def load_default_context(self):
+        c = self._get_local_tmpl_context()
+        c.allow_scoped_tokens = self.ALLOW_SCOPED_TOKENS
+        self._register_global_c(c)
+        return c
+
+    def _redirect_for_default_user(self, username):
+        _ = self.request.translate
+        if username == User.DEFAULT_USER:
+            h.flash(_("You can't edit this user"), category='warning')
+            # TODO(marcink): redirect to 'users' admin panel once this
+            # is a pyramid view
+            raise HTTPFound('/')
+
+    def _extract_ordering(self, request):
+        column_index = safe_int(request.GET.get('order[0][column]'))
+        order_dir = request.GET.get(
+            'order[0][dir]', 'desc')
+        order_by = request.GET.get(
+            'columns[%s][data][sort]' % column_index, 'name_raw')
+
+        # translate datatable to DB columns
+        order_by = {
+            'first_name': 'name',
+            'last_name': 'lastname',
+        }.get(order_by) or order_by
+
+        search_q = request.GET.get('search[value]')
+        return search_q, order_by, order_dir
+
+    def _extract_chunk(self, request):
+        start = safe_int(request.GET.get('start'), 0)
+        length = safe_int(request.GET.get('length'), 25)
+        draw = safe_int(request.GET.get('draw'))
+        return draw, start, length
+
+    @HasPermissionAllDecorator('hg.admin')
+    @view_config(
+        route_name='users', request_method='GET',
+        renderer='rhodecode:templates/admin/users/users.mako')
+    def users_list(self):
+        c = self.load_default_context()
+        return self._get_template_context(c)
+
+    @HasPermissionAllDecorator('hg.admin')
+    @view_config(
+        # renderer defined below
+        route_name='users_data', request_method='GET', renderer='json',
+        xhr=True)
+    def users_list_data(self):
+        draw, start, limit = self._extract_chunk(self.request)
+        search_q, order_by, order_dir = self._extract_ordering(self.request)
+
+        _render = PartialRenderer('data_table/_dt_elements.mako')
+
+        def user_actions(user_id, username):
+            return _render("user_actions", user_id, username)
+
+        users_data_total_count = User.query()\
+            .filter(User.username != User.DEFAULT_USER) \
+            .count()
+
+        # json generate
+        base_q = User.query().filter(User.username != User.DEFAULT_USER)
+
+        if search_q:
+            like_expression = u'%{}%'.format(safe_unicode(search_q))
+            base_q = base_q.filter(or_(
+                User.username.ilike(like_expression),
+                User._email.ilike(like_expression),
+                User.name.ilike(like_expression),
+                User.lastname.ilike(like_expression),
+            ))
+
+        users_data_total_filtered_count = base_q.count()
+
+        sort_col = getattr(User, order_by, None)
+        if sort_col and order_dir == 'asc':
+            base_q = base_q.order_by(sort_col.asc().nullslast())
+        elif sort_col:
+            base_q = base_q.order_by(sort_col.desc().nullslast())
+
+        base_q = base_q.offset(start).limit(limit)
+        users_list = base_q.all()
+
+        users_data = []
+        for user in users_list:
+            users_data.append({
+                "username": h.gravatar_with_user(user.username),
+                "email": user.email,
+                "first_name": h.escape(user.name),
+                "last_name": h.escape(user.lastname),
+                "last_login": h.format_date(user.last_login),
+                "last_activity": h.format_date(user.last_activity),
+                "active": h.bool2icon(user.active),
+                "active_raw": user.active,
+                "admin": h.bool2icon(user.admin),
+                "extern_type": user.extern_type,
+                "extern_name": user.extern_name,
+                "action": user_actions(user.user_id, user.username),
+            })
+
+        data = ({
+            'draw': draw,
+            'data': users_data,
+            'recordsTotal': users_data_total_count,
+            'recordsFiltered': users_data_total_filtered_count,
+        })
+
+        return data
+
+    @LoginRequired()
+    @HasPermissionAllDecorator('hg.admin')
+    @view_config(
+        route_name='edit_user_auth_tokens', request_method='GET',
+        renderer='rhodecode:templates/admin/users/user_edit.mako')
+    def auth_tokens(self):
+        _ = self.request.translate
+        c = self.load_default_context()
+
+        user_id = self.request.matchdict.get('user_id')
+        c.user = User.get_or_404(user_id, pyramid_exc=True)
+        self._redirect_for_default_user(c.user.username)
+
+        c.active = 'auth_tokens'
+
+        c.lifetime_values = [
+            (str(-1), _('forever')),
+            (str(5), _('5 minutes')),
+            (str(60), _('1 hour')),
+            (str(60 * 24), _('1 day')),
+            (str(60 * 24 * 30), _('1 month')),
+        ]
+        c.lifetime_options = [(c.lifetime_values, _("Lifetime"))]
+        c.role_values = [
+            (x, AuthTokenModel.cls._get_role_name(x))
+            for x in AuthTokenModel.cls.ROLES]
+        c.role_options = [(c.role_values, _("Role"))]
+        c.user_auth_tokens = AuthTokenModel().get_auth_tokens(
+            c.user.user_id, show_expired=True)
+        return self._get_template_context(c)
+
+    def maybe_attach_token_scope(self, token):
+        # implemented in EE edition
+        pass
+
+    @LoginRequired()
+    @HasPermissionAllDecorator('hg.admin')
+    @CSRFRequired()
+    @view_config(
+        route_name='edit_user_auth_tokens_add', request_method='POST')
+    def auth_tokens_add(self):
+        _ = self.request.translate
+        c = self.load_default_context()
+
+        user_id = self.request.matchdict.get('user_id')
+        c.user = User.get_or_404(user_id, pyramid_exc=True)
+        self._redirect_for_default_user(c.user.username)
+
+        lifetime = safe_int(self.request.POST.get('lifetime'), -1)
+        description = self.request.POST.get('description')
+        role = self.request.POST.get('role')
+
+        token = AuthTokenModel().create(
+            c.user.user_id, description, lifetime, role)
+        self.maybe_attach_token_scope(token)
+        Session().commit()
+
+        h.flash(_("Auth token successfully created"), category='success')
+        return HTTPFound(h.route_path('edit_user_auth_tokens', user_id=user_id))
+
+    @LoginRequired()
+    @HasPermissionAllDecorator('hg.admin')
+    @CSRFRequired()
+    @view_config(
+        route_name='edit_user_auth_tokens_delete', request_method='POST')
+    def auth_tokens_delete(self):
+        _ = self.request.translate
+        c = self.load_default_context()
+
+        user_id = self.request.matchdict.get('user_id')
+        c.user = User.get_or_404(user_id, pyramid_exc=True)
+        self._redirect_for_default_user(c.user.username)
+
+        del_auth_token = self.request.POST.get('del_auth_token')
+
+        if del_auth_token:
+            AuthTokenModel().delete(del_auth_token, c.user.user_id)
+            Session().commit()
+            h.flash(_("Auth token successfully deleted"), category='success')
+
+        return HTTPFound(h.route_path('edit_user_auth_tokens', user_id=user_id))
+
+    @LoginRequired()
+    @HasPermissionAllDecorator('hg.admin')
+    @view_config(
+        route_name='edit_user_groups_management', request_method='GET',
+        renderer='rhodecode:templates/admin/users/user_edit.mako')
+    def groups_management(self):
+        c = self.load_default_context()
+
+        user_id = self.request.matchdict.get('user_id')
+        c.user = User.get_or_404(user_id, pyramid_exc=True)
+        c.data = c.user.group_member
+        self._redirect_for_default_user(c.user.username)
+        groups = [UserGroupModel.get_user_groups_as_dict(group.users_group) for group in c.user.group_member]
+        c.groups = json.dumps(groups)
+        c.active = 'groups'
+
+        return self._get_template_context(c)
+
+    @LoginRequired()
+    @HasPermissionAllDecorator('hg.admin')
+    @view_config(
+        route_name='edit_user_groups_management_updates', request_method='POST')
+    def groups_management_updates(self):
+        _ = self.request.translate
+        c = self.load_default_context()
+
+        user_id = self.request.matchdict.get('user_id')
+        c.user = User.get_or_404(user_id, pyramid_exc=True)
+        self._redirect_for_default_user(c.user.username)
+
+        users_groups = set(self.request.POST.getall('users_group_id'))
+        users_groups_model = []
+
+        for ugid in users_groups:
+            users_groups_model.append(UserGroupModel().get_group(safe_int(ugid)))
+        user_group_model = UserGroupModel()
+        user_group_model.change_groups(c.user, users_groups_model)
+
+        Session().commit()
+        c.active = 'user_groups_management'
+        h.flash(_("Groups successfully changed"), category='success')
+
+        return HTTPFound(h.route_path(
+            'edit_user_groups_management', user_id=user_id))
+
+    @LoginRequired()
+    @HasPermissionAllDecorator('hg.admin')
+    @view_config(
+        route_name='edit_user_audit_logs', request_method='GET',
+        renderer='rhodecode:templates/admin/users/user_edit.mako')
+    def user_audit_logs(self):
+        _ = self.request.translate
+        c = self.load_default_context()
+
+        user_id = self.request.matchdict.get('user_id')
+        c.user = User.get_or_404(user_id, pyramid_exc=True)
+        self._redirect_for_default_user(c.user.username)
+        c.active = 'audit'
+
+        p = safe_int(self.request.GET.get('page', 1), 1)
+
+        filter_term = self.request.GET.get('filter')
+        c.user_log = UserModel().get_user_log(c.user, filter_term)
+
+        def url_generator(**kw):
+            if filter_term:
+                kw['filter'] = filter_term
+            return self.request.current_route_path(_query=kw)
+
+        c.user_log = Page(c.user_log, page=p, items_per_page=10,
+                          url=url_generator)
+        c.filter_term = filter_term
+        return self._get_template_context(c)
+

rhodecode/apps/my_account/__init__.py

@@ -0,0 +1,52 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2016-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+
+from rhodecode.apps._base import ADMIN_PREFIX
+
+
+def includeme(config):
+
+    config.add_route(
+        name='my_account_profile',
+        pattern=ADMIN_PREFIX + '/my_account/profile')
+
+    config.add_route(
+        name='my_account_password',
+        pattern=ADMIN_PREFIX + '/my_account/password')
+
+    config.add_route(
+        name='my_account_password_update',
+        pattern=ADMIN_PREFIX + '/my_account/password')
+
+    config.add_route(
+        name='my_account_auth_tokens',
+        pattern=ADMIN_PREFIX + '/my_account/auth_tokens')
+    config.add_route(
+        name='my_account_auth_tokens_add',
+        pattern=ADMIN_PREFIX + '/my_account/auth_tokens/new',
+        )
+    config.add_route(
+        name='my_account_auth_tokens_delete',
+        pattern=ADMIN_PREFIX + '/my_account/auth_tokens/delete',
+        )
+
+    # Scan module for configuration decorators.
+    config.scan()

rhodecode/apps/my_account/tests/__init__.py

@@ -0,0 +1,19 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2016-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/

rhodecode/apps/my_account/tests/test_my_account_auth_tokens.py

@@ -0,0 +1,111 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2010-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+import pytest
+
+from rhodecode.apps._base import ADMIN_PREFIX
+from rhodecode.model.db import User
+from rhodecode.tests import (
+    TestController, TEST_USER_ADMIN_LOGIN, TEST_USER_ADMIN_PASS,
+    TEST_USER_REGULAR_LOGIN, TEST_USER_REGULAR_PASS, assert_session_flash)
+from rhodecode.tests.fixture import Fixture
+from rhodecode.tests.utils import AssertResponse
+
+fixture = Fixture()
+
+
+def route_path(name, **kwargs):
+    return {
+        'my_account_auth_tokens':
+            ADMIN_PREFIX + '/my_account/auth_tokens',
+        'my_account_auth_tokens_add':
+            ADMIN_PREFIX + '/my_account/auth_tokens/new',
+        'my_account_auth_tokens_delete':
+            ADMIN_PREFIX + '/my_account/auth_tokens/delete',
+    }[name].format(**kwargs)
+
+
+class TestMyAccountAuthTokens(TestController):
+
+    def test_my_account_auth_tokens(self):
+        usr = self.log_user('test_regular2', 'test12')
+        user = User.get(usr['user_id'])
+        response = self.app.get(route_path('my_account_auth_tokens'))
+        for token in user.auth_tokens:
+            response.mustcontain(token)
+            response.mustcontain('never')
+
+    def test_my_account_add_auth_tokens_wrong_csrf(self, user_util):
+        user = user_util.create_user(password='qweqwe')
+        self.log_user(user.username, 'qweqwe')
+
+        self.app.post(
+            route_path('my_account_auth_tokens_add'),
+            {'description': 'desc', 'lifetime': -1}, status=403)
+
+    @pytest.mark.parametrize("desc, lifetime", [
+        ('forever', -1),
+        ('5mins', 60*5),
+        ('30days', 60*60*24*30),
+    ])
+    def test_my_account_add_auth_tokens(self, desc, lifetime, user_util):
+        user = user_util.create_user(password='qweqwe')
+        user_id = user.user_id
+        self.log_user(user.username, 'qweqwe')
+
+        response = self.app.post(
+            route_path('my_account_auth_tokens_add'),
+            {'description': desc, 'lifetime': lifetime,
+             'csrf_token': self.csrf_token})
+        assert_session_flash(response, 'Auth token successfully created')
+
+        response = response.follow()
+        user = User.get(user_id)
+        for auth_token in user.auth_tokens:
+            response.mustcontain(auth_token)
+
+    def test_my_account_delete_auth_token(self, user_util):
+        user = user_util.create_user(password='qweqwe')
+        user_id = user.user_id
+        self.log_user(user.username, 'qweqwe')
+
+        user = User.get(user_id)
+        keys = user.extra_auth_tokens
+        assert 2 == len(keys)
+
+        response = self.app.post(
+            route_path('my_account_auth_tokens_add'),
+            {'description': 'desc', 'lifetime': -1,
+             'csrf_token': self.csrf_token})
+        assert_session_flash(response, 'Auth token successfully created')
+        response.follow()
+
+        user = User.get(user_id)
+        keys = user.extra_auth_tokens
+        assert 3 == len(keys)
+
+        response = self.app.post(
+            route_path('my_account_auth_tokens_delete'),
+            {'del_auth_token': keys[0].api_key, 'csrf_token': self.csrf_token})
+        assert_session_flash(response, 'Auth token successfully deleted')
+
+        user = User.get(user_id)
+        keys = user.extra_auth_tokens
+        assert 2 == len(keys)

rhodecode/apps/my_account/tests/test_my_account_password.py

@@ -0,0 +1,137 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2010-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+import pytest
+import mock
+
+from rhodecode.apps._base import ADMIN_PREFIX
+from rhodecode.lib import helpers as h
+from rhodecode.lib.auth import check_password
+from rhodecode.model.meta import Session
+from rhodecode.model.user import UserModel
+from rhodecode.tests import assert_session_flash
+from rhodecode.tests.fixture import Fixture, TestController, error_function
+
+fixture = Fixture()
+
+
+def route_path(name, **kwargs):
+    return {
+        'home': '/',
+        'my_account_password':
+            ADMIN_PREFIX + '/my_account/password',
+    }[name].format(**kwargs)
+
+
+test_user_1 = 'testme'
+test_user_1_password = '0jd83nHNS/d23n'
+
+
+class TestMyAccountPassword(TestController):
+    def test_valid_change_password(self, user_util):
+        new_password = 'my_new_valid_password'
+        user = user_util.create_user(password=test_user_1_password)
+        self.log_user(user.username, test_user_1_password)
+
+        form_data = [
+            ('current_password', test_user_1_password),
+            ('__start__', 'new_password:mapping'),
+            ('new_password', new_password),
+            ('new_password-confirm', new_password),
+            ('__end__', 'new_password:mapping'),
+            ('csrf_token', self.csrf_token),
+        ]
+        response = self.app.post(route_path('my_account_password'), form_data).follow()
+        assert 'Successfully updated password' in response
+
+        # check_password depends on user being in session
+        Session().add(user)
+        try:
+            assert check_password(new_password, user.password)
+        finally:
+            Session().expunge(user)
+
+    @pytest.mark.parametrize('current_pw, new_pw, confirm_pw', [
+        ('', 'abcdef123', 'abcdef123'),
+        ('wrong_pw', 'abcdef123', 'abcdef123'),
+        (test_user_1_password, test_user_1_password, test_user_1_password),
+        (test_user_1_password, '', ''),
+        (test_user_1_password, 'abcdef123', ''),
+        (test_user_1_password, '', 'abcdef123'),
+        (test_user_1_password, 'not_the', 'same_pw'),
+        (test_user_1_password, 'short', 'short'),
+    ])
+    def test_invalid_change_password(self, current_pw, new_pw, confirm_pw,
+                                     user_util):
+        user = user_util.create_user(password=test_user_1_password)
+        self.log_user(user.username, test_user_1_password)
+
+        form_data = [
+            ('current_password', current_pw),
+            ('__start__', 'new_password:mapping'),
+            ('new_password', new_pw),
+            ('new_password-confirm', confirm_pw),
+            ('__end__', 'new_password:mapping'),
+            ('csrf_token', self.csrf_token),
+        ]
+        response = self.app.post(route_path('my_account_password'), form_data)
+
+        assert_response = response.assert_response()
+        assert assert_response.get_elements('.error-block')
+
+    @mock.patch.object(UserModel, 'update_user', error_function)
+    def test_invalid_change_password_exception(self, user_util):
+        user = user_util.create_user(password=test_user_1_password)
+        self.log_user(user.username, test_user_1_password)
+
+        form_data = [
+            ('current_password', test_user_1_password),
+            ('__start__', 'new_password:mapping'),
+            ('new_password', '123456'),
+            ('new_password-confirm', '123456'),
+            ('__end__', 'new_password:mapping'),
+            ('csrf_token', self.csrf_token),
+        ]
+        response = self.app.post(route_path('my_account_password'), form_data)
+        assert_session_flash(
+            response, 'Error occurred during update of user password')
+
+    def test_password_is_updated_in_session_on_password_change(self, user_util):
+        old_password = 'abcdef123'
+        new_password = 'abcdef124'
+
+        user = user_util.create_user(password=old_password)
+        session = self.log_user(user.username, old_password)
+        old_password_hash = session['password']
+
+        form_data = [
+            ('current_password', old_password),
+            ('__start__', 'new_password:mapping'),
+            ('new_password', new_password),
+            ('new_password-confirm', new_password),
+            ('__end__', 'new_password:mapping'),
+            ('csrf_token', self.csrf_token),
+        ]
+        self.app.post(route_path('my_account_password'), form_data)
+
+        response = self.app.get(route_path('home'))
+        new_password_hash = response.session['rhodecode_user']['password']
+
+        assert old_password_hash != new_password_hash
  No newline at end of file

rhodecode/apps/my_account/tests/test_my_account_profile.py

@@ -0,0 +1,55 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2010-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+import pytest
+
+from rhodecode.apps._base import ADMIN_PREFIX
+from rhodecode.tests import (
+    TestController, TEST_USER_ADMIN_LOGIN,
+    TEST_USER_REGULAR_LOGIN, TEST_USER_REGULAR_PASS)
+from rhodecode.tests.fixture import Fixture
+
+fixture = Fixture()
+
+
+def route_path(name, **kwargs):
+    return {
+        'my_account':
+            ADMIN_PREFIX + '/my_account/profile',
+    }[name].format(**kwargs)
+
+
+class TestMyAccountProfile(TestController):
+
+    def test_my_account(self):
+        self.log_user()
+        response = self.app.get(route_path('my_account'))
+
+        response.mustcontain(TEST_USER_ADMIN_LOGIN)
+        response.mustcontain('href="/_admin/my_account/edit"')
+        response.mustcontain('Photo')
+
+    def test_my_account_regular_user(self):
+        self.log_user(TEST_USER_REGULAR_LOGIN, TEST_USER_REGULAR_PASS)
+        response = self.app.get(route_path('my_account'))
+
+        response.mustcontain(TEST_USER_REGULAR_LOGIN)
+        response.mustcontain('href="/_admin/my_account/edit"')
+        response.mustcontain('Photo')

rhodecode/apps/my_account/views.py

@@ -0,0 +1,194 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2016-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+import logging
+
+from pyramid.httpexceptions import HTTPFound
+from pyramid.view import view_config
+
+from rhodecode.apps._base import BaseAppView
+from rhodecode import forms
+from rhodecode.lib.auth import LoginRequired, NotAnonymous, CSRFRequired
+from rhodecode.lib import helpers as h
+from rhodecode.lib.utils2 import safe_int, md5
+from rhodecode.model.auth_token import AuthTokenModel
+from rhodecode.model.meta import Session
+from rhodecode.model.user import UserModel
+from rhodecode.model.validation_schema.schemas import user_schema
+
+log = logging.getLogger(__name__)
+
+
+class MyAccountView(BaseAppView):
+    ALLOW_SCOPED_TOKENS = False
+    """
+    This view has alternative version inside EE, if modified please take a look
+    in there as well.
+    """
+
+    def load_default_context(self):
+        c = self._get_local_tmpl_context()
+        c.user = c.auth_user.get_instance()
+        c.allow_scoped_tokens = self.ALLOW_SCOPED_TOKENS
+        self._register_global_c(c)
+        return c
+
+    @LoginRequired()
+    @NotAnonymous()
+    @view_config(
+        route_name='my_account_profile', request_method='GET',
+        renderer='rhodecode:templates/admin/my_account/my_account.mako')
+    def my_account_profile(self):
+        c = self.load_default_context()
+        c.active = 'profile'
+        return self._get_template_context(c)
+
+    @LoginRequired()
+    @NotAnonymous()
+    @view_config(
+        route_name='my_account_password', request_method='GET',
+        renderer='rhodecode:templates/admin/my_account/my_account.mako')
+    def my_account_password(self):
+        c = self.load_default_context()
+        c.active = 'password'
+        c.extern_type = c.user.extern_type
+
+        schema = user_schema.ChangePasswordSchema().bind(
+            username=c.user.username)
+
+        form = forms.Form(
+            schema, buttons=(forms.buttons.save, forms.buttons.reset))
+
+        c.form = form
+        return self._get_template_context(c)
+
+    @LoginRequired()
+    @NotAnonymous()
+    @CSRFRequired()
+    @view_config(
+        route_name='my_account_password', request_method='POST',
+        renderer='rhodecode:templates/admin/my_account/my_account.mako')
+    def my_account_password_update(self):
+        _ = self.request.translate
+        c = self.load_default_context()
+        c.active = 'password'
+        c.extern_type = c.user.extern_type
+
+        schema = user_schema.ChangePasswordSchema().bind(
+            username=c.user.username)
+
+        form = forms.Form(
+            schema, buttons=(forms.buttons.save, forms.buttons.reset))
+
+        if c.extern_type != 'rhodecode':
+            raise HTTPFound(self.request.route_path('my_account_password'))
+
+        controls = self.request.POST.items()
+        try:
+            valid_data = form.validate(controls)
+            UserModel().update_user(c.user.user_id, **valid_data)
+            c.user.update_userdata(force_password_change=False)
+            Session().commit()
+        except forms.ValidationFailure as e:
+            c.form = e
+            return self._get_template_context(c)
+
+        except Exception:
+            log.exception("Exception updating password")
+            h.flash(_('Error occurred during update of user password'),
+                    category='error')
+        else:
+            instance = c.auth_user.get_instance()
+            self.session.setdefault('rhodecode_user', {}).update(
+                {'password': md5(instance.password)})
+            self.session.save()
+            h.flash(_("Successfully updated password"), category='success')
+
+        raise HTTPFound(self.request.route_path('my_account_password'))
+
+    @LoginRequired()
+    @NotAnonymous()
+    @view_config(
+        route_name='my_account_auth_tokens', request_method='GET',
+        renderer='rhodecode:templates/admin/my_account/my_account.mako')
+    def my_account_auth_tokens(self):
+        _ = self.request.translate
+
+        c = self.load_default_context()
+        c.active = 'auth_tokens'
+
+        c.lifetime_values = [
+            (str(-1), _('forever')),
+            (str(5), _('5 minutes')),
+            (str(60), _('1 hour')),
+            (str(60 * 24), _('1 day')),
+            (str(60 * 24 * 30), _('1 month')),
+        ]
+        c.lifetime_options = [(c.lifetime_values, _("Lifetime"))]
+        c.role_values = [
+            (x, AuthTokenModel.cls._get_role_name(x))
+            for x in AuthTokenModel.cls.ROLES]
+        c.role_options = [(c.role_values, _("Role"))]
+        c.user_auth_tokens = AuthTokenModel().get_auth_tokens(
+            c.user.user_id, show_expired=True)
+        return self._get_template_context(c)
+
+    def maybe_attach_token_scope(self, token):
+        # implemented in EE edition
+        pass
+
+    @LoginRequired()
+    @NotAnonymous()
+    @CSRFRequired()
+    @view_config(
+        route_name='my_account_auth_tokens_add', request_method='POST')
+    def my_account_auth_tokens_add(self):
+        _ = self.request.translate
+        c = self.load_default_context()
+
+        lifetime = safe_int(self.request.POST.get('lifetime'), -1)
+        description = self.request.POST.get('description')
+        role = self.request.POST.get('role')
+
+        token = AuthTokenModel().create(
+            c.user.user_id, description, lifetime, role)
+        self.maybe_attach_token_scope(token)
+        Session().commit()
+
+        h.flash(_("Auth token successfully created"), category='success')
+        return HTTPFound(h.route_path('my_account_auth_tokens'))
+
+    @LoginRequired()
+    @NotAnonymous()
+    @CSRFRequired()
+    @view_config(
+        route_name='my_account_auth_tokens_delete', request_method='POST')
+    def my_account_auth_tokens_delete(self):
+        _ = self.request.translate
+        c = self.load_default_context()
+
+        del_auth_token = self.request.POST.get('del_auth_token')
+
+        if del_auth_token:
+            AuthTokenModel().delete(del_auth_token, c.user.user_id)
+            Session().commit()
+            h.flash(_("Auth token successfully deleted"), category='success')
+
+        return HTTPFound(h.route_path('my_account_auth_tokens'))

rhodecode/apps/repository/__init__.py

@@ -0,0 +1,46 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2016-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+
+def includeme(config):
+
+    config.add_route(
+        name='repo_maintenance',
+        pattern='/{repo_name:.*?[^/]}/maintenance', repo_route=True)
+
+    config.add_route(
+        name='repo_maintenance_execute',
+        pattern='/{repo_name:.*?[^/]}/maintenance/execute', repo_route=True)
+
+
+    # Strip
+    config.add_route(
+        name='strip',
+        pattern='/{repo_name:.*?[^/]}/strip', repo_route=True)
+
+    config.add_route(
+        name='strip_check',
+        pattern='/{repo_name:.*?[^/]}/strip_check', repo_route=True)
+
+    config.add_route(
+        name='strip_execute',
+        pattern='/{repo_name:.*?[^/]}/strip_execute', repo_route=True)
+    # Scan module for configuration decorators.
+    config.scan()

rhodecode/apps/repository/views/__init__.py

@@ -0,0 +1,19 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2011-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/

rhodecode/apps/repository/views/repo_maintainance.py

@@ -0,0 +1,70 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2011-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+import logging
+
+from pyramid.view import view_config
+
+from rhodecode.apps._base import RepoAppView
+from rhodecode.lib.auth import (LoginRequired, HasRepoPermissionAnyDecorator,
+    NotAnonymous)
+from rhodecode.lib import repo_maintenance
+
+log = logging.getLogger(__name__)
+
+
+class RepoMaintenanceView(RepoAppView):
+    def load_default_context(self):
+        c = self._get_local_tmpl_context()
+
+        # TODO(marcink): remove repo_info and use c.rhodecode_db_repo instead
+        c.repo_info = self.db_repo
+
+        self._register_global_c(c)
+        return c
+
+    @LoginRequired()
+    @NotAnonymous()
+    @HasRepoPermissionAnyDecorator('repository.admin')
+    @view_config(
+        route_name='repo_maintenance', request_method='GET',
+        renderer='rhodecode:templates/admin/repos/repo_edit.mako')
+    def repo_maintenance(self):
+        c = self.load_default_context()
+        c.active = 'maintenance'
+        maintenance = repo_maintenance.RepoMaintenance()
+        c.executable_tasks = maintenance.get_tasks_for_repo(self.db_repo)
+        return self._get_template_context(c)
+
+    @LoginRequired()
+    @NotAnonymous()
+    @HasRepoPermissionAnyDecorator('repository.admin')
+    @view_config(
+        route_name='repo_maintenance_execute', request_method='GET',
+        renderer='json', xhr=True)
+    def repo_maintenance_execute(self):
+        c = self.load_default_context()
+        c.active = 'maintenance'
+        _ = self.request.translate
+
+        maintenance = repo_maintenance.RepoMaintenance()
+        executed_types = maintenance.execute(self.db_repo)
+
+        return executed_types

rhodecode/apps/repository/views/strip.py

@@ -0,0 +1,110 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2011-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+import logging
+from pyramid.view import view_config
+
+from rhodecode.apps._base import RepoAppView
+from rhodecode.lib.auth import (LoginRequired, HasRepoPermissionAnyDecorator,
+    NotAnonymous)
+
+
+log = logging.getLogger(__name__)
+
+
+class StripView(RepoAppView):
+    def load_default_context(self):
+        c = self._get_local_tmpl_context()
+
+        # TODO(marcink): remove repo_info and use c.rhodecode_db_repo instead
+        c.repo_info = self.db_repo
+
+        self._register_global_c(c)
+        return c
+
+    @LoginRequired()
+    @NotAnonymous()
+    @HasRepoPermissionAnyDecorator('repository.admin')
+    @view_config(
+        route_name='strip', request_method='GET',
+        renderer='rhodecode:templates/admin/repos/repo_edit.mako')
+    def strip(self):
+        c = self.load_default_context()
+        c.active = 'strip'
+        c.strip_limit = 10
+
+        return self._get_template_context(c)
+
+    @LoginRequired()
+    @NotAnonymous()
+    @HasRepoPermissionAnyDecorator('repository.admin')
+    @view_config(
+        route_name='strip_check', request_method='POST',
+        renderer='json', xhr=True
+    )
+    def strip_check(self):
+        from rhodecode.lib.vcs.backends.base import EmptyCommit
+        data = {}
+        rp = self.request.POST
+        for i in range(1, 11):
+            chset = 'changeset_id-%d'%(i,)
+            check = rp.get(chset)
+            if check:
+                data[i] = self.db_repo.get_changeset(rp[chset])
+                if isinstance(data[i], EmptyCommit):
+                    data[i] = {'rev': None, 'commit': rp[chset]}
+                else:
+                    data[i] = {'rev': data[i].raw_id, 'branch': data[i].branch, 'author': data[i].author,
+                               'comment': data[i].message}
+            else:
+                break
+        return data
+
+    @LoginRequired()
+    @NotAnonymous()
+    @HasRepoPermissionAnyDecorator('repository.admin')
+    @view_config(
+        route_name='strip_execute', request_method='POST',
+        renderer='json', xhr=True
+    )
+    def strip_execute(self):
+
+        from rhodecode.model.scm import ScmModel
+        from rhodecode.lib.ext_json import json
+
+        c = self.load_default_context()
+        user = self._rhodecode_user
+        rp = self.request.POST
+        data = {}
+        for idx in rp:
+            commit = json.loads(rp[idx])
+            #If someone put two times the same branch
+            if commit['branch'] in data.keys():
+                continue
+            try:
+                ScmModel().strip(repo=c.repo_info,
+                          commit_id=commit['rev'], branch=commit['branch'])
+                log.info('Stripped commit %s from repo `%s` by %s' % (commit['rev'], c.repo_info.repo_name, user))
+                data[commit['rev']] = True
+            except Exception, e:
+                data[commit['rev']] = False
+                log.debug('Stripped commit %s from repo `%s` failed by %s, exeption %s' % (commit['rev'],
+                                                                            c.repo_info.repo_name, user, e.message))
+        return data

rhodecode/apps/user_profile/__init__.py

@@ -0,0 +1,28 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2016-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+
+def includeme(config):
+    config.add_route(
+        name='user_profile',
+        pattern='/_profiles/{username}')
+
+    # Scan module for configuration decorators.
+    config.scan()

rhodecode/apps/user_profile/tests/__init__.py

@@ -0,0 +1,19 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2016-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/

rhodecode/apps/user_profile/tests/test_users.py

@@ -0,0 +1,75 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2010-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+import pytest
+
+from rhodecode.model.db import User
+from rhodecode.tests import (
+    TestController, TEST_USER_ADMIN_LOGIN, TEST_USER_ADMIN_PASS,
+    TEST_USER_REGULAR_LOGIN, TEST_USER_REGULAR_PASS)
+from rhodecode.tests.fixture import Fixture
+from rhodecode.tests.utils import AssertResponse
+
+fixture = Fixture()
+
+
+def route_path(name, **kwargs):
+    return '/_profiles/{username}'.format(**kwargs)
+
+
+class TestUsersController(TestController):
+
+    def test_user_profile(self, user_util):
+        edit_link_css = '.user-profile .panel-edit'
+        self.log_user(TEST_USER_REGULAR_LOGIN, TEST_USER_REGULAR_PASS)
+        user = user_util.create_user(
+            'test-my-user', password='qweqwe', email='[e-mail]')
+        username = user.username
+
+        response = self.app.get(route_path('user_profile', username=username))
+        response.mustcontain('testme')
+        response.mustcontain('[e-mail]')
+        assert_response = AssertResponse(response)
+        assert_response.no_element_exists(edit_link_css)
+
+        # edit should be available to superadmin users
+        self.logout_user()
+        self.log_user(TEST_USER_ADMIN_LOGIN, TEST_USER_ADMIN_PASS)
+        response = self.app.get(route_path('user_profile', username=username))
+        assert_response = AssertResponse(response)
+        assert_response.element_contains(edit_link_css, 'Edit')
+
+    def test_user_profile_not_available(self, user_util):
+        user = user_util.create_user()
+        username = user.username
+
+        # not logged in, redirect
+        self.app.get(route_path('user_profile', username=username), status=302)
+
+        self.log_user()
+        # after log-in show
+        self.app.get(route_path('user_profile', username=username), status=200)
+
+        # default user, not allowed to show it
+        self.app.get(
+            route_path('user_profile', username=User.DEFAULT_USER), status=404)
+
+        # actual 404
+        self.app.get(route_path('user_profile', username='unknown'), status=404)

rhodecode/apps/user_profile/views.py

@@ -0,0 +1,53 b''
+# -*- coding: utf-8 -*-
+
+# Copyright (C) 2016-2017 RhodeCode GmbH
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License, version 3
+# (only), as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# This program is dual-licensed. If you wish to learn more about the
+# RhodeCode Enterprise Edition, including its added features, Support services,
+# and proprietary license terms, please see https://rhodecode.com/licenses/
+
+import logging
+
+from pyramid.httpexceptions import HTTPNotFound
+from pyramid.view import view_config
+
+from rhodecode.apps._base import BaseAppView
+from rhodecode.lib.auth import LoginRequired, NotAnonymous
+
+from rhodecode.model.db import User
+from rhodecode.model.user import UserModel
+
+log = logging.getLogger(__name__)
+
+
+class UserProfileView(BaseAppView):
+
+    @LoginRequired()
+    @NotAnonymous()
+    @view_config(
+        route_name='user_profile', request_method='GET',
+        renderer='rhodecode:templates/users/user.mako')
+    def login(self):
+        # register local template context
+        c = self._get_local_tmpl_context()
+        c.active = 'user_profile'
+
+        username = self.request.matchdict.get('username')
+
+        c.user = UserModel().get_by_username(username)
+        if not c.user or c.user.username == User.DEFAULT_USER:
+            raise HTTPNotFound()
+
+        return self._get_template_context(c)

.bumpversion.cfg

@@ -1,5 +1,5 b''
 [bumpversion]
-current_version = 4.6.1
+current_version = 4.7.0
 message = release: Bump version {current_version} to {new_version}
 
 [bumpversion:file:rhodecode/VERSION]

.coveragerc

@@ -10,6 +10,8 b' include ='
 omit =
     rhodecode/lib/vcs/remote/*
     rhodecode/lib/dbmigrate/*
+    rhodecode/lib/paster_commands/*
+    rhodecode/tests/*
 
 [report]
 

.release.cfg

@@ -4,26 +4,21 b' done = false'
 [task:bump_version]
 done = true
 
-[task:rc_tools_pinned]
-done = true
-
 [task:fixes_on_stable]
-done = true
 
 [task:pip2nix_generated]
-done = true
 
 [task:changelog_updated]
-done = true
 
 [task:generate_api_docs]
-done = true
+
+[task:updated_translation]
 
 [release]
-state = prepared
-version = 4.6.1
+state = in_progress
+version = 4.7.0
 
-[task:updated_translation]
+[task:rc_tools_pinned]
 
 [task:generate_js_routes]
 

MANIFEST.in

@@ -1,5 +1,5 b''
 # top level files
-include test.ini
+
 include MANIFEST.in
 include README.rst
 include CHANGES.rst

configs/development.ini

@@ -539,18 +539,15 b' vcs.server = localhost:9900'
 
 ## Web server connectivity protocol, responsible for web based VCS operatations
 ## Available protocols are:
-## `pyro4` - use pyro4 server
 ## `http` - use http-rpc backend (default)
 vcs.server.protocol = http
 
 ## Push/Pull operations protocol, available options are:
-## `pyro4` - use pyro4 server
 ## `http` - use http-rpc backend (default)
 ##
 vcs.scm_app_implementation = http
 
 ## Push/Pull operations hooks protocol, available options are:
-## `pyro4` - use pyro4 server
 ## `http` - use http-rpc backend (default)
 vcs.hooks.protocol = http
 
@@ -666,7 +663,7 b' formatter = color_formatter_sql'
 ################
 
 [formatter_generic]
-class = rhodecode.lib.logging_formatter.Pyro4AwareFormatter
+class = rhodecode.lib.logging_formatter.ExceptionAwareFormatter
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 

configs/gunicorn_config.py

@@ -36,7 +36,7 b' tmp_upload_dir = None'
 
 # Custom log format
 access_log_format = (
-    '%(t)s GNCRN %(p)-8s %(h)-15s rqt:%(L)s %(s)s %(b)s "%(m)s:%(U)s %(q)s" usr:%(u)s "%(f)s" "%(a)s"')
+    '%(t)s GNCRN %(p)-8s %(h)-15s rqt:%(L)s %(s)s %(b)-6s "%(m)s:%(U)s %(q)s" usr:%(u)s "%(f)s" "%(a)s"')
 
 # self adjust workers based on CPU count
 # workers = multiprocessing.cpu_count() * 2 + 1

configs/production.ini

@@ -508,18 +508,15 b' vcs.server = localhost:9900'
 
 ## Web server connectivity protocol, responsible for web based VCS operatations
 ## Available protocols are:
-## `pyro4` - use pyro4 server
 ## `http` - use http-rpc backend (default)
 vcs.server.protocol = http
 
 ## Push/Pull operations protocol, available options are:
-## `pyro4` - use pyro4 server
 ## `http` - use http-rpc backend (default)
 ##
 vcs.scm_app_implementation = http
 
 ## Push/Pull operations hooks protocol, available options are:
-## `pyro4` - use pyro4 server
 ## `http` - use http-rpc backend (default)
 vcs.hooks.protocol = http
 
@@ -635,7 +632,7 b' formatter = generic'
 ################
 
 [formatter_generic]
-class = rhodecode.lib.logging_formatter.Pyro4AwareFormatter
+class = rhodecode.lib.logging_formatter.ExceptionAwareFormatter
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 

default.nix

@@ -124,6 +124,7 b' let'
       name = "rhodecode-enterprise-ce-${version}";
       releaseName = "RhodeCodeEnterpriseCE-${version}";
       src = rhodecode-enterprise-ce-src;
+      dontStrip = true; # prevent strip, we don't need it.
 
       buildInputs =
         attrs.buildInputs ++
@@ -225,8 +226,8 b' let'
   rhodecode-testdata-src = sources.rhodecode-testdata or (
     pkgs.fetchhg {
       url = "https://code.rhodecode.com/upstream/rc_testdata";
-      rev = "v0.9.0";
-      sha256 = "0k0ccb7cncd6mmzwckfbr6l7fsymcympwcm948qc3i0f0m6bbg1y";
+      rev = "v0.10.0";
+      sha256 = "0zn9swwvx4vgw4qn8q3ri26vvzgrxn15x6xnjrysi1bwmz01qjl0";
   });
 
   # Apply all overrides and fix the final package set

docs-internal/vcs/index.rst

@@ -17,7 +17,6 b' implemented in :mod:`rhodecode.lib.middl'
 .. toctree::
    :maxdepth: 2
 
-   http-transition
    middleware
    vcsserver
    subversion

docs/admin/apache-conf-example.rst

@@ -59,7 +59,7 b' Below config if for an Apache Reverse Pr'
 
         # Url to running RhodeCode instance. This is shown as `- URL:` when
         # running rccontrol status.
-        ProxyPass / http://127.0.0.1:10002/
+        ProxyPass / http://127.0.0.1:10002/ timeout=7200 Keepalive=On
         ProxyPassReverse / http://127.0.0.1:10002/
 
         # strict http prevents from https -> http downgrade

docs/admin/enable-debug.rst

@@ -47,7 +47,7 b' the ``debug`` level.'
     ### LOGGING CONFIGURATION   ####
     ################################
     [loggers]
-    keys = root, routes, rhodecode, sqlalchemy, beaker, pyro4, templates
+    keys = root, routes, rhodecode, sqlalchemy, beaker, templates
 
     [handlers]
     keys = console, console_sql, file, file_rotating
@@ -75,12 +75,6 b' the ``debug`` level.'
     qualname = beaker.container
     propagate = 1
 
-    [logger_pyro4]
-    level = DEBUG
-    handlers =
-    qualname = Pyro4
-    propagate = 1
-
     [logger_templates]
     level = INFO
     handlers =

docs/admin/svn-http.rst

@@ -114,8 +114,20 b' 4. Add the following configuration optio'
 
 This would create a special template file called ```mod_dav_svn.conf```. We
 used that file path in the apache config above inside the Include statement.
-It's also possible to generate the config from the
-:menuselection:`Admin --> Settings --> VCS` page.
+It's also possible to manually generate the config from the
+:menuselection:`Admin --> Settings --> VCS` page by clicking a
+`Generate Apache Config` button.
+
+5. Now only things left is to enable svn support, and generate the initial
+   configuration.
+
+   - Select `Proxy subversion HTTP requests` checkbox
+   - Enter http://localhost:8090 into `Subversion HTTP Server URL`
+   - Click the `Generate Apache Config` button.
+
+This config will be automatically re-generated once an user-groups is added
+to properly map the additional paths generated.
+
 
 
 Using |svn|

docs/admin/tuning-change-large-file-dir.rst

@@ -7,26 +7,8 b' Change the |hg| Large Files Location'
 :file:`/home/{user}/repos/.cache/largefiles`. If you wish to change this, use
 the following steps:
 
-1. Open ishell from the terminal and use it to log into the |RCE| database by
-   specifying the instance :file:`rhodecode.ini` file.
-
-.. code-block:: bash
-
-    # Open iShell from the terminal and set ini file
-    $ rccontrol ishell enterprise-1
-
-2. Run the following commands, and ensure that |RCE| has write access to the
-   new directory:
+1. Open :menuselection:`Admin --> Settings --> VCS` as super-admin.
 
-.. code-block:: bash
+In section called `Mercurial Settings` you can change where the largefiles
+objects should be stored.
 
-    # Once logged into the database, use SQL to redirect
-    # the large files location
-    In [1]: from rhodecode.model.settings import SettingsModel
-    In [2]: SettingsModel().get_ui_by_key('usercache')
-    Out[2]: <RhodeCodeUi[largefiles]usercache=>/mnt/hgfs/shared/workspace/xxxx/.cache/largefiles]>
-
-    In [3]: largefiles_cache = SettingsModel().get_ui_by_key('usercache')
-    In [4]: largefiles_cache.ui_value = '/new/path’
-    In [5]: Session().add(largefiles_cache);Session().commit()
-

docs/admin/tuning-rhodecode.rst

@@ -17,5 +17,6 b' may find some of the following methods u'
    tuning-mount-cache-memory
    tuning-change-encoding
    tuning-change-large-file-dir
+   tuning-change-lfs-dir
    tuning-hg-auth-loop
 

docs/admin/vcs-server.rst

@@ -232,7 +232,7 b' For a more detailed explanation of the l'
     ### LOGGING CONFIGURATION   ####
     ################################
     [loggers]
-    keys = root, vcsserver, pyro4, beaker
+    keys = root, vcsserver, beaker
 
     [handlers]
     keys = console
@@ -259,12 +259,6 b' For a more detailed explanation of the l'
     qualname = beaker
     propagate = 1
 
-    [logger_pyro4]
-    level = DEBUG
-    handlers =
-    qualname = Pyro4
-    propagate = 1
-
 
     ##############
     ## HANDLERS ##

docs/api/api.rst

@@ -75,7 +75,7 b' Example call for auto pulling from remot'
 .. code-block:: bash
 
     curl https://server.com/_admin/api -X POST -H 'content-type:text/plain' --data-binary '{"id":1,
-    "auth_token":"xe7cdb2v278e4evbdf5vs04v832v0efvcbcve4a3","method":"pull", "args":{"repo":"CPython"}}'
+    "auth_token":"xe7cdb2v278e4evbdf5vs04v832v0efvcbcve4a3","method":"pull", "args":{"repoid":"CPython"}}'
 
 Provide those parameters:
  - **id** A value of any type, which is used to match the response with the

docs/api/methods/pull-request-methods.rst

@@ -24,13 +24,12 b' close_pull_request'
 
    .. code-block:: bash
 
-     "id": <id_given_in_input>,
-     "result":
-       {
+       "id": <id_given_in_input>,
+       "result": {
            "pull_request_id":  "<int>",
            "closed":           "<bool>"
        },
-     "error": null
+       "error": null
 
 
 comment_pull_request 
@@ -67,15 +66,14 b' comment_pull_request'
 
    .. code-block:: bash
 
-     id : <id_given_in_input>
-     result :
-       {
+       id : <id_given_in_input>
+       result : {
            "pull_request_id":  "<Integer>",
            "comment_id":       "<Integer>",
            "status": {"given": <given_status>,
                       "was_changed": <bool status_was_actually_changed> },
-       }
-     error :  null
+       },
+       error :  null
 
 
 create_pull_request 
@@ -109,9 +107,8 b' create_pull_request'
    :param reviewers: Set the new pull request reviewers list.
    :type reviewers: Optional(list)
        Accepts username strings or objects of the format:
-       {
-           'username': 'nick', 'reasons': ['original author']
-       }
+
+           {'username': 'nick', 'reasons': ['original author']}
 
 
 get_pull_request 
@@ -305,9 +302,8 b' merge_pull_request'
 
    .. code-block:: bash
 
-     "id": <id_given_in_input>,
-     "result":
-       {
+       "id": <id_given_in_input>,
+       "result": {
            "executed":         "<bool>",
            "failure_reason":   "<int>",
            "merge_commit_id":  "<merge_commit_id>",
@@ -318,7 +314,7 b' merge_pull_request'
                                    "name":      "<name>"
                                }
        },
-     "error": null
+       "error": null
 
 
 update_pull_request 
@@ -349,9 +345,8 b' update_pull_request'
 
    .. code-block:: bash
 
-     id : <id_given_in_input>
-     result :
-       {
+       id : <id_given_in_input>
+       result : {
            "msg": "Updated pull request `63`",
            "pull_request": <pull_request_object>,
            "updated_reviewers": {
@@ -371,6 +366,6 b' update_pull_request'
              "removed": []
            }
        }
-     error :  null
+       error :  null
 
 

docs/api/methods/server-methods.rst

@@ -85,6 +85,58 b' get_ip'
        }
 
 
+get_method 
+----------
+
+.. py:function:: get_method(apiuser, pattern=<Optional:'*'>)
+
+   Returns list of all available API methods. By default match pattern
+   os "*" but any other pattern can be specified. eg *comment* will return
+   all methods with comment inside them. If just single method is matched
+   returned data will also include method specification
+
+   This command can only be run using an |authtoken| with admin rights to
+   the specified repository.
+
+   This command takes the following options:
+
+   :param apiuser: This is filled automatically from the |authtoken|.
+   :type apiuser: AuthUser
+   :param pattern: pattern to match method names against
+   :type older_then: Optional("*")
+
+   Example output:
+
+   .. code-block:: bash
+
+     id : <id_given_in_input>
+     "result": [
+       "changeset_comment",
+       "comment_pull_request",
+       "comment_commit"
+     ]
+     error :  null
+
+   .. code-block:: bash
+
+     id : <id_given_in_input>
+     "result": [
+       "comment_commit",
+       {
+         "apiuser": "<RequiredType>",
+         "comment_type": "<Optional:u'note'>",
+         "commit_id": "<RequiredType>",
+         "message": "<RequiredType>",
+         "repoid": "<RequiredType>",
+         "request": "<RequiredType>",
+         "resolves_comment_id": "<Optional:None>",
+         "status": "<Optional:None>",
+         "userid": "<Optional:<OptionalAttr:apiuser>>"
+       }
+     ]
+     error :  null
+
+
 get_server_info 
 ---------------
 

docs/api/methods/user-methods.rst

@@ -41,15 +41,16 b' create_user'
    :type force_password_change: Optional(``True`` | ``False``)
    :param create_personal_repo_group: Create personal repo group for this user
    :type create_personal_repo_group: Optional(``True`` | ``False``)
+
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
-                 "msg" : "created new user `<username>`",
-                 "user": <user_obj>
-               }
+           "msg" : "created new user `<username>`",
+           "user": <user_obj>
+       }
        error:  null
 
    Example error output:
@@ -98,9 +99,9 b' delete_user'
 
        id : <id_given_in_input>
        result: {
-                 "msg" : "deleted user ID:<userid> <username>",
-                 "user": null
-               }
+           "msg" : "deleted user ID:<userid> <username>",
+           "user": null
+       }
        error:  null
 
    Example error output:
@@ -145,8 +146,8 b' get_user'
          "result": {
            "active": true,
            "admin": false,
-           "api_key": "api-key",
            "api_keys": [ list of keys ],
+           "auth_tokens": [ list of tokens with details ],
            "email": "[e-mail]",
            "emails": [
              "[e-mail]"
@@ -157,6 +158,7 b' get_user'
            "ip_addresses": [],
            "language": null,
            "last_login": "Timestamp",
+           "last_activity": "Timestamp",
            "lastname": "surnae",
            "permissions": {
              "global": [
@@ -183,6 +185,32 b' get_user'
        }
 
 
+get_user_audit_logs 
+-------------------
+
+.. py:function:: get_user_audit_logs(apiuser, userid=<Optional:<OptionalAttr:apiuser>>)
+
+   Fetches all action logs made by the specified user.
+
+   This command takes the following options:
+
+   :param apiuser: This is filled automatically from the |authtoken|.
+   :type apiuser: AuthUser
+   :param userid: Sets the userid whose list of locked |repos| will be
+       displayed.
+   :type userid: Optional(str or int)
+
+   Example output:
+
+   .. code-block:: bash
+
+       id : <id_given_in_input>
+       result : {
+           [action, action,...]
+       }
+       error :  null
+
+
 get_user_locks 
 --------------
 
@@ -232,7 +260,7 b' get_users'
    .. code-block:: bash
 
        id : <id_given_in_input>
-           result: [<user_object>, ...]
+       result: [<user_object>, ...]
        error:  null
 
 
@@ -279,9 +307,9 b' update_user'
 
        id : <id_given_in_input>
        result: {
-                 "msg" : "updated user ID:<userid> <username>",
-                 "user": <user_object>,
-               }
+           "msg" : "updated user ID:<userid> <username>",
+           "user": <user_object>,
+       }
        error:  null
 
    Example error output:

docs/auth/ldap-active-directory.rst

@@ -19,4 +19,11 b' authentication.'
     # Set the Active Directory user surname
     Last Name Attribute  = user_surname
     # Set the Active Directory user email
-    E-mail Attribute     = userEmail
  No newline at end of file
+    E-mail Attribute     = userEmail
+
+
+Below is example setup that can be used with Active Directory and ldap groups.
+
+.. image:: ../images/ldap-groups-example.png
+   :alt: LDAP/AD setup example
+   :scale: 50 %
  No newline at end of file

docs/auth/ldap-authentication.rst

@@ -104,4 +104,10 b' The following are optional when enabling'
    following directory: `/etc/openldap/cacerts`
 
 
+Below is example setup that can be used with Active Directory and ldap groups.
+
+.. image:: ../images/ldap-groups-example.png
+   :alt: LDAP/AD setup example
+   :scale: 50 %
+
 .. _RFC 2254: http://www.rfc-base.org/rfc-2254.html
  No newline at end of file

docs/contributing/dev-setup.rst

@@ -114,6 +114,30 b' following command from inside the cloned'
    fine on both MacOS and Linux platforms.
 
 
+Create config.nix for development
+---------------------------------
+
+In order to run proper tests and setup linking across projects, a config.nix
+file needs to be setup::
+
+    # create config
+    mkdir -p ~/.nixpkgs
+    touch ~/.nixpkgs/config.nix
+
+    # put the below content into the ~/.nixpkgs/config.nix file
+    # adjusts, the path to where you cloned your repositories.
+
+    {
+      rc = {
+       sources = {
+        rhodecode-vcsserver = "/home/dev/rhodecode-vcsserver";
+        rhodecode-enterprise-ce = "/home/dev/rhodecode-enterprise-ce";
+        rhodecode-enterprise-ee = "/home/dev/rhodecode-enterprise-ee";
+       };
+      };
+    }
+
+
 
 Creating a Development Configuration
 ------------------------------------

docs/release-notes/release-notes-4.5.0.rst

@@ -21,7 +21,7 b' New Features'
 - Pull request reviewers (EE only): added new default reviewers functionality.
   Allows picking users or user groups defined as reviewers for new pull request.
   Picking reviewers can be based on branch name, changed file name patterns or
-  original author of changed source code. eg *.css -> design team.
+  original author of changed source code. eg \*.css -> design team.
   Master branch -> repo owner, fixes #1131.
 - Pull request reviewers: store and show reasons why given person is a reviewer.
   Manually adding reviewers after creating a PR will now be also indicated
@@ -85,7 +85,7 b' General'
   input for text box.
 - Api: WARNING DEPRECATION, refactor repository group schemas. Fixes #4133.
   When using create_repo, create_repo_group, update_repo, update_repo_group
-  the *_name parameter now takes full path including sub repository groups.
+  the \*_name parameter now takes full path including sub repository groups.
   This is the only way to add resource under another repository group.
   Furthermore giving non-existing path will no longer create the missing
   structure. This change makes the api more consistent, it better validates

docs/release-notes/release-notes.rst

@@ -9,6 +9,7 b' Release Notes'
 .. toctree::
    :maxdepth: 1
 
+   release-notes-4.7.0.rst
    release-notes-4.6.1.rst
    release-notes-4.6.0.rst
    release-notes-4.5.2.rst

docs/tutorials/hg-large-ext.rst

@@ -9,6 +9,36 b' has a large files extension which tracks'
 means that the large files are only downloaded when they are needed as part
 of the current revision. This saves both disk space and bandwidth.
 
+
+Enabling HG Largefiles
+++++++++++++++++++++++
+
+Mercurial Largefiles extension is disabled by default within |RC| Server.
+
+To enable Mercurial Largefiles Globally:
+
+    - Go to :menuselection:`Admin --> Settings --> VCS`
+
+    - Scroll down into `Mercurial settings`
+
+    - Tick `Enable largefiles extension`
+
+    - Save your settings.
+
+Those settings apply globally to each repository that inherits from the defaults
+You can leave `largefiles extension` disabled globally, and only enable it per
+repository that would use the largefiles.
+
+
+.. note::
+
+    You might want to adjust the global storage location at that point, however
+    we recommend leaving the default one created.
+
+
+Installing and using the |hg| Largefiles
+++++++++++++++++++++++++++++++++++++++++
+
 To find out more, see the |hg| `Large Files Extensions Documentation`_.
 
 To configure the large files extension, you need to set up your

docs/tutorials/tutorials.rst

@@ -12,6 +12,7 b' then please send a request to support@rh'
 
    deploy-from-host
    hg-large-ext
+   git-lfs-ext
    multi-instance-setup
    scaling-best-practices
    squash-commits

pkgs/python-packages-overrides.nix

@@ -48,12 +48,27 b' self: super: {'
     ];
   });
 
+  nbconvert = super.nbconvert.override (attrs: {
+    propagatedBuildInputs = attrs.propagatedBuildInputs ++ [
+      # marcink: plug in jupyter-client for notebook rendering
+      self.jupyter-client
+    ];
+  });
+
   ipython = super.ipython.override (attrs: {
     propagatedBuildInputs = attrs.propagatedBuildInputs ++ [
       self.gnureadline
     ];
   });
 
+  celery = super.celery.override (attrs: {
+    # The current version of kombu needs some patching to work with the
+    # other libs. Should be removed once we update celery and kombu.
+    patches = [
+      ./patch-celery-dateutil.diff
+    ];
+  });
+
   kombu = super.kombu.override (attrs: {
     # The current version of kombu needs some patching to work with the
     # other libs. Should be removed once we update celery and kombu.

pkgs/python-packages.nix

@@ -197,19 +197,6 b''
       license = [ pkgs.lib.licenses.bsdOriginal ];
     };
   };
-  Pyro4 = super.buildPythonPackage {
-    name = "Pyro4-4.41";
-    buildInputs = with self; [];
-    doCheck = false;
-    propagatedBuildInputs = with self; [serpent];
-    src = fetchurl {
-      url = "https://pypi.python.org/packages/56/2b/89b566b4bf3e7f8ba790db2d1223852f8cb454c52cab7693dd41f608ca2a/Pyro4-4.41.tar.gz";
-      md5 = "ed69e9bfafa9c06c049a87cb0c4c2b6c";
-    };
-    meta = {
-      license = [ pkgs.lib.licenses.mit ];
-    };
-  };
   Routes = super.buildPythonPackage {
     name = "Routes-1.13";
     buildInputs = with self; [];
@@ -444,6 +431,19 b''
       license = [ pkgs.lib.licenses.mit ];
     };
   };
+  bleach = super.buildPythonPackage {
+    name = "bleach-1.5.0";
+    buildInputs = with self; [];
+    doCheck = false;
+    propagatedBuildInputs = with self; [six html5lib];
+    src = fetchurl {
+      url = "https://pypi.python.org/packages/99/00/25a8fce4de102bf6e3cc76bc4ea60685b2fee33bde1b34830c70cacc26a7/bleach-1.5.0.tar.gz";
+      md5 = "b663300efdf421b3b727b19d7be9c7e7";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.asl20 ];
+    };
+  };
   bottle = super.buildPythonPackage {
     name = "bottle-0.12.8";
     buildInputs = with self; [];
@@ -535,6 +535,19 b''
       license = [ pkgs.lib.licenses.bsdOriginal ];
     };
   };
+  configparser = super.buildPythonPackage {
+    name = "configparser-3.5.0";
+    buildInputs = with self; [];
+    doCheck = false;
+    propagatedBuildInputs = with self; [];
+    src = fetchurl {
+      url = "https://pypi.python.org/packages/7c/69/c2ce7e91c89dc073eb1aa74c0621c3eefbffe8216b3f9af9d3885265c01c/configparser-3.5.0.tar.gz";
+      md5 = "cfdd915a5b7a6c09917a64a573140538";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.mit ];
+    };
+  };
   cov-core = super.buildPythonPackage {
     name = "cov-core-1.15.0";
     buildInputs = with self; [];
@@ -562,29 +575,29 b''
     };
   };
   cssselect = super.buildPythonPackage {
-    name = "cssselect-0.9.1";
+    name = "cssselect-1.0.1";
     buildInputs = with self; [];
     doCheck = false;
     propagatedBuildInputs = with self; [];
     src = fetchurl {
-      url = "https://pypi.python.org/packages/aa/e5/9ee1460d485b94a6d55732eb7ad5b6c084caf73dd6f9cb0bb7d2a78fafe8/cssselect-0.9.1.tar.gz";
-      md5 = "c74f45966277dc7a0f768b9b0f3522ac";
+      url = "https://pypi.python.org/packages/77/ff/9c865275cd19290feba56344eba570e719efb7ca5b34d67ed12b22ebbb0d/cssselect-1.0.1.tar.gz";
+      md5 = "3fa03bf82a9f0b1223c0f1eb1369e139";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal ];
     };
   };
   decorator = super.buildPythonPackage {
-    name = "decorator-3.4.2";
+    name = "decorator-4.0.11";
     buildInputs = with self; [];
     doCheck = false;
     propagatedBuildInputs = with self; [];
     src = fetchurl {
-      url = "https://pypi.python.org/packages/35/3a/42566eb7a2cbac774399871af04e11d7ae3fc2579e7dae85213b8d1d1c57/decorator-3.4.2.tar.gz";
-      md5 = "9e0536870d2b83ae27d58dbf22582f4d";
+      url = "https://pypi.python.org/packages/cc/ac/5a16f1fc0506ff72fcc8fd4e858e3a1c231f224ab79bb7c4c9b2094cc570/decorator-4.0.11.tar.gz";
+      md5 = "73644c8f0bd4983d1b6a34b49adec0ae";
     };
     meta = {
-      license = [ pkgs.lib.licenses.bsdOriginal ];
+      license = [ pkgs.lib.licenses.bsdOriginal { fullName = "new BSD License"; } ];
     };
   };
   deform = super.buildPythonPackage {
@@ -678,6 +691,19 b''
       license = [ pkgs.lib.licenses.asl20 ];
     };
   };
+  entrypoints = super.buildPythonPackage {
+    name = "entrypoints-0.2.2";
+    buildInputs = with self; [];
+    doCheck = false;
+    propagatedBuildInputs = with self; [configparser];
+    src = fetchurl {
+      url = "https://code.rhodecode.com/upstream/entrypoints/archive/96e6d645684e1af3d7df5b5272f3fe85a546b233.tar.gz?md5=7db37771aea9ac9fefe093e5d6987313";
+      md5 = "7db37771aea9ac9fefe093e5d6987313";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.mit ];
+    };
+  };
   enum34 = super.buildPythonPackage {
     name = "enum34-1.1.6";
     buildInputs = with self; [];
@@ -691,6 +717,19 b''
       license = [ pkgs.lib.licenses.bsdOriginal ];
     };
   };
+  functools32 = super.buildPythonPackage {
+    name = "functools32-3.2.3.post2";
+    buildInputs = with self; [];
+    doCheck = false;
+    propagatedBuildInputs = with self; [];
+    src = fetchurl {
+      url = "https://pypi.python.org/packages/5e/1a/0aa2c8195a204a9f51284018562dea77e25511f02fe924fac202fc012172/functools32-3.2.3-2.zip";
+      md5 = "d55232eb132ec779e6893c902a0bc5ad";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.psfl ];
+    };
+  };
   future = super.buildPythonPackage {
     name = "future-0.14.3";
     buildInputs = with self; [];
@@ -782,6 +821,19 b''
       license = [ pkgs.lib.licenses.mit ];
     };
   };
+  html5lib = super.buildPythonPackage {
+    name = "html5lib-0.9999999";
+    buildInputs = with self; [];
+    doCheck = false;
+    propagatedBuildInputs = with self; [six];
+    src = fetchurl {
+      url = "https://pypi.python.org/packages/ae/ae/bcb60402c60932b32dfaf19bb53870b29eda2cd17551ba5639219fb5ebf9/html5lib-0.9999999.tar.gz";
+      md5 = "ef43cb05e9e799f25d65d1135838a96f";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.mit ];
+    };
+  };
   infrae.cache = super.buildPythonPackage {
     name = "infrae.cache-1.0.1";
     buildInputs = with self; [];
@@ -873,6 +925,45 b''
       license = [ pkgs.lib.licenses.bsdOriginal ];
     };
   };
+  jsonschema = super.buildPythonPackage {
+    name = "jsonschema-2.6.0";
+    buildInputs = with self; [];
+    doCheck = false;
+    propagatedBuildInputs = with self; [functools32];
+    src = fetchurl {
+      url = "https://pypi.python.org/packages/58/b9/171dbb07e18c6346090a37f03c7e74410a1a56123f847efed59af260a298/jsonschema-2.6.0.tar.gz";
+      md5 = "50c6b69a373a8b55ff1e0ec6e78f13f4";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.mit ];
+    };
+  };
+  jupyter-client = super.buildPythonPackage {
+    name = "jupyter-client-5.0.0";
+    buildInputs = with self; [];
+    doCheck = false;
+    propagatedBuildInputs = with self; [traitlets jupyter-core pyzmq python-dateutil];
+    src = fetchurl {
+      url = "https://pypi.python.org/packages/e5/6f/65412ed462202b90134b7e761b0b7e7f949e07a549c1755475333727b3d0/jupyter_client-5.0.0.tar.gz";
+      md5 = "1acd331b5c9fb4d79dae9939e79f2426";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.bsdOriginal ];
+    };
+  };
+  jupyter-core = super.buildPythonPackage {
+    name = "jupyter-core-4.3.0";
+    buildInputs = with self; [];
+    doCheck = false;
+    propagatedBuildInputs = with self; [traitlets];
+    src = fetchurl {
+      url = "https://pypi.python.org/packages/2f/39/5138f975100ce14d150938df48a83cd852a3fd8e24b1244f4113848e69e2/jupyter_core-4.3.0.tar.gz";
+      md5 = "18819511a809afdeed9a995a9c27bcfb";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.bsdOriginal ];
+    };
+  };
   kombu = super.buildPythonPackage {
     name = "kombu-1.5.1";
     buildInputs = with self; [];
@@ -887,13 +978,13 b''
     };
   };
   lxml = super.buildPythonPackage {
-    name = "lxml-3.4.4";
+    name = "lxml-3.7.3";
     buildInputs = with self; [];
     doCheck = false;
     propagatedBuildInputs = with self; [];
     src = fetchurl {
-      url = "https://pypi.python.org/packages/63/c7/4f2a2a4ad6c6fa99b14be6b3c1cece9142e2d915aa7c43c908677afc8fa4/lxml-3.4.4.tar.gz";
-      md5 = "a9a65972afc173ec7a39c585f4eea69c";
+      url = "https://pypi.python.org/packages/39/e8/a8e0b1fa65dd021d48fe21464f71783655f39a41f218293c1c590d54eb82/lxml-3.7.3.tar.gz";
+      md5 = "075692ce442e69bbd604d44e21c02753";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal ];
@@ -912,6 +1003,19 b''
       license = [ { fullName = "BSD-derived (http://www.repoze.org/LICENSE.txt)"; } ];
     };
   };
+  mistune = super.buildPythonPackage {
+    name = "mistune-0.7.3";
+    buildInputs = with self; [];
+    doCheck = false;
+    propagatedBuildInputs = with self; [];
+    src = fetchurl {
+      url = "https://pypi.python.org/packages/88/1e/be99791262b3a794332fda598a07c2749a433b9378586361ba9d8e824607/mistune-0.7.3.tar.gz";
+      md5 = "4eba50bd121b83716fa4be6a4049004b";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.bsdOriginal ];
+    };
+  };
   mock = super.buildPythonPackage {
     name = "mock-1.0.1";
     buildInputs = with self; [];
@@ -938,6 +1042,32 b''
       license = [ pkgs.lib.licenses.asl20 ];
     };
   };
+  nbconvert = super.buildPythonPackage {
+    name = "nbconvert-5.1.1";
+    buildInputs = with self; [];
+    doCheck = false;
+    propagatedBuildInputs = with self; [mistune Jinja2 Pygments traitlets jupyter-core nbformat entrypoints bleach pandocfilters testpath];
+    src = fetchurl {
+      url = "https://pypi.python.org/packages/95/58/df1c91f1658ee5df19097f915a1e71c91fc824a708d82d2b2e35f8b80e9a/nbconvert-5.1.1.tar.gz";
+      md5 = "d0263fb03a44db2f94eea09a608ed813";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.bsdOriginal ];
+    };
+  };
+  nbformat = super.buildPythonPackage {
+    name = "nbformat-4.3.0";
+    buildInputs = with self; [];
+    doCheck = false;
+    propagatedBuildInputs = with self; [ipython-genutils traitlets jsonschema jupyter-core];
+    src = fetchurl {
+      url = "https://pypi.python.org/packages/f9/c5/89df4abf906f766727f976e170caa85b4f1c1d1feb1f45d716016e68e19f/nbformat-4.3.0.tar.gz";
+      md5 = "9a00d20425914cd5ba5f97769d9963ca";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.bsdOriginal ];
+    };
+  };
   nose = super.buildPythonPackage {
     name = "nose-1.3.6";
     buildInputs = with self; [];
@@ -977,6 +1107,19 b''
       license = [ pkgs.lib.licenses.asl20 ];
     };
   };
+  pandocfilters = super.buildPythonPackage {
+    name = "pandocfilters-1.4.1";
+    buildInputs = with self; [];
+    doCheck = false;
+    propagatedBuildInputs = with self; [];
+    src = fetchurl {
+      url = "https://pypi.python.org/packages/e3/1f/21d1b7e8ca571e80b796c758d361fdf5554335ff138158654684bc5401d8/pandocfilters-1.4.1.tar.gz";
+      md5 = "7680d9f9ec07397dd17f380ee3818b9d";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.bsdOriginal ];
+    };
+  };
   paramiko = super.buildPythonPackage {
     name = "paramiko-1.15.1";
     buildInputs = with self; [];
@@ -1043,13 +1186,13 b''
     };
   };
   prompt-toolkit = super.buildPythonPackage {
-    name = "prompt-toolkit-1.0.9";
+    name = "prompt-toolkit-1.0.13";
     buildInputs = with self; [];
     doCheck = false;
     propagatedBuildInputs = with self; [six wcwidth];
     src = fetchurl {
-      url = "https://pypi.python.org/packages/83/14/5ac258da6c530eca02852ee25c7a9ff3ca78287bb4c198d0d0055845d856/prompt_toolkit-1.0.9.tar.gz";
-      md5 = "a39f91a54308fb7446b1a421c11f227c";
+      url = "https://pypi.python.org/packages/23/be/4876b52d5cc159cbd4b0ff6e7aa419a26470849a43a8f647857a4a24467b/prompt_toolkit-1.0.13.tar.gz";
+      md5 = "427b496d2c147bd3819bc3a7f6e0d493";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal ];
@@ -1199,13 +1342,13 b''
     };
   };
   pyramid = super.buildPythonPackage {
-    name = "pyramid-1.6.1";
+    name = "pyramid-1.7.4";
     buildInputs = with self; [];
     doCheck = false;
     propagatedBuildInputs = with self; [setuptools WebOb repoze.lru zope.interface zope.deprecation venusian translationstring PasteDeploy];
     src = fetchurl {
-      url = "https://pypi.python.org/packages/30/b3/fcc4a2a4800cbf21989e00454b5828cf1f7fe35c63e0810b350e56d4c475/pyramid-1.6.1.tar.gz";
-      md5 = "b18688ff3cc33efdbb098a35b45dd122";
+      url = "https://pypi.python.org/packages/33/91/55f5c661f8923902cd1f68d75f2b937c45e7682857356cf18f0be5493899/pyramid-1.7.4.tar.gz";
+      md5 = "6ef1dfdcff9136d04490410757c4c446";
     };
     meta = {
       license = [ { fullName = "Repoze Public License"; } { fullName = "BSD-derived (http://www.repoze.org/LICENSE.txt)"; } ];
@@ -1225,13 +1368,13 b''
     };
   };
   pyramid-debugtoolbar = super.buildPythonPackage {
-    name = "pyramid-debugtoolbar-2.4.2";
+    name = "pyramid-debugtoolbar-3.0.5";
     buildInputs = with self; [];
     doCheck = false;
     propagatedBuildInputs = with self; [pyramid pyramid-mako repoze.lru Pygments];
     src = fetchurl {
-      url = "https://pypi.python.org/packages/89/00/ed5426ee41ed747ba3ffd30e8230841a6878286ea67d480b1444d24f06a2/pyramid_debugtoolbar-2.4.2.tar.gz";
-      md5 = "073ea67086cc4bd5decc3a000853642d";
+      url = "https://pypi.python.org/packages/64/0e/df00bfb55605900e7a2f7e4a18dd83575a6651688e297d5a0aa4c208fd7d/pyramid_debugtoolbar-3.0.5.tar.gz";
+      md5 = "aebab8c3bfdc6f89e4d3adc1d126538e";
     };
     meta = {
       license = [ { fullName = "Repoze Public License"; } pkgs.lib.licenses.bsdOriginal ];
@@ -1368,16 +1511,16 b''
     };
   };
   python-dateutil = super.buildPythonPackage {
-    name = "python-dateutil-1.5";
+    name = "python-dateutil-2.1";
     buildInputs = with self; [];
     doCheck = false;
-    propagatedBuildInputs = with self; [];
+    propagatedBuildInputs = with self; [six];
     src = fetchurl {
-      url = "https://pypi.python.org/packages/b4/7c/df59c89a753eb33c7c44e1dd42de0e9bc2ccdd5a4d576e0bfad97cc280cb/python-dateutil-1.5.tar.gz";
-      md5 = "0dcb1de5e5cad69490a3b6ab63f0cfa5";
+      url = "https://pypi.python.org/packages/65/52/9c18dac21f174ad31b65e22d24297864a954e6fe65876eba3f5773d2da43/python-dateutil-2.1.tar.gz";
+      md5 = "1534bb15cf311f07afaa3aacba1c028b";
     };
     meta = {
-      license = [ pkgs.lib.licenses.psfl ];
+      license = [ { fullName = "Simplified BSD"; } ];
     };
   };
   python-editor = super.buildPythonPackage {
@@ -1498,10 +1641,10 b''
     };
   };
   rhodecode-enterprise-ce = super.buildPythonPackage {
-    name = "rhodecode-enterprise-ce-4.6.1";
-    buildInputs = with self; [pytest py pytest-cov pytest-sugar pytest-runner pytest-catchlog pytest-profiling gprof2dot pytest-timeout mock WebTest cov-core coverage cssselect lxml configobj];
+    name = "rhodecode-enterprise-ce-4.7.0";
+    buildInputs = with self; [pytest py pytest-cov pytest-sugar pytest-runner pytest-catchlog pytest-profiling gprof2dot pytest-timeout mock WebTest cov-core coverage configobj];
     doCheck = true;
-    propagatedBuildInputs = with self; [Babel Beaker FormEncode Mako Markdown MarkupSafe MySQL-python Paste PasteDeploy PasteScript Pygments pygments-markdown-lexer Pylons Pyro4 Routes SQLAlchemy Tempita URLObject WebError WebHelpers WebHelpers2 WebOb WebTest Whoosh alembic amqplib anyjson appenlight-client authomatic backport-ipaddress celery channelstream colander decorator deform docutils gevent gunicorn infrae.cache ipython iso8601 kombu msgpack-python packaging psycopg2 py-gfm pycrypto pycurl pyparsing pyramid pyramid-debugtoolbar pyramid-mako pyramid-beaker pysqlite python-dateutil python-ldap python-memcached python-pam recaptcha-client repoze.lru requests simplejson subprocess32 waitress zope.cachedescriptors dogpile.cache dogpile.core psutil py-bcrypt];
+    propagatedBuildInputs = with self; [Babel Beaker FormEncode Mako Markdown MarkupSafe MySQL-python Paste PasteDeploy PasteScript Pygments pygments-markdown-lexer Pylons Routes SQLAlchemy Tempita URLObject WebError WebHelpers WebHelpers2 WebOb WebTest Whoosh alembic amqplib anyjson appenlight-client authomatic backport-ipaddress cssselect celery channelstream colander decorator deform docutils gevent gunicorn infrae.cache ipython iso8601 kombu lxml msgpack-python nbconvert packaging psycopg2 py-gfm pycrypto pycurl pyparsing pyramid pyramid-debugtoolbar pyramid-mako pyramid-beaker pysqlite python-dateutil python-ldap python-memcached python-pam recaptcha-client repoze.lru requests simplejson subprocess32 waitress zope.cachedescriptors dogpile.cache dogpile.core psutil py-bcrypt];
     src = ./.;
     meta = {
       license = [ { fullName = "Affero GNU General Public License v3 or later (AGPLv3+)"; } { fullName = "AGPLv3, and Commercial License"; } ];
@@ -1520,19 +1663,6 b''
       license = [ { fullName = "AGPLv3 and Proprietary"; } ];
     };
   };
-  serpent = super.buildPythonPackage {
-    name = "serpent-1.15";
-    buildInputs = with self; [];
-    doCheck = false;
-    propagatedBuildInputs = with self; [];
-    src = fetchurl {
-      url = "https://pypi.python.org/packages/7b/38/b2b27673a882ff2ea5871bb3e3e6b496ebbaafd1612e51990ffb158b9254/serpent-1.15.tar.gz";
-      md5 = "e27b1aad5c218e16442f52abb7c7053a";
-    };
-    meta = {
-      license = [ pkgs.lib.licenses.mit ];
-    };
-  };
   setproctitle = super.buildPythonPackage {
     name = "setproctitle-1.1.8";
     buildInputs = with self; [];
@@ -1650,14 +1780,27 b''
       license = [ pkgs.lib.licenses.mit ];
     };
   };
+  testpath = super.buildPythonPackage {
+    name = "testpath-0.1";
+    buildInputs = with self; [];
+    doCheck = false;
+    propagatedBuildInputs = with self; [];
+    src = fetchurl {
+      url = "https://pypi.python.org/packages/f9/c4/c0b22f35138bc26a6058c39cb61db1e8977e5e9550b12cd2cb02ef56fc51/testpath-0.1.tar.gz";
+      md5 = "401918bcd0b0e5b71a9b909835117bc6";
+    };
+    meta = {
+      license = [ pkgs.lib.licenses.mit ];
+    };
+  };
   traitlets = super.buildPythonPackage {
-    name = "traitlets-4.3.1";
+    name = "traitlets-4.3.2";
     buildInputs = with self; [];
     doCheck = false;
     propagatedBuildInputs = with self; [ipython-genutils six decorator enum34];
     src = fetchurl {
-      url = "https://pypi.python.org/packages/b1/d6/5b5aa6d5c474691909b91493da1e8972e309c9f01ecfe4aeafd272eb3234/traitlets-4.3.1.tar.gz";
-      md5 = "dd0b1b6e5d31ce446d55a4b5e5083c98";
+      url = "https://pypi.python.org/packages/a5/98/7f5ef2fe9e9e071813aaf9cb91d1a732e0a68b6c44a32b38cb8e14c3f069/traitlets-4.3.2.tar.gz";
+      md5 = "3068663f2f38fd939a9eb3a500ccc154";
     };
     meta = {
       license = [ pkgs.lib.licenses.bsdOriginal ];

pytest.ini

@@ -2,7 +2,6 b''
 testpaths = ./rhodecode
 pylons_config = rhodecode/tests/rhodecode.ini
 vcsserver_protocol = http
-vcsserver_config_pyro4 = rhodecode/tests/vcsserver_pyro4.ini
 vcsserver_config_http = rhodecode/tests/vcsserver_http.ini
 norecursedirs = tests/scripts
 addopts = -k "not _BaseTest"

requirements.txt

@@ -14,7 +14,8 b' channelstream==0.5.2'
 click==5.1
 colander==1.2
 configobj==5.0.6
-decorator==3.4.2
+cssselect==1.0.1
+decorator==4.0.11
 deform==2.0a2
 docutils==0.12
 dogpile.cache==0.6.1
@@ -29,6 +30,7 b' iso8601==0.1.11'
 itsdangerous==0.24
 Jinja2==2.7.3
 kombu==1.5.1
+lxml==3.7.3
 Mako==1.0.6
 Markdown==2.6.7
 MarkupSafe==0.23
@@ -42,6 +44,7 b' paramiko==1.15.1'
 Paste==2.0.3
 PasteDeploy==1.5.2
 PasteScript==1.7.5
+pathlib2==2.1.0
 psutil==4.3.1
 psycopg2==2.6.1
 py-bcrypt==0.4
@@ -52,12 +55,12 b' pygments-markdown-lexer==0.1.0.dev39'
 Pygments==2.2.0
 pyparsing==1.5.7
 pyramid-beaker==0.8
-pyramid-debugtoolbar==2.4.2
+pyramid-debugtoolbar==3.0.5
 pyramid-jinja2==2.5
 pyramid-mako==1.0.2
-pyramid==1.6.1
+pyramid==1.7.4
 pysqlite==2.6.3
-python-dateutil==1.5
+python-dateutil==2.1
 python-ldap==2.4.19
 python-memcached==1.57
 python-pam==1.8.2
@@ -97,6 +100,12 b' https://code.rhodecode.com/upstream/pylo'
 # not released py-gfm==0.1.3
 https://code.rhodecode.com/upstream/py-gfm/archive/0d66a19bc16e3d49de273c0f797d4e4781e8c0f2.tar.gz?md5=0d0d5385bfb629eea636a80b9c2bfd16#egg=py-gfm==0.1.3.rhodecode-upstream1
 
+# IPYTHON RENDERING
+# entrypoints backport, pypi version doesn't support egg installs
+https://code.rhodecode.com/upstream/entrypoints/archive/96e6d645684e1af3d7df5b5272f3fe85a546b233.tar.gz?md5=7db37771aea9ac9fefe093e5d6987313#egg=entrypoints==0.2.2.rhodecode-upstream1
+nbconvert==5.1.1
+nbformat==4.3.0
+jupyter_client==5.0.0
 
 ## cli tools
 alembic==0.8.4
@@ -123,9 +132,5 b' https://code.rhodecode.com/rhodecode-too'
 ## appenlight
 appenlight-client==0.6.14
 
-# Pyro/Deprecated TODO(Marcink): remove in 4.7 release.
-Pyro4==4.41
-serpent==1.15
-
 ## test related requirements
 -r requirements_test.txt

requirements_test.txt

@@ -13,5 +13,3 b' mock==1.0.1'
 WebTest==1.4.3
 cov-core==1.15.0
 coverage==3.7.1
-cssselect==0.9.1
-lxml==3.4.4

rhodecode/VERSION

@@ -1,1 +1,1 b''
-4.6.1
  No newline at end of file
+4.7.0
  No newline at end of file

rhodecode/__init__.py

@@ -51,7 +51,7 b' PYRAMID_SETTINGS = {}'
 EXTENSIONS = {}
 
 __version__ = ('.'.join((str(each) for each in VERSION[:3])))
-__dbversion__ = 64  # defines current db version for migrations
+__dbversion__ = 71  # defines current db version for migrations
 __platform__ = platform.system()
 __license__ = 'AGPLv3, and Commercial License'
 __author__ = 'RhodeCode GmbH'

rhodecode/api/__init__.py

@@ -22,6 +22,7 b' import inspect'
 import itertools
 import logging
 import types
+import fnmatch
 
 import decorator
 import venusian
@@ -47,6 +48,18 b" DEFAULT_RENDERER = 'jsonrpc_renderer'"
 DEFAULT_URL = '/_admin/apiv2'
 
 
+def find_methods(jsonrpc_methods, pattern):
+    matches = OrderedDict()
+    if not isinstance(pattern, (list, tuple)):
+        pattern = [pattern]
+
+    for single_pattern in pattern:
+        for method_name, method in jsonrpc_methods.items():
+            if fnmatch.fnmatch(method_name, single_pattern):
+                matches[method_name] = method
+    return matches
+
+
 class ExtJsonRenderer(object):
     """
     Custom renderer that mkaes use of our ext_json lib
@@ -143,7 +156,19 b' def exception_view(exc, request):'
         log.debug('json-rpc method `%s` not found in list of '
                   'api calls: %s, rpc_id:%s',
                   method, request.registry.jsonrpc_methods.keys(), rpc_id)
-        fault_message = "No such method: {}".format(method)
+
+        similar = 'none'
+        try:
+            similar_paterns = ['*{}*'.format(x) for x in method.split('_')]
+            similar_found = find_methods(
+                request.registry.jsonrpc_methods, similar_paterns)
+            similar = ', '.join(similar_found.keys()) or similar
+        except Exception:
+            # make the whole above block safe
+            pass
+
+        fault_message = "No such method: {}. Similar methods: {}".format(
+            method, similar)
 
     return jsonrpc_error(request, fault_message, rpc_id)
 
@@ -184,18 +209,18 b' def request_view(request):'
         request.rpc_user = auth_u
 
         # now check if token is valid for API
-        role = UserApiKeys.ROLE_API
-        extra_auth_tokens = [
-            x.api_key for x in User.extra_valid_auth_tokens(api_user, role=role)]
-        active_tokens = [api_user.api_key] + extra_auth_tokens
+        auth_token = request.rpc_api_key
+        token_match = api_user.authenticate_by_token(
+            auth_token, roles=[UserApiKeys.ROLE_API])
+        invalid_token = not token_match
 
-        log.debug('Checking if API key has proper role')
-        if request.rpc_api_key not in active_tokens:
+        log.debug('Checking if API KEY is valid with proper role')
+        if invalid_token:
             return jsonrpc_error(
                 request, retid=request.rpc_id,
-                message='API KEY has bad role for an API call')
+                message='API KEY invalid or, has bad role for an API call')
 
-    except Exception as e:
+    except Exception:
         log.exception('Error on API AUTH')
         return jsonrpc_error(
             request, retid=request.rpc_id, message='Invalid API KEY')
@@ -351,9 +376,10 b' class RoutePredicate(object):'
 class NotFoundPredicate(object):
     def __init__(self, val, config):
         self.val = val
+        self.methods = config.registry.jsonrpc_methods
 
     def text(self):
-        return 'jsonrpc method not found = %s' % self.val
+        return 'jsonrpc method not found = {}.'.format(self.val)
 
     phash = text
 

rhodecode/api/tests/conftest.py

@@ -22,14 +22,19 b' import pytest'
 
 from rhodecode.model.meta import Session
 from rhodecode.model.user import UserModel
+from rhodecode.model.auth_token import AuthTokenModel
 from rhodecode.tests import TEST_USER_ADMIN_LOGIN
 
 
 @pytest.fixture(scope="class")
 def testuser_api(request, pylonsapp):
     cls = request.cls
+
+    # ADMIN USER
     cls.usr = UserModel().get_by_username(TEST_USER_ADMIN_LOGIN)
     cls.apikey = cls.usr.api_key
+
+    # REGULAR USER
     cls.test_user = UserModel().create_or_update(
         username='test-api',
         password='test',
@@ -37,6 +42,11 b' def testuser_api(request, pylonsapp):'
         firstname='first',
         lastname='last'
     )
+    # create TOKEN for user, if he doesn't have one
+    if not cls.test_user.api_key:
+        AuthTokenModel().create(
+            user=cls.test_user, description='TEST_USER_TOKEN')
+
     Session().commit()
     cls.TEST_USER_LOGIN = cls.test_user.username
     cls.apikey_regular = cls.test_user.api_key

rhodecode/api/tests/test_api.py

@@ -80,7 +80,13 b' class TestApi(object):'
     def test_api_non_existing_method(self, request):
         id_, params = build_data(self.apikey, 'not_existing', args='xx')
         response = api_call(self.app, params)
-        expected = 'No such method: not_existing'
+        expected = 'No such method: not_existing. Similar methods: none'
+        assert_error(id_, expected, given=response.body)
+
+    def test_api_non_existing_method_have_similar(self, request):
+        id_, params = build_data(self.apikey, 'comment', args='xx')
+        response = api_call(self.app, params)
+        expected = 'No such method: comment. Similar methods: changeset_comment, comment_pull_request, comment_commit'
         assert_error(id_, expected, given=response.body)
 
     def test_api_disabled_user(self, request):

rhodecode/api/tests/test_close_pull_request.py

@@ -29,6 +29,7 b' from rhodecode.api.tests.utils import ('
 
 @pytest.mark.usefixtures("testuser_api", "app")
 class TestClosePullRequest(object):
+
     @pytest.mark.backends("git", "hg")
     def test_api_close_pull_request(self, pr_util):
         pull_request = pr_util.create_pull_request()

rhodecode/api/tests/test_delete_repo.py

@@ -53,11 +53,10 b' class TestApiDeleteRepo(object):'
         }
         assert_ok(id_, expected, given=response.body)
 
-    def test_api_delete_repo_by_non_admin_no_permission(
-            self, backend, user_regular):
+    def test_api_delete_repo_by_non_admin_no_permission(self, backend):
         repo = backend.create_repo()
         id_, params = build_data(
-            user_regular.api_key, 'delete_repo', repoid=repo.repo_name, )
+            self.apikey_regular, 'delete_repo', repoid=repo.repo_name, )
         response = api_call(self.app, params)
         expected = 'repository `%s` does not exist' % (repo.repo_name)
         assert_error(id_, expected, given=response.body)

rhodecode/api/tests/test_update_user_group.py

@@ -23,7 +23,7 b' import pytest'
 
 from rhodecode.model.user import UserModel
 from rhodecode.model.user_group import UserGroupModel
-from rhodecode.tests import TEST_USER_REGULAR_LOGIN
+from rhodecode.tests import TEST_USER_ADMIN_EMAIL
 from rhodecode.api.tests.utils import (
     build_data, api_call, assert_error, assert_ok, crash, jsonify)
 
@@ -33,7 +33,8 b' class TestUpdateUserGroup(object):'
     @pytest.mark.parametrize("changing_attr, updates", [
         ('group_name', {'group_name': 'new_group_name'}),
         ('group_name', {'group_name': 'test_group_for_update'}),
-        ('owner', {'owner': TEST_USER_REGULAR_LOGIN}),
+        # ('owner', {'owner': TEST_USER_REGULAR_LOGIN}),
+        ('owner_email', {'owner_email': TEST_USER_ADMIN_EMAIL}),
         ('active', {'active': False}),
         ('active', {'active': True})
     ])
@@ -59,7 +60,8 b' class TestUpdateUserGroup(object):'
         # TODO: mikhail: decide if we need to test against the commented params
         # ('group_name', {'group_name': 'new_group_name'}),
         # ('group_name', {'group_name': 'test_group_for_update'}),
-        ('owner', {'owner': TEST_USER_REGULAR_LOGIN}),
+        # ('owner', {'owner': TEST_USER_REGULAR_LOGIN}),
+        ('owner_email', {'owner_email': TEST_USER_ADMIN_EMAIL}),
         ('active', {'active': False}),
         ('active', {'active': True})
     ])

rhodecode/api/tests/test_utils.py

@@ -181,29 +181,56 b' class TestGetRefHash(object):'
 class TestUserByNameOrError(object):
     def test_user_found_by_id(self):
         fake_user = Mock(id=123)
+
+        patcher = patch('rhodecode.model.user.UserModel.get_user')
+        with patcher as get_user:
+            get_user.return_value = fake_user
+
+            patcher = patch('rhodecode.model.user.UserModel.get_by_username')
+            with patcher as get_by_username:
+                result = utils.get_user_or_error(123)
+                assert result == fake_user
+
+    def test_user_not_found_by_id_as_str(self):
+        fake_user = Mock(id=123)
+
         patcher = patch('rhodecode.model.user.UserModel.get_user')
         with patcher as get_user:
             get_user.return_value = fake_user
-            result = utils.get_user_or_error('123')
-            assert result == fake_user
+            patcher = patch('rhodecode.model.user.UserModel.get_by_username')
+            with patcher as get_by_username:
+                get_by_username.return_value = None
+
+                with pytest.raises(JSONRPCError):
+                    utils.get_user_or_error('123')
 
     def test_user_found_by_name(self):
         fake_user = Mock(id=123)
-        patcher = patch('rhodecode.model.user.UserModel.get_by_username')
-        with patcher as get_by_username:
-            get_by_username.return_value = fake_user
-            result = utils.get_user_or_error('test')
-            assert result == fake_user
+
+        patcher = patch('rhodecode.model.user.UserModel.get_user')
+        with patcher as get_user:
+            get_user.return_value = None
+
+            patcher = patch('rhodecode.model.user.UserModel.get_by_username')
+            with patcher as get_by_username:
+                get_by_username.return_value = fake_user
+
+                result = utils.get_user_or_error('test')
+                assert result == fake_user
 
     def test_user_not_found_by_id(self):
         patcher = patch('rhodecode.model.user.UserModel.get_user')
         with patcher as get_user:
             get_user.return_value = None
-            with pytest.raises(JSONRPCError) as excinfo:
-                utils.get_user_or_error('123')
+            patcher = patch('rhodecode.model.user.UserModel.get_by_username')
+            with patcher as get_by_username:
+                get_by_username.return_value = None
 
-        expected_message = 'user `123` does not exist'
-        assert excinfo.value.message == expected_message
+                with pytest.raises(JSONRPCError) as excinfo:
+                    utils.get_user_or_error(123)
+
+                expected_message = 'user `123` does not exist'
+                assert excinfo.value.message == expected_message
 
     def test_user_not_found_by_name(self):
         patcher = patch('rhodecode.model.user.UserModel.get_by_username')
@@ -216,8 +243,7 b' class TestUserByNameOrError(object):'
         assert excinfo.value.message == expected_message
 
 
-class TestGetCommitDict:
-
+class TestGetCommitDict(object):
     @pytest.mark.parametrize('filename, expected', [
         (b'sp\xc3\xa4cial', u'sp\xe4cial'),
         (b'sp\xa4cial', u'sp\ufffdcial'),

rhodecode/api/utils.py

@@ -213,15 +213,19 b' def get_user_or_error(userid):'
     :param userid:
     """
     from rhodecode.model.user import UserModel
+    user_model = UserModel()
 
-    user_model = UserModel()
-    try:
-        user = user_model.get_user(int(userid))
-    except ValueError:
+    if isinstance(userid, (int, long)):
+        try:
+            user = user_model.get_user(userid)
+        except ValueError:
+            user = None
+    else:
         user = user_model.get_by_username(userid)
 
     if user is None:
-        raise JSONRPCError("user `%s` does not exist" % (userid,))
+        raise JSONRPCError(
+            'user `%s` does not exist' % (userid,))
     return user
 
 
@@ -232,10 +236,19 b' def get_repo_or_error(repoid):'
     :param repoid:
     """
     from rhodecode.model.repo import RepoModel
+    repo_model = RepoModel()
 
-    repo = RepoModel().get_repo(repoid)
+    if isinstance(repoid, (int, long)):
+        try:
+            repo = repo_model.get_repo(repoid)
+        except ValueError:
+            repo = None
+    else:
+        repo = repo_model.get_by_repo_name(repoid)
+
     if repo is None:
-        raise JSONRPCError('repository `%s` does not exist' % (repoid,))
+        raise JSONRPCError(
+            'repository `%s` does not exist' % (repoid,))
     return repo
 
 
@@ -246,8 +259,16 b' def get_repo_group_or_error(repogroupid)'
     :param repogroupid:
     """
     from rhodecode.model.repo_group import RepoGroupModel
+    repo_group_model = RepoGroupModel()
 
-    repo_group = RepoGroupModel()._get_repo_group(repogroupid)
+    if isinstance(repogroupid, (int, long)):
+        try:
+            repo_group = repo_group_model._get_repo_group(repogroupid)
+        except ValueError:
+            repo_group = None
+    else:
+        repo_group = repo_group_model.get_by_group_name(repogroupid)
+
     if repo_group is None:
         raise JSONRPCError(
             'repository group `%s` does not exist' % (repogroupid,))
@@ -261,10 +282,19 b' def get_user_group_or_error(usergroupid)'
     :param usergroupid:
     """
     from rhodecode.model.user_group import UserGroupModel
+    user_group_model = UserGroupModel()
 
-    user_group = UserGroupModel().get_group(usergroupid)
+    if isinstance(usergroupid, (int, long)):
+        try:
+            user_group = user_group_model.get_group(usergroupid)
+        except ValueError:
+            user_group = None
+    else:
+        user_group = user_group_model.get_by_name(usergroupid)
+
     if user_group is None:
-        raise JSONRPCError('user group `%s` does not exist' % (usergroupid,))
+        raise JSONRPCError(
+            'user group `%s` does not exist' % (usergroupid,))
     return user_group
 
 

rhodecode/api/views/pull_request_api.py

@@ -244,9 +244,8 b' def merge_pull_request(request, apiuser,'
 
     .. code-block:: bash
 
-      "id": <id_given_in_input>,
-      "result":
-        {
+        "id": <id_given_in_input>,
+        "result": {
             "executed":         "<bool>",
             "failure_reason":   "<int>",
             "merge_commit_id":  "<merge_commit_id>",
@@ -257,8 +256,7 b' def merge_pull_request(request, apiuser,'
                                     "name":      "<name>"
                                 }
         },
-      "error": null
-
+        "error": null
     """
     repo = get_repo_or_error(repoid)
     if not isinstance(userid, Optional):
@@ -321,13 +319,12 b' def close_pull_request(request, apiuser,'
 
     .. code-block:: bash
 
-      "id": <id_given_in_input>,
-      "result":
-        {
+        "id": <id_given_in_input>,
+        "result": {
             "pull_request_id":  "<int>",
             "closed":           "<bool>"
         },
-      "error": null
+        "error": null
 
     """
     repo = get_repo_or_error(repoid)
@@ -396,15 +393,14 b' def comment_pull_request('
 
     .. code-block:: bash
 
-      id : <id_given_in_input>
-      result :
-        {
+        id : <id_given_in_input>
+        result : {
             "pull_request_id":  "<Integer>",
             "comment_id":       "<Integer>",
             "status": {"given": <given_status>,
                        "was_changed": <bool status_was_actually_changed> },
-        }
-      error :  null
+        },
+        error :  null
     """
     repo = get_repo_or_error(repoid)
     if not isinstance(userid, Optional):
@@ -535,9 +531,8 b' def create_pull_request('
     :param reviewers: Set the new pull request reviewers list.
     :type reviewers: Optional(list)
         Accepts username strings or objects of the format:
-        {
-            'username': 'nick', 'reasons': ['original author']
-        }
+
+            {'username': 'nick', 'reasons': ['original author']}
     """
 
     source = get_repo_or_error(source_repo)
@@ -633,9 +628,8 b' def update_pull_request('
 
     .. code-block:: bash
 
-      id : <id_given_in_input>
-      result :
-        {
+        id : <id_given_in_input>
+        result : {
             "msg": "Updated pull request `63`",
             "pull_request": <pull_request_object>,
             "updated_reviewers": {
@@ -655,7 +649,7 b' def update_pull_request('
               "removed": []
             }
         }
-      error :  null
+        error :  null
     """
 
     repo = get_repo_or_error(repoid)

rhodecode/api/views/server_api.py

@@ -18,10 +18,12 b''
 # RhodeCode Enterprise Edition, including its added features, Support services,
 # and proprietary license terms, please see https://rhodecode.com/licenses/
 
-
+import inspect
 import logging
+import itertools
 
-from rhodecode.api import jsonrpc_method, JSONRPCError, JSONRPCForbidden
+from rhodecode.api import (
+    jsonrpc_method, JSONRPCError, JSONRPCForbidden, find_methods)
 
 from rhodecode.api.utils import (
     Optional, OAttr, has_superadmin_permission, get_user_or_error)
@@ -243,3 +245,77 b' def cleanup_sessions(request, apiuser, o'
         raise JSONRPCError(
             'Error occurred during session cleanup'
         )
+
+
+@jsonrpc_method()
+def get_method(request, apiuser, pattern=Optional('*')):
+    """
+    Returns list of all available API methods. By default match pattern
+    os "*" but any other pattern can be specified. eg *comment* will return
+    all methods with comment inside them. If just single method is matched
+    returned data will also include method specification
+
+    This command can only be run using an |authtoken| with admin rights to
+    the specified repository.
+
+    This command takes the following options:
+
+    :param apiuser: This is filled automatically from the |authtoken|.
+    :type apiuser: AuthUser
+    :param pattern: pattern to match method names against
+    :type older_then: Optional("*")
+
+    Example output:
+
+    .. code-block:: bash
+
+      id : <id_given_in_input>
+      "result": [
+        "changeset_comment",
+        "comment_pull_request",
+        "comment_commit"
+      ]
+      error :  null
+
+    .. code-block:: bash
+
+      id : <id_given_in_input>
+      "result": [
+        "comment_commit",
+        {
+          "apiuser": "<RequiredType>",
+          "comment_type": "<Optional:u'note'>",
+          "commit_id": "<RequiredType>",
+          "message": "<RequiredType>",
+          "repoid": "<RequiredType>",
+          "request": "<RequiredType>",
+          "resolves_comment_id": "<Optional:None>",
+          "status": "<Optional:None>",
+          "userid": "<Optional:<OptionalAttr:apiuser>>"
+        }
+      ]
+      error :  null
+    """
+    if not has_superadmin_permission(apiuser):
+        raise JSONRPCForbidden()
+
+    pattern = Optional.extract(pattern)
+
+    matches = find_methods(request.registry.jsonrpc_methods, pattern)
+
+    args_desc = []
+    if len(matches) == 1:
+        func = matches[matches.keys()[0]]
+
+        argspec = inspect.getargspec(func)
+        arglist = argspec[0]
+        defaults = map(repr, argspec[3] or [])
+
+        default_empty = '<RequiredType>'
+
+        # kw arguments required by this method
+        func_kwargs = dict(itertools.izip_longest(
+            reversed(arglist), reversed(defaults), fillvalue=default_empty))
+        args_desc.append(func_kwargs)
+
+    return matches.keys() + args_desc

rhodecode/api/views/user_api.py

@@ -29,7 +29,6 b' from rhodecode.lib.utils2 import safe_in'
 from rhodecode.model.db import Session, User, Repository
 from rhodecode.model.user import UserModel
 
-
 log = logging.getLogger(__name__)
 
 
@@ -62,8 +61,8 b' def get_user(request, apiuser, userid=Op'
           "result": {
             "active": true,
             "admin": false,
-            "api_key": "api-key",
             "api_keys": [ list of keys ],
+            "auth_tokens": [ list of tokens with details ],
             "email": "[e-mail]",
             "emails": [
               "[e-mail]"
@@ -74,6 +73,7 b' def get_user(request, apiuser, userid=Op'
             "ip_addresses": [],
             "language": null,
             "last_login": "Timestamp",
+            "last_activity": "Timestamp",
             "lastname": "surnae",
             "permissions": {
               "global": [
@@ -133,7 +133,7 b' def get_users(request, apiuser):'
     .. code-block:: bash
 
         id : <id_given_in_input>
-            result: [<user_object>, ...]
+        result: [<user_object>, ...]
         error:  null
     """
 
@@ -191,15 +191,16 b' def create_user(request, apiuser, userna'
     :type force_password_change: Optional(``True`` | ``False``)
     :param create_personal_repo_group: Create personal repo group for this user
     :type create_personal_repo_group: Optional(``True`` | ``False``)
+
     Example output:
 
     .. code-block:: bash
 
         id : <id_given_in_input>
         result: {
-                  "msg" : "created new user `<username>`",
-                  "user": <user_obj>
-                }
+            "msg" : "created new user `<username>`",
+            "user": <user_obj>
+        }
         error:  null
 
     Example error output:
@@ -305,9 +306,9 b' def update_user(request, apiuser, userid'
 
         id : <id_given_in_input>
         result: {
-                  "msg" : "updated user ID:<userid> <username>",
-                  "user": <user_object>,
-                }
+            "msg" : "updated user ID:<userid> <username>",
+            "user": <user_object>,
+        }
         error:  null
 
     Example error output:
@@ -384,9 +385,9 b' def delete_user(request, apiuser, userid'
 
         id : <id_given_in_input>
         result: {
-                  "msg" : "deleted user ID:<userid> <username>",
-                  "user": null
-                }
+            "msg" : "deleted user ID:<userid> <username>",
+            "user": null
+        }
         error:  null
 
     Example error output:
@@ -470,3 +471,45 b' def get_user_locks(request, apiuser, use'
                 ret.append(_api_data)
 
     return ret
+
+
+@jsonrpc_method()
+def get_user_audit_logs(request, apiuser, userid=Optional(OAttr('apiuser'))):
+    """
+    Fetches all action logs made by the specified user.
+
+    This command takes the following options:
+
+    :param apiuser: This is filled automatically from the |authtoken|.
+    :type apiuser: AuthUser
+    :param userid: Sets the userid whose list of locked |repos| will be
+        displayed.
+    :type userid: Optional(str or int)
+
+    Example output:
+
+    .. code-block:: bash
+
+        id : <id_given_in_input>
+        result : {
+            [action, action,...]
+        }
+        error :  null
+    """
+
+    if not has_superadmin_permission(apiuser):
+        # make sure normal user does not pass someone else userid,
+        # he is not allowed to do that
+        if not isinstance(userid, Optional) and userid != apiuser.user_id:
+            raise JSONRPCError('userid is not the same as your user')
+
+    userid = Optional.extract(userid, evaluate_locals=locals())
+    userid = getattr(userid, 'user_id', userid)
+    user = get_user_or_error(userid)
+
+    ret = []
+
+    # show all user actions
+    for entry in UserModel().get_user_log(user, filter_term=None):
+        ret.append(entry)
+    return ret

rhodecode/apps/admin/__init__.py

@@ -19,11 +19,72 b''
 # and proprietary license terms, please see https://rhodecode.com/licenses/
 
 
-from rhodecode.admin.navigation import NavigationRegistry
+from rhodecode.apps.admin.navigation import NavigationRegistry
 from rhodecode.config.routing import ADMIN_PREFIX
 from rhodecode.lib.utils2 import str2bool
 
 
+def admin_routes(config):
+    """
+    Admin prefixed routes
+    """
+
+    config.add_route(
+        name='admin_settings_open_source',
+        pattern='/settings/open_source')
+    config.add_route(
+        name='admin_settings_vcs_svn_generate_cfg',
+        pattern='/settings/vcs/svn_generate_cfg')
+
+    config.add_route(
+        name='admin_settings_system',
+        pattern='/settings/system')
+    config.add_route(
+        name='admin_settings_system_update',
+        pattern='/settings/system/updates')
+
+    config.add_route(
+        name='admin_settings_sessions',
+        pattern='/settings/sessions')
+    config.add_route(
+        name='admin_settings_sessions_cleanup',
+        pattern='/settings/sessions/cleanup')
+
+    # users admin
+    config.add_route(
+        name='users',
+        pattern='/users')
+
+    config.add_route(
+        name='users_data',
+        pattern='/users_data')
+
+    # user auth tokens
+    config.add_route(
+        name='edit_user_auth_tokens',
+        pattern='/users/{user_id:\d+}/edit/auth_tokens')
+    config.add_route(
+        name='edit_user_auth_tokens_add',
+        pattern='/users/{user_id:\d+}/edit/auth_tokens/new')
+    config.add_route(
+        name='edit_user_auth_tokens_delete',
+        pattern='/users/{user_id:\d+}/edit/auth_tokens/delete')
+
+    # user groups management
+    config.add_route(
+        name='edit_user_groups_management',
+        pattern='/users/{user_id:\d+}/edit/groups_management')
+
+    config.add_route(
+        name='edit_user_groups_management_updates',
+        pattern='/users/{user_id:\d+}/edit/edit_user_groups_management/updates')
+
+    # user audit logs
+    config.add_route(
+        name='edit_user_audit_logs',
+        pattern='/users/{user_id:\d+}/edit/audit')
+
+
 def includeme(config):
     settings = config.get_settings()
 
@@ -32,26 +93,7 b' def includeme(config):'
     navigation_registry = NavigationRegistry(labs_active=labs_active)
     config.registry.registerUtility(navigation_registry)
 
-    config.add_route(
-        name='admin_settings_open_source',
-        pattern=ADMIN_PREFIX + '/settings/open_source')
-    config.add_route(
-        name='admin_settings_vcs_svn_generate_cfg',
-        pattern=ADMIN_PREFIX + '/settings/vcs/svn_generate_cfg')
-
-    config.add_route(
-        name='admin_settings_system',
-        pattern=ADMIN_PREFIX + '/settings/system')
-    config.add_route(
-        name='admin_settings_system_update',
-        pattern=ADMIN_PREFIX + '/settings/system/updates')
-
-    config.add_route(
-        name='admin_settings_sessions',
-        pattern=ADMIN_PREFIX + '/settings/sessions')
-    config.add_route(
-        name='admin_settings_sessions_cleanup',
-        pattern=ADMIN_PREFIX + '/settings/sessions/cleanup')
+    config.include(admin_routes, route_prefix=ADMIN_PREFIX)
 
     # Scan module for configuration decorators.
     config.scan()

rhodecode/apps/admin/navigation.py

@@ -25,7 +25,7 b' import collections'
 from pylons import url
 from zope.interface import implementer
 
-from rhodecode.admin.interfaces import IAdminNavigationRegistry
+from rhodecode.apps.admin.interfaces import IAdminNavigationRegistry
 from rhodecode.lib.utils import get_registry
 from rhodecode.translation import _
 

rhodecode/apps/admin/views/open_source_licenses.py

@@ -24,15 +24,15 b' import logging'
 from pylons import tmpl_context as c
 from pyramid.view import view_config
 
-from rhodecode.admin.views.base import AdminSettingsView
-from rhodecode.admin.navigation import navigation_list
+from rhodecode.apps._base import BaseAppView
+from rhodecode.apps.admin.navigation import navigation_list
 from rhodecode.lib.auth import (LoginRequired, HasPermissionAllDecorator)
 from rhodecode.lib.utils import read_opensource_licenses
 
 log = logging.getLogger(__name__)
 
 
-class OpenSourceLicensesAdminSettingsView(AdminSettingsView):
+class OpenSourceLicensesAdminSettingsView(BaseAppView):
 
     @LoginRequired()
     @HasPermissionAllDecorator('hg.admin')

rhodecode/apps/admin/views/sessions.py

@@ -24,9 +24,8 b' from pylons import tmpl_context as c'
 from pyramid.view import view_config
 from pyramid.httpexceptions import HTTPFound
 
-from rhodecode.translation import _
-
-from rhodecode.admin.views.base import AdminSettingsView
+from rhodecode.apps._base import BaseAppView
+from rhodecode.apps.admin.navigation import navigation_list
 from rhodecode.lib.auth import (
     LoginRequired, HasPermissionAllDecorator, CSRFRequired)
 from rhodecode.lib.utils2 import safe_int
@@ -34,13 +33,10 b' from rhodecode.lib import system_info'
 from rhodecode.lib import user_sessions
 
 
-from rhodecode.admin.navigation import navigation_list
-
-
 log = logging.getLogger(__name__)
 
 
-class AdminSessionSettingsView(AdminSettingsView):
+class AdminSessionSettingsView(BaseAppView):
 
     @LoginRequired()
     @HasPermissionAllDecorator('hg.admin')

rhodecode/apps/admin/views/svn_config.py

@@ -22,16 +22,15 b' import logging'
 
 from pyramid.view import view_config
 
-from rhodecode.svn_support.utils import generate_mod_dav_svn_config
-
-from rhodecode.admin.views.base import AdminSettingsView
+from rhodecode.apps._base import BaseAppView
+from rhodecode.apps.svn_support.utils import generate_mod_dav_svn_config
 from rhodecode.lib.auth import (
     LoginRequired, HasPermissionAllDecorator, CSRFRequired)
 
 log = logging.getLogger(__name__)
 
 
-class SvnConfigAdminSettingsView(AdminSettingsView):
+class SvnConfigAdminSettingsView(BaseAppView):
 
     @LoginRequired()
     @CSRFRequired()

rhodecode/apps/admin/views/system_info.py

@@ -26,20 +26,19 b' from pylons import tmpl_context as c'
 from pyramid.view import view_config
 
 import rhodecode
+from rhodecode.apps._base import BaseAppView
+from rhodecode.apps.admin.navigation import navigation_list
 from rhodecode.lib import helpers as h
 from rhodecode.lib.auth import (LoginRequired, HasPermissionAllDecorator)
 from rhodecode.lib.utils2 import str2bool
 from rhodecode.lib import system_info
 from rhodecode.lib.ext_json import json
-
-from rhodecode.admin.views.base import AdminSettingsView
-from rhodecode.admin.navigation import navigation_list
 from rhodecode.model.settings import SettingsModel
 
 log = logging.getLogger(__name__)
 
 
-class AdminSystemInfoSettingsView(AdminSettingsView):
+class AdminSystemInfoSettingsView(BaseAppView):
 
     @staticmethod
     def get_update_data(update_url):
@@ -107,6 +106,8 b' class AdminSystemInfoSettingsView(AdminS'
             (_('RhodeCode Server IP'), val('server')['server_ip'], state('server')),
             (_('RhodeCode Server ID'), val('server')['server_id'], state('server')),
             (_('RhodeCode Configuration'), val('rhodecode_config')['path'], state('rhodecode_config')),
+            (_('Workers'), val('rhodecode_config')['config']['server:main'].get('workers', '?'), state('rhodecode_config')),
+            (_('Worker Type'), val('rhodecode_config')['config']['server:main'].get('worker_class', 'sync'), state('rhodecode_config')),
             ('', '', ''),  # spacer
 
             # Database
@@ -122,7 +123,7 b' class AdminSystemInfoSettingsView(AdminS'
             ('', '', ''),  # spacer
 
             # Systems stats
-            (_('CPU'), val('cpu'), state('cpu')),
+            (_('CPU'), val('cpu')['text'], state('cpu')),
             (_('Load'), val('load')['text'], state('load')),
             (_('Memory'), val('memory')['text'], state('memory')),
             (_('Uptime'), val('uptime')['text'], state('uptime')),

rhodecode/apps/channelstream/__init__.py

@@ -85,4 +85,6 b' def includeme(config):'
     config.add_route(
         name='channelstream_proxy',
         pattern=settings.get('channelstream.proxy_path') or '/_channelstream')
-    config.scan('rhodecode.channelstream')
+
+    # Scan module for configuration decorators.
+    config.scan()

rhodecode/apps/login/tests/test_register_captcha.py

@@ -22,8 +22,9 b''
 import mock
 import pytest
 
+from rhodecode.apps.login.views import LoginView, CaptchaData
 from rhodecode.config.routing import ADMIN_PREFIX
-from rhodecode.login.views import LoginView, CaptchaData
+from rhodecode.lib.utils2 import AttributeDict
 from rhodecode.model.settings import SettingsModel
 from rhodecode.tests.utils import AssertResponse
 
@@ -40,7 +41,7 b' class RhodeCodeSetting(object):'
         model.create_or_update_setting(name=self.name, val=self.value)
         return self
 
-    def __exit__(self, type, value, traceback):
+    def __exit__(self, exc_type, exc_val, exc_tb):
         model = SettingsModel()
         if self.old_setting:
             model.create_or_update_setting(
@@ -57,8 +58,12 b' class TestRegisterCaptcha(object):'
         ('privkey', '',       CaptchaData(True,  'privkey', '')),
         ('privkey', 'pubkey', CaptchaData(True,  'privkey', 'pubkey')),
     ])
-    def test_get_captcha_data(self, private_key, public_key, expected, db):
-        login_view = LoginView(mock.Mock(), mock.Mock())
+    def test_get_captcha_data(self, private_key, public_key, expected, db,
+                              request_stub, user_util):
+        request_stub.user = user_util.create_user().AuthUser
+        request_stub.matched_route = AttributeDict({'name': 'login'})
+        login_view = LoginView(mock.Mock(), request_stub)
+
         with RhodeCodeSetting('captcha_private_key', private_key):
             with RhodeCodeSetting('captcha_public_key', public_key):
                 captcha = login_view._get_captcha_data()
@@ -92,7 +97,7 b' class TestRegisterCaptcha(object):'
             assertr.no_element_exists('#recaptcha_field')
 
     @pytest.mark.parametrize('valid', [False, True])
-    @mock.patch('rhodecode.login.views.submit')
+    @mock.patch('rhodecode.apps.login.views.submit')
     @mock.patch.object(LoginView, '_get_captcha_data')
     def test_register_with_active_captcha(
             self, m_get_captcha_data, m_submit, valid, app, csrf_token):

rhodecode/apps/login/views.py

@@ -18,6 +18,7 b''
 # RhodeCode Enterprise Edition, including its added features, Support services,
 # and proprietary license terms, please see https://rhodecode.com/licenses/
 
+import time
 import collections
 import datetime
 import formencode
@@ -29,6 +30,7 b' from pyramid.httpexceptions import HTTPF'
 from pyramid.view import view_config
 from recaptcha.client.captcha import submit
 
+from rhodecode.apps._base import BaseAppView
 from rhodecode.authentication.base import authenticate, HTTP_TYPE
 from rhodecode.events import UserRegistered
 from rhodecode.lib import helpers as h
@@ -37,9 +39,10 b' from rhodecode.lib.auth import ('
 from rhodecode.lib.base import get_ip_addr
 from rhodecode.lib.exceptions import UserCreationError
 from rhodecode.lib.utils2 import safe_str
-from rhodecode.model.db import User
+from rhodecode.model.db import User, UserApiKeys
 from rhodecode.model.forms import LoginForm, RegisterForm, PasswordResetForm
 from rhodecode.model.meta import Session
+from rhodecode.model.auth_token import AuthTokenModel
 from rhodecode.model.settings import SettingsModel
 from rhodecode.model.user import UserModel
 from rhodecode.translation import _
@@ -103,20 +106,13 b' def get_came_from(request):'
     return came_from or url('home')
 
 
-class LoginView(object):
+class LoginView(BaseAppView):
 
-    def __init__(self, context, request):
-        self.request = request
-        self.context = context
-        self.session = request.session
-        self._rhodecode_user = request.user
-
-    def _get_template_context(self):
-        return {
-            'came_from': get_came_from(self.request),
-            'defaults': {},
-            'errors': {},
-        }
+    def load_default_context(self):
+        c = self._get_local_tmpl_context()
+        c.came_from = get_came_from(self.request)
+        self._register_global_c(c)
+        return c
 
     def _get_captcha_data(self):
         settings = SettingsModel().get_all_settings()
@@ -130,12 +126,13 b' class LoginView(object):'
         route_name='login', request_method='GET',
         renderer='rhodecode:templates/login.mako')
     def login(self):
-        came_from = get_came_from(self.request)
-        user = self.request.user
+        c = self.load_default_context()
+        auth_user = self._rhodecode_user
 
         # redirect if already logged in
-        if user.is_authenticated and not user.is_default and user.ip_allowed:
-            raise HTTPFound(came_from)
+        if (auth_user.is_authenticated and
+                not auth_user.is_default and auth_user.ip_allowed):
+            raise HTTPFound(c.came_from)
 
         # check if we use headers plugin, and try to login using it.
         try:
@@ -145,18 +142,18 b' class LoginView(object):'
             if auth_info:
                 headers = _store_user_in_session(
                     self.session, auth_info.get('username'))
-                raise HTTPFound(came_from, headers=headers)
+                raise HTTPFound(c.came_from, headers=headers)
         except UserCreationError as e:
             log.error(e)
             self.session.flash(e, queue='error')
 
-        return self._get_template_context()
+        return self._get_template_context(c)
 
     @view_config(
         route_name='login', request_method='POST',
         renderer='rhodecode:templates/login.mako')
     def login_post(self):
-        came_from = get_came_from(self.request)
+        c = self.load_default_context()
 
         login_form = LoginForm()()
 
@@ -168,13 +165,13 b' class LoginView(object):'
                 self.session,
                 username=form_result['username'],
                 remember=form_result['remember'])
-            log.debug('Redirecting to "%s" after login.', came_from)
-            raise HTTPFound(came_from, headers=headers)
+            log.debug('Redirecting to "%s" after login.', c.came_from)
+            raise HTTPFound(c.came_from, headers=headers)
         except formencode.Invalid as errors:
             defaults = errors.value
             # remove password from filling in form again
             defaults.pop('password', None)
-            render_ctx = self._get_template_context()
+            render_ctx = self._get_template_context(c)
             render_ctx.update({
                 'errors': errors.error_dict,
                 'defaults': defaults,
@@ -187,13 +184,13 b' class LoginView(object):'
             # with user creation, explanation should be provided in
             # Exception itself
             self.session.flash(e, queue='error')
-            return self._get_template_context()
+            return self._get_template_context(c)
 
     @CSRFRequired()
     @view_config(route_name='logout', request_method='POST')
     def logout(self):
-        user = self.request.user
-        log.info('Deleting session for user: `%s`', user)
+        auth_user = self._rhodecode_user
+        log.info('Deleting session for user: `%s`', auth_user)
         self.session.delete()
         return HTTPFound(url('home'))
 
@@ -203,6 +200,7 b' class LoginView(object):'
         route_name='register', request_method='GET',
         renderer='rhodecode:templates/register.mako',)
     def register(self, defaults=None, errors=None):
+        c = self.load_default_context()
         defaults = defaults or {}
         errors = errors or {}
 
@@ -212,7 +210,7 b' class LoginView(object):'
         auto_active = 'hg.register.auto_activate' in User.get_default_user()\
             .AuthUser.permissions['global']
 
-        render_ctx = self._get_template_context()
+        render_ctx = self._get_template_context(c)
         render_ctx.update({
             'defaults': defaults,
             'errors': errors,
@@ -289,17 +287,24 b' class LoginView(object):'
             'errors': {},
         }
 
+        # always send implicit message to prevent from discovery of
+        # matching emails
+        msg = _('If such email exists, a password reset link was sent to it.')
+
         if self.request.POST:
+            if h.HasPermissionAny('hg.password_reset.disabled')():
+                _email = self.request.POST.get('email', '')
+                log.error('Failed attempt to reset password for `%s`.', _email)
+                self.session.flash(_('Password reset has been disabled.'),
+                                   queue='error')
+                return HTTPFound(self.request.route_path('reset_password'))
+
             password_reset_form = PasswordResetForm()()
             try:
                 form_result = password_reset_form.to_python(
                     self.request.params)
-                if h.HasPermissionAny('hg.password_reset.disabled')():
-                    log.error('Failed attempt to reset password for %s.', form_result['email'] )
-                    self.session.flash(
-                        _('Password reset has been disabled.'),
-                        queue='error')
-                    return HTTPFound(self.request.route_path('reset_password'))
+                user_email = form_result['email']
+
                 if captcha.active:
                     response = submit(
                         self.request.params.get('recaptcha_challenge_field'),
@@ -310,43 +315,76 b' class LoginView(object):'
                         _value = form_result
                         _msg = _('Bad captcha')
                         error_dict = {'recaptcha_field': _msg}
-                        raise formencode.Invalid(_msg, _value, None,
-                                                 error_dict=error_dict)
+                        raise formencode.Invalid(
+                            _msg, _value, None, error_dict=error_dict)
 
                 # Generate reset URL and send mail.
-                user_email = form_result['email']
                 user = User.get_by_email(user_email)
+
+                # generate password reset token that expires in 10minutes
+                desc = 'Generated token for password reset from {}'.format(
+                    datetime.datetime.now().isoformat())
+                reset_token = AuthTokenModel().create(
+                    user, lifetime=10,
+                    description=desc,
+                    role=UserApiKeys.ROLE_PASSWORD_RESET)
+                Session().commit()
+
+                log.debug('Successfully created password recovery token')
                 password_reset_url = self.request.route_url(
                     'reset_password_confirmation',
-                    _query={'key': user.api_key})
+                    _query={'key': reset_token.api_key})
                 UserModel().reset_password_link(
                     form_result, password_reset_url)
-
                 # Display success message and redirect.
-                self.session.flash(
-                    _('Your password reset link was sent'),
-                    queue='success')
-                return HTTPFound(self.request.route_path('login'))
+                self.session.flash(msg, queue='success')
+                return HTTPFound(self.request.route_path('reset_password'))
 
             except formencode.Invalid as errors:
                 render_ctx.update({
                     'defaults': errors.value,
                     'errors': errors.error_dict,
                 })
+                if not self.request.params.get('email'):
+                    # case of empty email, we want to report that
+                    return render_ctx
+
+                if 'recaptcha_field' in errors.error_dict:
+                    # case of failed captcha
+                    return render_ctx
+
+            log.debug('faking response on invalid password reset')
+            # make this take 2s, to prevent brute forcing.
+            time.sleep(2)
+            self.session.flash(msg, queue='success')
+            return HTTPFound(self.request.route_path('reset_password'))
 
         return render_ctx
 
     @view_config(route_name='reset_password_confirmation',
                  request_method='GET')
     def password_reset_confirmation(self):
+
         if self.request.GET and self.request.GET.get('key'):
+            # make this take 2s, to prevent brute forcing.
+            time.sleep(2)
+
+            token = AuthTokenModel().get_auth_token(
+                self.request.GET.get('key'))
+
+            # verify token is the correct role
+            if token is None or token.role != UserApiKeys.ROLE_PASSWORD_RESET:
+                log.debug('Got token with role:%s expected is %s',
+                          getattr(token, 'role', 'EMPTY_TOKEN'),
+                          UserApiKeys.ROLE_PASSWORD_RESET)
+                self.session.flash(
+                    _('Given reset token is invalid'), queue='error')
+                return HTTPFound(self.request.route_path('reset_password'))
+
             try:
-                user = User.get_by_auth_token(self.request.GET.get('key'))
-                password_reset_url = self.request.route_url(
-                    'reset_password_confirmation',
-                    _query={'key': user.api_key})
-                data = {'email': user.email}
-                UserModel().reset_password(data, password_reset_url)
+                owner = token.user
+                data = {'email': owner.email, 'token': token.api_key}
+                UserModel().reset_password(data)
                 self.session.flash(
                     _('Your password reset was successful, '
                       'a new password has been sent to your email'),

rhodecode/apps/svn_support/tests/test_mod_dav_svn_config.py

@@ -25,7 +25,7 b' import re'
 
 from pyramid import testing
 
-from rhodecode.svn_support import utils
+from rhodecode.apps.svn_support import utils
 
 
 class TestModDavSvnConfig(object):

rhodecode/apps/svn_support/utils.py

@@ -81,7 +81,7 b' def _render_mod_dav_svn_config('
     }
 
     # Render the configuration template to string.
-    template = 'rhodecode:svn_support/templates/mod-dav-svn.conf.mako'
+    template = 'rhodecode:apps/svn_support/templates/mod-dav-svn.conf.mako'
     return render(template, context)
 
 

rhodecode/authentication/base.py

@@ -91,6 +91,11 b' class RhodeCodeAuthPluginBase(object):'
     # set on authenticate() method and via set_auth_type func.
     auth_type = None
 
+    # set on authenticate() method and via set_calling_scope_repo, this is a
+    # calling scope repository when doing authentication most likely on VCS
+    # operations
+    acl_repo_name = None
+
     # List of setting names to store encrypted. Plugins may override this list
     # to store settings encrypted.
     _settings_encrypted = []
@@ -268,6 +273,9 b' class RhodeCodeAuthPluginBase(object):'
     def set_auth_type(self, auth_type):
         self.auth_type = auth_type
 
+    def set_calling_scope_repo(self, acl_repo_name):
+        self.acl_repo_name = acl_repo_name
+
     def allows_authentication_from(
             self, user, allows_non_existing_user=True,
             allowed_auth_plugins=None, allowed_auth_sources=None):
@@ -332,6 +340,8 b' class RhodeCodeAuthPluginBase(object):'
             log.debug('provided username:`%s` is empty skipping...', username)
         if not user:
             log.debug('User `%s` not found in database', username)
+        else:
+            log.debug('Got DB user:%s', user)
         return user
 
     def user_activation_state(self):
@@ -518,7 +528,7 b' def get_auth_cache_manager(custom_ttl=No'
 
 
 def authenticate(username, password, environ=None, auth_type=None,
-                 skip_missing=False, registry=None):
+                 skip_missing=False, registry=None, acl_repo_name=None):
     """
     Authentication function used for access control,
     It tries to authenticate based on enabled authentication modules.
@@ -538,6 +548,7 b' def authenticate(username, password, env'
     authn_registry = get_authn_registry(registry)
     for plugin in authn_registry.get_plugins_for_authentication():
         plugin.set_auth_type(auth_type)
+        plugin.set_calling_scope_repo(acl_repo_name)
         user = plugin.get_user(username)
         display_user = user.username if user else username
 
@@ -627,3 +638,21 b' def authenticate(username, password, env'
         log.debug("User `%s` failed to authenticate against %s",
                   display_user, plugin.get_id())
     return None
+
+
+def chop_at(s, sub, inclusive=False):
+    """Truncate string ``s`` at the first occurrence of ``sub``.
+
+    If ``inclusive`` is true, truncate just after ``sub`` rather than at it.
+
+    >>> chop_at("plutocratic brats", "rat")
+    'plutoc'
+    >>> chop_at("plutocratic brats", "rat", True)
+    'plutocrat'
+    """
+    pos = s.find(sub)
+    if pos == -1:
+        return s
+    if inclusive:
+        return s[:pos+len(sub)]
+    return s[:pos]

rhodecode/authentication/plugins/auth_crowd.py

@@ -28,10 +28,9 b' import base64'
 import logging
 import urllib2
 
-from pylons.i18n.translation import lazy_ugettext as _
-from sqlalchemy.ext.hybrid import hybrid_property
-
-from rhodecode.authentication.base import RhodeCodeExternalAuthPlugin
+from rhodecode.translation import _
+from rhodecode.authentication.base import (
+    RhodeCodeExternalAuthPlugin, hybrid_property)
 from rhodecode.authentication.schema import AuthnPluginSettingsSchemaBase
 from rhodecode.authentication.routes import AuthnPluginResourceBase
 from rhodecode.lib.colander_utils import strip_whitespace

rhodecode/authentication/plugins/auth_headers.py

@@ -21,15 +21,14 b''
 import colander
 import logging
 
-from sqlalchemy.ext.hybrid import hybrid_property
-
-from rhodecode.authentication.base import RhodeCodeExternalAuthPlugin
+from rhodecode.translation import _
+from rhodecode.authentication.base import (
+    RhodeCodeExternalAuthPlugin, hybrid_property)
 from rhodecode.authentication.schema import AuthnPluginSettingsSchemaBase
 from rhodecode.authentication.routes import AuthnPluginResourceBase
 from rhodecode.lib.colander_utils import strip_whitespace
 from rhodecode.lib.utils2 import str2bool, safe_unicode
 from rhodecode.model.db import User
-from rhodecode.translation import _
 
 
 log = logging.getLogger(__name__)

rhodecode/authentication/plugins/auth_jasig_cas.py

@@ -30,10 +30,9 b' import rhodecode'
 import urllib
 import urllib2
 
-from pylons.i18n.translation import lazy_ugettext as _
-from sqlalchemy.ext.hybrid import hybrid_property
-
-from rhodecode.authentication.base import RhodeCodeExternalAuthPlugin
+from rhodecode.translation import _
+from rhodecode.authentication.base import (
+    RhodeCodeExternalAuthPlugin, hybrid_property)
 from rhodecode.authentication.schema import AuthnPluginSettingsSchemaBase
 from rhodecode.authentication.routes import AuthnPluginResourceBase
 from rhodecode.lib.colander_utils import strip_whitespace

rhodecode/authentication/plugins/auth_ldap.py

@@ -27,10 +27,9 b' import colander'
 import logging
 import traceback
 
-from pylons.i18n.translation import lazy_ugettext as _
-from sqlalchemy.ext.hybrid import hybrid_property
-
-from rhodecode.authentication.base import RhodeCodeExternalAuthPlugin
+from rhodecode.translation import _
+from rhodecode.authentication.base import (
+    RhodeCodeExternalAuthPlugin, chop_at, hybrid_property)
 from rhodecode.authentication.schema import AuthnPluginSettingsSchemaBase
 from rhodecode.authentication.routes import AuthnPluginResourceBase
 from rhodecode.lib.colander_utils import strip_whitespace
@@ -72,14 +71,15 b' class LdapSettingsSchema(AuthnPluginSett'
     host = colander.SchemaNode(
         colander.String(),
         default='',
-        description=_('Host of the LDAP Server'),
+        description=_('Host of the LDAP Server \n'
+                      '(e.g., 192.168.2.154, or ldap-server.domain.com'),
         preparer=strip_whitespace,
         title=_('LDAP Host'),
         widget='string')
     port = colander.SchemaNode(
         colander.Int(),
         default=389,
-        description=_('Port that the LDAP server is listening on'),
+        description=_('Custom port that the LDAP server is listening on. Default: 389'),
         preparer=strip_whitespace,
         title=_('Port'),
         validator=colander.Range(min=0, max=65536),
@@ -87,7 +87,9 b' class LdapSettingsSchema(AuthnPluginSett'
     dn_user = colander.SchemaNode(
         colander.String(),
         default='',
-        description=_('User to connect to LDAP'),
+        description=_('Optional user DN/account to connect to LDAP if authentication is required. \n'
+                      'e.g., cn=admin,dc=mydomain,dc=com, or '
+                      'uid=root,cn=users,dc=mydomain,dc=com, or [e-mail]'),
         missing='',
         preparer=strip_whitespace,
         title=_('Account'),
@@ -95,7 +97,7 b' class LdapSettingsSchema(AuthnPluginSett'
     dn_pass = colander.SchemaNode(
         colander.String(),
         default='',
-        description=_('Password to connect to LDAP'),
+        description=_('Password to authenticate for given user DN.'),
         missing='',
         preparer=strip_whitespace,
         title=_('Password'),
@@ -117,7 +119,9 b' class LdapSettingsSchema(AuthnPluginSett'
     base_dn = colander.SchemaNode(
         colander.String(),
         default='',
-        description=_('Base DN to search (e.g., dc=mydomain,dc=com)'),
+        description=_('Base DN to search. Dynamic bind is supported. Add `$login` marker '
+                      'in it to be replaced with current user credentials \n'
+                      '(e.g., dc=mydomain,dc=com, or ou=Users,dc=mydomain,dc=com)'),
         missing='',
         preparer=strip_whitespace,
         title=_('Base DN'),
@@ -125,22 +129,25 b' class LdapSettingsSchema(AuthnPluginSett'
     filter = colander.SchemaNode(
         colander.String(),
         default='',
-        description=_('Filter to narrow results (e.g., ou=Users, etc)'),
+        description=_('Filter to narrow results \n'
+                      '(e.g., (&(objectCategory=Person)(objectClass=user)), or \n'
+                      '(memberof=cn=rc-login,ou=groups,ou=company,dc=mydomain,dc=com)))'),
         missing='',
         preparer=strip_whitespace,
         title=_('LDAP Search Filter'),
         widget='string')
+
     search_scope = colander.SchemaNode(
         colander.String(),
-        default=search_scope_choices[0],
-        description=_('How deep to search LDAP'),
+        default=search_scope_choices[2],
+        description=_('How deep to search LDAP. If unsure set to SUBTREE'),
         title=_('LDAP Search Scope'),
         validator=colander.OneOf(search_scope_choices),
         widget='select')
     attr_login = colander.SchemaNode(
         colander.String(),
-        default='',
-        description=_('LDAP Attribute to map to user name'),
+        default='uid',
+        description=_('LDAP Attribute to map to user name (e.g., uid, or sAMAccountName)'),
         preparer=strip_whitespace,
         title=_('Login Attribute'),
         missing_msg=_('The LDAP Login attribute of the CN must be specified'),
@@ -148,7 +155,7 b' class LdapSettingsSchema(AuthnPluginSett'
     attr_firstname = colander.SchemaNode(
         colander.String(),
         default='',
-        description=_('LDAP Attribute to map to first name'),
+        description=_('LDAP Attribute to map to first name (e.g., givenName)'),
         missing='',
         preparer=strip_whitespace,
         title=_('First Name Attribute'),
@@ -156,7 +163,7 b' class LdapSettingsSchema(AuthnPluginSett'
     attr_lastname = colander.SchemaNode(
         colander.String(),
         default='',
-        description=_('LDAP Attribute to map to last name'),
+        description=_('LDAP Attribute to map to last name (e.g., sn)'),
         missing='',
         preparer=strip_whitespace,
         title=_('Last Name Attribute'),
@@ -164,7 +171,9 b' class LdapSettingsSchema(AuthnPluginSett'
     attr_email = colander.SchemaNode(
         colander.String(),
         default='',
-        description=_('LDAP Attribute to map to email address'),
+        description=_('LDAP Attribute to map to email address (e.g., mail).\n'
+                      'Emails are a crucial part of RhodeCode. \n'
+                      'If possible add a valid email attribute to ldap users.'),
         missing='',
         preparer=strip_whitespace,
         title=_('Email Attribute'),
@@ -182,7 +191,7 b' class AuthLdap(object):'
     def __init__(self, server, base_dn, port=389, bind_dn='', bind_pass='',
                  tls_kind='PLAIN', tls_reqcert='DEMAND', ldap_version=3,
                  search_scope='SUBTREE', attr_login='uid',
-                 ldap_filter='(&(objectClass=user)(!(objectClass=computer)))'):
+                 ldap_filter=None):
         if ldap == Missing:
             raise LdapImportError("Missing or incompatible ldap library")
 
@@ -236,14 +245,13 b' class AuthLdap(object):'
             server.start_tls_s()
 
         if self.LDAP_BIND_DN and self.LDAP_BIND_PASS:
-            log.debug('Trying simple_bind with password and given DN: %s',
+            log.debug('Trying simple_bind with password and given login DN: %s',
                       self.LDAP_BIND_DN)
             server.simple_bind_s(self.LDAP_BIND_DN, self.LDAP_BIND_PASS)
 
         return server
 
     def get_uid(self, username):
-        from rhodecode.lib.helpers import chop_at
         uid = username
         for server_addr in self.SERVER_ADDRESSES:
             uid = chop_at(username, "@%s" % server_addr)
@@ -292,8 +300,11 b' class AuthLdap(object):'
                 self.BASE_DN, self.SEARCH_SCOPE, filter_)
 
             if not lobjects:
+                log.debug("No matching LDAP objects for authentication "
+                          "of UID:'%s' username:(%s)", uid, username)
                 raise ldap.NO_SUCH_OBJECT()
 
+            log.debug('Found matching ldap object, trying to authenticate')
             for (dn, _attrs) in lobjects:
                 if dn is None:
                     continue
@@ -304,15 +315,13 b' class AuthLdap(object):'
                     break
 
             else:
-                log.debug("No matching LDAP objects for authentication "
-                          "of '%s' (%s)", uid, username)
                 raise LdapPasswordError('Failed to authenticate user '
                                         'with given password')
 
         except ldap.NO_SUCH_OBJECT:
             log.debug("LDAP says no such user '%s' (%s), org_exc:",
                       uid, username, exc_info=True)
-            raise LdapUsernameError()
+            raise LdapUsernameError('Unable to find user')
         except ldap.SERVER_DOWN:
             org_exc = traceback.format_exc()
             raise LdapConnectionError(
@@ -447,7 +456,7 b' class RhodeCodeAuthPlugin(RhodeCodeExter'
                 'email': get_ldap_attr('attr_email') or email,
                 'admin': admin,
                 'active': active,
-                "active_from_extern": None,
+                'active_from_extern': None,
                 'extern_name': user_dn,
                 'extern_type': extern_type,
             }

rhodecode/authentication/plugins/auth_pam.py

@@ -17,6 +17,7 b''
 # This program is dual-licensed. If you wish to learn more about the
 # RhodeCode Enterprise Edition, including its added features, Support services,
 # and proprietary license terms, please see https://rhodecode.com/licenses/
+
 """
 RhodeCode authentication library for PAM
 """
@@ -29,10 +30,9 b' import pwd'
 import re
 import socket
 
-from pylons.i18n.translation import lazy_ugettext as _
-from sqlalchemy.ext.hybrid import hybrid_property
-
-from rhodecode.authentication.base import RhodeCodeExternalAuthPlugin
+from rhodecode.translation import _
+from rhodecode.authentication.base import (
+    RhodeCodeExternalAuthPlugin, hybrid_property)
 from rhodecode.authentication.schema import AuthnPluginSettingsSchemaBase
 from rhodecode.authentication.routes import AuthnPluginResourceBase
 from rhodecode.lib.colander_utils import strip_whitespace

rhodecode/authentication/plugins/auth_rhodecode.py

@@ -25,9 +25,8 b' RhodeCode authentication plugin for buil'
 import logging
 
 from pylons.i18n.translation import lazy_ugettext as _
-from sqlalchemy.ext.hybrid import hybrid_property
 
-from rhodecode.authentication.base import RhodeCodeAuthPluginBase
+from rhodecode.authentication.base import RhodeCodeAuthPluginBase, hybrid_property
 from rhodecode.authentication.routes import AuthnPluginResourceBase
 from rhodecode.lib.utils2 import safe_str
 from rhodecode.model.db import User

rhodecode/authentication/plugins/auth_token.py

@@ -24,12 +24,11 b' RhodeCode authentication token plugin fo'
 
 import logging
 
-from sqlalchemy.ext.hybrid import hybrid_property
-
 from rhodecode.translation import _
-from rhodecode.authentication.base import RhodeCodeAuthPluginBase, VCS_TYPE
+from rhodecode.authentication.base import (
+    RhodeCodeAuthPluginBase, VCS_TYPE, hybrid_property)
 from rhodecode.authentication.routes import AuthnPluginResourceBase
-from rhodecode.model.db import User, UserApiKeys
+from rhodecode.model.db import User, UserApiKeys, Repository
 
 
 log = logging.getLogger(__name__)
@@ -122,10 +121,17 b' class RhodeCodeAuthPlugin(RhodeCodeAuthP'
 
         log.debug('Authenticating user with args %s', user_attrs)
         if userobj.active:
-            role = UserApiKeys.ROLE_VCS
-            active_tokens = [x.api_key for x in
-                             User.extra_valid_auth_tokens(userobj, role=role)]
-            if userobj.username == username and password in active_tokens:
+            # calling context repo for token scopes
+            scope_repo_id = None
+            if self.acl_repo_name:
+                repo = Repository.get_by_repo_name(self.acl_repo_name)
+                scope_repo_id = repo.repo_id if repo else None
+
+            token_match = userobj.authenticate_by_token(
+                password, roles=[UserApiKeys.ROLE_VCS],
+                scope_repo_id=scope_repo_id)
+
+            if userobj.username == username and token_match:
                 log.info(
                     'user `%s` successfully authenticated via %s',
                     user_attrs['username'], self.name)

rhodecode/authentication/schema.py

@@ -40,10 +40,11 b' class AuthnPluginSettingsSchemaBase(cola'
     cache_ttl = colander.SchemaNode(
         colander.Int(),
         default=0,
-        description=_('Amount of seconds to cache the authentication '
-                      'call for this plugin. Useful for long calls like '
-                      'LDAP to improve the responsiveness of the '
-                      'authentication system (0 means disabled).'),
+        description=_('Amount of seconds to cache the authentication response'
+                      'call for this plugin. \n'
+                      'Useful for long calls like LDAP to improve the '
+                      'performance of the authentication system '
+                      '(0 means disabled).'),
         missing=0,
         title=_('Auth Cache TTL'),
         validator=colander.Range(min=0, max=None),

rhodecode/config/environment.py

@@ -87,15 +87,6 b' def load_environment(global_conf, app_co'
 
     config['routes.map'] = make_map(config)
 
-    if asbool(config.get('generate_js_files', 'false')):
-        jsroutes = config['routes.map'].jsroutes()
-        jsroutes_file_content = generate_jsroutes_content(jsroutes)
-        jsroutes_file_path = os.path.join(
-            paths['static_files'], 'js', 'rhodecode', 'routes.js')
-
-        with io.open(jsroutes_file_path, 'w', encoding='utf-8') as f:
-            f.write(jsroutes_file_content)
-
     config['pylons.app_globals'] = app_globals.Globals(config)
     config['pylons.h'] = helpers
     rhodecode.CONFIG = config
@@ -184,7 +175,6 b' def load_pyramid_environment(global_conf'
                          protocol=utils.get_vcs_server_protocol(settings),
                          log_level=settings['vcs.server.log_level'])
 
-    utils.configure_pyro4(settings)
     utils.configure_vcs(settings)
     if vcs_server_enabled:
         connect_vcs(vcs_server_uri, utils.get_vcs_server_protocol(settings))

rhodecode/config/licenses.json

@@ -40,10 +40,7 b''
   }, 
   "python2.7-Pylons-1.0.1-patch1": {
     "BSD 4-clause \"Original\" or \"Old\" License": "http://spdx.org/licenses/BSD-4-Clause"
-  }, 
-  "python2.7-Pyro4-4.35": {
-    "MIT License": "http://spdx.org/licenses/MIT"
-  }, 
+  },
   "python2.7-Routes-1.13": {
     "BSD 4-clause \"Original\" or \"Old\" License": "http://spdx.org/licenses/BSD-4-Clause"
   }, 
@@ -214,10 +211,7 b''
   }, 
   "python2.7-requests-2.9.1": {
     "Apache License 2.0": "http://spdx.org/licenses/Apache-2.0"
-  }, 
-  "python2.7-serpent-1.12": {
-    "MIT License": "http://spdx.org/licenses/MIT"
-  }, 
+  },
   "python2.7-setuptools-19.4": {
     "Python Software Foundation License version 2": "http://spdx.org/licenses/Python-2.0", 
     "Zope Public License 2.0": "http://spdx.org/licenses/ZPL-2.0"

rhodecode/config/middleware.py

@@ -32,7 +32,7 b' from pyramid.config import Configurator'
 from pyramid.settings import asbool, aslist
 from pyramid.wsgi import wsgiapp
 from pyramid.httpexceptions import (
-    HTTPError, HTTPInternalServerError, HTTPFound)
+    HTTPException, HTTPError, HTTPInternalServerError, HTTPFound)
 from pyramid.events import ApplicationCreated
 from pyramid.renderers import render_to_response
 from routes.middleware import RoutesMiddleware
@@ -53,7 +53,8 b' from rhodecode.lib.middleware.vcs import'
 from rhodecode.lib.plugins.utils import register_rhodecode_plugin
 from rhodecode.lib.utils2 import aslist as rhodecode_aslist
 from rhodecode.subscribers import (
-    scan_repositories_if_enabled, write_metadata_if_needed)
+    scan_repositories_if_enabled, write_metadata_if_needed,
+    write_js_routes_if_enabled)
 
 
 log = logging.getLogger(__name__)
@@ -219,18 +220,14 b' def add_pylons_compat_data(registry, glo'
 
 
 def error_handler(exception, request):
-    from rhodecode.model.settings import SettingsModel
+    import rhodecode
     from rhodecode.lib.utils2 import AttributeDict
 
-    try:
-        rc_config = SettingsModel().get_all_settings()
-    except Exception:
-        log.exception('failed to fetch settings')
-        rc_config = {}
+    rhodecode_title = rhodecode.CONFIG.get('rhodecode_title') or 'RhodeCode'
 
     base_response = HTTPInternalServerError()
     # prefer original exception for the response since it may have headers set
-    if isinstance(exception, HTTPError):
+    if isinstance(exception, HTTPException):
         base_response = exception
 
     def is_http_error(response):
@@ -251,7 +248,7 b' def error_handler(exception, request):'
         request.route_url('rhodecode_support')
     )
     c.redirect_time = 0
-    c.rhodecode_name = rc_config.get('rhodecode_title', '')
+    c.rhodecode_name = rhodecode_title
     if not c.rhodecode_name:
         c.rhodecode_name = 'Rhodecode'
 
@@ -281,14 +278,24 b' def includeme(config):'
     # Includes which are required. The application would fail without them.
     config.include('pyramid_mako')
     config.include('pyramid_beaker')
-    config.include('rhodecode.channelstream')
-    config.include('rhodecode.admin')
+
     config.include('rhodecode.authentication')
     config.include('rhodecode.integrations')
-    config.include('rhodecode.login')
+
+    # apps
+    config.include('rhodecode.apps._base')
+
+    config.include('rhodecode.apps.admin')
+    config.include('rhodecode.apps.channelstream')
+    config.include('rhodecode.apps.login')
+    config.include('rhodecode.apps.repository')
+    config.include('rhodecode.apps.user_profile')
+    config.include('rhodecode.apps.my_account')
+    config.include('rhodecode.apps.svn_support')
+
     config.include('rhodecode.tweens')
     config.include('rhodecode.api')
-    config.include('rhodecode.svn_support')
+
     config.add_route(
         'rhodecode_support', 'https://rhodecode.com/help/', static=True)
 
@@ -298,6 +305,7 b' def includeme(config):'
     # Add subscribers.
     config.add_subscriber(scan_repositories_if_enabled, ApplicationCreated)
     config.add_subscriber(write_metadata_if_needed, ApplicationCreated)
+    config.add_subscriber(write_js_routes_if_enabled, ApplicationCreated)
 
     # Set the authorization policy.
     authz_policy = ACLAuthorizationPolicy()

rhodecode/config/routing.py

@@ -92,7 +92,7 b' class JSRoutesMapper(Mapper):'
     def _extract_route_information(self, route):
         """
         Convert a route into tuple(name, path, args), eg:
-            ('user_profile', '/profile/%(username)s', ['username'])
+            ('show_user', '/profile/%(username)s', ['username'])
         """
         routepath = route.routepath
         def replace(matchobj):
@@ -198,10 +198,6 b' def make_map(config):'
     rmap.connect('user_group_autocomplete_data', '/_user_groups', controller='home',
                  action='user_group_autocomplete_data', jsroute=True)
 
-    rmap.connect(
-        'user_profile', '/_profiles/{username}', controller='users',
-        action='user_profile')
-
     # TODO: johbo: Static links, to be replaced by our redirection mechanism
     rmap.connect('rst_help',
                  'http://docutils.sourceforge.net/docs/user/rst/quickref.html',
@@ -296,8 +292,6 b' def make_map(config):'
                         controller='admin/users') as m:
         m.connect('users', '/users',
                   action='create', conditions={'method': ['POST']})
-        m.connect('users', '/users',
-                  action='index', conditions={'method': ['GET']})
         m.connect('new_user', '/users/new',
                   action='new', conditions={'method': ['GET']})
         m.connect('update_user', '/users/{user_id}',
@@ -319,13 +313,6 b' def make_map(config):'
         m.connect('edit_user_advanced', '/users/{user_id}/edit/advanced',
                   action='update_advanced', conditions={'method': ['PUT']})
 
-        m.connect('edit_user_auth_tokens', '/users/{user_id}/edit/auth_tokens',
-                  action='edit_auth_tokens', conditions={'method': ['GET']})
-        m.connect('edit_user_auth_tokens', '/users/{user_id}/edit/auth_tokens',
-                  action='add_auth_token', conditions={'method': ['PUT']})
-        m.connect('edit_user_auth_tokens', '/users/{user_id}/edit/auth_tokens',
-                  action='delete_auth_token', conditions={'method': ['DELETE']})
-
         m.connect('edit_user_global_perms', '/users/{user_id}/edit/global_permissions',
                   action='edit_global_perms', conditions={'method': ['GET']})
         m.connect('edit_user_global_perms', '/users/{user_id}/edit/global_permissions',
@@ -387,6 +374,10 b' def make_map(config):'
                   '/user_groups/{user_group_id}/edit/advanced',
                   action='edit_advanced', conditions={'method': ['GET']})
 
+        m.connect('edit_user_group_advanced_sync',
+                  '/user_groups/{user_group_id}/edit/advanced/sync',
+                  action='edit_advanced_set_synchronization', conditions={'method': ['POST']})
+
         m.connect('edit_user_group_members',
                   '/user_groups/{user_group_id}/edit/members', jsroute=True,
                   action='user_group_members', conditions={'method': ['GET']})
@@ -518,15 +509,15 b' def make_map(config):'
     with rmap.submapper(path_prefix=ADMIN_PREFIX,
                         controller='admin/my_account') as m:
 
-        m.connect('my_account', '/my_account',
-                  action='my_account', conditions={'method': ['GET']})
         m.connect('my_account_edit', '/my_account/edit',
                   action='my_account_edit', conditions={'method': ['GET']})
-        m.connect('my_account', '/my_account',
+        m.connect('my_account', '/my_account/update',
                   action='my_account_update', conditions={'method': ['POST']})
 
+        # NOTE(marcink): this needs to be kept for password force flag to be
+        # handler, remove after migration to pyramid
         m.connect('my_account_password', '/my_account/password',
-                  action='my_account_password', conditions={'method': ['GET', 'POST']})
+                  action='my_account_password', conditions={'method': ['GET']})
 
         m.connect('my_account_repos', '/my_account/repos',
                   action='my_account_repos', conditions={'method': ['GET']})
@@ -547,12 +538,6 b' def make_map(config):'
         m.connect('my_account_emails', '/my_account/emails',
                   action='my_account_emails_delete', conditions={'method': ['DELETE']})
 
-        m.connect('my_account_auth_tokens', '/my_account/auth_tokens',
-                  action='my_account_auth_tokens', conditions={'method': ['GET']})
-        m.connect('my_account_auth_tokens', '/my_account/auth_tokens',
-                  action='my_account_auth_tokens_add', conditions={'method': ['POST']})
-        m.connect('my_account_auth_tokens', '/my_account/auth_tokens',
-                  action='my_account_auth_tokens_delete', conditions={'method': ['DELETE']})
         m.connect('my_account_notifications', '/my_account/notifications',
                   action='my_notifications',
                   conditions={'method': ['GET']})
@@ -1066,6 +1051,12 b' def make_map(config):'
                  f_path='', annotate=True, conditions={'function': check_repo},
                  requirements=URL_NAME_REQUIREMENTS, jsroute=True)
 
+    rmap.connect('files_annotate_previous',
+                 '/{repo_name}/annotate-previous/{revision}/{f_path}',
+                 controller='files', action='annotate_previous', revision='tip',
+                 f_path='', annotate=True, conditions={'function': check_repo},
+                 requirements=URL_NAME_REQUIREMENTS, jsroute=True)
+
     rmap.connect('files_edit',
                  '/{repo_name}/edit/{revision}/{f_path}',
                  controller='files', action='edit', revision='tip',
@@ -1137,11 +1128,6 b' def make_map(config):'
                  conditions={'function': check_repo},
                  requirements=URL_NAME_REQUIREMENTS)
 
-    rmap.connect('repo_followers_home', '/{repo_name}/followers',
-                 controller='followers', action='followers',
-                 conditions={'function': check_repo},
-                 requirements=URL_NAME_REQUIREMENTS)
-
     # must be here for proper group/repo catching pattern
     _connect_with_slash(
         rmap, 'repo_group_home', '/{group_name}',

rhodecode/config/utils.py

@@ -20,29 +20,11 b''
 
 import os
 import shlex
-import Pyro4
 import platform
 
 from rhodecode.model import init_model
 
 
-def configure_pyro4(config):
-    """
-    Configure Pyro4 based on `config`.
-
-    This will mainly set the different configuration parameters of the Pyro4
-    library based on the settings in our INI files. The Pyro4 documentation
-    lists more details about the specific settings and their meaning.
-    """
-    Pyro4.config.COMMTIMEOUT = float(config['vcs.connection_timeout'])
-    Pyro4.config.SERIALIZER = 'pickle'
-    Pyro4.config.SERIALIZERS_ACCEPTED.add('pickle')
-
-    # Note: We need server configuration in the WSGI processes
-    # because we provide a callback server in certain vcs operations.
-    Pyro4.config.SERVERTYPE = "multiplex"
-    Pyro4.config.POLLTIMEOUT = 0.01
-
 
 def configure_vcs(config):
     """
@@ -77,9 +59,16 b' def initialize_test_environment(settings'
         create_test_directory, create_test_database, create_test_repositories,
         create_test_index)
     from rhodecode.tests import TESTS_TMP_PATH
+    from rhodecode.lib.vcs.backends.hg import largefiles_store
+    from rhodecode.lib.vcs.backends.git import lfs_store
+
     # test repos
     if test_env:
         create_test_directory(TESTS_TMP_PATH)
+        # large object stores
+        create_test_directory(largefiles_store(TESTS_TMP_PATH))
+        create_test_directory(lfs_store(TESTS_TMP_PATH))
+
         create_test_database(TESTS_TMP_PATH, settings)
         create_test_repositories(TESTS_TMP_PATH, settings)
         create_test_index(TESTS_TMP_PATH, settings)

rhodecode/controllers/admin/admin.py

@@ -28,101 +28,17 b' import logging'
 from pylons import request, tmpl_context as c, url
 from pylons.controllers.util import redirect
 from sqlalchemy.orm import joinedload
-from whoosh.qparser.default import QueryParser, query
-from whoosh.qparser.dateparse import DateParserPlugin
-from whoosh.fields import (TEXT, Schema, DATETIME)
-from sqlalchemy.sql.expression import or_, and_, func
 
 from rhodecode.model.db import UserLog, PullRequest
+from rhodecode.lib.user_log_filter import user_log_filter
 from rhodecode.lib.auth import LoginRequired, HasPermissionAllDecorator
 from rhodecode.lib.base import BaseController, render
-from rhodecode.lib.utils2 import safe_int, remove_prefix, remove_suffix
+from rhodecode.lib.utils2 import safe_int
 from rhodecode.lib.helpers import Page
 
 
 log = logging.getLogger(__name__)
 
-# JOURNAL SCHEMA used only to generate queries in journal. We use whoosh
-# querylang to build sql queries and filter journals
-JOURNAL_SCHEMA = Schema(
-    username=TEXT(),
-    date=DATETIME(),
-    action=TEXT(),
-    repository=TEXT(),
-    ip=TEXT(),
-)
-
-
-def _journal_filter(user_log, search_term):
-    """
-    Filters sqlalchemy user_log based on search_term with whoosh Query language
-    http://packages.python.org/Whoosh/querylang.html
-
-    :param user_log:
-    :param search_term:
-    """
-    log.debug('Initial search term: %r' % search_term)
-    qry = None
-    if search_term:
-        qp = QueryParser('repository', schema=JOURNAL_SCHEMA)
-        qp.add_plugin(DateParserPlugin())
-        qry = qp.parse(unicode(search_term))
-        log.debug('Filtering using parsed query %r' % qry)
-
-    def wildcard_handler(col, wc_term):
-        if wc_term.startswith('*') and not wc_term.endswith('*'):
-            # postfix == endswith
-            wc_term = remove_prefix(wc_term, prefix='*')
-            return func.lower(col).endswith(wc_term)
-        elif wc_term.startswith('*') and wc_term.endswith('*'):
-            # wildcard == ilike
-            wc_term = remove_prefix(wc_term, prefix='*')
-            wc_term = remove_suffix(wc_term, suffix='*')
-            return func.lower(col).contains(wc_term)
-
-    def get_filterion(field, val, term):
-
-        if field == 'repository':
-            field = getattr(UserLog, 'repository_name')
-        elif field == 'ip':
-            field = getattr(UserLog, 'user_ip')
-        elif field == 'date':
-            field = getattr(UserLog, 'action_date')
-        elif field == 'username':
-            field = getattr(UserLog, 'username')
-        else:
-            field = getattr(UserLog, field)
-        log.debug('filter field: %s val=>%s' % (field, val))
-
-        # sql filtering
-        if isinstance(term, query.Wildcard):
-            return wildcard_handler(field, val)
-        elif isinstance(term, query.Prefix):
-            return func.lower(field).startswith(func.lower(val))
-        elif isinstance(term, query.DateRange):
-            return and_(field >= val[0], field <= val[1])
-        return func.lower(field) == func.lower(val)
-
-    if isinstance(qry, (query.And, query.Term, query.Prefix, query.Wildcard,
-                        query.DateRange)):
-        if not isinstance(qry, query.And):
-            qry = [qry]
-        for term in qry:
-            field = term.fieldname
-            val = (term.text if not isinstance(term, query.DateRange)
-                   else [term.startdate, term.enddate])
-            user_log = user_log.filter(get_filterion(field, val, term))
-    elif isinstance(qry, query.Or):
-        filters = []
-        for term in qry:
-            field = term.fieldname
-            val = (term.text if not isinstance(term, query.DateRange)
-                   else [term.startdate, term.enddate])
-            filters.append(get_filterion(field, val, term))
-        user_log = user_log.filter(or_(*filters))
-
-    return user_log
-
 
 class AdminController(BaseController):
 
@@ -139,7 +55,7 b' class AdminController(BaseController):'
         # FILTERING
         c.search_term = request.GET.get('filter')
         try:
-            users_log = _journal_filter(users_log, c.search_term)
+            users_log = user_log_filter(users_log, c.search_term)
         except Exception:
             # we want this to crash for now
             raise

rhodecode/controllers/admin/my_account.py

@@ -29,32 +29,30 b' import datetime'
 import formencode
 from formencode import htmlfill
 from pyramid.threadlocal import get_current_registry
-from pylons import request, tmpl_context as c, url, session
+from pyramid.httpexceptions import HTTPFound
+
+from pylons import request, tmpl_context as c, url
 from pylons.controllers.util import redirect
 from pylons.i18n.translation import _
 from sqlalchemy.orm import joinedload
-from webob.exc import HTTPBadGateway
 
-from rhodecode import forms
 from rhodecode.lib import helpers as h
 from rhodecode.lib import auth
 from rhodecode.lib.auth import (
-    LoginRequired, NotAnonymous, AuthUser, generate_auth_token)
+    LoginRequired, NotAnonymous, AuthUser)
 from rhodecode.lib.base import BaseController, render
 from rhodecode.lib.utils import jsonify
-from rhodecode.lib.utils2 import safe_int, md5, str2bool
+from rhodecode.lib.utils2 import safe_int, str2bool
 from rhodecode.lib.ext_json import json
 from rhodecode.lib.channelstream import channelstream_request, \
     ChannelstreamException
 
-from rhodecode.model.validation_schema.schemas import user_schema
 from rhodecode.model.db import (
     Repository, PullRequest, UserEmailMap, User, UserFollowing)
 from rhodecode.model.forms import UserForm
 from rhodecode.model.scm import RepoList
 from rhodecode.model.user import UserModel
 from rhodecode.model.repo import RepoModel
-from rhodecode.model.auth_token import AuthTokenModel
 from rhodecode.model.meta import Session
 from rhodecode.model.pull_request import PullRequestModel
 from rhodecode.model.comment import CommentsModel
@@ -79,7 +77,7 b' class MyAccountController(BaseController'
         if c.user.username == User.DEFAULT_USER:
             h.flash(_("You can't edit this user since it's"
                       " crucial for entire application"), category='warning')
-            return redirect(url('users'))
+            return redirect(h.route_path('users'))
 
         c.auth_user = AuthUser(
             user_id=c.rhodecode_user.user_id, ip_addr=self.ip_addr)
@@ -156,7 +154,7 b' class MyAccountController(BaseController'
                     % form_result.get('username'), category='error')
 
         if update:
-            return redirect('my_account')
+            raise HTTPFound(h.route_path('my_account_profile'))
 
         return htmlfill.render(
             render('admin/my_account/my_account.mako'),
@@ -165,19 +163,6 b' class MyAccountController(BaseController'
             force_defaults=False
         )
 
-    def my_account(self):
-        """
-        GET /_admin/my_account Displays info about my account
-        """
-        # url('my_account')
-        c.active = 'profile'
-        self.__load_data()
-
-        defaults = c.user.get_dict()
-        return htmlfill.render(
-            render('admin/my_account/my_account.mako'),
-            defaults=defaults, encoding="UTF-8", force_defaults=False)
-
     def my_account_edit(self):
         """
         GET /_admin/my_account/edit Displays edit form of my account
@@ -196,47 +181,6 b' class MyAccountController(BaseController'
             force_defaults=False
         )
 
-    @auth.CSRFRequired(except_methods=['GET'])
-    def my_account_password(self):
-        c.active = 'password'
-        self.__load_data()
-        c.extern_type = c.user.extern_type
-
-        schema = user_schema.ChangePasswordSchema().bind(
-            username=c.rhodecode_user.username)
-
-        form = forms.Form(schema,
-            buttons=(forms.buttons.save, forms.buttons.reset))
-
-        if request.method == 'POST' and c.extern_type == 'rhodecode':
-            controls = request.POST.items()
-            try:
-                valid_data = form.validate(controls)
-                UserModel().update_user(c.rhodecode_user.user_id, **valid_data)
-                instance = c.rhodecode_user.get_instance()
-                instance.update_userdata(force_password_change=False)
-                Session().commit()
-            except forms.ValidationFailure as e:
-                request.session.flash(
-                    _('Error occurred during update of user password'),
-                    queue='error')
-                form = e
-            except Exception:
-                log.exception("Exception updating password")
-                request.session.flash(
-                    _('Error occurred during update of user password'),
-                    queue='error')
-            else:
-                session.setdefault('rhodecode_user', {}).update(
-                    {'password': md5(instance.password)})
-                session.save()
-                request.session.flash(
-                    _("Successfully updated password"), queue='success')
-                return redirect(url('my_account_password'))
-
-        c.form = form
-        return render('admin/my_account/my_account.mako')
-
     def my_account_repos(self):
         c.active = 'repos'
         self.__load_data()
@@ -376,54 +320,6 b' class MyAccountController(BaseController'
         else:
             return json.dumps(data)
 
-    def my_account_auth_tokens(self):
-        c.active = 'auth_tokens'
-        self.__load_data()
-        show_expired = True
-        c.lifetime_values = [
-            (str(-1), _('forever')),
-            (str(5), _('5 minutes')),
-            (str(60), _('1 hour')),
-            (str(60 * 24), _('1 day')),
-            (str(60 * 24 * 30), _('1 month')),
-        ]
-        c.lifetime_options = [(c.lifetime_values, _("Lifetime"))]
-        c.role_values = [(x, AuthTokenModel.cls._get_role_name(x))
-                         for x in AuthTokenModel.cls.ROLES]
-        c.role_options = [(c.role_values, _("Role"))]
-        c.user_auth_tokens = AuthTokenModel().get_auth_tokens(
-            c.rhodecode_user.user_id, show_expired=show_expired)
-        return render('admin/my_account/my_account.mako')
-
-    @auth.CSRFRequired()
-    def my_account_auth_tokens_add(self):
-        lifetime = safe_int(request.POST.get('lifetime'), -1)
-        description = request.POST.get('description')
-        role = request.POST.get('role')
-        AuthTokenModel().create(c.rhodecode_user.user_id, description, lifetime,
-                                role)
-        Session().commit()
-        h.flash(_("Auth token successfully created"), category='success')
-        return redirect(url('my_account_auth_tokens'))
-
-    @auth.CSRFRequired()
-    def my_account_auth_tokens_delete(self):
-        auth_token = request.POST.get('del_auth_token')
-        user_id = c.rhodecode_user.user_id
-        if request.POST.get('del_auth_token_builtin'):
-            user = User.get(user_id)
-            if user:
-                user.api_key = generate_auth_token(user.username)
-                Session().add(user)
-                Session().commit()
-                h.flash(_("Auth token successfully reset"), category='success')
-        elif auth_token:
-            AuthTokenModel().delete(auth_token, c.rhodecode_user.user_id)
-            Session().commit()
-            h.flash(_("Auth token successfully deleted"), category='success')
-
-        return redirect(url('my_account_auth_tokens'))
-
     def my_notifications(self):
         c.active = 'notifications'
         return render('admin/my_account/my_account.mako')

rhodecode/controllers/admin/settings.py

@@ -36,7 +36,8 b' from pyramid.threadlocal import get_curr'
 from webob.exc import HTTPBadRequest
 
 import rhodecode
-from rhodecode.admin.navigation import navigation_list
+from rhodecode.apps.admin.navigation import navigation_list
+from rhodecode.apps.svn_support.config_keys import generate_config
 from rhodecode.lib import auth
 from rhodecode.lib import helpers as h
 from rhodecode.lib.auth import LoginRequired, HasPermissionAllDecorator
@@ -62,7 +63,6 b' from rhodecode.model.settings import ('
     SettingsModel)
 
 from rhodecode.model.supervisor import SupervisorModel, SUPERVISOR_MASTER
-from rhodecode.svn_support.config_keys import generate_config
 
 
 log = logging.getLogger(__name__)
@@ -82,7 +82,7 b' class SettingsController(BaseController)'
             rhodecode.CONFIG.get('labs_settings_active', 'true'))
         c.navlist = navigation_list(request)
 
-    def _get_hg_ui_settings(self):
+    def _get_ui_settings(self):
         ret = RhodeCodeUi.query().all()
 
         if not ret:
@@ -94,7 +94,7 b' class SettingsController(BaseController)'
             if k == '/':
                 k = 'root_path'
 
-            if k in ['push_ssl', 'publish']:
+            if k in ['push_ssl', 'publish', 'enabled']:
                 v = str2bool(v)
 
             if k.find('.') != -1:
@@ -165,6 +165,7 b' class SettingsController(BaseController)'
 
             model.create_or_update_global_svn_settings(form_result)
             model.create_or_update_global_hg_settings(form_result)
+            model.create_or_update_global_git_settings(form_result)
             model.create_or_update_global_pr_settings(form_result)
         except Exception:
             log.exception("Exception while updating settings")
@@ -668,7 +669,8 b' class SettingsController(BaseController)'
 
     def _form_defaults(self):
         defaults = SettingsModel().get_all_settings()
-        defaults.update(self._get_hg_ui_settings())
+        defaults.update(self._get_ui_settings())
+
         defaults.update({
             'new_svn_branch': '',
             'new_svn_tag': '',

rhodecode/controllers/admin/user_groups.py

@@ -117,7 +117,7 b' class UserGroupsController(BaseControlle'
         def user_group_actions(user_group_id, user_group_name):
             return _render("user_group_actions", user_group_id, user_group_name)
 
-        ## json generate
+        # json generate
         group_iter = UserGroupList(UserGroup.query().all(),
                                    perm_set=['usergroup.admin'])
 
@@ -129,6 +129,7 b' class UserGroupsController(BaseControlle'
                 "group_name_raw": user_gr.users_group_name,
                 "desc": h.escape(user_gr.user_group_description),
                 "members": len(user_gr.members),
+                "sync": user_gr.group_data.get('extern_type'),
                 "active": h.bool2icon(user_gr.users_group_active),
                 "owner": h.escape(h.link_to_user(user_gr.user.username)),
                 "action": user_group_actions(
@@ -431,7 +432,6 b' class UserGroupsController(BaseControlle'
                 prefix_error=False,
                 encoding="UTF-8",
                 force_defaults=False)
-
         except Exception:
             log.exception("Exception during permissions saving")
             h.flash(_('An error occurred during permissions saving'),
@@ -459,6 +459,36 b' class UserGroupsController(BaseControlle'
         return render('admin/user_groups/user_group_edit.mako')
 
     @HasUserGroupPermissionAnyDecorator('usergroup.admin')
+    def edit_advanced_set_synchronization(self, user_group_id):
+        user_group_id = safe_int(user_group_id)
+        user_group = UserGroup.get_or_404(user_group_id)
+
+        existing = user_group.group_data.get('extern_type')
+
+        if existing:
+            new_state = user_group.group_data
+            new_state['extern_type'] = None
+        else:
+            new_state = user_group.group_data
+            new_state['extern_type'] = 'manual'
+            new_state['extern_type_set_by'] = c.rhodecode_user.username
+
+        try:
+            user_group.group_data = new_state
+            Session().add(user_group)
+            Session().commit()
+
+            h.flash(_('User Group synchronization updated successfully'),
+                    category='success')
+        except Exception:
+            log.exception("Exception during sync settings saving")
+            h.flash(_('An error occurred during synchronization update'),
+                    category='error')
+
+        return redirect(
+            url('edit_user_group_advanced', user_group_id=user_group_id))
+
+    @HasUserGroupPermissionAnyDecorator('usergroup.admin')
     @XHRRequired()
     @jsonify
     def user_group_members(self, user_group_id):

rhodecode/controllers/admin/users.py

@@ -50,7 +50,6 b' from rhodecode.model.user import UserMod'
 from rhodecode.model.meta import Session
 from rhodecode.model.permission import PermissionModel
 from rhodecode.lib.utils import action_logger
-from rhodecode.lib.ext_json import json
 from rhodecode.lib.utils2 import datetime_to_time, safe_int, AttributeDict
 
 log = logging.getLogger(__name__)
@@ -76,51 +75,6 b' class UsersController(BaseController):'
         ]
         PermissionModel().set_global_permission_choices(c, gettext_translator=_)
 
-    @HasPermissionAllDecorator('hg.admin')
-    def index(self):
-        """GET /users: All items in the collection"""
-        # url('users')
-
-        from rhodecode.lib.utils import PartialRenderer
-        _render = PartialRenderer('data_table/_dt_elements.mako')
-
-        def username(user_id, username):
-            return _render("user_name", user_id, username)
-
-        def user_actions(user_id, username):
-            return _render("user_actions", user_id, username)
-
-        # json generate
-        c.users_list = User.query()\
-            .filter(User.username != User.DEFAULT_USER) \
-            .all()
-
-        users_data = []
-        for user in c.users_list:
-            users_data.append({
-                "username": h.gravatar_with_user(user.username),
-                "username_raw": user.username,
-                "email": user.email,
-                "first_name": h.escape(user.name),
-                "last_name": h.escape(user.lastname),
-                "last_login": h.format_date(user.last_login),
-                "last_login_raw": datetime_to_time(user.last_login),
-                "last_activity": h.format_date(
-                    h.time_to_datetime(user.user_data.get('last_activity', 0))),
-                "last_activity_raw": user.user_data.get('last_activity', 0),
-                "active": h.bool2icon(user.active),
-                "active_raw": user.active,
-                "admin": h.bool2icon(user.admin),
-                "admin_raw": user.admin,
-                "extern_type": user.extern_type,
-                "extern_name": user.extern_name,
-                "action": user_actions(user.user_id, user.username),
-            })
-
-
-        c.data = json.dumps(users_data)
-        return render('admin/users/users.mako')
-
     def _get_personal_repo_group_template_vars(self):
         DummyUser = AttributeDict({
             'username': '${username}',
@@ -135,7 +89,6 b' class UsersController(BaseController):'
     @auth.CSRFRequired()
     def create(self):
         """POST /users: Create a new item"""
-        # url('users')
         c.default_extern_type = auth_rhodecode.RhodeCodeAuthPlugin.name
         user_model = UserModel()
         user_form = UserForm()()
@@ -168,7 +121,7 b' class UsersController(BaseController):'
             log.exception("Exception creation of user")
             h.flash(_('Error occurred during creation of user %s')
                     % request.POST.get('username'), category='error')
-        return redirect(url('users'))
+        return redirect(h.route_path('users'))
 
     @HasPermissionAllDecorator('hg.admin')
     def new(self):
@@ -312,7 +265,7 b' class UsersController(BaseController):'
             log.exception("Exception during deletion of user")
             h.flash(_('An error occurred during deletion of user'),
                     category='error')
-        return redirect(url('users'))
+        return redirect(h.route_path('users'))
 
     @HasPermissionAllDecorator('hg.admin')
     @auth.CSRFRequired()
@@ -404,7 +357,7 b' class UsersController(BaseController):'
         c.user = User.get_or_404(user_id)
         if c.user.username == User.DEFAULT_USER:
             h.flash(_("You can't edit this user"), category='warning')
-            return redirect(url('users'))
+            return redirect(h.route_path('users'))
 
         c.active = 'profile'
         c.extern_type = c.user.extern_type
@@ -425,7 +378,7 b' class UsersController(BaseController):'
         user = c.user = User.get_or_404(user_id)
         if user.username == User.DEFAULT_USER:
             h.flash(_("You can't edit this user"), category='warning')
-            return redirect(url('users'))
+            return redirect(h.route_path('users'))
 
         c.active = 'advanced'
         c.perm_user = AuthUser(user_id=user_id, ip_addr=self.ip_addr)
@@ -452,83 +405,12 b' class UsersController(BaseController):'
             force_defaults=False)
 
     @HasPermissionAllDecorator('hg.admin')
-    def edit_auth_tokens(self, user_id):
-        user_id = safe_int(user_id)
-        c.user = User.get_or_404(user_id)
-        if c.user.username == User.DEFAULT_USER:
-            h.flash(_("You can't edit this user"), category='warning')
-            return redirect(url('users'))
-
-        c.active = 'auth_tokens'
-        show_expired = True
-        c.lifetime_values = [
-            (str(-1), _('forever')),
-            (str(5), _('5 minutes')),
-            (str(60), _('1 hour')),
-            (str(60 * 24), _('1 day')),
-            (str(60 * 24 * 30), _('1 month')),
-        ]
-        c.lifetime_options = [(c.lifetime_values, _("Lifetime"))]
-        c.role_values = [(x, AuthTokenModel.cls._get_role_name(x))
-                         for x in AuthTokenModel.cls.ROLES]
-        c.role_options = [(c.role_values, _("Role"))]
-        c.user_auth_tokens = AuthTokenModel().get_auth_tokens(
-            c.user.user_id, show_expired=show_expired)
-        defaults = c.user.get_dict()
-        return htmlfill.render(
-            render('admin/users/user_edit.mako'),
-            defaults=defaults,
-            encoding="UTF-8",
-            force_defaults=False)
-
-    @HasPermissionAllDecorator('hg.admin')
-    @auth.CSRFRequired()
-    def add_auth_token(self, user_id):
-        user_id = safe_int(user_id)
-        c.user = User.get_or_404(user_id)
-        if c.user.username == User.DEFAULT_USER:
-            h.flash(_("You can't edit this user"), category='warning')
-            return redirect(url('users'))
-
-        lifetime = safe_int(request.POST.get('lifetime'), -1)
-        description = request.POST.get('description')
-        role = request.POST.get('role')
-        AuthTokenModel().create(c.user.user_id, description, lifetime, role)
-        Session().commit()
-        h.flash(_("Auth token successfully created"), category='success')
-        return redirect(url('edit_user_auth_tokens', user_id=c.user.user_id))
-
-    @HasPermissionAllDecorator('hg.admin')
-    @auth.CSRFRequired()
-    def delete_auth_token(self, user_id):
-        user_id = safe_int(user_id)
-        c.user = User.get_or_404(user_id)
-        if c.user.username == User.DEFAULT_USER:
-            h.flash(_("You can't edit this user"), category='warning')
-            return redirect(url('users'))
-
-        auth_token = request.POST.get('del_auth_token')
-        if request.POST.get('del_auth_token_builtin'):
-            user = User.get(c.user.user_id)
-            if user:
-                user.api_key = generate_auth_token(user.username)
-                Session().add(user)
-                Session().commit()
-                h.flash(_("Auth token successfully reset"), category='success')
-        elif auth_token:
-            AuthTokenModel().delete(auth_token, c.user.user_id)
-            Session().commit()
-            h.flash(_("Auth token successfully deleted"), category='success')
-
-        return redirect(url('edit_user_auth_tokens', user_id=c.user.user_id))
-
-    @HasPermissionAllDecorator('hg.admin')
     def edit_global_perms(self, user_id):
         user_id = safe_int(user_id)
         c.user = User.get_or_404(user_id)
         if c.user.username == User.DEFAULT_USER:
             h.flash(_("You can't edit this user"), category='warning')
-            return redirect(url('users'))
+            return redirect(h.route_path('users'))
 
         c.active = 'global_perms'
 
@@ -602,7 +484,7 b' class UsersController(BaseController):'
         c.user = User.get_or_404(user_id)
         if c.user.username == User.DEFAULT_USER:
             h.flash(_("You can't edit this user"), category='warning')
-            return redirect(url('users'))
+            return redirect(h.route_path('users'))
 
         c.active = 'perms_summary'
         c.perm_user = AuthUser(user_id=user_id, ip_addr=self.ip_addr)
@@ -615,7 +497,7 b' class UsersController(BaseController):'
         c.user = User.get_or_404(user_id)
         if c.user.username == User.DEFAULT_USER:
             h.flash(_("You can't edit this user"), category='warning')
-            return redirect(url('users'))
+            return redirect(h.route_path('users'))
 
         c.active = 'emails'
         c.user_email_map = UserEmailMap.query() \
@@ -673,7 +555,7 b' class UsersController(BaseController):'
         c.user = User.get_or_404(user_id)
         if c.user.username == User.DEFAULT_USER:
             h.flash(_("You can't edit this user"), category='warning')
-            return redirect(url('users'))
+            return redirect(h.route_path('users'))
 
         c.active = 'ips'
         c.user_ip_map = UserIpMap.query() \

rhodecode/controllers/feed.py

@@ -28,12 +28,11 b' import pytz'
 from pylons import url, response, tmpl_context as c
 from pylons.i18n.translation import _
 
-from beaker.cache import cache_region, region_invalidate
+from beaker.cache import cache_region
 from webhelpers.feedgenerator import Atom1Feed, Rss201rev2Feed
 
-from rhodecode.model.db import CacheKey
+from rhodecode.model.db import CacheKey, UserApiKeys
 from rhodecode.lib import helpers as h
-from rhodecode.lib import caches
 from rhodecode.lib.auth import LoginRequired, HasRepoPermissionAnyDecorator
 from rhodecode.lib.base import BaseRepoController
 from rhodecode.lib.diffs import DiffProcessor, LimitedDiffContainer
@@ -62,7 +61,7 b' class FeedController(BaseRepoController)'
                 safe_int(config.get('rss_cut_off_limit', 32 * 1024)),
         }
 
-    @LoginRequired(auth_token_access=True)
+    @LoginRequired(auth_token_access=[UserApiKeys.ROLE_FEED])
     def __before__(self):
         super(FeedController, self).__before__()
         config = self._get_config()

rhodecode/controllers/files.py

@@ -223,6 +223,8 b' class FilesController(BaseRepoController'
             c.file_author = True
             c.file_tree = ''
             if c.file.is_file():
+                c.lf_node = c.file.get_largefile_node()
+
                 c.file_source_page = 'true'
                 c.file_last_commit = c.file.last_commit
                 if c.file.size < self.cut_off_limit_file:
@@ -239,7 +241,11 b' class FilesController(BaseRepoController'
 
                 c.on_branch_head = self._is_valid_head(
                     commit_id, c.rhodecode_repo)
-                c.branch_or_raw_id = c.commit.branch or c.commit.raw_id
+
+                branch = c.commit.branch if (
+                    c.commit.branch and '/' not in c.commit.branch) else None
+                c.branch_or_raw_id = branch or c.commit.raw_id
+                c.branch_name = c.commit.branch or h.short_id(c.commit.raw_id)
 
                 author = c.file_last_commit.author
                 c.authors = [(h.email(author),
@@ -260,6 +266,32 b' class FilesController(BaseRepoController'
         return render('files/files.mako')
 
     @LoginRequired()
+    @HasRepoPermissionAnyDecorator(
+        'repository.read', 'repository.write', 'repository.admin')
+    def annotate_previous(self, repo_name, revision, f_path):
+
+        commit_id = revision
+        commit = self.__get_commit_or_redirect(commit_id, repo_name)
+        prev_commit_id = commit.raw_id
+
+        f_path = f_path
+        is_file = False
+        try:
+            _file = commit.get_node(f_path)
+            is_file = _file.is_file()
+        except (NodeDoesNotExistError, CommitDoesNotExistError, VCSError):
+            pass
+
+        if is_file:
+            history = commit.get_file_history(f_path)
+            prev_commit_id = history[1].raw_id \
+                if len(history) > 1 else prev_commit_id
+
+        return redirect(h.url(
+            'files_annotate_home', repo_name=repo_name,
+            revision=prev_commit_id, f_path=f_path))
+
+    @LoginRequired()
     @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',
                                    'repository.admin')
     @jsonify
@@ -317,6 +349,14 b' class FilesController(BaseRepoController'
         commit = self.__get_commit_or_redirect(revision, repo_name)
         file_node = self.__get_filenode_or_redirect(repo_name, commit, f_path)
 
+        if request.GET.get('lf'):
+            # only if lf get flag is passed, we download this file
+            # as LFS/Largefile
+            lf_node = file_node.get_largefile_node()
+            if lf_node:
+                # overwrite our pointer with the REAL large-file
+                file_node = lf_node
+
         response.content_disposition = 'attachment; filename=%s' % \
             safe_str(f_path.split(Repository.NAME_SEP)[-1])
 
@@ -352,6 +392,7 b' class FilesController(BaseRepoController'
             'image/png': ('image/png', 'inline'),
             'image/gif': ('image/gif', 'inline'),
             'image/jpeg': ('image/jpeg', 'inline'),
+            'application/pdf': ('application/pdf', 'inline'),
         }
 
         mimetype = file_node.mimetype

rhodecode/controllers/journal.py

@@ -34,8 +34,8 b' from webob.exc import HTTPBadRequest'
 from pylons import request, tmpl_context as c, response, url
 from pylons.i18n.translation import _
 
-from rhodecode.controllers.admin.admin import _journal_filter
-from rhodecode.model.db import UserLog, UserFollowing, User
+from rhodecode.controllers.admin.admin import user_log_filter
+from rhodecode.model.db import UserLog, UserFollowing, User, UserApiKeys
 from rhodecode.model.meta import Session
 import rhodecode.lib.helpers as h
 from rhodecode.lib.helpers import Page
@@ -89,7 +89,7 b' class JournalController(BaseController):'
                 .options(joinedload(UserLog.repository))
             #filter
             try:
-                journal = _journal_filter(journal, c.search_term)
+                journal = user_log_filter(journal, c.search_term)
             except Exception:
                 # we want this to crash for now
                 raise
@@ -211,7 +211,7 b' class JournalController(BaseController):'
 
         return render('journal/journal.mako')
 
-    @LoginRequired(auth_token_access=True)
+    @LoginRequired(auth_token_access=[UserApiKeys.ROLE_FEED])
     @NotAnonymous()
     def journal_atom(self):
         """
@@ -223,7 +223,7 b' class JournalController(BaseController):'
             .all()
         return self._atom_feed(following, public=False)
 
-    @LoginRequired(auth_token_access=True)
+    @LoginRequired(auth_token_access=[UserApiKeys.ROLE_FEED])
     @NotAnonymous()
     def journal_rss(self):
         """
@@ -281,7 +281,7 b' class JournalController(BaseController):'
             return c.journal_data
         return render('journal/public_journal.mako')
 
-    @LoginRequired(auth_token_access=True)
+    @LoginRequired(auth_token_access=[UserApiKeys.ROLE_FEED])
     def public_journal_atom(self):
         """
         Produce an atom-1.0 feed via feedgenerator module
@@ -293,7 +293,7 b' class JournalController(BaseController):'
 
         return self._atom_feed(c.following)
 
-    @LoginRequired(auth_token_access=True)
+    @LoginRequired(auth_token_access=[UserApiKeys.ROLE_FEED])
     def public_journal_rss(self):
         """
         Produce an rss2 feed via feedgenerator module

rhodecode/controllers/pullrequests.py

@@ -69,6 +69,8 b' class PullrequestsController(BaseRepoCon'
 
     def __before__(self):
         super(PullrequestsController, self).__before__()
+        c.REVIEW_STATUS_APPROVED = ChangesetStatus.STATUS_APPROVED
+        c.REVIEW_STATUS_REJECTED = ChangesetStatus.STATUS_REJECTED
 
     def _extract_ordering(self, request):
         column_index = safe_int(request.GET.get('order[0][column]'))
@@ -440,13 +442,25 b' class PullrequestsController(BaseRepoCon'
         resp = PullRequestModel().update_commits(pull_request)
 
         if resp.executed:
+
+            if resp.target_changed and resp.source_changed:
+                changed = 'target and source repositories'
+            elif resp.target_changed and not resp.source_changed:
+                changed = 'target repository'
+            elif not resp.target_changed and resp.source_changed:
+                changed = 'source repository'
+            else:
+                changed = 'nothing'
+
             msg = _(
                 u'Pull request updated to "{source_commit_id}" with '
-                u'{count_added} added, {count_removed} removed commits.')
+                u'{count_added} added, {count_removed} removed commits. '
+                u'Source of changes: {change_source}')
             msg = msg.format(
                 source_commit_id=pull_request.source_ref_parts.commit_id,
                 count_added=len(resp.changes.added),
-                count_removed=len(resp.changes.removed))
+                count_removed=len(resp.changes.removed),
+                change_source=changed)
             h.flash(msg, category='success')
 
             registry = get_current_registry()
@@ -562,13 +576,21 b' class PullrequestsController(BaseRepoCon'
     def delete(self, repo_name, pull_request_id):
         pull_request_id = safe_int(pull_request_id)
         pull_request = PullRequest.get_or_404(pull_request_id)
+
+        pr_closed = pull_request.is_closed()
+        allowed_to_delete = PullRequestModel().check_user_delete(
+            pull_request, c.rhodecode_user) and not pr_closed
+
         # only owner can delete it !
-        if pull_request.author.user_id == c.rhodecode_user.user_id:
+        if allowed_to_delete:
             PullRequestModel().delete(pull_request)
             Session().commit()
             h.flash(_('Successfully deleted pull request'),
                     category='success')
             return redirect(url('my_account_pullrequests'))
+
+        h.flash(_('Your are not allowed to delete this pull request'),
+                category='error')
         raise HTTPForbidden()
 
     def _get_pr_version(self, pull_request_id, version=None):
@@ -642,6 +664,13 b' class PullrequestsController(BaseRepoCon'
          pull_request_display_obj,
          at_version) = self._get_pr_version(
             pull_request_id, version=version)
+        pr_closed = pull_request_latest.is_closed()
+
+        if pr_closed and (version or from_version):
+            # not allow to browse versions
+            return redirect(h.url('pullrequest_show', repo_name=repo_name,
+                                  pull_request_id=pull_request_id))
+
         versions = pull_request_display_obj.versions()
 
         c.at_version = at_version
@@ -675,20 +704,21 b' class PullrequestsController(BaseRepoCon'
         c.shadow_clone_url = PullRequestModel().get_shadow_clone_url(
             pull_request_at_ver)
 
-        c.ancestor = None  # empty ancestor hidden in display
         c.pull_request = pull_request_display_obj
         c.pull_request_latest = pull_request_latest
 
-        pr_closed = pull_request_latest.is_closed()
         if compare or (at_version and not at_version == 'latest'):
             c.allowed_to_change_status = False
             c.allowed_to_update = False
             c.allowed_to_merge = False
             c.allowed_to_delete = False
             c.allowed_to_comment = False
+            c.allowed_to_close = False
         else:
             c.allowed_to_change_status = PullRequestModel(). \
-                check_user_change_status(pull_request_at_ver, c.rhodecode_user)
+                check_user_change_status(pull_request_at_ver, c.rhodecode_user) \
+                                         and not pr_closed
+
             c.allowed_to_update = PullRequestModel().check_user_update(
                 pull_request_latest, c.rhodecode_user) and not pr_closed
             c.allowed_to_merge = PullRequestModel().check_user_merge(
@@ -696,6 +726,7 b' class PullrequestsController(BaseRepoCon'
             c.allowed_to_delete = PullRequestModel().check_user_delete(
                 pull_request_latest, c.rhodecode_user) and not pr_closed
             c.allowed_to_comment = not pr_closed
+            c.allowed_to_close = c.allowed_to_change_status and not pr_closed
 
         # check merge capabilities
         _merge_check = MergeCheck.validate(
@@ -704,6 +735,7 b' class PullrequestsController(BaseRepoCon'
         c.pr_merge_possible = not _merge_check.failed
         c.pr_merge_message = _merge_check.merge_msg
 
+        c.pull_request_review_status = _merge_check.review_status
         if merge_checks:
             return render('/pullrequests/pullrequest_merge_checks.mako')
 
@@ -712,7 +744,6 b' class PullrequestsController(BaseRepoCon'
         # reviewers and statuses
         c.pull_request_reviewers = pull_request_at_ver.reviewers_statuses()
         allowed_reviewers = [x[0].user_id for x in c.pull_request_reviewers]
-        c.pull_request_review_status = pull_request_at_ver.calculated_review_status()
 
         # GENERAL COMMENTS with versions #
         q = comments_model._all_general_comments_of_pull_request(pull_request_latest)
@@ -789,12 +820,15 b' class PullrequestsController(BaseRepoCon'
         target_commit = EmptyCommit()
         c.missing_requirements = False
 
+        source_scm = source_repo.scm_instance()
+        target_scm = target_repo.scm_instance()
+
         # try first shadow repo, fallback to regular repo
         try:
             commits_source_repo = pull_request_latest.get_shadow_repo()
         except Exception:
             log.debug('Failed to get shadow repo', exc_info=True)
-            commits_source_repo = source_repo.scm_instance()
+            commits_source_repo = source_scm
 
         c.commits_source_repo = commits_source_repo
         commit_cache = {}
@@ -818,6 +852,15 b' class PullrequestsController(BaseRepoCon'
                 'Failed to get all required data from repo', exc_info=True)
             c.missing_requirements = True
 
+        c.ancestor = None  # set it to None, to hide it from PR view
+
+        try:
+            ancestor_id = source_scm.get_common_ancestor(
+                source_commit.raw_id, target_commit.raw_id, target_scm)
+            c.ancestor_commit = source_scm.get_commit(ancestor_id)
+        except Exception:
+            c.ancestor_commit = None
+
         c.statuses = source_repo.statuses(
             [x.raw_id for x in c.commit_ranges])
 
@@ -860,12 +903,7 b' class PullrequestsController(BaseRepoCon'
         # We need to swap that here to generate it properly on the html side
         c.target_repo = c.source_repo
 
-        if c.allowed_to_update:
-            force_close = ('forced_closed', _('Close Pull Request'))
-            statuses = ChangesetStatus.STATUSES + [force_close]
-        else:
-            statuses = ChangesetStatus.STATUSES
-        c.commit_statuses = statuses
+        c.commit_statuses = ChangesetStatus.STATUSES
 
         c.show_version_changes = not pr_closed
         if c.show_version_changes:
@@ -924,22 +962,21 b' class PullrequestsController(BaseRepoCon'
         if pull_request.is_closed():
             raise HTTPForbidden()
 
-        # TODO: johbo: Re-think this bit, "approved_closed" does not exist
-        # as a changeset status, still we want to send it in one value.
         status = request.POST.get('changeset_status', None)
         text = request.POST.get('text')
         comment_type = request.POST.get('comment_type')
         resolves_comment_id = request.POST.get('resolves_comment_id', None)
+        close_pull_request = request.POST.get('close_pull_request')
 
-        if status and '_closed' in status:
+        close_pr = False
+        if close_pull_request:
             close_pr = True
-            status = status.replace('_closed', '')
-        else:
-            close_pr = False
-
-        forced = (status == 'forced')
-        if forced:
-            status = 'rejected'
+            pull_request_review_status = pull_request.calculated_review_status()
+            if pull_request_review_status == ChangesetStatus.STATUS_APPROVED:
+                # approved only if we have voting consent
+                status = ChangesetStatus.STATUS_APPROVED
+            else:
+                status = ChangesetStatus.STATUS_REJECTED
 
         allowed_to_change_status = PullRequestModel().check_user_change_status(
             pull_request, c.rhodecode_user)
@@ -995,7 +1032,7 b' class PullrequestsController(BaseRepoCon'
                 status_completed = (
                     calculated_status in [ChangesetStatus.STATUS_APPROVED,
                                           ChangesetStatus.STATUS_REJECTED])
-                if forced or status_completed:
+                if close_pull_request or status_completed:
                     PullRequestModel().close_pull_request(
                         pull_request_id, c.rhodecode_user)
                 else:

rhodecode/controllers/summary.py

@@ -38,7 +38,7 b' from rhodecode.lib.utils2 import safe_st'
 from rhodecode.lib.auth import (
     LoginRequired, HasRepoPermissionAnyDecorator, NotAnonymous, XHRRequired)
 from rhodecode.lib.base import BaseRepoController, render
-from rhodecode.lib.markup_renderer import MarkupRenderer
+from rhodecode.lib.markup_renderer import MarkupRenderer, relative_links
 from rhodecode.lib.ext_json import json
 from rhodecode.lib.vcs.backends.base import EmptyCommit
 from rhodecode.lib.vcs.exceptions import (
@@ -70,7 +70,12 b' class SummaryController(BaseRepoControll'
                 log.debug("Searching for a README file.")
                 readme_node = ReadmeFinder(default_renderer).search(commit)
             if readme_node:
-                readme_data = self._render_readme_or_none(commit, readme_node)
+                relative_url = h.url('files_raw_home',
+                                     repo_name=repo_name,
+                                     revision=commit.raw_id,
+                                     f_path=readme_node.path)
+                readme_data = self._render_readme_or_none(
+                    commit, readme_node, relative_url)
                 readme_filename = readme_node.path
             return readme_data, readme_filename
 
@@ -95,13 +100,16 b' class SummaryController(BaseRepoControll'
             log.exception(
                 "Problem getting commit when trying to render the README.")
 
-    def _render_readme_or_none(self, commit, readme_node):
+    def _render_readme_or_none(self, commit, readme_node, relative_url):
         log.debug(
             'Found README file `%s` rendering...', readme_node.path)
         renderer = MarkupRenderer()
         try:
-            return renderer.render(
+            html_source = renderer.render(
                 readme_node.content, filename=readme_node.path)
+            if relative_url:
+                return relative_links(html_source, relative_url)
+            return html_source
         except Exception:
             log.exception(
                 "Exception while trying to render the README")

rhodecode/events/base.py

@@ -56,8 +56,15 b' class RhodecodeEvent(object):'
     @property
     def actor(self):
         auth_user = self.auth_user
+
         if auth_user:
-            return auth_user.get_instance()
+            instance = auth_user.get_instance()
+            if not instance:
+                return AttributeDict(dict(
+                    username=auth_user.username
+                ))
+            return instance
+
         return SYSTEM_USER
 
     @property

rhodecode/events/pullrequest.py

@@ -128,6 +128,9 b' class PullRequestCommentEvent(PullReques'
             'comment': {
                 'status': status,
                 'text': self.comment.text,
+                'type': self.comment.comment_type,
+                'file': self.comment.f_path,
+                'line': self.comment.line_no,
                 'url': CommentsModel().get_url(self.comment)
             }
         })

rhodecode/events/repo.py

@@ -34,10 +34,9 b' def _commits_as_dict(commit_ids, repos):'
     :param repos: list of repos to check
     """
     from rhodecode.lib.utils2 import extract_mentioned_users
-    from rhodecode.model.db import Repository
     from rhodecode.lib import helpers as h
-    from rhodecode.lib.helpers import process_patterns
-    from rhodecode.lib.helpers import urlify_commit_message
+    from rhodecode.lib.helpers import (
+        urlify_commit_message, process_patterns, chop_at_smart)
 
     if not repos:
         raise Exception('no repo defined')
@@ -78,14 +77,15 b' def _commits_as_dict(commit_ids, repos):'
                 cs_data['issues'] = issues_data
                 cs_data['message_html'] = urlify_commit_message(cs_data['message'],
                     repo.repo_name)
+                cs_data['message_html_title'] = chop_at_smart(cs_data['message'], '\n', suffix_if_chopped='...')
                 commits.append(cs_data)
 
                 needed_commits.remove(commit_id)
 
         except Exception as e:
             log.exception(e)
-            # we don't send any commits when crash happens, only full list matters
-            # we short circuit then.
+            # we don't send any commits when crash happens, only full list
+            # matters we short circuit then.
             return []
 
     missing_commits = set(commit_ids) - set(c['raw_id'] for c in commits)

rhodecode/integrations/types/slack.py

@@ -19,12 +19,14 b''
 # and proprietary license terms, please see https://rhodecode.com/licenses/
 
 from __future__ import unicode_literals
+import re
+import time
+import textwrap
+import logging
+
 import deform
-import re
-import logging
 import requests
 import colander
-import textwrap
 from celery.task import task
 from mako.template import Template
 
@@ -85,17 +87,6 b' class SlackSettingsSchema(colander.Schem'
     )
 
 
-repo_push_template = Template(r'''
-*${data['actor']['username']}* pushed to repo <${data['repo']['url']}|${data['repo']['repo_name']}>:
-%for branch, branch_commits in branches_commits.items():
-branch: <${branch_commits['branch']['url']}|${branch_commits['branch']['name']}>
-    %for commit in branch_commits['commits']:
-> <${commit['url']}|${commit['short_id']}> - ${commit['message_html']|html_to_slack_links}
-    %endfor
-%endfor
-''')
-
-
 class SlackIntegrationType(IntegrationTypeBase):
     key = 'slack'
     display_name = _('Slack')
@@ -124,25 +115,31 b' class SlackIntegrationType(IntegrationTy'
 
         data = event.as_dict()
 
+        # defaults
+        title = '*%s* caused a *%s* event' % (
+            data['actor']['username'], event.name)
         text = '*%s* caused a *%s* event' % (
             data['actor']['username'], event.name)
+        fields = None
+        overrides = None
 
         log.debug('handling slack event for %s' % event.name)
 
         if isinstance(event, events.PullRequestCommentEvent):
-            text = self.format_pull_request_comment_event(event, data)
+            (title, text, fields, overrides) \
+                = self.format_pull_request_comment_event(event, data)
         elif isinstance(event, events.PullRequestReviewEvent):
-            text = self.format_pull_request_review_event(event, data)
+            title, text = self.format_pull_request_review_event(event, data)
         elif isinstance(event, events.PullRequestEvent):
-            text = self.format_pull_request_event(event, data)
+            title, text = self.format_pull_request_event(event, data)
         elif isinstance(event, events.RepoPushEvent):
-            text = self.format_repo_push_event(data)
+            title, text = self.format_repo_push_event(data)
         elif isinstance(event, events.RepoCreateEvent):
-            text = self.format_repo_create_event(data)
+            title, text = self.format_repo_create_event(data)
         else:
             log.error('unhandled event type: %r' % event)
 
-        run_task(post_text_to_slack, self.settings, text)
+        run_task(post_text_to_slack, self.settings, title, text, fields, overrides)
 
     def settings_schema(self):
         schema = SlackSettingsSchema()
@@ -167,37 +164,60 b' class SlackIntegrationType(IntegrationTy'
                 comment_url=data['comment']['url'],
             )
 
-        comment_status = ''
+        fields = None
+        overrides = None
+        status_text = None
+
         if data['comment']['status']:
-            comment_status = '[{}]: '.format(data['comment']['status'])
+            status_color = {
+                'approved': '#0ac878',
+                'rejected': '#e85e4d'}.get(data['comment']['status'])
+
+            if status_color:
+                overrides = {"color": status_color}
+
+            status_text = data['comment']['status']
+
+        if data['comment']['file']:
+            fields = [
+                {
+                    "title": "file",
+                    "value": data['comment']['file']
+                },
+                {
+                    "title": "line",
+                    "value": data['comment']['line']
+                }
+            ]
 
-        return (textwrap.dedent(
-            '''
-            *{user}* commented on pull request <{pr_url}|#{number}> - {pr_title}:
-            >>> {comment_status}{comment_text}
-            ''').format(
-                comment_status=comment_status,
-                user=data['actor']['username'],
-                number=data['pullrequest']['pull_request_id'],
-                pr_url=data['pullrequest']['url'],
-                pr_status=data['pullrequest']['status'],
-                pr_title=data['pullrequest']['title'],
-                comment_text=comment_text
-            )
+        title = Template(textwrap.dedent(r'''
+        *${data['actor']['username']}* left ${data['comment']['type']} on pull request <${data['pullrequest']['url']}|#${data['pullrequest']['pull_request_id']}>:
+        ''')).render(data=data, comment=event.comment)
+
+        text = Template(textwrap.dedent(r'''
+        *pull request title*: ${pr_title}
+        % if status_text:
+        *submitted status*: `${status_text}`
+        % endif
+        >>> ${comment_text}
+        ''')).render(comment_text=comment_text,
+                     pr_title=data['pullrequest']['title'],
+                     status_text=status_text)
+
+        return title, text, fields, overrides
+
+    def format_pull_request_review_event(self, event, data):
+        title = Template(textwrap.dedent(r'''
+        *${data['actor']['username']}* changed status of pull request <${data['pullrequest']['url']}|#${data['pullrequest']['pull_request_id']} to `${data['pullrequest']['status']}`>:
+        ''')).render(data=data)
+
+        text = Template(textwrap.dedent(r'''
+        *pull request title*: ${pr_title}
+        ''')).render(
+            pr_title=data['pullrequest']['title'],
         )
 
-    def format_pull_request_review_event(self, event, data):
-        return (textwrap.dedent(
-            '''
-            Status changed to {pr_status} for pull request <{pr_url}|#{number}> - {pr_title}
-            ''').format(
-                user=data['actor']['username'],
-                number=data['pullrequest']['pull_request_id'],
-                pr_url=data['pullrequest']['url'],
-                pr_status=data['pullrequest']['status'],
-                pr_title=data['pullrequest']['title'],
-            )
-        )
+        return title, text
 
     def format_pull_request_event(self, event, data):
         action = {
@@ -207,15 +227,22 b' class SlackIntegrationType(IntegrationTy'
             events.PullRequestCreateEvent: 'created',
         }.get(event.__class__, str(event.__class__))
 
-        return ('Pull request <{url}|#{number}> - {title} '
-                '`{action}` by *{user}*').format(
-            user=data['actor']['username'],
-            number=data['pullrequest']['pull_request_id'],
-            url=data['pullrequest']['url'],
-            title=data['pullrequest']['title'],
-            action=action
+        title = Template(textwrap.dedent(r'''
+        *${data['actor']['username']}* `${action}` pull request <${data['pullrequest']['url']}|#${data['pullrequest']['pull_request_id']}>:
+        ''')).render(data=data, action=action)
+
+        text = Template(textwrap.dedent(r'''
+        *pull request title*: ${pr_title}
+        %if data['pullrequest']['commits']:
+        *commits*: ${len(data['pullrequest']['commits'])}
+        %endif
+        ''')).render(
+            pr_title=data['pullrequest']['title'],
+            data=data
         )
 
+        return title, text
+
     def format_repo_push_event(self, data):
         branch_data = {branch['name']: branch
                        for branch in data['push']['branches']}
@@ -230,20 +257,38 b' class SlackIntegrationType(IntegrationTy'
             branch_commits = branches_commits[commit['branch']]
             branch_commits['commits'].append(commit)
 
-        result = repo_push_template.render(
+        title = Template(r'''
+        *${data['actor']['username']}* pushed to repo <${data['repo']['url']}|${data['repo']['repo_name']}>:
+        ''').render(data=data)
+
+        repo_push_template = Template(textwrap.dedent(r'''
+        %for branch, branch_commits in branches_commits.items():
+        ${len(branch_commits['commits'])} ${'commit' if len(branch_commits['commits']) == 1 else 'commits'} on branch: <${branch_commits['branch']['url']}|${branch_commits['branch']['name']}>
+        %for commit in branch_commits['commits']:
+        `<${commit['url']}|${commit['short_id']}>` - ${commit['message_html_title']|html_to_slack_links}
+        %endfor
+        %endfor
+        '''))
+
+        text = repo_push_template.render(
             data=data,
             branches_commits=branches_commits,
             html_to_slack_links=html_to_slack_links,
         )
-        return result
+
+        return title, text
 
     def format_repo_create_event(self, data):
-        return '<{}|{}> ({}) repository created by *{}*'.format(
-            data['repo']['url'],
-            data['repo']['repo_name'],
-            data['repo']['repo_type'],
-            data['actor']['username'],
-        )
+        title = Template(r'''
+        *${data['actor']['username']}* created new repository ${data['repo']['repo_name']}:
+        ''').render(data=data)
+
+        text = Template(textwrap.dedent(r'''
+        repo_url: ${data['repo']['url']}
+        repo_type: ${data['repo']['repo_type']}
+        ''')).render(data=data)
+
+        return title, text
 
 
 def html_to_slack_links(message):
@@ -252,12 +297,38 b' def html_to_slack_links(message):'
 
 
 @task(ignore_result=True)
-def post_text_to_slack(settings, text):
-    log.debug('sending %s to slack %s' % (text, settings['service']))
-    resp = requests.post(settings['service'], json={
+def post_text_to_slack(settings, title, text, fields=None, overrides=None):
+    log.debug('sending %s (%s) to slack %s' % (
+        title, text, settings['service']))
+
+    fields = fields or []
+    overrides = overrides or {}
+
+    message_data = {
+                "fallback": text,
+                "color": "#427cc9",
+                "pretext": title,
+                #"author_name": "Bobby Tables",
+                #"author_link": "http://flickr.com/bobby/",
+                #"author_icon": "http://flickr.com/icons/bobby.jpg",
+                #"title": "Slack API Documentation",
+                #"title_link": "https://api.slack.com/",
+                "text": text,
+                "fields": fields,
+                #"image_url": "http://my-website.com/path/to/image.jpg",
+                #"thumb_url": "http://example.com/path/to/thumb.png",
+                "footer": "RhodeCode",
+                #"footer_icon": "",
+                "ts": time.time(),
+                "mrkdwn_in": ["pretext", "text"]
+    }
+    message_data.update(overrides)
+    json_message = {
+        "icon_emoji": settings.get('icon_emoji', ':studio_microphone:'),
         "channel": settings.get('channel', ''),
         "username": settings.get('username', 'Rhodecode'),
-        "text": text,
-        "icon_emoji": settings.get('icon_emoji', ':studio_microphone:')
-    })
+        "attachments": [message_data]
+    }
+
+    resp = requests.post(settings['service'], json=json_message)
     resp.raise_for_status()  # raise exception on a failed request

rhodecode/integrations/views.py

@@ -29,6 +29,7 b' from pyramid.httpexceptions import HTTPF'
 from pyramid.renderers import render
 from pyramid.response import Response
 
+from rhodecode.apps.admin.navigation import navigation_list
 from rhodecode.lib import auth
 from rhodecode.lib.auth import LoginRequired, HasPermissionAllDecorator
 from rhodecode.lib.utils2 import safe_int
@@ -36,7 +37,6 b' from rhodecode.lib.helpers import Page'
 from rhodecode.model.db import Repository, RepoGroup, Session, Integration
 from rhodecode.model.scm import ScmModel
 from rhodecode.model.integration import IntegrationModel
-from rhodecode.admin.navigation import navigation_list
 from rhodecode.translation import _
 from rhodecode.integrations import integration_type_registry
 from rhodecode.model.validation_schema.schemas.integration_schema import (
@@ -75,11 +75,10 b' class IntegrationSettingsViewBase(object'
             repo_name = request.matchdict['repo_name']
             self.repo = Repository.get_by_repo_name(repo_name)
 
-        if 'repo_group_name' in request.matchdict: # in group settings context
+        if 'repo_group_name' in request.matchdict:  # in group settings context
             repo_group_name = request.matchdict['repo_group_name']
             self.repo_group = RepoGroup.get_by_group_name(repo_group_name)
 
-
         if 'integration' in request.matchdict:  # integration type context
             integration_type = request.matchdict['integration']
             self.IntegrationType = integration_type_registry[integration_type]
@@ -380,12 +379,12 b' class GlobalIntegrationsView(Integration'
 
 class RepoIntegrationsView(IntegrationSettingsViewBase):
     def perm_check(self, user):
-        return auth.HasRepoPermissionAll('repository.admin'
-            )(repo_name=self.repo.repo_name, user=user)
+        return auth.HasRepoPermissionAll('repository.admin')(
+            repo_name=self.repo.repo_name, user=user)
 
 
 class RepoGroupIntegrationsView(IntegrationSettingsViewBase):
     def perm_check(self, user):
-        return auth.HasRepoGroupPermissionAll('group.admin'
-            )(group_name=self.repo_group.group_name, user=user)
+        return auth.HasRepoGroupPermissionAll('group.admin')(
+            group_name=self.repo_group.group_name, user=user)
 

rhodecode/lib/auth.py

@@ -22,24 +22,22 b''
 authentication and permission libraries
 """
 
+import os
 import inspect
 import collections
 import fnmatch
 import hashlib
 import itertools
 import logging
-import os
 import random
-import time
 import traceback
 from functools import wraps
 
 import ipaddress
-from pyramid.httpexceptions import HTTPForbidden
+from pyramid.httpexceptions import HTTPForbidden, HTTPFound
 from pylons import url, request
 from pylons.controllers.util import abort, redirect
 from pylons.i18n.translation import _
-from sqlalchemy import or_
 from sqlalchemy.orm.exc import ObjectDeletedError
 from sqlalchemy.orm import joinedload
 from zope.cachedescriptors.property import Lazy as LazyProperty
@@ -99,6 +97,7 b' class PasswordGenerator(object):'
 
 
 class _RhodeCodeCryptoBase(object):
+    ENC_PREF = None
 
     def hash_create(self, str_):
         """
@@ -139,6 +138,7 b' class _RhodeCodeCryptoBase(object):'
 
 
 class _RhodeCodeCryptoBCrypt(_RhodeCodeCryptoBase):
+    ENC_PREF = '$2a$10'
 
     def hash_create(self, str_):
         self._assert_bytes(str_)
@@ -194,6 +194,7 b' class _RhodeCodeCryptoBCrypt(_RhodeCodeC'
 
 
 class _RhodeCodeCryptoSha256(_RhodeCodeCryptoBase):
+    ENC_PREF = '_'
 
     def hash_create(self, str_):
         self._assert_bytes(str_)
@@ -211,6 +212,7 b' class _RhodeCodeCryptoSha256(_RhodeCodeC'
 
 
 class _RhodeCodeCryptoMd5(_RhodeCodeCryptoBase):
+    ENC_PREF = '_'
 
     def hash_create(self, str_):
         self._assert_bytes(str_)
@@ -567,8 +569,14 b' class PermissionCalculator(object):'
         # on given user group
         for perm in self.default_user_group_perms:
             u_k = perm.UserUserGroupToPerm.user_group.users_group_name
-            p = perm.Permission.permission_name
             o = PermOrigin.USERGROUP_DEFAULT
+            if perm.UserGroup.user_id == self.user_id:
+                # set admin if owner
+                p = 'usergroup.admin'
+                o = PermOrigin.USERGROUP_OWNER
+            else:
+                p = perm.Permission.permission_name
+
             # if we decide this user isn't inheriting permissions from default
             # user we set him to .none so only explicit permissions work
             if not user_inherit_object_permissions:
@@ -647,7 +655,7 b' class PermissionCalculator(object):'
             multiple_counter[g_k] += 1
             p = perm.Permission.permission_name
             if perm.RepoGroup.user_id == self.user_id:
-                # set admin if owner
+                # set admin if owner, even for member of other user group
                 p = 'group.admin'
                 o = PermOrigin.REPOGROUP_OWNER
             else:
@@ -683,7 +691,7 b' class PermissionCalculator(object):'
         # user group for user group permissions
         user_group_from_user_group = Permission\
             .get_default_user_group_perms_from_user_group(
-                self.user_id, self.scope_repo_group_id)
+                self.user_id, self.scope_user_group_id)
 
         multiple_counter = collections.defaultdict(int)
         for perm in user_group_from_user_group:
@@ -694,9 +702,15 b' class PermissionCalculator(object):'
             o = PermOrigin.USERGROUP_USERGROUP % u_k
             multiple_counter[g_k] += 1
             p = perm.Permission.permission_name
-            if multiple_counter[g_k] > 1:
-                cur_perm = self.permissions_user_groups[g_k]
-                p = self._choose_permission(p, cur_perm)
+
+            if perm.UserGroup.user_id == self.user_id:
+                # set admin if owner, even for member of other user group
+                p = 'usergroup.admin'
+                o = PermOrigin.USERGROUP_OWNER
+            else:
+                if multiple_counter[g_k] > 1:
+                    cur_perm = self.permissions_user_groups[g_k]
+                    p = self._choose_permission(p, cur_perm)
             self.permissions_user_groups[g_k] = p, o
 
         # user explicit permission for user groups
@@ -705,12 +719,18 b' class PermissionCalculator(object):'
         for perm in user_user_groups_perms:
             ug_k = perm.UserUserGroupToPerm.user_group.users_group_name
             u_k = perm.UserUserGroupToPerm.user.username
-            p = perm.Permission.permission_name
             o = PermOrigin.USERGROUP_USER % u_k
-            if not self.explicit:
-                cur_perm = self.permissions_user_groups.get(
-                    ug_k, 'usergroup.none')
-                p = self._choose_permission(p, cur_perm)
+
+            if perm.UserGroup.user_id == self.user_id:
+                # set admin if owner
+                p = 'usergroup.admin'
+                o = PermOrigin.USERGROUP_OWNER
+            else:
+                p = perm.Permission.permission_name
+                if not self.explicit:
+                    cur_perm = self.permissions_user_groups.get(
+                        ug_k, 'usergroup.none')
+                    p = self._choose_permission(p, cur_perm)
             self.permissions_user_groups[ug_k] = p, o
 
     def _choose_permission(self, new_perm, cur_perm):
@@ -831,10 +851,6 b' class AuthUser(object):'
             self._permissions_scoped_cache[cache_key] = res
         return self._permissions_scoped_cache[cache_key]
 
-    @property
-    def auth_tokens(self):
-        return self.get_auth_tokens()
-
     def get_instance(self):
         return User.get(self.user_id)
 
@@ -925,16 +941,6 b' class AuthUser(object):'
         log.debug('PERMISSION tree computed %s' % (result_repr,))
         return result
 
-    def get_auth_tokens(self):
-        auth_tokens = [self.api_key]
-        for api_key in UserApiKeys.query()\
-                .filter(UserApiKeys.user_id == self.user_id)\
-                .filter(or_(UserApiKeys.expires == -1,
-                            UserApiKeys.expires >= time.time())).all():
-            auth_tokens.append(api_key.api_key)
-
-        return auth_tokens
-
     @property
     def is_default(self):
         return self.username == User.DEFAULT_USER
@@ -952,25 +958,27 b' class AuthUser(object):'
         """
         Returns list of repositories you're an admin of
         """
-        return [x[0] for x in self.permissions['repositories'].iteritems()
-                if x[1] == 'repository.admin']
+        return [
+            x[0] for x in self.permissions['repositories'].iteritems()
+            if x[1] == 'repository.admin']
 
     @property
     def repository_groups_admin(self):
         """
         Returns list of repository groups you're an admin of
         """
-        return [x[0]
-                for x in self.permissions['repositories_groups'].iteritems()
-                if x[1] == 'group.admin']
+        return [
+            x[0] for x in self.permissions['repositories_groups'].iteritems()
+            if x[1] == 'group.admin']
 
     @property
     def user_groups_admin(self):
         """
         Returns list of user groups you're an admin of
         """
-        return [x[0] for x in self.permissions['user_groups'].iteritems()
-                if x[1] == 'usergroup.admin']
+        return [
+            x[0] for x in self.permissions['user_groups'].iteritems()
+            if x[1] == 'usergroup.admin']
 
     @property
     def ip_allowed(self):
@@ -1171,7 +1179,7 b' class LoginRequired(object):'
     :param api_access: if enabled this checks only for valid auth token
         and grants access based on valid token
     """
-    def __init__(self, auth_token_access=False):
+    def __init__(self, auth_token_access=None):
         self.auth_token_access = auth_token_access
 
     def __call__(self, func):
@@ -1191,7 +1199,7 b' class LoginRequired(object):'
             ip_access_valid = False
 
         # check if we used an APIKEY and it's a valid one
-        # defined whitelist of controllers which API access will be enabled
+        # defined white-list of controllers which API access will be enabled
         _auth_token = request.GET.get(
             'auth_token', '') or request.GET.get('api_key', '')
         auth_token_access_valid = allowed_auth_token_access(
@@ -1200,8 +1208,20 b' class LoginRequired(object):'
         # explicit controller is enabled or API is in our whitelist
         if self.auth_token_access or auth_token_access_valid:
             log.debug('Checking AUTH TOKEN access for %s' % (cls,))
+            db_user = user.get_instance()
 
-            if _auth_token and _auth_token in user.auth_tokens:
+            if db_user:
+                if self.auth_token_access:
+                    roles = self.auth_token_access
+                else:
+                    roles = [UserApiKeys.ROLE_HTTP]
+                token_match = db_user.authenticate_by_token(
+                    _auth_token, roles=roles)
+            else:
+                log.debug('Unable to fetch db instance for auth user: %s', user)
+                token_match = False
+
+            if _auth_token and token_match:
                 auth_token_access_valid = True
                 log.debug('AUTH TOKEN ****%s is VALID' % (_auth_token[-4:],))
             else:
@@ -1234,7 +1254,6 b' class LoginRequired(object):'
                    auth_token_access_valid))
             # we preserve the get PARAM
             came_from = request.path_qs
-
             log.debug('redirecting to login page with %s' % (came_from,))
             return redirect(
                 h.route_path('login', _query={'came_from': came_from}))
@@ -1249,6 +1268,7 b' class NotAnonymous(object):'
         return get_cython_compat_decorator(self.__wrapper, func)
 
     def __wrapper(self, func, *fargs, **fkwargs):
+        import rhodecode.lib.helpers as h
         cls = fargs[0]
         self.user = cls._rhodecode_user
 
@@ -1258,8 +1278,6 b' class NotAnonymous(object):'
 
         if anonymous:
             came_from = request.path_qs
-
-            import rhodecode.lib.helpers as h
             h.flash(_('You need to be a registered user to '
                       'perform this action'),
                     category='warning')
@@ -1296,6 +1314,7 b' class HasAcceptedRepoType(object):'
         return get_cython_compat_decorator(self.__wrapper, func)
 
     def __wrapper(self, func, *fargs, **fkwargs):
+        import rhodecode.lib.helpers as h
         cls = fargs[0]
         rhodecode_repo = cls.rhodecode_repo
 
@@ -1306,7 +1325,6 b' class HasAcceptedRepoType(object):'
         if rhodecode_repo.alias in self.repo_type_list:
             return func(*fargs, **fkwargs)
         else:
-            import rhodecode.lib.helpers as h
             h.flash(h.literal(
                 _('Action not supported for %s.' % rhodecode_repo.alias)),
                 category='warning')
@@ -1326,7 +1344,22 b' class PermsDecorator(object):'
     def __call__(self, func):
         return get_cython_compat_decorator(self.__wrapper, func)
 
+    def _get_request(self):
+        from pyramid.threadlocal import get_current_request
+        pyramid_request = get_current_request()
+        if not pyramid_request:
+            # return global request of pylons in case pyramid isn't available
+            return request
+        return pyramid_request
+
+    def _get_came_from(self):
+        _request = self._get_request()
+
+        # both pylons/pyramid has this attribute
+        return _request.path_qs
+
     def __wrapper(self, func, *fargs, **fkwargs):
+        import rhodecode.lib.helpers as h
         cls = fargs[0]
         _user = cls._rhodecode_user
 
@@ -1342,17 +1375,15 b' class PermsDecorator(object):'
             anonymous = _user.username == User.DEFAULT_USER
 
             if anonymous:
-                came_from = request.path_qs
-
-                import rhodecode.lib.helpers as h
+                came_from = self._get_came_from()
                 h.flash(_('You need to be signed in to view this page'),
-                        category='warning')
-                return redirect(
+                               category='warning')
+                raise HTTPFound(
                     h.route_path('login', _query={'came_from': came_from}))
 
             else:
                 # redirect with forbidden ret code
-                return abort(403)
+                raise HTTPForbidden()
 
     def check_permissions(self, user):
         """Dummy function for overriding"""
@@ -1391,10 +1422,13 b' class HasRepoPermissionAllDecorator(Perm'
     Checks for access permission for all given predicates for specific
     repository. All of them have to be meet in order to fulfill the request
     """
+    def _get_repo_name(self):
+        _request = self._get_request()
+        return get_repo_slug(_request)
 
     def check_permissions(self, user):
         perms = user.permissions
-        repo_name = get_repo_slug(request)
+        repo_name = self._get_repo_name()
         try:
             user_perms = set([perms['repositories'][repo_name]])
         except KeyError:
@@ -1409,10 +1443,13 b' class HasRepoPermissionAnyDecorator(Perm'
     Checks for access permission for any of given predicates for specific
     repository. In order to fulfill the request any of predicates must be meet
     """
+    def _get_repo_name(self):
+        _request = self._get_request()
+        return get_repo_slug(_request)
 
     def check_permissions(self, user):
         perms = user.permissions
-        repo_name = get_repo_slug(request)
+        repo_name = self._get_repo_name()
         try:
             user_perms = set([perms['repositories'][repo_name]])
         except KeyError:
@@ -1429,10 +1466,13 b' class HasRepoGroupPermissionAllDecorator'
     repository group. All of them have to be meet in order to
     fulfill the request
     """
+    def _get_repo_group_name(self):
+        _request = self._get_request()
+        return get_repo_group_slug(_request)
 
     def check_permissions(self, user):
         perms = user.permissions
-        group_name = get_repo_group_slug(request)
+        group_name = self._get_repo_group_name()
         try:
             user_perms = set([perms['repositories_groups'][group_name]])
         except KeyError:
@@ -1449,10 +1489,13 b' class HasRepoGroupPermissionAnyDecorator'
     repository group. In order to fulfill the request any
     of predicates must be met
     """
+    def _get_repo_group_name(self):
+        _request = self._get_request()
+        return get_repo_group_slug(_request)
 
     def check_permissions(self, user):
         perms = user.permissions
-        group_name = get_repo_group_slug(request)
+        group_name = self._get_repo_group_name()
         try:
             user_perms = set([perms['repositories_groups'][group_name]])
         except KeyError:
@@ -1468,10 +1511,13 b' class HasUserGroupPermissionAllDecorator'
     Checks for access permission for all given predicates for specific
     user group. All of them have to be meet in order to fulfill the request
     """
+    def _get_user_group_name(self):
+        _request = self._get_request()
+        return get_user_group_slug(_request)
 
     def check_permissions(self, user):
         perms = user.permissions
-        group_name = get_user_group_slug(request)
+        group_name = self._get_user_group_name()
         try:
             user_perms = set([perms['user_groups'][group_name]])
         except KeyError:
@@ -1487,10 +1533,13 b' class HasUserGroupPermissionAnyDecorator'
     Checks for access permission for any of given predicates for specific
     user group. In order to fulfill the request any of predicates must be meet
     """
+    def _get_user_group_name(self):
+        _request = self._get_request()
+        return get_user_group_slug(_request)
 
     def check_permissions(self, user):
         perms = user.permissions
-        group_name = get_user_group_slug(request)
+        group_name = self._get_user_group_name()
         try:
             user_perms = set([perms['user_groups'][group_name]])
         except KeyError:
@@ -1553,6 +1602,14 b' class PermsFunction(object):'
                       check_scope, user, check_location)
             return False
 
+    def _get_request(self):
+        from pyramid.threadlocal import get_current_request
+        pyramid_request = get_current_request()
+        if not pyramid_request:
+            # return global request of pylons incase pyramid one isn't available
+            return request
+        return pyramid_request
+
     def _get_check_scope(self, cls_name):
         return {
             'HasPermissionAll':          'GLOBAL',
@@ -1591,10 +1648,14 b' class HasRepoPermissionAll(PermsFunction'
         self.repo_name = repo_name
         return super(HasRepoPermissionAll, self).__call__(check_location, user)
 
-    def check_permissions(self, user):
+    def _get_repo_name(self):
         if not self.repo_name:
-            self.repo_name = get_repo_slug(request)
+            _request = self._get_request()
+            self.repo_name = get_repo_slug(_request)
+        return self.repo_name
 
+    def check_permissions(self, user):
+        self.repo_name = self._get_repo_name()
         perms = user.permissions
         try:
             user_perms = set([perms['repositories'][self.repo_name]])
@@ -1610,10 +1671,13 b' class HasRepoPermissionAny(PermsFunction'
         self.repo_name = repo_name
         return super(HasRepoPermissionAny, self).__call__(check_location, user)
 
-    def check_permissions(self, user):
+    def _get_repo_name(self):
         if not self.repo_name:
             self.repo_name = get_repo_slug(request)
+        return self.repo_name
 
+    def check_permissions(self, user):
+        self.repo_name = self._get_repo_name()
         perms = user.permissions
         try:
             user_perms = set([perms['repositories'][self.repo_name]])
@@ -1905,3 +1969,5 b' def get_cython_compat_decorator(wrapper,'
         return wrapper(func, *args, **kwds)
     local_wrapper.__wrapped__ = func
     return local_wrapper
+
+

rhodecode/lib/base.py

@@ -209,11 +209,12 b' def vcs_operation_context('
 class BasicAuth(AuthBasicAuthenticator):
 
     def __init__(self, realm, authfunc, registry, auth_http_code=None,
-                 initial_call_detection=False):
+                 initial_call_detection=False, acl_repo_name=None):
         self.realm = realm
         self.initial_call = initial_call_detection
         self.authfunc = authfunc
         self.registry = registry
+        self.acl_repo_name = acl_repo_name
         self._rc_auth_http_code = auth_http_code
 
     def _get_response_from_code(self, http_code):
@@ -247,7 +248,7 b' class BasicAuth(AuthBasicAuthenticator):'
             username, password = _parts
             if self.authfunc(
                     username, password, environ, VCS_TYPE,
-                    registry=self.registry):
+                    registry=self.registry, acl_repo_name=self.acl_repo_name):
                 return username
             if username and password:
                 # we mark that we actually executed authentication once, at
@@ -488,21 +489,12 b' class BaseController(WSGIController):'
             _route_name)
         )
 
-        # TODO: Maybe this should be move to pyramid to cover all views.
-        # check user attributes for password change flag
         user_obj = auth_user.get_instance()
         if user_obj and user_obj.user_data.get('force_password_change'):
             h.flash('You are required to change your password', 'warning',
                     ignore_duplicate=True)
-
-            skip_user_check_urls = [
-                'error.document', 'login.logout', 'login.index',
-                'admin/my_account.my_account_password',
-                'admin/my_account.my_account_password_update'
-            ]
-            if _route_name not in skip_user_check_urls:
-                return self._dispatch_redirect(
-                    url('my_account_password'), environ, start_response)
+            return self._dispatch_redirect(
+                url('my_account_password'), environ, start_response)
 
         return WSGIController.__call__(self, environ, start_response)
 

rhodecode/lib/celerylib/__init__.py

@@ -226,7 +226,6 b' def vcsconnection(func):'
             for alias in rhodecode.BACKENDS.keys():
                 if alias not in backends:
                     del rhodecode.BACKENDS[alias]
-            utils.configure_pyro4(settings)
             utils.configure_vcs(settings)
             connect_vcs(
                 settings['vcs.server'],

rhodecode/lib/codeblocks.py

@@ -25,7 +25,7 b' from itertools import groupby'
 from pygments import lex
 from pygments.formatters.html import _get_ttype_class as pygment_token_class
 from rhodecode.lib.helpers import (
-    get_lexer_for_filenode, html_escape)
+    get_lexer_for_filenode, html_escape, get_custom_lexer)
 from rhodecode.lib.utils2 import AttributeDict
 from rhodecode.lib.vcs.nodes import FileNode
 from rhodecode.lib.diff_match_patch import diff_match_patch
@@ -407,8 +407,12 b' class DiffSet(object):'
         if filename not in self._lexer_cache:
             if filenode:
                 lexer = filenode.lexer
+                extension = filenode.extension
             else:
                 lexer = FileNode.get_lexer(filename=filename)
+                extension = filename.split('.')[-1]
+
+            lexer = get_custom_lexer(extension) or lexer
             self._lexer_cache[filename] = lexer
         return self._lexer_cache[filename]
 

rhodecode/lib/db_manage.py

@@ -292,13 +292,13 b' class DbManage(object):'
             TEST_USER_REGULAR2_PASS, TEST_USER_REGULAR2_EMAIL
 
         self.create_user(TEST_USER_ADMIN_LOGIN, TEST_USER_ADMIN_PASS,
-                         TEST_USER_ADMIN_EMAIL, True)
+                         TEST_USER_ADMIN_EMAIL, True, api_key=True)
 
         self.create_user(TEST_USER_REGULAR_LOGIN, TEST_USER_REGULAR_PASS,
-                         TEST_USER_REGULAR_EMAIL, False)
+                         TEST_USER_REGULAR_EMAIL, False, api_key=True)
 
         self.create_user(TEST_USER_REGULAR2_LOGIN, TEST_USER_REGULAR2_PASS,
-                         TEST_USER_REGULAR2_EMAIL, False)
+                         TEST_USER_REGULAR2_EMAIL, False, api_key=True)
 
     def create_ui_settings(self, repo_store_path):
         """
@@ -306,6 +306,8 b' class DbManage(object):'
         and disables dotencode
         """
         settings_model = SettingsModel(sa=self.sa)
+        from rhodecode.lib.vcs.backends.hg import largefiles_store
+        from rhodecode.lib.vcs.backends.git import lfs_store
 
         # Build HOOKS
         hooks = [
@@ -315,6 +317,7 b' class DbManage(object):'
             (RhodeCodeUi.HOOK_PRE_PULL, 'python:vcsserver.hooks.pre_pull'),
             (RhodeCodeUi.HOOK_PULL, 'python:vcsserver.hooks.log_pull_action'),
             (RhodeCodeUi.HOOK_PRE_PUSH, 'python:vcsserver.hooks.pre_push'),
+            (RhodeCodeUi.HOOK_PRETX_PUSH, 'python:vcsserver.hooks.pre_push'),
             (RhodeCodeUi.HOOK_PUSH, 'python:vcsserver.hooks.log_push_action'),
 
         ]
@@ -335,14 +338,23 b' class DbManage(object):'
         self.sa.add(largefiles)
 
         # set default largefiles cache dir, defaults to
-        # /repo location/.cache/largefiles
+        # /repo_store_location/.cache/largefiles
         largefiles = RhodeCodeUi()
         largefiles.ui_section = 'largefiles'
         largefiles.ui_key = 'usercache'
-        largefiles.ui_value = os.path.join(repo_store_path, '.cache',
-                                           'largefiles')
+        largefiles.ui_value = largefiles_store(repo_store_path)
+
         self.sa.add(largefiles)
 
+        # set default lfs cache dir, defaults to
+        # /repo_store_location/.cache/lfs_store
+        lfsstore = RhodeCodeUi()
+        lfsstore.ui_section = 'vcs_git_lfs'
+        lfsstore.ui_key = 'store_location'
+        lfsstore.ui_value = lfs_store(repo_store_path)
+
+        self.sa.add(lfsstore)
+
         # enable hgsubversion disabled by default
         hgsubversion = RhodeCodeUi()
         hgsubversion.ui_section = 'extensions'
@@ -506,12 +518,12 b' class DbManage(object):'
         self.create_ui_settings(path)
 
         ui_config = [
-            ('web', 'push_ssl', 'false'),
+            ('web', 'push_ssl', 'False'),
             ('web', 'allow_archive', 'gz zip bz2'),
             ('web', 'allow_push', '*'),
             ('web', 'baseurl', '/'),
             ('paths', '/', path),
-            ('phases', 'publish', 'true')
+            ('phases', 'publish', 'True')
         ]
         for section, key, value in ui_config:
             ui_conf = RhodeCodeUi()
@@ -560,7 +572,9 b' class DbManage(object):'
 
         if api_key:
             log.info('setting a provided api key for the user %s', username)
-            user.api_key = api_key
+            from rhodecode.model.auth_token import AuthTokenModel
+            AuthTokenModel().create(
+                user=user, description='BUILTIN TOKEN')
 
     def create_default_user(self):
         log.info('creating default user')