release: merge back stable branch into default

docs/release-notes/release-notes-4.11.4.rst

@@ -0,0 +1,44 b''
+|RCE| 4.11.4 |RNS|
+------------------
+
+Release Date
+^^^^^^^^^^^^
+
+- 2018-02-18
+
+
+New Features
+^^^^^^^^^^^^
+
+
+
+General
+^^^^^^^
+
+
+
+Security
+^^^^^^^^
+
+
+
+Performance
+^^^^^^^^^^^
+
+
+
+Fixes
+^^^^^
+
+- Ssh: fixed problems with svn ssh clones in certain configurations.
+- Caches: use individual namespaces per user to prevent beaker caching problems in database backends.
+  In some cases of large LDAP data we could generate mysql table errors
+  with too big data. Now each user will use individual row in db cache
+- Reviewers: fixed logic with wildcard (*) match for source/target branches.
+  Reverting a regression when * didn't work as expected in branch matching.
+
+
+Upgrade notes
+^^^^^^^^^^^^^
+
+- Unscheduled bugfix release fixing several reported issues.

.hgtags

@@ -1,34 +1,35 b''
 1bd3e92b7e2e2d2024152b34bb88dff1db544a71 v4.0.0
 170c5398320ea6cddd50955e88d408794c21d43a v4.0.1
 c3fe200198f5aa34cf2e4066df2881a9cefe3704 v4.1.0
 7fd5c850745e2ea821fb4406af5f4bff9b0a7526 v4.1.1
 41c87da28a179953df86061d817bc35533c66dd2 v4.1.2
 baaf9f5bcea3bae0ef12ae20c8b270482e62abb6 v4.2.0
 32a70c7e56844a825f61df496ee5eaf8c3c4e189 v4.2.1
 fa695cdb411d294679ac081d595ac654e5613b03 v4.3.0
 0e4dc11b58cad833c513fe17bac39e6850edf959 v4.3.1
 8a876f48f5cb1d018b837db28ff928500cb32cfb v4.4.0
 8dd86b410b1aac086ffdfc524ef300f896af5047 v4.4.1
 d2514226abc8d3b4f6fb57765f47d1b6fb360a05 v4.4.2
 27d783325930af6dad2741476c0d0b1b7c8415c2 v4.5.0
 7f2016f352abcbdba4a19d4039c386e9629449da v4.5.1
 416fec799314c70a5c780fb28b3357b08869333a v4.5.2
 27c3b85fafc83143e6678fbc3da69e1615bcac55 v4.6.0
 5ad13deb9118c2a5243d4032d4d9cc174e5872db v4.6.1
 2be921e01fa24bb102696ada596f87464c3666f6 v4.7.0
 7198bdec29c2872c974431d55200d0398354cdb1 v4.7.1
 bd1c8d230fe741c2dfd7100a0ef39fd0774fd581 v4.7.2
 9731914f89765d9628dc4dddc84bc9402aa124c8 v4.8.0
 c5a2b7d0e4bbdebc4a62d7b624befe375207b659 v4.9.0
 d9aa3b27ac9f7e78359775c75fedf7bfece232f1 v4.9.1
 4ba4d74981cec5d6b28b158f875a2540952c2f74 v4.10.0
 0a6821cbd6b0b3c21503002f88800679fa35ab63 v4.10.1
 434ad90ec8d621f4416074b84f6e9ce03964defb v4.10.2
 68baee10e698da2724c6e0f698c03a6abb993bf2 v4.10.3
 00821d3afd1dce3f4767cc353f84a17f7d5218a1 v4.10.4
 22f6744ad8cc274311825f63f953e4dee2ea5cb9 v4.10.5
 96eb24bea2f5f9258775245e3f09f6fa0a4dda01 v4.10.6
 3121217a812c956d7dd5a5875821bd73e8002a32 v4.11.0
 fa98b454715ac5b912f39e84af54345909a2a805 v4.11.1
 3982abcfdcc229a723cebe52d3a9bcff10bba08e v4.11.2
 33195f145db9172f0a8f1487e09207178a6ab065 v4.11.3
+194c74f33e32bbae6fc4d71ec5a999cff3c13605 v4.11.4

docs/issue-trackers/issue-trackers.rst

@@ -1,81 +1,81 b''
 .. _rhodecode-issue-trackers-ref:
 
 Issue Tracker Integration
 =========================
 
 You can set an issue tracker connection in two ways with |RCE|.
 
 * At the instance level, you can set a default issue tracker.
 * At the |repo| level, you can configure an integration with a different issue
   tracker.
 
 To integrate |RCM| with an issue tracker, you need to define a regular
 expression that will fetch the issue ID stored in commit messages, and replace
 it with a URL. This enables |RCE| to generate a link matching each issue to the
 target |repo|.
 
 Default Issue Tracker Configuration
 -----------------------------------
 
 To integrate your issue tracker, use the following steps:
 
 1. Open :menuselection:`Admin --> Settings --> Issue Tracker`.
 2. In the new entry field, enter the following information:
 
     * :guilabel:`Description`: A name for this set of rules.
     * :guilabel:`Pattern`: The regular expression that will match issues
       tagged in commit messages, or more see :ref:`issue-tr-eg-ref`.
     * :guilabel:`URL`: The URL to your issue tracker.
     * :guilabel:`Prefix`: The prefix with which you want to mark issues.
 
 3. Select **Add** so save the rule to your issue tracker configuration.
 
 Repository Issue Tracker Configuration
 --------------------------------------
 
 You can configure specific |repos| to use a different issue tracker than the
 default one. See the instructions in :ref:`repo-it`
 
 .. _issue-tr-eg-ref:
 
 Jira Integration
 ----------------
 
 * Regex = ``(?:^#|\s#)(\w+-\d+)``
-* URL = ``https://myissueserver.com/issue/${id}``
+* URL = ``https://myissueserver.com/browse/${id}``
 * Issue Prefix = ``#``
 
 Confluence (Wiki)
 -----------------
 
 * Regex = ``(?:conf-)([A-Z0-9]+)``
 * URL = ``https://example.atlassian.net/display/wiki/${id}/${repo_name}``
 * issue prefix = ``CONF-``
 
 Redmine Integration
 -------------------
 
 * Regex = ``(issue-+\d+)``
 * URL = ``https://myissueserver.com/redmine/issue/${id}``
 * Issue Prefix = ``issue-``
 
 Redmine (wiki)
 --------------
 
 * Regex = ``(?:wiki-)([a-zA-Z0-9]+)``
 * URL = ``https://example.com/redmine/projects/wiki/${repo_name}``
 * Issue prefix = ``Issue-``
 
 Pivotal Tracker
 ---------------
 
 * Regex = ``(?:pivot-)(?<project_id>\d+)-(?<story>\d+)``
 * URL = ``https://www.pivotaltracker.com/s/projects/${project_id}/stories/${story}``
 * Issue prefix = ``Piv-``
 
 Trello
 ------
 
 * Regex = ``(?:trello-)(?<card_id>[a-zA-Z0-9]+)``
 * URL = ``https://trello.com/example.com/${card_id}``
 * Issue prefix = ``Trello-``

docs/release-notes/release-notes.rst

@@ -1,111 +1,112 b''
 .. _rhodecode-release-notes-ref:
 
 Release Notes
 =============
 
 |RCE| 4.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
+   release-notes-4.11.4.rst
    release-notes-4.11.3.rst
    release-notes-4.11.2.rst
    release-notes-4.11.1.rst
    release-notes-4.11.0.rst
    release-notes-4.10.6.rst
    release-notes-4.10.5.rst
    release-notes-4.10.4.rst
    release-notes-4.10.3.rst
    release-notes-4.10.2.rst
    release-notes-4.10.1.rst
    release-notes-4.10.0.rst
    release-notes-4.9.1.rst
    release-notes-4.9.0.rst
    release-notes-4.8.0.rst
    release-notes-4.7.2.rst
    release-notes-4.7.1.rst
    release-notes-4.7.0.rst
    release-notes-4.6.1.rst
    release-notes-4.6.0.rst
    release-notes-4.5.2.rst
    release-notes-4.5.1.rst
    release-notes-4.5.0.rst
    release-notes-4.4.2.rst
    release-notes-4.4.1.rst
    release-notes-4.4.0.rst
    release-notes-4.3.1.rst
    release-notes-4.3.0.rst
    release-notes-4.2.1.rst
    release-notes-4.2.0.rst
    release-notes-4.1.2.rst
    release-notes-4.1.1.rst
    release-notes-4.1.0.rst
    release-notes-4.0.1.rst
    release-notes-4.0.0.rst
 
 |RCE| 3.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-3.8.4.rst
    release-notes-3.8.3.rst
    release-notes-3.8.2.rst
    release-notes-3.8.1.rst
    release-notes-3.8.0.rst
    release-notes-3.7.1.rst
    release-notes-3.7.0.rst
    release-notes-3.6.1.rst
    release-notes-3.6.0.rst
    release-notes-3.5.2.rst
    release-notes-3.5.1.rst
    release-notes-3.5.0.rst
    release-notes-3.4.1.rst
    release-notes-3.4.0.rst
    release-notes-3.3.4.rst
    release-notes-3.3.3.rst
    release-notes-3.3.2.rst
    release-notes-3.3.1.rst
    release-notes-3.3.0.rst
    release-notes-3.2.3.rst
    release-notes-3.2.2.rst
    release-notes-3.2.1.rst
    release-notes-3.2.0.rst
    release-notes-3.1.1.rst
    release-notes-3.1.0.rst
    release-notes-3.0.2.rst
    release-notes-3.0.1.rst
    release-notes-3.0.0.rst
 
 |RCE| 2.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-2.2.8.rst
    release-notes-2.2.7.rst
    release-notes-2.2.6.rst
    release-notes-2.2.5.rst
    release-notes-2.2.4.rst
    release-notes-2.2.3.rst
    release-notes-2.2.2.rst
    release-notes-2.2.1.rst
    release-notes-2.2.0.rst
    release-notes-2.1.0.rst
    release-notes-2.0.2.rst
    release-notes-2.0.1.rst
    release-notes-2.0.0.rst
 
 |RCE| 1.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-1.7.2.rst
    release-notes-1.7.1.rst
    release-notes-1.7.0.rst
    release-notes-1.6.0.rst

rhodecode/apps/ssh_support/lib/backends/svn.py

@@ -1,228 +1,227 b''
 # -*- coding: utf-8 -*-
 
 # Copyright (C) 2016-2018 RhodeCode GmbH
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License, version 3
 # (only), as published by the Free Software Foundation.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 # This program is dual-licensed. If you wish to learn more about the
 # RhodeCode Enterprise Edition, including its added features, Support services,
 # and proprietary license terms, please see https://rhodecode.com/licenses/
 
 import os
 import re
 import sys
 import logging
 import signal
 import tempfile
 from subprocess import Popen, PIPE
 import urlparse
 
 from .base import VcsServer
 
 log = logging.getLogger(__name__)
 
 
 class SubversionTunnelWrapper(object):
     process = None
 
     def __init__(self, server):
         self.server = server
         self.timeout = 30
         self.stdin = sys.stdin
         self.stdout = sys.stdout
         self.svn_conf_fd, self.svn_conf_path = tempfile.mkstemp()
         self.hooks_env_fd, self.hooks_env_path = tempfile.mkstemp()
 
         self.read_only = True  # flag that we set to make the hooks readonly
 
     def create_svn_config(self):
         content = (
             '[general]\n'
             'hooks-env = {}\n').format(self.hooks_env_path)
         with os.fdopen(self.svn_conf_fd, 'w') as config_file:
             config_file.write(content)
 
     def create_hooks_env(self):
         content = (
             '[default]\n'
             'LANG = en_US.UTF-8\n')
         if self.read_only:
             content += 'SSH_READ_ONLY = 1\n'
         with os.fdopen(self.hooks_env_fd, 'w') as hooks_env_file:
             hooks_env_file.write(content)
 
     def remove_configs(self):
         os.remove(self.svn_conf_path)
         os.remove(self.hooks_env_path)
 
     def command(self):
         root = self.server.get_root_store()
         command = [
             self.server.svn_path, '-t',
             '--config-file', self.svn_conf_path,
             '-r', root]
-        log.debug("Final CMD: %s", command)
+        log.debug("Final CMD: %s", ' '.join(command))
         return command
 
     def start(self):
         command = self.command()
-        self.process = Popen(command, stdin=PIPE)
+        self.process = Popen(' '.join(command), stdin=PIPE, shell=True)
 
     def sync(self):
         while self.process.poll() is None:
             next_byte = self.stdin.read(1)
             if not next_byte:
                 break
             self.process.stdin.write(next_byte)
         self.remove_configs()
 
     @property
     def return_code(self):
         return self.process.returncode
 
     def get_first_client_response(self):
         signal.signal(signal.SIGALRM, self.interrupt)
         signal.alarm(self.timeout)
         first_response = self._read_first_client_response()
         signal.alarm(0)
         return (
             self._parse_first_client_response(first_response)
             if first_response else None)
 
     def patch_first_client_response(self, response, **kwargs):
         self.create_hooks_env()
         data = response.copy()
         data.update(kwargs)
         data['url'] = self._svn_string(data['url'])
         data['ra_client'] = self._svn_string(data['ra_client'])
         data['client'] = data['client'] or ''
         buffer_ = (
             "( {version} ( {capabilities} ) {url}{ra_client}"
             "( {client}) ) ".format(**data))
         self.process.stdin.write(buffer_)
 
     def fail(self, message):
         print(
             "( failure ( ( 210005 {message} 0: 0 ) ) )".format(
                 message=self._svn_string(message)))
         self.remove_configs()
         self.process.kill()
 
     def interrupt(self, signum, frame):
         self.fail("Exited by timeout")
 
     def _svn_string(self, str_):
         if not str_:
             return ''
         return '{length}:{string} '.format(length=len(str_), string=str_)
 
     def _read_first_client_response(self):
         buffer_ = ""
         brackets_stack = []
         while True:
             next_byte = self.stdin.read(1)
             buffer_ += next_byte
             if next_byte == "(":
                 brackets_stack.append(next_byte)
             elif next_byte == ")":
                 brackets_stack.pop()
             elif next_byte == " " and not brackets_stack:
                 break
         return buffer_
 
     def _parse_first_client_response(self, buffer_):
         """
         According to the Subversion RA protocol, the first request
         should look like:
 
         ( version:number ( cap:word ... ) url:string ? ra-client:string
            ( ? client:string ) )
 
         Please check https://svn.apache.org/repos/asf/subversion/trunk/
            subversion/libsvn_ra_svn/protocol
         """
         version_re = r'(?P<version>\d+)'
         capabilities_re = r'\(\s(?P<capabilities>[\w\d\-\ ]+)\s\)'
         url_re = r'\d+\:(?P<url>[\W\w]+)'
         ra_client_re = r'(\d+\:(?P<ra_client>[\W\w]+)\s)'
         client_re = r'(\d+\:(?P<client>[\W\w]+)\s)*'
         regex = re.compile(
             r'^\(\s{version}\s{capabilities}\s{url}\s{ra_client}'
             r'\(\s{client}\)\s\)\s*$'.format(
                 version=version_re, capabilities=capabilities_re,
                 url=url_re, ra_client=ra_client_re, client=client_re))
         matcher = regex.match(buffer_)
         return matcher.groupdict() if matcher else None
 
     def run(self, extras):
         action = 'pull'
         self.create_svn_config()
         self.start()
 
         first_response = self.get_first_client_response()
         if not first_response:
             self.fail("Repository name cannot be extracted")
-            return 1
 
         url_parts = urlparse.urlparse(first_response['url'])
         self.server.repo_name = url_parts.path.strip('/')
 
         exit_code = self.server._check_permissions(action)
         if exit_code:
             return exit_code
 
         # set the readonly flag to False if we have proper permissions
         if self.server.has_write_perm():
             self.read_only = False
         self.server.update_environment(action=action, extras=extras)
 
         self.patch_first_client_response(first_response)
         self.sync()
         return self.return_code
 
 
 class SubversionServer(VcsServer):
     backend = 'svn'
 
     def __init__(self, store, ini_path, repo_name,
                  user, user_permissions, config, env):
         super(SubversionServer, self)\
             .__init__(user, user_permissions, config, env)
         self.store = store
         self.ini_path = ini_path
         # this is set in .run() from input stream
         self.repo_name = repo_name
         self._path = self.svn_path = config.get(
             'app:main', 'ssh.executable.svn')
 
         self.tunnel = SubversionTunnelWrapper(server=self)
 
     def _handle_tunnel(self, extras):
 
         # pre-auth
         action = 'pull'
         # Special case for SVN, we extract repo name at later stage
         # exit_code = self._check_permissions(action)
         # if exit_code:
         #     return exit_code, False
 
         req = self.env['request']
         server_url = req.host_url + req.script_name
         extras['server_url'] = server_url
 
         log.debug('Using %s binaries from path %s', self.backend, self._path)
         exit_code = self.tunnel.run(extras)
 
         return exit_code, action == "push"