release: Merge default into stable for release preparation

docs/admin/restore-deleted-repositories.rst

@@ -0,0 +1,42 b''
+.. _restore-deleted-repositories:
+
+Restoring Deleted Repositories
+==============================
+
+By default when repository or whole repository group is deleted an archived copy
+of filesystem repositories are kept. You can see them as special entries in the
+repository storage such as::
+
+    drwxrwxr-x   3 rcdev rcdev  4096 Dec  4  2017 rm__20171204_105727_400795__ce-import
+    drwxrwxr-x   6 rcdev rcdev  4096 Nov 21  2017 rm__20180221_152430_675047__svn-repo
+    drwxr-xr-x   7 rcdev rcdev  4096 Mar 28  2018 rm__20180328_143124_617576__test-git
+    drwxr-xr-x   7 rcdev rcdev  4096 Mar 28  2018 rm__20180328_144954_317729__test-git-install-hooks
+
+
+Data from those repositories can be restored by simply removing the
+`rm_YYYYDDMM_HHMMSS_DDDDDD__` prefix and additionally only in case of Mercurial
+repositories remove the `.hg` store prefix.::
+
+    rm__.hg => .hg
+
+
+For Git or SVN repositories this operation is not required.
+
+After removing the prefix repository can be brought by opening
+:menuselection:`Admin --> Settings --> Remap and Rescan` and running `Rescan Filesystem`
+
+This will create a new DB entry restoring the data previously removed.
+To restore OLD database entries this should be done by restoring from a Database backup.
+
+RhodeCode also keeps the repository group structure, this is marked by entries that
+in addition have GROUP in the prefix, eg::
+
+    drwxr-xr-x   2 rcdev rcdev  4096 Jan 18 16:13 rm__20181130_120650_977082_GROUP_Test1
+    drwxr-xr-x   2 rcdev rcdev  4096 Jan 18 16:13 rm__20181130_120659_922952_GROUP_Docs
+
+
+
+.. note::
+
+    RhodeCode Tools have a special cleanup tool for the archived repositories. Please
+    see :ref:`clean-up-cmds`

docs/api/methods/store-methods.rst

@@ -0,0 +1,37 b''
+.. _store-methods-ref:
+
+store methods
+=============
+
+file_store_add (EE only)
+------------------------
+
+.. py:function:: file_store_add(apiuser, filename, content)
+
+   Upload API for the file_store
+
+   Example usage from CLI::
+       rhodecode-api --instance-name=enterprise-1 upload_file "{"content": "$(cat image.jpg | base64)", "filename":"image.jpg"}"
+
+   This command takes the following options:
+
+   :param apiuser: This is filled automatically from the |authtoken|.
+   :type apiuser: AuthUser
+   :param filename: name of the file uploaded
+   :type filename: str
+   :param content: base64 encoded content of the uploaded file
+   :type content: str
+
+   Example output:
+
+   .. code-block:: bash
+
+     id : <id_given_in_input>
+     result: {
+       "access_path": "/_file_store/download/84d156f7-8323-4ad3-9fce-4a8e88e1deaf-0.jpg",
+       "access_path_fqn": "http://server.domain.com/_file_store/download/84d156f7-8323-4ad3-9fce-4a8e88e1deaf-0.jpg",
+       "store_fid": "84d156f7-8323-4ad3-9fce-4a8e88e1deaf-0.jpg"
+     }
+     error :  null
+
+

docs/auth/auth-saml-bulk-enroll-users.rst

@@ -0,0 +1,88 b''
+.. _auth-saml-bulk-enroll-users-ref:
+
+
+Bulk enroll multiple existing users
+-----------------------------------
+
+
+RhodeCode Supports standard SAML 2.0 SSO for the web-application part.
+Below is an example how to enroll list of all or some users to use SAML authentication.
+This method simply enables SAML authentication for many users at once.
+
+
+From the server RhodeCode Enterprise is running run ishell on the instance which we
+want to apply the SAML migration::
+
+    rccontrol ishell enterprise-1
+
+Follow these steps to enable SAML authentication for multiple users.
+
+
+1) Create a user_id => attribute mapping
+
+
+`saml2user` is a mapping of external ID from SAML provider such as OneLogin, DuoSecurity, Google.
+This mapping consists of local rhodecode user_id mapped to set of required attributes needed to bind SAML
+account to internal rhodecode user.
+For example, 123 is local rhodecode user_id, and '48253211' is OneLogin ID.
+For other providers you'd have to figure out what would be the user-id, sometimes it's the email, i.e for Google
+The most important this id needs to be unique for each user.
+
+.. code-block:: python
+
+    In [1]: saml2user = {
+       ...: # OneLogin, uses externalID available to read from in the UI
+       ...: 123: {'id: '48253211'},
+       ...: # for Google/DuoSecurity email is also an option for unique ID
+       ...: 124: {'id: '[e-mail]'},
+       ...: }
+
+
+2) Import the plugin you want to run migration for.
+
+From available options pick only one and run the `import` statement
+
+.. code-block:: python
+
+    # for Duo Security
+    In [2]: from rc_auth_plugins.auth_duo_security import RhodeCodeAuthPlugin
+    # for OneLogin
+    In [2]: from rc_auth_plugins.auth_onelogin import RhodeCodeAuthPlugin
+    # generic SAML plugin
+    In [2]: from rc_auth_plugins.auth_saml import RhodeCodeAuthPlugin
+
+3) Run the migration based on saml2user mapping.
+
+Enter in the ishell prompt
+
+.. code-block:: python
+
+    In [3]: for user in User.get_all():
+       ...:     existing_identity = ExternalIdentity().query().filter(ExternalIdentity.local_user_id == user.user_id).scalar()
+       ...:     attrs = saml2user.get(user.user_id)
+       ...:     provider = RhodeCodeAuthPlugin.uid
+       ...:     if existing_identity:
+       ...:         print('Identity for user `{}` already exists, skipping'.format(user.username))
+       ...:         continue
+       ...:     if attrs:
+       ...:         external_id = attrs['id']
+       ...:         new_external_identity = ExternalIdentity()
+       ...:         new_external_identity.external_id = external_id
+       ...:         new_external_identity.external_username = '{}-saml-{}'.format(user.username, user.user_id)
+       ...:         new_external_identity.provider_name = provider
+       ...:         new_external_identity.local_user_id = user_id
+       ...:         new_external_identity.access_token = ''
+       ...:         new_external_identity.token_secret = ''
+       ...:         new_external_identity.alt_token = ''
+       ...:         Session().add(ex_identity)
+       ...:         Session().commit()
+       ...:         print('Set user `{}` external identity bound to ExternalID:{}'.format(user.username, external_id))
+
+.. note::
+
+    saml2user can be really big and hard to maintain in ishell. It's also possible
+    to load it as a JSON file prepared before and stored on disk. To do so run::
+
+        import json
+        saml2user = json.loads(open('/path/to/saml2user.json','rb').read())
+

docs/release-notes/release-notes-4.16.0.rst

@@ -0,0 +1,148 b''
+|RCE| 4.16.0 |RNS|
+------------------
+
+Release Date
+^^^^^^^^^^^^
+
+- 2019-02-15
+
+
+New Features
+^^^^^^^^^^^^
+
+
+- Full-text search: added support for ElasticSearch 6.X (ES6)
+- Full-text search: Expose a quick way to search within repository groups using ES6.
+- Full-text search: Add quick links to broaden/narrow search scope to repositories or
+  repository groups from global search.
+- Full-text search: ES6 backend adds new highlighter, and search markers for better UX when searching.
+- Full-text search: ES6 backend has enabled advanced `query string syntax`
+  adding more search and filtering capabilities.
+- Full-text search: ES6 engine will now show added information where available such as line numbers file size.
+- Files: added option to use highlight marker to show keywords inside file source. This
+  is used now for ES6 backend extended highlighting capabilities
+- Artifacts (beta): EE edition exposes new feature called storage_api this allows storing
+  binary files outside of Version Control System, but in the scope of a repository or group.
+  This will soon become an Artifacts functionality available in EE edition.
+- Authentication: introduced `User restriction` and `Scope restriction` for RhodeCode authentication plugins.
+  Admins can limit usage of RhodeCode plugins to super-admins user types, and usage in Web, or VCS protocol only.
+  This is mostly to help to migrate users to SAML, keeping the super-admins to manage instances via local-logins,
+  and secondly to force usage of AuthenticationTokens instead of re-using same credentials for
+  WEB and VCS authentication.
+- API: added basic upload API for the storage_api. It's possible to store files using internal
+  API. This is a start for attachments upload in RhodeCode.
+- API: added store_exception_api for remote exception storage. This is used by a new
+  indexer that will report any problems back into the RhodeCode instance in case of indexing problems.
+- API: added function to fetch comments for a repository.
+- Quick search: improve the styling of search input and results.
+- Pull requests: allowed to select all forks and parent forks of target repository in creation UI.
+  This is a common workflow supported by GitHub etc.
+
+
+General
+^^^^^^^
+
+- Users/Repositories/Repository groups: expose IDs of those objects in advanced views.
+  Useful for API calls or usage in ishell.
+- UI: moved repo group select next to the name as it's very relevant to each other.
+- Pull requests: increase the stability of concurrent pull requests created.
+- Pull requests: introduced operation state for pull requests to prevent from
+  locks during merge/update operations in concurrent busy environments.
+- Pull requests: ensure that merge response provide more details about failed operations.
+- UI / Files: expose downloads options onto files view similar as in summary page.
+- Repositories: show hooks version and update link in the advanced section of repository page.
+- Events: trigger 'review_status_change' in all cases when reviewers are changed
+  influencing review status.
+- Files: display submodules in a sorted way, equal to how Directories are sorted.
+- API: fetching all pull-requests now sorts the results and exposed a flag to show/hide
+  the merge result state for faster result fetching.
+- API: merge_pull_request expose detailed merge message in the merge operation
+  next to numeric merge response code.
+- API: added possibility to specify owner to create_pull_request API.
+- SSH: Added ability to disable server-side SSH key generation to enforce users
+  generated SSH keys only outside of the server.
+- Integrations: allow PUT method for WebHook integration.
+- Dependencies: bumped git to 2.19.2 release.
+- Dependencies: dropped pygments-markdown-lexer as it's natively supported by pygments now.
+- Dependencies: bumped pyramid to 1.10.1
+- Dependencies: bumped pastedeploy to 2.0.1
+- Dependencies: bumped pastescript to 3.0.0
+- Dependencies: bumped pathlib2 to 2.3.3
+- Dependencies: bumped webob to 1.8.4
+- Dependencies: bumped iso8601 to 0.1.12
+- Dependencies: bumped more-itertools to 5.0.0
+- Dependencies: bumped psutil to 5.4.8
+- Dependencies: bumped pyasn1 to 0.4.5
+- Dependencies: bumped pygments to 2.3.1
+- Dependencies: bumped pyramid-debugtoolbar to 4.5.0
+- Dependencies: bumped subprocess32 to 3.5.3
+- Dependencies: bumped supervisor to 3.3.5
+- Dependencies: bumped dogpile.cache to 0.7.1
+- Dependencies: bumped simplejson to 3.16.0
+- Dependencies: bumped gevent to 1.4.0
+- Dependencies: bumped configparser to 3.5.1
+
+
+Security
+^^^^^^^^
+
+- Fork page: don't expose fork origin link if we don't have permission to access this repository.
+  Additionally don't pre-select such repository in pull request ref selector.
+- Security: fix possible XSS in the issue tracker URL.
+- Security: sanitize plaintext renderer with bleach, preventing XSS in rendered html.
+- Audit logs: added audit logs for API permission calls.
+
+
+Performance
+^^^^^^^^^^^
+
+- Summary page: don't load repo size when showing expanded information about repository.
+  Size calculation needs to be triggered manually.
+- Git: use rev-list for fetching last commit data in case of single commit history.
+  In some cases, it is much faster than previously used git log command.
+
+
+Fixes
+^^^^^
+
+- Installer: fixed 32bit package builds broken in previous releases.
+- Git: use iterative fetch to prevent errors about too many arguments on
+  synchronizing very large repositories.
+- Git: pass in the SSL dir that is exposed from wire for remote GIT commands.
+- LDAP+Groups: improve logging, and fix the case when extracting group name from LDAP
+  returned nothing. We should warn about that, but not FAIL on login.
+- Default reviewers: fixed submodule support in picking reviewers from annotation for files.
+- Hooks: handle non-ascii characters in hooks new pull-requests open template.
+- Diffs: fixed missing limited diff container display on over-size limit diffs.
+- Diffs: fixed 500 error in case of some very uncommon diffs containing only Unicode characters.
+- Repositories: handle VCS backend unavailable correctly in advanced settings for the repository.
+- Remap & rescan: prevent empty/damaged repositories to break the remap operation.
+- Visual: fixed show revision/commit length settings.
+- Mercurial submodules: only show submodule in the path that it belongs too.
+  Before even submodules from root node were shown in subdirectories.
+- UI/Files: fixed icons in file tree search.
+- WebHook integration: quote URL variables to prevent URL errors with special chars
+  like # in the title.
+- API: pull-requests, fixed invocation of merge as another user.
+- VCS: limit fd leaks on subprocessio calls.
+- VCS: expose SSL certificate path over the wire to the vcsserver, this solves some
+  remote SSL import problems reported.
+
+
+Upgrade notes
+^^^^^^^^^^^^^
+
+This release brings the new Full-text search capabilities using ElasticSearch 6.
+If you use Elastic Search backend a backward compatibility mode is enabled and
+ElasticSearch backend defaults to previously used ElasticSearch 2.
+
+To use new features a full index rebuild is required, in addition ```--es-version=6``` flag
+needs to be used with indexer and ```search.es_version = 6``` should be set in rhodecode.ini
+
+Additionally new mapping format is available for the indexer that has additional capabilities
+for include/exclude rules. Old format should work as well, but we encourage to
+generate a new mapping.ini file using rhodecode-index command, and migrate your repositories
+to the new format.
+
+Please refer to the :ref:`indexing-ref` documentation for more details.
+

.bumpversion.cfg

@@ -1,6 +1,6 b''
 [bumpversion]
-current_version = 4.15.2
+current_version = 4.16.0
 message = release: Bump version {current_version} to {new_version}
 
 [bumpversion:file:rhodecode/VERSION]
 

.release.cfg

@@ -1,33 +1,28 b''
 [DEFAULT]
 done = false
 
 [task:bump_version]
 done = true
 
 [task:rc_tools_pinned]
-done = true
 
 [task:fixes_on_stable]
-done = true
 
 [task:pip2nix_generated]
-done = true
 
 [task:changelog_updated]
-done = true
 
 [task:generate_api_docs]
-done = true
+
+[task:updated_translation]
 
 [release]
-state = prepared
-version = 4.15.2
-
-[task:updated_translation]
+state = in_progress
+version = 4.16.0
 
 [task:generate_js_routes]
 
 [task:updated_trial_license]
 
 [task:generate_oss_licenses]
 

LICENSE.txt

@@ -1,696 +1,694 b''
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU Affero General Public License, version 3
 (only), as published by the Free Software Foundation.
 
 
 This program incorporates work covered by the following copyright and
 permission notice:
 
    Copyright (c) 2014-2016 - packaging
        file:
    Copyright (c) 2008-2011 - msgpack-python
        file:licenses/msgpack_license.txt
    Copyright (c) 2009 - tornado
        file:licenses/tornado_license.txt
-   Copyright (c) 2015 - pygments-markdown-lexer
-       file:licenses/pygments_markdown_lexer_license.txt
    Copyright 2006 - diff_match_patch
        file:licenses/diff_match_patch_license.txt
 
 All licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
 
     http://www.apache.org/licenses/LICENSE-2.0
 
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 imitations under the License.
 
 
 Below is the full text of GNU Affero General Public License, version 3
 
 
                     GNU AFFERO GENERAL PUBLIC LICENSE
                        Version 3, 19 November 2007
 
  Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
  Everyone is permitted to copy and distribute verbatim copies
  of this license document, but changing it is not allowed.
 
                             Preamble
 
   The GNU Affero General Public License is a free, copyleft license for
 software and other kinds of works, specifically designed to ensure
 cooperation with the community in the case of network server software.
 
   The licenses for most software and other practical works are designed
 to take away your freedom to share and change the works.  By contrast,
 our General Public Licenses are intended to guarantee your freedom to
 share and change all versions of a program--to make sure it remains free
 software for all its users.
 
   When we speak of free software, we are referring to freedom, not
 price.  Our General Public Licenses are designed to make sure that you
 have the freedom to distribute copies of free software (and charge for
 them if you wish), that you receive source code or can get it if you
 want it, that you can change the software or use pieces of it in new
 free programs, and that you know you can do these things.
 
   Developers that use our General Public Licenses protect your rights
 with two steps: (1) assert copyright on the software, and (2) offer
 you this License which gives you legal permission to copy, distribute
 and/or modify the software.
 
   A secondary benefit of defending all users' freedom is that
 improvements made in alternate versions of the program, if they
 receive widespread use, become available for other developers to
 incorporate.  Many developers of free software are heartened and
 encouraged by the resulting cooperation.  However, in the case of
 software used on network servers, this result may fail to come about.
 The GNU General Public License permits making a modified version and
 letting the public access it on a server without ever releasing its
 source code to the public.
 
   The GNU Affero General Public License is designed specifically to
 ensure that, in such cases, the modified source code becomes available
 to the community.  It requires the operator of a network server to
 provide the source code of the modified version running there to the
 users of that server.  Therefore, public use of a modified version, on
 a publicly accessible server, gives the public access to the source
 code of the modified version.
 
   An older license, called the Affero General Public License and
 published by Affero, was designed to accomplish similar goals.  This is
 a different license, not a version of the Affero GPL, but Affero has
 released a new version of the Affero GPL which permits relicensing under
 this license.
 
   The precise terms and conditions for copying, distribution and
 modification follow.
 
                        TERMS AND CONDITIONS
 
   0. Definitions.
 
   "This License" refers to version 3 of the GNU Affero General Public License.
 
   "Copyright" also means copyright-like laws that apply to other kinds of
 works, such as semiconductor masks.
 
   "The Program" refers to any copyrightable work licensed under this
 License.  Each licensee is addressed as "you".  "Licensees" and
 "recipients" may be individuals or organizations.
 
   To "modify" a work means to copy from or adapt all or part of the work
 in a fashion requiring copyright permission, other than the making of an
 exact copy.  The resulting work is called a "modified version" of the
 earlier work or a work "based on" the earlier work.
 
   A "covered work" means either the unmodified Program or a work based
 on the Program.
 
   To "propagate" a work means to do anything with it that, without
 permission, would make you directly or secondarily liable for
 infringement under applicable copyright law, except executing it on a
 computer or modifying a private copy.  Propagation includes copying,
 distribution (with or without modification), making available to the
 public, and in some countries other activities as well.
 
   To "convey" a work means any kind of propagation that enables other
 parties to make or receive copies.  Mere interaction with a user through
 a computer network, with no transfer of a copy, is not conveying.
 
   An interactive user interface displays "Appropriate Legal Notices"
 to the extent that it includes a convenient and prominently visible
 feature that (1) displays an appropriate copyright notice, and (2)
 tells the user that there is no warranty for the work (except to the
 extent that warranties are provided), that licensees may convey the
 work under this License, and how to view a copy of this License.  If
 the interface presents a list of user commands or options, such as a
 menu, a prominent item in the list meets this criterion.
 
   1. Source Code.
 
   The "source code" for a work means the preferred form of the work
 for making modifications to it.  "Object code" means any non-source
 form of a work.
 
   A "Standard Interface" means an interface that either is an official
 standard defined by a recognized standards body, or, in the case of
 interfaces specified for a particular programming language, one that
 is widely used among developers working in that language.
 
   The "System Libraries" of an executable work include anything, other
 than the work as a whole, that (a) is included in the normal form of
 packaging a Major Component, but which is not part of that Major
 Component, and (b) serves only to enable use of the work with that
 Major Component, or to implement a Standard Interface for which an
 implementation is available to the public in source code form.  A
 "Major Component", in this context, means a major essential component
 (kernel, window system, and so on) of the specific operating system
 (if any) on which the executable work runs, or a compiler used to
 produce the work, or an object code interpreter used to run it.
 
   The "Corresponding Source" for a work in object code form means all
 the source code needed to generate, install, and (for an executable
 work) run the object code and to modify the work, including scripts to
 control those activities.  However, it does not include the work's
 System Libraries, or general-purpose tools or generally available free
 programs which are used unmodified in performing those activities but
 which are not part of the work.  For example, Corresponding Source
 includes interface definition files associated with source files for
 the work, and the source code for shared libraries and dynamically
 linked subprograms that the work is specifically designed to require,
 such as by intimate data communication or control flow between those
 subprograms and other parts of the work.
 
   The Corresponding Source need not include anything that users
 can regenerate automatically from other parts of the Corresponding
 Source.
 
   The Corresponding Source for a work in source code form is that
 same work.
 
   2. Basic Permissions.
 
   All rights granted under this License are granted for the term of
 copyright on the Program, and are irrevocable provided the stated
 conditions are met.  This License explicitly affirms your unlimited
 permission to run the unmodified Program.  The output from running a
 covered work is covered by this License only if the output, given its
 content, constitutes a covered work.  This License acknowledges your
 rights of fair use or other equivalent, as provided by copyright law.
 
   You may make, run and propagate covered works that you do not
 convey, without conditions so long as your license otherwise remains
 in force.  You may convey covered works to others for the sole purpose
 of having them make modifications exclusively for you, or provide you
 with facilities for running those works, provided that you comply with
 the terms of this License in conveying all material for which you do
 not control copyright.  Those thus making or running the covered works
 for you must do so exclusively on your behalf, under your direction
 and control, on terms that prohibit them from making any copies of
 your copyrighted material outside their relationship with you.
 
   Conveying under any other circumstances is permitted solely under
 the conditions stated below.  Sublicensing is not allowed; section 10
 makes it unnecessary.
 
   3. Protecting Users' Legal Rights From Anti-Circumvention Law.
 
   No covered work shall be deemed part of an effective technological
 measure under any applicable law fulfilling obligations under article
 11 of the WIPO copyright treaty adopted on 20 December 1996, or
 similar laws prohibiting or restricting circumvention of such
 measures.
 
   When you convey a covered work, you waive any legal power to forbid
 circumvention of technological measures to the extent such circumvention
 is effected by exercising rights under this License with respect to
 the covered work, and you disclaim any intention to limit operation or
 modification of the work as a means of enforcing, against the work's
 users, your or third parties' legal rights to forbid circumvention of
 technological measures.
 
   4. Conveying Verbatim Copies.
 
   You may convey verbatim copies of the Program's source code as you
 receive it, in any medium, provided that you conspicuously and
 appropriately publish on each copy an appropriate copyright notice;
 keep intact all notices stating that this License and any
 non-permissive terms added in accord with section 7 apply to the code;
 keep intact all notices of the absence of any warranty; and give all
 recipients a copy of this License along with the Program.
 
   You may charge any price or no price for each copy that you convey,
 and you may offer support or warranty protection for a fee.
 
   5. Conveying Modified Source Versions.
 
   You may convey a work based on the Program, or the modifications to
 produce it from the Program, in the form of source code under the
 terms of section 4, provided that you also meet all of these conditions:
 
     a) The work must carry prominent notices stating that you modified
     it, and giving a relevant date.
 
     b) The work must carry prominent notices stating that it is
     released under this License and any conditions added under section
     7.  This requirement modifies the requirement in section 4 to
     "keep intact all notices".
 
     c) You must license the entire work, as a whole, under this
     License to anyone who comes into possession of a copy.  This
     License will therefore apply, along with any applicable section 7
     additional terms, to the whole of the work, and all its parts,
     regardless of how they are packaged.  This License gives no
     permission to license the work in any other way, but it does not
     invalidate such permission if you have separately received it.
 
     d) If the work has interactive user interfaces, each must display
     Appropriate Legal Notices; however, if the Program has interactive
     interfaces that do not display Appropriate Legal Notices, your
     work need not make them do so.
 
   A compilation of a covered work with other separate and independent
 works, which are not by their nature extensions of the covered work,
 and which are not combined with it such as to form a larger program,
 in or on a volume of a storage or distribution medium, is called an
 "aggregate" if the compilation and its resulting copyright are not
 used to limit the access or legal rights of the compilation's users
 beyond what the individual works permit.  Inclusion of a covered work
 in an aggregate does not cause this License to apply to the other
 parts of the aggregate.
 
   6. Conveying Non-Source Forms.
 
   You may convey a covered work in object code form under the terms
 of sections 4 and 5, provided that you also convey the
 machine-readable Corresponding Source under the terms of this License,
 in one of these ways:
 
     a) Convey the object code in, or embodied in, a physical product
     (including a physical distribution medium), accompanied by the
     Corresponding Source fixed on a durable physical medium
     customarily used for software interchange.
 
     b) Convey the object code in, or embodied in, a physical product
     (including a physical distribution medium), accompanied by a
     written offer, valid for at least three years and valid for as
     long as you offer spare parts or customer support for that product
     model, to give anyone who possesses the object code either (1) a
     copy of the Corresponding Source for all the software in the
     product that is covered by this License, on a durable physical
     medium customarily used for software interchange, for a price no
     more than your reasonable cost of physically performing this
     conveying of source, or (2) access to copy the
     Corresponding Source from a network server at no charge.
 
     c) Convey individual copies of the object code with a copy of the
     written offer to provide the Corresponding Source.  This
     alternative is allowed only occasionally and noncommercially, and
     only if you received the object code with such an offer, in accord
     with subsection 6b.
 
     d) Convey the object code by offering access from a designated
     place (gratis or for a charge), and offer equivalent access to the
     Corresponding Source in the same way through the same place at no
     further charge.  You need not require recipients to copy the
     Corresponding Source along with the object code.  If the place to
     copy the object code is a network server, the Corresponding Source
     may be on a different server (operated by you or a third party)
     that supports equivalent copying facilities, provided you maintain
     clear directions next to the object code saying where to find the
     Corresponding Source.  Regardless of what server hosts the
     Corresponding Source, you remain obligated to ensure that it is
     available for as long as needed to satisfy these requirements.
 
     e) Convey the object code using peer-to-peer transmission, provided
     you inform other peers where the object code and Corresponding
     Source of the work are being offered to the general public at no
     charge under subsection 6d.
 
   A separable portion of the object code, whose source code is excluded
 from the Corresponding Source as a System Library, need not be
 included in conveying the object code work.
 
   A "User Product" is either (1) a "consumer product", which means any
 tangible personal property which is normally used for personal, family,
 or household purposes, or (2) anything designed or sold for incorporation
 into a dwelling.  In determining whether a product is a consumer product,
 doubtful cases shall be resolved in favor of coverage.  For a particular
 product received by a particular user, "normally used" refers to a
 typical or common use of that class of product, regardless of the status
 of the particular user or of the way in which the particular user
 actually uses, or expects or is expected to use, the product.  A product
 is a consumer product regardless of whether the product has substantial
 commercial, industrial or non-consumer uses, unless such uses represent
 the only significant mode of use of the product.
 
   "Installation Information" for a User Product means any methods,
 procedures, authorization keys, or other information required to install
 and execute modified versions of a covered work in that User Product from
 a modified version of its Corresponding Source.  The information must
 suffice to ensure that the continued functioning of the modified object
 code is in no case prevented or interfered with solely because
 modification has been made.
 
   If you convey an object code work under this section in, or with, or
 specifically for use in, a User Product, and the conveying occurs as
 part of a transaction in which the right of possession and use of the
 User Product is transferred to the recipient in perpetuity or for a
 fixed term (regardless of how the transaction is characterized), the
 Corresponding Source conveyed under this section must be accompanied
 by the Installation Information.  But this requirement does not apply
 if neither you nor any third party retains the ability to install
 modified object code on the User Product (for example, the work has
 been installed in ROM).
 
   The requirement to provide Installation Information does not include a
 requirement to continue to provide support service, warranty, or updates
 for a work that has been modified or installed by the recipient, or for
 the User Product in which it has been modified or installed.  Access to a
 network may be denied when the modification itself materially and
 adversely affects the operation of the network or violates the rules and
 protocols for communication across the network.
 
   Corresponding Source conveyed, and Installation Information provided,
 in accord with this section must be in a format that is publicly
 documented (and with an implementation available to the public in
 source code form), and must require no special password or key for
 unpacking, reading or copying.
 
   7. Additional Terms.
 
   "Additional permissions" are terms that supplement the terms of this
 License by making exceptions from one or more of its conditions.
 Additional permissions that are applicable to the entire Program shall
 be treated as though they were included in this License, to the extent
 that they are valid under applicable law.  If additional permissions
 apply only to part of the Program, that part may be used separately
 under those permissions, but the entire Program remains governed by
 this License without regard to the additional permissions.
 
   When you convey a copy of a covered work, you may at your option
 remove any additional permissions from that copy, or from any part of
 it.  (Additional permissions may be written to require their own
 removal in certain cases when you modify the work.)  You may place
 additional permissions on material, added by you to a covered work,
 for which you have or can give appropriate copyright permission.
 
   Notwithstanding any other provision of this License, for material you
 add to a covered work, you may (if authorized by the copyright holders of
 that material) supplement the terms of this License with terms:
 
     a) Disclaiming warranty or limiting liability differently from the
     terms of sections 15 and 16 of this License; or
 
     b) Requiring preservation of specified reasonable legal notices or
     author attributions in that material or in the Appropriate Legal
     Notices displayed by works containing it; or
 
     c) Prohibiting misrepresentation of the origin of that material, or
     requiring that modified versions of such material be marked in
     reasonable ways as different from the original version; or
 
     d) Limiting the use for publicity purposes of names of licensors or
     authors of the material; or
 
     e) Declining to grant rights under trademark law for use of some
     trade names, trademarks, or service marks; or
 
     f) Requiring indemnification of licensors and authors of that
     material by anyone who conveys the material (or modified versions of
     it) with contractual assumptions of liability to the recipient, for
     any liability that these contractual assumptions directly impose on
     those licensors and authors.
 
   All other non-permissive additional terms are considered "further
 restrictions" within the meaning of section 10.  If the Program as you
 received it, or any part of it, contains a notice stating that it is
 governed by this License along with a term that is a further
 restriction, you may remove that term.  If a license document contains
 a further restriction but permits relicensing or conveying under this
 License, you may add to a covered work material governed by the terms
 of that license document, provided that the further restriction does
 not survive such relicensing or conveying.
 
   If you add terms to a covered work in accord with this section, you
 must place, in the relevant source files, a statement of the
 additional terms that apply to those files, or a notice indicating
 where to find the applicable terms.
 
   Additional terms, permissive or non-permissive, may be stated in the
 form of a separately written license, or stated as exceptions;
 the above requirements apply either way.
 
   8. Termination.
 
   You may not propagate or modify a covered work except as expressly
 provided under this License.  Any attempt otherwise to propagate or
 modify it is void, and will automatically terminate your rights under
 this License (including any patent licenses granted under the third
 paragraph of section 11).
 
   However, if you cease all violation of this License, then your
 license from a particular copyright holder is reinstated (a)
 provisionally, unless and until the copyright holder explicitly and
 finally terminates your license, and (b) permanently, if the copyright
 holder fails to notify you of the violation by some reasonable means
 prior to 60 days after the cessation.
 
   Moreover, your license from a particular copyright holder is
 reinstated permanently if the copyright holder notifies you of the
 violation by some reasonable means, this is the first time you have
 received notice of violation of this License (for any work) from that
 copyright holder, and you cure the violation prior to 30 days after
 your receipt of the notice.
 
   Termination of your rights under this section does not terminate the
 licenses of parties who have received copies or rights from you under
 this License.  If your rights have been terminated and not permanently
 reinstated, you do not qualify to receive new licenses for the same
 material under section 10.
 
   9. Acceptance Not Required for Having Copies.
 
   You are not required to accept this License in order to receive or
 run a copy of the Program.  Ancillary propagation of a covered work
 occurring solely as a consequence of using peer-to-peer transmission
 to receive a copy likewise does not require acceptance.  However,
 nothing other than this License grants you permission to propagate or
 modify any covered work.  These actions infringe copyright if you do
 not accept this License.  Therefore, by modifying or propagating a
 covered work, you indicate your acceptance of this License to do so.
 
   10. Automatic Licensing of Downstream Recipients.
 
   Each time you convey a covered work, the recipient automatically
 receives a license from the original licensors, to run, modify and
 propagate that work, subject to this License.  You are not responsible
 for enforcing compliance by third parties with this License.
 
   An "entity transaction" is a transaction transferring control of an
 organization, or substantially all assets of one, or subdividing an
 organization, or merging organizations.  If propagation of a covered
 work results from an entity transaction, each party to that
 transaction who receives a copy of the work also receives whatever
 licenses to the work the party's predecessor in interest had or could
 give under the previous paragraph, plus a right to possession of the
 Corresponding Source of the work from the predecessor in interest, if
 the predecessor has it or can get it with reasonable efforts.
 
   You may not impose any further restrictions on the exercise of the
 rights granted or affirmed under this License.  For example, you may
 not impose a license fee, royalty, or other charge for exercise of
 rights granted under this License, and you may not initiate litigation
 (including a cross-claim or counterclaim in a lawsuit) alleging that
 any patent claim is infringed by making, using, selling, offering for
 sale, or importing the Program or any portion of it.
 
   11. Patents.
 
   A "contributor" is a copyright holder who authorizes use under this
 License of the Program or a work on which the Program is based.  The
 work thus licensed is called the contributor's "contributor version".
 
   A contributor's "essential patent claims" are all patent claims
 owned or controlled by the contributor, whether already acquired or
 hereafter acquired, that would be infringed by some manner, permitted
 by this License, of making, using, or selling its contributor version,
 but do not include claims that would be infringed only as a
 consequence of further modification of the contributor version.  For
 purposes of this definition, "control" includes the right to grant
 patent sublicenses in a manner consistent with the requirements of
 this License.
 
   Each contributor grants you a non-exclusive, worldwide, royalty-free
 patent license under the contributor's essential patent claims, to
 make, use, sell, offer for sale, import and otherwise run, modify and
 propagate the contents of its contributor version.
 
   In the following three paragraphs, a "patent license" is any express
 agreement or commitment, however denominated, not to enforce a patent
 (such as an express permission to practice a patent or covenant not to
 sue for patent infringement).  To "grant" such a patent license to a
 party means to make such an agreement or commitment not to enforce a
 patent against the party.
 
   If you convey a covered work, knowingly relying on a patent license,
 and the Corresponding Source of the work is not available for anyone
 to copy, free of charge and under the terms of this License, through a
 publicly available network server or other readily accessible means,
 then you must either (1) cause the Corresponding Source to be so
 available, or (2) arrange to deprive yourself of the benefit of the
 patent license for this particular work, or (3) arrange, in a manner
 consistent with the requirements of this License, to extend the patent
 license to downstream recipients.  "Knowingly relying" means you have
 actual knowledge that, but for the patent license, your conveying the
 covered work in a country, or your recipient's use of the covered work
 in a country, would infringe one or more identifiable patents in that
 country that you have reason to believe are valid.
 
   If, pursuant to or in connection with a single transaction or
 arrangement, you convey, or propagate by procuring conveyance of, a
 covered work, and grant a patent license to some of the parties
 receiving the covered work authorizing them to use, propagate, modify
 or convey a specific copy of the covered work, then the patent license
 you grant is automatically extended to all recipients of the covered
 work and works based on it.
 
   A patent license is "discriminatory" if it does not include within
 the scope of its coverage, prohibits the exercise of, or is
 conditioned on the non-exercise of one or more of the rights that are
 specifically granted under this License.  You may not convey a covered
 work if you are a party to an arrangement with a third party that is
 in the business of distributing software, under which you make payment
 to the third party based on the extent of your activity of conveying
 the work, and under which the third party grants, to any of the
 parties who would receive the covered work from you, a discriminatory
 patent license (a) in connection with copies of the covered work
 conveyed by you (or copies made from those copies), or (b) primarily
 for and in connection with specific products or compilations that
 contain the covered work, unless you entered into that arrangement,
 or that patent license was granted, prior to 28 March 2007.
 
   Nothing in this License shall be construed as excluding or limiting
 any implied license or other defenses to infringement that may
 otherwise be available to you under applicable patent law.
 
   12. No Surrender of Others' Freedom.
 
   If conditions are imposed on you (whether by court order, agreement or
 otherwise) that contradict the conditions of this License, they do not
 excuse you from the conditions of this License.  If you cannot convey a
 covered work so as to satisfy simultaneously your obligations under this
 License and any other pertinent obligations, then as a consequence you may
 not convey it at all.  For example, if you agree to terms that obligate you
 to collect a royalty for further conveying from those to whom you convey
 the Program, the only way you could satisfy both those terms and this
 License would be to refrain entirely from conveying the Program.
 
   13. Remote Network Interaction; Use with the GNU General Public License.
 
   Notwithstanding any other provision of this License, if you modify the
 Program, your modified version must prominently offer all users
 interacting with it remotely through a computer network (if your version
 supports such interaction) an opportunity to receive the Corresponding
 Source of your version by providing access to the Corresponding Source
 from a network server at no charge, through some standard or customary
 means of facilitating copying of software.  This Corresponding Source
 shall include the Corresponding Source for any work covered by version 3
 of the GNU General Public License that is incorporated pursuant to the
 following paragraph.
 
   Notwithstanding any other provision of this License, you have
 permission to link or combine any covered work with a work licensed
 under version 3 of the GNU General Public License into a single
 combined work, and to convey the resulting work.  The terms of this
 License will continue to apply to the part which is the covered work,
 but the work with which it is combined will remain governed by version
 3 of the GNU General Public License.
 
   14. Revised Versions of this License.
 
   The Free Software Foundation may publish revised and/or new versions of
 the GNU Affero General Public License from time to time.  Such new versions
 will be similar in spirit to the present version, but may differ in detail to
 address new problems or concerns.
 
   Each version is given a distinguishing version number.  If the
 Program specifies that a certain numbered version of the GNU Affero General
 Public License "or any later version" applies to it, you have the
 option of following the terms and conditions either of that numbered
 version or of any later version published by the Free Software
 Foundation.  If the Program does not specify a version number of the
 GNU Affero General Public License, you may choose any version ever published
 by the Free Software Foundation.
 
   If the Program specifies that a proxy can decide which future
 versions of the GNU Affero General Public License can be used, that proxy's
 public statement of acceptance of a version permanently authorizes you
 to choose that version for the Program.
 
   Later license versions may give you additional or different
 permissions.  However, no additional obligations are imposed on any
 author or copyright holder as a result of your choosing to follow a
 later version.
 
   15. Disclaimer of Warranty.
 
   THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
 APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
 HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
 OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
 THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
 IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
 ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
 
   16. Limitation of Liability.
 
   IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
 WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
 THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
 GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
 USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
 DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
 PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
 EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
 SUCH DAMAGES.
 
   17. Interpretation of Sections 15 and 16.
 
   If the disclaimer of warranty and limitation of liability provided
 above cannot be given local legal effect according to their terms,
 reviewing courts shall apply local law that most closely approximates
 an absolute waiver of all civil liability in connection with the
 Program, unless a warranty or assumption of liability accompanies a
 copy of the Program in return for a fee.
 
                      END OF TERMS AND CONDITIONS
 
             How to Apply These Terms to Your New Programs
 
   If you develop a new program, and you want it to be of the greatest
 possible use to the public, the best way to achieve this is to make it
 free software which everyone can redistribute and change under these terms.
 
   To do so, attach the following notices to the program.  It is safest
 to attach them to the start of each source file to most effectively
 state the exclusion of warranty; and each file should have at least
 the "copyright" line and a pointer to where the full notice is found.
 
     <one line to give the program's name and a brief idea of what it does.>
     Copyright (C) <year>  <name of author>
 
     This program is free software: you can redistribute it and/or modify
     it under the terms of the GNU Affero General Public License as published by
     the Free Software Foundation, either version 3 of the License, or
     (at your option) any later version.
 
     This program is distributed in the hope that it will be useful,
     but WITHOUT ANY WARRANTY; without even the implied warranty of
     MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
     GNU Affero General Public License for more details.
 
     You should have received a copy of the GNU Affero General Public License
     along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 Also add information on how to contact you by electronic and paper mail.
 
   If your software can interact with users remotely through a computer
 network, you should also make sure that it provides a way for users to
 get its source.  For example, if your program is a web application, its
 interface could display a "Source" link that leads users to an archive
 of the code.  There are many ways you could offer source, and different
 solutions will be better for different programs; see section 13 for the
 specific requirements.
 
   You should also get your employer (if you work as a programmer) or school,
 if any, to sign a "copyright disclaimer" for the program, if necessary.
 For more information on this, and how to apply and follow the GNU AGPL, see
 <http://www.gnu.org/licenses/>.

Makefile

@@ -1,53 +1,55 b''
 
-.PHONY: clean docs docs-clean docs-cleanup test test-clean test-only test-only-postgres test-only-mysql web-build
+.PHONY: clean docs docs-clean docs-cleanup test test-clean test-only test-only-postgres test-only-mysql web-build generate-pkgs
 
 NODE_PATH=./node_modules
 WEBPACK=./node_binaries/webpack
 GRUNT=./node_binaries/grunt
 
 
 clean:
 	make test-clean
-	find . -type f \( -iname '*.c' -o -iname '*.pyc' -o -iname '*.so' \) -exec rm '{}' ';'
+	find . -type f \( -iname '*.c' -o -iname '*.pyc' -o -iname '*.so' -o -iname '*.orig' \) -exec rm '{}' ';'
 
 test:
 	make test-clean
 	make test-only
 
 test-clean:
 	rm -rf coverage.xml htmlcov junit.xml pylint.log result
 	find . -type d -name "__pycache__" -prune -exec rm -rf '{}' ';'
 
 test-only:
 	PYTHONHASHSEED=random \
 	py.test -x -vv -r xw -p no:sugar --cov=rhodecode \
     --cov-report=term-missing --cov-report=html \
     rhodecode
 
 test-only-mysql:
 	PYTHONHASHSEED=random \
 	py.test -x -vv -r xw -p no:sugar --cov=rhodecode \
     --cov-report=term-missing --cov-report=html \
     --ini-config-override='{"app:main": {"sqlalchemy.db1.url": "mysql://root:qweqwe@localhost/rhodecode_test"}}' \
     rhodecode
 
 test-only-postgres:
 	PYTHONHASHSEED=random \
 	py.test -x -vv -r xw -p no:sugar --cov=rhodecode \
     --cov-report=term-missing --cov-report=html \
     --ini-config-override='{"app:main": {"sqlalchemy.db1.url": "postgresql://postgres:qweqwe@localhost/rhodecode_test"}}' \
     rhodecode
 
 
 docs:
 	(cd docs; nix-build default.nix -o result; make clean html)
 
 docs-clean:
 	(cd docs; make clean)
 
 docs-cleanup:
 	(cd docs; make cleanup)
 
 web-build:
 	NODE_PATH=$(NODE_PATH) $(GRUNT)
 
+generate-pkgs:
+	nix-shell pkgs/shell-generate.nix --command "pip2nix generate --licenses"

configs/development.ini

@@ -1,718 +1,732 b''
 
 
 ################################################################################
 ##                RHODECODE COMMUNITY  EDITION CONFIGURATION                  ##
 ################################################################################
 
 [DEFAULT]
 ## Debug flag sets all loggers to debug, and enables request tracking
 debug = true
 
 ################################################################################
 ##                            EMAIL CONFIGURATION                             ##
 ## Uncomment and replace with the email address which should receive          ##
 ## any error reports after an application crash                               ##
 ## Additionally these settings will be used by the RhodeCode mailing system   ##
 ################################################################################
 
 ## prefix all emails subjects with given prefix, helps filtering out emails
 #email_prefix = [RhodeCode]
 
 ## email FROM address all mails will be sent
 #app_email_from = rhodecode-noreply@localhost
 
 #smtp_server = mail.server.com
 #smtp_username =
 #smtp_password =
 #smtp_port =
 #smtp_use_tls = false
 #smtp_use_ssl = true
 
 [server:main]
 ## COMMON ##
 host = 127.0.0.1
 port = 5000
 
 ###########################################################
 ## WAITRESS WSGI SERVER - Recommended for Development  ####
 ###########################################################
 
 use = egg:waitress#main
 ## number of worker threads
 threads = 5
 ## MAX BODY SIZE 100GB
 max_request_body_size = 107374182400
 ## Use poll instead of select, fixes file descriptors limits problems.
 ## May not work on old windows systems.
 asyncore_use_poll = true
 
 
 ##########################
 ## GUNICORN WSGI SERVER ##
 ##########################
 ## run with gunicorn --log-config rhodecode.ini --paste rhodecode.ini
 
 #use = egg:gunicorn#main
-## Sets the number of process workers. More workers means more concurent connections
+## Sets the number of process workers. More workers means more concurrent connections
 ## RhodeCode can handle at the same time. Each additional worker also it increases
 ## memory usage as each has it's own set of caches.
 ## Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
 ## than 8-10 unless for really big deployments .e.g 700-1000 users.
 ## `instance_id = *` must be set in the [app:main] section below (which is the default)
 ## when using more than 1 worker.
 #workers = 2
 ## process name visible in process list
 #proc_name = rhodecode
 ## type of worker class, one of sync, gevent
 ## recommended for bigger setup is using of of other than sync one
 #worker_class = gevent
 ## The maximum number of simultaneous clients. Valid only for Gevent
 #worker_connections = 10
 ## max number of requests that worker will handle before being gracefully
 ## restarted, could prevent memory leaks
 #max_requests = 1000
 #max_requests_jitter = 30
 ## amount of time a worker can spend with handling a request before it
 ## gets killed and restarted. Set to 6hrs
 #timeout = 21600
 
 
 ## prefix middleware for RhodeCode.
 ## recommended when using proxy setup.
 ## allows to set RhodeCode under a prefix in server.
 ## eg https://server.com/custom_prefix. Enable `filter-with =` option below as well.
 ## And set your prefix like: `prefix = /custom_prefix`
 ## be sure to also set beaker.session.cookie_path = /custom_prefix if you need
 ## to make your cookies only work on prefix url
 [filter:proxy-prefix]
 use = egg:PasteDeploy#prefix
 prefix = /
 
 [app:main]
 ## The %(here)s variable will be replaced with the absolute path of parent directory
 ## of this file
 ## In addition ENVIRONMENT variables usage is possible, e.g
 ## sqlalchemy.db1.url = {ENV_RC_DB_URL}
 
 use = egg:rhodecode-enterprise-ce
 
 ## enable proxy prefix middleware, defined above
 #filter-with = proxy-prefix
 
 # During development the we want to have the debug toolbar enabled
 pyramid.includes =
     pyramid_debugtoolbar
     rhodecode.lib.middleware.request_wrapper
 
 pyramid.reload_templates = true
 
 debugtoolbar.hosts = 0.0.0.0/0
 debugtoolbar.exclude_prefixes =
     /css
     /fonts
     /images
     /js
 
 ## RHODECODE PLUGINS ##
 rhodecode.includes =
     rhodecode.api
 
 
 # api prefix url
 rhodecode.api.url = /_admin/api
 
 
 ## END RHODECODE PLUGINS ##
 
 ## encryption key used to encrypt social plugin tokens,
 ## remote_urls with credentials etc, if not set it defaults to
 ## `beaker.session.secret`
 #rhodecode.encrypted_values.secret =
 
 ## decryption strict mode (enabled by default). It controls if decryption raises
 ## `SignatureVerificationError` in case of wrong key, or damaged encryption data.
 #rhodecode.encrypted_values.strict = false
 
-## return gzipped responses from Rhodecode (static files/application)
+## return gzipped responses from RhodeCode (static files/application)
 gzip_responses = false
 
-## autogenerate javascript routes file on startup
+## auto-generate javascript routes file on startup
 generate_js_files = false
 
 ## System global default language.
 ## All available languages: en(default), be, de, es, fr, it, ja, pl, pt, ru, zh
 lang = en
 
 ## Perform a full repository scan and import on each server start.
 ## Settings this to true could lead to very long startup time.
 startup.import_repos = false
 
 ## Uncomment and set this path to use archive download cache.
 ## Once enabled, generated archives will be cached at this location
 ## and served from the cache during subsequent requests for the same archive of
 ## the repository.
 #archive_cache_dir = /tmp/tarballcache
 
-## URL at which the application is running. This is used for bootstraping
+## URL at which the application is running. This is used for Bootstrapping
 ## requests in context when no web request is available. Used in ishell, or
 ## SSH calls. Set this for events to receive proper url for SSH calls.
 app.base_url = http://rhodecode.local
 
 ## Unique application ID. Should be a random unique string for security.
 app_instance_uuid = rc-production
 
 ## Cut off limit for large diffs (size in bytes). If overall diff size on
 ## commit, or pull request exceeds this limit this diff will be displayed
 ## partially. E.g 512000 == 512Kb
 cut_off_limit_diff = 512000
 
 ## Cut off limit for large files inside diffs (size in bytes). Each individual
 ## file inside diff which exceeds this limit will be displayed partially.
 ##  E.g 128000 == 128Kb
 cut_off_limit_file = 128000
 
 ## use cached version of vcs repositories everywhere. Recommended to be `true`
 vcs_full_cache = true
 
 ## Force https in RhodeCode, fixes https redirects, assumes it's always https.
 ## Normally this is controlled by proper http flags sent from http server
 force_https = false
 
 ## use Strict-Transport-Security headers
 use_htsts = false
 
 ## git rev filter option, --all is the default filter, if you need to
 ## hide all refs in changelog switch this to --branches --tags
 git_rev_filter = --branches --tags
 
 # Set to true if your repos are exposed using the dumb protocol
 git_update_server_info = false
 
 ## RSS/ATOM feed options
 rss_cut_off_limit = 256000
 rss_items_per_page = 10
 rss_include_diff = false
 
 ## gist URL alias, used to create nicer urls for gist. This should be an
 ## url that does rewrites to _admin/gists/{gistid}.
 ## example: http://gist.rhodecode.org/{gistid}. Empty means use the internal
 ## RhodeCode url, ie. http[s]://rhodecode.server/_admin/gists/{gistid}
 gist_alias_url =
 
 ## List of views (using glob pattern syntax) that AUTH TOKENS could be
 ## used for access.
 ## Adding ?auth_token=TOKEN_HASH to the url authenticates this request as if it
 ## came from the the logged in user who own this authentication token.
-## Additionally @TOKEN syntaxt can be used to bound the view to specific
+## Additionally @TOKEN syntax can be used to bound the view to specific
 ## authentication token. Such view would be only accessible when used together
 ## with this authentication token
 ##
 ## list of all views can be found under `/_admin/permissions/auth_token_access`
 ## The list should be "," separated and on a single line.
 ##
 ## Most common views to enable:
 #    RepoCommitsView:repo_commit_download
 #    RepoCommitsView:repo_commit_patch
 #    RepoCommitsView:repo_commit_raw
 #    RepoCommitsView:repo_commit_raw@TOKEN
 #    RepoFilesView:repo_files_diff
 #    RepoFilesView:repo_archivefile
 #    RepoFilesView:repo_file_raw
 #    GistView:*
 api_access_controllers_whitelist =
 
 ## Default encoding used to convert from and to unicode
 ## can be also a comma separated list of encoding in case of mixed encodings
 default_encoding = UTF-8
 
 ## instance-id prefix
 ## a prefix key for this instance used for cache invalidation when running
-## multiple instances of rhodecode, make sure it's globally unique for
-## all running rhodecode instances. Leave empty if you don't use it
+## multiple instances of RhodeCode, make sure it's globally unique for
+## all running RhodeCode instances. Leave empty if you don't use it
 instance_id =
 
 ## Fallback authentication plugin. Set this to a plugin ID to force the usage
 ## of an authentication plugin also if it is disabled by it's settings.
 ## This could be useful if you are unable to log in to the system due to broken
-## authentication settings. Then you can enable e.g. the internal rhodecode auth
+## authentication settings. Then you can enable e.g. the internal RhodeCode auth
 ## module to log in again and fix the settings.
 ##
 ## Available builtin plugin IDs (hash is part of the ID):
 ## egg:rhodecode-enterprise-ce#rhodecode
 ## egg:rhodecode-enterprise-ce#pam
 ## egg:rhodecode-enterprise-ce#ldap
 ## egg:rhodecode-enterprise-ce#jasig_cas
 ## egg:rhodecode-enterprise-ce#headers
 ## egg:rhodecode-enterprise-ce#crowd
 #rhodecode.auth_plugin_fallback = egg:rhodecode-enterprise-ce#rhodecode
 
 ## alternative return HTTP header for failed authentication. Default HTTP
 ## response is 401 HTTPUnauthorized. Currently HG clients have troubles with
 ## handling that causing a series of failed authentication calls.
 ## Set this variable to 403 to return HTTPForbidden, or any other HTTP code
-## This will be served instead of default 401 on bad authnetication
+## This will be served instead of default 401 on bad authentication
 auth_ret_code =
 
 ## use special detection method when serving auth_ret_code, instead of serving
 ## ret_code directly, use 401 initially (Which triggers credentials prompt)
 ## and then serve auth_ret_code to clients
 auth_ret_code_detection = false
 
 ## locking return code. When repository is locked return this HTTP code. 2XX
 ## codes don't break the transactions while 4XX codes do
 lock_ret_code = 423
 
 ## allows to change the repository location in settings page
 allow_repo_location_change = true
 
 ## allows to setup custom hooks in settings page
 allow_custom_hooks_settings = true
 
 ## Generated license token required for EE edition license.
 ## New generated token value can be found in Admin > settings > license page.
 license_token =
 
 ## supervisor connection uri, for managing supervisor and logs.
 supervisor.uri =
 ## supervisord group name/id we only want this RC instance to handle
 supervisor.group_id = dev
 
 ## Display extended labs settings
 labs_settings_active = true
 
 ## Custom exception store path, defaults to TMPDIR
 ## This is used to store exception from RhodeCode in shared directory
 #exception_tracker.store_path =
 
+## File store configuration. This is used to store and serve uploaded files
+file_store.enabled = true
+## Storage backend, available options are: local
+file_store.backend = local
+## path to store the uploaded binaries
+file_store.storage_path = %(here)s/data/file_store
+
 
 ####################################
 ###        CELERY CONFIG        ####
 ####################################
 ## run: /path/to/celery worker \
 ##     -E --beat --app rhodecode.lib.celerylib.loader \
 ##     --scheduler rhodecode.lib.celerylib.scheduler.RcScheduler \
 ##     --loglevel DEBUG --ini /path/to/rhodecode.ini
 
 use_celery = false
 
 ## connection url to the message broker (default rabbitmq)
 celery.broker_url = amqp://rabbitmq:qweqwe@localhost:5672/rabbitmqhost
 
 ## maximum tasks to execute before worker restart
 celery.max_tasks_per_child = 100
 
 ## tasks will never be sent to the queue, but executed locally instead.
 celery.task_always_eager = false
 
 #####################################
 ###         DOGPILE CACHE        ####
 #####################################
 ## Default cache dir for caches.  Putting this into a ramdisk
 ## can boost performance, eg. /tmpfs/data_ramdisk, however this directory might require
 ## large amount of space
 cache_dir = %(here)s/data
 
 ## `cache_perms` cache settings for permission tree, auth TTL.
 rc_cache.cache_perms.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_perms.expiration_time = 300
 
 ## alternative `cache_perms` redis backend with distributed lock
 #rc_cache.cache_perms.backend = dogpile.cache.rc.redis
 #rc_cache.cache_perms.expiration_time = 300
 ## redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_perms.arguments.redis_expiration_time = 7200
 #rc_cache.cache_perms.arguments.socket_timeout = 30
 #rc_cache.cache_perms.arguments.host = localhost
 #rc_cache.cache_perms.arguments.port = 6379
 #rc_cache.cache_perms.arguments.db = 0
+## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_perms.arguments.distributed_lock = true
 
 ## `cache_repo` cache settings for FileTree, Readme, RSS FEEDS
 rc_cache.cache_repo.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_repo.expiration_time = 2592000
 
 ## alternative `cache_repo` redis backend with distributed lock
 #rc_cache.cache_repo.backend = dogpile.cache.rc.redis
 #rc_cache.cache_repo.expiration_time = 2592000
 ## redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_repo.arguments.redis_expiration_time = 2678400
 #rc_cache.cache_repo.arguments.socket_timeout = 30
 #rc_cache.cache_repo.arguments.host = localhost
 #rc_cache.cache_repo.arguments.port = 6379
 #rc_cache.cache_repo.arguments.db = 1
+## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_repo.arguments.distributed_lock = true
 
 ## cache settings for SQL queries, this needs to use memory type backend
 rc_cache.sql_cache_short.backend = dogpile.cache.rc.memory_lru
 rc_cache.sql_cache_short.expiration_time = 30
 
 ## `cache_repo_longterm` cache for repo object instances, this needs to use memory
 ## type backend as the objects kept are not pickle serializable
 rc_cache.cache_repo_longterm.backend = dogpile.cache.rc.memory_lru
 ## by default we use 96H, this is using invalidation on push anyway
 rc_cache.cache_repo_longterm.expiration_time = 345600
 ## max items in LRU cache, reduce this number to save memory, and expire last used
 ## cached objects
 rc_cache.cache_repo_longterm.max_size = 10000
 
 
 ####################################
 ###       BEAKER SESSION        ####
 ####################################
 
 ## .session.type is type of storage options for the session, current allowed
 ## types are file, ext:memcached, ext:redis, ext:database, and memory (default).
 beaker.session.type = file
 beaker.session.data_dir = %(here)s/data/sessions
 
 ## db based session, fast, and allows easy management over logged in users
 #beaker.session.type = ext:database
 #beaker.session.table_name = db_session
 #beaker.session.sa.url = postgresql://postgres:secret@localhost/rhodecode
 #beaker.session.sa.url = mysql://root:secret@127.0.0.1/rhodecode
 #beaker.session.sa.pool_recycle = 3600
 #beaker.session.sa.echo = false
 
 beaker.session.key = rhodecode
 beaker.session.secret = develop-rc-uytcxaz
 beaker.session.lock_dir = %(here)s/data/sessions/lock
 
 ## Secure encrypted cookie. Requires AES and AES python libraries
 ## you must disable beaker.session.secret to use this
 #beaker.session.encrypt_key = key_for_encryption
 #beaker.session.validate_key = validation_key
 
 ## sets session as invalid(also logging out user) if it haven not been
 ## accessed for given amount of time in seconds
 beaker.session.timeout = 2592000
 beaker.session.httponly = true
 ## Path to use for the cookie. Set to prefix if you use prefix middleware
 #beaker.session.cookie_path = /custom_prefix
 
 ## uncomment for https secure cookie
 beaker.session.secure = false
 
 ## auto save the session to not to use .save()
 beaker.session.auto = false
 
 ## default cookie expiration time in seconds, set to `true` to set expire
 ## at browser close
 #beaker.session.cookie_expires = 3600
 
 ###################################
 ## SEARCH INDEXING CONFIGURATION ##
 ###################################
 ## Full text search indexer is available in rhodecode-tools under
 ## `rhodecode-tools index` command
 
 ## WHOOSH Backend, doesn't require additional services to run
 ## it works good with few dozen repos
 search.module = rhodecode.lib.index.whoosh
 search.location = %(here)s/data/index
 
 ########################################
 ###    CHANNELSTREAM CONFIG         ####
 ########################################
 ## channelstream enables persistent connections and live notification
 ## in the system. It's also used by the chat system
 
 channelstream.enabled = false
 
 ## server address for channelstream server on the backend
 channelstream.server = 127.0.0.1:9800
 
 ## location of the channelstream server from outside world
 ## use ws:// for http or wss:// for https. This address needs to be handled
 ## by external HTTP server such as Nginx or Apache
-## see nginx/apache configuration examples in our docs
+## see Nginx/Apache configuration examples in our docs
 channelstream.ws_url = ws://rhodecode.yourserver.com/_channelstream
 channelstream.secret = secret
 channelstream.history.location = %(here)s/channelstream_history
 
 ## Internal application path that Javascript uses to connect into.
 ## If you use proxy-prefix the prefix should be added before /_channelstream
 channelstream.proxy_path = /_channelstream
 
 
 ###################################
 ##       APPENLIGHT CONFIG       ##
 ###################################
 
 ## Appenlight is tailored to work with RhodeCode, see
 ## http://appenlight.com for details how to obtain an account
 
-## appenlight integration enabled
+## Appenlight integration enabled
 appenlight = false
 
 appenlight.server_url = https://api.appenlight.com
 appenlight.api_key = YOUR_API_KEY
 #appenlight.transport_config = https://api.appenlight.com?threaded=1&timeout=5
 
-# used for JS client
+## used for JS client
 appenlight.api_public_key = YOUR_API_PUBLIC_KEY
 
 ## TWEAK AMOUNT OF INFO SENT HERE
 
 ## enables 404 error logging (default False)
 appenlight.report_404 = false
 
 ## time in seconds after request is considered being slow (default 1)
 appenlight.slow_request_time = 1
 
 ## record slow requests in application
 ## (needs to be enabled for slow datastore recording and time tracking)
 appenlight.slow_requests = true
 
 ## enable hooking to application loggers
 appenlight.logging = true
 
 ## minimum log level for log capture
 appenlight.logging.level = WARNING
 
 ## send logs only from erroneous/slow requests
 ## (saves API quota for intensive logging)
 appenlight.logging_on_error = false
 
-## list of additonal keywords that should be grabbed from environ object
+## list of additional keywords that should be grabbed from environ object
 ## can be string with comma separated list of words in lowercase
 ## (by default client will always send following info:
 ## 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
 ## start with HTTP* this list be extended with additional keywords here
 appenlight.environ_keys_whitelist =
 
 ## list of keywords that should be blanked from request object
 ## can be string with comma separated list of words in lowercase
 ## (by default client will always blank keys that contain following words
 ## 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
 ## this list be extended with additional keywords set here
 appenlight.request_keys_blacklist =
 
 ## list of namespaces that should be ignores when gathering log entries
 ## can be string with comma separated list of namespaces
 ## (by default the client ignores own entries: appenlight_client.client)
 appenlight.log_namespace_blacklist =
 
 # enable debug style page
 debug_style = true
 
 ###########################################
 ###   MAIN RHODECODE DATABASE CONFIG    ###
 ###########################################
 #sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
 #sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
 #sqlalchemy.db1.url = mysql://root:qweqwe@localhost/rhodecode?charset=utf8
 # pymysql is an alternative driver for MySQL, use in case of problems with default one
 #sqlalchemy.db1.url = mysql+pymysql://root:qweqwe@localhost/rhodecode
 
 sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
 
 # see sqlalchemy docs for other advanced settings
 
 ## print the sql statements to output
 sqlalchemy.db1.echo = false
 ## recycle the connections after this amount of seconds
 sqlalchemy.db1.pool_recycle = 3600
 sqlalchemy.db1.convert_unicode = true
 
 ## the number of connections to keep open inside the connection pool.
 ## 0 indicates no limit
 #sqlalchemy.db1.pool_size = 5
 
 ## the number of connections to allow in connection pool "overflow", that is
 ## connections that can be opened above and beyond the pool_size setting,
 ## which defaults to five.
 #sqlalchemy.db1.max_overflow = 10
 
 ## Connection check ping, used to detect broken database connections
 ## could be enabled to better handle cases if MySQL has gone away errors
 #sqlalchemy.db1.ping_connection = true
 
 ##################
 ### VCS CONFIG ###
 ##################
 vcs.server.enable = true
 vcs.server = localhost:9900
 
-## Web server connectivity protocol, responsible for web based VCS operatations
+## Web server connectivity protocol, responsible for web based VCS operations
 ## Available protocols are:
 ## `http` - use http-rpc backend (default)
 vcs.server.protocol = http
 
 ## Push/Pull operations protocol, available options are:
 ## `http` - use http-rpc backend (default)
 vcs.scm_app_implementation = http
 
 ## Push/Pull operations hooks protocol, available options are:
 ## `http` - use http-rpc backend (default)
 vcs.hooks.protocol = http
 
 ## Host on which this instance is listening for hooks. If vcsserver is in other location
 ## this should be adjusted.
 vcs.hooks.host = 127.0.0.1
 
 vcs.server.log_level = debug
 ## Start VCSServer with this instance as a subprocess, useful for development
 vcs.start_server = false
 
 ## List of enabled VCS backends, available options are:
 ## `hg`  - mercurial
 ## `git` - git
 ## `svn` - subversion
 vcs.backends = hg, git, svn
 
 vcs.connection_timeout = 3600
 ## Compatibility version when creating SVN repositories. Defaults to newest version when commented out.
 ## Available options are: pre-1.4-compatible, pre-1.5-compatible, pre-1.6-compatible, pre-1.8-compatible, pre-1.9-compatible
 #vcs.svn.compatible_version = pre-1.8-compatible
 
 
 ############################################################
 ### Subversion proxy support (mod_dav_svn)               ###
 ### Maps RhodeCode repo groups into SVN paths for Apache ###
 ############################################################
 ## Enable or disable the config file generation.
 svn.proxy.generate_config = false
 ## Generate config file with `SVNListParentPath` set to `On`.
 svn.proxy.list_parent_path = true
 ## Set location and file name of generated config file.
 svn.proxy.config_file_path = %(here)s/mod_dav_svn.conf
 ## alternative mod_dav config template. This needs to be a mako template
 #svn.proxy.config_template = ~/.rccontrol/enterprise-1/custom_svn_conf.mako
 ## Used as a prefix to the `Location` block in the generated config file.
 ## In most cases it should be set to `/`.
 svn.proxy.location_root = /
 ## Command to reload the mod dav svn configuration on change.
-## Example: `/etc/init.d/apache2 reload`
+## Example: `/etc/init.d/apache2 reload` or /home/USER/apache_reload.sh
+## Make sure user who runs RhodeCode process is allowed to reload Apache
 #svn.proxy.reload_cmd = /etc/init.d/apache2 reload
 ## If the timeout expires before the reload command finishes, the command will
 ## be killed. Setting it to zero means no timeout. Defaults to 10 seconds.
 #svn.proxy.reload_timeout = 10
 
 ############################################################
 ### SSH Support Settings                                 ###
 ############################################################
 
 ## Defines if a custom authorized_keys file should be created and written on
-## any change user ssh keys. Setting this to false also disables posibility
+## any change user ssh keys. Setting this to false also disables possibility
 ## of adding SSH keys by users from web interface. Super admins can still
 ## manage SSH Keys.
 ssh.generate_authorized_keyfile = false
 
 ## Options for ssh, default is `no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding`
 # ssh.authorized_keys_ssh_opts =
 
-## Path to the authrozied_keys file where the generate entries are placed.
+## Path to the authorized_keys file where the generate entries are placed.
 ## It is possible to have multiple key files specified in `sshd_config` e.g.
 ## AuthorizedKeysFile %h/.ssh/authorized_keys %h/.ssh/authorized_keys_rhodecode
 ssh.authorized_keys_file_path = ~/.ssh/authorized_keys_rhodecode
 
 ## Command to execute the SSH wrapper. The binary is available in the
-## rhodecode installation directory.
+## RhodeCode installation directory.
 ## e.g ~/.rccontrol/community-1/profile/bin/rc-ssh-wrapper
 ssh.wrapper_cmd = ~/.rccontrol/community-1/rc-ssh-wrapper
 
 ## Allow shell when executing the ssh-wrapper command
 ssh.wrapper_cmd_allow_shell = false
 
 ## Enables logging, and detailed output send back to the client during SSH
-## operations. Usefull for debugging, shouldn't be used in production.
+## operations. Useful for debugging, shouldn't be used in production.
 ssh.enable_debug_logging = true
 
 ## Paths to binary executable, by default they are the names, but we can
 ## override them if we want to use a custom one
 ssh.executable.hg = ~/.rccontrol/vcsserver-1/profile/bin/hg
 ssh.executable.git = ~/.rccontrol/vcsserver-1/profile/bin/git
 ssh.executable.svn = ~/.rccontrol/vcsserver-1/profile/bin/svnserve
 
+## Enables SSH key generator web interface. Disabling this still allows users
+## to add their own keys.
+ssh.enable_ui_key_generator = true
+
 
 ## Dummy marker to add new entries after.
 ## Add any custom entries below. Please don't remove.
 custom.conf = 1
 
 
 ################################
 ### LOGGING CONFIGURATION   ####
 ################################
 [loggers]
 keys = root, sqlalchemy, beaker, celery, rhodecode, ssh_wrapper
 
 [handlers]
 keys = console, console_sql
 
 [formatters]
 keys = generic, color_formatter, color_formatter_sql
 
 #############
 ## LOGGERS ##
 #############
 [logger_root]
 level = NOTSET
 handlers = console
 
 [logger_sqlalchemy]
 level = INFO
 handlers = console_sql
 qualname = sqlalchemy.engine
 propagate = 0
 
 [logger_beaker]
 level = DEBUG
 handlers =
 qualname = beaker.container
 propagate = 1
 
 [logger_rhodecode]
 level = DEBUG
 handlers =
 qualname = rhodecode
 propagate = 1
 
 [logger_ssh_wrapper]
 level = DEBUG
 handlers =
 qualname = ssh_wrapper
 propagate = 1
 
 [logger_celery]
 level = DEBUG
 handlers =
 qualname = celery
 
 
 ##############
 ## HANDLERS ##
 ##############
 
 [handler_console]
 class = StreamHandler
 args = (sys.stderr, )
 level = DEBUG
 formatter = color_formatter
 
 [handler_console_sql]
 # "level = DEBUG" logs SQL queries and results.
 # "level = INFO" logs SQL queries.
 # "level = WARN" logs neither.  (Recommended for production systems.)
 class = StreamHandler
 args = (sys.stderr, )
 level = WARN
 formatter = color_formatter_sql
 
 ################
 ## FORMATTERS ##
 ################
 
 [formatter_generic]
 class = rhodecode.lib.logging_formatter.ExceptionAwareFormatter
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter]
 class = rhodecode.lib.logging_formatter.ColorFormatter
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter_sql]
 class = rhodecode.lib.logging_formatter.ColorFormatterSql
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S

configs/production.ini

@@ -1,691 +1,705 b''
 
 
 ################################################################################
 ##                RHODECODE COMMUNITY  EDITION CONFIGURATION                  ##
 ################################################################################
 
 [DEFAULT]
 ## Debug flag sets all loggers to debug, and enables request tracking
 debug = false
 
 ################################################################################
 ##                            EMAIL CONFIGURATION                             ##
 ## Uncomment and replace with the email address which should receive          ##
 ## any error reports after an application crash                               ##
 ## Additionally these settings will be used by the RhodeCode mailing system   ##
 ################################################################################
 
 ## prefix all emails subjects with given prefix, helps filtering out emails
 #email_prefix = [RhodeCode]
 
 ## email FROM address all mails will be sent
 #app_email_from = rhodecode-noreply@localhost
 
 #smtp_server = mail.server.com
 #smtp_username =
 #smtp_password =
 #smtp_port =
 #smtp_use_tls = false
 #smtp_use_ssl = true
 
 [server:main]
 ## COMMON ##
 host = 127.0.0.1
 port = 5000
 
 ###########################################################
 ## WAITRESS WSGI SERVER - Recommended for Development  ####
 ###########################################################
 
 #use = egg:waitress#main
 ## number of worker threads
 #threads = 5
 ## MAX BODY SIZE 100GB
 #max_request_body_size = 107374182400
 ## Use poll instead of select, fixes file descriptors limits problems.
 ## May not work on old windows systems.
 #asyncore_use_poll = true
 
 
 ##########################
 ## GUNICORN WSGI SERVER ##
 ##########################
 ## run with gunicorn --log-config rhodecode.ini --paste rhodecode.ini
 
 use = egg:gunicorn#main
-## Sets the number of process workers. More workers means more concurent connections
+## Sets the number of process workers. More workers means more concurrent connections
 ## RhodeCode can handle at the same time. Each additional worker also it increases
 ## memory usage as each has it's own set of caches.
 ## Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
 ## than 8-10 unless for really big deployments .e.g 700-1000 users.
 ## `instance_id = *` must be set in the [app:main] section below (which is the default)
 ## when using more than 1 worker.
 workers = 2
 ## process name visible in process list
 proc_name = rhodecode
 ## type of worker class, one of sync, gevent
 ## recommended for bigger setup is using of of other than sync one
 worker_class = gevent
 ## The maximum number of simultaneous clients. Valid only for Gevent
 worker_connections = 10
 ## max number of requests that worker will handle before being gracefully
 ## restarted, could prevent memory leaks
 max_requests = 1000
 max_requests_jitter = 30
 ## amount of time a worker can spend with handling a request before it
 ## gets killed and restarted. Set to 6hrs
 timeout = 21600
 
 
 ## prefix middleware for RhodeCode.
 ## recommended when using proxy setup.
 ## allows to set RhodeCode under a prefix in server.
 ## eg https://server.com/custom_prefix. Enable `filter-with =` option below as well.
 ## And set your prefix like: `prefix = /custom_prefix`
 ## be sure to also set beaker.session.cookie_path = /custom_prefix if you need
 ## to make your cookies only work on prefix url
 [filter:proxy-prefix]
 use = egg:PasteDeploy#prefix
 prefix = /
 
 [app:main]
 ## The %(here)s variable will be replaced with the absolute path of parent directory
 ## of this file
 ## In addition ENVIRONMENT variables usage is possible, e.g
 ## sqlalchemy.db1.url = {ENV_RC_DB_URL}
 
 use = egg:rhodecode-enterprise-ce
 
 ## enable proxy prefix middleware, defined above
 #filter-with = proxy-prefix
 
 ## encryption key used to encrypt social plugin tokens,
 ## remote_urls with credentials etc, if not set it defaults to
 ## `beaker.session.secret`
 #rhodecode.encrypted_values.secret =
 
 ## decryption strict mode (enabled by default). It controls if decryption raises
 ## `SignatureVerificationError` in case of wrong key, or damaged encryption data.
 #rhodecode.encrypted_values.strict = false
 
-## return gzipped responses from Rhodecode (static files/application)
+## return gzipped responses from RhodeCode (static files/application)
 gzip_responses = false
 
-## autogenerate javascript routes file on startup
+## auto-generate javascript routes file on startup
 generate_js_files = false
 
 ## System global default language.
 ## All available languages: en(default), be, de, es, fr, it, ja, pl, pt, ru, zh
 lang = en
 
 ## Perform a full repository scan and import on each server start.
 ## Settings this to true could lead to very long startup time.
 startup.import_repos = false
 
 ## Uncomment and set this path to use archive download cache.
 ## Once enabled, generated archives will be cached at this location
 ## and served from the cache during subsequent requests for the same archive of
 ## the repository.
 #archive_cache_dir = /tmp/tarballcache
 
-## URL at which the application is running. This is used for bootstraping
+## URL at which the application is running. This is used for Bootstrapping
 ## requests in context when no web request is available. Used in ishell, or
 ## SSH calls. Set this for events to receive proper url for SSH calls.
 app.base_url = http://rhodecode.local
 
 ## Unique application ID. Should be a random unique string for security.
 app_instance_uuid = rc-production
 
 ## Cut off limit for large diffs (size in bytes). If overall diff size on
 ## commit, or pull request exceeds this limit this diff will be displayed
 ## partially. E.g 512000 == 512Kb
 cut_off_limit_diff = 512000
 
 ## Cut off limit for large files inside diffs (size in bytes). Each individual
 ## file inside diff which exceeds this limit will be displayed partially.
 ##  E.g 128000 == 128Kb
 cut_off_limit_file = 128000
 
 ## use cached version of vcs repositories everywhere. Recommended to be `true`
 vcs_full_cache = true
 
 ## Force https in RhodeCode, fixes https redirects, assumes it's always https.
 ## Normally this is controlled by proper http flags sent from http server
 force_https = false
 
 ## use Strict-Transport-Security headers
 use_htsts = false
 
 ## git rev filter option, --all is the default filter, if you need to
 ## hide all refs in changelog switch this to --branches --tags
 git_rev_filter = --branches --tags
 
 # Set to true if your repos are exposed using the dumb protocol
 git_update_server_info = false
 
 ## RSS/ATOM feed options
 rss_cut_off_limit = 256000
 rss_items_per_page = 10
 rss_include_diff = false
 
 ## gist URL alias, used to create nicer urls for gist. This should be an
 ## url that does rewrites to _admin/gists/{gistid}.
 ## example: http://gist.rhodecode.org/{gistid}. Empty means use the internal
 ## RhodeCode url, ie. http[s]://rhodecode.server/_admin/gists/{gistid}
 gist_alias_url =
 
 ## List of views (using glob pattern syntax) that AUTH TOKENS could be
 ## used for access.
 ## Adding ?auth_token=TOKEN_HASH to the url authenticates this request as if it
 ## came from the the logged in user who own this authentication token.
-## Additionally @TOKEN syntaxt can be used to bound the view to specific
+## Additionally @TOKEN syntax can be used to bound the view to specific
 ## authentication token. Such view would be only accessible when used together
 ## with this authentication token
 ##
 ## list of all views can be found under `/_admin/permissions/auth_token_access`
 ## The list should be "," separated and on a single line.
 ##
 ## Most common views to enable:
 #    RepoCommitsView:repo_commit_download
 #    RepoCommitsView:repo_commit_patch
 #    RepoCommitsView:repo_commit_raw
 #    RepoCommitsView:repo_commit_raw@TOKEN
 #    RepoFilesView:repo_files_diff
 #    RepoFilesView:repo_archivefile
 #    RepoFilesView:repo_file_raw
 #    GistView:*
 api_access_controllers_whitelist =
 
 ## Default encoding used to convert from and to unicode
 ## can be also a comma separated list of encoding in case of mixed encodings
 default_encoding = UTF-8
 
 ## instance-id prefix
 ## a prefix key for this instance used for cache invalidation when running
-## multiple instances of rhodecode, make sure it's globally unique for
-## all running rhodecode instances. Leave empty if you don't use it
+## multiple instances of RhodeCode, make sure it's globally unique for
+## all running RhodeCode instances. Leave empty if you don't use it
 instance_id =
 
 ## Fallback authentication plugin. Set this to a plugin ID to force the usage
 ## of an authentication plugin also if it is disabled by it's settings.
 ## This could be useful if you are unable to log in to the system due to broken
-## authentication settings. Then you can enable e.g. the internal rhodecode auth
+## authentication settings. Then you can enable e.g. the internal RhodeCode auth
 ## module to log in again and fix the settings.
 ##
 ## Available builtin plugin IDs (hash is part of the ID):
 ## egg:rhodecode-enterprise-ce#rhodecode
 ## egg:rhodecode-enterprise-ce#pam
 ## egg:rhodecode-enterprise-ce#ldap
 ## egg:rhodecode-enterprise-ce#jasig_cas
 ## egg:rhodecode-enterprise-ce#headers
 ## egg:rhodecode-enterprise-ce#crowd
 #rhodecode.auth_plugin_fallback = egg:rhodecode-enterprise-ce#rhodecode
 
 ## alternative return HTTP header for failed authentication. Default HTTP
 ## response is 401 HTTPUnauthorized. Currently HG clients have troubles with
 ## handling that causing a series of failed authentication calls.
 ## Set this variable to 403 to return HTTPForbidden, or any other HTTP code
-## This will be served instead of default 401 on bad authnetication
+## This will be served instead of default 401 on bad authentication
 auth_ret_code =
 
 ## use special detection method when serving auth_ret_code, instead of serving
 ## ret_code directly, use 401 initially (Which triggers credentials prompt)
 ## and then serve auth_ret_code to clients
 auth_ret_code_detection = false
 
 ## locking return code. When repository is locked return this HTTP code. 2XX
 ## codes don't break the transactions while 4XX codes do
 lock_ret_code = 423
 
 ## allows to change the repository location in settings page
 allow_repo_location_change = true
 
 ## allows to setup custom hooks in settings page
 allow_custom_hooks_settings = true
 
 ## Generated license token required for EE edition license.
 ## New generated token value can be found in Admin > settings > license page.
 license_token =
 
 ## supervisor connection uri, for managing supervisor and logs.
 supervisor.uri =
 ## supervisord group name/id we only want this RC instance to handle
 supervisor.group_id = prod
 
 ## Display extended labs settings
 labs_settings_active = true
 
 ## Custom exception store path, defaults to TMPDIR
 ## This is used to store exception from RhodeCode in shared directory
 #exception_tracker.store_path =
 
+## File store configuration. This is used to store and serve uploaded files
+file_store.enabled = true
+## Storage backend, available options are: local
+file_store.backend = local
+## path to store the uploaded binaries
+file_store.storage_path = %(here)s/data/file_store
+
 
 ####################################
 ###        CELERY CONFIG        ####
 ####################################
 ## run: /path/to/celery worker \
 ##     -E --beat --app rhodecode.lib.celerylib.loader \
 ##     --scheduler rhodecode.lib.celerylib.scheduler.RcScheduler \
 ##     --loglevel DEBUG --ini /path/to/rhodecode.ini
 
 use_celery = false
 
 ## connection url to the message broker (default rabbitmq)
 celery.broker_url = amqp://rabbitmq:qweqwe@localhost:5672/rabbitmqhost
 
 ## maximum tasks to execute before worker restart
 celery.max_tasks_per_child = 100
 
 ## tasks will never be sent to the queue, but executed locally instead.
 celery.task_always_eager = false
 
 #####################################
 ###         DOGPILE CACHE        ####
 #####################################
 ## Default cache dir for caches.  Putting this into a ramdisk
 ## can boost performance, eg. /tmpfs/data_ramdisk, however this directory might require
 ## large amount of space
 cache_dir = %(here)s/data
 
 ## `cache_perms` cache settings for permission tree, auth TTL.
 rc_cache.cache_perms.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_perms.expiration_time = 300
 
 ## alternative `cache_perms` redis backend with distributed lock
 #rc_cache.cache_perms.backend = dogpile.cache.rc.redis
 #rc_cache.cache_perms.expiration_time = 300
 ## redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_perms.arguments.redis_expiration_time = 7200
 #rc_cache.cache_perms.arguments.socket_timeout = 30
 #rc_cache.cache_perms.arguments.host = localhost
 #rc_cache.cache_perms.arguments.port = 6379
 #rc_cache.cache_perms.arguments.db = 0
+## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_perms.arguments.distributed_lock = true
 
 ## `cache_repo` cache settings for FileTree, Readme, RSS FEEDS
 rc_cache.cache_repo.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_repo.expiration_time = 2592000
 
 ## alternative `cache_repo` redis backend with distributed lock
 #rc_cache.cache_repo.backend = dogpile.cache.rc.redis
 #rc_cache.cache_repo.expiration_time = 2592000
 ## redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_repo.arguments.redis_expiration_time = 2678400
 #rc_cache.cache_repo.arguments.socket_timeout = 30
 #rc_cache.cache_repo.arguments.host = localhost
 #rc_cache.cache_repo.arguments.port = 6379
 #rc_cache.cache_repo.arguments.db = 1
+## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_repo.arguments.distributed_lock = true
 
 ## cache settings for SQL queries, this needs to use memory type backend
 rc_cache.sql_cache_short.backend = dogpile.cache.rc.memory_lru
 rc_cache.sql_cache_short.expiration_time = 30
 
 ## `cache_repo_longterm` cache for repo object instances, this needs to use memory
 ## type backend as the objects kept are not pickle serializable
 rc_cache.cache_repo_longterm.backend = dogpile.cache.rc.memory_lru
 ## by default we use 96H, this is using invalidation on push anyway
 rc_cache.cache_repo_longterm.expiration_time = 345600
 ## max items in LRU cache, reduce this number to save memory, and expire last used
 ## cached objects
 rc_cache.cache_repo_longterm.max_size = 10000
 
 
 ####################################
 ###       BEAKER SESSION        ####
 ####################################
 
 ## .session.type is type of storage options for the session, current allowed
 ## types are file, ext:memcached, ext:redis, ext:database, and memory (default).
 beaker.session.type = file
 beaker.session.data_dir = %(here)s/data/sessions
 
 ## db based session, fast, and allows easy management over logged in users
 #beaker.session.type = ext:database
 #beaker.session.table_name = db_session
 #beaker.session.sa.url = postgresql://postgres:secret@localhost/rhodecode
 #beaker.session.sa.url = mysql://root:secret@127.0.0.1/rhodecode
 #beaker.session.sa.pool_recycle = 3600
 #beaker.session.sa.echo = false
 
 beaker.session.key = rhodecode
 beaker.session.secret = production-rc-uytcxaz
 beaker.session.lock_dir = %(here)s/data/sessions/lock
 
 ## Secure encrypted cookie. Requires AES and AES python libraries
 ## you must disable beaker.session.secret to use this
 #beaker.session.encrypt_key = key_for_encryption
 #beaker.session.validate_key = validation_key
 
 ## sets session as invalid(also logging out user) if it haven not been
 ## accessed for given amount of time in seconds
 beaker.session.timeout = 2592000
 beaker.session.httponly = true
 ## Path to use for the cookie. Set to prefix if you use prefix middleware
 #beaker.session.cookie_path = /custom_prefix
 
 ## uncomment for https secure cookie
 beaker.session.secure = false
 
 ## auto save the session to not to use .save()
 beaker.session.auto = false
 
 ## default cookie expiration time in seconds, set to `true` to set expire
 ## at browser close
 #beaker.session.cookie_expires = 3600
 
 ###################################
 ## SEARCH INDEXING CONFIGURATION ##
 ###################################
 ## Full text search indexer is available in rhodecode-tools under
 ## `rhodecode-tools index` command
 
 ## WHOOSH Backend, doesn't require additional services to run
 ## it works good with few dozen repos
 search.module = rhodecode.lib.index.whoosh
 search.location = %(here)s/data/index
 
 ########################################
 ###    CHANNELSTREAM CONFIG         ####
 ########################################
 ## channelstream enables persistent connections and live notification
 ## in the system. It's also used by the chat system
 
 channelstream.enabled = false
 
 ## server address for channelstream server on the backend
 channelstream.server = 127.0.0.1:9800
 
 ## location of the channelstream server from outside world
 ## use ws:// for http or wss:// for https. This address needs to be handled
 ## by external HTTP server such as Nginx or Apache
-## see nginx/apache configuration examples in our docs
+## see Nginx/Apache configuration examples in our docs
 channelstream.ws_url = ws://rhodecode.yourserver.com/_channelstream
 channelstream.secret = secret
 channelstream.history.location = %(here)s/channelstream_history
 
 ## Internal application path that Javascript uses to connect into.
 ## If you use proxy-prefix the prefix should be added before /_channelstream
 channelstream.proxy_path = /_channelstream
 
 
 ###################################
 ##       APPENLIGHT CONFIG       ##
 ###################################
 
 ## Appenlight is tailored to work with RhodeCode, see
 ## http://appenlight.com for details how to obtain an account
 
-## appenlight integration enabled
+## Appenlight integration enabled
 appenlight = false
 
 appenlight.server_url = https://api.appenlight.com
 appenlight.api_key = YOUR_API_KEY
 #appenlight.transport_config = https://api.appenlight.com?threaded=1&timeout=5
 
-# used for JS client
+## used for JS client
 appenlight.api_public_key = YOUR_API_PUBLIC_KEY
 
 ## TWEAK AMOUNT OF INFO SENT HERE
 
 ## enables 404 error logging (default False)
 appenlight.report_404 = false
 
 ## time in seconds after request is considered being slow (default 1)
 appenlight.slow_request_time = 1
 
 ## record slow requests in application
 ## (needs to be enabled for slow datastore recording and time tracking)
 appenlight.slow_requests = true
 
 ## enable hooking to application loggers
 appenlight.logging = true
 
 ## minimum log level for log capture
 appenlight.logging.level = WARNING
 
 ## send logs only from erroneous/slow requests
 ## (saves API quota for intensive logging)
 appenlight.logging_on_error = false
 
-## list of additonal keywords that should be grabbed from environ object
+## list of additional keywords that should be grabbed from environ object
 ## can be string with comma separated list of words in lowercase
 ## (by default client will always send following info:
 ## 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
 ## start with HTTP* this list be extended with additional keywords here
 appenlight.environ_keys_whitelist =
 
 ## list of keywords that should be blanked from request object
 ## can be string with comma separated list of words in lowercase
 ## (by default client will always blank keys that contain following words
 ## 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
 ## this list be extended with additional keywords set here
 appenlight.request_keys_blacklist =
 
 ## list of namespaces that should be ignores when gathering log entries
 ## can be string with comma separated list of namespaces
 ## (by default the client ignores own entries: appenlight_client.client)
 appenlight.log_namespace_blacklist =
 
 
 ###########################################
 ###   MAIN RHODECODE DATABASE CONFIG    ###
 ###########################################
 #sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
 #sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
 #sqlalchemy.db1.url = mysql://root:qweqwe@localhost/rhodecode?charset=utf8
 # pymysql is an alternative driver for MySQL, use in case of problems with default one
 #sqlalchemy.db1.url = mysql+pymysql://root:qweqwe@localhost/rhodecode
 
 sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
 
 # see sqlalchemy docs for other advanced settings
 
 ## print the sql statements to output
 sqlalchemy.db1.echo = false
 ## recycle the connections after this amount of seconds
 sqlalchemy.db1.pool_recycle = 3600
 sqlalchemy.db1.convert_unicode = true
 
 ## the number of connections to keep open inside the connection pool.
 ## 0 indicates no limit
 #sqlalchemy.db1.pool_size = 5
 
 ## the number of connections to allow in connection pool "overflow", that is
 ## connections that can be opened above and beyond the pool_size setting,
 ## which defaults to five.
 #sqlalchemy.db1.max_overflow = 10
 
 ## Connection check ping, used to detect broken database connections
 ## could be enabled to better handle cases if MySQL has gone away errors
 #sqlalchemy.db1.ping_connection = true
 
 ##################
 ### VCS CONFIG ###
 ##################
 vcs.server.enable = true
 vcs.server = localhost:9900
 
-## Web server connectivity protocol, responsible for web based VCS operatations
+## Web server connectivity protocol, responsible for web based VCS operations
 ## Available protocols are:
 ## `http` - use http-rpc backend (default)
 vcs.server.protocol = http
 
 ## Push/Pull operations protocol, available options are:
 ## `http` - use http-rpc backend (default)
 vcs.scm_app_implementation = http
 
 ## Push/Pull operations hooks protocol, available options are:
 ## `http` - use http-rpc backend (default)
 vcs.hooks.protocol = http
 
 ## Host on which this instance is listening for hooks. If vcsserver is in other location
 ## this should be adjusted.
 vcs.hooks.host = 127.0.0.1
 
 vcs.server.log_level = info
 ## Start VCSServer with this instance as a subprocess, useful for development
 vcs.start_server = false
 
 ## List of enabled VCS backends, available options are:
 ## `hg`  - mercurial
 ## `git` - git
 ## `svn` - subversion
 vcs.backends = hg, git, svn
 
 vcs.connection_timeout = 3600
 ## Compatibility version when creating SVN repositories. Defaults to newest version when commented out.
 ## Available options are: pre-1.4-compatible, pre-1.5-compatible, pre-1.6-compatible, pre-1.8-compatible, pre-1.9-compatible
 #vcs.svn.compatible_version = pre-1.8-compatible
 
 
 ############################################################
 ### Subversion proxy support (mod_dav_svn)               ###
 ### Maps RhodeCode repo groups into SVN paths for Apache ###
 ############################################################
 ## Enable or disable the config file generation.
 svn.proxy.generate_config = false
 ## Generate config file with `SVNListParentPath` set to `On`.
 svn.proxy.list_parent_path = true
 ## Set location and file name of generated config file.
 svn.proxy.config_file_path = %(here)s/mod_dav_svn.conf
 ## alternative mod_dav config template. This needs to be a mako template
 #svn.proxy.config_template = ~/.rccontrol/enterprise-1/custom_svn_conf.mako
 ## Used as a prefix to the `Location` block in the generated config file.
 ## In most cases it should be set to `/`.
 svn.proxy.location_root = /
 ## Command to reload the mod dav svn configuration on change.
-## Example: `/etc/init.d/apache2 reload`
+## Example: `/etc/init.d/apache2 reload` or /home/USER/apache_reload.sh
+## Make sure user who runs RhodeCode process is allowed to reload Apache
 #svn.proxy.reload_cmd = /etc/init.d/apache2 reload
 ## If the timeout expires before the reload command finishes, the command will
 ## be killed. Setting it to zero means no timeout. Defaults to 10 seconds.
 #svn.proxy.reload_timeout = 10
 
 ############################################################
 ### SSH Support Settings                                 ###
 ############################################################
 
 ## Defines if a custom authorized_keys file should be created and written on
-## any change user ssh keys. Setting this to false also disables posibility
+## any change user ssh keys. Setting this to false also disables possibility
 ## of adding SSH keys by users from web interface. Super admins can still
 ## manage SSH Keys.
 ssh.generate_authorized_keyfile = false
 
 ## Options for ssh, default is `no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding`
 # ssh.authorized_keys_ssh_opts =
 
-## Path to the authrozied_keys file where the generate entries are placed.
+## Path to the authorized_keys file where the generate entries are placed.
 ## It is possible to have multiple key files specified in `sshd_config` e.g.
 ## AuthorizedKeysFile %h/.ssh/authorized_keys %h/.ssh/authorized_keys_rhodecode
 ssh.authorized_keys_file_path = ~/.ssh/authorized_keys_rhodecode
 
 ## Command to execute the SSH wrapper. The binary is available in the
-## rhodecode installation directory.
+## RhodeCode installation directory.
 ## e.g ~/.rccontrol/community-1/profile/bin/rc-ssh-wrapper
 ssh.wrapper_cmd = ~/.rccontrol/community-1/rc-ssh-wrapper
 
 ## Allow shell when executing the ssh-wrapper command
 ssh.wrapper_cmd_allow_shell = false
 
 ## Enables logging, and detailed output send back to the client during SSH
-## operations. Usefull for debugging, shouldn't be used in production.
+## operations. Useful for debugging, shouldn't be used in production.
 ssh.enable_debug_logging = false
 
 ## Paths to binary executable, by default they are the names, but we can
 ## override them if we want to use a custom one
 ssh.executable.hg = ~/.rccontrol/vcsserver-1/profile/bin/hg
 ssh.executable.git = ~/.rccontrol/vcsserver-1/profile/bin/git
 ssh.executable.svn = ~/.rccontrol/vcsserver-1/profile/bin/svnserve
 
+## Enables SSH key generator web interface. Disabling this still allows users
+## to add their own keys.
+ssh.enable_ui_key_generator = true
+
 
 ## Dummy marker to add new entries after.
 ## Add any custom entries below. Please don't remove.
 custom.conf = 1
 
 
 ################################
 ### LOGGING CONFIGURATION   ####
 ################################
 [loggers]
 keys = root, sqlalchemy, beaker, celery, rhodecode, ssh_wrapper
 
 [handlers]
 keys = console, console_sql
 
 [formatters]
 keys = generic, color_formatter, color_formatter_sql
 
 #############
 ## LOGGERS ##
 #############
 [logger_root]
 level = NOTSET
 handlers = console
 
 [logger_sqlalchemy]
 level = INFO
 handlers = console_sql
 qualname = sqlalchemy.engine
 propagate = 0
 
 [logger_beaker]
 level = DEBUG
 handlers =
 qualname = beaker.container
 propagate = 1
 
 [logger_rhodecode]
 level = DEBUG
 handlers =
 qualname = rhodecode
 propagate = 1
 
 [logger_ssh_wrapper]
 level = DEBUG
 handlers =
 qualname = ssh_wrapper
 propagate = 1
 
 [logger_celery]
 level = DEBUG
 handlers =
 qualname = celery
 
 
 ##############
 ## HANDLERS ##
 ##############
 
 [handler_console]
 class = StreamHandler
 args = (sys.stderr, )
 level = INFO
 formatter = generic
 
 [handler_console_sql]
 # "level = DEBUG" logs SQL queries and results.
 # "level = INFO" logs SQL queries.
 # "level = WARN" logs neither.  (Recommended for production systems.)
 class = StreamHandler
 args = (sys.stderr, )
 level = WARN
 formatter = generic
 
 ################
 ## FORMATTERS ##
 ################
 
 [formatter_generic]
 class = rhodecode.lib.logging_formatter.ExceptionAwareFormatter
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter]
 class = rhodecode.lib.logging_formatter.ColorFormatter
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter_sql]
 class = rhodecode.lib.logging_formatter.ColorFormatterSql
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S

default.nix

@@ -1,294 +1,296 b''
 # Nix environment for the community edition
 #
 # This shall be as lean as possible, just producing the enterprise-ce
 # derivation. For advanced tweaks to pimp up the development environment we use
 # "shell.nix" so that it does not have to clutter this file.
 #
 # Configuration, set values in "~/.nixpkgs/config.nix".
 # example
 #  {
 #    # Thoughts on how to configure the dev environment
 #    rc = {
 #      codeInternalUrl = "https://usr:[e-mail]/internal";
 #      sources = {
 #        rhodecode-vcsserver = "/home/user/work/rhodecode-vcsserver";
 #        rhodecode-enterprise-ce = "/home/user/work/rhodecode-enterprise-ce";
 #        rhodecode-enterprise-ee = "/home/user/work/rhodecode-enterprise-ee";
 #      };
 #    };
 #  }
 
 args@
-{ pythonPackages ? "python27Packages"
+{ system ? builtins.currentSystem
+, pythonPackages ? "python27Packages"
 , pythonExternalOverrides ? self: super: {}
 , doCheck ? false
 , ...
 }:
 
 let
-  pkgs_ = (import <nixpkgs> {});
+  pkgs_ = args.pkgs or (import <nixpkgs> { inherit system; });
 in
 
 let
   pkgs = import <nixpkgs> {
     overlays = [
       (import ./pkgs/overlays.nix)
     ];
     inherit
       (pkgs_)
       system;
   };
 
   # Works with the new python-packages, still can fallback to the old
   # variant.
   basePythonPackagesUnfix = basePythonPackages.__unfix__ or (
     self: basePythonPackages.override (a: { inherit self; }));
 
   # Evaluates to the last segment of a file system path.
   basename = path: with pkgs.lib; last (splitString "/" path);
 
   # source code filter used as arugment to builtins.filterSource.
   src-filter = path: type: with pkgs.lib;
     let
       ext = last (splitString "." path);
     in
       !builtins.elem (basename path) [
         ".git" ".hg" "__pycache__" ".eggs" ".idea" ".dev"
         "node_modules" "node_binaries"
         "build" "data" "result" "tmp"] &&
       !builtins.elem ext ["egg-info" "pyc"] &&
       # TODO: johbo: This check is wrong, since "path" contains an absolute path,
       # it would still be good to restore it since we want to ignore "result-*".
       !hasPrefix "result" path;
 
   sources =
     let
       inherit
         (pkgs.lib)
         all
         isString
         attrValues;
       sourcesConfig = pkgs.config.rc.sources or {};
     in
       # Ensure that sources are configured as strings. Using a path
       # would result in a copy into the nix store.
       assert all isString (attrValues sourcesConfig);
       sourcesConfig;
 
   version = builtins.readFile "${rhodecode-enterprise-ce-src}/rhodecode/VERSION";
   rhodecode-enterprise-ce-src = builtins.filterSource src-filter ./.;
 
   nodeEnv = import ./pkgs/node-default.nix {
     inherit
-      pkgs;
+      pkgs
+      system;
   };
   nodeDependencies = nodeEnv.shell.nodeDependencies;
 
   rhodecode-testdata-src = sources.rhodecode-testdata or (
     pkgs.fetchhg {
       url = "https://code.rhodecode.com/upstream/rc_testdata";
       rev = "v0.10.0";
       sha256 = "0zn9swwvx4vgw4qn8q3ri26vvzgrxn15x6xnjrysi1bwmz01qjl0";
   });
 
   rhodecode-testdata = import "${rhodecode-testdata-src}/default.nix" {
   inherit
     doCheck
     pkgs
     pythonPackages;
   };
 
   pythonLocalOverrides = self: super: {
     rhodecode-enterprise-ce =
       let
         linkNodePackages = ''
           export RHODECODE_CE_PATH=${rhodecode-enterprise-ce-src}
 
           echo "[BEGIN]: Link node packages and binaries"
           # johbo: Linking individual packages allows us to run "npm install"
           # inside of a shell to try things out. Re-entering the shell will
           # restore a clean environment.
           rm -fr node_modules
           mkdir node_modules
           ln -s ${nodeDependencies}/lib/node_modules/* node_modules/
           export NODE_PATH=./node_modules
 
           rm -fr node_binaries
           mkdir node_binaries
           ln -s ${nodeDependencies}/bin/* node_binaries/
           echo "[DONE ]: Link node packages and binaries"
         '';
 
         releaseName = "RhodeCodeEnterpriseCE-${version}";
       in super.rhodecode-enterprise-ce.override (attrs: {
       inherit
         doCheck
         version;
 
       name = "rhodecode-enterprise-ce-${version}";
       releaseName = releaseName;
       src = rhodecode-enterprise-ce-src;
       dontStrip = true; # prevent strip, we don't need it.
 
       # expose following attributed outside
       passthru = {
         inherit
           rhodecode-testdata
           linkNodePackages
           myPythonPackagesUnfix
           pythonLocalOverrides
           pythonCommunityOverrides;
 
         pythonPackages = self;
       };
 
       buildInputs =
         attrs.buildInputs or [] ++ [
           rhodecode-testdata
         ];
 
       #NOTE: option to inject additional propagatedBuildInputs
       propagatedBuildInputs =
         attrs.propagatedBuildInputs or [] ++ [
 
         ];
 
       LC_ALL = "en_US.UTF-8";
       LOCALE_ARCHIVE =
         if pkgs.stdenv.isLinux
         then "${pkgs.glibcLocales}/lib/locale/locale-archive"
         else "";
 
       # Add bin directory to path so that tests can find 'rhodecode'.
       preCheck = ''
         export PATH="$out/bin:$PATH"
       '';
 
       # custom check phase for testing
       checkPhase = ''
         runHook preCheck
         PYTHONHASHSEED=random py.test -vv -p no:sugar -r xw --cov-config=.coveragerc --cov=rhodecode --cov-report=term-missing rhodecode
         runHook postCheck
       '';
 
       postCheck = ''
         echo "Cleanup of rhodecode/tests"
         rm -rf $out/lib/${self.python.libPrefix}/site-packages/rhodecode/tests
       '';
 
       preBuild = ''
         echo "[BEGIN]: Building frontend assets"
         ${linkNodePackages}
         make web-build
         rm -fr node_modules
         rm -fr node_binaries
         echo "[DONE ]: Building frontend assets"
       '';
 
       postInstall = ''
         # check required files
         STATIC_CHECK="/robots.txt /502.html
                       /js/scripts.js /js/rhodecode-components.js
                       /css/style.css /css/style-polymer.css"
 
         for file in $STATIC_CHECK;
         do
             if [ ! -f rhodecode/public/$file ]; then
               echo "Missing $file"
               exit 1
             fi
         done
 
         echo "Writing enterprise-ce meta information for rccontrol to nix-support/rccontrol"
         mkdir -p $out/nix-support/rccontrol
         cp -v rhodecode/VERSION $out/nix-support/rccontrol/version
         echo "[DONE ]: enterprise-ce meta information for rccontrol written"
 
         mkdir -p $out/etc
         cp configs/production.ini $out/etc
         echo "[DONE ]: saved enterprise-ce production.ini into $out/etc"
 
         cp -Rf rhodecode/config/rcextensions $out/etc/rcextensions.tmpl
         echo "[DONE ]: saved enterprise-ce rcextensions into $out/etc/rcextensions.tmpl"
 
         # python based programs need to be wrapped
         mkdir -p $out/bin
 
         # required binaries from dependencies
         ln -s ${self.supervisor}/bin/supervisorctl $out/bin/
         ln -s ${self.supervisor}/bin/supervisord $out/bin/
         ln -s ${self.pastescript}/bin/paster $out/bin/
         ln -s ${self.channelstream}/bin/channelstream $out/bin/
         ln -s ${self.celery}/bin/celery $out/bin/
         ln -s ${self.gunicorn}/bin/gunicorn $out/bin/
         ln -s ${self.pyramid}/bin/prequest $out/bin/
         ln -s ${self.pyramid}/bin/pserve $out/bin/
 
         echo "[DONE ]: created symlinks into $out/bin"
         DEPS="$out/bin/supervisorctl \
               $out/bin/supervisord \
               $out/bin/paster \
               $out/bin/channelstream \
               $out/bin/celery \
               $out/bin/gunicorn \
               $out/bin/prequest \
               $out/bin/pserve"
 
         # wrap only dependency scripts, they require to have full PYTHONPATH set
         # to be able to import all packages
         for file in $DEPS;
         do
           wrapProgram $file \
             --prefix PATH : $PATH \
             --prefix PYTHONPATH : $PYTHONPATH \
             --set PYTHONHASHSEED random
         done
 
         echo "[DONE ]: enterprise-ce binary wrapping"
 
         # rhodecode-tools don't need wrapping
         ln -s ${self.rhodecode-tools}/bin/rhodecode-* $out/bin/
 
         # expose sources of CE
         ln -s $out $out/etc/rhodecode_enterprise_ce_source
 
         # expose static files folder
         cp -Rf $out/lib/${self.python.libPrefix}/site-packages/rhodecode/public/ $out/etc/static
         chmod 755 -R $out/etc/static
 
       '';
     });
 
   };
 
   basePythonPackages = with builtins;
     if isAttrs pythonPackages then
       pythonPackages
     else
       getAttr pythonPackages pkgs;
 
   pythonGeneratedPackages = import ./pkgs/python-packages.nix {
     inherit
       pkgs;
     inherit
       (pkgs)
       fetchurl
       fetchgit
       fetchhg;
   };
 
   pythonCommunityOverrides = import ./pkgs/python-packages-overrides.nix {
     inherit pkgs basePythonPackages;
   };
 
   # Apply all overrides and fix the final package set
   myPythonPackagesUnfix = with pkgs.lib;
     (extends pythonExternalOverrides
     (extends pythonLocalOverrides
     (extends pythonCommunityOverrides
     (extends pythonGeneratedPackages
              basePythonPackagesUnfix))));
 
   myPythonPackages = (pkgs.lib.fix myPythonPackagesUnfix);
 
 in myPythonPackages.rhodecode-enterprise-ce

docs/admin/backup-restore.rst

@@ -1,151 +1,151 b''
 .. _backup-ref:
 
 Backup and Restore
 ==================
 
 *“The condition of any backup is unknown until a restore is attempted.”*
 `Schrödinger's Backup`_
 
 To snapshot an instance of |RCE|, and save its settings, you need to backup the
 following parts of the system at the same time.
 
 * The |repos| managed by the instance together with the stored Gists.
 * The |RCE| database.
 * Any configuration files or extensions that you've configured. In most
   cases it's only the :file:`rhodecode.ini` file.
 * Installer files such as those in `/opt/rhodecode` can be backed-up, however
   it's not required since in case of a recovery installer simply
   re-creates those.
 
 
 .. important::
 
    Ideally you should script all of these functions so that it creates a
    backup snapshot of your system at a particular timestamp and then run that
    script regularly.
 
 Backup Details
 --------------
 
 To backup the relevant parts of |RCE| required to restore your system, use
 the information in this section to identify what is important to you.
 
 Repository Backup
 ^^^^^^^^^^^^^^^^^
 
 To back up your |repos|, use the API to get a list of all |repos| managed,
 and then clone them to your backup location. This is the most safe backup option.
 Backing up the storage directory could potentially result in a backup of
 partially committed files or commits. (Backup taking place during a big push)
 As an alternative you could use a rsync or simple `cp` commands if you can
 ensure your instance is only in read-only mode or stopped at the moment.
 
 
 Use the ``get_repos`` method to list all your managed |repos|,
 and use the ``clone_uri`` information that is returned. See the :ref:`api`
 for more information. Be sure to keep the structure or repositories with their
 repository groups.
 
 .. important::
 
    This will not work for |svn| |repos|. Currently the only way to back up
    your |svn| |repos| is to make a copy of them.
 
    It is also important to note, that you can only restore the |svn| |repos|
    using the same version as they were saved with.
 
 Database Backup
 ^^^^^^^^^^^^^^^
 
 The instance database contains all the |RCE| permissions settings,
 and user management information. To backup your database,
 export it using the following appropriate example, and then move it to your
 backup location:
 
 .. code-block:: bash
 
    # For MySQL DBs
    $ mysqldump -u <uname> -p <pass> rhodecode_db_name > mysql-db-backup
    # MySQL restore command
    $ mysql -u <uname> -p <pass> rhodecode_db_name < mysql-db-backup
 
    # For PostgreSQL DBs
    $ PGPASSWORD=<pass> pg_dump --inserts -U <uname> -h localhost rhodecode_db_name > postgresql-db-backup
    # PosgreSQL restore
    $ PGPASSWORD=<pass> psql -U <uname> -h localhost -d rhodecode_db_name -1 -f postgresql-db-backup
 
    # For SQLite
    $ sqlite3 rhodecode.db ‘.dump’ > sqlite-db-backup
    # SQLite restore
    $ copy sqlite-db-backup rhodecode.db
 
 
 The default |RCE| SQLite database location is
 :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.db`
 
 If running MySQL or PostgreSQL databases, you will have configured these
 separately, for more information see :ref:`rhodecode-database-ref`
 
 Configuration File Backup
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Depending on your setup, you could have a number of configuration files that
 should be backed up. You may have some, or all of the configuration files
 listed in the :ref:`config-rce-files` section. Ideally you should back these
 up at the same time as the database and |repos|. It really depends on if you need
 the configuration file like logs, custom modules. We always recommend backing
 those up.
 
 Gist Backup
 ^^^^^^^^^^^
 
 To backup the gists on your |RCE| instance you usually have to backup the
 gist storage path. If this haven't been changed it's located inside
 :file:`.rc_gist_store` and the metadata in :file:`.rc_gist_metadata`.
 You can use the ``get_users`` and ``get_gists`` API methods to fetch the
 gists for each user on the instance.
 
 Extension Backups
 ^^^^^^^^^^^^^^^^^
 
 You should also backup any extensions added in the
 :file:`home/{user}/.rccontrol/{instance-id}/rcextensions` directory.
 
 Full-text Search Backup
 ^^^^^^^^^^^^^^^^^^^^^^^
 
 You may also have full text search set up, but the index can be rebuild from
 re-imported |repos| if necessary. You will most likely want to backup your
-:file:`mapping.ini` file if you've configured that. For more information, see
+:file:`search_mapping.ini` file if you've configured that. For more information, see
 the :ref:`indexing-ref` section.
 
 Restoration Steps
 -----------------
 
 To restore an instance of |RCE| from its backed up components, to a fresh
 system use the following steps.
 
 1. Install a new instance of |RCE| using sqlite option as database.
 2. Restore your database.
 3. Once installed, replace you backed up the :file:`rhodecode.ini` with your
    backup version. Ensure this file points to the restored
    database, see the :ref:`config-database` section.
 4. Restart |RCE| and remap and rescan your |repos| to verify filesystem access,
    see the :ref:`remap-rescan` section.
 
 
 Post Restoration Steps
 ^^^^^^^^^^^^^^^^^^^^^^
 
 Once you have restored your |RCE| instance to basic functionality, you can
 then work on restoring any specific setup changes you had made.
 
-* To recreate the |RCE| index, use the backed up :file:`mapping.ini` file if
+* To recreate the |RCE| index, use the backed up :file:`search_mapping.ini` file if
   you had made changes and rerun the indexer. See the
   :ref:`indexing-ref` section for details.
 * To reconfigure any extensions, copy the backed up extensions into the
   :file:`/home/{user}/.rccontrol/{instance-id}/rcextensions` and also specify
   any custom hooks if necessary. See the :ref:`extensions-hooks-ref` section for
   details.
 
 .. _Schrödinger's Backup: http://novabackup.novastor.com/blog/schrodingers-backup-good-bad-backup/

docs/admin/config-files-overview.rst

@@ -1,74 +1,74 b''
 .. _config-files:
 
 Configuration Files Overview
 ============================
 
 |RCE| and |RCC| have a number of different configuration files. The following
 is a brief explanation of each, and links to their associated configuration
 sections.
 
 .. rst-class:: dl-horizontal
 
     \- **rhodecode.ini**
         Default location:
         :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
 
         This is the main |RCE| configuration file and controls much of its
         default behaviour. It is also used to configure certain customer
         settings. Here are some of the most common reasons to make changes to
         this file.
 
         * :ref:`config-database`
         * :ref:`set-up-mail`
         * :ref:`increase-gunicorn`
         * :ref:`x-frame`
 
-    \- **mapping.ini**
+    \- **search_mapping.ini**
         Default location:
-        :file:`/home/{user}/.rccontrol/{instance-id}/mapping.ini`
+        :file:`/home/{user}/.rccontrol/{instance-id}/search_mapping.ini`
 
         This file is used to control the |RCE| indexer. It comes configured
         to index your instance. To change the default configuration, see
         :ref:`advanced-indexing`.
 
     \- **vcsserver.ini**
         Default location:
         :file:`/home/{user}/.rccontrol/{vcsserver-id}/vcsserver.ini`
 
         The VCS Server handles the connection between your |repos| and |RCE|.
         See the :ref:`vcs-server` section for configuration options and more
         detailed information.
 
     \- **supervisord.ini**
         Default location:
         :file:`/home/{user}/.rccontrol/supervisor/supervisord.ini`
 
         |RCC| uses Supervisor to monitor and manage installed instances of
         |RCE| and the VCS Server. |RCC| will manage this file completely,
         unless you install |RCE| in self-managed mode. For more information,
         see the :ref:`Supervisor Setup<control:supervisor-setup>` section.
 
     \- **.rccontrol.ini**
         Default location: :file:`/home/{user}/.rccontrol.ini`
 
         This file contains the instances that |RCC| starts at boot, which is all
         by default, but for more information, see
         the :ref:`Manually Start At Boot <control:set-start-boot>` section.
 
     \- **.rhoderc**
         Default location: :file:`/home/{user}/.rhoderc`
 
         This file is used by the |RCE| API when accessing an instance from a
         remote machine. The API checks this file for connection and
         authentication details. For more details, see the :ref:`config-rhoderc`
         section.
 
     \- **MANIFEST**
         Default location: :file:`/home/{user}/.rccontrol/cache/MANIFEST`
 
         |RCC| uses this file to source the latest available builds from the
         secure RhodeCode download channels. The only reason to mess with this file
         is if you need to do an offline installation,
         see the :ref:`Offline Installation<control:offline-installer-ref>`
         instructions, otherwise |RCC| will completely manage this file.
 

docs/admin/indexing.rst

@@ -1,276 +1,359 b''
 .. _indexing-ref:
 
 Full-text Search
 ----------------
 
-By default RhodeCode is configured to use `Whoosh`_ to index |repos| and
-provide full-text search.
+RhodeCode provides a full text search capabilities to search inside file content,
+commit message, and file paths. Indexing is not enabled by default and to use
+full text search building an index is a pre-requisite.
 
-|RCE| also provides support for `Elasticsearch`_ as a backend for scalable
-search. See :ref:`enable-elasticsearch` for details.
+By default RhodeCode is configured to use `Whoosh`_ to index |repos| and
+provide full-text search. `Whoosh`_ works well for a small amount of data and
+shouldn't be used in case of large code-bases and lots of repositories.
+
+|RCE| also provides support for `ElasticSearch 6`_ as a backend more for advanced
+and scalable search. See :ref:`enable-elasticsearch` for details.
 
 Indexing
 ^^^^^^^^
 
-To run the indexer you need to use an |authtoken| with admin rights to all
-|repos|.
+To run the indexer you need to have an |authtoken| with admin rights to all |repos|.
 
-To index new content added, you have the option to set the indexer up in a
+To index repositories stored in RhodeCode, you have the option to set the indexer up in a
 number of ways, for example:
 
-* Call the indexer via a cron job. We recommend running this nightly,
-  unless you need everything indexed immediately.
-* Set the indexer to infinitely loop and reindex as soon as it has run its
-  cycle.
+* Call the indexer via a cron job. We recommend running this once at night.
+  In case you need everything indexed immediately it's possible to index few
+  times during the day. Indexer has a special locking mechanism that won't allow
+  two instances of indexer running at once. It's safe to run it even every 1hr.
+* Set the indexer to infinitely loop and reindex as soon as it has run its previous cycle.
 * Hook the indexer up with your CI server to reindex after each push.
 
-The indexer works by indexing new commits added since the last run. If you
-wish to build a brand new index from scratch each time,
-use the ``force`` option in the configuration file.
+The indexer works by indexing new commits added since the last run, and comparing
+file changes to index only new or modified files.
+If you wish to build a brand new index from scratch each time, use the ``force``
+option in the configuration file, or run it with --force flag.
 
 .. important::
 
    You need to have |RCT| installed, see :ref:`install-tools`. Since |RCE|
-   3.5.0 they are installed by default.
+   3.5.0 they are installed by default and available with community/enterprise installations.
 
 To set up indexing, use the following steps:
 
 1. :ref:`config-rhoderc`, if running tools remotely.
 2. :ref:`run-index`
 3. :ref:`set-index`
 4. :ref:`advanced-indexing`
 
 .. _config-rhoderc:
 
 Configure the ``.rhoderc`` File
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
+.. note::
+
+    Optionally it's possible to use indexer without the ``.rhoderc``. Simply instead of
+    executing with `--instance-name=enterprise-1` execute providing the host and token
+    directly: `--api-host=http://127.0.0.1:10000 --api-key=<auth-token-goes-here>`
+
+
 |RCT| uses the :file:`/home/{user}/.rhoderc` file for connection details
 to |RCE| instances. If this file is not automatically created,
 you can configure it using the following example. You need to configure the
 details for each instance you want to index.
 
 .. code-block:: bash
 
     # Check the instance details
     # of the instance you want to index
     $ rccontrol status
 
-     - NAME: enterprise-1
-     - STATUS: RUNNING
-     - TYPE: Momentum
-     - VERSION: 1.5.0
-     - URL: http://127.0.0.1:10000
+    - NAME: enterprise-1
+    - STATUS: RUNNING
+    - TYPE: Enterprise
+    - VERSION: 4.1.0
+    - URL: http://127.0.0.1:10003
 
 To get your API Token, on the |RCE| interface go to
 :menuselection:`username --> My Account --> Auth tokens`
 
 .. code-block:: ini
 
     # Configure .rhoderc with matching details
     # This allows the indexer to connect to the instance
     [instance:enterprise-1]
     api_host = http://127.0.0.1:10000
     api_key = <auth token goes here>
-    repo_dir = /home/<username>/repos
+
 
 .. _run-index:
 
 Run the Indexer
 ^^^^^^^^^^^^^^^
 
-Run the indexer using the following command, and specify the instance you
-want to index:
+Run the indexer using the following command, and specify the instance you want to index:
 
 .. code-block:: bash
 
-   # From inside a virtualevv
-   (venv)$ rhodecode-index --instance-name=enterprise-1
-
-   # Using default installation
+   # Using default simples indexing of all repositories
    $ /home/user/.rccontrol/enterprise-1/profile/bin/rhodecode-index \
        --instance-name=enterprise-1
 
-   # Using a custom mapping file
+   # Using a custom mapping file with indexing rules, and using elasticsearch 6 backend
    $ /home/user/.rccontrol/enterprise-1/profile/bin/rhodecode-index \
        --instance-name=enterprise-1 \
-       --mapping=/home/user/.rccontrol/enterprise-1/mapping.ini
+       --mapping=/home/user/.rccontrol/enterprise-1/search_mapping.ini \
+       --es-version=6 --engine-location=http://elasticsearch-host:9200
+
+   # Using a custom mapping file and invocation without ``.rhoderc``
+   $ /home/user/.rccontrol/enterprise-1/profile/bin/rhodecode-index \
+       --api-host=http://rhodecodecode.myserver.com --api-key=xxxxx \
+       --mapping=/home/user/.rccontrol/enterprise-1/search_mapping.ini
+
+   # From inside a virtualev on your local machine or CI server.
+   (venv)$ rhodecode-index --instance-name=enterprise-1
+
 
 .. note::
 
    In case of often indexing the index may become fragmented. Most often a result of that
    is error about `too many open files`. To fix this indexer needs to be executed with
    --optimize flag. E.g `rhodecode-index --instance-name=enterprise-1 --optimize`
    This should be executed regularly, once a week is recommended.
 
 
 .. _set-index:
 
 Schedule the Indexer
 ^^^^^^^^^^^^^^^^^^^^
 
 To schedule the indexer, configure the crontab file to run the indexer inside
 your |RCT| virtualenv using the following steps.
 
 1. Open the crontab file, using ``crontab -e``.
 2. Add the indexer to the crontab, and schedule it to run as regularly as you
    wish.
 3. Save the file.
 
 .. code-block:: bash
 
     $ crontab -e
 
     # The virtualenv can be called using its full path, so for example you can
     # put this example into the crontab
 
     # Run the indexer daily at 4am using the default mapping settings
     * 4 * * * /home/ubuntu/.virtualenv/rhodecode-venv/bin/rhodecode-index \
     --instance-name=enterprise-1
 
     # Run the indexer every Sunday at 3am using default mapping
     * 3 * * 0 /home/ubuntu/.virtualenv/rhodecode-venv/bin/rhodecode-index \
     --instance-name=enterprise-1
 
     # Run the indexer every 15 minutes
     # using a specially configured mapping file
     */15 * * * * ~/.rccontrol/enterprise-4/profile/bin/rhodecode-index \
        --instance-name=enterprise-4 \
-       --mapping=/home/user/.rccontrol/enterprise-4/mapping.ini
+       --mapping=/home/user/.rccontrol/enterprise-4/search_mapping.ini
 
 .. _advanced-indexing:
 
 Advanced Indexing
 ^^^^^^^^^^^^^^^^^
 
-|RCT| indexes based on the :file:`mapping.ini` file. To configure your index,
-you can specify different options in this file. The default location is:
+
+Force Re-Indexing single repository
++++++++++++++++++++++++++++++++++++
+
+Often it's required to re-index whole repository because of some repository changes,
+or to remove some indexed secrets, or files. There's a special `--repo-name=` flag
+for the indexer that limits execution to a single repository. For example to force-reindex
+single repository such call can be made::
+
+    rhodecode-index --instance-name=enterprise-1 --force --repo-name=rhodecode-vcsserver
+
+
+Removing repositories from index
+++++++++++++++++++++++++++++++++
 
-* :file:`/home/{user}/.rccontrol/{instance-id}/mapping.ini`, using default
-  |RCT|.
+The indexer automatically removes renamed repositories and builds index for new names.
+In the same way if a listed repository in mapping.ini is not reported existing by the
+server it's removed from the index.
+In case that you wish to remove indexed repository manually such call would allow that::
+
+    rhodecode-index --instance-name=enterprise-1 --remove-only --repo-name=rhodecode-vcsserver
+
+
+Using search_mapping.ini file for advanced index rules
+++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+By default rhodecode-index runs for all repositories, all files with parsing limits
+defined by the CLI default arguments. You can change those limits by calling with
+different flags such as `--max-filesize=2048kb` or `--repo-limit=10`
+
+For more advanced execution logic it's possible to use a configuration file that
+would define detailed rules which repositories and how should be indexed.
+
+|RCT| provides an example index configuration file called :file:`search_mapping.ini`.
+This file is created by default during installation and is located at:
+
+* :file:`/home/{user}/.rccontrol/{instance-id}/search_mapping.ini`, using default |RCT|.
 * :file:`~/venv/lib/python2.7/site-packages/rhodecode_tools/templates/mapping.ini`,
   when using ``virtualenv``.
 
 .. note::
 
-    If you need to create the :file:`mapping.ini` file, use the |RCT|
-    ``rhodecode-index --create-mapping path/to/file`` API call. For details,
-    see the :ref:`tools-cli` section.
-
-The indexer runs in a random order to prevent a failing |repo| from stopping
-a build. To configure different indexing scenarios, set the following options
-inside the :file:`mapping.ini` and specify the altered file using the
-``--mapping`` option.
+    If you need to create the :file:`search_mapping.ini` file manually, use the |RCT|
+    ``rhodecode-index --create-mapping path/to/search_mapping.ini`` API call.
+    For details, see the :ref:`tools-cli` section.
 
-* ``index_files`` : Index the specified file types.
-* ``skip_files`` : Do not index the specified file types.
-* ``index_files_content`` : Index the content of the specified file types.
-* ``skip_files_content`` : Do not index the content of the specified files.
-* ``force`` : Create a fresh index on each run.
-* ``max_filesize`` : Files larger than the set size will not be indexed.
-* ``commit_parse_limit`` : Set the batch size when indexing commit messages.
-  Set to a lower number to lessen memory load.
-* ``repo_limit`` : Set the maximum number or |repos| indexed per run.
-* ``[INCLUDE]`` : Set |repos| you want indexed. This takes precedent over
-  ``[EXCLUDE]``.
-* ``[EXCLUDE]`` : Set |repos| you do not want indexed. Exclude can be used to
-  not index branches, forks, or log |repos|.
+To Run the indexer with mapping file provide it using `--mapping` flag::
 
-At the end of the file you can specify conditions for specific |repos| that
-will override the default values. To configure your indexer,
-use the following example :file:`mapping.ini` file.
+    rhodecode-index --instance-name=enterprise-1 --mapping=/my/path/search_mapping.ini
+
+
+Here's a detailed example of using :file:`search_mapping.ini` file.
 
 .. code-block:: ini
 
     [__DEFAULT__]
-    # default patterns for indexing files and content of files.
-    # Binary files are skipped by default.
+    ; Create index on commits data, and files data in this order. Available options
+    ; are `commits`, `files`
+    index_types = commits,files
+
+    ; Commit fetch limit. In what amount of chunks commits should be fetched
+    ; via api and parsed. This allows server to transfer smaller chunks and be less loaded
+    commit_fetch_limit = 1000
 
-    # Index python and markdown files
-    index_files = *.py, *.md
+    ; Commit process limit. Limit the number of commits indexer should fetch, and
+    ; store inside the full text search index. eg. if repo has 2000 commits, and
+    ; limit is 1000, on the first run it will process commits 0-1000 and on the
+    ; second 1000-2000 commits. Help reduce memory usage, default is 50000
+    ; (set -1 for unlimited)
+    commit_process_limit = 20000
 
-    # Do not index these file types
-    skip_files = *.svg, *.log, *.dump, *.txt
+    ; Limit of how many repositories each run can process, default is -1 (unlimited)
+    ; in case of 1000s of repositories it's better to execute in chunks to not overload
+    ; the server.
+    repo_limit = -1
 
-    # Index both file types and their content
-    index_files_content = *.cpp, *.ini, *.py
+    ; Default patterns for indexing files and content of files. Binary files
+    ; are skipped by default.
+
+    ; Add to index those comma separated files; globs syntax
+    ; e.g index_files = *.py, *.c, *.h, *.js
+    index_files = *,
+
+    ; Do not add to index those comma separated files, this excludes
+    ; both search by name and content; globs syntax
+    ; e.g index_files = *.key, *.sql, *.xml, *.pem, *.crt
+    skip_files = ,
 
-    # Index file names, but not file content
-    skip_files_content = *.svg,
+    ; Add to index content of those comma separated files; globs syntax
+    ; e.g index_files = *.h, *.obj
+    index_files_content = *,
 
-    # Force rebuilding an index from scratch. Each repository will be rebuild
-    # from scratch with a global flag. Use local flag to rebuild single repos
+    ; Do not add to index content of those comma separated files; globs syntax
+    ; Binary files are not indexed by default.
+    ; e.g index_files = *.min.js, *.xml, *.dump, *.log, *.dump
+    skip_files_content = ,
+
+    ; Force rebuilding an index from scratch. Each repository will be rebuild from
+    ; scratch with a global flag. Use --repo-name=NAME --force to rebuild single repo
     force = false
 
-    # Do not index files larger than 385KB
-    max_filesize = 385KB
+    ; maximum file size that indexer will use, files above that limit are not going
+    ; to have they content indexed.
+    ; Possible options are KB (kilobytes), MB (megabytes), eg 1MB or 1024KB
+    max_filesize = 10MB
 
-    # Limit commit indexing to 500 per batch
-    commit_parse_limit = 500
-
-    # Limit each index run to 25 repos
-    repo_limit = 25
 
-    # __INCLUDE__ is more important that __EXCLUDE__.
-
-    [__INCLUDE__]
-    # Include all repos with these names
+    [__INDEX_RULES__]
+    ; Ordered match rules for repositories. A list of all repositories will be fetched
+    ; using API and this list will be filtered using those rules.
+    ; Syntax for entry: `glob_pattern_OR_full_repo_name = 0 OR 1` where 0=exclude, 1=include
+    ; When this ordered list is traversed first match will return the include/exclude marker
+    ; For example:
+    ;    upstream/binary_repo = 0
+    ;    upstream/subrepo/xml_files = 0
+    ;    upstream/* = 1
+    ;    special-repo = 1
+    ;    * = 0
+    ; This will index all repositories under upstream/*, but skip upstream/binary_repo
+    ; and upstream/sub_repo/xml_files, last * = 0 means skip all other matches
 
-    docs/* = 1
-    lib/* = 1
-
-    [__EXCLUDE__]
-    # Do not include the following repo in index
 
-    dev-docs/* = 1
-    legacy-repos/* = 1
-    *-dev/* = 1
-
-    # Each repo that needs special indexing is a separate section below.
-    # In each section set the options to override the global configuration
-    # parameters above.
-    # If special settings are not configured, the global configuration values
-    # above are inherited. If no special repositories are
-    # defined here RhodeCode will use the API to ask for all repositories
+    ; == EXPLICIT REPOSITORY INDEXING ==
+    ; If defined this will skip using __INDEX_RULES__, and will not use API to fetch
+    ; list of repositories, it will explicitly take names defined with [NAME] format and
+    ; try to build the index, to build index just for repo_name_1 and special-repo use:
+    ;    [repo_name_1]
+    ;    [special-repo]
 
-    # For this repo use different settings
-    [special-repo]
-    commit_parse_limit = 20,
-    skip_files = *.idea, *.xml,
+    ; == PER REPOSITORY CONFIGURATION ==
+    ; This allows overriding the global configuration per repository.
+    ; example to set specific file limit, and skip certain files for repository special-repo
+    ; the CLI flags doesn't override the conf settings.
+    ;    [conf:special-repo]
+    ;    max_filesize = 5mb
+    ;    skip_files = *.xml, *.sql
+
 
-    # For another repo use different settings
-    [another-special-repo]
-    index_files = *,
-    max_filesize = 800MB
-    commit_parse_limit = 20000
+
+In case of 1000s of repositories it can be tricky to write the include/exclude rules at first.
+There's a special flag to test the mapping file rules and list repositories that would
+be indexed. Run the indexer with `--show-matched-repos` to list only the
+match repositories defined in .ini file rules::
+
+    rhodecode-index --instance-name=enterprise-1 --show-matched-repos --mapping=/my/path/search_mapping.ini
+
 
 .. _enable-elasticsearch:
 
-Enabling Elasticsearch
+Enabling ElasticSearch
 ^^^^^^^^^^^^^^^^^^^^^^
 
+ElasticSearch is available in EE edition only. It provides much scalable and more advanced
+search capabilities. While Whoosh is fine for upto 1-2GB of data, beyond that amount it
+starts slowing down, and can cause other problems.
+New ElasticSearch 6 also provides much more advanced query language.
+It allows advanced filtering by file paths, extensions, use OR statements, ranges etc.
+Please check query language examples in the search field for some advanced query language usage.
+
+
 1. Open the :file:`rhodecode.ini` file for the instance you wish to edit. The
    default location is
    :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
 2. Find the search configuration section:
 
 .. code-block:: ini
 
     ###################################
     ## SEARCH INDEXING CONFIGURATION ##
     ###################################
 
     search.module = rhodecode.lib.index.whoosh
     search.location = %(here)s/data/index
 
 and change it to:
 
 .. code-block:: ini
 
     search.module = rc_elasticsearch
-    search.location = http://localhost:9200/
+    search.location = http://localhost:9200
+    ## specify Elastic Search version, 6 for latest or 2 for legacy
+    search.es_version = 6
+
+where ``search.location`` points to the ElasticSearch server
+by default running on port 9200.
 
-where ``search.location`` points to the elasticsearch server.
+Index invocation also needs change. Please provide --es-version= and
+--engine-location= parameters to define ElasticSearch server location and it's version.
+For example::
+
+    rhodecode-index --instace-name=enterprise-1 --es-version=6 --engine-location=http://localhost:9200
+
 
 .. _Whoosh: https://pypi.python.org/pypi/Whoosh/
-.. _Elasticsearch: https://www.elastic.co/
  No newline at end of file
+.. _ElasticSearch 6: https://www.elastic.co/

docs/admin/nginx-config-example.rst

@@ -1,161 +1,192 b''
 Nginx Configuration Example
 ---------------------------
 
 Use the following example to configure Nginx as a your web server.
 
 
 .. code-block:: nginx
 
     ## Rate limiter for certain pages to prevent brute force attacks
     limit_req_zone  $binary_remote_addr  zone=req_limit:10m   rate=1r/s;
 
+    ## cache zone
+    proxy_cache_path /etc/nginx/nginx_cache levels=1:2 use_temp_path=off keys_zone=cache_zone:10m inactive=720h max_size=10g;
+
     ## Custom log format
     log_format log_custom '$remote_addr - $remote_user [$time_local] '
                           '"$request" $status $body_bytes_sent '
                           '"$http_referer" "$http_user_agent" '
                           '$request_time $upstream_response_time $pipe';
 
     ## Define one or more upstreams (local RhodeCode instance) to connect to
     upstream rc {
         # Url to running RhodeCode instance.
         # This is shown as `- URL: <host>` in output from rccontrol status.
         server 127.0.0.1:10002;
 
         # add more instances for load balancing
         # server 127.0.0.1:10003;
         # server 127.0.0.1:10004;
     }
 
     ## HTTP to HTTPS rewrite
     server {
         listen          80;
         server_name     rhodecode.myserver.com;
 
         if ($http_host = rhodecode.myserver.com) {
             rewrite  (.*)  https://rhodecode.myserver.com$1 permanent;
         }
     }
 
     ## Optional gist alias server, for serving nicer GIST urls.
     server {
         listen          443;
         server_name     gist.myserver.com;
         access_log      /var/log/nginx/gist.access.log log_custom;
         error_log       /var/log/nginx/gist.error.log;
 
         ssl on;
         ssl_certificate     gist.rhodecode.myserver.com.crt;
         ssl_certificate_key gist.rhodecode.myserver.com.key;
 
         ssl_session_timeout 5m;
 
         ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
         ssl_prefer_server_ciphers on;
         ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
 
         ## Strict http prevents from https -> http downgrade
         add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
 
         ## Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
         #ssl_dhparam /etc/nginx/ssl/dhparam.pem;
 
         rewrite ^/(.+)$ https://rhodecode.myserver.com/_admin/gists/$1;
         rewrite (.*)    https://rhodecode.myserver.com/_admin/gists;
     }
 
 
     ## MAIN SSL enabled server
     server {
         listen       443 ssl http2;
         server_name  rhodecode.myserver.com;
 
         access_log   /var/log/nginx/rhodecode.access.log log_custom;
         error_log    /var/log/nginx/rhodecode.error.log;
 
         ssl_certificate     rhodecode.myserver.com.crt;
         ssl_certificate_key rhodecode.myserver.com.key;
 
         # enable session resumption to improve https performance
         # http://vincent.bernat.im/en/blog/2011-ssl-session-reuse-rfc5077.html
         ssl_session_cache shared:SSL:50m;
         ssl_session_timeout 5m;
 
         ## Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
         #ssl_dhparam /etc/nginx/ssl/dhparam.pem;
 
         # enables server-side protection from BEAST attacks
         # http://blog.ivanristic.com/2013/09/is-beast-still-a-threat.html
         ssl_prefer_server_ciphers on;
 
         # disable SSLv3(enabled by default since nginx 0.8.19) since it's less secure then TLS http://en.wikipedia.org/wiki/Secure_Sockets_Layer#SSL_3.0
         ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
 
         # ciphers chosen for forward secrecy and compatibility
         # http://blog.ivanristic.com/2013/08/configuring-apache-nginx-and-openssl-for-forward-secrecy.html
         ssl_ciphers "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
 
         client_body_buffer_size     128k;
         # maximum number and size of buffers for large headers to read from client request
         large_client_header_buffers 16 256k;
 
         ## uncomment to serve static files by Nginx, recommended for performance
         # location /_static/rhodecode {
         #    gzip on;
         #    gzip_min_length  500;
         #    gzip_proxied     any;
         #    gzip_comp_level 4;
         #    gzip_types  text/css text/javascript text/xml text/plain text/x-component application/javascript application/json application/xml application/rss+xml font/truetype font/opentype application/vnd.ms-fontobject image/svg+xml;
         #    gzip_vary on;
         #    gzip_disable     "msie6";
         #    alias /path/to/.rccontrol/community-1/static;
         #    alias /path/to/.rccontrol/enterprise-1/static;
         # }
 
         ## channelstream location handler, if channelstream live chat and notifications
         ## are enable this will proxy the requests to channelstream websocket server
         location /_channelstream {
             rewrite /_channelstream/(.*) /$1 break;
             gzip                         off;
             tcp_nodelay                  off;
 
             proxy_connect_timeout        10;
             proxy_send_timeout           10m;
             proxy_read_timeout           10m;
 
             proxy_set_header             Host $host;
             proxy_set_header             X-Real-IP $remote_addr;
             proxy_set_header             X-Url-Scheme $scheme;
             proxy_set_header             X-Forwarded-Proto $scheme;
             proxy_set_header             X-Forwarded-For $proxy_add_x_forwarded_for;
 
             proxy_http_version           1.1;
             proxy_set_header Upgrade     $http_upgrade;
             proxy_set_header Connection  "upgrade";
 
             proxy_pass                  http://127.0.0.1:9800;
         }
 
         ## rate limit this endpoint to prevent login page brute-force attacks
         location /_admin/login {
             limit_req  zone=req_limit  burst=10  nodelay;
             try_files $uri @rhodecode_http;
         }
 
+        ## Special Cache for file store, make sure you enable this intentionally as
+        ## it could bypass upload files permissions
+        # location /_file_store/download {
+        #
+        #    proxy_cache cache_zone;
+        #    # ignore Set-Cookie
+        #    proxy_ignore_headers Set-Cookie;
+        #    proxy_ignore_headers Cookie;
+        #
+        #    proxy_cache_key $host$uri$is_args$args;
+        #    proxy_cache_methods GET;
+        #
+        #    proxy_cache_bypass  $http_cache_control;
+        #    proxy_cache_valid 200 302 720h;
+        #
+        #    proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
+        #
+        #    # returns cache status in headers
+        #    add_header X-Proxy-Cache $upstream_cache_status;
+        #    add_header Cache-Control "public";
+        #
+        #    proxy_cache_lock on;
+        #    proxy_cache_lock_age 5m;
+        #
+        #    proxy_pass  http://rc;
+        #
+        # }
+
         location / {
             try_files $uri @rhodecode_http;
         }
 
         location @rhodecode_http {
             # example of proxy.conf can be found in our docs.
             include     /etc/nginx/proxy.conf;
             proxy_pass  http://rc;
         }
 
         ## Custom 502 error page.
         ## Will be displayed while RhodeCode server is turned off
         error_page 502 /502.html;
         location = /502.html {
            #root  /path/to/.rccontrol/community-1/static;
            root  /path/to/.rccontrol/enterprise-1/static;
         }
     }
  No newline at end of file

docs/admin/repo-perm-steps.rst

@@ -1,45 +1,45 b''
 .. _set-repo-perms:
 
 Setting Repository Permissions
 ------------------------------
 
 To set the permissions on an individual |repo|, use the following steps:
 
 1. Open :menuselection:`Admin --> Repositories` and select
    :guilabel:`edit` beside the |repo| you wish to configure.
 2. On the |repo| settings page you will see a number of tabs. Exploring these
    you will find the following main configuration options for a |repo|.
 3. Once you make changes, select :guilabel:`Save`
 
 * :guilabel:`Repository group`: Lets you to add a |repo| to a |repo| group.
 * :guilabel:`Owner`: Lets you change the |repo| owner. Useful when users are
   moving roles within an organisation.
 * :guilabel:`Enable automatic locking`: For more information,
   see :ref:`repo-locking`
 * :guilabel:`User Access`: On the permissions tab you can add users,
   or user groups, and set the permissions each has for that |repo|.
 * :guilabel:`Invalidate repository cache`: On the Caches tab you can delete
   the |repo| cache, sometimes needed when mirroring.
 
 .. _set-repo-group-perms:
 
 Setting Repository Group Permissions
 ------------------------------------
 
 To set the permissions on a Repository Group, use the following steps:
 
 1. Open :menuselection:`Admin --> Repository groups` and select
    :guilabel:`edit` beside the |repo| you wish to configure.
 2. On the |repo| group settings page you will see a number of tabs. Exploring
    these you will find the following main configuration options:
 
 * :guilabel:`Owner`: Lets you change the group owner. Useful when users are
   moving roles within an organisation.
-* :guilabel:`Group parent`: Lets you add the |repo| group as a sub-group
+* :guilabel:`Repository group`: Lets you add the |repo| group as a sub-group
   of a larger group, i.e. :guilabel:`QA-Repos >> QA-Repos-Berlin`
 * :guilabel:`Enable automatic locking`: For more information,
   see :ref:`repo-locking`
 * :guilabel:`User Access`: On the permissions tab you can add users,
   or user groups, and set the permissions each has for that |repo| group.
 * :guilabel:`Add Child Group`: Allows you to add sub-repository-groups
   that will all share the same permissions.

docs/admin/system-admin.rst

@@ -1,32 +1,34 b''
 .. _rhodecode-admin-ref:
 
 System Administration
 =====================
 
 The following are the most common system administration tasks.
 
 .. only:: latex
 
     * :ref:`vcs-server`
     * :ref:`apache-ws-ref`
     * :ref:`nginx-ws-ref`
     * :ref:`rhodecode-tuning-ref`
     * :ref:`indexing-ref`
     * :ref:`rhodecode-reset-ref`
 
 .. toctree::
 
     config-files-overview
     vcs-server
     svn-http
     svn-path-permissions
     gunicorn-ssl-support
     apache-config
     nginx-config
     backup-restore
     tuning-rhodecode
     indexing
     reset-information
     enable-debug
     admin-tricks
     cleanup-cmds
+    restore-deleted-repositories
+

docs/admin/system-overview.rst

@@ -1,171 +1,171 b''
 .. _system-overview-ref:
 
 System Overview
 ===============
 
 Latest Version
 --------------
 
 * |release| on Unix and Windows systems.
 
 System Architecture
 -------------------
 
 The following diagram shows a typical production architecture.
 
 .. image:: ../images/architecture-diagram.png
   :align: center
 
 Supported Operating Systems
 ---------------------------
 
 Linux
 ^^^^^
 
 * Ubuntu 14.04
 * CentOS 6.2 and 7
 * Debian 7.8
 * RedHat Fedora
 * Arch Linux
 * SUSE Linux
 
 Windows
 ^^^^^^^
 
 * Windows Vista Ultimate 64bit
 * Windows 7 Ultimate 64bit
 * Windows 8 Professional 64bit
 * Windows 8.1 Enterprise 64bit
 * Windows Server 2008 64bit
 * Windows Server 2008-R2 64bit
 * Windows Server 2012 64bit
 
 Supported Databases
 -------------------
 
 * SQLite
 * MySQL
 * MariaDB
 * PostgreSQL
 
 Supported Browsers
 ------------------
 
 * Chrome
 * Safari
 * Firefox
 * Internet Explorer 10 & 11
 
 System Requirements
 -------------------
 
 |RCE| performs best on machines with ultra-fast hard disks. Generally disk
 performance is more important than CPU performance. In a corporate production
 environment handling 1000s of users and |repos| you should deploy on a 12+
 core 64GB RAM server. In short, the more RAM the better.
 
 
 For example:
 
  - for team of 1 - 5 active users you can run on 1GB RAM machine with 1CPU
  - above 250 active users, |RCE| needs at least 8GB of memory.
    Number of CPUs is less important, but recommended to have at least 2-3 CPUs
 
 
 .. _config-rce-files:
 
 Configuration Files
 -------------------
 
 * :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
-* :file:`/home/{user}/.rccontrol/{instance-id}/mapping.ini`
+* :file:`/home/{user}/.rccontrol/{instance-id}/search_mapping.ini`
 * :file:`/home/{user}/.rccontrol/{vcsserver-id}/vcsserver.ini`
 * :file:`/home/{user}/.rccontrol/supervisor/supervisord.ini`
 * :file:`/home/{user}/.rccontrol.ini`
 * :file:`/home/{user}/.rhoderc`
 * :file:`/home/{user}/.rccontrol/cache/MANIFEST`
 
 For more information, see the :ref:`config-files` section.
 
 Log Files
 ---------
 
 * :file:`/home/{user}/.rccontrol/{instance-id}/enterprise.log`
 * :file:`/home/{user}/.rccontrol/{vcsserver-id}/vcsserver.log`
 * :file:`/home/{user}/.rccontrol/supervisor/supervisord.log`
 * :file:`/tmp/rccontrol.log`
 * :file:`/tmp/rhodecode_tools.log`
 
 Storage Files
 -------------
 
 * :file:`/home/{user}/.rccontrol/{instance-id}/data/index/{index-file.toc}`
 * :file:`/home/{user}/repos/.rc_gist_store`
 * :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.db`
 * :file:`/opt/rhodecode/store/{unique-hash}`
 
 Default Repositories Location
 -----------------------------
 
 * :file:`/home/{user}/repos`
 
 Connection Methods
 ------------------
 
 * HTTPS
 * SSH
 * |RCE| API
 
 Internationalization Support
 ----------------------------
 
 Currently available in the following languages, see `Transifex`_ for the
 latest details. If you want a new language added, please contact us. To
 configure your language settings, see the :ref:`set-lang` section.
 
 .. hlist::
 
   * Belorussian
   * Chinese
   * French
   * German
   * Italian
   * Japanese
   * Portuguese
   * Polish
   * Russian
   * Spanish
 
 Licencing Information
 ---------------------
 
 * See licencing information `here`_
 
 Peer-to-peer Failover Support
 -----------------------------
 
 * Yes
 
 Additional Binaries
 -------------------
 
 * Yes, see :ref:`rhodecode-nix-ref` for full details.
 
 Remote Connectivity
 -------------------
 
 * Available
 
 Executable Files
 ----------------
 
 Windows: :file:`RhodeCode-installer-{version}.exe`
 
 Deprecated Support
 ------------------
 
 - Internet Explorer 8 support deprecated since version 3.7.0.
 - Internet Explorer 9 support deprecated since version 3.8.0.
 
 .. _here: https://rhodecode.com/licenses/
 .. _Transifex: https://www.transifex.com/projects/p/RhodeCode/

docs/api/api.rst

@@ -1,208 +1,209 b''
 .. _api:
 
 API Documentation
 =================
 
 The |RCE| API uses a single scheme for calling all API methods. The API is
 implemented with JSON protocol in both directions. To send API requests to
 your instance of |RCE|, use the following URL format
 ``<your_server>/_admin``
 
 .. note::
 
    To use the API, you should configure the :file:`~/.rhoderc` file with
    access details per instance. For more information, see
    :ref:`config-rhoderc`.
 
 
 API ACCESS FOR WEB VIEWS
 ------------------------
 
 API access can also be turned on for each web view in |RCE| that is
 decorated with a `@LoginRequired` decorator. To enable API access, change
 the standard login decorator to `@LoginRequired(api_access=True)`.
 
 From |RCE| version 1.7.0 you can configure a white list
 of views that have API access enabled by default. To enable these,
 edit the |RCE| configuration ``.ini`` file. The default location is:
 
 * |RCE| Pre-2.2.7 :file:`root/rhodecode/data/production.ini`
 * |RCE| 3.0 :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
 
 To configure the white list, edit this section of the file. In this
 configuration example, API access is granted to the patch/diff raw file and
 archive.
 
 .. code-block:: ini
 
     ## List of controllers (using glob syntax) that AUTH TOKENS could be used for access.
     ## Adding ?auth_token = <token> to the url authenticates this request as if it
     ## came from the the logged in user who own this authentication token.
     ##
     ## Syntax is <ControllerClass>:<function_pattern>.
     ## The list should be "," separated and on a single line.
     ##
     api_access_controllers_whitelist = RepoCommitsView:repo_commit_raw,RepoCommitsView:repo_commit_patch,RepoCommitsView:repo_commit_download
 
 After this change, a |RCE| view can be accessed without login by adding a
 GET parameter ``?auth_token=<auth_token>`` to a url. For example to
 access the raw diff.
 
 .. code-block:: html
 
    http://<server>/<repo>/changeset-diff/<sha>?auth_token=<auth_token>
 
 By default this is only enabled on RSS/ATOM feed views. Exposing raw diffs is a
 good way to integrate with 3rd party services like code review, or build farms
 that could download archives.
 
 API ACCESS
 ----------
 
 All clients are required to send JSON-RPC spec JSON data.
 
 .. code-block:: bash
 
     {
         "id:"<id>",
         "auth_token":"<auth_token>",
         "method":"<method_name>",
         "args":{"<arg_key>":"<arg_val>"}
     }
 
 Example call for auto pulling from remote repositories using curl:
 
 .. code-block:: bash
 
     curl https://server.com/_admin/api -X POST -H 'content-type:text/plain' --data-binary '{"id":1,
     "auth_token":"xe7cdb2v278e4evbdf5vs04v832v0efvcbcve4a3","method":"pull", "args":{"repoid":"CPython"}}'
 
 Provide those parameters:
  - **id** A value of any type, which is used to match the response with the
    request that it is replying to.
  - **auth_token** for access and permission validation.
  - **method** is name of method to call
  - **args** is an ``key:value`` list of arguments to pass to method
 
 .. note::
 
     To get your |authtoken|, from the |RCE| interface,
     go to:
     :menuselection:`username --> My account --> Auth tokens`
 
     For security reasons you should always create a dedicated |authtoken| for
     API use only.
 
 
 The |RCE| API will always return a JSON-RPC response:
 
 .. code-block:: bash
 
     {
         "id": <id>, # matching id sent by request
         "result": "<result>"|null, # JSON formatted result, null if any errors
         "error": "null"|<error_message> # JSON formatted error (if any)
     }
 
 All responses from API will be with `HTTP/1.0 200 OK` status code.
 If there is an error when calling the API, the *error* key will contain a
 failure description and the *result* will be `null`.
 
 API CLIENT
 ----------
 
 To install the |RCE| API, see :ref:`install-tools`. To configure the API per
 instance, see the :ref:`rc-tools` section as you need to configure a
 :file:`~/.rhoderc` file with your |authtokens|.
 
 Once you have set up your instance API access, use the following examples to
 get started.
 
 .. code-block:: bash
 
     # Getting the 'rhodecode' repository
     # from a RhodeCode Enterprise instance
     rhodecode-api --instance-name=enterprise-1 get_repo repoid:rhodecode
 
     Calling method get_repo => http://127.0.0.1:5000
     Server response
     {
         <json data>
     }
 
     # Creating a new mercurial repository called 'brand-new'
     # with a description 'Repo-description'
     rhodecode-api --instance-name=enterprise-1 create_repo repo_name:brand-new repo_type:hg description:Repo-description
     {
       "error": null,
       "id": 1110,
       "result": {
         "msg": "Created new repository `brand-new`",
         "success": true,
         "task": null
       }
     }
 
 A broken example, what not to do.
 
 .. code-block:: bash
 
     # A call missing the required arguments
     # and not specifying the instance
     rhodecode-api get_repo
 
     Calling method get_repo => http://127.0.0.1:5000
     Server response
     "Missing non optional `repoid` arg in JSON DATA"
 
 You can specify pure JSON using the ``--format`` parameter.
 
 .. code-block:: bash
 
     rhodecode-api --format=json get_repo repoid:rhodecode
 
 In such case only output that this function shows is pure JSON, we can use that
 and pipe output to some json formatter.
 
 If output is in pure JSON format, you can pipe output to a JSON formatter.
 
 .. code-block:: bash
 
     rhodecode-api --instance-name=enterprise-1 --format=json get_repo repoid:rhodecode | python -m json.tool
 
 API METHODS
 -----------
 
 Each method by default required following arguments.
 
 .. code-block:: bash
 
     id :      "<id_for_response>"
     auth_token : "<auth_token>"
     method :  "<method name>"
     args :    {}
 
 Use each **param** from docs and put it in args, Optional parameters
 are not required in args.
 
 .. code-block:: bash
 
     args: {"repoid": "rhodecode"}
 
 .. Note: From this point on things are generated by the script in
    `scripts/fabfile.py`. To change things below, update the docstrings in the
    ApiController.
 
 .. --- API DEFS MARKER ---
 .. toctree::
 
-   methods/views
+   methods/repo-methods
+   methods/store-methods
    methods/license-methods
    methods/deprecated-methods
    methods/gist-methods
    methods/pull-request-methods
    methods/repo-methods
    methods/repo-group-methods
    methods/server-methods
    methods/user-methods
    methods/user-group-methods

docs/api/methods/pull-request-methods.rst

@@ -1,428 +1,434 b''
 .. _pull-request-methods-ref:
 
 pull_request methods
 ====================
 
 close_pull_request 
 ------------------
 
 .. py:function:: close_pull_request(apiuser, pullrequestid, repoid=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>, message=<Optional:''>)
 
    Close the pull request specified by `pullrequestid`.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Repository name or repository ID to which the pull
        request belongs.
    :type repoid: str or int
    :param pullrequestid: ID of the pull request to be closed.
    :type pullrequestid: int
    :param userid: Close the pull request as this user.
    :type userid: Optional(str or int)
    :param message: Optional message to close the Pull Request with. If not
        specified it will be generated automatically.
    :type message: Optional(str)
 
    Example output:
 
    .. code-block:: bash
 
        "id": <id_given_in_input>,
        "result": {
            "pull_request_id":  "<int>",
            "close_status":     "<str:status_lbl>,
            "closed":           "<bool>"
        },
        "error": null
 
 
 comment_pull_request 
 --------------------
 
 .. py:function:: comment_pull_request(apiuser, pullrequestid, repoid=<Optional:None>, message=<Optional:None>, commit_id=<Optional:None>, status=<Optional:None>, comment_type=<Optional:u'note'>, resolves_comment_id=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Comment on the pull request specified with the `pullrequestid`,
    in the |repo| specified by the `repoid`, and optionally change the
    review status.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Optional repository name or repository ID.
    :type repoid: str or int
    :param pullrequestid: The pull request ID.
    :type pullrequestid: int
    :param commit_id: Specify the commit_id for which to set a comment. If
        given commit_id is different than latest in the PR status
        change won't be performed.
    :type commit_id: str
    :param message: The text content of the comment.
    :type message: str
    :param status: (**Optional**) Set the approval status of the pull
        request. One of: 'not_reviewed', 'approved', 'rejected',
        'under_review'
    :type status: str
    :param comment_type: Comment type, one of: 'note', 'todo'
    :type comment_type: Optional(str), default: 'note'
    :param userid: Comment on the pull request as this user
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result : {
            "pull_request_id":  "<Integer>",
            "comment_id":       "<Integer>",
            "status": {"given": <given_status>,
                       "was_changed": <bool status_was_actually_changed> },
        },
        error :  null
 
 
 create_pull_request 
 -------------------
 
-.. py:function:: create_pull_request(apiuser, source_repo, target_repo, source_ref, target_ref, title=<Optional:''>, description=<Optional:''>, description_renderer=<Optional:''>, reviewers=<Optional:None>)
+.. py:function:: create_pull_request(apiuser, source_repo, target_repo, source_ref, target_ref, owner=<Optional:<OptionalAttr:apiuser>>, title=<Optional:''>, description=<Optional:''>, description_renderer=<Optional:''>, reviewers=<Optional:None>)
 
    Creates a new pull request.
 
    Accepts refs in the following formats:
 
        * branch:<branch_name>:<sha>
        * branch:<branch_name>
        * bookmark:<bookmark_name>:<sha> (Mercurial only)
        * bookmark:<bookmark_name> (Mercurial only)
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param source_repo: Set the source repository name.
    :type source_repo: str
    :param target_repo: Set the target repository name.
    :type target_repo: str
    :param source_ref: Set the source ref name.
    :type source_ref: str
    :param target_ref: Set the target ref name.
    :type target_ref: str
+   :param owner: user_id or username
+   :type owner: Optional(str)
    :param title: Optionally Set the pull request title, it's generated otherwise
    :type title: str
    :param description: Set the pull request description.
    :type description: Optional(str)
    :type description_renderer: Optional(str)
    :param description_renderer: Set pull request renderer for the description.
        It should be 'rst', 'markdown' or 'plain'. If not give default
        system renderer will be used
    :param reviewers: Set the new pull request reviewers list.
        Reviewer defined by review rules will be added automatically to the
        defined list.
    :type reviewers: Optional(list)
        Accepts username strings or objects of the format:
 
            [{'username': 'nick', 'reasons': ['original author'], 'mandatory': <bool>}]
 
 
 get_pull_request 
 ----------------
 
 .. py:function:: get_pull_request(apiuser, pullrequestid, repoid=<Optional:None>)
 
    Get a pull request based on the given ID.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Optional, repository name or repository ID from where
        the pull request was opened.
    :type repoid: str or int
    :param pullrequestid: ID of the requested pull request.
    :type pullrequestid: int
 
    Example output:
 
    .. code-block:: bash
 
      "id": <id_given_in_input>,
      "result":
        {
            "pull_request_id":   "<pull_request_id>",
            "url":               "<url>",
            "title":             "<title>",
            "description":       "<description>",
            "status" :           "<status>",
            "created_on":        "<date_time_created>",
            "updated_on":        "<date_time_updated>",
            "commit_ids":        [
                                     ...
                                     "<commit_id>",
                                     "<commit_id>",
                                     ...
                                 ],
            "review_status":    "<review_status>",
            "mergeable":         {
                                     "status":  "<bool>",
                                     "message": "<message>",
                                 },
            "source":            {
                                     "clone_url":     "<clone_url>",
                                     "repository":    "<repository_name>",
                                     "reference":
                                     {
                                         "name":      "<name>",
                                         "type":      "<type>",
                                         "commit_id": "<commit_id>",
                                     }
                                 },
            "target":            {
                                     "clone_url":   "<clone_url>",
                                     "repository":    "<repository_name>",
                                     "reference":
                                     {
                                         "name":      "<name>",
                                         "type":      "<type>",
                                         "commit_id": "<commit_id>",
                                     }
                                 },
            "merge":             {
                                     "clone_url":   "<clone_url>",
                                     "reference":
                                     {
                                         "name":      "<name>",
                                         "type":      "<type>",
                                         "commit_id": "<commit_id>",
                                     }
                                 },
           "author":             <user_obj>,
           "reviewers":          [
                                     ...
                                     {
                                        "user":          "<user_obj>",
                                        "review_status": "<review_status>",
                                     }
                                     ...
                                 ]
        },
       "error": null
 
 
 get_pull_request_comments 
 -------------------------
 
 .. py:function:: get_pull_request_comments(apiuser, pullrequestid, repoid=<Optional:None>)
 
    Get all comments of pull request specified with the `pullrequestid`
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Optional repository name or repository ID.
    :type repoid: str or int
    :param pullrequestid: The pull request ID.
    :type pullrequestid: int
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result : [
            {
              "comment_author": {
                "active": true,
                "full_name_or_username": "Tom Gore",
                "username": "admin"
              },
              "comment_created_on": "2017-01-02T18:43:45.533",
              "comment_f_path": null,
              "comment_id": 25,
              "comment_lineno": null,
              "comment_status": {
                "status": "under_review",
                "status_lbl": "Under Review"
              },
              "comment_text": "Example text",
              "comment_type": null,
              "pull_request_version": null
            }
        ],
        error :  null
 
 
 get_pull_requests 
 -----------------
 
-.. py:function:: get_pull_requests(apiuser, repoid, status=<Optional:'new'>)
+.. py:function:: get_pull_requests(apiuser, repoid, status=<Optional:'new'>, merge_state=<Optional:True>)
 
    Get all pull requests from the repository specified in `repoid`.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Optional repository name or repository ID.
    :type repoid: str or int
    :param status: Only return pull requests with the specified status.
        Valid options are.
        * ``new`` (default)
        * ``open``
        * ``closed``
    :type status: str
+   :param merge_state: Optional calculate merge state for each repository.
+       This could result in longer time to fetch the data
+   :type merge_state: bool
 
    Example output:
 
    .. code-block:: bash
 
      "id": <id_given_in_input>,
      "result":
        [
            ...
            {
                "pull_request_id":   "<pull_request_id>",
                "url":               "<url>",
                "title" :            "<title>",
                "description":       "<description>",
                "status":            "<status>",
                "created_on":        "<date_time_created>",
                "updated_on":        "<date_time_updated>",
                "commit_ids":        [
                                         ...
                                         "<commit_id>",
                                         "<commit_id>",
                                         ...
                                     ],
                "review_status":    "<review_status>",
                "mergeable":         {
                                        "status":      "<bool>",
                                        "message:      "<message>",
                                     },
                "source":            {
                                         "clone_url":     "<clone_url>",
                                         "reference":
                                         {
                                             "name":      "<name>",
                                             "type":      "<type>",
                                             "commit_id": "<commit_id>",
                                         }
                                     },
                "target":            {
                                         "clone_url":   "<clone_url>",
                                         "reference":
                                         {
                                             "name":      "<name>",
                                             "type":      "<type>",
                                             "commit_id": "<commit_id>",
                                         }
                                     },
                "merge":             {
                                         "clone_url":   "<clone_url>",
                                         "reference":
                                         {
                                             "name":      "<name>",
                                             "type":      "<type>",
                                             "commit_id": "<commit_id>",
                                         }
                                     },
               "author":             <user_obj>,
               "reviewers":          [
                                         ...
                                         {
                                            "user":          "<user_obj>",
                                            "review_status": "<review_status>",
                                         }
                                         ...
                                     ]
            }
            ...
        ],
      "error": null
 
 
 merge_pull_request 
 ------------------
 
 .. py:function:: merge_pull_request(apiuser, pullrequestid, repoid=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Merge the pull request specified by `pullrequestid` into its target
    repository.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Optional, repository name or repository ID of the
        target repository to which the |pr| is to be merged.
    :type repoid: str or int
    :param pullrequestid: ID of the pull request which shall be merged.
    :type pullrequestid: int
    :param userid: Merge the pull request as this user.
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
        "id": <id_given_in_input>,
        "result": {
-           "executed":         "<bool>",
-           "failure_reason":   "<int>",
-           "merge_commit_id":  "<merge_commit_id>",
-           "possible":         "<bool>",
+           "executed":               "<bool>",
+           "failure_reason":         "<int>",
+           "merge_status_message":   "<str>",
+           "merge_commit_id":        "<merge_commit_id>",
+           "possible":               "<bool>",
            "merge_ref":        {
                                    "commit_id": "<commit_id>",
                                    "type":      "<type>",
                                    "name":      "<name>"
                                }
        },
        "error": null
 
 
 update_pull_request 
 -------------------
 
 .. py:function:: update_pull_request(apiuser, pullrequestid, repoid=<Optional:None>, title=<Optional:''>, description=<Optional:''>, description_renderer=<Optional:''>, reviewers=<Optional:None>, update_commits=<Optional:None>)
 
    Updates a pull request.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Optional repository name or repository ID.
    :type repoid: str or int
    :param pullrequestid: The pull request ID.
    :type pullrequestid: int
    :param title: Set the pull request title.
    :type title: str
    :param description: Update pull request description.
    :type description: Optional(str)
    :type description_renderer: Optional(str)
    :param description_renderer: Update pull request renderer for the description.
        It should be 'rst', 'markdown' or 'plain'
    :param reviewers: Update pull request reviewers list with new value.
    :type reviewers: Optional(list)
        Accepts username strings or objects of the format:
 
            [{'username': 'nick', 'reasons': ['original author'], 'mandatory': <bool>}]
 
    :param update_commits: Trigger update of commits for this pull request
    :type: update_commits: Optional(bool)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result : {
            "msg": "Updated pull request `63`",
            "pull_request": <pull_request_object>,
            "updated_reviewers": {
              "added": [
                "username"
              ],
              "removed": []
            },
            "updated_commits": {
              "added": [
                "<sha1_hash>"
              ],
              "common": [
                "<sha1_hash>",
                "<sha1_hash>",
              ],
              "removed": []
            }
        }
        error :  null
 
 

docs/api/methods/repo-methods.rst

@@ -1,1028 +1,1134 b''
 .. _repo-methods-ref:
 
 repo methods
 ============
 
 add_field_to_repo 
 -----------------
 
 .. py:function:: add_field_to_repo(apiuser, repoid, key, label=<Optional:''>, description=<Optional:''>)
 
    Adds an extra field to a repository.
 
    This command can only be run using an |authtoken| with at least
    write permissions to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository id.
    :type repoid: str or int
    :param key: Create a unique field key for this repository.
    :type key: str
    :param label:
    :type label: Optional(str)
    :param description:
    :type description: Optional(str)
 
 
 comment_commit 
 --------------
 
 .. py:function:: comment_commit(apiuser, repoid, commit_id, message, status=<Optional:None>, comment_type=<Optional:u'note'>, resolves_comment_id=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Set a commit comment, and optionally change the status of the commit.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param commit_id: Specify the commit_id for which to set a comment.
    :type commit_id: str
    :param message: The comment text.
    :type message: str
    :param status: (**Optional**) status of commit, one of: 'not_reviewed',
        'approved', 'rejected', 'under_review'
    :type status: str
    :param comment_type: Comment type, one of: 'note', 'todo'
    :type comment_type: Optional(str), default: 'note'
    :param userid: Set the user name of the comment creator.
    :type userid: Optional(str or int)
 
    Example error output:
 
    .. code-block:: bash
 
        {
            "id" : <id_given_in_input>,
            "result" : {
                "msg": "Commented on commit `<commit_id>` for repository `<repoid>`",
                "status_change": null or <status>,
                "success": true
            },
            "error" :  null
        }
 
 
 create_repo 
 -----------
 
 .. py:function:: create_repo(apiuser, repo_name, repo_type, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, push_uri=<Optional:None>, landing_rev=<Optional:'rev:tip'>, enable_statistics=<Optional:False>, enable_locking=<Optional:False>, enable_downloads=<Optional:False>, copy_permissions=<Optional:False>)
 
    Creates a repository.
 
    * If the repository name contains "/", repository will be created inside
      a repository group or nested repository groups
 
      For example "foo/bar/repo1" will create |repo| called "repo1" inside
      group "foo/bar". You have to have permissions to access and write to
      the last repository group ("bar" in this example)
 
    This command can only be run using an |authtoken| with at least
    permissions to create repositories, or write permissions to
    parent repository groups.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repo_name: Set the repository name.
    :type repo_name: str
    :param repo_type: Set the repository type; 'hg','git', or 'svn'.
    :type repo_type: str
    :param owner: user_id or username
    :type owner: Optional(str)
    :param description: Set the repository description.
    :type description: Optional(str)
    :param private: set repository as private
    :type private: bool
    :param clone_uri: set clone_uri
    :type clone_uri: str
    :param push_uri: set push_uri
    :type push_uri: str
    :param landing_rev: <rev_type>:<rev>
    :type landing_rev: str
    :param enable_locking:
    :type enable_locking: bool
    :param enable_downloads:
    :type enable_downloads: bool
    :param enable_statistics:
    :type enable_statistics: bool
    :param copy_permissions: Copy permission from group in which the
        repository is being created.
    :type copy_permissions: bool
 
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg": "Created new repository `<reponame>`",
                  "success": true,
                  "task": "<celery task id or None if done sync>"
                }
        error:  null
 
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
         'failed to create repository `<repo_name>`'
      }
 
 
 delete_repo 
 -----------
 
 .. py:function:: delete_repo(apiuser, repoid, forks=<Optional:''>)
 
    Deletes a repository.
 
    * When the `forks` parameter is set it's possible to detach or delete
      forks of deleted repository.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param forks: Set to `detach` or `delete` forks from the |repo|.
    :type forks: Optional(str)
 
    Example error output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg": "Deleted repository `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 fork_repo 
 ---------
 
 .. py:function:: fork_repo(apiuser, repoid, fork_name, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, landing_rev=<Optional:'rev:tip'>, copy_permissions=<Optional:False>)
 
    Creates a fork of the specified |repo|.
 
    * If the fork_name contains "/", fork will be created inside
      a repository group or nested repository groups
 
      For example "foo/bar/fork-repo" will create fork called "fork-repo"
      inside group "foo/bar". You have to have permissions to access and
      write to the last repository group ("bar" in this example)
 
    This command can only be run using an |authtoken| with minimum
    read permissions of the forked repo, create fork permissions for an user.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set repository name or repository ID.
    :type repoid: str or int
    :param fork_name: Set the fork name, including it's repository group membership.
    :type fork_name: str
    :param owner: Set the fork owner.
    :type owner: str
    :param description: Set the fork description.
    :type description: str
    :param copy_permissions: Copy permissions from parent |repo|. The
        default is False.
    :type copy_permissions: bool
    :param private: Make the fork private. The default is False.
    :type private: bool
    :param landing_rev: Set the landing revision. The default is tip.
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_for_response>
        api_key : "<api_key>"
        args:     {
                    "repoid" :          "<reponame or repo_id>",
                    "fork_name":        "<forkname>",
                    "owner":            "<username or user_id = Optional(=apiuser)>",
                    "description":      "<description>",
                    "copy_permissions": "<bool>",
                    "private":          "<bool>",
                    "landing_rev":      "<landing_rev>"
                  }
 
    Example error output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg": "Created fork of `<reponame>` as `<forkname>`",
                  "success": true,
                  "task": "<celery task id or None if done sync>"
                }
        error:  null
 
 
 get_repo 
 --------
 
 .. py:function:: get_repo(apiuser, repoid, cache=<Optional:True>)
 
    Gets an existing repository by its name or repository_id.
 
    The members section so the output returns users groups or users
    associated with that repository.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository id.
    :type repoid: str or int
    :param cache: use the cached value for last changeset
    :type: cache: Optional(bool)
 
    Example output:
 
    .. code-block:: bash
 
        {
          "error": null,
          "id": <repo_id>,
          "result": {
            "clone_uri": null,
            "created_on": "timestamp",
            "description": "repo description",
            "enable_downloads": false,
            "enable_locking": false,
            "enable_statistics": false,
            "followers": [
              {
                "active": true,
                "admin": false,
                "api_key": "****************************************",
                "api_keys": [
                  "****************************************"
                ],
                "email": "[e-mail]",
                "emails": [
                  "[e-mail]"
                ],
                "extern_name": "rhodecode",
                "extern_type": "rhodecode",
                "firstname": "username",
                "ip_addresses": [],
                "language": null,
                "last_login": "2015-09-16T17:16:35.854",
                "lastname": "surname",
                "user_id": <user_id>,
                "username": "name"
              }
            ],
            "fork_of": "parent-repo",
            "landing_rev": [
              "rev",
              "tip"
            ],
            "last_changeset": {
              "author": "User <[e-mail]>",
              "branch": "default",
              "date": "timestamp",
              "message": "last commit message",
              "parents": [
                {
                  "raw_id": "commit-id"
                }
              ],
              "raw_id": "commit-id",
              "revision": <revision number>,
              "short_id": "short id"
            },
            "lock_reason": null,
            "locked_by": null,
            "locked_date": null,
            "owner": "owner-name",
            "permissions": [
              {
                "name": "super-admin-name",
                "origin": "super-admin",
                "permission": "repository.admin",
                "type": "user"
              },
              {
                "name": "owner-name",
                "origin": "owner",
                "permission": "repository.admin",
                "type": "user"
              },
              {
                "name": "user-group-name",
                "origin": "permission",
                "permission": "repository.write",
                "type": "user_group"
              }
            ],
            "private": true,
            "repo_id": 676,
            "repo_name": "user-group/repo-name",
            "repo_type": "hg"
          }
        }
 
 
 get_repo_changeset 
 ------------------
 
 .. py:function:: get_repo_changeset(apiuser, repoid, revision, details=<Optional:'basic'>)
 
    Returns information about a changeset.
 
    Additionally parameters define the amount of details returned by
    this function.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository id
    :type repoid: str or int
    :param revision: revision for which listing should be done
    :type revision: str
    :param details: details can be 'basic|extended|full' full gives diff
        info details like the diff itself, and number of changed files etc.
    :type details: Optional(str)
 
 
 get_repo_changesets 
 -------------------
 
 .. py:function:: get_repo_changesets(apiuser, repoid, start_rev, limit, details=<Optional:'basic'>)
 
    Returns a set of commits limited by the number starting
    from the `start_rev` option.
 
    Additional parameters define the amount of details returned by this
    function.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param start_rev: The starting revision from where to get changesets.
    :type start_rev: str
    :param limit: Limit the number of commits to this amount
    :type limit: str or int
    :param details: Set the level of detail returned. Valid option are:
        ``basic``, ``extended`` and ``full``.
    :type details: Optional(str)
 
    .. note::
 
       Setting the parameter `details` to the value ``full`` is extensive
       and returns details like the diff itself, and the number
       of changed files.
 
 
-get_repo_nodes 
---------------
+get_repo_comments 
+-----------------
+
+.. py:function:: get_repo_comments(apiuser, repoid, commit_id=<Optional:None>, comment_type=<Optional:None>, userid=<Optional:None>)
+
+   Get all comments for a repository
 
-.. py:function:: get_repo_nodes(apiuser, repoid, revision, root_path, ret_type=<Optional:'all'>, details=<Optional:'basic'>, max_file_bytes=<Optional:None>)
+   :param apiuser: This is filled automatically from the |authtoken|.
+   :type apiuser: AuthUser
+   :param repoid: Set the repository name or repository ID.
+   :type repoid: str or int
+   :param commit_id: Optionally filter the comments by the commit_id
+   :type commit_id: Optional(str), default: None
+   :param comment_type: Optionally filter the comments by the comment_type
+       one of: 'note', 'todo'
+   :type comment_type: Optional(str), default: None
+   :param userid: Optionally filter the comments by the author of comment
+   :type userid: Optional(str or int), Default: None
+
+   Example error output:
+
+   .. code-block:: bash
 
-   Returns a list of nodes and children in a flat list for a given
-   path at given revision.
+       {
+           "id" : <id_given_in_input>,
+           "result" : [
+               {
+                 "comment_author": <USER_DETAILS>,
+                 "comment_created_on": "2017-02-01T14:38:16.309",
+                 "comment_f_path": "file.txt",
+                 "comment_id": 282,
+                 "comment_lineno": "n1",
+                 "comment_resolved_by": null,
+                 "comment_status": [],
+                 "comment_text": "This file needs a header",
+                 "comment_type": "todo"
+               }
+           ],
+           "error" :  null
+       }
 
-   It's possible to specify ret_type to show only `files` or `dirs`.
+
+get_repo_file 
+-------------
+
+.. py:function:: get_repo_file(apiuser, repoid, commit_id, file_path, max_file_bytes=<Optional:None>, details=<Optional:'basic'>, cache=<Optional:True>)
+
+   Returns a single file from repository at given revision.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
-   :param revision: The revision for which listing should be done.
-   :type revision: str
-   :param root_path: The path from which to start displaying.
-   :type root_path: str
-   :param ret_type: Set the return type. Valid options are
-       ``all`` (default), ``files`` and ``dirs``.
-   :type ret_type: Optional(str)
-   :param details: Returns extended information about nodes, such as
-       md5, binary, and or content.  The valid options are ``basic`` and
-       ``full``.
+   :param commit_id: The revision for which listing should be done.
+   :type commit_id: str
+   :param file_path: The path from which to start displaying.
+   :type file_path: str
+   :param details: Returns different set of information about nodes.
+       The valid options are ``minimal`` ``basic`` and ``full``.
    :type details: Optional(str)
    :param max_file_bytes: Only return file content under this file size bytes
-   :type details: Optional(int)
-
+   :type max_file_bytes: Optional(int)
+   :param cache: Use internal caches for fetching files. If disabled fetching
+       files is slower but more memory efficient
+   :type cache: Optional(bool)
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
-       result: [
-                 {
-                   "name" : "<name>"
-                   "type" : "<type>",
-                   "binary": "<true|false>" (only in extended mode)
-                   "md5"  : "<md5 of file content>" (only in extended mode)
-                 },
-                 ...
-               ]
+       result: {
+           "binary": false,
+           "extension": "py",
+           "lines": 35,
+           "content": "....",
+           "md5": "76318336366b0f17ee249e11b0c99c41",
+           "mimetype": "text/x-python",
+           "name": "python.py",
+           "size": 817,
+           "type": "file",
+       }
        error:  null
 
 
+get_repo_fts_tree 
+-----------------
+
+.. py:function:: get_repo_fts_tree(apiuser, repoid, commit_id, root_path)
+
+   Returns a list of tree nodes for path at given revision. This api is built
+   strictly for usage in full text search building, and shouldn't be consumed
+
+   This command can only be run using an |authtoken| with admin rights,
+   or users with at least read rights to |repos|.
+
+
+get_repo_nodes 
+--------------
+
+.. py:function:: get_repo_nodes(apiuser, repoid, revision, root_path, ret_type=<Optional:'all'>, details=<Optional:'basic'>, max_file_bytes=<Optional:None>)
+
+   Returns a list of nodes and children in a flat list for a given
+       path at given revision.
+
+       It's possible to specify ret_type to show only `files` or `dirs`.
+
+       This command can only be run using an |authtoken| with admin rights,
+       or users with at least read rights to |repos|.
+
+       :param apiuser: This is filled automatically from the |authtoken|.
+       :type apiuser: AuthUser
+       :param repoid: The repository name or repository ID.
+       :type repoid: str or int
+       :param revision: The revision for which listing should be done.
+       :type revision: str
+       :param root_path: The path from which to start displaying.
+       :type root_path: str
+       :param ret_type: Set the return type. Valid options are
+           ``all`` (default), ``files`` and ``dirs``.
+       :type ret_type: Optional(str)
+       :param details: Returns extended information about nodes, such as
+           md5, binary, and or content.
+           The valid options are ``basic`` and ``full``.
+       :type details: Optional(str)
+       :param max_file_bytes: Only return file content under this file size bytes
+       :type details: Optional(int)
+
+       Example output:
+
+       .. code-block:: bash
+
+           id : <id_given_in_input>
+           result: [
+                       {
+                         "binary": false,
+                         "content": "File line
+   Line2
+   ",
+                         "extension": "md",
+                         "lines": 2,
+                         "md5": "059fa5d29b19c0657e384749480f6422",
+                         "mimetype": "text/x-minidsrc",
+                         "name": "file.md",
+                         "size": 580,
+                         "type": "file"
+                       },
+                     ...
+                   ]
+           error:  null
+
+
 get_repo_refs 
 -------------
 
 .. py:function:: get_repo_refs(apiuser, repoid)
 
    Returns a dictionary of current references. It returns
    bookmarks, branches, closed_branches, and tags for given repository
 
    It's possible to specify ret_type to show only `files` or `dirs`.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        "result": {
            "bookmarks": {
              "dev": "5611d30200f4040ba2ab4f3d64e5b06408a02188",
              "master": "367f590445081d8ec8c2ea0456e73ae1f1c3d6cf"
            },
            "branches": {
              "default": "5611d30200f4040ba2ab4f3d64e5b06408a02188",
              "stable": "367f590445081d8ec8c2ea0456e73ae1f1c3d6cf"
            },
            "branches_closed": {},
            "tags": {
              "tip": "5611d30200f4040ba2ab4f3d64e5b06408a02188",
              "v4.4.0": "1232313f9e6adac5ce5399c2a891dc1e72b79022",
              "v4.4.1": "cbb9f1d329ae5768379cdec55a62ebdd546c4e27",
              "v4.4.2": "24ffe44a27fcd1c5b6936144e176b9f6dd2f3a17",
            }
        }
        error:  null
 
 
 get_repo_settings 
 -----------------
 
 .. py:function:: get_repo_settings(apiuser, repoid, key=<Optional:None>)
 
    Returns all settings for a repository. If key is given it only returns the
    setting identified by the key or null.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository id.
    :type repoid: str or int
    :param key: Key of the setting to return.
    :type: key: Optional(str)
 
    Example output:
 
    .. code-block:: bash
 
        {
            "error": null,
            "id": 237,
            "result": {
                "extensions_largefiles": true,
                "extensions_evolve": true,
                "hooks_changegroup_push_logger": true,
                "hooks_changegroup_repo_size": false,
                "hooks_outgoing_pull_logger": true,
                "phases_publish": "True",
                "rhodecode_hg_use_rebase_for_merging": true,
                "rhodecode_pr_merge_enabled": true,
                "rhodecode_use_outdated_comments": true
            }
        }
 
 
 get_repos 
 ---------
 
 .. py:function:: get_repos(apiuser, root=<Optional:None>, traverse=<Optional:True>)
 
    Lists all existing repositories.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param root: specify root repository group to fetch repositories.
        filters the returned repositories to be members of given root group.
    :type root: Optional(None)
    :param traverse: traverse given root into subrepositories. With this flag
        set to False, it will only return top-level repositories from `root`.
        if root is empty it will return just top-level repositories.
    :type traverse: Optional(True)
 
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: [
                  {
                    "repo_id" :          "<repo_id>",
                    "repo_name" :        "<reponame>"
                    "repo_type" :        "<repo_type>",
                    "clone_uri" :        "<clone_uri>",
                    "private": :         "<bool>",
                    "created_on" :       "<datetimecreated>",
                    "description" :      "<description>",
                    "landing_rev":       "<landing_rev>",
                    "owner":             "<repo_owner>",
                    "fork_of":           "<name_of_fork_parent>",
                    "enable_downloads":  "<bool>",
                    "enable_locking":    "<bool>",
                    "enable_statistics": "<bool>",
                  },
                  ...
                ]
        error:  null
 
 
 grant_user_group_permission 
 ---------------------------
 
 .. py:function:: grant_user_group_permission(apiuser, repoid, usergroupid, perm)
 
    Grant permission for a user group on the specified repository,
    or update existing permissions.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param usergroupid: Specify the ID of the user group.
    :type usergroupid: str or int
    :param perm: Set the user group permissions using the following
        format: (repository.(none|read|write|admin))
    :type perm: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg" : "Granted perm: `<perm>` for group: `<usersgroupname>` in repo: `<reponame>`",
        "success": true
 
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to edit permission for user group: `<usergroup>` in repo `<repo>`'
      }
 
 
 grant_user_permission 
 ---------------------
 
 .. py:function:: grant_user_permission(apiuser, repoid, userid, perm)
 
    Grant permissions for the specified user on the given repository,
    or update existing permissions if found.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param userid: Set the user name.
    :type userid: str
    :param perm: Set the user permissions, using the following format
        ``(repository.(none|read|write|admin))``
    :type perm: str
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Granted perm: `<perm>` for user: `<username>` in repo: `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 invalidate_cache 
 ----------------
 
 .. py:function:: invalidate_cache(apiuser, repoid, delete_keys=<Optional:False>)
 
    Invalidates the cache for the specified repository.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Sets the repository name or repository ID.
    :type repoid: str or int
    :param delete_keys: This deletes the invalidated keys instead of
        just flagging them.
    :type delete_keys: Optional(``True`` | ``False``)
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'msg': Cache for repository `<repository name>` was invalidated,
        'repository': <repository name>
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error : {
         'Error occurred during cache invalidation action'
      }
 
 
 lock 
 ----
 
 .. py:function:: lock(apiuser, repoid, locked=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Sets the lock state of the specified |repo| by the given user.
    From more information, see :ref:`repo-locking`.
 
    * If the ``userid`` option is not set, the repository is locked to the
      user who called the method.
    * If the ``locked`` parameter is not set, the current lock state of the
      repository is displayed.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Sets the repository name or repository ID.
    :type repoid: str or int
    :param locked: Sets the lock state.
    :type locked: Optional(``True`` | ``False``)
    :param userid: Set the repository lock to this user.
    :type userid: Optional(str or int)
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'repo': '<reponame>',
        'locked': <bool: lock state>,
        'locked_since': <int: lock timestamp>,
        'locked_by': <username of person who made the lock>,
        'lock_reason': <str: reason for locking>,
        'lock_state_changed': <bool: True if lock state has been changed in this request>,
        'msg': 'Repo `<reponame>` locked by `<username>` on <timestamp>.'
        or
        'msg': 'Repo `<repository name>` not locked.'
        or
        'msg': 'User `<user name>` set lock state for repo `<repository name>` to `<new lock state>`'
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        'Error occurred locking repository `<reponame>`'
      }
 
 
 maintenance 
 -----------
 
 .. py:function:: maintenance(apiuser, repoid)
 
    Triggers a maintenance on the given repository.
 
    This command can only be run using an |authtoken| with admin
    rights to the specified repository. For more information,
    see :ref:`config-token-ref`.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "executed maintenance command",
        "executed_actions": [
           <action_message>, <action_message2>...
        ],
        "repository": "<repository name>"
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "Unable to execute maintenance on `<reponame>`"
      }
 
 
 pull 
 ----
 
 .. py:function:: pull(apiuser, repoid, remote_uri=<Optional:None>)
 
    Triggers a pull on the given repository from a remote location. You
    can use this to keep remote repositories up-to-date.
 
    This command can only be run using an |authtoken| with admin
    rights to the specified repository. For more information,
    see :ref:`config-token-ref`.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param remote_uri: Optional remote URI to pass in for pull
    :type remote_uri: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "Pulled from url `<remote_url>` on repo `<repository name>`"
        "repository": "<repository name>"
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "Unable to push changes from `<remote_url>`"
      }
 
 
 remove_field_from_repo 
 ----------------------
 
 .. py:function:: remove_field_from_repo(apiuser, repoid, key)
 
    Removes an extra field from a repository.
 
    This command can only be run using an |authtoken| with at least
    write permissions to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param key: Set the unique field key for this repository.
    :type key: str
 
 
 revoke_user_group_permission 
 ----------------------------
 
 .. py:function:: revoke_user_group_permission(apiuser, repoid, usergroupid)
 
    Revoke the permissions of a user group on a given repository.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param usergroupid: Specify the user group ID.
    :type usergroupid: str or int
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Revoked perm for group: `<usersgroupname>` in repo: `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 revoke_user_permission 
 ----------------------
 
 .. py:function:: revoke_user_permission(apiuser, repoid, userid)
 
    Revoke permission for a user on the specified repository.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param userid: Set the user name of revoked user.
    :type userid: str or int
 
    Example error output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Revoked perm for user: `<username>` in repo: `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 set_repo_settings 
 -----------------
 
 .. py:function:: set_repo_settings(apiuser, repoid, settings)
 
    Update repository settings. Returns true on success.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository id.
    :type repoid: str or int
    :param settings: The new settings for the repository.
    :type: settings: dict
 
    Example output:
 
    .. code-block:: bash
 
        {
            "error": null,
            "id": 237,
            "result": true
        }
 
 
 strip 
 -----
 
 .. py:function:: strip(apiuser, repoid, revision, branch)
 
    Strips the given revision from the specified repository.
 
    * This will remove the revision and all of its decendants.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param revision: The revision you wish to strip.
    :type revision: str
    :param branch: The branch from which to strip the revision.
    :type branch: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "'Stripped commit <commit_hash> from repo `<repository name>`'"
        "repository": "<repository name>"
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "Unable to strip commit <commit_hash> from repo `<repository name>`"
      }
 
 
 update_repo 
 -----------
 
 .. py:function:: update_repo(apiuser, repoid, repo_name=<Optional:None>, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, push_uri=<Optional:None>, landing_rev=<Optional:'rev:tip'>, fork_of=<Optional:None>, enable_statistics=<Optional:False>, enable_locking=<Optional:False>, enable_downloads=<Optional:False>, fields=<Optional:''>)
 
    Updates a repository with the given information.
 
    This command can only be run using an |authtoken| with at least
    admin permissions to the |repo|.
 
    * If the repository name contains "/", repository will be updated
      accordingly with a repository group or nested repository groups
 
      For example repoid=repo-test name="foo/bar/repo-test" will update |repo|
      called "repo-test" and place it inside group "foo/bar".
      You have to have permissions to access and write to the last repository
      group ("bar" in this example)
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: repository name or repository ID.
    :type repoid: str or int
    :param repo_name: Update the |repo| name, including the
        repository group it's in.
    :type repo_name: str
    :param owner: Set the |repo| owner.
    :type owner: str
    :param fork_of: Set the |repo| as fork of another |repo|.
    :type fork_of: str
    :param description: Update the |repo| description.
    :type description: str
    :param private: Set the |repo| as private. (True | False)
    :type private: bool
    :param clone_uri: Update the |repo| clone URI.
    :type clone_uri: str
    :param landing_rev: Set the |repo| landing revision. Default is ``rev:tip``.
    :type landing_rev: str
    :param enable_statistics: Enable statistics on the |repo|, (True | False).
    :type enable_statistics: bool
    :param enable_locking: Enable |repo| locking.
    :type enable_locking: bool
    :param enable_downloads: Enable downloads from the |repo|, (True | False).
    :type enable_downloads: bool
    :param fields: Add extra fields to the |repo|. Use the following
        example format: ``field_key=field_val,field_key2=fieldval2``.
        Escape ', ' with \,
    :type fields: str
 
 

docs/api/methods/server-methods.rst

@@ -1,234 +1,268 b''
 .. _server-methods-ref:
 
 server methods
 ==============
 
 cleanup_sessions 
 ----------------
 
 .. py:function:: cleanup_sessions(apiuser, older_then=<Optional:60>)
 
    Triggers a session cleanup action.
 
    If the ``older_then`` option is set, only sessions that hasn't been
    accessed in the given number of days will be removed.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param older_then: Deletes session that hasn't been accessed
        in given number of days.
    :type older_then: Optional(int)
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result: {
        "backend": "<type of backend>",
        "sessions_removed": <number_of_removed_sessions>
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        'Error occurred during session cleanup'
      }
 
 
 get_ip 
 ------
 
 .. py:function:: get_ip(apiuser, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Displays the IP Address as seen from the |RCE| server.
 
    * This command displays the IP Address, as well as all the defined IP
      addresses for the specified user. If the ``userid`` is not set, the
      data returned is for the user calling the method.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from |authtoken|.
    :type apiuser: AuthUser
    :param userid: Sets the userid for which associated IP Address data
        is returned.
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result : {
                     "server_ip_addr": "<ip_from_clien>",
                     "user_ips": [
                                    {
                                       "ip_addr": "<ip_with_mask>",
                                       "ip_range": ["<start_ip>", "<end_ip>"],
                                    },
                                    ...
                                 ]
        }
 
 
 get_method 
 ----------
 
 .. py:function:: get_method(apiuser, pattern=<Optional:'*'>)
 
    Returns list of all available API methods. By default match pattern
    os "*" but any other pattern can be specified. eg *comment* will return
    all methods with comment inside them. If just single method is matched
    returned data will also include method specification
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param pattern: pattern to match method names against
-   :type older_then: Optional("*")
+   :type pattern: Optional("*")
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      "result": [
        "changeset_comment",
        "comment_pull_request",
        "comment_commit"
      ]
      error :  null
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      "result": [
        "comment_commit",
        {
          "apiuser": "<RequiredType>",
          "comment_type": "<Optional:u'note'>",
          "commit_id": "<RequiredType>",
          "message": "<RequiredType>",
          "repoid": "<RequiredType>",
          "request": "<RequiredType>",
          "resolves_comment_id": "<Optional:None>",
          "status": "<Optional:None>",
          "userid": "<Optional:<OptionalAttr:apiuser>>"
        }
      ]
      error :  null
 
 
 get_repo_store 
 --------------
 
 .. py:function:: get_repo_store(apiuser)
 
    Returns the |RCE| repository storage information.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'modules': [<module name>,...]
        'py_version': <python version>,
        'platform': <platform type>,
        'rhodecode_version': <rhodecode version>
      }
      error :  null
 
 
 get_server_info 
 ---------------
 
 .. py:function:: get_server_info(apiuser)
 
    Returns the |RCE| server information.
 
    This includes the running version of |RCE| and all installed
    packages. This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'modules': [<module name>,...]
        'py_version': <python version>,
        'platform': <platform type>,
        'rhodecode_version': <rhodecode version>
      }
      error :  null
 
 
 rescan_repos 
 ------------
 
 .. py:function:: rescan_repos(apiuser, remove_obsolete=<Optional:False>)
 
    Triggers a rescan of the specified repositories.
 
    * If the ``remove_obsolete`` option is set, it also deletes repositories
      that are found in the database but not on the file system, so called
      "clean zombies".
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param remove_obsolete: Deletes repositories from the database that
        are not found on the filesystem.
    :type remove_obsolete: Optional(``True`` | ``False``)
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'added': [<added repository name>,...]
        'removed': [<removed repository name>,...]
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        'Error occurred during rescan repositories action'
      }
 
 
+store_exception 
+---------------
+
+.. py:function:: store_exception(apiuser, exc_data_json, prefix=<Optional:'rhodecode'>)
+
+   Stores sent exception inside the built-in exception tracker in |RCE| server.
+
+   This command can only be run using an |authtoken| with admin rights to
+   the specified repository.
+
+   This command takes the following options:
+
+   :param apiuser: This is filled automatically from the |authtoken|.
+   :type apiuser: AuthUser
+
+   :param exc_data_json: JSON data with exception e.g
+       {"exc_traceback": "Value `1` is not allowed", "exc_type_name": "ValueError"}
+   :type exc_data_json: JSON data
+
+   :param prefix: prefix for error type, e.g 'rhodecode', 'vcsserver', 'rhodecode-tools'
+   :type prefix: Optional("rhodecode")
+
+   Example output:
+
+   .. code-block:: bash
+
+     id : <id_given_in_input>
+     "result": {
+       "exc_id": 139718459226384,
+       "exc_url": "http://localhost:8080/_admin/settings/exceptions/139718459226384"
+     }
+     error :  null
+
+

docs/auth/auth-saml-generic.rst

@@ -1,18 +1,19 b''
 .. _config-saml-generic-ref:
 
 
 SAML 2.0 Authentication
 -----------------------
 
 
 **This plugin is available only in EE Edition.**
 
 RhodeCode Supports standard SAML 2.0 SSO for the web-application part.
 
 Please check for reference two example providers:
 
 .. toctree::
 
    auth-saml-duosecurity
    auth-saml-onelogin
+   auth-saml-bulk-enroll-users
 

docs/auth/ssh-connection.rst

@@ -1,140 +1,144 b''
 .. _ssh-connection:
 
 SSH Connection
 --------------
 
 If you wish to connect to your |repos| using SSH protocol, use the
 following instructions.
 
 1. Include |RCE| generated `authorized_keys` file into your sshd_config.
 
    By default a file `authorized_keys_rhodecode` is created containing
    configuration and all allowed user connection keys are stored inside.
    On each change of stored keys inside |RCE| this file is updated with
    proper data.
 
    .. code-block:: bash
 
        # Edit sshd_config file most likely at /etc/ssh/sshd_config
        # add or edit the AuthorizedKeysFile, and set to use custom files
 
        AuthorizedKeysFile %h/.ssh/authorized_keys %h/.ssh/authorized_keys_rhodecode
 
    This way we use a separate file for SSH access and separate one for
    SSH access to |RCE| repositories.
 
 
 2. Enable the SSH module on instance.
 
    On the server where |RCE| is running executing:
 
    .. code-block:: bash
 
        rccontrol enable-module ssh {instance-id}
 
    This will add the following configuration into :file:`rhodecode.ini`.
    This also can be done manually:
 
    .. code-block:: ini
 
         ############################################################
         ### SSH Support Settings                                 ###
         ############################################################
 
         ## Defines if a custom authorized_keys file should be created and written on
         ## any change user ssh keys. Setting this to false also disables posibility
         ## of adding SSH keys by users from web interface. Super admins can still
         ## manage SSH Keys.
         ssh.generate_authorized_keyfile = true
 
         ## Options for ssh, default is `no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding`
         # ssh.authorized_keys_ssh_opts =
 
         ## Path to the authrozied_keys file where the generate entries are placed.
         ## It is possible to have multiple key files specified in `sshd_config` e.g.
         ## AuthorizedKeysFile %h/.ssh/authorized_keys %h/.ssh/authorized_keys_rhodecode
         ssh.authorized_keys_file_path = ~/.ssh/authorized_keys_rhodecode
 
         ## Command to execute the SSH wrapper. The binary is available in the
         ## rhodecode installation directory.
         ## e.g ~/.rccontrol/community-1/profile/bin/rc-ssh-wrapper
         ssh.wrapper_cmd = ~/.rccontrol/community-1/rc-ssh-wrapper
 
         ## Allow shell when executing the ssh-wrapper command
         ssh.wrapper_cmd_allow_shell = false
 
         ## Enables logging, and detailed output send back to the client during SSH
         ## operations. Useful for debugging, shouldn't be used in production.
         ssh.enable_debug_logging = false
 
         ## Paths to binary executable, by default they are the names, but we can
         ## override them if we want to use a custom one
         ssh.executable.hg = ~/.rccontrol/vcsserver-1/profile/bin/hg
         ssh.executable.git = ~/.rccontrol/vcsserver-1/profile/bin/git
         ssh.executable.svn = ~/.rccontrol/vcsserver-1/profile/bin/svnserve
 
+        ## Enables SSH key generator web interface. Disabling this still allows users
+        ## to add their own keys.
+        ssh.enable_ui_key_generator = true
+
 
 3. Set base_url for instance to enable proper event handling (Optional):
 
    If you wish to have integrations working correctly via SSH please configure
    The Application base_url.
 
    Use the ``rccontrol status`` command to view instance details.
    Hostname is required for the integration to properly set the instance URL.
 
    When your hostname is known (e.g https://code.rhodecode.com) please set it
    inside :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
 
    add into `[app:main]` section the following configuration:
 
    .. code-block:: ini
 
        app.base_url = https://code.rhodecode.com
 
 
 4. Add the public key to your user account for testing.
    First generate a new key, or use your existing one and have your public key
    at hand.
 
    Go to
    :menuselection:`My Account --> SSH Keys` and add the public key with proper description.
 
    This will generate a new entry inside our configured `authorized_keys_rhodecode` file.
 
    Test the connection from your local machine using the following example:
 
    .. note::
 
        In case of connection problems please set
        `ssh.enable_debug_logging = true` inside the SSH configuration of
        :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
        Then add, remove your SSH key and try connecting again.
        Debug logging will be printed to help find the problems on the server side.
 
    Test connection using the ssh command from the local machine. Make sure
    to use the use who is running the |RCE| server, and not your username from
    the web interface.
 
 
    For SVN:
 
    .. code-block:: bash
 
        SVN_SSH="ssh -i ~/.ssh/id_rsa_test_ssh_private.key" svn checkout svn+ssh://rhodecode@rc-server/repo_name
 
    For GIT:
 
    .. code-block:: bash
 
        GIT_SSH_COMMAND='ssh -i ~/.ssh/id_rsa_test_ssh_private.key' git clone ssh://rhodecode@rc-server/repo_name
 
    For Mercurial:
 
    .. code-block:: bash
 
        Add to hgrc:
 
        [ui]
        ssh = ssh -C -i ~/.ssh/id_rsa_test_ssh_private.key
 
        hg clone ssh://rhodecode@rc-server/repo_name

docs/contributing/dev-setup.rst

@@ -1,226 +1,243 b''
 .. _dev-setup:
 
 ===================
  Development setup
 ===================
 
 
 RhodeCode Enterprise runs inside a Nix managed environment. This ensures build
 environment dependencies are correctly declared and installed during setup.
 It also enables atomic upgrades, rollbacks, and multiple instances of RhodeCode
 Enterprise running with isolation.
 
 To set up RhodeCode Enterprise inside the Nix environment, use the following steps:
 
 
 
 Setup Nix Package Manager
 -------------------------
 
 To install the Nix Package Manager, please run::
 
    $ curl https://nixos.org/nix/install | sh
 
 or go to https://nixos.org/nix/ and follow the installation instructions.
 Once this is correctly set up on your system, you should be able to use the
 following commands:
 
 * `nix-env`
 
 * `nix-shell`
 
 
 .. tip::
 
    Update your channels frequently by running ``nix-channel --update``.
 
 .. note::
 
    To uninstall nix run the following:
 
     remove the . "$HOME/.nix-profile/etc/profile.d/nix.sh" line in your ~/.profile or ~/.bash_profile
     rm -rf $HOME/{.nix-channels,.nix-defexpr,.nix-profile,.config/nixpkgs}
     sudo rm -rf /nix
 
 Switch nix to the latest STABLE channel
 ---------------------------------------
 
 run::
 
    nix-channel --add https://nixos.org/channels/nixos-18.03 nixpkgs
 
 Followed by::
 
    nix-channel --update
+   nix-env -i nix-2.0.4
 
 
 Install required binaries
 -------------------------
 
 We need some handy tools first.
 
 run::
 
     nix-env -i nix-prefetch-hg
     nix-env -i nix-prefetch-git
 
 
+Speed up JS build by installing PhantomJS
+-----------------------------------------
+
+PhantomJS will be downloaded each time nix-shell is invoked. To speed this by
+setting already downloaded version do this::
+
+    nix-env -i phantomjs-2.1.1
+
+    # and set nix bin path
+    export PATH=$PATH:~/.nix-profile/bin
+
+
 Clone the required repositories
 -------------------------------
 
 After Nix is set up, clone the RhodeCode Enterprise Community Edition and
 RhodeCode VCSServer repositories into the same directory.
 RhodeCode currently is using Mercurial Version Control System, please make sure
 you have it installed before continuing.
 
 To obtain the required sources, use the following commands::
 
     mkdir rhodecode-develop && cd rhodecode-develop
-    hg clone https://code.rhodecode.com/rhodecode-enterprise-ce
-    hg clone https://code.rhodecode.com/rhodecode-vcsserver
+    hg clone -u default https://code.rhodecode.com/rhodecode-enterprise-ce
+    hg clone -u default https://code.rhodecode.com/rhodecode-vcsserver
 
 .. note::
 
    If you cannot clone the repository, please contact us via [e-mail]
 
 
 Install some required libraries
 -------------------------------
 
 There are some required drivers and dev libraries that we need to install to
 test RhodeCode under different types of databases. For example in Ubuntu we
 need to install the following.
 
 required libraries::
 
+    # svn related
     sudo apt-get install libapr1-dev libaprutil1-dev
     sudo apt-get install libsvn-dev
+    # libcurl required too
+    sudo apt-get install libcurl4-openssl-dev
+    # mysql/pg server for development, optional
     sudo apt-get install mysql-server libmysqlclient-dev
     sudo apt-get install postgresql postgresql-contrib libpq-dev
-    sudo apt-get install libcurl4-openssl-dev
+
 
 
 Enter the Development Shell
 ---------------------------
 
 The final step is to start the development shells. To do this, run the
 following command from inside the cloned repository::
 
-   #first, the vcsserver
+   # first, the vcsserver
    cd ~/rhodecode-vcsserver
    nix-shell
 
    # then enterprise sources
    cd ~/rhodecode-enterprise-ce
    nix-shell
 
 .. note::
 
    On the first run, this will take a while to download and optionally compile
    a few things. The following runs will be faster. The development shell works
    fine on both MacOS and Linux platforms.
 
 
 Create config.nix for development
 ---------------------------------
 
 In order to run proper tests and setup linking across projects, a config.nix
 file needs to be setup::
 
     # create config
     mkdir -p ~/.nixpkgs
     touch ~/.nixpkgs/config.nix
 
     # put the below content into the ~/.nixpkgs/config.nix file
     # adjusts, the path to where you cloned your repositories.
 
     {
       rc = {
        sources = {
         rhodecode-vcsserver = "/home/dev/rhodecode-vcsserver";
         rhodecode-enterprise-ce = "/home/dev/rhodecode-enterprise-ce";
         rhodecode-enterprise-ee = "/home/dev/rhodecode-enterprise-ee";
        };
       };
     }
 
 
 
 Creating a Development Configuration
 ------------------------------------
 
 To create a development environment for RhodeCode Enterprise,
 use the following steps:
 
 1. Create a copy of vcsserver config:
     `cp ~/rhodecode-vcsserver/configs/development.ini ~/rhodecode-vcsserver/configs/dev.ini`
 2. Create a copy of rhodocode config:
     `cp ~/rhodecode-enterprise-ce/configs/development.ini ~/rhodecode-enterprise-ce/configs/dev.ini`
 3. Adjust the configuration settings to your needs if needed.
 
 .. note::
 
   It is recommended to use the name `dev.ini` since it's included in .hgignore file.
 
 
 Setup the Development Database
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To create a development database, use the following example. This is a one
 time operation executed from the nix-shell of rhodecode-enterprise-ce sources ::
 
     rc-setup-app dev.ini \
         --user=admin --password=secret \
         --email=[e-mail] \
         --repos=~/my_dev_repos
 
 
 Compile CSS and JavaScript
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To use the application's frontend and prepare it for production deployment,
 you will need to compile the CSS and JavaScript with Grunt.
 This is easily done from within the nix-shell using the following command::
 
-    grunt
+    make web-build
 
 When developing new features you will need to recompile following any
 changes made to the CSS or JavaScript files when developing the code::
 
     grunt watch
 
 This prepares the development (with comments/whitespace) versions of files.
 
 Start the Development Servers
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 From the rhodecode-vcsserver directory, start the development server in another
 nix-shell, using the following command::
 
       pserve configs/dev.ini
 
 In the adjacent nix-shell which you created for your development server, you may
 now start CE with the following command::
 
 
       pserve --reload configs/dev.ini
 
 .. note::
 
   `--reload` flag will automatically reload the server when source file changes.
 
 
 Run the Environment Tests
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Please make sure that the tests are passing to verify that your environment is
 set up correctly. RhodeCode uses py.test to run tests.
 While your instance is running, start a new nix-shell and simply run
 ``make test`` to run the basic test suite.
 
 
 Need Help?
 ^^^^^^^^^^
 
 Join us on Slack via https://rhodecode.com/join or post questions in our
 Community Portal at https://community.rhodecode.com

docs/integrations/integrations-rcextensions.rst

@@ -1,44 +1,44 b''
 .. _integrations-rcextensions:
 
 
 rcextensions integrations
 =========================
 
 
 Since RhodeCode 4.14 release rcextensions aren't part of rhodecode-tools, and instead
 they are shipped with the new or upgraded installations.
 
 The rcextensions template `rcextensions.tmpl` is created in the `etc/` directory
 of enterprise or community installation. It's always re-created and updated on upgrades.
 
 
 Activating rcextensions
 +++++++++++++++++++++++
 
 To activate rcextensions simply copy or rename the created template rcextensions
 into the path where the rhodecode.ini file is located::
 
-    pushd ~/rccontrol/enterprise-1/
+    pushd ~/.rccontrol/enterprise-1/
     or
-    pushd ~/rccontrol/community-1/
+    pushd ~/.rccontrol/community-1/
 
-    mv etc/rcextensions.tmpl rcextensions
+    mv profile/etc/rcextensions.tmpl rcextensions
 
 
 rcextensions are loaded when |RCE| starts. So a restart is required after activation or
 change of code in rcextensions.
 
 Simply restart only the enterprise/community instance::
 
     rccontrol restart enterprise-1
     or
     rccontrol restart community-1
 
 
 Example usage
 +++++++++++++
 
 
 To see examples of usage please check the examples directory under:
 
 https://code.rhodecode.com/rhodecode-enterprise-ce/files/stable/rhodecode/config/rcextensions/examples

docs/release-notes/release-notes-4.14.0.rst

@@ -1,114 +1,114 b''
 |RCE| 4.14.0 |RNS|
 ------------------
 
 Release Date
 ^^^^^^^^^^^^
 
 - 2018-11-02
 
 
 New Features
 ^^^^^^^^^^^^
 
 - Diffs: expose range diff inside the PR view. It's now possible to show
   commit-per-commit view of changes in pull request.
 - Diffs: new sticky context bar.
   When browsing diffs we show file path of the current diff so users are aware all the time
   what file they are reviewing.
 - Diffs: added quick file selector in diffs views. Now it's possible to select a file
   in large diffs from the sticky header for quicker access to certain interesting files
   in diffs.
 - Diffs: introducing diff menu for whitespace toggle and context changes.
   It's now possible to show/hide whitespace changes and toggle the file context in
   all diff places including pull requests.
 - Comments: allow commenting on empty files without content.
 - Repositories: added option to archive repositories instead of deleting them.
   Archived repositories are useful for future auditing, but they are read-only.
 - rcextensions: new rcextensions. We're introducing new `rcextensions` that will be base
   for future low-level integrations. It's now possible to expose nice messages back
   to the users when using `rcextensions`.
 - Summary page: slightly re-organize summary page for better user experience.
 
 
 General
 ^^^^^^^
 
 - Mailing: switched from custom library to pyramid_mailer with python3 compatibility.
 - Frontend: Switched to Polymer 3.0.
 - Frontend: fixed problems with IE11 and brought back support for that browser.
 - Git: use a fetch_sync based creation of remote repositories.
   This fixes problems with importing from Bitbucket.
 - Comments: update comments email templates.
 - Packaging: only wrap external dependency scripts. This makes execution of scripts
   roughly 5x faster due to much smaller PATH tree.
 - HTTP: use application wide detection of invalid bytes sent via URL/GET/POST data.
 - Fonts/UI: use consistent fonts across the whole application.
   Few places had non-standard custom fonts.
 - Google: updated google auth plugin with latest API changes.
 - Core: handle edge case requesting matched routes but with hg/svn/git or api context.
 - Dependencies: bumped rhodecode-tools to 1.0.0 release using Apache2 license.
 - Dependencies: atomicwrites==1.2.1
 - Dependencies: attrs==18.2.0
 - Dependencies: dogpile.cache==0.6.7
 - Dependencies: psutil==5.4.7
 - Dependencies: pathlib2==2.3.2
 - Dependencies: subprocess32==3.5.2
 - Dependencies: gevent==1.3.6
 - Dependencies: greenlet==0.4.15
 - Dependencies: pytest==3.8.2
 - Dependencies: py==1.6.0
 - Dependencies: pytest-cov==2.6.0
 - Dependencies: pytest-timeout==1.3.2
 - Dependencies: coverage==4.5.1
 - Dependencies: psycopg2==2.7.5
 
 
 Security
 ^^^^^^^^
 
 - RST: improve Javascript RST sandbox.
 - Jupyter: sanitize markdown cells similar as we do for our own markdown cleanup.
 
 
 Performance
 ^^^^^^^^^^^
 
 - SSH: improved SSH wrapper execution speed by using optimized binary script wrapping.
 - Core: reduced font and JavaScript load times.
 
 
 Fixes
 ^^^^^
 
 - Comments: ensure we always display unmatched comments.
 - Branch Permissions: fixed changing rule order for branch permissions.
 - Users: ensure get_first_superadmin actually gets the 1st created super-admin.
 - Users: when deleting users ensure we also clear personal flag.
   This fixes some problems with multiple personal groups.
 - Diffs: disable the error border on highlight errors.
 - Integrations: implement retry to HTTP[S] calls for integrations.
   Web parts will do a 3x retry call in case service is not reachable or
   responds with 5XX codes.
 - Git: fixed pull-request updates in case branch names are the same as the file names.
 - Supervisor: add patch for older kernel support.
 - Compare: fixed file after/before links in compare view for cross repo compare.
 - Emails: improve fonts and rendering of email HTML.
 - Permissions: flush members of user groups permissions to clear caches.
 - Repository: add check preventing of removal of repo with attached pull requests. Users
   should use the new archive repo function instead.
 
 
 Upgrade notes
 ^^^^^^^^^^^^^
 
 - In this release, we're shipping a new `rcextensions`. The changes made are
   backward incompatible. An update of `rcextensions` is required
   prior to using them again. Please check the new `rcextensions.tmpl` directory
-  located in `etc/rcextensions.tmpl` in your instance installation path.
+  located in `profile/etc/rcextensions.tmpl` in your instance installation path.
   Old code should be 100% portable by just copy&paste to the right function.
 
 - Mailing: We introduced a new mailing library. The older options should be compatible and
   generally, old configuration doesn't need any changes in order to send emails.
   We, however, encourage users to re-check mailing setup in case of some more
   sophisticated email setups.
   There's a possibility to send a test email from admin > settings > email section.

docs/release-notes/release-notes.rst

@@ -1,128 +1,129 b''
 .. _rhodecode-release-notes-ref:
 
 Release Notes
 =============
 
 |RCE| 4.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
+   release-notes-4.16.0.rst
    release-notes-4.15.2.rst
    release-notes-4.15.1.rst
    release-notes-4.15.0.rst
    release-notes-4.14.1.rst
    release-notes-4.14.0.rst
    release-notes-4.13.3.rst
    release-notes-4.13.2.rst
    release-notes-4.13.1.rst
    release-notes-4.13.0.rst
    release-notes-4.12.4.rst
    release-notes-4.12.3.rst
    release-notes-4.12.2.rst
    release-notes-4.12.1.rst
    release-notes-4.12.0.rst
    release-notes-4.11.6.rst
    release-notes-4.11.5.rst
    release-notes-4.11.4.rst
    release-notes-4.11.3.rst
    release-notes-4.11.2.rst
    release-notes-4.11.1.rst
    release-notes-4.11.0.rst
    release-notes-4.10.6.rst
    release-notes-4.10.5.rst
    release-notes-4.10.4.rst
    release-notes-4.10.3.rst
    release-notes-4.10.2.rst
    release-notes-4.10.1.rst
    release-notes-4.10.0.rst
    release-notes-4.9.1.rst
    release-notes-4.9.0.rst
    release-notes-4.8.0.rst
    release-notes-4.7.2.rst
    release-notes-4.7.1.rst
    release-notes-4.7.0.rst
    release-notes-4.6.1.rst
    release-notes-4.6.0.rst
    release-notes-4.5.2.rst
    release-notes-4.5.1.rst
    release-notes-4.5.0.rst
    release-notes-4.4.2.rst
    release-notes-4.4.1.rst
    release-notes-4.4.0.rst
    release-notes-4.3.1.rst
    release-notes-4.3.0.rst
    release-notes-4.2.1.rst
    release-notes-4.2.0.rst
    release-notes-4.1.2.rst
    release-notes-4.1.1.rst
    release-notes-4.1.0.rst
    release-notes-4.0.1.rst
    release-notes-4.0.0.rst
 
 |RCE| 3.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-3.8.4.rst
    release-notes-3.8.3.rst
    release-notes-3.8.2.rst
    release-notes-3.8.1.rst
    release-notes-3.8.0.rst
    release-notes-3.7.1.rst
    release-notes-3.7.0.rst
    release-notes-3.6.1.rst
    release-notes-3.6.0.rst
    release-notes-3.5.2.rst
    release-notes-3.5.1.rst
    release-notes-3.5.0.rst
    release-notes-3.4.1.rst
    release-notes-3.4.0.rst
    release-notes-3.3.4.rst
    release-notes-3.3.3.rst
    release-notes-3.3.2.rst
    release-notes-3.3.1.rst
    release-notes-3.3.0.rst
    release-notes-3.2.3.rst
    release-notes-3.2.2.rst
    release-notes-3.2.1.rst
    release-notes-3.2.0.rst
    release-notes-3.1.1.rst
    release-notes-3.1.0.rst
    release-notes-3.0.2.rst
    release-notes-3.0.1.rst
    release-notes-3.0.0.rst
 
 |RCE| 2.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-2.2.8.rst
    release-notes-2.2.7.rst
    release-notes-2.2.6.rst
    release-notes-2.2.5.rst
    release-notes-2.2.4.rst
    release-notes-2.2.3.rst
    release-notes-2.2.2.rst
    release-notes-2.2.1.rst
    release-notes-2.2.0.rst
    release-notes-2.1.0.rst
    release-notes-2.0.2.rst
    release-notes-2.0.1.rst
    release-notes-2.0.0.rst
 
 |RCE| 1.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-1.7.2.rst
    release-notes-1.7.1.rst
    release-notes-1.7.0.rst
    release-notes-1.6.0.rst

docs/tools/tools-cli.rst

@@ -1,578 +1,578 b''
 .. _tools-cli:
 
 |RCT| CLI
 ---------
 
 The commands available with |RCT| can be split into three categories:
 
 - Remotely executable commands that can be run from your local machine once you
   have your connection details to |RCE| configured.
 - Locally executable commands the can be run on the server to carry out
   general maintenance.
 - Local configuration commands used to help set up your |RCT| configuration.
 
 
 rhodecode-tools
 ---------------
 
 Use |RCT| to setup automation, run the indexer, and install extensions for
 your |RCE| instances. Options:
 
 .. rst-class:: dl-horizontal
 
     \ - -apihost <api_host>
         Set the API host value.
 
     \ - -apikey <apikey_value>
         Set the API key value.
 
     \-c, - -config <config_file>
         Create a configuration file. The default file is created
         in ``~/.rhoderc``
 
     \ - -save-config
         Save the configuration file.
 
     \ - -show-config
         Show the current configuration values.
 
     \ - -format {json,pretty}
         Set the formatted representation.
 
 Example usage:
 
 .. code-block:: bash
 
    $ rhodecode-tools --apikey=key --apihost=http://rhodecode.server \
        --save-config
 
 rhodecode-api
 -------------
 
 The RhodeCode API lets you connect to |RCE| and carry out management tasks from a
 remote machine, for more information about the API, see the :ref:`api`. To
 pass arguments on the command-line use the ``method:option`` syntax.
 
 Example usage:
 
 .. code-block:: bash
 
     # Run the get_repos API call and sample output
     $ rhodecode-api --instance-name=enterprise-1 create_repo \
         repo_name:brand-new repo_type:hg description:repo-desc
 
     {
       "error": null,
       "id": 1110,
       "result": {
         "msg": "Created new repository `brand-new`",
         "success": true,
         "task": null
       }
     }
 
 Options:
 
 .. rst-class:: dl-horizontal
 
     \ - -api-cache-only
         Requires a cache to be present when running this call
 
     \ - -api-cache-rebuild
         Replaces existing cached values with new ones from server
 
     \ - -api-cache <PATH>
         Use a special cache dir to read responses from instead of the server
 
     \ - -api-cert-verify
          Verify the endpoint ssl certificate
 
     \ - -api-cert <PATH>
          Path to alternate CA bundle.
 
     \ - -apihost <api_host>
         Set the API host value.
 
     \ - -apikey <apikey_value>
         Set the API key value.
 
     \ - -instance-name <instance-id>
         Set the instance name
 
     \-I, - -install-dir <DIR>
         Location of application instances
 
     \-c, - -config <.rhoderc-file>
         Location of the :file:`.rhoderc`
 
     \-F, - -format {json,pretty}
         Set the formatted representation.
 
     \-h, - -help
         Show help messages.
 
     \-v, - -verbose
         Enable verbose messaging
 
 rhodecode-cleanup-gists
 -----------------------
 
 Use this to delete gists within |RCE|. Options:
 
 .. rst-class:: dl-horizontal
 
     \-c, - -config <config_file>
         Set the file path to the configuration file. The default file is
         :file:`/home/{user}/.rhoderc`
 
     \ - -corrupted
         Remove gists with corrupted metadata.
 
     \ - -dont-ask
         Remove gists without asking for confirmation.
 
     \-h, - -help
         Show help messages. current configuration values.
 
     \ - -instance-name <instance-id>
         Set the instance name.
 
     \-R, - -repo-dir
         Set the repository file path.
 
     \ - -version
         Display your |RCT| version.
 
 Example usage:
 
 .. code-block:: bash
 
     # Clean up gists related to an instance
     $ rhodecode-cleanup-gists --instance-name=enterprise-1
     Scanning for gists in /home/brian/repos/.rc_gist_store...
     preparing to remove [3] found gists
 
     # Clean up corrupted gists in an instance
     $ rhodecode-cleanup-gists --instance-name=enterprise-1 --corrupted
     Scanning for gists in /home/brian/repos/.rc_gist_store...
     preparing to remove [2] found gists
     the following gists will be archived:
       * EXPIRED: BAD METADATA        | /home/brian/repos/.rc_gist_store/5
       * EXPIRED: BAD METADATA        | /home/brian/repos/.rc_gist_store/8FtC
     are you sure you want to archive them? [y/N]: y
     removing gist /home/brian/repos/.rc_gist_store/5
     removing gist /home/brian/repos/.rc_gist_store/8FtCKdcbRKmEvRzTVsEt
 
 rhodecode-cleanup-repos
 -----------------------
 
 Use this to manage |repos| and |repo| groups within |RCE|. Options:
 
 .. rst-class:: dl-horizontal
 
     \-c, - -config <config_file>
         Set the file path to the configuration file. The default file is
         :file:`/home/{user}/.rhoderc`.
 
     \-h, - -help
         Show help messages. current configuration values.
 
     \ - -interactive
         Enable an interactive prompt for each repository when deleting.
 
     \ - -include-groups
         Remove repository groups.
 
     \ - -instance-name <instance-id>
         Set the instance name.
 
     \ - -list-only
         Display repositories selected for deletion.
 
     \ - -older-than <str>
         Delete repositories older that a specified time.
         You can use the following suffixes; d for days, h for hours,
         m for minutes, s for seconds.
 
     \-R, - -repo-dir
         Set the repository file path
 
 Example usage:
 
 .. code-block:: bash
 
     # Cleaning up repos using tools installed with RCE 350 and above
     $ ~/.rccontrol/enterprise-4/profile/bin/rhodecode-cleanup-repos \
         --instance-name=enterprise-4 --older-than=1d
     Scanning for repositories in /home/brian/repos...
     preparing to remove [2] found repositories older than 1 day, 0:00:00 (1d)
 
     the following repositories will be deleted completely:
     * REMOVED: 2015-08-05 00:23:18 | /home/brian/repos/rm__20150805_002318_831
     * REMOVED: 2015-08-04 01:22:10 | /home/brian/repos/rm__20150804_012210_336
     are you sure you want to remove them? [y/N]:
 
     # Clean up repos older than 1 year
     # If using virtualenv and pre RCE 350 tools installation
     (venv)$ rhodecode-cleanup-repos --instance-name=enterprise-1 \
         --older-than=365d
 
     Scanning for repositories in /home/brian/repos...
     preparing to remove [343] found repositories older than 365 days
 
     # clean up repos older than 3 days
     # If using virtualenv and pre RCE 350 tools installation
     (venv)$ rhodecode-cleanup-repos --instance-name=enterprise-1 \
         --older-than=3d
     Scanning for repositories in /home/brian/repos...
     preparing to remove [3] found repositories older than 3 days
 
 .. _tools-config:
 
 rhodecode-config
 ----------------
 
 Use this to create or update a |RCE| configuration file on the local machine.
 
 .. rst-class:: dl-horizontal
 
     \- -filename </path/to/config_file>
         Set the file path to the |RCE| configuration file.
 
     \- -show-defaults
         Display the defaults set in the |RCE| configuration file.
 
     \- -update
         Update the configuration with the new settings passed on the command
         line.
 
 .. code-block:: bash
 
     # Create a new config file
     $ rhodecode-config --filename=dev.ini
     Wrote new config file in /Users/user/dev.ini
 
     # Update config value for given section:
     $ rhodecode-config --update --filename=prod.ini [handler_console]level=INFO
 
     $ rhodecode-config --filename=dev.ini --show-defaults
     lang=en
     cpu_number=4
     uuid=<function <lambda> at 0x10d86ac08>
     license_token=ff1e-aa9c-bb66-11e5
     host=127.0.0.1
     here=/Users/brian
     error_aggregation_service=None
     database_url=sqlite:///%(here)s/rhodecode.db?timeout=30
     git_path=git
     http_server=waitress
     port=5000
 
 .. _tools-rhodecode-extensions:
 
 rhodecode-extensions
 --------------------
 
 The `rcextensions` since version 4.14 are now shipped together with |RCE| please check
 the using :ref:`integrations-rcextensions` section.
 
 
 rhodecode-gist
 --------------
 
 Use this to create, list, show, or delete gists within |RCE|. Options:
 
 .. rst-class:: dl-horizontal
 
     \ - -api-cache-only
         Requires a cache to be present when running this call
 
     \ - -api-cache-rebuild
         Replaces existing cached values with new ones from server
 
     \ - -api-cache PATH
         Use a special cache dir to read responses from instead of the server
 
     \ - -api-cert-verify
          Verify the endpoint ssl certificate
 
     \ - -api-cert PATH
          Path to alternate CA bundle.
 
     \ - -apihost <api_host>
         Set the API host value.
 
     \ - -apikey <apikey_value>
         Set the API key value.
 
     \-c, - -config <config_file>
         Create a configuration file.
         The default file is created in :file:`~/.rhoderc`
 
     \ - -create <gistname>
         create the gist
 
     \-d, - -description <str>
         Set gist description
 
     \ - -delete <gistid>
         Delete the gist
 
     \-f, - -file
         Specify the filename The file extension will enable syntax highlighting.
 
     \-F, - -format {json,pretty}
         Set the formatted representation.
 
     \ - -help
         Show help messages.
 
     \-I, - -install-dir <DIR>
         Location of application instances
 
     \ - -instance-name <instance-id>
         Set the instance name.
 
     \ - -list
         Display instance gists.
 
     \-l, --lifetime <minutes>
         Set the gist lifetime. The default value is (-1) forever
 
     \ - -show <gistname>
         Show the content of the gist
 
     \-o, - -open
         After creating Gist open it in browser
 
     \-p, - -private
         Create a private gist
 
     \ - -version
          Display your |RCT| version.
 
 Example usage:
 
 .. code-block:: bash
 
     # List the gists in an instance
     (venv)brian@ubuntu:~$ rhodecode-gist --instance-name=enterprise-1 list
     {
       "error": null,
       "id": 7102,
       "result": [
         {
           "access_id": "2",
           "content": null,
           "created_on": "2015-01-19T12:52:26.494",
           "description": "A public gust",
           "expires": -1.0,
           "gist_id": 2,
           "type": "public",
           "url": "http://127.0.0.1:10003/_admin/gists/2"
         },
         {
           "access_id": "7gs6BsSEC4pKUEPLz5AB",
           "content": null,
           "created_on": "2015-01-19T11:27:40.812",
           "description": "Gist testing API",
           "expires": -1.0,
           "gist_id": 1,
           "type": "private",
           "url": "http://127.0.0.1:10003/_admin/gists/7gs6BsSEC4pKUEPLz5AB"
         }
       ]
     }
 
     # delete a particular gist
     # You use the access_id to specify the gist to delete
     (venv)brian@ubuntu:~$ rhodecode-gist delete 2  --instance-name=enterprise-1
     {
       "error": null,
       "id": 6284,
       "result": {
         "gist": null,
         "msg": "deleted gist ID:2"
       }
     }
 
     # cat a file and pipe to new gist
     # This is if you are using virtualenv
     (venv)$ cat ~/.rhoderc | rhodecode-gist --instance-name=enterprise-1 \
         -d '.rhoderc copy' create
 
     {
       "error": null,
       "id": 5374,
       "result": {
         "gist": {
           "access_id": "7",
           "content": null,
           "created_on": "2015-01-26T11:31:58.774",
           "description": ".rhoderc copy",
           "expires": -1.0,
           "gist_id": 7,
           "type": "public",
           "url": "http://127.0.0.1:10003/_admin/gists/7"
         },
         "msg": "created new gist"
       }
     }
 
     # Cat a file and pipe to gist
     # in RCE 3.5.0 tools and above
     $ cat ~/.rhoderc | ~/.rccontrol/{instance-id}/profile/bin/rhodecode-gist \
        --instance-name=enterprise-4 -d '.rhoderc copy' create
     {
       "error": null,
       "id": 9253,
       "result": {
         "gist": {
           "access_id": "4",
           "acl_level": "acl_public",
           "content": null,
           "created_on": "2015-08-20T05:54:11.250",
           "description": ".rhoderc copy",
           "expires": -1.0,
           "gist_id": 4,
           "modified_at": "2015-08-20T05:54:11.250",
           "type": "public",
           "url": "http://127.0.0.1:10000/_admin/gists/4"
         },
         "msg": "created new gist"
       }
     }
 
 
 rhodecode-index
 ---------------
 
 More detailed information regarding setting up the indexer is available in
 the :ref:`indexing-ref` section. Options:
 
 .. rst-class:: dl-horizontal
 
     \ - -api-cache-only
         Requires a cache to be present when running this call
 
     \ - -api-cache-rebuild
         Replaces existing cached values with new ones from server
 
     \ - -api-cache PATH
         Use a special cache dir to read responses from instead of the server
 
     \ - -api-cert-verify
          Verify the endpoint ssl certificate
 
     \ - -api-cert PATH
          Path to alternate CA bundle.
 
     \ - -apihost <api_host>
         Set the API host value.
 
     \ - -apikey <apikey_value>
         Set the API key value.
 
     \-c, --config <config_file>
         Create a configuration file.
         The default file is created in :file:`~/.rhoderc`
 
     \ - -create-mapping <PATH>
          Creates an example mapping configuration for indexer.
 
     \-F, - -format {json,pretty}
          Set the formatted representation.
 
     \-h, - -help
          Show help messages.
 
     \ - -instance-name <instance-id>
         Set the instance name
 
     \-I, - -install-dir <DIR>
         Location of application instances
 
     \-m, - -mapping <file_name>
          Parse the output to the .ini mapping file.
 
     \ - -optimize
          Optimize index for performance by amalgamating multiple index files
          into one. Greatly increases incremental indexing speed.
 
     \-R, - -repo-dir <DIRECTORY>
          Location of repositories
 
     \ - -source <PATH>
          Use a special source JSON file to feed the indexer
 
     \ - -version
          Display your |RCT| version.
 
 Example usage:
 
 .. code-block:: bash
 
     # Run the indexer
     $ ~/.rccontrol/enterprise-4/profile/bin/rhodecode-index \
         --instance-name=enterprise-4
 
-    # Run indexer based on mapping.ini file
+    # Run indexer based on search_mapping.ini file
     # This is using pre-350 virtualenv
     (venv)$ rhodecode-index --instance-name=enterprise-1
 
     # Index from the command line without creating
     # the .rhoderc file
     $ rhodecode-index --apikey=key --apihost=http://rhodecode.server \
         --instance-name=enterprise-2 --save-config
 
     # Create the indexing mapping file
     $ ~/.rccontrol/enterprise-4/profile/bin/rhodecode-index \
-        --create-mapping mapping.ini --instance-name=enterprise-4
+        --create-mapping search_mapping.ini --instance-name=enterprise-4
 
 .. _tools-rhodecode-list-instance:
 
 rhodecode-list-instances
 ------------------------
 
 Use this command to list the instance details configured in the
 :file:`~/.rhoderc` file.
 
 .. code-block:: bash
 
    $ .rccontrol/enterprise-1/profile/bin/rhodecode-list-instances
    [instance:production] - Config only
    API-HOST: https://some.url.com
    API-KEY:  some.auth.token
 
    [instance:development] - Config only
    API-HOST: http://some.ip.address
    API-KEY:  some.auth.token
 
 
 .. _tools-setup-config:
 
 rhodecode-setup-config
 ----------------------
 
 Use this command to create the ``~.rhoderc`` file required by |RCT| to access
 remote instances.
 
 .. rst-class:: dl-horizontal
 
     \- -instance-name <name>
         Specify the instance name in the :file:`~/.rhoderc`
 
     \api_host <hostname>
         Create a configuration file. The default file is created
         in ``~/.rhoderc``
 
     \api_key <auth-token>
         Create a configuration file. The default file is created
         in ``~/.rhoderc``
 
 
 .. code-block:: bash
 
     (venv)$ rhodecode-setup-config --instance-name=tea api_host=URL api_key=xyz
     Config not found under /Users/username/.rhoderc, creating a new one
     Wrote new configuration into /Users/username/.rhoderc

grunt_config.json

@@ -1,174 +1,174 b''
 {
   "dirs": {
     "css": {
       "src": "rhodecode/public/css",
       "dest": "rhodecode/public/css"
     },
     "js": {
       "src": "rhodecode/public/js/src",
       "src_rc": "rhodecode/public/js/rhodecode",
       "dest": "rhodecode/public/js",
       "node_modules": "node_modules"
     }
   },
   "copy": {
     "main": {
       "files": [
         {
           "expand": true,
           "cwd": "node_modules/@webcomponents",
           "src": "webcomponentsjs/*.*",
           "dest": "<%= dirs.js.dest %>/vendors"
         },
         {
           "src": "<%= dirs.css.src %>/style-polymer.css",
           "dest": "<%= dirs.js.dest %>/src/components/style-polymer.css"
         }
       ]
     }
   },
   "concat": {
     "dist": {
       "src": [
         "<%= dirs.js.node_modules %>/jquery/dist/jquery.min.js",
         "<%= dirs.js.node_modules %>/mousetrap/mousetrap.min.js",
         "<%= dirs.js.node_modules %>/moment/min/moment.min.js",
         "<%= dirs.js.node_modules %>/clipboard/dist/clipboard.min.js",
         "<%= dirs.js.node_modules %>/favico.js/favico-0.3.10.min.js",
         "<%= dirs.js.node_modules %>/sticky-sidebar/dist/sticky-sidebar.min.js",
         "<%= dirs.js.node_modules %>/sticky-sidebar/dist/jquery.sticky-sidebar.min.js",
         "<%= dirs.js.node_modules %>/waypoints/lib/noframework.waypoints.min.js",
         "<%= dirs.js.node_modules %>/waypoints/lib/jquery.waypoints.min.js",
         "<%= dirs.js.node_modules %>/appenlight-client/appenlight-client.min.js",
         "<%= dirs.js.src %>/logging.js",
         "<%= dirs.js.src %>/bootstrap.js",
         "<%= dirs.js.src %>/i18n_utils.js",
         "<%= dirs.js.src %>/deform.js",
         "<%= dirs.js.src %>/ejs.js",
         "<%= dirs.js.src %>/ejs_templates/utils.js",
         "<%= dirs.js.src %>/plugins/jquery.pjax.js",
         "<%= dirs.js.src %>/plugins/jquery.dataTables.js",
         "<%= dirs.js.src %>/plugins/flavoured_checkbox.js",
         "<%= dirs.js.src %>/plugins/jquery.auto-grow-input.js",
         "<%= dirs.js.src %>/plugins/jquery.autocomplete.js",
         "<%= dirs.js.src %>/plugins/jquery.debounce.js",
-        "<%= dirs.js.src %>/plugins/jquery.mark.js",
+        "<%= dirs.js.node_modules %>/mark.js/dist/jquery.mark.min.js",
         "<%= dirs.js.src %>/plugins/jquery.timeago.js",
         "<%= dirs.js.src %>/plugins/jquery.timeago-extension.js",
         "<%= dirs.js.src %>/select2/select2.js",
         "<%= dirs.js.src %>/codemirror/codemirror.js",
         "<%= dirs.js.src %>/codemirror/codemirror_loadmode.js",
         "<%= dirs.js.src %>/codemirror/codemirror_hint.js",
         "<%= dirs.js.src %>/codemirror/codemirror_overlay.js",
         "<%= dirs.js.src %>/codemirror/codemirror_placeholder.js",
         "<%= dirs.js.src %>/codemirror/codemirror_simplemode.js",
         "<%= dirs.js.dest %>/mode/meta.js",
         "<%= dirs.js.dest %>/mode/meta_ext.js",
         "<%= dirs.js.src_rc %>/i18n/select2/translations.js",
         "<%= dirs.js.src %>/rhodecode/utils/array.js",
         "<%= dirs.js.src %>/rhodecode/utils/string.js",
         "<%= dirs.js.src %>/rhodecode/utils/pyroutes.js",
         "<%= dirs.js.src %>/rhodecode/utils/ajax.js",
         "<%= dirs.js.src %>/rhodecode/utils/autocomplete.js",
         "<%= dirs.js.src %>/rhodecode/utils/colorgenerator.js",
         "<%= dirs.js.src %>/rhodecode/utils/ie.js",
         "<%= dirs.js.src %>/rhodecode/utils/os.js",
         "<%= dirs.js.src %>/rhodecode/utils/topics.js",
         "<%= dirs.js.src %>/rhodecode/init.js",
         "<%= dirs.js.src %>/rhodecode/changelog.js",
         "<%= dirs.js.src %>/rhodecode/codemirror.js",
         "<%= dirs.js.src %>/rhodecode/comments.js",
         "<%= dirs.js.src %>/rhodecode/constants.js",
         "<%= dirs.js.src %>/rhodecode/files.js",
         "<%= dirs.js.src %>/rhodecode/followers.js",
         "<%= dirs.js.src %>/rhodecode/menus.js",
         "<%= dirs.js.src %>/rhodecode/notifications.js",
         "<%= dirs.js.src %>/rhodecode/permissions.js",
         "<%= dirs.js.src %>/rhodecode/pjax.js",
         "<%= dirs.js.src %>/rhodecode/pullrequests.js",
         "<%= dirs.js.src %>/rhodecode/settings.js",
         "<%= dirs.js.src %>/rhodecode/select2_widgets.js",
         "<%= dirs.js.src %>/rhodecode/tooltips.js",
         "<%= dirs.js.src %>/rhodecode/users.js",
         "<%= dirs.js.src %>/rhodecode/appenlight.js",
         "<%= dirs.js.src %>/rhodecode.js",
         "<%= dirs.js.dest %>/rhodecode-components.js"
       ],
       "dest": "<%= dirs.js.dest %>/scripts.js",
       "nonull": true
     }
   },
   "less": {
     "development": {
       "options": {
         "compress": false,
         "yuicompress": false,
         "optimization": 0
       },
       "files": {
         "<%= dirs.css.dest %>/style.css": "<%= dirs.css.src %>/main.less",
         "<%= dirs.css.dest %>/style-polymer.css": "<%= dirs.css.src %>/polymer.less"
       }
     },
     "production": {
       "options": {
         "compress": true,
         "yuicompress": true,
         "optimization": 2
       },
       "files": {
         "<%= dirs.css.dest %>/style.css": "<%= dirs.css.src %>/main.less",
         "<%= dirs.css.dest %>/style-polymer.css": "<%= dirs.css.src %>/polymer.less"
       }
     },
     "components": {
       "files": [
         {
           "cwd": "<%= dirs.js.src %>/components/",
           "dest": "<%= dirs.js.src %>/components/",
           "src": [
             "**/*.less"
           ],
           "expand": true,
           "ext": ".css"
         }
       ]
     }
   },
   "watch": {
     "less": {
       "files": [
         "<%= dirs.css.src %>/**/*.less",
         "<%= dirs.js.src %>/components/**/*.less"
       ],
       "tasks": [
         "less:development",
         "less:components",
         "concat:polymercss",
         "webpack",
         "concat:dist"
       ]
     },
     "js": {
       "files": [
         "!<%= dirs.js.src %>/components/root-styles.gen.html",
         "<%= dirs.js.src %>/**/*.js",
         "<%= dirs.js.src %>/components/**/*.html"
       ],
       "tasks": [
         "less:components",
         "concat:polymercss",
         "webpack",
         "concat:dist"
       ]
     }
   },
   "jshint": {
     "rhodecode": {
       "src": "<%= dirs.js.src %>/rhodecode/**/*.js",
       "options": {
         "jshintrc": ".jshintrc"
       }
     }
   }
 }

package.json

@@ -1,58 +1,59 b''
 {
   "name": "rhodecode-enterprise",
   "version": "2.0.0",
   "private": true,
   "description" : "RhodeCode JS packaged",
   "license": "SEE LICENSE IN LICENSE.txt",
   "repository" : {
     "type" : "hg",
     "url" : "https://code.rhodecode.com/rhodecode-enterprise-ce"
   },
   "devDependencies": {
     "appenlight-client": "git+https://[e-mail]/AppEnlight/appenlight-client-js.git#0.5.1",
     "clipboard": "^2.0.1",
     "exports-loader": "^0.6.4",
     "favico.js": "^0.3.10",
     "grunt": "^0.4.5",
     "grunt-cli": "^1.3.1",
     "grunt-contrib-concat": "^0.5.1",
     "grunt-contrib-copy": "^1.0.0",
     "grunt-contrib-jshint": "^0.12.0",
     "grunt-contrib-less": "^1.1.0",
     "grunt-contrib-watch": "^0.6.1",
     "grunt-webpack": "^3.1.3",
     "jquery": "1.11.3",
+    "mark.js": "8.11.1",
     "jshint": "^2.9.1-rc3",
     "moment": "^2.18.1",
     "mousetrap": "^1.6.1",
     "qrious": "^4.0.2",
     "sticky-sidebar": "3.3.1",
     "waypoints": "4.0.1",
     "webpack": "4.23.1",
     "webpack-cli": "3.1.2",
     "babel-core": "^6.26.3",
     "babel-loader": "^7.1.2",
     "babel-plugin-transform-object-rest-spread": "^6.26.0",
     "babel-preset-env": "^1.6.0",
     "copy-webpack-plugin": "^4.4.2",
     "css-loader": "^0.28.11",
     "html-loader": "^0.4.4",
     "html-webpack-plugin": "^3.2.0",
     "imports-loader": "^0.7.1",
     "polymer-webpack-loader": "^2.0.1",
     "style-loader": "^0.21.0",
     "webpack-uglify-js-plugin": "^1.1.9",
     "raw-loader": "1.0.0-beta.0",
     "ts-loader": "^1.3.3",
     "@webcomponents/webcomponentsjs": "^2.0.0",
     "@polymer/polymer": "^3.0.0",
     "@polymer/paper-button": "^3.0.0",
     "@polymer/paper-spinner": "^3.0.0",
     "@polymer/paper-tooltip": "^3.0.0",
     "@polymer/paper-toast": "^3.0.0",
     "@polymer/paper-toggle-button": "^3.0.0",
     "@polymer/iron-ajax": "^3.0.0",
     "@polymer/iron-autogrow-textarea": "^3.0.0",
     "@polymer/iron-a11y-keys": "^3.0.0"
   }
 }