release: Merge default into stable for release preparation

docs/admin/repo-admin.rst

@@ -0,0 +1,28 b''
+.. _repo-admin-set:
+.. _permissions-info-add-group-ref:
+
+Repository Administration
+=========================
+
+Repository permissions in |RCE| can be managed in a number of different ways.
+This overview should give you an insight into how you could adopt particular
+settings for your needs:
+
+* Global |repo| permissions: This allows you to set the default permissions
+  for each new |repo| created within |RCE|, see :ref:`repo-default-ref`. All
+  |repos| created will inherit these permissions unless explicitly configured.
+* Individual |repo| permissions: To set individual |repo| permissions,
+  see :ref:`set-repo-perms`.
+* Repository Group permissions: This allows you to define the permissions for
+  a group, and all |repos| created within that group will inherit the same
+  permissions.
+
+.. toctree::
+
+    repo_admin/repo-perm-steps
+    repo_admin/repo-extra-fields
+    repo_admin/repo-hooks
+    repo_admin/repo-issue-tracker
+    repo_admin/repo-vcs
+    repo_admin/restore-deleted-repositories
+    repo_admin/repo-admin-tasks
  No newline at end of file

docs/admin/repo_admin/repo-admin-tasks.rst

@@ -0,0 +1,24 b''
+.. _repo-admin-tasks:
+
+Common Admin Tasks for Repositories
+-----------------------------------
+
+
+Manually Force Delete Repository
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In case of attached forks or pull-requests repositories should be archived.
+Here is how to force delete a repository and remove all dependent objects
+
+
+.. code-block:: bash
+
+    # starts the ishell interactive prompt
+    $ rccontrol ishell enterprise-1
+
+.. code-block:: python
+
+    In [4]: from rhodecode.model.repo import RepoModel
+    In [3]: repo = Repository.get_by_repo_name('test_repos/repo_with_prs')
+    In [5]: RepoModel().delete(repo, forks='detach', pull_requests='delete')
+    In [6]: Session().commit()

docs/admin/user_admin/user-admin-tasks.rst

@@ -0,0 +1,24 b''
+.. _user-admin-tasks:
+
+Common Admin Tasks for Users
+----------------------------
+
+
+Manually Set Personal Repository Group
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Here is how to set a repository group as personal for a user using ishell.
+
+
+.. code-block:: bash
+
+    # starts the ishell interactive prompt
+    $ rccontrol ishell enterprise-1
+
+.. code-block:: python
+
+    In [1]: repo_group = RepoGroup.get_by_group_name('some_group_name')
+    In [2]: user = User.get_by_username('some_user')
+    In [3]: repo_group.user = user
+    In [4]: repo_group.personal = True
+    In [5]: Session().add(repo_group);Session().commit()

docs/api/methods/search-methods.rst

@@ -0,0 +1,35 b''
+.. _search-methods-ref:
+
+search methods
+==============
+
+search 
+------
+
+.. py:function:: search(apiuser, search_query, search_type, page_limit=<Optional:10>, page=<Optional:1>, search_sort=<Optional:'newfirst'>, repo_name=<Optional:None>, repo_group_name=<Optional:None>)
+
+   Fetch Full Text Search results using API.
+
+   :param apiuser: This is filled automatically from the |authtoken|.
+   :type apiuser: AuthUser
+   :param search_query: Search query.
+   :type search_query: str
+   :param search_type: Search type. The following are valid options:
+       * commit
+       * content
+       * path
+   :type search_type: str
+   :param page_limit: Page item limit, from 1 to 500. Default 10 items.
+   :type page_limit: Optional(int)
+   :param page: Page number. Default first page.
+   :type page: Optional(int)
+   :param search_sort: Search sort order. Default newfirst. The following are valid options:
+       * newfirst
+       * oldfirst
+   :type search_sort: Optional(str)
+   :param repo_name: Filter by one repo. Default is all.
+   :type repo_name: Optional(str)
+   :param repo_group_name: Filter by one repo group. Default is all.
+   :type repo_group_name: Optional(str)
+
+

docs/release-notes/release-notes-4.17.0.rst

@@ -0,0 +1,85 b''
+|RCE| 4.17.0 |RNS|
+------------------
+
+Release Date
+^^^^^^^^^^^^
+
+- 2019-07-04
+
+
+New Features
+^^^^^^^^^^^^
+
+- New artifacts feature.
+  Ability to store binary artifacts for repository with ACL
+- UI/UX refresh for most of the pages. This includes multiple fixes and improvements.
+- Diffs: store wide-diff mode in user sessions to store user preference for diff display.
+- Mercurial: added support for Mercurial 4.9
+- API: Added search API methods
+- Files: adding/editing allows previews for generated content.
+- Files: allowed multi file upload using UI.
+- Repository Groups: last change is now smartly calculated based on latest change
+  from all it's children repositories.
+- Archives: it's now currently possible to download partial directories from files view.
+- SVN: allowed executing pre-commit code with rcextensions, also added example to
+  validate SVN file size and paths on pre-commit level.
+
+
+General
+^^^^^^^
+- Exception store: add filter for display and deletion.
+- Files: loading history doesn't display hidden and obsolete commits anymore.
+- Repositories: bring back missing watch action in summary view.
+- Admin: user groups is now using pure DB filtering to speed up display
+  for large number of groups.
+- Mercurial: enabled full evolve+topic extensions when evolve is enabled.
+- Dependencies: bumped evolve to 8.5.1
+- Dependencies: bumped pyramid to 1.10.4
+- Dependencies: bumped psutil to 5.5.1
+- Dependencies: bumped pygments to 2.4.2
+- Dependencies: bumped pyramid to 1.10.4
+- Dependencies: bumped psycopg2 to 2.8.3
+- Dependencies [security]: updated colander to 1.7.0
+
+
+Security
+^^^^^^^^
+
+- SSH: replaced pycrypto with cryptography to generate SSH keys as pycrypto isn't
+  considered safe anymore.
+
+
+Performance
+^^^^^^^^^^^
+
+- Config: updated header limits on gunicorn to prevent errors on large Mercurial repositories.
+- User sessions: added option to cleanup redis based sessions in user session interface.
+- Authentication: reduced usage of raw auth calls inside templates to speed up rendering.
+- Sessions: don't touch session for API calls. Before each API call created new session
+  object which wasn't required.
+
+
+Fixes
+^^^^^
+
+- hHooks: fixed more unicode problems with new pull-request link generator.
+- Mercurial: fix ssh-server support for mercurial custom options.
+- Pull requests: updated metadata information for failed merges with multiple heads.
+- Pull requests: calculate ancestor in the same way as creation mode.
+  Fixed problem with updates generating wrong diffs in case of merges.
+- Pull requests: fixed a bug in removal of multiple reviewers at once.
+- Summary: fix timeout issues loading summary page without styling.
+- SSH: fix invocation of custom hgrc.
+- SSH: call custom hooks via SSH backend
+- Markup: fix styling for check-lists.
+- Archives: allows downloading refs that have slashes and special refs. e.g f/feat1 branch names.
+- Files: ensure we generate archives with consistent hashing (except for .tar.gz which uses temp files names in header)
+- Files: fixed rendering of readme files under non-ascii paths.
+
+
+Upgrade notes
+^^^^^^^^^^^^^
+
+- In this release we introduced new UI across the application.
+  In case of problems with the display on your systems please send us info to [e-mail].
+

.bumpversion.cfg

@@ -1,6 +1,6 b''
 [bumpversion]
-current_version = 4.16.2
+current_version = 4.17.0
 message = release: Bump version {current_version} to {new_version}
 
 [bumpversion:file:rhodecode/VERSION]
 

.release.cfg

@@ -1,33 +1,28 b''
 [DEFAULT]
 done = false
 
 [task:bump_version]
 done = true
 
 [task:rc_tools_pinned]
-done = true
 
 [task:fixes_on_stable]
-done = true
 
 [task:pip2nix_generated]
-done = true
 
 [task:changelog_updated]
-done = true
 
 [task:generate_api_docs]
-done = true
+
+[task:updated_translation]
 
 [release]
-state = prepared
-version = 4.16.2
-
-[task:updated_translation]
+state = in_progress
+version = 4.17.0
 
 [task:generate_js_routes]
 
 [task:updated_trial_license]
 
 [task:generate_oss_licenses]
 

Makefile

@@ -1,55 +1,64 b''
 
 .PHONY: clean docs docs-clean docs-cleanup test test-clean test-only test-only-postgres test-only-mysql web-build generate-pkgs
 
 NODE_PATH=./node_modules
 WEBPACK=./node_binaries/webpack
 GRUNT=./node_binaries/grunt
 
 
 clean:
 	make test-clean
 	find . -type f \( -iname '*.c' -o -iname '*.pyc' -o -iname '*.so' -o -iname '*.orig' \) -exec rm '{}' ';'
 
 test:
 	make test-clean
 	make test-only
 
 test-clean:
 	rm -rf coverage.xml htmlcov junit.xml pylint.log result
 	find . -type d -name "__pycache__" -prune -exec rm -rf '{}' ';'
 
 test-only:
 	PYTHONHASHSEED=random \
 	py.test -x -vv -r xw -p no:sugar --cov=rhodecode \
     --cov-report=term-missing --cov-report=html \
     rhodecode
 
 test-only-mysql:
 	PYTHONHASHSEED=random \
 	py.test -x -vv -r xw -p no:sugar --cov=rhodecode \
     --cov-report=term-missing --cov-report=html \
     --ini-config-override='{"app:main": {"sqlalchemy.db1.url": "mysql://root:qweqwe@localhost/rhodecode_test"}}' \
     rhodecode
 
 test-only-postgres:
 	PYTHONHASHSEED=random \
 	py.test -x -vv -r xw -p no:sugar --cov=rhodecode \
     --cov-report=term-missing --cov-report=html \
     --ini-config-override='{"app:main": {"sqlalchemy.db1.url": "postgresql://postgres:qweqwe@localhost/rhodecode_test"}}' \
     rhodecode
 
 
 docs:
 	(cd docs; nix-build default.nix -o result; make clean html)
 
 docs-clean:
 	(cd docs; make clean)
 
 docs-cleanup:
 	(cd docs; make cleanup)
 
 web-build:
 	NODE_PATH=$(NODE_PATH) $(GRUNT)
 
 generate-pkgs:
 	nix-shell pkgs/shell-generate.nix --command "pip2nix generate --licenses"
+
+generate-js-pkgs:
+	rm -rf node_modules && \
+	nix-shell pkgs/shell-generate.nix --command "node2nix --input package.json -o pkgs/node-packages.nix -e pkgs/node-env.nix -c pkgs/node-default.nix -d --flatten --nodejs-8" && \
+	sed -i -e 's/http:\/\//https:\/\//g' pkgs/node-packages.nix
+
+generate-license-meta:
+	nix-build pkgs/license-generate.nix -o result-license && \
+	cat result-license/licenses.json | python -m json.tool > rhodecode/config/licenses.json
  No newline at end of file

configs/development.ini

@@ -1,732 +1,741 b''
 
 
 ################################################################################
 ##                RHODECODE COMMUNITY  EDITION CONFIGURATION                  ##
 ################################################################################
 
 [DEFAULT]
 ## Debug flag sets all loggers to debug, and enables request tracking
 debug = true
 
 ################################################################################
 ##                            EMAIL CONFIGURATION                             ##
 ## Uncomment and replace with the email address which should receive          ##
 ## any error reports after an application crash                               ##
 ## Additionally these settings will be used by the RhodeCode mailing system   ##
 ################################################################################
 
 ## prefix all emails subjects with given prefix, helps filtering out emails
 #email_prefix = [RhodeCode]
 
 ## email FROM address all mails will be sent
 #app_email_from = rhodecode-noreply@localhost
 
 #smtp_server = mail.server.com
 #smtp_username =
 #smtp_password =
 #smtp_port =
 #smtp_use_tls = false
 #smtp_use_ssl = true
 
 [server:main]
 ## COMMON ##
 host = 127.0.0.1
 port = 5000
 
 ###########################################################
 ## WAITRESS WSGI SERVER - Recommended for Development  ####
 ###########################################################
 
 use = egg:waitress#main
 ## number of worker threads
 threads = 5
 ## MAX BODY SIZE 100GB
 max_request_body_size = 107374182400
 ## Use poll instead of select, fixes file descriptors limits problems.
 ## May not work on old windows systems.
 asyncore_use_poll = true
 
 
 ##########################
 ## GUNICORN WSGI SERVER ##
 ##########################
 ## run with gunicorn --log-config rhodecode.ini --paste rhodecode.ini
 
 #use = egg:gunicorn#main
 ## Sets the number of process workers. More workers means more concurrent connections
 ## RhodeCode can handle at the same time. Each additional worker also it increases
 ## memory usage as each has it's own set of caches.
 ## Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
 ## than 8-10 unless for really big deployments .e.g 700-1000 users.
 ## `instance_id = *` must be set in the [app:main] section below (which is the default)
 ## when using more than 1 worker.
 #workers = 2
 ## process name visible in process list
 #proc_name = rhodecode
 ## type of worker class, one of sync, gevent
 ## recommended for bigger setup is using of of other than sync one
 #worker_class = gevent
 ## The maximum number of simultaneous clients. Valid only for Gevent
 #worker_connections = 10
 ## max number of requests that worker will handle before being gracefully
 ## restarted, could prevent memory leaks
 #max_requests = 1000
 #max_requests_jitter = 30
 ## amount of time a worker can spend with handling a request before it
 ## gets killed and restarted. Set to 6hrs
 #timeout = 21600
 
 
 ## prefix middleware for RhodeCode.
 ## recommended when using proxy setup.
 ## allows to set RhodeCode under a prefix in server.
 ## eg https://server.com/custom_prefix. Enable `filter-with =` option below as well.
 ## And set your prefix like: `prefix = /custom_prefix`
 ## be sure to also set beaker.session.cookie_path = /custom_prefix if you need
 ## to make your cookies only work on prefix url
 [filter:proxy-prefix]
 use = egg:PasteDeploy#prefix
 prefix = /
 
 [app:main]
 ## The %(here)s variable will be replaced with the absolute path of parent directory
 ## of this file
 ## In addition ENVIRONMENT variables usage is possible, e.g
 ## sqlalchemy.db1.url = {ENV_RC_DB_URL}
 
 use = egg:rhodecode-enterprise-ce
 
 ## enable proxy prefix middleware, defined above
 #filter-with = proxy-prefix
 
 # During development the we want to have the debug toolbar enabled
 pyramid.includes =
     pyramid_debugtoolbar
     rhodecode.lib.middleware.request_wrapper
 
 pyramid.reload_templates = true
 
 debugtoolbar.hosts = 0.0.0.0/0
 debugtoolbar.exclude_prefixes =
     /css
     /fonts
     /images
     /js
 
 ## RHODECODE PLUGINS ##
 rhodecode.includes =
     rhodecode.api
 
 
 # api prefix url
 rhodecode.api.url = /_admin/api
 
 
 ## END RHODECODE PLUGINS ##
 
 ## encryption key used to encrypt social plugin tokens,
 ## remote_urls with credentials etc, if not set it defaults to
 ## `beaker.session.secret`
 #rhodecode.encrypted_values.secret =
 
 ## decryption strict mode (enabled by default). It controls if decryption raises
 ## `SignatureVerificationError` in case of wrong key, or damaged encryption data.
 #rhodecode.encrypted_values.strict = false
 
+## Pick algorithm for encryption. Either fernet (more secure) or aes (default)
+## fernet is safer, and we strongly recommend switching to it.
+## Due to backward compatibility aes is used as default.
+#rhodecode.encrypted_values.algorithm = fernet
+
 ## return gzipped responses from RhodeCode (static files/application)
 gzip_responses = false
 
 ## auto-generate javascript routes file on startup
 generate_js_files = false
 
 ## System global default language.
 ## All available languages: en(default), be, de, es, fr, it, ja, pl, pt, ru, zh
 lang = en
 
 ## Perform a full repository scan and import on each server start.
 ## Settings this to true could lead to very long startup time.
 startup.import_repos = false
 
 ## Uncomment and set this path to use archive download cache.
 ## Once enabled, generated archives will be cached at this location
 ## and served from the cache during subsequent requests for the same archive of
 ## the repository.
 #archive_cache_dir = /tmp/tarballcache
 
 ## URL at which the application is running. This is used for Bootstrapping
 ## requests in context when no web request is available. Used in ishell, or
 ## SSH calls. Set this for events to receive proper url for SSH calls.
 app.base_url = http://rhodecode.local
 
 ## Unique application ID. Should be a random unique string for security.
 app_instance_uuid = rc-production
 
 ## Cut off limit for large diffs (size in bytes). If overall diff size on
 ## commit, or pull request exceeds this limit this diff will be displayed
 ## partially. E.g 512000 == 512Kb
 cut_off_limit_diff = 512000
 
 ## Cut off limit for large files inside diffs (size in bytes). Each individual
 ## file inside diff which exceeds this limit will be displayed partially.
 ##  E.g 128000 == 128Kb
 cut_off_limit_file = 128000
 
 ## use cached version of vcs repositories everywhere. Recommended to be `true`
 vcs_full_cache = true
 
 ## Force https in RhodeCode, fixes https redirects, assumes it's always https.
 ## Normally this is controlled by proper http flags sent from http server
 force_https = false
 
 ## use Strict-Transport-Security headers
 use_htsts = false
 
 ## git rev filter option, --all is the default filter, if you need to
 ## hide all refs in changelog switch this to --branches --tags
 git_rev_filter = --branches --tags
 
 # Set to true if your repos are exposed using the dumb protocol
 git_update_server_info = false
 
 ## RSS/ATOM feed options
 rss_cut_off_limit = 256000
 rss_items_per_page = 10
 rss_include_diff = false
 
 ## gist URL alias, used to create nicer urls for gist. This should be an
 ## url that does rewrites to _admin/gists/{gistid}.
 ## example: http://gist.rhodecode.org/{gistid}. Empty means use the internal
 ## RhodeCode url, ie. http[s]://rhodecode.server/_admin/gists/{gistid}
 gist_alias_url =
 
 ## List of views (using glob pattern syntax) that AUTH TOKENS could be
 ## used for access.
 ## Adding ?auth_token=TOKEN_HASH to the url authenticates this request as if it
 ## came from the the logged in user who own this authentication token.
 ## Additionally @TOKEN syntax can be used to bound the view to specific
 ## authentication token. Such view would be only accessible when used together
 ## with this authentication token
 ##
 ## list of all views can be found under `/_admin/permissions/auth_token_access`
 ## The list should be "," separated and on a single line.
 ##
 ## Most common views to enable:
 #    RepoCommitsView:repo_commit_download
 #    RepoCommitsView:repo_commit_patch
 #    RepoCommitsView:repo_commit_raw
 #    RepoCommitsView:repo_commit_raw@TOKEN
 #    RepoFilesView:repo_files_diff
 #    RepoFilesView:repo_archivefile
 #    RepoFilesView:repo_file_raw
 #    GistView:*
 api_access_controllers_whitelist =
 
 ## Default encoding used to convert from and to unicode
 ## can be also a comma separated list of encoding in case of mixed encodings
 default_encoding = UTF-8
 
 ## instance-id prefix
 ## a prefix key for this instance used for cache invalidation when running
 ## multiple instances of RhodeCode, make sure it's globally unique for
 ## all running RhodeCode instances. Leave empty if you don't use it
 instance_id =
 
 ## Fallback authentication plugin. Set this to a plugin ID to force the usage
 ## of an authentication plugin also if it is disabled by it's settings.
 ## This could be useful if you are unable to log in to the system due to broken
 ## authentication settings. Then you can enable e.g. the internal RhodeCode auth
 ## module to log in again and fix the settings.
 ##
 ## Available builtin plugin IDs (hash is part of the ID):
 ## egg:rhodecode-enterprise-ce#rhodecode
 ## egg:rhodecode-enterprise-ce#pam
 ## egg:rhodecode-enterprise-ce#ldap
 ## egg:rhodecode-enterprise-ce#jasig_cas
 ## egg:rhodecode-enterprise-ce#headers
 ## egg:rhodecode-enterprise-ce#crowd
 #rhodecode.auth_plugin_fallback = egg:rhodecode-enterprise-ce#rhodecode
 
 ## alternative return HTTP header for failed authentication. Default HTTP
 ## response is 401 HTTPUnauthorized. Currently HG clients have troubles with
 ## handling that causing a series of failed authentication calls.
 ## Set this variable to 403 to return HTTPForbidden, or any other HTTP code
 ## This will be served instead of default 401 on bad authentication
 auth_ret_code =
 
 ## use special detection method when serving auth_ret_code, instead of serving
 ## ret_code directly, use 401 initially (Which triggers credentials prompt)
 ## and then serve auth_ret_code to clients
 auth_ret_code_detection = false
 
 ## locking return code. When repository is locked return this HTTP code. 2XX
 ## codes don't break the transactions while 4XX codes do
 lock_ret_code = 423
 
 ## allows to change the repository location in settings page
 allow_repo_location_change = true
 
 ## allows to setup custom hooks in settings page
 allow_custom_hooks_settings = true
 
 ## Generated license token required for EE edition license.
 ## New generated token value can be found in Admin > settings > license page.
 license_token =
 
 ## supervisor connection uri, for managing supervisor and logs.
 supervisor.uri =
 ## supervisord group name/id we only want this RC instance to handle
 supervisor.group_id = dev
 
 ## Display extended labs settings
 labs_settings_active = true
 
 ## Custom exception store path, defaults to TMPDIR
 ## This is used to store exception from RhodeCode in shared directory
 #exception_tracker.store_path =
 
 ## File store configuration. This is used to store and serve uploaded files
 file_store.enabled = true
 ## Storage backend, available options are: local
 file_store.backend = local
 ## path to store the uploaded binaries
 file_store.storage_path = %(here)s/data/file_store
 
 
 ####################################
 ###        CELERY CONFIG        ####
 ####################################
 ## run: /path/to/celery worker \
 ##     -E --beat --app rhodecode.lib.celerylib.loader \
 ##     --scheduler rhodecode.lib.celerylib.scheduler.RcScheduler \
 ##     --loglevel DEBUG --ini /path/to/rhodecode.ini
 
 use_celery = false
 
 ## connection url to the message broker (default rabbitmq)
 celery.broker_url = amqp://rabbitmq:qweqwe@localhost:5672/rabbitmqhost
 
 ## maximum tasks to execute before worker restart
 celery.max_tasks_per_child = 100
 
 ## tasks will never be sent to the queue, but executed locally instead.
 celery.task_always_eager = false
 
 #####################################
 ###         DOGPILE CACHE        ####
 #####################################
 ## Default cache dir for caches.  Putting this into a ramdisk
 ## can boost performance, eg. /tmpfs/data_ramdisk, however this directory might require
 ## large amount of space
 cache_dir = %(here)s/data
 
 ## `cache_perms` cache settings for permission tree, auth TTL.
 rc_cache.cache_perms.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_perms.expiration_time = 300
 
 ## alternative `cache_perms` redis backend with distributed lock
 #rc_cache.cache_perms.backend = dogpile.cache.rc.redis
 #rc_cache.cache_perms.expiration_time = 300
 ## redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_perms.arguments.redis_expiration_time = 7200
 #rc_cache.cache_perms.arguments.socket_timeout = 30
 #rc_cache.cache_perms.arguments.host = localhost
 #rc_cache.cache_perms.arguments.port = 6379
 #rc_cache.cache_perms.arguments.db = 0
 ## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_perms.arguments.distributed_lock = true
 
 ## `cache_repo` cache settings for FileTree, Readme, RSS FEEDS
 rc_cache.cache_repo.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_repo.expiration_time = 2592000
 
 ## alternative `cache_repo` redis backend with distributed lock
 #rc_cache.cache_repo.backend = dogpile.cache.rc.redis
 #rc_cache.cache_repo.expiration_time = 2592000
 ## redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_repo.arguments.redis_expiration_time = 2678400
 #rc_cache.cache_repo.arguments.socket_timeout = 30
 #rc_cache.cache_repo.arguments.host = localhost
 #rc_cache.cache_repo.arguments.port = 6379
 #rc_cache.cache_repo.arguments.db = 1
 ## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_repo.arguments.distributed_lock = true
 
 ## cache settings for SQL queries, this needs to use memory type backend
 rc_cache.sql_cache_short.backend = dogpile.cache.rc.memory_lru
 rc_cache.sql_cache_short.expiration_time = 30
 
 ## `cache_repo_longterm` cache for repo object instances, this needs to use memory
 ## type backend as the objects kept are not pickle serializable
 rc_cache.cache_repo_longterm.backend = dogpile.cache.rc.memory_lru
 ## by default we use 96H, this is using invalidation on push anyway
 rc_cache.cache_repo_longterm.expiration_time = 345600
 ## max items in LRU cache, reduce this number to save memory, and expire last used
 ## cached objects
 rc_cache.cache_repo_longterm.max_size = 10000
 
 
 ####################################
 ###       BEAKER SESSION        ####
 ####################################
 
 ## .session.type is type of storage options for the session, current allowed
 ## types are file, ext:memcached, ext:redis, ext:database, and memory (default).
 beaker.session.type = file
 beaker.session.data_dir = %(here)s/data/sessions
 
+## redis sessions
+#beaker.session.type = ext:redis
+#beaker.session.url = redis://127.0.0.1:6379/2
+
 ## db based session, fast, and allows easy management over logged in users
 #beaker.session.type = ext:database
 #beaker.session.table_name = db_session
 #beaker.session.sa.url = postgresql://postgres:secret@localhost/rhodecode
 #beaker.session.sa.url = mysql://root:secret@127.0.0.1/rhodecode
 #beaker.session.sa.pool_recycle = 3600
 #beaker.session.sa.echo = false
 
 beaker.session.key = rhodecode
 beaker.session.secret = develop-rc-uytcxaz
 beaker.session.lock_dir = %(here)s/data/sessions/lock
 
 ## Secure encrypted cookie. Requires AES and AES python libraries
 ## you must disable beaker.session.secret to use this
 #beaker.session.encrypt_key = key_for_encryption
 #beaker.session.validate_key = validation_key
 
 ## sets session as invalid(also logging out user) if it haven not been
 ## accessed for given amount of time in seconds
 beaker.session.timeout = 2592000
 beaker.session.httponly = true
 ## Path to use for the cookie. Set to prefix if you use prefix middleware
 #beaker.session.cookie_path = /custom_prefix
 
 ## uncomment for https secure cookie
 beaker.session.secure = false
 
 ## auto save the session to not to use .save()
 beaker.session.auto = false
 
 ## default cookie expiration time in seconds, set to `true` to set expire
 ## at browser close
 #beaker.session.cookie_expires = 3600
 
 ###################################
 ## SEARCH INDEXING CONFIGURATION ##
 ###################################
 ## Full text search indexer is available in rhodecode-tools under
 ## `rhodecode-tools index` command
 
 ## WHOOSH Backend, doesn't require additional services to run
 ## it works good with few dozen repos
 search.module = rhodecode.lib.index.whoosh
 search.location = %(here)s/data/index
 
 ########################################
 ###    CHANNELSTREAM CONFIG         ####
 ########################################
 ## channelstream enables persistent connections and live notification
 ## in the system. It's also used by the chat system
 
 channelstream.enabled = false
 
 ## server address for channelstream server on the backend
 channelstream.server = 127.0.0.1:9800
 
 ## location of the channelstream server from outside world
 ## use ws:// for http or wss:// for https. This address needs to be handled
 ## by external HTTP server such as Nginx or Apache
 ## see Nginx/Apache configuration examples in our docs
 channelstream.ws_url = ws://rhodecode.yourserver.com/_channelstream
 channelstream.secret = secret
 channelstream.history.location = %(here)s/channelstream_history
 
 ## Internal application path that Javascript uses to connect into.
 ## If you use proxy-prefix the prefix should be added before /_channelstream
 channelstream.proxy_path = /_channelstream
 
 
 ###################################
 ##       APPENLIGHT CONFIG       ##
 ###################################
 
 ## Appenlight is tailored to work with RhodeCode, see
 ## http://appenlight.com for details how to obtain an account
 
 ## Appenlight integration enabled
 appenlight = false
 
 appenlight.server_url = https://api.appenlight.com
 appenlight.api_key = YOUR_API_KEY
 #appenlight.transport_config = https://api.appenlight.com?threaded=1&timeout=5
 
 ## used for JS client
 appenlight.api_public_key = YOUR_API_PUBLIC_KEY
 
 ## TWEAK AMOUNT OF INFO SENT HERE
 
 ## enables 404 error logging (default False)
 appenlight.report_404 = false
 
 ## time in seconds after request is considered being slow (default 1)
 appenlight.slow_request_time = 1
 
 ## record slow requests in application
 ## (needs to be enabled for slow datastore recording and time tracking)
 appenlight.slow_requests = true
 
 ## enable hooking to application loggers
 appenlight.logging = true
 
 ## minimum log level for log capture
 appenlight.logging.level = WARNING
 
 ## send logs only from erroneous/slow requests
 ## (saves API quota for intensive logging)
 appenlight.logging_on_error = false
 
 ## list of additional keywords that should be grabbed from environ object
 ## can be string with comma separated list of words in lowercase
 ## (by default client will always send following info:
 ## 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
 ## start with HTTP* this list be extended with additional keywords here
 appenlight.environ_keys_whitelist =
 
 ## list of keywords that should be blanked from request object
 ## can be string with comma separated list of words in lowercase
 ## (by default client will always blank keys that contain following words
 ## 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
 ## this list be extended with additional keywords set here
 appenlight.request_keys_blacklist =
 
 ## list of namespaces that should be ignores when gathering log entries
 ## can be string with comma separated list of namespaces
 ## (by default the client ignores own entries: appenlight_client.client)
 appenlight.log_namespace_blacklist =
 
 # enable debug style page
 debug_style = true
 
 ###########################################
 ###   MAIN RHODECODE DATABASE CONFIG    ###
 ###########################################
 #sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
 #sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
 #sqlalchemy.db1.url = mysql://root:qweqwe@localhost/rhodecode?charset=utf8
 # pymysql is an alternative driver for MySQL, use in case of problems with default one
 #sqlalchemy.db1.url = mysql+pymysql://root:qweqwe@localhost/rhodecode
 
 sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
 
 # see sqlalchemy docs for other advanced settings
 
 ## print the sql statements to output
 sqlalchemy.db1.echo = false
 ## recycle the connections after this amount of seconds
 sqlalchemy.db1.pool_recycle = 3600
 sqlalchemy.db1.convert_unicode = true
 
 ## the number of connections to keep open inside the connection pool.
 ## 0 indicates no limit
 #sqlalchemy.db1.pool_size = 5
 
 ## the number of connections to allow in connection pool "overflow", that is
 ## connections that can be opened above and beyond the pool_size setting,
 ## which defaults to five.
 #sqlalchemy.db1.max_overflow = 10
 
 ## Connection check ping, used to detect broken database connections
 ## could be enabled to better handle cases if MySQL has gone away errors
 #sqlalchemy.db1.ping_connection = true
 
 ##################
 ### VCS CONFIG ###
 ##################
 vcs.server.enable = true
 vcs.server = localhost:9900
 
 ## Web server connectivity protocol, responsible for web based VCS operations
 ## Available protocols are:
 ## `http` - use http-rpc backend (default)
 vcs.server.protocol = http
 
 ## Push/Pull operations protocol, available options are:
 ## `http` - use http-rpc backend (default)
 vcs.scm_app_implementation = http
 
 ## Push/Pull operations hooks protocol, available options are:
 ## `http` - use http-rpc backend (default)
 vcs.hooks.protocol = http
 
 ## Host on which this instance is listening for hooks. If vcsserver is in other location
 ## this should be adjusted.
 vcs.hooks.host = 127.0.0.1
 
 vcs.server.log_level = debug
 ## Start VCSServer with this instance as a subprocess, useful for development
 vcs.start_server = false
 
 ## List of enabled VCS backends, available options are:
 ## `hg`  - mercurial
 ## `git` - git
 ## `svn` - subversion
 vcs.backends = hg, git, svn
 
 vcs.connection_timeout = 3600
 ## Compatibility version when creating SVN repositories. Defaults to newest version when commented out.
 ## Available options are: pre-1.4-compatible, pre-1.5-compatible, pre-1.6-compatible, pre-1.8-compatible, pre-1.9-compatible
 #vcs.svn.compatible_version = pre-1.8-compatible
 
 
 ############################################################
 ### Subversion proxy support (mod_dav_svn)               ###
 ### Maps RhodeCode repo groups into SVN paths for Apache ###
 ############################################################
 ## Enable or disable the config file generation.
 svn.proxy.generate_config = false
 ## Generate config file with `SVNListParentPath` set to `On`.
 svn.proxy.list_parent_path = true
 ## Set location and file name of generated config file.
 svn.proxy.config_file_path = %(here)s/mod_dav_svn.conf
 ## alternative mod_dav config template. This needs to be a mako template
 #svn.proxy.config_template = ~/.rccontrol/enterprise-1/custom_svn_conf.mako
 ## Used as a prefix to the `Location` block in the generated config file.
 ## In most cases it should be set to `/`.
 svn.proxy.location_root = /
 ## Command to reload the mod dav svn configuration on change.
 ## Example: `/etc/init.d/apache2 reload` or /home/USER/apache_reload.sh
 ## Make sure user who runs RhodeCode process is allowed to reload Apache
 #svn.proxy.reload_cmd = /etc/init.d/apache2 reload
 ## If the timeout expires before the reload command finishes, the command will
 ## be killed. Setting it to zero means no timeout. Defaults to 10 seconds.
 #svn.proxy.reload_timeout = 10
 
 ############################################################
 ### SSH Support Settings                                 ###
 ############################################################
 
 ## Defines if a custom authorized_keys file should be created and written on
 ## any change user ssh keys. Setting this to false also disables possibility
 ## of adding SSH keys by users from web interface. Super admins can still
 ## manage SSH Keys.
 ssh.generate_authorized_keyfile = false
 
 ## Options for ssh, default is `no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding`
 # ssh.authorized_keys_ssh_opts =
 
 ## Path to the authorized_keys file where the generate entries are placed.
 ## It is possible to have multiple key files specified in `sshd_config` e.g.
 ## AuthorizedKeysFile %h/.ssh/authorized_keys %h/.ssh/authorized_keys_rhodecode
 ssh.authorized_keys_file_path = ~/.ssh/authorized_keys_rhodecode
 
 ## Command to execute the SSH wrapper. The binary is available in the
 ## RhodeCode installation directory.
 ## e.g ~/.rccontrol/community-1/profile/bin/rc-ssh-wrapper
 ssh.wrapper_cmd = ~/.rccontrol/community-1/rc-ssh-wrapper
 
 ## Allow shell when executing the ssh-wrapper command
 ssh.wrapper_cmd_allow_shell = false
 
 ## Enables logging, and detailed output send back to the client during SSH
 ## operations. Useful for debugging, shouldn't be used in production.
 ssh.enable_debug_logging = true
 
 ## Paths to binary executable, by default they are the names, but we can
 ## override them if we want to use a custom one
 ssh.executable.hg = ~/.rccontrol/vcsserver-1/profile/bin/hg
 ssh.executable.git = ~/.rccontrol/vcsserver-1/profile/bin/git
 ssh.executable.svn = ~/.rccontrol/vcsserver-1/profile/bin/svnserve
 
 ## Enables SSH key generator web interface. Disabling this still allows users
 ## to add their own keys.
 ssh.enable_ui_key_generator = true
 
 
 ## Dummy marker to add new entries after.
 ## Add any custom entries below. Please don't remove.
 custom.conf = 1
 
 
 ################################
 ### LOGGING CONFIGURATION   ####
 ################################
 [loggers]
 keys = root, sqlalchemy, beaker, celery, rhodecode, ssh_wrapper
 
 [handlers]
 keys = console, console_sql
 
 [formatters]
 keys = generic, color_formatter, color_formatter_sql
 
 #############
 ## LOGGERS ##
 #############
 [logger_root]
 level = NOTSET
 handlers = console
 
 [logger_sqlalchemy]
 level = INFO
 handlers = console_sql
 qualname = sqlalchemy.engine
 propagate = 0
 
 [logger_beaker]
 level = DEBUG
 handlers =
 qualname = beaker.container
 propagate = 1
 
 [logger_rhodecode]
 level = DEBUG
 handlers =
 qualname = rhodecode
 propagate = 1
 
 [logger_ssh_wrapper]
 level = DEBUG
 handlers =
 qualname = ssh_wrapper
 propagate = 1
 
 [logger_celery]
 level = DEBUG
 handlers =
 qualname = celery
 
 
 ##############
 ## HANDLERS ##
 ##############
 
 [handler_console]
 class = StreamHandler
 args = (sys.stderr, )
 level = DEBUG
 formatter = color_formatter
 
 [handler_console_sql]
 # "level = DEBUG" logs SQL queries and results.
 # "level = INFO" logs SQL queries.
 # "level = WARN" logs neither.  (Recommended for production systems.)
 class = StreamHandler
 args = (sys.stderr, )
 level = WARN
 formatter = color_formatter_sql
 
 ################
 ## FORMATTERS ##
 ################
 
 [formatter_generic]
 class = rhodecode.lib.logging_formatter.ExceptionAwareFormatter
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter]
 class = rhodecode.lib.logging_formatter.ColorFormatter
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter_sql]
 class = rhodecode.lib.logging_formatter.ColorFormatterSql
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S

configs/gunicorn_config.py

@@ -1,152 +1,154 b''
 """
 gunicorn config extension and hooks. Sets additional configuration that is
 available post the .ini config.
 
 - workers = ${cpu_number}
 - threads = 1
 - proc_name = ${gunicorn_proc_name}
 - worker_class = sync
 - worker_connections = 10
 - max_requests = 1000
 - max_requests_jitter = 30
 - timeout = 21600
 
 """
 
 import multiprocessing
 import sys
 import time
 import datetime
 import threading
 import traceback
 from gunicorn.glogging import Logger
 
 
 # GLOBAL
 errorlog = '-'
 accesslog = '-'
 loglevel = 'debug'
 
 # SECURITY
 
 # The maximum size of HTTP request line in bytes.
-limit_request_line = 4094
+# 0 for unlimited
+limit_request_line = 0
 
 # Limit the number of HTTP headers fields in a request.
-limit_request_fields = 1024
+# By default this value is 100 and can’t be larger than 32768.
+limit_request_fields = 10240
 
 # Limit the allowed size of an HTTP request header field.
 # Value is a positive number or 0.
 # Setting it to 0 will allow unlimited header field sizes.
 limit_request_field_size = 0
 
 
 # Timeout for graceful workers restart.
 # After receiving a restart signal, workers have this much time to finish
 # serving requests. Workers still alive after the timeout (starting from the
 # receipt of the restart signal) are force killed.
 graceful_timeout = 30
 
 
 # The number of seconds to wait for requests on a Keep-Alive connection.
 # Generally set in the 1-5 seconds range.
 keepalive = 2
 
 
 # SERVER MECHANICS
 # None == system temp dir
 # worker_tmp_dir is recommended to be set to some tmpfs
 worker_tmp_dir = None
 tmp_upload_dir = None
 
 # Custom log format
 access_log_format = (
     '%(t)s [%(p)-8s] GNCRN %(h)-15s rqt:%(L)s %(s)s %(b)-6s "%(m)s:%(U)s %(q)s" usr:%(u)s "%(f)s" "%(a)s"')
 
 # self adjust workers based on CPU count
 # workers = multiprocessing.cpu_count() * 2 + 1
 
 
 def post_fork(server, worker):
     server.log.info("[<%-10s>] WORKER spawned", worker.pid)
 
 
 def pre_fork(server, worker):
     pass
 
 
 def pre_exec(server):
     server.log.info("Forked child, re-executing.")
 
 
 def on_starting(server):
     server.log.info("Server is starting.")
 
 
 def when_ready(server):
     server.log.info("Server is ready. Spawning workers")
 
 
 def on_reload(server):
     pass
 
 
 def worker_int(worker):
     worker.log.info("[<%-10s>] worker received INT or QUIT signal", worker.pid)
 
     # get traceback info, on worker crash
     id2name = dict([(th.ident, th.name) for th in threading.enumerate()])
     code = []
     for thread_id, stack in sys._current_frames().items():
         code.append(
             "\n# Thread: %s(%d)" % (id2name.get(thread_id, ""), thread_id))
         for fname, lineno, name, line in traceback.extract_stack(stack):
             code.append('File: "%s", line %d, in %s' % (fname, lineno, name))
             if line:
                 code.append("  %s" % (line.strip()))
     worker.log.debug("\n".join(code))
 
 
 def worker_abort(worker):
     worker.log.info("[<%-10s>] worker received SIGABRT signal", worker.pid)
 
 
 def worker_exit(server, worker):
     worker.log.info("[<%-10s>] worker exit", worker.pid)
 
 
 def child_exit(server, worker):
     worker.log.info("[<%-10s>] worker child exit", worker.pid)
 
 
 def pre_request(worker, req):
     worker.start_time = time.time()
     worker.log.debug(
         "GNCRN PRE  WORKER [cnt:%s]: %s %s", worker.nr, req.method, req.path)
 
 
 def post_request(worker, req, environ, resp):
     total_time = time.time() - worker.start_time
     worker.log.debug(
         "GNCRN POST WORKER [cnt:%s]: %s %s resp: %s, Load Time: %.3fs",
         worker.nr, req.method, req.path, resp.status_code, total_time)
 
 
 class RhodeCodeLogger(Logger):
     """
     Custom Logger that allows some customization that gunicorn doesn't allow
     """
 
     datefmt = r"%Y-%m-%d %H:%M:%S"
 
     def __init__(self, cfg):
         Logger.__init__(self, cfg)
 
     def now(self):
         """ return date in RhodeCode Log format """
         now = time.time()
         msecs = int((now - long(now)) * 1000)
         return time.strftime(self.datefmt, time.localtime(now)) + '.{0:03d}'.format(msecs)
 
 
 logger_class = RhodeCodeLogger

configs/production.ini

@@ -1,705 +1,714 b''
 
 
 ################################################################################
 ##                RHODECODE COMMUNITY  EDITION CONFIGURATION                  ##
 ################################################################################
 
 [DEFAULT]
 ## Debug flag sets all loggers to debug, and enables request tracking
 debug = false
 
 ################################################################################
 ##                            EMAIL CONFIGURATION                             ##
 ## Uncomment and replace with the email address which should receive          ##
 ## any error reports after an application crash                               ##
 ## Additionally these settings will be used by the RhodeCode mailing system   ##
 ################################################################################
 
 ## prefix all emails subjects with given prefix, helps filtering out emails
 #email_prefix = [RhodeCode]
 
 ## email FROM address all mails will be sent
 #app_email_from = rhodecode-noreply@localhost
 
 #smtp_server = mail.server.com
 #smtp_username =
 #smtp_password =
 #smtp_port =
 #smtp_use_tls = false
 #smtp_use_ssl = true
 
 [server:main]
 ## COMMON ##
 host = 127.0.0.1
 port = 5000
 
 ###########################################################
 ## WAITRESS WSGI SERVER - Recommended for Development  ####
 ###########################################################
 
 #use = egg:waitress#main
 ## number of worker threads
 #threads = 5
 ## MAX BODY SIZE 100GB
 #max_request_body_size = 107374182400
 ## Use poll instead of select, fixes file descriptors limits problems.
 ## May not work on old windows systems.
 #asyncore_use_poll = true
 
 
 ##########################
 ## GUNICORN WSGI SERVER ##
 ##########################
 ## run with gunicorn --log-config rhodecode.ini --paste rhodecode.ini
 
 use = egg:gunicorn#main
 ## Sets the number of process workers. More workers means more concurrent connections
 ## RhodeCode can handle at the same time. Each additional worker also it increases
 ## memory usage as each has it's own set of caches.
 ## Recommended value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers, but no more
 ## than 8-10 unless for really big deployments .e.g 700-1000 users.
 ## `instance_id = *` must be set in the [app:main] section below (which is the default)
 ## when using more than 1 worker.
 workers = 2
 ## process name visible in process list
 proc_name = rhodecode
 ## type of worker class, one of sync, gevent
 ## recommended for bigger setup is using of of other than sync one
 worker_class = gevent
 ## The maximum number of simultaneous clients. Valid only for Gevent
 worker_connections = 10
 ## max number of requests that worker will handle before being gracefully
 ## restarted, could prevent memory leaks
 max_requests = 1000
 max_requests_jitter = 30
 ## amount of time a worker can spend with handling a request before it
 ## gets killed and restarted. Set to 6hrs
 timeout = 21600
 
 
 ## prefix middleware for RhodeCode.
 ## recommended when using proxy setup.
 ## allows to set RhodeCode under a prefix in server.
 ## eg https://server.com/custom_prefix. Enable `filter-with =` option below as well.
 ## And set your prefix like: `prefix = /custom_prefix`
 ## be sure to also set beaker.session.cookie_path = /custom_prefix if you need
 ## to make your cookies only work on prefix url
 [filter:proxy-prefix]
 use = egg:PasteDeploy#prefix
 prefix = /
 
 [app:main]
 ## The %(here)s variable will be replaced with the absolute path of parent directory
 ## of this file
 ## In addition ENVIRONMENT variables usage is possible, e.g
 ## sqlalchemy.db1.url = {ENV_RC_DB_URL}
 
 use = egg:rhodecode-enterprise-ce
 
 ## enable proxy prefix middleware, defined above
 #filter-with = proxy-prefix
 
 ## encryption key used to encrypt social plugin tokens,
 ## remote_urls with credentials etc, if not set it defaults to
 ## `beaker.session.secret`
 #rhodecode.encrypted_values.secret =
 
 ## decryption strict mode (enabled by default). It controls if decryption raises
 ## `SignatureVerificationError` in case of wrong key, or damaged encryption data.
 #rhodecode.encrypted_values.strict = false
 
+## Pick algorithm for encryption. Either fernet (more secure) or aes (default)
+## fernet is safer, and we strongly recommend switching to it.
+## Due to backward compatibility aes is used as default.
+#rhodecode.encrypted_values.algorithm = fernet
+
 ## return gzipped responses from RhodeCode (static files/application)
 gzip_responses = false
 
 ## auto-generate javascript routes file on startup
 generate_js_files = false
 
 ## System global default language.
 ## All available languages: en(default), be, de, es, fr, it, ja, pl, pt, ru, zh
 lang = en
 
 ## Perform a full repository scan and import on each server start.
 ## Settings this to true could lead to very long startup time.
 startup.import_repos = false
 
 ## Uncomment and set this path to use archive download cache.
 ## Once enabled, generated archives will be cached at this location
 ## and served from the cache during subsequent requests for the same archive of
 ## the repository.
 #archive_cache_dir = /tmp/tarballcache
 
 ## URL at which the application is running. This is used for Bootstrapping
 ## requests in context when no web request is available. Used in ishell, or
 ## SSH calls. Set this for events to receive proper url for SSH calls.
 app.base_url = http://rhodecode.local
 
 ## Unique application ID. Should be a random unique string for security.
 app_instance_uuid = rc-production
 
 ## Cut off limit for large diffs (size in bytes). If overall diff size on
 ## commit, or pull request exceeds this limit this diff will be displayed
 ## partially. E.g 512000 == 512Kb
 cut_off_limit_diff = 512000
 
 ## Cut off limit for large files inside diffs (size in bytes). Each individual
 ## file inside diff which exceeds this limit will be displayed partially.
 ##  E.g 128000 == 128Kb
 cut_off_limit_file = 128000
 
 ## use cached version of vcs repositories everywhere. Recommended to be `true`
 vcs_full_cache = true
 
 ## Force https in RhodeCode, fixes https redirects, assumes it's always https.
 ## Normally this is controlled by proper http flags sent from http server
 force_https = false
 
 ## use Strict-Transport-Security headers
 use_htsts = false
 
 ## git rev filter option, --all is the default filter, if you need to
 ## hide all refs in changelog switch this to --branches --tags
 git_rev_filter = --branches --tags
 
 # Set to true if your repos are exposed using the dumb protocol
 git_update_server_info = false
 
 ## RSS/ATOM feed options
 rss_cut_off_limit = 256000
 rss_items_per_page = 10
 rss_include_diff = false
 
 ## gist URL alias, used to create nicer urls for gist. This should be an
 ## url that does rewrites to _admin/gists/{gistid}.
 ## example: http://gist.rhodecode.org/{gistid}. Empty means use the internal
 ## RhodeCode url, ie. http[s]://rhodecode.server/_admin/gists/{gistid}
 gist_alias_url =
 
 ## List of views (using glob pattern syntax) that AUTH TOKENS could be
 ## used for access.
 ## Adding ?auth_token=TOKEN_HASH to the url authenticates this request as if it
 ## came from the the logged in user who own this authentication token.
 ## Additionally @TOKEN syntax can be used to bound the view to specific
 ## authentication token. Such view would be only accessible when used together
 ## with this authentication token
 ##
 ## list of all views can be found under `/_admin/permissions/auth_token_access`
 ## The list should be "," separated and on a single line.
 ##
 ## Most common views to enable:
 #    RepoCommitsView:repo_commit_download
 #    RepoCommitsView:repo_commit_patch
 #    RepoCommitsView:repo_commit_raw
 #    RepoCommitsView:repo_commit_raw@TOKEN
 #    RepoFilesView:repo_files_diff
 #    RepoFilesView:repo_archivefile
 #    RepoFilesView:repo_file_raw
 #    GistView:*
 api_access_controllers_whitelist =
 
 ## Default encoding used to convert from and to unicode
 ## can be also a comma separated list of encoding in case of mixed encodings
 default_encoding = UTF-8
 
 ## instance-id prefix
 ## a prefix key for this instance used for cache invalidation when running
 ## multiple instances of RhodeCode, make sure it's globally unique for
 ## all running RhodeCode instances. Leave empty if you don't use it
 instance_id =
 
 ## Fallback authentication plugin. Set this to a plugin ID to force the usage
 ## of an authentication plugin also if it is disabled by it's settings.
 ## This could be useful if you are unable to log in to the system due to broken
 ## authentication settings. Then you can enable e.g. the internal RhodeCode auth
 ## module to log in again and fix the settings.
 ##
 ## Available builtin plugin IDs (hash is part of the ID):
 ## egg:rhodecode-enterprise-ce#rhodecode
 ## egg:rhodecode-enterprise-ce#pam
 ## egg:rhodecode-enterprise-ce#ldap
 ## egg:rhodecode-enterprise-ce#jasig_cas
 ## egg:rhodecode-enterprise-ce#headers
 ## egg:rhodecode-enterprise-ce#crowd
 #rhodecode.auth_plugin_fallback = egg:rhodecode-enterprise-ce#rhodecode
 
 ## alternative return HTTP header for failed authentication. Default HTTP
 ## response is 401 HTTPUnauthorized. Currently HG clients have troubles with
 ## handling that causing a series of failed authentication calls.
 ## Set this variable to 403 to return HTTPForbidden, or any other HTTP code
 ## This will be served instead of default 401 on bad authentication
 auth_ret_code =
 
 ## use special detection method when serving auth_ret_code, instead of serving
 ## ret_code directly, use 401 initially (Which triggers credentials prompt)
 ## and then serve auth_ret_code to clients
 auth_ret_code_detection = false
 
 ## locking return code. When repository is locked return this HTTP code. 2XX
 ## codes don't break the transactions while 4XX codes do
 lock_ret_code = 423
 
 ## allows to change the repository location in settings page
 allow_repo_location_change = true
 
 ## allows to setup custom hooks in settings page
 allow_custom_hooks_settings = true
 
 ## Generated license token required for EE edition license.
 ## New generated token value can be found in Admin > settings > license page.
 license_token =
 
 ## supervisor connection uri, for managing supervisor and logs.
 supervisor.uri =
 ## supervisord group name/id we only want this RC instance to handle
 supervisor.group_id = prod
 
 ## Display extended labs settings
 labs_settings_active = true
 
 ## Custom exception store path, defaults to TMPDIR
 ## This is used to store exception from RhodeCode in shared directory
 #exception_tracker.store_path =
 
 ## File store configuration. This is used to store and serve uploaded files
 file_store.enabled = true
 ## Storage backend, available options are: local
 file_store.backend = local
 ## path to store the uploaded binaries
 file_store.storage_path = %(here)s/data/file_store
 
 
 ####################################
 ###        CELERY CONFIG        ####
 ####################################
 ## run: /path/to/celery worker \
 ##     -E --beat --app rhodecode.lib.celerylib.loader \
 ##     --scheduler rhodecode.lib.celerylib.scheduler.RcScheduler \
 ##     --loglevel DEBUG --ini /path/to/rhodecode.ini
 
 use_celery = false
 
 ## connection url to the message broker (default rabbitmq)
 celery.broker_url = amqp://rabbitmq:qweqwe@localhost:5672/rabbitmqhost
 
 ## maximum tasks to execute before worker restart
 celery.max_tasks_per_child = 100
 
 ## tasks will never be sent to the queue, but executed locally instead.
 celery.task_always_eager = false
 
 #####################################
 ###         DOGPILE CACHE        ####
 #####################################
 ## Default cache dir for caches.  Putting this into a ramdisk
 ## can boost performance, eg. /tmpfs/data_ramdisk, however this directory might require
 ## large amount of space
 cache_dir = %(here)s/data
 
 ## `cache_perms` cache settings for permission tree, auth TTL.
 rc_cache.cache_perms.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_perms.expiration_time = 300
 
 ## alternative `cache_perms` redis backend with distributed lock
 #rc_cache.cache_perms.backend = dogpile.cache.rc.redis
 #rc_cache.cache_perms.expiration_time = 300
 ## redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_perms.arguments.redis_expiration_time = 7200
 #rc_cache.cache_perms.arguments.socket_timeout = 30
 #rc_cache.cache_perms.arguments.host = localhost
 #rc_cache.cache_perms.arguments.port = 6379
 #rc_cache.cache_perms.arguments.db = 0
 ## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_perms.arguments.distributed_lock = true
 
 ## `cache_repo` cache settings for FileTree, Readme, RSS FEEDS
 rc_cache.cache_repo.backend = dogpile.cache.rc.file_namespace
 rc_cache.cache_repo.expiration_time = 2592000
 
 ## alternative `cache_repo` redis backend with distributed lock
 #rc_cache.cache_repo.backend = dogpile.cache.rc.redis
 #rc_cache.cache_repo.expiration_time = 2592000
 ## redis_expiration_time needs to be greater then expiration_time
 #rc_cache.cache_repo.arguments.redis_expiration_time = 2678400
 #rc_cache.cache_repo.arguments.socket_timeout = 30
 #rc_cache.cache_repo.arguments.host = localhost
 #rc_cache.cache_repo.arguments.port = 6379
 #rc_cache.cache_repo.arguments.db = 1
 ## more Redis options: https://dogpilecache.sqlalchemy.org/en/latest/api.html#redis-backends
 #rc_cache.cache_repo.arguments.distributed_lock = true
 
 ## cache settings for SQL queries, this needs to use memory type backend
 rc_cache.sql_cache_short.backend = dogpile.cache.rc.memory_lru
 rc_cache.sql_cache_short.expiration_time = 30
 
 ## `cache_repo_longterm` cache for repo object instances, this needs to use memory
 ## type backend as the objects kept are not pickle serializable
 rc_cache.cache_repo_longterm.backend = dogpile.cache.rc.memory_lru
 ## by default we use 96H, this is using invalidation on push anyway
 rc_cache.cache_repo_longterm.expiration_time = 345600
 ## max items in LRU cache, reduce this number to save memory, and expire last used
 ## cached objects
 rc_cache.cache_repo_longterm.max_size = 10000
 
 
 ####################################
 ###       BEAKER SESSION        ####
 ####################################
 
 ## .session.type is type of storage options for the session, current allowed
 ## types are file, ext:memcached, ext:redis, ext:database, and memory (default).
 beaker.session.type = file
 beaker.session.data_dir = %(here)s/data/sessions
 
+## redis sessions
+#beaker.session.type = ext:redis
+#beaker.session.url = redis://127.0.0.1:6379/2
+
 ## db based session, fast, and allows easy management over logged in users
 #beaker.session.type = ext:database
 #beaker.session.table_name = db_session
 #beaker.session.sa.url = postgresql://postgres:secret@localhost/rhodecode
 #beaker.session.sa.url = mysql://root:secret@127.0.0.1/rhodecode
 #beaker.session.sa.pool_recycle = 3600
 #beaker.session.sa.echo = false
 
 beaker.session.key = rhodecode
 beaker.session.secret = production-rc-uytcxaz
 beaker.session.lock_dir = %(here)s/data/sessions/lock
 
 ## Secure encrypted cookie. Requires AES and AES python libraries
 ## you must disable beaker.session.secret to use this
 #beaker.session.encrypt_key = key_for_encryption
 #beaker.session.validate_key = validation_key
 
 ## sets session as invalid(also logging out user) if it haven not been
 ## accessed for given amount of time in seconds
 beaker.session.timeout = 2592000
 beaker.session.httponly = true
 ## Path to use for the cookie. Set to prefix if you use prefix middleware
 #beaker.session.cookie_path = /custom_prefix
 
 ## uncomment for https secure cookie
 beaker.session.secure = false
 
 ## auto save the session to not to use .save()
 beaker.session.auto = false
 
 ## default cookie expiration time in seconds, set to `true` to set expire
 ## at browser close
 #beaker.session.cookie_expires = 3600
 
 ###################################
 ## SEARCH INDEXING CONFIGURATION ##
 ###################################
 ## Full text search indexer is available in rhodecode-tools under
 ## `rhodecode-tools index` command
 
 ## WHOOSH Backend, doesn't require additional services to run
 ## it works good with few dozen repos
 search.module = rhodecode.lib.index.whoosh
 search.location = %(here)s/data/index
 
 ########################################
 ###    CHANNELSTREAM CONFIG         ####
 ########################################
 ## channelstream enables persistent connections and live notification
 ## in the system. It's also used by the chat system
 
 channelstream.enabled = false
 
 ## server address for channelstream server on the backend
 channelstream.server = 127.0.0.1:9800
 
 ## location of the channelstream server from outside world
 ## use ws:// for http or wss:// for https. This address needs to be handled
 ## by external HTTP server such as Nginx or Apache
 ## see Nginx/Apache configuration examples in our docs
 channelstream.ws_url = ws://rhodecode.yourserver.com/_channelstream
 channelstream.secret = secret
 channelstream.history.location = %(here)s/channelstream_history
 
 ## Internal application path that Javascript uses to connect into.
 ## If you use proxy-prefix the prefix should be added before /_channelstream
 channelstream.proxy_path = /_channelstream
 
 
 ###################################
 ##       APPENLIGHT CONFIG       ##
 ###################################
 
 ## Appenlight is tailored to work with RhodeCode, see
 ## http://appenlight.com for details how to obtain an account
 
 ## Appenlight integration enabled
 appenlight = false
 
 appenlight.server_url = https://api.appenlight.com
 appenlight.api_key = YOUR_API_KEY
 #appenlight.transport_config = https://api.appenlight.com?threaded=1&timeout=5
 
 ## used for JS client
 appenlight.api_public_key = YOUR_API_PUBLIC_KEY
 
 ## TWEAK AMOUNT OF INFO SENT HERE
 
 ## enables 404 error logging (default False)
 appenlight.report_404 = false
 
 ## time in seconds after request is considered being slow (default 1)
 appenlight.slow_request_time = 1
 
 ## record slow requests in application
 ## (needs to be enabled for slow datastore recording and time tracking)
 appenlight.slow_requests = true
 
 ## enable hooking to application loggers
 appenlight.logging = true
 
 ## minimum log level for log capture
 appenlight.logging.level = WARNING
 
 ## send logs only from erroneous/slow requests
 ## (saves API quota for intensive logging)
 appenlight.logging_on_error = false
 
 ## list of additional keywords that should be grabbed from environ object
 ## can be string with comma separated list of words in lowercase
 ## (by default client will always send following info:
 ## 'REMOTE_USER', 'REMOTE_ADDR', 'SERVER_NAME', 'CONTENT_TYPE' + all keys that
 ## start with HTTP* this list be extended with additional keywords here
 appenlight.environ_keys_whitelist =
 
 ## list of keywords that should be blanked from request object
 ## can be string with comma separated list of words in lowercase
 ## (by default client will always blank keys that contain following words
 ## 'password', 'passwd', 'pwd', 'auth_tkt', 'secret', 'csrf'
 ## this list be extended with additional keywords set here
 appenlight.request_keys_blacklist =
 
 ## list of namespaces that should be ignores when gathering log entries
 ## can be string with comma separated list of namespaces
 ## (by default the client ignores own entries: appenlight_client.client)
 appenlight.log_namespace_blacklist =
 
 
 ###########################################
 ###   MAIN RHODECODE DATABASE CONFIG    ###
 ###########################################
 #sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db?timeout=30
 #sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
 #sqlalchemy.db1.url = mysql://root:qweqwe@localhost/rhodecode?charset=utf8
 # pymysql is an alternative driver for MySQL, use in case of problems with default one
 #sqlalchemy.db1.url = mysql+pymysql://root:qweqwe@localhost/rhodecode
 
 sqlalchemy.db1.url = postgresql://postgres:qweqwe@localhost/rhodecode
 
 # see sqlalchemy docs for other advanced settings
 
 ## print the sql statements to output
 sqlalchemy.db1.echo = false
 ## recycle the connections after this amount of seconds
 sqlalchemy.db1.pool_recycle = 3600
 sqlalchemy.db1.convert_unicode = true
 
 ## the number of connections to keep open inside the connection pool.
 ## 0 indicates no limit
 #sqlalchemy.db1.pool_size = 5
 
 ## the number of connections to allow in connection pool "overflow", that is
 ## connections that can be opened above and beyond the pool_size setting,
 ## which defaults to five.
 #sqlalchemy.db1.max_overflow = 10
 
 ## Connection check ping, used to detect broken database connections
 ## could be enabled to better handle cases if MySQL has gone away errors
 #sqlalchemy.db1.ping_connection = true
 
 ##################
 ### VCS CONFIG ###
 ##################
 vcs.server.enable = true
 vcs.server = localhost:9900
 
 ## Web server connectivity protocol, responsible for web based VCS operations
 ## Available protocols are:
 ## `http` - use http-rpc backend (default)
 vcs.server.protocol = http
 
 ## Push/Pull operations protocol, available options are:
 ## `http` - use http-rpc backend (default)
 vcs.scm_app_implementation = http
 
 ## Push/Pull operations hooks protocol, available options are:
 ## `http` - use http-rpc backend (default)
 vcs.hooks.protocol = http
 
 ## Host on which this instance is listening for hooks. If vcsserver is in other location
 ## this should be adjusted.
 vcs.hooks.host = 127.0.0.1
 
 vcs.server.log_level = info
 ## Start VCSServer with this instance as a subprocess, useful for development
 vcs.start_server = false
 
 ## List of enabled VCS backends, available options are:
 ## `hg`  - mercurial
 ## `git` - git
 ## `svn` - subversion
 vcs.backends = hg, git, svn
 
 vcs.connection_timeout = 3600
 ## Compatibility version when creating SVN repositories. Defaults to newest version when commented out.
 ## Available options are: pre-1.4-compatible, pre-1.5-compatible, pre-1.6-compatible, pre-1.8-compatible, pre-1.9-compatible
 #vcs.svn.compatible_version = pre-1.8-compatible
 
 
 ############################################################
 ### Subversion proxy support (mod_dav_svn)               ###
 ### Maps RhodeCode repo groups into SVN paths for Apache ###
 ############################################################
 ## Enable or disable the config file generation.
 svn.proxy.generate_config = false
 ## Generate config file with `SVNListParentPath` set to `On`.
 svn.proxy.list_parent_path = true
 ## Set location and file name of generated config file.
 svn.proxy.config_file_path = %(here)s/mod_dav_svn.conf
 ## alternative mod_dav config template. This needs to be a mako template
 #svn.proxy.config_template = ~/.rccontrol/enterprise-1/custom_svn_conf.mako
 ## Used as a prefix to the `Location` block in the generated config file.
 ## In most cases it should be set to `/`.
 svn.proxy.location_root = /
 ## Command to reload the mod dav svn configuration on change.
 ## Example: `/etc/init.d/apache2 reload` or /home/USER/apache_reload.sh
 ## Make sure user who runs RhodeCode process is allowed to reload Apache
 #svn.proxy.reload_cmd = /etc/init.d/apache2 reload
 ## If the timeout expires before the reload command finishes, the command will
 ## be killed. Setting it to zero means no timeout. Defaults to 10 seconds.
 #svn.proxy.reload_timeout = 10
 
 ############################################################
 ### SSH Support Settings                                 ###
 ############################################################
 
 ## Defines if a custom authorized_keys file should be created and written on
 ## any change user ssh keys. Setting this to false also disables possibility
 ## of adding SSH keys by users from web interface. Super admins can still
 ## manage SSH Keys.
 ssh.generate_authorized_keyfile = false
 
 ## Options for ssh, default is `no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding`
 # ssh.authorized_keys_ssh_opts =
 
 ## Path to the authorized_keys file where the generate entries are placed.
 ## It is possible to have multiple key files specified in `sshd_config` e.g.
 ## AuthorizedKeysFile %h/.ssh/authorized_keys %h/.ssh/authorized_keys_rhodecode
 ssh.authorized_keys_file_path = ~/.ssh/authorized_keys_rhodecode
 
 ## Command to execute the SSH wrapper. The binary is available in the
 ## RhodeCode installation directory.
 ## e.g ~/.rccontrol/community-1/profile/bin/rc-ssh-wrapper
 ssh.wrapper_cmd = ~/.rccontrol/community-1/rc-ssh-wrapper
 
 ## Allow shell when executing the ssh-wrapper command
 ssh.wrapper_cmd_allow_shell = false
 
 ## Enables logging, and detailed output send back to the client during SSH
 ## operations. Useful for debugging, shouldn't be used in production.
 ssh.enable_debug_logging = false
 
 ## Paths to binary executable, by default they are the names, but we can
 ## override them if we want to use a custom one
 ssh.executable.hg = ~/.rccontrol/vcsserver-1/profile/bin/hg
 ssh.executable.git = ~/.rccontrol/vcsserver-1/profile/bin/git
 ssh.executable.svn = ~/.rccontrol/vcsserver-1/profile/bin/svnserve
 
 ## Enables SSH key generator web interface. Disabling this still allows users
 ## to add their own keys.
 ssh.enable_ui_key_generator = true
 
 
 ## Dummy marker to add new entries after.
 ## Add any custom entries below. Please don't remove.
 custom.conf = 1
 
 
 ################################
 ### LOGGING CONFIGURATION   ####
 ################################
 [loggers]
 keys = root, sqlalchemy, beaker, celery, rhodecode, ssh_wrapper
 
 [handlers]
 keys = console, console_sql
 
 [formatters]
 keys = generic, color_formatter, color_formatter_sql
 
 #############
 ## LOGGERS ##
 #############
 [logger_root]
 level = NOTSET
 handlers = console
 
 [logger_sqlalchemy]
 level = INFO
 handlers = console_sql
 qualname = sqlalchemy.engine
 propagate = 0
 
 [logger_beaker]
 level = DEBUG
 handlers =
 qualname = beaker.container
 propagate = 1
 
 [logger_rhodecode]
 level = DEBUG
 handlers =
 qualname = rhodecode
 propagate = 1
 
 [logger_ssh_wrapper]
 level = DEBUG
 handlers =
 qualname = ssh_wrapper
 propagate = 1
 
 [logger_celery]
 level = DEBUG
 handlers =
 qualname = celery
 
 
 ##############
 ## HANDLERS ##
 ##############
 
 [handler_console]
 class = StreamHandler
 args = (sys.stderr, )
 level = INFO
 formatter = generic
 
 [handler_console_sql]
 # "level = DEBUG" logs SQL queries and results.
 # "level = INFO" logs SQL queries.
 # "level = WARN" logs neither.  (Recommended for production systems.)
 class = StreamHandler
 args = (sys.stderr, )
 level = WARN
 formatter = generic
 
 ################
 ## FORMATTERS ##
 ################
 
 [formatter_generic]
 class = rhodecode.lib.logging_formatter.ExceptionAwareFormatter
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter]
 class = rhodecode.lib.logging_formatter.ColorFormatter
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter_sql]
 class = rhodecode.lib.logging_formatter.ColorFormatterSql
 format = %(asctime)s.%(msecs)03d [%(process)d] %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S

default.nix

@@ -1,296 +1,296 b''
 # Nix environment for the community edition
 #
 # This shall be as lean as possible, just producing the enterprise-ce
 # derivation. For advanced tweaks to pimp up the development environment we use
 # "shell.nix" so that it does not have to clutter this file.
 #
 # Configuration, set values in "~/.nixpkgs/config.nix".
 # example
 #  {
 #    # Thoughts on how to configure the dev environment
 #    rc = {
 #      codeInternalUrl = "https://usr:[e-mail]/internal";
 #      sources = {
 #        rhodecode-vcsserver = "/home/user/work/rhodecode-vcsserver";
 #        rhodecode-enterprise-ce = "/home/user/work/rhodecode-enterprise-ce";
 #        rhodecode-enterprise-ee = "/home/user/work/rhodecode-enterprise-ee";
 #      };
 #    };
 #  }
 
 args@
 { system ? builtins.currentSystem
 , pythonPackages ? "python27Packages"
 , pythonExternalOverrides ? self: super: {}
 , doCheck ? false
 , ...
 }:
 
 let
   pkgs_ = args.pkgs or (import <nixpkgs> { inherit system; });
 in
 
 let
   pkgs = import <nixpkgs> {
     overlays = [
       (import ./pkgs/overlays.nix)
     ];
     inherit
       (pkgs_)
       system;
   };
 
   # Works with the new python-packages, still can fallback to the old
   # variant.
   basePythonPackagesUnfix = basePythonPackages.__unfix__ or (
     self: basePythonPackages.override (a: { inherit self; }));
 
   # Evaluates to the last segment of a file system path.
   basename = path: with pkgs.lib; last (splitString "/" path);
 
   # source code filter used as arugment to builtins.filterSource.
   src-filter = path: type: with pkgs.lib;
     let
       ext = last (splitString "." path);
     in
       !builtins.elem (basename path) [
         ".git" ".hg" "__pycache__" ".eggs" ".idea" ".dev"
         "node_modules" "node_binaries"
         "build" "data" "result" "tmp"] &&
       !builtins.elem ext ["egg-info" "pyc"] &&
       # TODO: johbo: This check is wrong, since "path" contains an absolute path,
       # it would still be good to restore it since we want to ignore "result-*".
       !hasPrefix "result" path;
 
   sources =
     let
       inherit
         (pkgs.lib)
         all
         isString
         attrValues;
       sourcesConfig = pkgs.config.rc.sources or {};
     in
       # Ensure that sources are configured as strings. Using a path
       # would result in a copy into the nix store.
       assert all isString (attrValues sourcesConfig);
       sourcesConfig;
 
   version = builtins.readFile "${rhodecode-enterprise-ce-src}/rhodecode/VERSION";
   rhodecode-enterprise-ce-src = builtins.filterSource src-filter ./.;
 
   nodeEnv = import ./pkgs/node-default.nix {
     inherit
       pkgs
       system;
   };
   nodeDependencies = nodeEnv.shell.nodeDependencies;
 
   rhodecode-testdata-src = sources.rhodecode-testdata or (
     pkgs.fetchhg {
       url = "https://code.rhodecode.com/upstream/rc_testdata";
       rev = "v0.10.0";
       sha256 = "0zn9swwvx4vgw4qn8q3ri26vvzgrxn15x6xnjrysi1bwmz01qjl0";
   });
 
   rhodecode-testdata = import "${rhodecode-testdata-src}/default.nix" {
   inherit
     doCheck
     pkgs
     pythonPackages;
   };
 
   pythonLocalOverrides = self: super: {
     rhodecode-enterprise-ce =
       let
         linkNodePackages = ''
           export RHODECODE_CE_PATH=${rhodecode-enterprise-ce-src}
 
           echo "[BEGIN]: Link node packages and binaries"
           # johbo: Linking individual packages allows us to run "npm install"
           # inside of a shell to try things out. Re-entering the shell will
           # restore a clean environment.
           rm -fr node_modules
           mkdir node_modules
           ln -s ${nodeDependencies}/lib/node_modules/* node_modules/
           export NODE_PATH=./node_modules
 
           rm -fr node_binaries
           mkdir node_binaries
           ln -s ${nodeDependencies}/bin/* node_binaries/
           echo "[DONE ]: Link node packages and binaries"
         '';
 
         releaseName = "RhodeCodeEnterpriseCE-${version}";
       in super.rhodecode-enterprise-ce.override (attrs: {
       inherit
         doCheck
         version;
 
       name = "rhodecode-enterprise-ce-${version}";
       releaseName = releaseName;
       src = rhodecode-enterprise-ce-src;
       dontStrip = true; # prevent strip, we don't need it.
 
       # expose following attributed outside
       passthru = {
         inherit
           rhodecode-testdata
           linkNodePackages
           myPythonPackagesUnfix
           pythonLocalOverrides
           pythonCommunityOverrides;
 
         pythonPackages = self;
       };
 
       buildInputs =
         attrs.buildInputs or [] ++ [
           rhodecode-testdata
         ];
 
       #NOTE: option to inject additional propagatedBuildInputs
       propagatedBuildInputs =
         attrs.propagatedBuildInputs or [] ++ [
 
         ];
 
       LC_ALL = "en_US.UTF-8";
       LOCALE_ARCHIVE =
         if pkgs.stdenv.isLinux
         then "${pkgs.glibcLocales}/lib/locale/locale-archive"
         else "";
 
       # Add bin directory to path so that tests can find 'rhodecode'.
       preCheck = ''
         export PATH="$out/bin:$PATH"
       '';
 
       # custom check phase for testing
       checkPhase = ''
         runHook preCheck
         PYTHONHASHSEED=random py.test -vv -p no:sugar -r xw --cov-config=.coveragerc --cov=rhodecode --cov-report=term-missing rhodecode
         runHook postCheck
       '';
 
       postCheck = ''
         echo "Cleanup of rhodecode/tests"
         rm -rf $out/lib/${self.python.libPrefix}/site-packages/rhodecode/tests
       '';
 
       preBuild = ''
         echo "[BEGIN]: Building frontend assets"
         ${linkNodePackages}
         make web-build
         rm -fr node_modules
         rm -fr node_binaries
         echo "[DONE ]: Building frontend assets"
       '';
 
       postInstall = ''
         # check required files
         STATIC_CHECK="/robots.txt /502.html
                       /js/scripts.js /js/rhodecode-components.js
-                      /css/style.css /css/style-polymer.css"
+                      /css/style.css /css/style-polymer.css /css/style-ipython.css"
 
         for file in $STATIC_CHECK;
         do
             if [ ! -f rhodecode/public/$file ]; then
               echo "Missing $file"
               exit 1
             fi
         done
 
         echo "Writing enterprise-ce meta information for rccontrol to nix-support/rccontrol"
         mkdir -p $out/nix-support/rccontrol
         cp -v rhodecode/VERSION $out/nix-support/rccontrol/version
         echo "[DONE ]: enterprise-ce meta information for rccontrol written"
 
         mkdir -p $out/etc
         cp configs/production.ini $out/etc
         echo "[DONE ]: saved enterprise-ce production.ini into $out/etc"
 
         cp -Rf rhodecode/config/rcextensions $out/etc/rcextensions.tmpl
         echo "[DONE ]: saved enterprise-ce rcextensions into $out/etc/rcextensions.tmpl"
 
         # python based programs need to be wrapped
         mkdir -p $out/bin
 
         # required binaries from dependencies
         ln -s ${self.supervisor}/bin/supervisorctl $out/bin/
         ln -s ${self.supervisor}/bin/supervisord $out/bin/
         ln -s ${self.pastescript}/bin/paster $out/bin/
         ln -s ${self.channelstream}/bin/channelstream $out/bin/
         ln -s ${self.celery}/bin/celery $out/bin/
         ln -s ${self.gunicorn}/bin/gunicorn $out/bin/
         ln -s ${self.pyramid}/bin/prequest $out/bin/
         ln -s ${self.pyramid}/bin/pserve $out/bin/
 
         echo "[DONE ]: created symlinks into $out/bin"
         DEPS="$out/bin/supervisorctl \
               $out/bin/supervisord \
               $out/bin/paster \
               $out/bin/channelstream \
               $out/bin/celery \
               $out/bin/gunicorn \
               $out/bin/prequest \
               $out/bin/pserve"
 
         # wrap only dependency scripts, they require to have full PYTHONPATH set
         # to be able to import all packages
         for file in $DEPS;
         do
           wrapProgram $file \
             --prefix PATH : $PATH \
             --prefix PYTHONPATH : $PYTHONPATH \
             --set PYTHONHASHSEED random
         done
 
         echo "[DONE ]: enterprise-ce binary wrapping"
 
         # rhodecode-tools don't need wrapping
         ln -s ${self.rhodecode-tools}/bin/rhodecode-* $out/bin/
 
         # expose sources of CE
         ln -s $out $out/etc/rhodecode_enterprise_ce_source
 
         # expose static files folder
         cp -Rf $out/lib/${self.python.libPrefix}/site-packages/rhodecode/public/ $out/etc/static
         chmod 755 -R $out/etc/static
 
       '';
     });
 
   };
 
   basePythonPackages = with builtins;
     if isAttrs pythonPackages then
       pythonPackages
     else
       getAttr pythonPackages pkgs;
 
   pythonGeneratedPackages = import ./pkgs/python-packages.nix {
     inherit
       pkgs;
     inherit
       (pkgs)
       fetchurl
       fetchgit
       fetchhg;
   };
 
   pythonCommunityOverrides = import ./pkgs/python-packages-overrides.nix {
     inherit pkgs basePythonPackages;
   };
 
   # Apply all overrides and fix the final package set
   myPythonPackagesUnfix = with pkgs.lib;
     (extends pythonExternalOverrides
     (extends pythonLocalOverrides
     (extends pythonCommunityOverrides
     (extends pythonGeneratedPackages
              basePythonPackagesUnfix))));
 
   myPythonPackages = (pkgs.lib.fix myPythonPackagesUnfix);
 
 in myPythonPackages.rhodecode-enterprise-ce

docs/admin/repo_admin/repo-extra-fields.rst

@@ -1,77 +1,77 b''
 .. _repo-xtra:
 
 Repository Extra Fields
 =======================
 
 Extra fields attached to a |repo| allow you to configure additional fields for
 each repository. This allows storing custom data per-repository.
 
 It can be used in :ref:`integrations-webhook` or in |RCX|.
 To read more about |RCX|, see the :ref:`integrations-rcextensions` section.
 
 
 Enabling Extra Fields
 ---------------------
 
 To enable extra fields on |repos|, use the following steps:
 
 1. Go to the :menuselection:`Admin --> Settings --> Visual` page.
 2. Check the :guilabel:`Use repository extra fields` box.
 3. Save your changes.
 
 
 Configuring Extra Fields
 ------------------------
 
 To configure extra fields per repository, use the following steps:
 
 1. Go to :menuselection:`Admin --> Repositories` and select :guilabel:`Edit`
    beside the |repo| to which you wish to add extra fields.
 2. On the |repo| settings page, select the :guilabel:`Extra fields` tab.
 
-.. image:: ../images/extra-repo-fields.png
+.. image:: ../../images/extra-repo-fields.png
 
 The most important is the `New field key` variable which under the value will
 be stored. It needs to be unique for each repository. The label and description
 will be generated in repository settings where users can actually save some
 values inside generated extra fields.
 
 
 Example Usage in extensions
 ---------------------------
 
 To use the extra fields in an extension, see the example below. For more
 information and examples, see the :ref:`extensions-hooks-ref` section.
 
 .. code-block:: python
 
     call = load_extension('http_notify.py')
     if call:
         url = 'http://default.url' # <url for post data>
 
         # possibly extract the URL from extra fields
         call = load_extension('extra_fields.py')
         if call:
             repo_extra_fields = call(**kwargs)
             # now update if we have extra fields, they have precedence
             # this way users can store any configuration inside the database per
             # repo
             for key, data in repo_extra_fields.items():
                 kwargs[key] = data['field_value']
 
         # an endpoint url data will be sent to, fetched from extra fields
         # if exists, or fallback to default
         kwargs['URL'] = kwargs.pop('webhook_url', None) or url
 
         # fetch pushed commits, from commit_ids list
         call = load_extension('extract_commits.py')
         extracted_commits = {}
         if call:
             extracted_commits = call(**kwargs)
             # store the commits for the next call chain
         kwargs['COMMITS'] = extracted_commits
 
         # set additional keys and values to be sent via POST to given URL
         kwargs['caller_type'] = 'rhodecode'
         kwargs['date'] = time.time() # import time before
         call(**kwargs)

docs/admin/system-admin.rst

@@ -1,34 +1,32 b''
 .. _rhodecode-admin-ref:
 
 System Administration
 =====================
 
 The following are the most common system administration tasks.
 
 .. only:: latex
 
     * :ref:`vcs-server`
     * :ref:`apache-ws-ref`
     * :ref:`nginx-ws-ref`
     * :ref:`rhodecode-tuning-ref`
     * :ref:`indexing-ref`
     * :ref:`rhodecode-reset-ref`
 
 .. toctree::
 
-    config-files-overview
-    vcs-server
-    svn-http
-    svn-path-permissions
-    gunicorn-ssl-support
-    apache-config
-    nginx-config
-    backup-restore
-    tuning-rhodecode
-    indexing
-    reset-information
-    enable-debug
-    admin-tricks
-    cleanup-cmds
-    restore-deleted-repositories
-
+    system_admin/config-files-overview
+    system_admin/vcs-server
+    system_admin/svn-http
+    system_admin/svn-path-permissions
+    system_admin/gunicorn-ssl-support
+    system_admin/apache-config
+    system_admin/nginx-config
+    system_admin/backup-restore
+    system_admin/tuning-rhodecode
+    system_admin/indexing
+    system_admin/reset-information
+    system_admin/enable-debug
+    system_admin/admin-tricks
+    system_admin/cleanup-cmds

docs/admin/system_admin/admin-tricks.rst

@@ -1,241 +1,264 b''
 .. _admin-tricks:
 
 One-time Admin Tasks
 --------------------
 
 * :ref:`web-analytics`
 * :ref:`admin-tricks-license`
 * :ref:`announcements`
 * :ref:`md-rst`
 * :ref:`repo-stats`
 * :ref:`server-side-merge`
 * :ref:`remap-rescan`
 * :ref:`custom-hooks`
 * :ref:`clear-repo-cache`
 * :ref:`set-repo-pub`
 * :ref:`ping`
 
 .. _web-analytics:
 
 Adding Web Analytics
 ^^^^^^^^^^^^^^^^^^^^
 
 If you wish to add a Google Analytics, or any other kind of tracker to your
 |RCE| instance you can add the necessary codes to the header or footer
 section of each instance using the following steps:
 
 1. From the |RCE| interface, select
    :menuselection:`Admin --> Settings --> Global`
 2. To add a tracking code to you instance, enter it in the header or footer
    section and select **Save**
 
 Use the example templates in the drop-down menu to set up your configuration.
 
 .. _admin-tricks-license:
 
 Licence Key Management
 ^^^^^^^^^^^^^^^^^^^^^^
 
 To manage your license key, go to
 :menuselection:`Admin --> Settings --> License`.
 On this page you can see the license key details. If you need a new license,
 or have questions about your current one, contact [e-mail]
 
 .. _announcements:
 
 Server-wide Announcements
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
 If you need to make a server-wide announcement to all users,
 you can add a message to be displayed using the following steps:
 
 1. From the |RCE| interface, select
    :menuselection:`Admin --> Settings --> Global`
 2. To add a message that will be displayed to all users,
    select :guilabel:`Server Announcement` from the drop-down menu and
    change the ``var message = "TYPE YOUR MESSAGE HERE";`` example line.
 3. Select :guilabel:`Save`, and you will see the message once your page
    refreshes.
 
-.. image:: ../images/server-wide-announcement.png
+.. image:: ../../images/server-wide-announcement.png
    :alt: Server Wide Announcement
 
 .. _md-rst:
 
 
 Suppress license warnings or errors
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 In case you're running on maximum allowed users, RhodeCode will display a
 warning message on pages that you're close to the license limits.
 It's often not desired to show that all the time. Here's how you can suppress
 the license messages.
 
 1. From the |RCE| interface, select
    :menuselection:`Admin --> Settings --> Global`
 2. Select :guilabel:`Flash message filtering` from the drop-down menu.
 3. Select :guilabel:`Save`, and you will no longer see the license message
    once your page refreshes.
 
 .. _admin-tricks-suppress-license-messages:
 
 
 Markdown or RST Rendering
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
 |RCE| can use `Markdown`_ or `reStructured Text`_ in commit message,
 code review messages, and inline comments. To set the default to either,
 select your preference from the drop-down menu on the
 :menuselection:`Admin --> Settings --> Visual` page and select
 :guilabel:`Save settings`.
 
 .. _repo-stats:
 
 Enabling Repository Statistics
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To enable |repo| statistics, use the following steps:
 
 1. From the |RCE| interface, open
    :menuselection:`Admin --> Repositories` and select
    :guilabel:`Edit` beside the |repo| for which you wish to enable statistics.
 2. Check the :guilabel:`Enable statistics` box, and select :guilabel:`Save`
 
 .. _server-side-merge:
 
 Enabling Server-side Merging
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To enable server-side merging, use the following steps:
 
 1. From the |RCE| interface, open :menuselection:`Admin --> Settings --> VCS`
 2. Check the :guilabel:`Server-side merge` box, and select
    :guilabel:`Save Settings`
 
 If you encounter slow performance with server-side merging enabled, check the
 speed at which your server is performing actions. When server-side merging is
 enabled, the following actions occurs on the server.
 
 * A |pr| is created in the database.
 * A shadow |repo| is created as a working environment for the |pr|.
 * On display, |RCE| checks if the |pr| can be merged.
 
 To check how fast the shadow |repo| creation is occurring on your server, use
 the following steps:
 
 1. Log into your server and create a directory in your |repos| folder.
 2. Clone a |repo| that is showing slow performance and time the action.
 
 .. code-block:: bash
 
    # One option is to use the time command
    $ time hg clone SOURCE_REPO TARGET
 
 .. _remap-rescan:
 
 Remap and Rescan Repositories
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 You may want to Remap and rescan the |repos| that |RCE| is managing to ensure
 the system is always up-to-date. This is useful after importing, deleting,
 or carrying out general cleaning up operations. To do this use the
 following steps:
 
 1. From the |RCE|, open
    :menuselection:`Admin --> Settings --> Remap and rescan`
 2. Click :guilabel:`Rescan Repositories`
 
 Check the additional options if needed:
 
 * :guilabel:`Destroy old data`: Useful for purging deleted repository
   information from the database.
 * :guilabel:`Invalidate cache for all repositories`: Use this to completely
   remap all |repos|. Useful when importing or migrating |repos| to ensure all
   new information is picked up.
 
 .. _custom-hooks:
 
 Adding Custom Hooks
 ^^^^^^^^^^^^^^^^^^^
 
 To add custom hooks to your instance, use the following steps:
 
 1. Open :menuselection:`Admin --> Settings --> Hooks`
 2. Add your custom hook details, you can use a file path to specify custom
    hook scripts, for example:
    ``pretxnchangegroup.example`` with value ``python:/path/to/custom_hook.py:my_func_name``
 3. Select :guilabel:`Save`
 
 Also, see the RhodeCode Extensions section of the :ref:`rc-tools` guide. RhodeCode
 Extensions can be used to add additional hooks to your instance and comes
 with a number of pre-built plugins if you chose to install them.
 
 .. _clear-repo-cache:
 
 Clearing |repo| cache
 ^^^^^^^^^^^^^^^^^^^^^
 
 If you need to clear the cache for a particular |repo|, use the following steps:
 
 1. Open :menuselection:`Admin --> Repositories` and select :guilabel:`Edit`
    beside the |repo| whose cache you wish to clear.
 2. On the |repo| settings page, go to the :guilabel:`Caches` tab and select
    :guilabel:`Invalidate repository cache`.
 
 .. _set-lang:
 
 Changing Default Language
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To change the default language of a |RCE| instance, change the language code
 in the :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file. To
 do this, use the following steps.
 
 1. Open the :file:`rhodecode.ini` file and set the required language code.
 
 .. code-block:: ini
 
    ## Optional Languages
    ## en(default), de, fr, it, ja, pl, pt, ru, zh
    lang = de
 
 2. Restart the |RCE| instance and check that the language has been updated.
 
 .. code-block:: bash
 
    $ rccontrol restart enterprise-2
    Instance "enterprise-2" successfully stopped.
    Instance "enterprise-2" successfully started.
 
-.. image:: ../images/language.png
+.. image:: ../../images/language.png
 
 .. _set-repo-pub:
 
 Setting Repositories to Publish
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 To automatically promote your local |repos| to public after pushing to |RCE|,
 enable the :guilabel:`Set repositories as publishing` option on the
 :menuselection:`Admin --> Settings --> VCS` page.
 
 .. note::
 
    This option is enabled by default on most |RCE| versions, but if upgrading
    from a 1.7.x version it could be disabled on upgrade due to inheriting
    older default settings.
 
 .. _ping:
 
 Pinging the |RCE| Server
 ^^^^^^^^^^^^^^^^^^^^^^^^
 
 You can check the IP Address of your |RCE| instance using the
 following URL: ``{instance-URL}/_admin/ping``.
 
 .. code-block:: bash
 
    $ curl https://your.rhodecode.url/_admin/ping
    pong[rce-7880] => 203.0.113.23
 
 .. _Markdown: http://daringfireball.net/projects/markdown/
 .. _reStructured Text: http://docutils.sourceforge.net/docs/index.html
+
+
+Unarchiving a repository
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Archive operation for the repository is similar as delete. Archive keeps the data for future references
+but makes the repository read-only. After archiving the repository it shouldn't be modified in any way.
+This is why repository settings are disabled for an archived repository.
+
+If there's a need for unarchiving a repository for some reasons, the interactive
+ishell interface should be used.
+
+.. code-block:: bash
+
+  # Open iShell from the terminal
+  $ rccontrol ishell enterprise-1/community-1
+
+.. code-block:: python
+
+  # Set repository as un-archived
+  In [1]: repo = Repository.get_by_repo_name('SOME_REPO_NAME')
+  In [2]: repo.archived = False
+  In [3]: Session().add(repo);Session().commit()

docs/admin/system_admin/apache-config.rst

@@ -1,14 +1,14 b''
 .. _apache-ws-ref:
 
 Apache HTTP Server Configuration
 --------------------------------
 
 To set up your Apache Web Server for optimal performance and security, use
 the information in the following sections.
 
 .. toctree::
 
-   apache-conf-example
-   apache-diffie-hellman
-   apache-subdirectory
-   apache-wsgi-coding
+   apache/apache-conf-example
+   apache/apache-diffie-hellman
+   apache/apache-subdirectory
+   apache/apache-wsgi-coding

docs/admin/system_admin/apache/apache-conf-example.rst

@@ -1,103 +1,107 b''
 .. _apache-conf-eg:
 
 Apache Configuration Example
 ----------------------------
 
 Use the following example to configure Apache as a your web server.
 Below config if for an Apache Reverse Proxy configuration.
 
 .. note::
 
    Apache requires the following modules to be enabled. Below is an example
    how to enable them on Ubuntu Server
 
 
 .. code-block:: bash
 
     $ sudo a2enmod proxy
     $ sudo a2enmod proxy_http
     $ sudo a2enmod proxy_balancer
     $ sudo a2enmod headers
     $ sudo a2enmod ssl
     $ sudo a2enmod rewrite
 
     # requires Apache 2.4+, required to handle websockets/channelstream
     $ sudo a2enmod proxy_wstunnel
 
 
 .. code-block:: apache
 
     ## HTTP to HTTPS rewrite
     <VirtualHost *:80>
        ServerName rhodecode.myserver.com
        DocumentRoot /var/www/html
        Redirect permanent / https://rhodecode.myserver.com/
     </VirtualHost>
 
     ## MAIN SSL enabled server
     <VirtualHost *:443>
 
         ServerName rhodecode.myserver.com
         ServerAlias rhodecode.myserver.com
 
         ## Skip ProxyPass the _static to backend server
         #ProxyPass /_static !
 
         ## serve static files by Apache, recommended for performance
         #Alias /_static/rhodecode /home/ubuntu/.rccontrol/community-1/static
 
         ## Allow Apache to access the static files in this directory
         #<Directory /home/ubuntu/.rccontrol/community-1/static/>
         #    AllowOverride none
         #    Require all granted
         #</Directory>
 
         RequestHeader set X-Forwarded-Proto "https"
 
         ## channelstream websocket handling
         ProxyPass /_channelstream ws://localhost:9800
         ProxyPassReverse /_channelstream ws://localhost:9800
 
         <Proxy *>
           Order allow,deny
           Allow from all
         </Proxy>
 
         # Directive to properly generate url (clone url) for RhodeCode
         ProxyPreserveHost On
 
+        # It allows request bodies to be sent to the backend using chunked transfer encoding.
+        SetEnv proxy-sendchunked 1
+
+        # Increase headers size for large Mercurial headers sent with many branches
+        LimitRequestLine 16380
+
         # Url to running RhodeCode instance. This is shown as `- URL:` when
         # running rccontrol status.
+
         ProxyPass / http://127.0.0.1:10002/ timeout=7200 Keepalive=On
         ProxyPassReverse / http://127.0.0.1:10002/
 
-        # Increase headers for large Mercurial headers
-        LimitRequestLine 16380
-
         # strict http prevents from https -> http downgrade
         Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains; preload"
 
         # Set x-frame options
         Header always append X-Frame-Options SAMEORIGIN
 
         # To enable https use line below
         # SetEnvIf X-Url-Scheme https HTTPS=1
 
         # SSL setup
         SSLEngine On
         SSLCertificateFile /etc/apache2/ssl/rhodecode.myserver.pem
         SSLCertificateKeyFile /etc/apache2/ssl/rhodecode.myserver.key
 
         SSLProtocol             all -SSLv2 -SSLv3
         SSLCipherSuite          ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
         SSLHonorCipherOrder     on
 
         # Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
         #SSLOpenSSLConfCmd DHParameters "/etc/apache2/dhparam.pem"
 
         ## custom 502 error page. Will be displayed while RhodeCode server
         ## is turned off
         ErrorDocument 502 /path/to/.rccontrol/enterprise-1/static/502.html
 
     </VirtualHost>
 

docs/admin/system_admin/nginx-config.rst

@@ -1,14 +1,14 b''
 .. _nginx-ws-ref:
 
 Nginx HTTP Server Configuration
 -------------------------------
 
 To set up your Nginx Web Server for optimal performance and security, use
 the information in the following sections.
 
 .. toctree::
 
-   nginx-config-example
-   nginx-diffie-hellman
-   nginx-proxy-conf
-   nginx-url-prefix
+   nginx/nginx-config-example
+   nginx/nginx-diffie-hellman
+   nginx/nginx-proxy-conf
+   nginx/nginx-url-prefix

docs/admin/system_admin/tuning-rhodecode.rst

@@ -1,21 +1,21 b''
 .. _rhodecode-tuning-ref:
 
 Tuning |RCE|
 ============
 
 To customize your |RCE| |version| installation for maximum performance you
 may find some of the following methods useful.
 
 .. toctree::
 
-   tuning-gunicorn
-   tuning-vcs-memory-cache
-   tuning-user-sessions-performance
-   tuning-increase-db-performance
-   tuning-scale-horizontally-cluster
-   tuning-mount-cache-memory
-   tuning-change-encoding
-   tuning-change-large-file-dir
-   tuning-change-lfs-dir
-   tuning-hg-auth-loop
+   tuning/tuning-gunicorn
+   tuning/tuning-vcs-memory-cache
+   tuning/tuning-user-sessions-performance
+   tuning/tuning-increase-db-performance
+   tuning/tuning-scale-horizontally-cluster
+   tuning/tuning-mount-cache-memory
+   tuning/tuning-change-encoding
+   tuning/tuning-change-large-file-dir
+   tuning/tuning-change-lfs-dir
+   tuning/tuning-hg-auth-loop
 

docs/admin/system_admin/tuning/tuning-gunicorn.rst

@@ -1,126 +1,126 b''
 .. _increase-gunicorn:
 
 Configure Gunicorn Workers
 --------------------------
 
 
 |RCE| comes with `Gunicorn`_ which is a Python WSGI HTTP Server for UNIX.
 
 To improve |RCE| performance you can increase the number of `Gunicorn`_  workers.
 This allows to handle more connections concurrently, and provide better
 responsiveness and performance.
 
 By default during installation |RCC|  tries to detect how many CPUs are
 available in the system, and set the number workers based on that information.
 However sometimes it's better to manually set the number of workers.
 
 To do this, use the following steps:
 
 1. Open the :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
 2. In the ``[server:main]`` section, change the number of Gunicorn
    ``workers`` using the following default formula :math:`(2 * Cores) + 1`.
    We however not recommend using more than 8-12 workers per server. It's better
    to start using the :ref:`scale-horizontal-cluster` in case that performance
    with 8-12 workers is not enough.
 
 .. code-block:: ini
 
     use = egg:gunicorn#main
     ## Sets the number of process workers. You must set `instance_id = *`
     ## when this option is set to more than one worker, recommended
     ## value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers
     ## The `instance_id = *` must be set in the [app:main] section below
     workers = 4
     ## process name
     proc_name = rhodecode
     ## type of worker class, one of sync, gevent
     ## recommended for bigger setup is using of of other than sync one
     worker_class = sync
     ## The maximum number of simultaneous clients. Valid only for Gevent
     #worker_connections = 10
     ## max number of requests that worker will handle before being gracefully
     ## restarted, could prevent memory leaks
     max_requests = 1000
     max_requests_jitter = 30
-    ## amount of time a worker can spend with handling a request before it
+    ## amount of time a worker can spend with handling a request tuning-change-lfs-dir.before it
     ## gets killed and restarted. Set to 6hrs
     timeout = 21600
 
 3. In the ``[app:main]`` section, set the ``instance_id`` property to ``*``.
 
 .. code-block:: ini
 
     # In the [app:main] section
     [app:main]
     # You must set `instance_id = *`
     instance_id = *
 
 4. Change the VCSServer workers too. Open the
    :file:`home/{user}/.rccontrol/{instance-id}/vcsserver.ini` file.
 
 5. In the ``[server:main]`` section, increase the number of Gunicorn
    ``workers`` using the following formula :math:`(2 * Cores) + 1`.
 
 .. code-block:: ini
 
     ## run with gunicorn --log-config vcsserver.ini --paste vcsserver.ini
     use = egg:gunicorn#main
     ## Sets the number of process workers. Recommended
     ## value is (2 * NUMBER_OF_CPUS + 1), eg 2CPU = 5 workers
     workers = 4
     ## process name
     proc_name = rhodecode_vcsserver
     ## type of worker class, currently `sync` is the only option allowed.
     worker_class = sync
     ## The maximum number of simultaneous clients. Valid only for Gevent
     #worker_connections = 10
     ## max number of requests that worker will handle before being gracefully
     ## restarted, could prevent memory leaks
     max_requests = 1000
     max_requests_jitter = 30
     ## amount of time a worker can spend with handling a request before it
     ## gets killed and restarted. Set to 6hrs
     timeout = 21600
 
 6. Save your changes.
 7. Restart your |RCE| instances, using the following command:
 
 .. code-block:: bash
 
     $ rccontrol restart '*'
 
 
 Gunicorn Gevent Backend
 -----------------------
 
 Gevent is an asynchronous worker type for Gunicorn. It allows accepting multiple
 connections on a single `Gunicorn`_  worker. This means you can handle 100s
 of concurrent clones, or API calls using just few workers. A setting called
 `worker_connections` defines on how many connections each worker can
 handle using `Gevent`.
 
 
 To enable `Gevent` on |RCE| do the following:
 
 
 1. Open the :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
 2. In the ``[server:main]`` section, change `worker_class` for Gunicorn.
 
 
 .. code-block:: ini
 
     ## type of worker class, one of sync, gevent
     ## recommended for bigger setup is using of of other than sync one
     worker_class = gevent
     ## The maximum number of simultaneous clients. Valid only for Gevent
     worker_connections = 30
 
 
 .. note::
 
     `Gevent` is currently only supported for Enterprise/Community instances.
     VCSServer doesn't yet support gevent.
 
 
 
 .. _Gunicorn: http://gunicorn.org/

docs/admin/user-admin.rst

@@ -1,23 +1,24 b''
 .. _user-admin-set:
 
 User Administration
 ===================
 
 |RCE| enables you to define permissions for the following entities within the
 system; **users**, **user groups**, **repositories**, **repository groups**.
 
 Within each one of these entities you can set default settings,
 and then all users or |repos| inherit those default permission settings
 unless individually defined. Each of these entities can have the following
 permissions applied to it; |perm|.
 
 .. toctree::
 
-    public-access
-    default-user-perms
-    adding-anonymous-user
-    adding-new-user
-    setting-default-permissions
-    setting-usergroup-permissions
+    user_admin/public-access
+    user_admin/default-user-perms
+    user_admin/adding-anonymous-user
+    user_admin/adding-new-user
+    user_admin/setting-default-permissions
+    user_admin/setting-usergroup-permissions
+    user_admin/user-admin-tasks
 
-.. |perm| replace:: **None**, **Read**, **Write**, or **Admin**
  No newline at end of file
+.. |perm| replace:: **None**, **Read**, **Write**, or **Admin**

docs/api/api.rst

@@ -1,209 +1,210 b''
 .. _api:
 
 API Documentation
 =================
 
 The |RCE| API uses a single scheme for calling all API methods. The API is
 implemented with JSON protocol in both directions. To send API requests to
 your instance of |RCE|, use the following URL format
 ``<your_server>/_admin``
 
 .. note::
 
    To use the API, you should configure the :file:`~/.rhoderc` file with
    access details per instance. For more information, see
    :ref:`config-rhoderc`.
 
 
 API ACCESS FOR WEB VIEWS
 ------------------------
 
 API access can also be turned on for each web view in |RCE| that is
 decorated with a `@LoginRequired` decorator. To enable API access, change
 the standard login decorator to `@LoginRequired(api_access=True)`.
 
 From |RCE| version 1.7.0 you can configure a white list
 of views that have API access enabled by default. To enable these,
 edit the |RCE| configuration ``.ini`` file. The default location is:
 
 * |RCE| Pre-2.2.7 :file:`root/rhodecode/data/production.ini`
 * |RCE| 3.0 :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
 
 To configure the white list, edit this section of the file. In this
 configuration example, API access is granted to the patch/diff raw file and
 archive.
 
 .. code-block:: ini
 
     ## List of controllers (using glob syntax) that AUTH TOKENS could be used for access.
     ## Adding ?auth_token = <token> to the url authenticates this request as if it
     ## came from the the logged in user who own this authentication token.
     ##
     ## Syntax is <ControllerClass>:<function_pattern>.
     ## The list should be "," separated and on a single line.
     ##
     api_access_controllers_whitelist = RepoCommitsView:repo_commit_raw,RepoCommitsView:repo_commit_patch,RepoCommitsView:repo_commit_download
 
 After this change, a |RCE| view can be accessed without login by adding a
 GET parameter ``?auth_token=<auth_token>`` to a url. For example to
 access the raw diff.
 
 .. code-block:: html
 
    http://<server>/<repo>/changeset-diff/<sha>?auth_token=<auth_token>
 
 By default this is only enabled on RSS/ATOM feed views. Exposing raw diffs is a
 good way to integrate with 3rd party services like code review, or build farms
 that could download archives.
 
 API ACCESS
 ----------
 
 All clients are required to send JSON-RPC spec JSON data.
 
 .. code-block:: bash
 
     {
         "id:"<id>",
         "auth_token":"<auth_token>",
         "method":"<method_name>",
         "args":{"<arg_key>":"<arg_val>"}
     }
 
 Example call for auto pulling from remote repositories using curl:
 
 .. code-block:: bash
 
     curl https://server.com/_admin/api -X POST -H 'content-type:text/plain' --data-binary '{"id":1,
     "auth_token":"xe7cdb2v278e4evbdf5vs04v832v0efvcbcve4a3","method":"pull", "args":{"repoid":"CPython"}}'
 
 Provide those parameters:
  - **id** A value of any type, which is used to match the response with the
    request that it is replying to.
  - **auth_token** for access and permission validation.
  - **method** is name of method to call
  - **args** is an ``key:value`` list of arguments to pass to method
 
 .. note::
 
     To get your |authtoken|, from the |RCE| interface,
     go to:
     :menuselection:`username --> My account --> Auth tokens`
 
     For security reasons you should always create a dedicated |authtoken| for
     API use only.
 
 
 The |RCE| API will always return a JSON-RPC response:
 
 .. code-block:: bash
 
     {
         "id": <id>, # matching id sent by request
         "result": "<result>"|null, # JSON formatted result, null if any errors
         "error": "null"|<error_message> # JSON formatted error (if any)
     }
 
 All responses from API will be with `HTTP/1.0 200 OK` status code.
 If there is an error when calling the API, the *error* key will contain a
 failure description and the *result* will be `null`.
 
 API CLIENT
 ----------
 
 To install the |RCE| API, see :ref:`install-tools`. To configure the API per
 instance, see the :ref:`rc-tools` section as you need to configure a
 :file:`~/.rhoderc` file with your |authtokens|.
 
 Once you have set up your instance API access, use the following examples to
 get started.
 
 .. code-block:: bash
 
     # Getting the 'rhodecode' repository
     # from a RhodeCode Enterprise instance
     rhodecode-api --instance-name=enterprise-1 get_repo repoid:rhodecode
 
     Calling method get_repo => http://127.0.0.1:5000
     Server response
     {
         <json data>
     }
 
     # Creating a new mercurial repository called 'brand-new'
     # with a description 'Repo-description'
     rhodecode-api --instance-name=enterprise-1 create_repo repo_name:brand-new repo_type:hg description:Repo-description
     {
       "error": null,
       "id": 1110,
       "result": {
         "msg": "Created new repository `brand-new`",
         "success": true,
         "task": null
       }
     }
 
 A broken example, what not to do.
 
 .. code-block:: bash
 
     # A call missing the required arguments
     # and not specifying the instance
     rhodecode-api get_repo
 
     Calling method get_repo => http://127.0.0.1:5000
     Server response
     "Missing non optional `repoid` arg in JSON DATA"
 
 You can specify pure JSON using the ``--format`` parameter.
 
 .. code-block:: bash
 
     rhodecode-api --format=json get_repo repoid:rhodecode
 
 In such case only output that this function shows is pure JSON, we can use that
 and pipe output to some json formatter.
 
 If output is in pure JSON format, you can pipe output to a JSON formatter.
 
 .. code-block:: bash
 
     rhodecode-api --instance-name=enterprise-1 --format=json get_repo repoid:rhodecode | python -m json.tool
 
 API METHODS
 -----------
 
 Each method by default required following arguments.
 
 .. code-block:: bash
 
     id :      "<id_for_response>"
     auth_token : "<auth_token>"
     method :  "<method name>"
     args :    {}
 
 Use each **param** from docs and put it in args, Optional parameters
 are not required in args.
 
 .. code-block:: bash
 
     args: {"repoid": "rhodecode"}
 
 .. Note: From this point on things are generated by the script in
    `scripts/fabfile.py`. To change things below, update the docstrings in the
    ApiController.
 
 .. --- API DEFS MARKER ---
 .. toctree::
 
    methods/repo-methods
    methods/store-methods
    methods/license-methods
    methods/deprecated-methods
    methods/gist-methods
    methods/pull-request-methods
    methods/repo-methods
    methods/repo-group-methods
+   methods/search-methods
    methods/server-methods
    methods/user-methods
    methods/user-group-methods

docs/api/methods/repo-methods.rst

@@ -1,1134 +1,1133 b''
 .. _repo-methods-ref:
 
 repo methods
 ============
 
 add_field_to_repo 
 -----------------
 
 .. py:function:: add_field_to_repo(apiuser, repoid, key, label=<Optional:''>, description=<Optional:''>)
 
    Adds an extra field to a repository.
 
    This command can only be run using an |authtoken| with at least
    write permissions to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository id.
    :type repoid: str or int
    :param key: Create a unique field key for this repository.
    :type key: str
    :param label:
    :type label: Optional(str)
    :param description:
    :type description: Optional(str)
 
 
 comment_commit 
 --------------
 
 .. py:function:: comment_commit(apiuser, repoid, commit_id, message, status=<Optional:None>, comment_type=<Optional:u'note'>, resolves_comment_id=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Set a commit comment, and optionally change the status of the commit.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param commit_id: Specify the commit_id for which to set a comment.
    :type commit_id: str
    :param message: The comment text.
    :type message: str
    :param status: (**Optional**) status of commit, one of: 'not_reviewed',
        'approved', 'rejected', 'under_review'
    :type status: str
    :param comment_type: Comment type, one of: 'note', 'todo'
    :type comment_type: Optional(str), default: 'note'
    :param userid: Set the user name of the comment creator.
    :type userid: Optional(str or int)
 
    Example error output:
 
    .. code-block:: bash
 
        {
            "id" : <id_given_in_input>,
            "result" : {
                "msg": "Commented on commit `<commit_id>` for repository `<repoid>`",
                "status_change": null or <status>,
                "success": true
            },
            "error" :  null
        }
 
 
 create_repo 
 -----------
 
 .. py:function:: create_repo(apiuser, repo_name, repo_type, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, push_uri=<Optional:None>, landing_rev=<Optional:'rev:tip'>, enable_statistics=<Optional:False>, enable_locking=<Optional:False>, enable_downloads=<Optional:False>, copy_permissions=<Optional:False>)
 
    Creates a repository.
 
    * If the repository name contains "/", repository will be created inside
      a repository group or nested repository groups
 
      For example "foo/bar/repo1" will create |repo| called "repo1" inside
      group "foo/bar". You have to have permissions to access and write to
      the last repository group ("bar" in this example)
 
    This command can only be run using an |authtoken| with at least
    permissions to create repositories, or write permissions to
    parent repository groups.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repo_name: Set the repository name.
    :type repo_name: str
    :param repo_type: Set the repository type; 'hg','git', or 'svn'.
    :type repo_type: str
    :param owner: user_id or username
    :type owner: Optional(str)
    :param description: Set the repository description.
    :type description: Optional(str)
    :param private: set repository as private
    :type private: bool
    :param clone_uri: set clone_uri
    :type clone_uri: str
    :param push_uri: set push_uri
    :type push_uri: str
    :param landing_rev: <rev_type>:<rev>
    :type landing_rev: str
    :param enable_locking:
    :type enable_locking: bool
    :param enable_downloads:
    :type enable_downloads: bool
    :param enable_statistics:
    :type enable_statistics: bool
    :param copy_permissions: Copy permission from group in which the
        repository is being created.
    :type copy_permissions: bool
 
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg": "Created new repository `<reponame>`",
                  "success": true,
                  "task": "<celery task id or None if done sync>"
                }
        error:  null
 
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
         'failed to create repository `<repo_name>`'
      }
 
 
 delete_repo 
 -----------
 
 .. py:function:: delete_repo(apiuser, repoid, forks=<Optional:''>)
 
    Deletes a repository.
 
    * When the `forks` parameter is set it's possible to detach or delete
      forks of deleted repository.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param forks: Set to `detach` or `delete` forks from the |repo|.
    :type forks: Optional(str)
 
    Example error output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg": "Deleted repository `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 fork_repo 
 ---------
 
 .. py:function:: fork_repo(apiuser, repoid, fork_name, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, landing_rev=<Optional:'rev:tip'>, copy_permissions=<Optional:False>)
 
    Creates a fork of the specified |repo|.
 
    * If the fork_name contains "/", fork will be created inside
      a repository group or nested repository groups
 
      For example "foo/bar/fork-repo" will create fork called "fork-repo"
      inside group "foo/bar". You have to have permissions to access and
      write to the last repository group ("bar" in this example)
 
    This command can only be run using an |authtoken| with minimum
    read permissions of the forked repo, create fork permissions for an user.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set repository name or repository ID.
    :type repoid: str or int
    :param fork_name: Set the fork name, including it's repository group membership.
    :type fork_name: str
    :param owner: Set the fork owner.
    :type owner: str
    :param description: Set the fork description.
    :type description: str
    :param copy_permissions: Copy permissions from parent |repo|. The
        default is False.
    :type copy_permissions: bool
    :param private: Make the fork private. The default is False.
    :type private: bool
    :param landing_rev: Set the landing revision. The default is tip.
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_for_response>
        api_key : "<api_key>"
        args:     {
                    "repoid" :          "<reponame or repo_id>",
                    "fork_name":        "<forkname>",
                    "owner":            "<username or user_id = Optional(=apiuser)>",
                    "description":      "<description>",
                    "copy_permissions": "<bool>",
                    "private":          "<bool>",
                    "landing_rev":      "<landing_rev>"
                  }
 
    Example error output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg": "Created fork of `<reponame>` as `<forkname>`",
                  "success": true,
                  "task": "<celery task id or None if done sync>"
                }
        error:  null
 
 
 get_repo 
 --------
 
 .. py:function:: get_repo(apiuser, repoid, cache=<Optional:True>)
 
    Gets an existing repository by its name or repository_id.
 
    The members section so the output returns users groups or users
    associated with that repository.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository id.
    :type repoid: str or int
    :param cache: use the cached value for last changeset
    :type: cache: Optional(bool)
 
    Example output:
 
    .. code-block:: bash
 
        {
          "error": null,
          "id": <repo_id>,
          "result": {
            "clone_uri": null,
            "created_on": "timestamp",
            "description": "repo description",
            "enable_downloads": false,
            "enable_locking": false,
            "enable_statistics": false,
            "followers": [
              {
                "active": true,
                "admin": false,
                "api_key": "****************************************",
                "api_keys": [
                  "****************************************"
                ],
                "email": "[e-mail]",
                "emails": [
                  "[e-mail]"
                ],
                "extern_name": "rhodecode",
                "extern_type": "rhodecode",
                "firstname": "username",
                "ip_addresses": [],
                "language": null,
                "last_login": "2015-09-16T17:16:35.854",
                "lastname": "surname",
                "user_id": <user_id>,
                "username": "name"
              }
            ],
            "fork_of": "parent-repo",
            "landing_rev": [
              "rev",
              "tip"
            ],
            "last_changeset": {
              "author": "User <[e-mail]>",
              "branch": "default",
              "date": "timestamp",
              "message": "last commit message",
              "parents": [
                {
                  "raw_id": "commit-id"
                }
              ],
              "raw_id": "commit-id",
              "revision": <revision number>,
              "short_id": "short id"
            },
            "lock_reason": null,
            "locked_by": null,
            "locked_date": null,
            "owner": "owner-name",
            "permissions": [
              {
                "name": "super-admin-name",
                "origin": "super-admin",
                "permission": "repository.admin",
                "type": "user"
              },
              {
                "name": "owner-name",
                "origin": "owner",
                "permission": "repository.admin",
                "type": "user"
              },
              {
                "name": "user-group-name",
                "origin": "permission",
                "permission": "repository.write",
                "type": "user_group"
              }
            ],
            "private": true,
            "repo_id": 676,
            "repo_name": "user-group/repo-name",
            "repo_type": "hg"
          }
        }
 
 
 get_repo_changeset 
 ------------------
 
 .. py:function:: get_repo_changeset(apiuser, repoid, revision, details=<Optional:'basic'>)
 
    Returns information about a changeset.
 
    Additionally parameters define the amount of details returned by
    this function.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository id
    :type repoid: str or int
    :param revision: revision for which listing should be done
    :type revision: str
    :param details: details can be 'basic|extended|full' full gives diff
        info details like the diff itself, and number of changed files etc.
    :type details: Optional(str)
 
 
 get_repo_changesets 
 -------------------
 
 .. py:function:: get_repo_changesets(apiuser, repoid, start_rev, limit, details=<Optional:'basic'>)
 
    Returns a set of commits limited by the number starting
    from the `start_rev` option.
 
    Additional parameters define the amount of details returned by this
    function.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param start_rev: The starting revision from where to get changesets.
    :type start_rev: str
    :param limit: Limit the number of commits to this amount
    :type limit: str or int
    :param details: Set the level of detail returned. Valid option are:
        ``basic``, ``extended`` and ``full``.
    :type details: Optional(str)
 
    .. note::
 
       Setting the parameter `details` to the value ``full`` is extensive
       and returns details like the diff itself, and the number
       of changed files.
 
 
 get_repo_comments 
 -----------------
 
 .. py:function:: get_repo_comments(apiuser, repoid, commit_id=<Optional:None>, comment_type=<Optional:None>, userid=<Optional:None>)
 
    Get all comments for a repository
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param commit_id: Optionally filter the comments by the commit_id
    :type commit_id: Optional(str), default: None
    :param comment_type: Optionally filter the comments by the comment_type
        one of: 'note', 'todo'
    :type comment_type: Optional(str), default: None
    :param userid: Optionally filter the comments by the author of comment
    :type userid: Optional(str or int), Default: None
 
    Example error output:
 
    .. code-block:: bash
 
        {
            "id" : <id_given_in_input>,
            "result" : [
                {
                  "comment_author": <USER_DETAILS>,
                  "comment_created_on": "2017-02-01T14:38:16.309",
                  "comment_f_path": "file.txt",
                  "comment_id": 282,
                  "comment_lineno": "n1",
                  "comment_resolved_by": null,
                  "comment_status": [],
                  "comment_text": "This file needs a header",
                  "comment_type": "todo"
                }
            ],
            "error" :  null
        }
 
 
 get_repo_file 
 -------------
 
 .. py:function:: get_repo_file(apiuser, repoid, commit_id, file_path, max_file_bytes=<Optional:None>, details=<Optional:'basic'>, cache=<Optional:True>)
 
    Returns a single file from repository at given revision.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param commit_id: The revision for which listing should be done.
    :type commit_id: str
    :param file_path: The path from which to start displaying.
    :type file_path: str
    :param details: Returns different set of information about nodes.
        The valid options are ``minimal`` ``basic`` and ``full``.
    :type details: Optional(str)
    :param max_file_bytes: Only return file content under this file size bytes
    :type max_file_bytes: Optional(int)
    :param cache: Use internal caches for fetching files. If disabled fetching
        files is slower but more memory efficient
    :type cache: Optional(bool)
+
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
            "binary": false,
            "extension": "py",
            "lines": 35,
            "content": "....",
            "md5": "76318336366b0f17ee249e11b0c99c41",
            "mimetype": "text/x-python",
            "name": "python.py",
            "size": 817,
            "type": "file",
        }
        error:  null
 
 
 get_repo_fts_tree 
 -----------------
 
 .. py:function:: get_repo_fts_tree(apiuser, repoid, commit_id, root_path)
 
    Returns a list of tree nodes for path at given revision. This api is built
    strictly for usage in full text search building, and shouldn't be consumed
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
 
 get_repo_nodes 
 --------------
 
 .. py:function:: get_repo_nodes(apiuser, repoid, revision, root_path, ret_type=<Optional:'all'>, details=<Optional:'basic'>, max_file_bytes=<Optional:None>)
 
    Returns a list of nodes and children in a flat list for a given
-       path at given revision.
+   path at given revision.
 
-       It's possible to specify ret_type to show only `files` or `dirs`.
+   It's possible to specify ret_type to show only `files` or `dirs`.
 
-       This command can only be run using an |authtoken| with admin rights,
-       or users with at least read rights to |repos|.
+   This command can only be run using an |authtoken| with admin rights,
+   or users with at least read rights to |repos|.
 
-       :param apiuser: This is filled automatically from the |authtoken|.
-       :type apiuser: AuthUser
-       :param repoid: The repository name or repository ID.
-       :type repoid: str or int
-       :param revision: The revision for which listing should be done.
-       :type revision: str
-       :param root_path: The path from which to start displaying.
-       :type root_path: str
-       :param ret_type: Set the return type. Valid options are
-           ``all`` (default), ``files`` and ``dirs``.
-       :type ret_type: Optional(str)
-       :param details: Returns extended information about nodes, such as
-           md5, binary, and or content.
-           The valid options are ``basic`` and ``full``.
-       :type details: Optional(str)
-       :param max_file_bytes: Only return file content under this file size bytes
-       :type details: Optional(int)
-
-       Example output:
+   :param apiuser: This is filled automatically from the |authtoken|.
+   :type apiuser: AuthUser
+   :param repoid: The repository name or repository ID.
+   :type repoid: str or int
+   :param revision: The revision for which listing should be done.
+   :type revision: str
+   :param root_path: The path from which to start displaying.
+   :type root_path: str
+   :param ret_type: Set the return type. Valid options are
+       ``all`` (default), ``files`` and ``dirs``.
+   :type ret_type: Optional(str)
+   :param details: Returns extended information about nodes, such as
+       md5, binary, and or content.
+       The valid options are ``basic`` and ``full``.
+   :type details: Optional(str)
+   :param max_file_bytes: Only return file content under this file size bytes
+   :type details: Optional(int)
 
-       .. code-block:: bash
+   Example output:
+
+   .. code-block:: bash
 
-           id : <id_given_in_input>
-           result: [
-                       {
-                         "binary": false,
-                         "content": "File line
-   Line2
-   ",
-                         "extension": "md",
-                         "lines": 2,
-                         "md5": "059fa5d29b19c0657e384749480f6422",
-                         "mimetype": "text/x-minidsrc",
-                         "name": "file.md",
-                         "size": 580,
-                         "type": "file"
-                       },
-                     ...
-                   ]
-           error:  null
+       id : <id_given_in_input>
+       result: [
+                   {
+                     "binary": false,
+                     "content": "File line",
+                     "extension": "md",
+                     "lines": 2,
+                     "md5": "059fa5d29b19c0657e384749480f6422",
+                     "mimetype": "text/x-minidsrc",
+                     "name": "file.md",
+                     "size": 580,
+                     "type": "file"
+                   },
+                 ...
+               ]
+       error:  null
 
 
 get_repo_refs 
 -------------
 
 .. py:function:: get_repo_refs(apiuser, repoid)
 
    Returns a dictionary of current references. It returns
    bookmarks, branches, closed_branches, and tags for given repository
 
    It's possible to specify ret_type to show only `files` or `dirs`.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        "result": {
            "bookmarks": {
              "dev": "5611d30200f4040ba2ab4f3d64e5b06408a02188",
              "master": "367f590445081d8ec8c2ea0456e73ae1f1c3d6cf"
            },
            "branches": {
              "default": "5611d30200f4040ba2ab4f3d64e5b06408a02188",
              "stable": "367f590445081d8ec8c2ea0456e73ae1f1c3d6cf"
            },
            "branches_closed": {},
            "tags": {
              "tip": "5611d30200f4040ba2ab4f3d64e5b06408a02188",
              "v4.4.0": "1232313f9e6adac5ce5399c2a891dc1e72b79022",
              "v4.4.1": "cbb9f1d329ae5768379cdec55a62ebdd546c4e27",
              "v4.4.2": "24ffe44a27fcd1c5b6936144e176b9f6dd2f3a17",
            }
        }
        error:  null
 
 
 get_repo_settings 
 -----------------
 
 .. py:function:: get_repo_settings(apiuser, repoid, key=<Optional:None>)
 
    Returns all settings for a repository. If key is given it only returns the
    setting identified by the key or null.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository id.
    :type repoid: str or int
    :param key: Key of the setting to return.
    :type: key: Optional(str)
 
    Example output:
 
    .. code-block:: bash
 
        {
            "error": null,
            "id": 237,
            "result": {
                "extensions_largefiles": true,
                "extensions_evolve": true,
                "hooks_changegroup_push_logger": true,
                "hooks_changegroup_repo_size": false,
                "hooks_outgoing_pull_logger": true,
                "phases_publish": "True",
                "rhodecode_hg_use_rebase_for_merging": true,
                "rhodecode_pr_merge_enabled": true,
                "rhodecode_use_outdated_comments": true
            }
        }
 
 
 get_repos 
 ---------
 
 .. py:function:: get_repos(apiuser, root=<Optional:None>, traverse=<Optional:True>)
 
    Lists all existing repositories.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param root: specify root repository group to fetch repositories.
        filters the returned repositories to be members of given root group.
    :type root: Optional(None)
    :param traverse: traverse given root into subrepositories. With this flag
        set to False, it will only return top-level repositories from `root`.
        if root is empty it will return just top-level repositories.
    :type traverse: Optional(True)
 
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: [
                  {
                    "repo_id" :          "<repo_id>",
                    "repo_name" :        "<reponame>"
                    "repo_type" :        "<repo_type>",
                    "clone_uri" :        "<clone_uri>",
                    "private": :         "<bool>",
                    "created_on" :       "<datetimecreated>",
                    "description" :      "<description>",
                    "landing_rev":       "<landing_rev>",
                    "owner":             "<repo_owner>",
                    "fork_of":           "<name_of_fork_parent>",
                    "enable_downloads":  "<bool>",
                    "enable_locking":    "<bool>",
                    "enable_statistics": "<bool>",
                  },
                  ...
                ]
        error:  null
 
 
 grant_user_group_permission 
 ---------------------------
 
 .. py:function:: grant_user_group_permission(apiuser, repoid, usergroupid, perm)
 
    Grant permission for a user group on the specified repository,
    or update existing permissions.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param usergroupid: Specify the ID of the user group.
    :type usergroupid: str or int
    :param perm: Set the user group permissions using the following
        format: (repository.(none|read|write|admin))
    :type perm: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg" : "Granted perm: `<perm>` for group: `<usersgroupname>` in repo: `<reponame>`",
        "success": true
 
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to edit permission for user group: `<usergroup>` in repo `<repo>`'
      }
 
 
 grant_user_permission 
 ---------------------
 
 .. py:function:: grant_user_permission(apiuser, repoid, userid, perm)
 
    Grant permissions for the specified user on the given repository,
    or update existing permissions if found.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param userid: Set the user name.
    :type userid: str
    :param perm: Set the user permissions, using the following format
        ``(repository.(none|read|write|admin))``
    :type perm: str
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Granted perm: `<perm>` for user: `<username>` in repo: `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 invalidate_cache 
 ----------------
 
 .. py:function:: invalidate_cache(apiuser, repoid, delete_keys=<Optional:False>)
 
    Invalidates the cache for the specified repository.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Sets the repository name or repository ID.
    :type repoid: str or int
    :param delete_keys: This deletes the invalidated keys instead of
        just flagging them.
    :type delete_keys: Optional(``True`` | ``False``)
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'msg': Cache for repository `<repository name>` was invalidated,
        'repository': <repository name>
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error : {
         'Error occurred during cache invalidation action'
      }
 
 
 lock 
 ----
 
 .. py:function:: lock(apiuser, repoid, locked=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Sets the lock state of the specified |repo| by the given user.
    From more information, see :ref:`repo-locking`.
 
    * If the ``userid`` option is not set, the repository is locked to the
      user who called the method.
    * If the ``locked`` parameter is not set, the current lock state of the
      repository is displayed.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Sets the repository name or repository ID.
    :type repoid: str or int
    :param locked: Sets the lock state.
    :type locked: Optional(``True`` | ``False``)
    :param userid: Set the repository lock to this user.
    :type userid: Optional(str or int)
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'repo': '<reponame>',
        'locked': <bool: lock state>,
        'locked_since': <int: lock timestamp>,
        'locked_by': <username of person who made the lock>,
        'lock_reason': <str: reason for locking>,
        'lock_state_changed': <bool: True if lock state has been changed in this request>,
        'msg': 'Repo `<reponame>` locked by `<username>` on <timestamp>.'
        or
        'msg': 'Repo `<repository name>` not locked.'
        or
        'msg': 'User `<user name>` set lock state for repo `<repository name>` to `<new lock state>`'
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        'Error occurred locking repository `<reponame>`'
      }
 
 
 maintenance 
 -----------
 
 .. py:function:: maintenance(apiuser, repoid)
 
    Triggers a maintenance on the given repository.
 
    This command can only be run using an |authtoken| with admin
    rights to the specified repository. For more information,
    see :ref:`config-token-ref`.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "executed maintenance command",
        "executed_actions": [
           <action_message>, <action_message2>...
        ],
        "repository": "<repository name>"
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "Unable to execute maintenance on `<reponame>`"
      }
 
 
 pull 
 ----
 
 .. py:function:: pull(apiuser, repoid, remote_uri=<Optional:None>)
 
    Triggers a pull on the given repository from a remote location. You
    can use this to keep remote repositories up-to-date.
 
    This command can only be run using an |authtoken| with admin
    rights to the specified repository. For more information,
    see :ref:`config-token-ref`.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param remote_uri: Optional remote URI to pass in for pull
    :type remote_uri: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "Pulled from url `<remote_url>` on repo `<repository name>`"
        "repository": "<repository name>"
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "Unable to push changes from `<remote_url>`"
      }
 
 
 remove_field_from_repo 
 ----------------------
 
 .. py:function:: remove_field_from_repo(apiuser, repoid, key)
 
    Removes an extra field from a repository.
 
    This command can only be run using an |authtoken| with at least
    write permissions to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param key: Set the unique field key for this repository.
    :type key: str
 
 
 revoke_user_group_permission 
 ----------------------------
 
 .. py:function:: revoke_user_group_permission(apiuser, repoid, usergroupid)
 
    Revoke the permissions of a user group on a given repository.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param usergroupid: Specify the user group ID.
    :type usergroupid: str or int
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Revoked perm for group: `<usersgroupname>` in repo: `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 revoke_user_permission 
 ----------------------
 
 .. py:function:: revoke_user_permission(apiuser, repoid, userid)
 
    Revoke permission for a user on the specified repository.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param userid: Set the user name of revoked user.
    :type userid: str or int
 
    Example error output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Revoked perm for user: `<username>` in repo: `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 set_repo_settings 
 -----------------
 
 .. py:function:: set_repo_settings(apiuser, repoid, settings)
 
    Update repository settings. Returns true on success.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository id.
    :type repoid: str or int
    :param settings: The new settings for the repository.
    :type: settings: dict
 
    Example output:
 
    .. code-block:: bash
 
        {
            "error": null,
            "id": 237,
            "result": true
        }
 
 
 strip 
 -----
 
 .. py:function:: strip(apiuser, repoid, revision, branch)
 
    Strips the given revision from the specified repository.
 
    * This will remove the revision and all of its decendants.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param revision: The revision you wish to strip.
    :type revision: str
    :param branch: The branch from which to strip the revision.
    :type branch: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "'Stripped commit <commit_hash> from repo `<repository name>`'"
        "repository": "<repository name>"
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "Unable to strip commit <commit_hash> from repo `<repository name>`"
      }
 
 
 update_repo 
 -----------
 
 .. py:function:: update_repo(apiuser, repoid, repo_name=<Optional:None>, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, push_uri=<Optional:None>, landing_rev=<Optional:'rev:tip'>, fork_of=<Optional:None>, enable_statistics=<Optional:False>, enable_locking=<Optional:False>, enable_downloads=<Optional:False>, fields=<Optional:''>)
 
    Updates a repository with the given information.
 
    This command can only be run using an |authtoken| with at least
    admin permissions to the |repo|.
 
    * If the repository name contains "/", repository will be updated
      accordingly with a repository group or nested repository groups
 
      For example repoid=repo-test name="foo/bar/repo-test" will update |repo|
      called "repo-test" and place it inside group "foo/bar".
      You have to have permissions to access and write to the last repository
      group ("bar" in this example)
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: repository name or repository ID.
    :type repoid: str or int
    :param repo_name: Update the |repo| name, including the
        repository group it's in.
    :type repo_name: str
    :param owner: Set the |repo| owner.
    :type owner: str
    :param fork_of: Set the |repo| as fork of another |repo|.
    :type fork_of: str
    :param description: Update the |repo| description.
    :type description: str
    :param private: Set the |repo| as private. (True | False)
    :type private: bool
    :param clone_uri: Update the |repo| clone URI.
    :type clone_uri: str
    :param landing_rev: Set the |repo| landing revision. Default is ``rev:tip``.
    :type landing_rev: str
    :param enable_statistics: Enable statistics on the |repo|, (True | False).
    :type enable_statistics: bool
    :param enable_locking: Enable |repo| locking.
    :type enable_locking: bool
    :param enable_downloads: Enable downloads from the |repo|, (True | False).
    :type enable_downloads: bool
    :param fields: Add extra fields to the |repo|. Use the following
        example format: ``field_key=field_val,field_key2=fieldval2``.
        Escape ', ' with \,
    :type fields: str
 
 

docs/auth/auth-saml-bulk-enroll-users.rst

@@ -1,88 +1,88 b''
 .. _auth-saml-bulk-enroll-users-ref:
 
 
 Bulk enroll multiple existing users
 -----------------------------------
 
 
 RhodeCode Supports standard SAML 2.0 SSO for the web-application part.
 Below is an example how to enroll list of all or some users to use SAML authentication.
 This method simply enables SAML authentication for many users at once.
 
 
 From the server RhodeCode Enterprise is running run ishell on the instance which we
 want to apply the SAML migration::
 
     rccontrol ishell enterprise-1
 
 Follow these steps to enable SAML authentication for multiple users.
 
 
 1) Create a user_id => attribute mapping
 
 
 `saml2user` is a mapping of external ID from SAML provider such as OneLogin, DuoSecurity, Google.
 This mapping consists of local rhodecode user_id mapped to set of required attributes needed to bind SAML
 account to internal rhodecode user.
 For example, 123 is local rhodecode user_id, and '48253211' is OneLogin ID.
 For other providers you'd have to figure out what would be the user-id, sometimes it's the email, i.e for Google
 The most important this id needs to be unique for each user.
 
 .. code-block:: python
 
     In [1]: saml2user = {
        ...: # OneLogin, uses externalID available to read from in the UI
-       ...: 123: {'id: '48253211'},
+       ...: 123: {'id': '48253211'},
        ...: # for Google/DuoSecurity email is also an option for unique ID
-       ...: 124: {'id: '[e-mail]'},
+       ...: 124: {'id': '[e-mail]'},
        ...: }
 
 
 2) Import the plugin you want to run migration for.
 
 From available options pick only one and run the `import` statement
 
 .. code-block:: python
 
     # for Duo Security
     In [2]: from rc_auth_plugins.auth_duo_security import RhodeCodeAuthPlugin
     # for OneLogin
     In [2]: from rc_auth_plugins.auth_onelogin import RhodeCodeAuthPlugin
     # generic SAML plugin
     In [2]: from rc_auth_plugins.auth_saml import RhodeCodeAuthPlugin
 
 3) Run the migration based on saml2user mapping.
 
 Enter in the ishell prompt
 
 .. code-block:: python
 
     In [3]: for user in User.get_all():
        ...:     existing_identity = ExternalIdentity().query().filter(ExternalIdentity.local_user_id == user.user_id).scalar()
        ...:     attrs = saml2user.get(user.user_id)
        ...:     provider = RhodeCodeAuthPlugin.uid
        ...:     if existing_identity:
        ...:         print('Identity for user `{}` already exists, skipping'.format(user.username))
        ...:         continue
        ...:     if attrs:
        ...:         external_id = attrs['id']
        ...:         new_external_identity = ExternalIdentity()
        ...:         new_external_identity.external_id = external_id
        ...:         new_external_identity.external_username = '{}-saml-{}'.format(user.username, user.user_id)
        ...:         new_external_identity.provider_name = provider
-       ...:         new_external_identity.local_user_id = user_id
+       ...:         new_external_identity.local_user_id = user.user_id
        ...:         new_external_identity.access_token = ''
        ...:         new_external_identity.token_secret = ''
        ...:         new_external_identity.alt_token = ''
        ...:         Session().add(ex_identity)
        ...:         Session().commit()
        ...:         print('Set user `{}` external identity bound to ExternalID:{}'.format(user.username, external_id))
 
 .. note::
 
     saml2user can be really big and hard to maintain in ishell. It's also possible
     to load it as a JSON file prepared before and stored on disk. To do so run::
 
         import json
         saml2user = json.loads(open('/path/to/saml2user.json','rb').read())
 

docs/index.rst

@@ -1,87 +1,87 b''
 |RCE|
 =====
 
 |RCE| is a high-performance source code management and collaboration system.
 It enables you to develop projects securely behind the firewall while
 providing collaboration tools that work with |git|, |hg|,
 and |svn| |repos|. The user interface allows you to create, edit,
 and commit files and |repos| while managing their security permissions.
 
 |RCE| provides the following features:
 
 * Source code management.
 * Extended permissions management.
 * Integrated code collaboration tools.
 * Integrated code review and notifications.
 * Scalability provided by multi-node setup.
 * Fully programmable automation API.
 * Web-based hook management.
 * Native |svn| support.
 * Migration from existing databases.
 * |RCE| SDK.
 * Built-in analytics
 * Built in integrations including: Slack, Webhooks (used for Jenkins/TeamCity and other CIs), Jira, Redmine, Hipchat
 * Pluggable authentication system.
 * Support for AD, |LDAP|, Crowd, CAS, PAM.
 * Support for external authentication via Oauth Google, Github, Bitbucket, Twitter.
 * Debug modes of operation.
 * Private and public gists.
 * Gists with limited lifetimes and within instance only sharing.
 * Fully integrated code search function.
 * Always on SSL connectivity.
 
 .. only:: html
 
    Table of Contents
    -----------------
 
 .. toctree::
    :maxdepth: 1
    :caption: Admin Documentation
 
    install/quick-start
    install/install-database
    install/install-steps
    admin/system-overview
    nix/default-env
    admin/system-admin
    admin/user-admin
-   admin/setting-repo-perms
+   admin/repo-admin
    admin/security-tips
    auth/auth
    issue-trackers/issue-trackers
    admin/lab-settings
 
 .. toctree::
    :maxdepth: 1
    :caption: Feature Documentation
 
    collaboration/collaboration
    collaboration/review-notifications
    collaboration/pull-requests
    code-review/code-review
    integrations/integrations
 
 .. toctree::
    :maxdepth: 1
+   :caption: User Documentation
+
+   usage/basic-usage
+   tutorials/tutorials
+
+.. toctree::
+   :maxdepth: 1
    :caption: Developer Documentation
 
    api/api
    tools/rhodecode-tools
    extensions/extensions-hooks
    contributing/contributing
 
 .. toctree::
    :maxdepth: 1
-   :caption: User Documentation
-
-   usage/basic-usage
-   tutorials/tutorials
-
-.. toctree::
-   :maxdepth: 1
    :caption: About
 
    known-issues/known-issues
    release-notes/release-notes
    admin/glossary

docs/release-notes/release-notes.rst

@@ -1,131 +1,132 b''
 .. _rhodecode-release-notes-ref:
 
 Release Notes
 =============
 
 |RCE| 4.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
+   release-notes-4.17.0.rst
    release-notes-4.16.2.rst
    release-notes-4.16.1.rst
    release-notes-4.16.0.rst
    release-notes-4.15.2.rst
    release-notes-4.15.1.rst
    release-notes-4.15.0.rst
    release-notes-4.14.1.rst
    release-notes-4.14.0.rst
    release-notes-4.13.3.rst
    release-notes-4.13.2.rst
    release-notes-4.13.1.rst
    release-notes-4.13.0.rst
    release-notes-4.12.4.rst
    release-notes-4.12.3.rst
    release-notes-4.12.2.rst
    release-notes-4.12.1.rst
    release-notes-4.12.0.rst
    release-notes-4.11.6.rst
    release-notes-4.11.5.rst
    release-notes-4.11.4.rst
    release-notes-4.11.3.rst
    release-notes-4.11.2.rst
    release-notes-4.11.1.rst
    release-notes-4.11.0.rst
    release-notes-4.10.6.rst
    release-notes-4.10.5.rst
    release-notes-4.10.4.rst
    release-notes-4.10.3.rst
    release-notes-4.10.2.rst
    release-notes-4.10.1.rst
    release-notes-4.10.0.rst
    release-notes-4.9.1.rst
    release-notes-4.9.0.rst
    release-notes-4.8.0.rst
    release-notes-4.7.2.rst
    release-notes-4.7.1.rst
    release-notes-4.7.0.rst
    release-notes-4.6.1.rst
    release-notes-4.6.0.rst
    release-notes-4.5.2.rst
    release-notes-4.5.1.rst
    release-notes-4.5.0.rst
    release-notes-4.4.2.rst
    release-notes-4.4.1.rst
    release-notes-4.4.0.rst
    release-notes-4.3.1.rst
    release-notes-4.3.0.rst
    release-notes-4.2.1.rst
    release-notes-4.2.0.rst
    release-notes-4.1.2.rst
    release-notes-4.1.1.rst
    release-notes-4.1.0.rst
    release-notes-4.0.1.rst
    release-notes-4.0.0.rst
 
 |RCE| 3.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-3.8.4.rst
    release-notes-3.8.3.rst
    release-notes-3.8.2.rst
    release-notes-3.8.1.rst
    release-notes-3.8.0.rst
    release-notes-3.7.1.rst
    release-notes-3.7.0.rst
    release-notes-3.6.1.rst
    release-notes-3.6.0.rst
    release-notes-3.5.2.rst
    release-notes-3.5.1.rst
    release-notes-3.5.0.rst
    release-notes-3.4.1.rst
    release-notes-3.4.0.rst
    release-notes-3.3.4.rst
    release-notes-3.3.3.rst
    release-notes-3.3.2.rst
    release-notes-3.3.1.rst
    release-notes-3.3.0.rst
    release-notes-3.2.3.rst
    release-notes-3.2.2.rst
    release-notes-3.2.1.rst
    release-notes-3.2.0.rst
    release-notes-3.1.1.rst
    release-notes-3.1.0.rst
    release-notes-3.0.2.rst
    release-notes-3.0.1.rst
    release-notes-3.0.0.rst
 
 |RCE| 2.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-2.2.8.rst
    release-notes-2.2.7.rst
    release-notes-2.2.6.rst
    release-notes-2.2.5.rst
    release-notes-2.2.4.rst
    release-notes-2.2.3.rst
    release-notes-2.2.2.rst
    release-notes-2.2.1.rst
    release-notes-2.2.0.rst
    release-notes-2.1.0.rst
    release-notes-2.0.2.rst
    release-notes-2.0.1.rst
    release-notes-2.0.0.rst
 
 |RCE| 1.x Versions
 ------------------
 
 .. toctree::
    :maxdepth: 1
 
    release-notes-1.7.2.rst
    release-notes-1.7.1.rst
    release-notes-1.7.0.rst
    release-notes-1.6.0.rst

grunt_config.json

@@ -1,174 +1,177 b''
 {
   "dirs": {
     "css": {
       "src": "rhodecode/public/css",
       "dest": "rhodecode/public/css"
     },
     "js": {
       "src": "rhodecode/public/js/src",
       "src_rc": "rhodecode/public/js/rhodecode",
       "dest": "rhodecode/public/js",
       "node_modules": "node_modules"
     }
   },
   "copy": {
     "main": {
       "files": [
         {
           "expand": true,
           "cwd": "node_modules/@webcomponents",
           "src": "webcomponentsjs/*.*",
           "dest": "<%= dirs.js.dest %>/vendors"
         },
         {
           "src": "<%= dirs.css.src %>/style-polymer.css",
           "dest": "<%= dirs.js.dest %>/src/components/style-polymer.css"
         }
       ]
     }
   },
   "concat": {
     "dist": {
       "src": [
         "<%= dirs.js.node_modules %>/jquery/dist/jquery.min.js",
         "<%= dirs.js.node_modules %>/mousetrap/mousetrap.min.js",
         "<%= dirs.js.node_modules %>/moment/min/moment.min.js",
         "<%= dirs.js.node_modules %>/clipboard/dist/clipboard.min.js",
         "<%= dirs.js.node_modules %>/favico.js/favico-0.3.10.min.js",
+        "<%= dirs.js.node_modules %>/dropzone/dist/dropzone.js",
         "<%= dirs.js.node_modules %>/sticky-sidebar/dist/sticky-sidebar.min.js",
         "<%= dirs.js.node_modules %>/sticky-sidebar/dist/jquery.sticky-sidebar.min.js",
         "<%= dirs.js.node_modules %>/waypoints/lib/noframework.waypoints.min.js",
         "<%= dirs.js.node_modules %>/waypoints/lib/jquery.waypoints.min.js",
         "<%= dirs.js.node_modules %>/appenlight-client/appenlight-client.min.js",
         "<%= dirs.js.src %>/logging.js",
         "<%= dirs.js.src %>/bootstrap.js",
         "<%= dirs.js.src %>/i18n_utils.js",
         "<%= dirs.js.src %>/deform.js",
         "<%= dirs.js.src %>/ejs.js",
         "<%= dirs.js.src %>/ejs_templates/utils.js",
         "<%= dirs.js.src %>/plugins/jquery.pjax.js",
         "<%= dirs.js.src %>/plugins/jquery.dataTables.js",
         "<%= dirs.js.src %>/plugins/flavoured_checkbox.js",
         "<%= dirs.js.src %>/plugins/jquery.auto-grow-input.js",
         "<%= dirs.js.src %>/plugins/jquery.autocomplete.js",
         "<%= dirs.js.src %>/plugins/jquery.debounce.js",
         "<%= dirs.js.node_modules %>/mark.js/dist/jquery.mark.min.js",
         "<%= dirs.js.src %>/plugins/jquery.timeago.js",
         "<%= dirs.js.src %>/plugins/jquery.timeago-extension.js",
         "<%= dirs.js.src %>/select2/select2.js",
         "<%= dirs.js.src %>/codemirror/codemirror.js",
         "<%= dirs.js.src %>/codemirror/codemirror_loadmode.js",
         "<%= dirs.js.src %>/codemirror/codemirror_hint.js",
         "<%= dirs.js.src %>/codemirror/codemirror_overlay.js",
         "<%= dirs.js.src %>/codemirror/codemirror_placeholder.js",
         "<%= dirs.js.src %>/codemirror/codemirror_simplemode.js",
         "<%= dirs.js.dest %>/mode/meta.js",
         "<%= dirs.js.dest %>/mode/meta_ext.js",
         "<%= dirs.js.src_rc %>/i18n/select2/translations.js",
         "<%= dirs.js.src %>/rhodecode/utils/array.js",
         "<%= dirs.js.src %>/rhodecode/utils/string.js",
         "<%= dirs.js.src %>/rhodecode/utils/pyroutes.js",
         "<%= dirs.js.src %>/rhodecode/utils/ajax.js",
         "<%= dirs.js.src %>/rhodecode/utils/autocomplete.js",
         "<%= dirs.js.src %>/rhodecode/utils/colorgenerator.js",
         "<%= dirs.js.src %>/rhodecode/utils/ie.js",
         "<%= dirs.js.src %>/rhodecode/utils/os.js",
         "<%= dirs.js.src %>/rhodecode/utils/topics.js",
         "<%= dirs.js.src %>/rhodecode/init.js",
         "<%= dirs.js.src %>/rhodecode/changelog.js",
         "<%= dirs.js.src %>/rhodecode/codemirror.js",
         "<%= dirs.js.src %>/rhodecode/comments.js",
         "<%= dirs.js.src %>/rhodecode/constants.js",
         "<%= dirs.js.src %>/rhodecode/files.js",
         "<%= dirs.js.src %>/rhodecode/followers.js",
         "<%= dirs.js.src %>/rhodecode/menus.js",
         "<%= dirs.js.src %>/rhodecode/notifications.js",
         "<%= dirs.js.src %>/rhodecode/permissions.js",
         "<%= dirs.js.src %>/rhodecode/pjax.js",
         "<%= dirs.js.src %>/rhodecode/pullrequests.js",
         "<%= dirs.js.src %>/rhodecode/settings.js",
         "<%= dirs.js.src %>/rhodecode/select2_widgets.js",
         "<%= dirs.js.src %>/rhodecode/tooltips.js",
         "<%= dirs.js.src %>/rhodecode/users.js",
         "<%= dirs.js.src %>/rhodecode/appenlight.js",
         "<%= dirs.js.src %>/rhodecode.js",
         "<%= dirs.js.dest %>/rhodecode-components.js"
       ],
       "dest": "<%= dirs.js.dest %>/scripts.js",
       "nonull": true
     }
   },
   "less": {
     "development": {
       "options": {
         "compress": false,
         "yuicompress": false,
         "optimization": 0
       },
       "files": {
         "<%= dirs.css.dest %>/style.css": "<%= dirs.css.src %>/main.less",
-        "<%= dirs.css.dest %>/style-polymer.css": "<%= dirs.css.src %>/polymer.less"
+        "<%= dirs.css.dest %>/style-polymer.css": "<%= dirs.css.src %>/polymer.less",
+        "<%= dirs.css.dest %>/style-ipython.css": "<%= dirs.css.src %>/ipython.less"
       }
     },
     "production": {
       "options": {
         "compress": true,
         "yuicompress": true,
         "optimization": 2
       },
       "files": {
         "<%= dirs.css.dest %>/style.css": "<%= dirs.css.src %>/main.less",
-        "<%= dirs.css.dest %>/style-polymer.css": "<%= dirs.css.src %>/polymer.less"
+        "<%= dirs.css.dest %>/style-polymer.css": "<%= dirs.css.src %>/polymer.less",
+        "<%= dirs.css.dest %>/style-ipython.css": "<%= dirs.css.src %>/ipython.less"
       }
     },
     "components": {
       "files": [
         {
           "cwd": "<%= dirs.js.src %>/components/",
           "dest": "<%= dirs.js.src %>/components/",
           "src": [
             "**/*.less"
           ],
           "expand": true,
           "ext": ".css"
         }
       ]
     }
   },
   "watch": {
     "less": {
       "files": [
         "<%= dirs.css.src %>/**/*.less",
         "<%= dirs.js.src %>/components/**/*.less"
       ],
       "tasks": [
         "less:development",
         "less:components",
         "concat:polymercss",
         "webpack",
         "concat:dist"
       ]
     },
     "js": {
       "files": [
         "!<%= dirs.js.src %>/components/root-styles.gen.html",
         "<%= dirs.js.src %>/**/*.js",
         "<%= dirs.js.src %>/components/**/*.html"
       ],
       "tasks": [
         "less:components",
         "concat:polymercss",
         "webpack",
         "concat:dist"
       ]
     }
   },
   "jshint": {
     "rhodecode": {
       "src": "<%= dirs.js.src %>/rhodecode/**/*.js",
       "options": {
         "jshintrc": ".jshintrc"
       }
     }
   }
 }

package.json

@@ -1,59 +1,60 b''
 {
   "name": "rhodecode-enterprise",
   "version": "2.0.0",
   "private": true,
   "description" : "RhodeCode JS packaged",
   "license": "SEE LICENSE IN LICENSE.txt",
   "repository" : {
     "type" : "hg",
     "url" : "https://code.rhodecode.com/rhodecode-enterprise-ce"
   },
   "devDependencies": {
     "appenlight-client": "git+https://[e-mail]/AppEnlight/appenlight-client-js.git#0.5.1",
     "clipboard": "^2.0.1",
     "exports-loader": "^0.6.4",
     "favico.js": "^0.3.10",
+    "dropzone": "^5.5.0",
     "grunt": "^0.4.5",
     "grunt-cli": "^1.3.1",
     "grunt-contrib-concat": "^0.5.1",
     "grunt-contrib-copy": "^1.0.0",
     "grunt-contrib-jshint": "^0.12.0",
     "grunt-contrib-less": "^1.1.0",
     "grunt-contrib-watch": "^0.6.1",
     "grunt-webpack": "^3.1.3",
     "jquery": "1.11.3",
     "mark.js": "8.11.1",
     "jshint": "^2.9.1-rc3",
     "moment": "^2.18.1",
     "mousetrap": "^1.6.1",
     "qrious": "^4.0.2",
     "sticky-sidebar": "3.3.1",
     "waypoints": "4.0.1",
     "webpack": "4.23.1",
     "webpack-cli": "3.1.2",
     "babel-core": "^6.26.3",
     "babel-loader": "^7.1.2",
     "babel-plugin-transform-object-rest-spread": "^6.26.0",
     "babel-preset-env": "^1.6.0",
     "copy-webpack-plugin": "^4.4.2",
     "css-loader": "^0.28.11",
     "html-loader": "^0.4.4",
     "html-webpack-plugin": "^3.2.0",
     "imports-loader": "^0.7.1",
     "polymer-webpack-loader": "^2.0.1",
     "style-loader": "^0.21.0",
     "webpack-uglify-js-plugin": "^1.1.9",
     "raw-loader": "1.0.0-beta.0",
     "ts-loader": "^1.3.3",
     "@webcomponents/webcomponentsjs": "^2.0.0",
     "@polymer/polymer": "^3.0.0",
     "@polymer/paper-button": "^3.0.0",
     "@polymer/paper-spinner": "^3.0.0",
     "@polymer/paper-tooltip": "^3.0.0",
     "@polymer/paper-toast": "^3.0.0",
     "@polymer/paper-toggle-button": "^3.0.0",
     "@polymer/iron-ajax": "^3.0.0",
     "@polymer/iron-autogrow-textarea": "^3.0.0",
     "@polymer/iron-a11y-keys": "^3.0.0"
   }
 }

pip2nix.ini

@@ -1,3 +1,3 b''
 [pip2nix]
-requirements = ., -r ./requirements.txt
+requirements = ., -r ./requirements.txt, -r ./requirements_pinned.txt
 output = ./pkgs/python-packages.nix