release: merge back stable branch into default

docs/release-notes/release-notes-4.11.1.rst

@@ -0,0 +1,54 b''
+|RCE| 4.11.1 |RNS|
+------------------
+
+Release Date
+^^^^^^^^^^^^
+
+- 2018-02-02
+
+
+New Features
+^^^^^^^^^^^^
+
+
+
+General
+^^^^^^^
+
+
+
+Security
+^^^^^^^^
+
+
+
+Performance
+^^^^^^^^^^^
+
+
+
+Fixes
+^^^^^
+
+
+- Audit logs: handle query syntax errors in whoosh query parser. Prevents 500
+  errors on wrongly entered search terms in audit logs.
+
+- Mercurial: fix new 4.4.X code change that does strict requirement checks. Fixes
+  problems with Mercurial Largefiles repositories.
+
+- VCSServer: in case of errors in the VCSServer worker restart it gracefully.
+  In rare cases certain errors caused locking of workers, and
+  unresponsive connections. Now we restart a worker freeing up memory and
+  connection.
+
+- Git: handle flaky and slow connection issues with git. Due to the changes in
+  Pyramid, flaky connections started affecting git clones.
+
+
+Upgrade notes
+^^^^^^^^^^^^^
+
+- Fixed regression on git with high latency connections.
+  No upgrade problems should be expected, however please check GIT repos
+  behaviour on upgrade.

.hgtags

@@ -28,3 +28,5 b' 68baee10e698da2724c6e0f698c03a6abb993bf2'
 00821d3afd1dce3f4767cc353f84a17f7d5218a1 v4.10.4
 22f6744ad8cc274311825f63f953e4dee2ea5cb9 v4.10.5
 96eb24bea2f5f9258775245e3f09f6fa0a4dda01 v4.10.6
+3121217a812c956d7dd5a5875821bd73e8002a32 v4.11.0
+fa98b454715ac5b912f39e84af54345909a2a805 v4.11.1

docs/install/configure-celery.rst

@@ -4,27 +4,48 b''
 Configure Celery
 ----------------
 
-To improve |RCM| performance you should configure and enabled Celery_ as it makes
-asynchronous tasks work efficiently. Most important it allows sending notification
-emails, create repository forks, and import repositories in async way.
+Celery_ is an asynchronous task queue. It's a part of RhodeCode scheduler
+functionality. Celery_ makes certain heavy tasks perform more efficiently.
+Most important it allows sending notification emails, create repository forks,
+and import repositories in async way. It is also used for bi-directional
+repository sync in scheduler.
 
 If you decide to use Celery you also need a working message queue.
-The recommended message broker is rabbitmq_.
+The recommended and fully supported message broker is rabbitmq_.
 
 
-In order to have install and configure Celery, follow these steps:
+In order to install and configure Celery, follow these steps:
 
 1. Install RabbitMQ, see the documentation on the Celery website for
-   `rabbitmq installation`_.
+   `rabbitmq installation`_, or `rabbitmq website installation`_
+
+
+1a. As en example configuration after installation, you can run::
+
+   sudo rabbitmqctl add_user rcuser secret_password
+   sudo rabbitmqctl add_vhost rhodevhost
+   sudo rabbitmqctl set_user_tags rcuser rhodecode
+   sudo rabbitmqctl set_permissions -p rhodevhost rcuser ".*" ".*" ".*"
+
 
-2. Configure Celery in the
+2. Enable celery, and install `celery worker` process script using the `enable-module`::
+
+    rccontrol enable-module celery {instance-id}
+
+.. note::
+
+   In case when using multiple instances in one or multiple servers it's highly
+   recommended that celery is running only once, for all servers connected to
+   the same database. Having multiple celery instances running without special
+   reconfiguration could cause scheduler issues.
+
+
+3. Configure Celery in the
    :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
-   Set the following minimal settings, that are set during rabbitmq_ installation::
+   Set the broker_url as minimal settings required to enable operation.
+   If used our example data from pt 1a, here is how the broker url should look like::
 
-        broker.host =
-        broker.vhost =
-        broker.user =
-        broker.password =
+        celery.broker_url = amqp://rcuser:secret_password@localhost:5672/rhodevhost
 
    Full configuration example is below:
 
@@ -34,35 +55,15 b' 2. Configure Celery in the'
         ####################################
         ###        CELERY CONFIG        ####
         ####################################
-        ## Set to true
-        use_celery = true
-        broker.host = localhost
-        broker.vhost = rabbitmqvhost
-        broker.port = 5672
-        broker.user = rabbitmq
-        broker.password = secret
-
-        celery.imports = rhodecode.lib.celerylib.tasks
 
-        celery.result.backend = amqp
-        celery.result.dburi = amqp://
-        celery.result.serialier = json
+        use_celery = true
+        celery.broker_url = amqp://rcuser:secret@localhost:5672/rhodevhost
 
-        #celery.send.task.error.emails = true
-        #celery.amqp.task.result.expires = 18000
-
-        celeryd.concurrency = 2
-        #celeryd.log.file = celeryd.log
-        celeryd.log.level = debug
-        celeryd.max.tasks.per.child = 1
+        # maximum tasks to execute before worker restart
+        celery.max_tasks_per_child = 100
 
         ## tasks will never be sent to the queue, but executed locally instead.
-        celery.always.eager = false
-
-
-3. Enable celery, and install `celeryd` process script using the `enable-module`::
-
-    rccontrol enable-module celery {instance-id}
+        celery.task_always_eager = false
 
 
 .. _python: http://www.python.org/
@@ -70,4 +71,5 b' 3. Enable celery, and install `celeryd` '
 .. _celery: http://celeryproject.org/
 .. _rabbitmq: http://www.rabbitmq.com/
 .. _rabbitmq installation: http://docs.celeryproject.org/en/latest/getting-started/brokers/rabbitmq.html
+.. _rabbitmq website installation: http://www.rabbitmq.com/download.html
 .. _Celery installation: http://docs.celeryproject.org/en/latest/getting-started/introduction.html#bundles

docs/release-notes/release-notes.rst

@@ -9,6 +9,7 b' Release Notes'
 .. toctree::
    :maxdepth: 1
 
+   release-notes-4.11.1.rst
    release-notes-4.11.0.rst
    release-notes-4.10.6.rst
    release-notes-4.10.5.rst

pkgs/python-packages.nix

@@ -1680,7 +1680,7 b''
     };
   };
   rhodecode-enterprise-ce = super.buildPythonPackage {
-    name = "rhodecode-enterprise-ce-4.11.0";
+    name = "rhodecode-enterprise-ce-4.11.1";
     buildInputs = with self; [pytest py pytest-cov pytest-sugar pytest-runner pytest-catchlog pytest-profiling gprof2dot pytest-timeout mock WebTest cov-core coverage configobj];
     doCheck = true;
     propagatedBuildInputs = with self; [setuptools-scm amqp authomatic Babel Beaker celery Chameleon channelstream click colander configobj cssselect decorator deform docutils dogpile.cache dogpile.core ecdsa FormEncode future futures gnureadline infrae.cache iso8601 itsdangerous Jinja2 billiard kombu lxml Mako Markdown MarkupSafe msgpack-python MySQL-python objgraph packaging Paste PasteDeploy PasteScript pathlib2 peppercorn psutil psycopg2 py-bcrypt pycrypto pycurl pyflakes pygments-markdown-lexer Pygments pyparsing pyramid-beaker pyramid-debugtoolbar pyramid-jinja2 pyramid-mako pyramid pysqlite python-dateutil python-ldap python-memcached python-pam pytz pyzmq py-gfm recaptcha-client redis repoze.lru requests Routes setproctitle simplejson six SQLAlchemy sshpubkeys subprocess32 supervisor Tempita translationstring trollius urllib3 URLObject venusian WebError WebHelpers2 WebHelpers WebOb Whoosh wsgiref zope.cachedescriptors zope.deprecation zope.event zope.interface nbconvert bleach nbformat jupyter-client alembic invoke bumpversion transifex-client gevent greenlet gunicorn waitress uWSGI ipdb ipython CProfileV bottle rhodecode-tools appenlight-client pytest py pytest-cov pytest-sugar pytest-runner pytest-catchlog pytest-profiling gprof2dot pytest-timeout mock WebTest cov-core coverage];