# HG changeset patch # User Marcin Kuzminski # Date 2018-02-02 12:58:07 # Node ID be1491740e53d96a66b6704e12461ccee78cec40 # Parent c537fbaec0b3ca4ed1931ddfad491e7031cecfef # Parent 445c68311f91c9c47492a68f00a580d35015209f release: merge back stable branch into default diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -28,3 +28,5 @@ 68baee10e698da2724c6e0f698c03a6abb993bf2 00821d3afd1dce3f4767cc353f84a17f7d5218a1 v4.10.4 22f6744ad8cc274311825f63f953e4dee2ea5cb9 v4.10.5 96eb24bea2f5f9258775245e3f09f6fa0a4dda01 v4.10.6 +3121217a812c956d7dd5a5875821bd73e8002a32 v4.11.0 +fa98b454715ac5b912f39e84af54345909a2a805 v4.11.1 diff --git a/docs/install/configure-celery.rst b/docs/install/configure-celery.rst --- a/docs/install/configure-celery.rst +++ b/docs/install/configure-celery.rst @@ -4,27 +4,48 @@ Configure Celery ---------------- -To improve |RCM| performance you should configure and enabled Celery_ as it makes -asynchronous tasks work efficiently. Most important it allows sending notification -emails, create repository forks, and import repositories in async way. +Celery_ is an asynchronous task queue. It's a part of RhodeCode scheduler +functionality. Celery_ makes certain heavy tasks perform more efficiently. +Most important it allows sending notification emails, create repository forks, +and import repositories in async way. It is also used for bi-directional +repository sync in scheduler. If you decide to use Celery you also need a working message queue. -The recommended message broker is rabbitmq_. +The recommended and fully supported message broker is rabbitmq_. -In order to have install and configure Celery, follow these steps: +In order to install and configure Celery, follow these steps: 1. Install RabbitMQ, see the documentation on the Celery website for - `rabbitmq installation`_. + `rabbitmq installation`_, or `rabbitmq website installation`_ + + +1a. As en example configuration after installation, you can run:: + + sudo rabbitmqctl add_user rcuser secret_password + sudo rabbitmqctl add_vhost rhodevhost + sudo rabbitmqctl set_user_tags rcuser rhodecode + sudo rabbitmqctl set_permissions -p rhodevhost rcuser ".*" ".*" ".*" + -2. Configure Celery in the +2. Enable celery, and install `celery worker` process script using the `enable-module`:: + + rccontrol enable-module celery {instance-id} + +.. note:: + + In case when using multiple instances in one or multiple servers it's highly + recommended that celery is running only once, for all servers connected to + the same database. Having multiple celery instances running without special + reconfiguration could cause scheduler issues. + + +3. Configure Celery in the :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file. - Set the following minimal settings, that are set during rabbitmq_ installation:: + Set the broker_url as minimal settings required to enable operation. + If used our example data from pt 1a, here is how the broker url should look like:: - broker.host = - broker.vhost = - broker.user = - broker.password = + celery.broker_url = amqp://rcuser:secret_password@localhost:5672/rhodevhost Full configuration example is below: @@ -34,35 +55,15 @@ 2. Configure Celery in the #################################### ### CELERY CONFIG #### #################################### - ## Set to true - use_celery = true - broker.host = localhost - broker.vhost = rabbitmqvhost - broker.port = 5672 - broker.user = rabbitmq - broker.password = secret - - celery.imports = rhodecode.lib.celerylib.tasks - celery.result.backend = amqp - celery.result.dburi = amqp:// - celery.result.serialier = json + use_celery = true + celery.broker_url = amqp://rcuser:secret@localhost:5672/rhodevhost - #celery.send.task.error.emails = true - #celery.amqp.task.result.expires = 18000 - - celeryd.concurrency = 2 - #celeryd.log.file = celeryd.log - celeryd.log.level = debug - celeryd.max.tasks.per.child = 1 + # maximum tasks to execute before worker restart + celery.max_tasks_per_child = 100 ## tasks will never be sent to the queue, but executed locally instead. - celery.always.eager = false - - -3. Enable celery, and install `celeryd` process script using the `enable-module`:: - - rccontrol enable-module celery {instance-id} + celery.task_always_eager = false .. _python: http://www.python.org/ @@ -70,4 +71,5 @@ 3. Enable celery, and install `celeryd` .. _celery: http://celeryproject.org/ .. _rabbitmq: http://www.rabbitmq.com/ .. _rabbitmq installation: http://docs.celeryproject.org/en/latest/getting-started/brokers/rabbitmq.html +.. _rabbitmq website installation: http://www.rabbitmq.com/download.html .. _Celery installation: http://docs.celeryproject.org/en/latest/getting-started/introduction.html#bundles diff --git a/docs/release-notes/release-notes-4.11.1.rst b/docs/release-notes/release-notes-4.11.1.rst new file mode 100644 --- /dev/null +++ b/docs/release-notes/release-notes-4.11.1.rst @@ -0,0 +1,54 @@ +|RCE| 4.11.1 |RNS| +------------------ + +Release Date +^^^^^^^^^^^^ + +- 2018-02-02 + + +New Features +^^^^^^^^^^^^ + + + +General +^^^^^^^ + + + +Security +^^^^^^^^ + + + +Performance +^^^^^^^^^^^ + + + +Fixes +^^^^^ + + +- Audit logs: handle query syntax errors in whoosh query parser. Prevents 500 + errors on wrongly entered search terms in audit logs. + +- Mercurial: fix new 4.4.X code change that does strict requirement checks. Fixes + problems with Mercurial Largefiles repositories. + +- VCSServer: in case of errors in the VCSServer worker restart it gracefully. + In rare cases certain errors caused locking of workers, and + unresponsive connections. Now we restart a worker freeing up memory and + connection. + +- Git: handle flaky and slow connection issues with git. Due to the changes in + Pyramid, flaky connections started affecting git clones. + + +Upgrade notes +^^^^^^^^^^^^^ + +- Fixed regression on git with high latency connections. + No upgrade problems should be expected, however please check GIT repos + behaviour on upgrade. diff --git a/docs/release-notes/release-notes.rst b/docs/release-notes/release-notes.rst --- a/docs/release-notes/release-notes.rst +++ b/docs/release-notes/release-notes.rst @@ -9,6 +9,7 @@ Release Notes .. toctree:: :maxdepth: 1 + release-notes-4.11.1.rst release-notes-4.11.0.rst release-notes-4.10.6.rst release-notes-4.10.5.rst diff --git a/pkgs/python-packages.nix b/pkgs/python-packages.nix --- a/pkgs/python-packages.nix +++ b/pkgs/python-packages.nix @@ -1680,7 +1680,7 @@ }; }; rhodecode-enterprise-ce = super.buildPythonPackage { - name = "rhodecode-enterprise-ce-4.11.0"; + name = "rhodecode-enterprise-ce-4.11.1"; buildInputs = with self; [pytest py pytest-cov pytest-sugar pytest-runner pytest-catchlog pytest-profiling gprof2dot pytest-timeout mock WebTest cov-core coverage configobj]; doCheck = true; propagatedBuildInputs = with self; [setuptools-scm amqp authomatic Babel Beaker celery Chameleon channelstream click colander configobj cssselect decorator deform docutils dogpile.cache dogpile.core ecdsa FormEncode future futures gnureadline infrae.cache iso8601 itsdangerous Jinja2 billiard kombu lxml Mako Markdown MarkupSafe msgpack-python MySQL-python objgraph packaging Paste PasteDeploy PasteScript pathlib2 peppercorn psutil psycopg2 py-bcrypt pycrypto pycurl pyflakes pygments-markdown-lexer Pygments pyparsing pyramid-beaker pyramid-debugtoolbar pyramid-jinja2 pyramid-mako pyramid pysqlite python-dateutil python-ldap python-memcached python-pam pytz pyzmq py-gfm recaptcha-client redis repoze.lru requests Routes setproctitle simplejson six SQLAlchemy sshpubkeys subprocess32 supervisor Tempita translationstring trollius urllib3 URLObject venusian WebError WebHelpers2 WebHelpers WebOb Whoosh wsgiref zope.cachedescriptors zope.deprecation zope.event zope.interface nbconvert bleach nbformat jupyter-client alembic invoke bumpversion transifex-client gevent greenlet gunicorn waitress uWSGI ipdb ipython CProfileV bottle rhodecode-tools appenlight-client pytest py pytest-cov pytest-sugar pytest-runner pytest-catchlog pytest-profiling gprof2dot pytest-timeout mock WebTest cov-core coverage]; diff --git a/rhodecode/i18n/de/LC_MESSAGES/rhodecode.mo b/rhodecode/i18n/de/LC_MESSAGES/rhodecode.mo index 59ecb1ac1088091803ad0a62ebd7bad77a96399c..40ea57040d2f4ca6ab943f315fb91ab4d5887f2f GIT binary patch literal 159919 zc$~bQ34qPj`|yWKQVA(3aw%(L?Aw&e7P2;$h~mz@cV;d#_fGfT!H`ly*(ys}3fU4O zYnGz4DJ_H)DUpz!cBS_@=kwfc4C?=X-?v}QbI&=?Ip;agcAm3+f8Un4bYg=1Gw||+ zgfaq;+@28k&zt=d5=skvOJGTXJNhf0=3g-Yc#FUy0vitiua&?X1U@LRxWIk_uM_x` zz#9cl6Ie~)YK1=tOcZ!cF7OV4_XzX~tRe7qfp-YpBe1K$vI7$mY6}brtS9glf#n76 z7T8eWKLR@mY&a+(p`F0d0$T~(DbQu;eMD#zI6`10f%^qk6?oO)goK&`>kF(SFkRDA zHT{XeiiR)oSVF>;0zCq+5csgB2R#PAMhLu0 z;5dcTwS1wbS82WX75{UEJGFejz`_DgY5I)7YXlY^LjDZ`uN7EcU@?L9hS2^!0?P_a z8A5y60xvgo4MDy`1x^w;L*VrS8$Ax}_&DvP3oIdUz~jh!l)%djzLw7wSV-U-k0bwe zkJJ8Ut@qXAjO(!0|3zRIfkmDGzxNa9Pv8mY9rXnDUl3T-@JZlp0vA0&KRXrgdw~@M z9##BeLlY97F?<*b-aLVq2wXfAJ}nyxpWYn`|2`Q?`(F=*u7g_sx8@gm5u~HWcQ|sHFr5CU45!^!hC}yT!{PU5T7S=Q`a3Q# zRp9T#;a86l@T-?X{|NZ`*a*spkD%R=Bha52BeeYy*x6?Srx|`dm5?xA;HIb0uU=21 z2fshfxC)L$A1jSS{Nf8Sqy=L;HILRu)*|S=vv2mhnFLEd0uNmUeQVMStdL{_E4@G(>2Fo6z%)1Jd$u73`F{{A`W`29KhyJQ^tP;wk} z+%gXN)DT!sU?+t^EgwG)IZhu({a3aAnsK!M(KzVYGY&oZeH{8zbUggHPT)-f8;_@6 z_wo4GjPc+ zZ6frJok;!Z6XEC5iPT#=k#T(}@KS+WCPMd@n!j7|4=6k_5j*>1BI7Cad_uylMlR3O zf6M2wmk!Tkmz|!+4|EsUU10y`p=ZnU^t1PQv3HvHD8NF$y&^;MB44Vx9 zUzm)X7EXpQ8z!S~pG-!t_6zK6_%oS)T1-Lz+6lZ`pnnSO4W7dIpP7Qb%$Ne7R!xDg z>!u*zO;hN1*A(>SgyR3B`K6}PPU2MNC@rs{ORA zq2t)g_??0?>A&<$@TzLM)=d0KlbOV?J!dkWelwX*N6&=LH8YXthcnTSZ)Oq){;7CL zv#^Jrv!HLVrY9?0qi~nPzh==++1ZT$_SuZ1m%?GQ(TfEFTN^!}jh96`+>b02*e?pr7)Lh0ncP{hcGJ*9?{kh1?@e2OA z)+^{$_gCOo;1%>{^efoibWOkU3iA2*725qqV6tiN73jTn9b;)6dcQ=+)ozk=qRmu*0-vwdNF#DvY7eMwHW-2#qcY% znDIZdn0e@h#jLy5EQbG=ypEpU_&WaNZh>tCroN7yFA^wIRl-k-_rwwzi@wLn_6ph`yaGB#uVCJIQS0qofgS&@^-HXT zztvWfuCtPHwbA_4mH6ep3UgON*NBzq$&{7E6|*(}^OeLKdj-nUF5#*-q>iyufijeY z!mH4qw^nJqunIaqS%sZ_qxq*+G48)sq0dEEW4HBJGtT>0S_^+Vy@|eWf0K64yonsHdW(7H*0+fJQ{F=F z9(xNpzW5e?Xw6&b?U!%iPY=I^JW8#je$#c#3)$<)pRf+P)~`cOTh<}(GwYzU#Cq(x z-Fo=fWj+0Q*2Djy0=F4?uZMrRZ`03=w^?_6_BQrE;T`&U;~nOskKTd)L+`+^LK_%w z!wvM`dIROXHz3!+8}M_>HbCdW4XmR|ybIr(z6)N?yNqvv!q49&{ynX*!h7h?eec1a zT!nMq!=G+>5Be{AAG|v6lkTDL>G#2V<9+0C_^~DGH)zTlL zSI!UNqfel#fW&_ScMIIRk$P``47_?1@_ayHzfH(>+$Qw#l}+$#(I)I<{U+w8?VHfs zJ)4l*nN7&~l27PA=@a6_+dskYKJy9wcF89?Z+*gg=x>1?4gWvIK9_t-`zt@iUf%hX z{y*3JJ)a_v!=J+En?9rb-p?3+kIyKd^%;I->u31KZ$87WF8drhuKXPQmIB8b`aWm8 zH*coBiksOtsJ|IM_wi=>|8g_@`Cik%ZpLm3Y$08G3+>#xg>m1u1^jkf;9u$%