Show More
configure-celery.rst
85 lines
| 3.2 KiB
| text/x-rst
|
RstLexer
r2034 | ||||
.. _config-celery: | ||||
Configure Celery | ||||
---------------- | ||||
r2516 | Celery_ is an asynchronous task queue. It's a part of RhodeCode scheduler | |||
functionality. Celery_ makes certain heavy tasks perform more efficiently. | ||||
Most important it allows sending notification emails, create repository forks, | ||||
and import repositories in async way. It is also used for bi-directional | ||||
repository sync in scheduler. | ||||
r2034 | ||||
If you decide to use Celery you also need a working message queue. | ||||
r3913 | There are two fully supported message brokers is rabbitmq_ and redis_ (recommended). | |||
Since release 4.18.X we recommend using redis_ as a backend since it's generally | ||||
easier to work with, and results in simpler stack as redis is generally recommended | ||||
for caching purposes. | ||||
r2034 | ||||
r2516 | In order to install and configure Celery, follow these steps: | |||
r2034 | ||||
r3913 | 1. Install RabbitMQ or Redis for a message queue, see the documentation on the Celery website for | |||
`redis installation`_ or `rabbitmq installation`_ | ||||
r2516 | ||||
r3913 | 1a. If you choose RabbitMQ example configuration after installation would look like that:: | |||
r2516 | ||||
sudo rabbitmqctl add_user rcuser secret_password | ||||
sudo rabbitmqctl add_vhost rhodevhost | ||||
sudo rabbitmqctl set_user_tags rcuser rhodecode | ||||
sudo rabbitmqctl set_permissions -p rhodevhost rcuser ".*" ".*" ".*" | ||||
r2191 | ||||
r2516 | 2. Enable celery, and install `celery worker` process script using the `enable-module`:: | |||
rccontrol enable-module celery {instance-id} | ||||
.. note:: | ||||
In case when using multiple instances in one or multiple servers it's highly | ||||
recommended that celery is running only once, for all servers connected to | ||||
the same database. Having multiple celery instances running without special | ||||
reconfiguration could cause scheduler issues. | ||||
3. Configure Celery in the | ||||
r2034 | :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file. | |||
r2516 | Set the broker_url as minimal settings required to enable operation. | |||
If used our example data from pt 1a, here is how the broker url should look like:: | ||||
r2034 | ||||
r3913 | # for Redis | |||
celery.broker_url = redis://localhost:6379/8 | ||||
# for RabbitMQ | ||||
r2516 | celery.broker_url = amqp://rcuser:secret_password@localhost:5672/rhodevhost | |||
r2034 | ||||
r2191 | Full configuration example is below: | |||
r2034 | ||||
r2191 | .. code-block:: ini | |||
r2034 | ||||
# Set this section of the ini file to match your Celery installation | ||||
#################################### | ||||
### CELERY CONFIG #### | ||||
#################################### | ||||
r2516 | use_celery = true | |||
r3913 | celery.broker_url = redis://localhost:6379/8 | |||
r2034 | ||||
r2516 | # maximum tasks to execute before worker restart | |||
r4890 | celery.max_tasks_per_child = 20 | |||
r2034 | ||||
## tasks will never be sent to the queue, but executed locally instead. | ||||
r2516 | celery.task_always_eager = false | |||
r2034 | ||||
.. _python: http://www.python.org/ | ||||
.. _mercurial: http://mercurial.selenic.com/ | ||||
.. _celery: http://celeryproject.org/ | ||||
r3913 | .. _redis: http://redis.io | |||
.. _redis installation: https://redis.io/topics/quickstart | ||||
r2034 | .. _rabbitmq: http://www.rabbitmq.com/ | |||
.. _rabbitmq installation: http://docs.celeryproject.org/en/latest/getting-started/brokers/rabbitmq.html | ||||
r2516 | .. _rabbitmq website installation: http://www.rabbitmq.com/download.html | |||
r2034 | .. _Celery installation: http://docs.celeryproject.org/en/latest/getting-started/introduction.html#bundles | |||