##// END OF EJS Templates
fix(file-caching): fixed cases when old cache was used before changes to operate on bytestrings
r5651:bad147da default
Show More
configure-celery.rst
85 lines | 3.2 KiB | text/x-rst | RstLexer
/ docs / install / configure-celery.rst
docs: updated enable celery documentation
r2034
.. _config-celery:
Configure Celery
----------------
docs: updated celery docs
r2516 Celery_ is an asynchronous task queue. It's a part of RhodeCode scheduler
functionality. Celery_ makes certain heavy tasks perform more efficiently.
Most important it allows sending notification emails, create repository forks,
and import repositories in async way. It is also used for bi-directional
repository sync in scheduler.
docs: updated enable celery documentation
r2034
If you decide to use Celery you also need a working message queue.
packages: updated celery to 4.3.0 and switch default backend to redis...
r3913 There are two fully supported message brokers is rabbitmq_ and redis_ (recommended).
Since release 4.18.X we recommend using redis_ as a backend since it's generally
easier to work with, and results in simpler stack as redis is generally recommended
for caching purposes.
docs: updated enable celery documentation
r2034
docs: updated celery docs
r2516 In order to install and configure Celery, follow these steps:
docs: updated enable celery documentation
r2034
packages: updated celery to 4.3.0 and switch default backend to redis...
r3913 1. Install RabbitMQ or Redis for a message queue, see the documentation on the Celery website for
`redis installation`_ or `rabbitmq installation`_
docs: updated celery docs
r2516
packages: updated celery to 4.3.0 and switch default backend to redis...
r3913 1a. If you choose RabbitMQ example configuration after installation would look like that::
docs: updated celery docs
r2516
sudo rabbitmqctl add_user rcuser secret_password
sudo rabbitmqctl add_vhost rhodevhost
sudo rabbitmqctl set_user_tags rcuser rhodecode
sudo rabbitmqctl set_permissions -p rhodevhost rcuser ".*" ".*" ".*"
docs: update celery docs
r2191
docs: updated celery docs
r2516 2. Enable celery, and install `celery worker` process script using the `enable-module`::
rccontrol enable-module celery {instance-id}
.. note::
In case when using multiple instances in one or multiple servers it's highly
recommended that celery is running only once, for all servers connected to
the same database. Having multiple celery instances running without special
reconfiguration could cause scheduler issues.
3. Configure Celery in the
docs: updated enable celery documentation
r2034 :file:`home/{user}/.rccontrol/{instance-id}/rhodecode.ini` file.
docs: updated celery docs
r2516 Set the broker_url as minimal settings required to enable operation.
If used our example data from pt 1a, here is how the broker url should look like::
docs: updated enable celery documentation
r2034
packages: updated celery to 4.3.0 and switch default backend to redis...
r3913 # for Redis
celery.broker_url = redis://localhost:6379/8
# for RabbitMQ
docs: updated celery docs
r2516 celery.broker_url = amqp://rcuser:secret_password@localhost:5672/rhodevhost
docs: updated enable celery documentation
r2034
docs: update celery docs
r2191 Full configuration example is below:
docs: updated enable celery documentation
r2034
docs: update celery docs
r2191 .. code-block:: ini
docs: updated enable celery documentation
r2034
# Set this section of the ini file to match your Celery installation
####################################
### CELERY CONFIG ####
####################################
docs: updated celery docs
r2516 use_celery = true
packages: updated celery to 4.3.0 and switch default backend to redis...
r3913 celery.broker_url = redis://localhost:6379/8
docs: updated enable celery documentation
r2034
docs: updated celery docs
r2516 # maximum tasks to execute before worker restart
config: reduce max-task-per-child to 20
r4890 celery.max_tasks_per_child = 20
docs: updated enable celery documentation
r2034
## tasks will never be sent to the queue, but executed locally instead.
docs: updated celery docs
r2516 celery.task_always_eager = false
docs: updated enable celery documentation
r2034
.. _python: http://www.python.org/
.. _mercurial: http://mercurial.selenic.com/
.. _celery: http://celeryproject.org/
packages: updated celery to 4.3.0 and switch default backend to redis...
r3913 .. _redis: http://redis.io
.. _redis installation: https://redis.io/topics/quickstart
docs: updated enable celery documentation
r2034 .. _rabbitmq: http://www.rabbitmq.com/
.. _rabbitmq installation: http://docs.celeryproject.org/en/latest/getting-started/brokers/rabbitmq.html
docs: updated celery docs
r2516 .. _rabbitmq website installation: http://www.rabbitmq.com/download.html
docs: updated enable celery documentation
r2034 .. _Celery installation: http://docs.celeryproject.org/en/latest/getting-started/introduction.html#bundles