Show More
@@ -70,16 +70,16 b' pdebug = false' | |||||
70 | ## PASTE ## |
|
70 | ## PASTE ## | |
71 | #use = egg:Paste#http |
|
71 | #use = egg:Paste#http | |
72 | ## nr of worker threads to spawn |
|
72 | ## nr of worker threads to spawn | |
73 |
#threadpool_workers = |
|
73 | #threadpool_workers = 1 | |
74 | ## max request before thread respawn |
|
74 | ## max request before thread respawn | |
75 | #threadpool_max_requests = 10 |
|
75 | #threadpool_max_requests = 100 | |
76 | ## option to use threads of process |
|
76 | ## option to use threads of process | |
77 | #use_threadpool = true |
|
77 | #use_threadpool = true | |
78 |
|
78 | |||
79 | ## WAITRESS ## |
|
79 | ## WAITRESS ## | |
80 | use = egg:waitress#main |
|
80 | use = egg:waitress#main | |
81 | ## number of worker threads |
|
81 | ## number of worker threads | |
82 |
threads = |
|
82 | threads = 1 | |
83 | ## MAX BODY SIZE 100GB |
|
83 | ## MAX BODY SIZE 100GB | |
84 | max_request_body_size = 107374182400 |
|
84 | max_request_body_size = 107374182400 | |
85 | ## use poll instead of select, fixes fd limits, may not work on old |
|
85 | ## use poll instead of select, fixes fd limits, may not work on old |
@@ -785,8 +785,7 b' Here is a sample excerpt from an Apache ' | |||||
785 |
|
785 | |||
786 | .. code-block:: apache |
|
786 | .. code-block:: apache | |
787 |
|
787 | |||
788 | WSGIDaemonProcess kallithea \ |
|
788 | WSGIDaemonProcess kallithea processes=5 threads=1 maximum-requests=100 \ | |
789 | threads=4 \ |
|
|||
790 | python-home=/srv/kallithea/venv |
|
789 | python-home=/srv/kallithea/venv | |
791 | WSGIProcessGroup kallithea |
|
790 | WSGIProcessGroup kallithea | |
792 | WSGIScriptAlias / /srv/kallithea/dispatch.wsgi |
|
791 | WSGIScriptAlias / /srv/kallithea/dispatch.wsgi | |
@@ -796,7 +795,7 b' Or if using a dispatcher WSGI script wit' | |||||
796 |
|
795 | |||
797 | .. code-block:: apache |
|
796 | .. code-block:: apache | |
798 |
|
797 | |||
799 |
WSGIDaemonProcess kallithea threads= |
|
798 | WSGIDaemonProcess kallithea processes=5 threads=1 maximum-requests=100 | |
800 | WSGIProcessGroup kallithea |
|
799 | WSGIProcessGroup kallithea | |
801 | WSGIScriptAlias / /srv/kallithea/dispatch.wsgi |
|
800 | WSGIScriptAlias / /srv/kallithea/dispatch.wsgi | |
802 | WSGIPassAuthorization On |
|
801 | WSGIPassAuthorization On |
@@ -31,8 +31,26 b' 4. Scale Kallithea horizontally' | |||||
31 |
|
31 | |||
32 | Scaling horizontally can give huge performance benefits when dealing with |
|
32 | Scaling horizontally can give huge performance benefits when dealing with | |
33 | large amounts of traffic (many users, CI servers, etc.). Kallithea can be |
|
33 | large amounts of traffic (many users, CI servers, etc.). Kallithea can be | |
34 |
scaled horizontally on one (recommended) or multiple machines. |
|
34 | scaled horizontally on one (recommended) or multiple machines. | |
35 | to scale horizontally you need to do the following: |
|
35 | ||
|
36 | It is generally possible to run WSGI applications multithreaded, so that | |||
|
37 | several HTTP requests are served from the same Python process at once. That | |||
|
38 | can in principle give better utilization of internal caches and less | |||
|
39 | process overhead. | |||
|
40 | ||||
|
41 | One danger of running multithreaded is that program execution becomes much | |||
|
42 | more complex; programs must be written to consider all combinations of | |||
|
43 | events and problems might depend on timing and be impossible to reproduce. | |||
|
44 | ||||
|
45 | Kallithea can't promise to be thread-safe, just like the embedded Mercurial | |||
|
46 | backend doesn't make any strong promises when used as Kallithea uses it. | |||
|
47 | Instead, we recommend scaling by using multiple server processes. | |||
|
48 | ||||
|
49 | Web servers with multiple worker processes (such as ``mod_wsgi`` with the | |||
|
50 | ``WSGIDaemonProcess`` ``processes`` parameter) will work out of the box. | |||
|
51 | ||||
|
52 | In order to scale horizontally on multiple machines, you need to do the | |||
|
53 | following: | |||
36 |
|
54 | |||
37 | - Each instance's ``data`` storage needs to be configured to be stored on a |
|
55 | - Each instance's ``data`` storage needs to be configured to be stored on a | |
38 | shared disk storage, preferably together with repositories. This ``data`` |
|
56 | shared disk storage, preferably together with repositories. This ``data`` | |
@@ -40,7 +58,7 b' 4. Scale Kallithea horizontally' | |||||
40 | task locking (so it is safe across multiple instances). Set the |
|
58 | task locking (so it is safe across multiple instances). Set the | |
41 | ``cache_dir``, ``index_dir``, ``beaker.cache.data_dir``, ``beaker.cache.lock_dir`` |
|
59 | ``cache_dir``, ``index_dir``, ``beaker.cache.data_dir``, ``beaker.cache.lock_dir`` | |
42 | variables in each .ini file to a shared location across Kallithea instances |
|
60 | variables in each .ini file to a shared location across Kallithea instances | |
43 | - If celery is used each instance should run a separate Celery instance, but |
|
61 | - If using several Celery instances, | |
44 | the message broker should be common to all of them (e.g., one |
|
62 | the message broker should be common to all of them (e.g., one | |
45 | shared RabbitMQ server) |
|
63 | shared RabbitMQ server) | |
46 | - Load balance using round robin or IP hash, recommended is writing LB rules |
|
64 | - Load balance using round robin or IP hash, recommended is writing LB rules |
@@ -65,9 +65,9 b' pdebug = false' | |||||
65 | <%text>## PASTE ##</%text> |
|
65 | <%text>## PASTE ##</%text> | |
66 | use = egg:Paste#http |
|
66 | use = egg:Paste#http | |
67 | <%text>## nr of worker threads to spawn</%text> |
|
67 | <%text>## nr of worker threads to spawn</%text> | |
68 |
threadpool_workers = |
|
68 | threadpool_workers = 1 | |
69 | <%text>## max request before thread respawn</%text> |
|
69 | <%text>## max request before thread respawn</%text> | |
70 | threadpool_max_requests = 10 |
|
70 | threadpool_max_requests = 100 | |
71 | <%text>## option to use threads of process</%text> |
|
71 | <%text>## option to use threads of process</%text> | |
72 | use_threadpool = true |
|
72 | use_threadpool = true | |
73 |
|
73 | |||
@@ -75,7 +75,7 b' use_threadpool = true' | |||||
75 | <%text>## WAITRESS ##</%text> |
|
75 | <%text>## WAITRESS ##</%text> | |
76 | use = egg:waitress#main |
|
76 | use = egg:waitress#main | |
77 | <%text>## number of worker threads</%text> |
|
77 | <%text>## number of worker threads</%text> | |
78 |
threads = |
|
78 | threads = 1 | |
79 | <%text>## MAX BODY SIZE 100GB</%text> |
|
79 | <%text>## MAX BODY SIZE 100GB</%text> | |
80 | max_request_body_size = 107374182400 |
|
80 | max_request_body_size = 107374182400 | |
81 | <%text>## use poll instead of select, fixes fd limits, may not work on old</%text> |
|
81 | <%text>## use poll instead of select, fixes fd limits, may not work on old</%text> |
@@ -65,16 +65,16 b' pdebug = false' | |||||
65 | ## PASTE ## |
|
65 | ## PASTE ## | |
66 | #use = egg:Paste#http |
|
66 | #use = egg:Paste#http | |
67 | ## nr of worker threads to spawn |
|
67 | ## nr of worker threads to spawn | |
68 |
#threadpool_workers = |
|
68 | #threadpool_workers = 1 | |
69 | ## max request before thread respawn |
|
69 | ## max request before thread respawn | |
70 | #threadpool_max_requests = 10 |
|
70 | #threadpool_max_requests = 100 | |
71 | ## option to use threads of process |
|
71 | ## option to use threads of process | |
72 | #use_threadpool = true |
|
72 | #use_threadpool = true | |
73 |
|
73 | |||
74 | ## WAITRESS ## |
|
74 | ## WAITRESS ## | |
75 | use = egg:waitress#main |
|
75 | use = egg:waitress#main | |
76 | ## number of worker threads |
|
76 | ## number of worker threads | |
77 |
threads = |
|
77 | threads = 1 | |
78 | ## MAX BODY SIZE 100GB |
|
78 | ## MAX BODY SIZE 100GB | |
79 | max_request_body_size = 107374182400 |
|
79 | max_request_body_size = 107374182400 | |
80 | ## use poll instead of select, fixes fd limits, may not work on old |
|
80 | ## use poll instead of select, fixes fd limits, may not work on old |
@@ -68,16 +68,16 b' pdebug = false' | |||||
68 | ## PASTE ## |
|
68 | ## PASTE ## | |
69 | #use = egg:Paste#http |
|
69 | #use = egg:Paste#http | |
70 | ## nr of worker threads to spawn |
|
70 | ## nr of worker threads to spawn | |
71 |
#threadpool_workers = |
|
71 | #threadpool_workers = 1 | |
72 | ## max request before thread respawn |
|
72 | ## max request before thread respawn | |
73 | #threadpool_max_requests = 10 |
|
73 | #threadpool_max_requests = 100 | |
74 | ## option to use threads of process |
|
74 | ## option to use threads of process | |
75 | #use_threadpool = true |
|
75 | #use_threadpool = true | |
76 |
|
76 | |||
77 | ## WAITRESS ## |
|
77 | ## WAITRESS ## | |
78 | use = egg:waitress#main |
|
78 | use = egg:waitress#main | |
79 | ## number of worker threads |
|
79 | ## number of worker threads | |
80 |
threads = |
|
80 | threads = 1 | |
81 | ## MAX BODY SIZE 100GB |
|
81 | ## MAX BODY SIZE 100GB | |
82 | max_request_body_size = 107374182400 |
|
82 | max_request_body_size = 107374182400 | |
83 | ## use poll instead of select, fixes fd limits, may not work on old |
|
83 | ## use poll instead of select, fixes fd limits, may not work on old |
General Comments 0
You need to be logged in to leave comments.
Login now