Show More
@@ -71,16 +71,16 b' pdebug = false' | |||||
71 | ## PASTE ## |
|
71 | ## PASTE ## | |
72 | #use = egg:Paste#http |
|
72 | #use = egg:Paste#http | |
73 | ## nr of worker threads to spawn |
|
73 | ## nr of worker threads to spawn | |
74 |
#threadpool_workers = |
|
74 | #threadpool_workers = 1 | |
75 | ## max request before thread respawn |
|
75 | ## max request before thread respawn | |
76 | #threadpool_max_requests = 10 |
|
76 | #threadpool_max_requests = 100 | |
77 | ## option to use threads of process |
|
77 | ## option to use threads of process | |
78 | #use_threadpool = true |
|
78 | #use_threadpool = true | |
79 |
|
79 | |||
80 | ## WAITRESS ## |
|
80 | ## WAITRESS ## | |
81 | use = egg:waitress#main |
|
81 | use = egg:waitress#main | |
82 | ## number of worker threads |
|
82 | ## number of worker threads | |
83 |
threads = |
|
83 | threads = 1 | |
84 | ## MAX BODY SIZE 100GB |
|
84 | ## MAX BODY SIZE 100GB | |
85 | max_request_body_size = 107374182400 |
|
85 | max_request_body_size = 107374182400 | |
86 | ## use poll instead of select, fixes fd limits, may not work on old |
|
86 | ## use poll instead of select, fixes fd limits, may not work on old |
@@ -735,8 +735,7 b' Here is a sample excerpt from an Apache ' | |||||
735 |
|
735 | |||
736 | .. code-block:: apache |
|
736 | .. code-block:: apache | |
737 |
|
737 | |||
738 | WSGIDaemonProcess kallithea \ |
|
738 | WSGIDaemonProcess kallithea processes=5 threads=1 maximum-requests=100 \ | |
739 | threads=4 \ |
|
|||
740 | python-home=/srv/kallithea/venv |
|
739 | python-home=/srv/kallithea/venv | |
741 | WSGIProcessGroup kallithea |
|
740 | WSGIProcessGroup kallithea | |
742 | WSGIScriptAlias / /srv/kallithea/dispatch.wsgi |
|
741 | WSGIScriptAlias / /srv/kallithea/dispatch.wsgi | |
@@ -746,7 +745,7 b' Or if using a dispatcher WSGI script wit' | |||||
746 |
|
745 | |||
747 | .. code-block:: apache |
|
746 | .. code-block:: apache | |
748 |
|
747 | |||
749 |
WSGIDaemonProcess kallithea threads= |
|
748 | WSGIDaemonProcess kallithea processes=5 threads=1 maximum-requests=100 | |
750 | WSGIProcessGroup kallithea |
|
749 | WSGIProcessGroup kallithea | |
751 | WSGIScriptAlias / /srv/kallithea/dispatch.wsgi |
|
750 | WSGIScriptAlias / /srv/kallithea/dispatch.wsgi | |
752 | WSGIPassAuthorization On |
|
751 | WSGIPassAuthorization On |
@@ -35,8 +35,26 b' 3. Scale Kallithea horizontally' | |||||
35 |
|
35 | |||
36 | Scaling horizontally can give huge performance benefits when dealing with |
|
36 | Scaling horizontally can give huge performance benefits when dealing with | |
37 | large amounts of traffic (many users, CI servers, etc.). Kallithea can be |
|
37 | large amounts of traffic (many users, CI servers, etc.). Kallithea can be | |
38 |
scaled horizontally on one (recommended) or multiple machines. |
|
38 | scaled horizontally on one (recommended) or multiple machines. | |
39 | to scale horizontally you need to do the following: |
|
39 | ||
|
40 | It is generally possible to run WSGI applications multithreaded, so that | |||
|
41 | several HTTP requests are served from the same Python process at once. That | |||
|
42 | can in principle give better utilization of internal caches and less | |||
|
43 | process overhead. | |||
|
44 | ||||
|
45 | One danger of running multithreaded is that program execution becomes much | |||
|
46 | more complex; programs must be written to consider all combinations of | |||
|
47 | events and problems might depend on timing and be impossible to reproduce. | |||
|
48 | ||||
|
49 | Kallithea can't promise to be thread-safe, just like the embedded Mercurial | |||
|
50 | backend doesn't make any strong promises when used as Kallithea uses it. | |||
|
51 | Instead, we recommend scaling by using multiple server processes. | |||
|
52 | ||||
|
53 | Web servers with multiple worker processes (such as ``mod_wsgi`` with the | |||
|
54 | ``WSGIDaemonProcess`` ``processes`` parameter) will work out of the box. | |||
|
55 | ||||
|
56 | In order to scale horizontally on multiple machines, you need to do the | |||
|
57 | following: | |||
40 |
|
58 | |||
41 | - Each instance needs its own .ini file and unique ``instance_id`` set. |
|
59 | - Each instance needs its own .ini file and unique ``instance_id`` set. | |
42 | - Each instance's ``data`` storage needs to be configured to be stored on a |
|
60 | - Each instance's ``data`` storage needs to be configured to be stored on a | |
@@ -45,7 +63,7 b' 3. Scale Kallithea horizontally' | |||||
45 | task locking (so it is safe across multiple instances). Set the |
|
63 | task locking (so it is safe across multiple instances). Set the | |
46 | ``cache_dir``, ``index_dir``, ``beaker.cache.data_dir``, ``beaker.cache.lock_dir`` |
|
64 | ``cache_dir``, ``index_dir``, ``beaker.cache.data_dir``, ``beaker.cache.lock_dir`` | |
47 | variables in each .ini file to a shared location across Kallithea instances |
|
65 | variables in each .ini file to a shared location across Kallithea instances | |
48 | - If celery is used each instance should run a separate Celery instance, but |
|
66 | - If using several Celery instances, | |
49 | the message broker should be common to all of them (e.g., one |
|
67 | the message broker should be common to all of them (e.g., one | |
50 | shared RabbitMQ server) |
|
68 | shared RabbitMQ server) | |
51 | - Load balance using round robin or IP hash, recommended is writing LB rules |
|
69 | - Load balance using round robin or IP hash, recommended is writing LB rules |
@@ -66,9 +66,9 b' pdebug = false' | |||||
66 | <%text>## PASTE ##</%text> |
|
66 | <%text>## PASTE ##</%text> | |
67 | use = egg:Paste#http |
|
67 | use = egg:Paste#http | |
68 | <%text>## nr of worker threads to spawn</%text> |
|
68 | <%text>## nr of worker threads to spawn</%text> | |
69 |
threadpool_workers = |
|
69 | threadpool_workers = 1 | |
70 | <%text>## max request before thread respawn</%text> |
|
70 | <%text>## max request before thread respawn</%text> | |
71 | threadpool_max_requests = 10 |
|
71 | threadpool_max_requests = 100 | |
72 | <%text>## option to use threads of process</%text> |
|
72 | <%text>## option to use threads of process</%text> | |
73 | use_threadpool = true |
|
73 | use_threadpool = true | |
74 |
|
74 | |||
@@ -76,7 +76,7 b' use_threadpool = true' | |||||
76 | <%text>## WAITRESS ##</%text> |
|
76 | <%text>## WAITRESS ##</%text> | |
77 | use = egg:waitress#main |
|
77 | use = egg:waitress#main | |
78 | <%text>## number of worker threads</%text> |
|
78 | <%text>## number of worker threads</%text> | |
79 |
threads = |
|
79 | threads = 1 | |
80 | <%text>## MAX BODY SIZE 100GB</%text> |
|
80 | <%text>## MAX BODY SIZE 100GB</%text> | |
81 | max_request_body_size = 107374182400 |
|
81 | max_request_body_size = 107374182400 | |
82 | <%text>## use poll instead of select, fixes fd limits, may not work on old</%text> |
|
82 | <%text>## use poll instead of select, fixes fd limits, may not work on old</%text> |
@@ -66,16 +66,16 b' pdebug = false' | |||||
66 | ## PASTE ## |
|
66 | ## PASTE ## | |
67 | #use = egg:Paste#http |
|
67 | #use = egg:Paste#http | |
68 | ## nr of worker threads to spawn |
|
68 | ## nr of worker threads to spawn | |
69 |
#threadpool_workers = |
|
69 | #threadpool_workers = 1 | |
70 | ## max request before thread respawn |
|
70 | ## max request before thread respawn | |
71 | #threadpool_max_requests = 10 |
|
71 | #threadpool_max_requests = 100 | |
72 | ## option to use threads of process |
|
72 | ## option to use threads of process | |
73 | #use_threadpool = true |
|
73 | #use_threadpool = true | |
74 |
|
74 | |||
75 | ## WAITRESS ## |
|
75 | ## WAITRESS ## | |
76 | use = egg:waitress#main |
|
76 | use = egg:waitress#main | |
77 | ## number of worker threads |
|
77 | ## number of worker threads | |
78 |
threads = |
|
78 | threads = 1 | |
79 | ## MAX BODY SIZE 100GB |
|
79 | ## MAX BODY SIZE 100GB | |
80 | max_request_body_size = 107374182400 |
|
80 | max_request_body_size = 107374182400 | |
81 | ## use poll instead of select, fixes fd limits, may not work on old |
|
81 | ## use poll instead of select, fixes fd limits, may not work on old |
@@ -70,16 +70,16 b' pdebug = false' | |||||
70 | ## PASTE ## |
|
70 | ## PASTE ## | |
71 | #use = egg:Paste#http |
|
71 | #use = egg:Paste#http | |
72 | ## nr of worker threads to spawn |
|
72 | ## nr of worker threads to spawn | |
73 |
#threadpool_workers = |
|
73 | #threadpool_workers = 1 | |
74 | ## max request before thread respawn |
|
74 | ## max request before thread respawn | |
75 | #threadpool_max_requests = 10 |
|
75 | #threadpool_max_requests = 100 | |
76 | ## option to use threads of process |
|
76 | ## option to use threads of process | |
77 | #use_threadpool = true |
|
77 | #use_threadpool = true | |
78 |
|
78 | |||
79 | ## WAITRESS ## |
|
79 | ## WAITRESS ## | |
80 | use = egg:waitress#main |
|
80 | use = egg:waitress#main | |
81 | ## number of worker threads |
|
81 | ## number of worker threads | |
82 |
threads = |
|
82 | threads = 1 | |
83 | ## MAX BODY SIZE 100GB |
|
83 | ## MAX BODY SIZE 100GB | |
84 | max_request_body_size = 107374182400 |
|
84 | max_request_body_size = 107374182400 | |
85 | ## use poll instead of select, fixes fd limits, may not work on old |
|
85 | ## use poll instead of select, fixes fd limits, may not work on old |
General Comments 0
You need to be logged in to leave comments.
Login now