##// END OF EJS Templates
docs: reduce double nesting level in performance.rst...
Thomas De Schampheleire -
r8370:d442d839 default
parent child Browse files
Show More
@@ -1,125 +1,125 b''
1 .. _performance:
1 .. _performance:
2
2
3 ================================
3 ================================
4 Optimizing Kallithea performance
4 Optimizing Kallithea performance
5 ================================
5 ================================
6
6
7 When serving a large amount of big repositories, Kallithea can start performing
7 When serving a large amount of big repositories, Kallithea can start performing
8 slower than expected. Because of the demanding nature of handling large amounts
8 slower than expected. Because of the demanding nature of handling large amounts
9 of data from version control systems, here are some tips on how to get the best
9 of data from version control systems, here are some tips on how to get the best
10 performance.
10 performance.
11
11
12
12
13 Fast storage
13 Fast storage
14 ------------
14 ------------
15
15
16 Kallithea is often I/O bound, and hence a fast disk (SSD/SAN) and plenty of RAM
16 Kallithea is often I/O bound, and hence a fast disk (SSD/SAN) and plenty of RAM
17 is usually more important than a fast CPU.
17 is usually more important than a fast CPU.
18
18
19
19
20 Caching
20 Caching
21 -------
21 -------
22
22
23 Tweak beaker cache settings in the ini file. The actual effect of that is
23 Tweak beaker cache settings in the ini file. The actual effect of that is
24 questionable.
24 questionable.
25
25
26 .. note::
26 .. note::
27
27
28 Beaker has no upper bound on cache size and will never drop any caches. For
28 Beaker has no upper bound on cache size and will never drop any caches. For
29 memory cache, the only option is to regularly restart the worker process.
29 memory cache, the only option is to regularly restart the worker process.
30 For file cache, it must be cleaned manually, as described in the `Beaker
30 For file cache, it must be cleaned manually, as described in the `Beaker
31 documentation <https://beaker.readthedocs.io/en/latest/sessions.html#removing-expired-old-sessions>`_::
31 documentation <https://beaker.readthedocs.io/en/latest/sessions.html#removing-expired-old-sessions>`_::
32
32
33 find data/cache -type f -mtime +30 -print -exec rm {} \;
33 find data/cache -type f -mtime +30 -print -exec rm {} \;
34
34
35
35
36 Database
36 Database
37 --------
37 --------
38
38
39 SQLite is a good option when having a small load on the system. But due to
39 SQLite is a good option when having a small load on the system. But due to
40 locking issues with SQLite, it is not recommended to use it for larger
40 locking issues with SQLite, it is not recommended to use it for larger
41 deployments.
41 deployments.
42
42
43 Switching to PostgreSQL or MariaDB/MySQL will result in an immediate performance
43 Switching to PostgreSQL or MariaDB/MySQL will result in an immediate performance
44 increase. A tool like SQLAlchemyGrate_ can be used for migrating to another
44 increase. A tool like SQLAlchemyGrate_ can be used for migrating to another
45 database platform.
45 database platform.
46
46
47
47
48 Horizontal scaling
48 Horizontal scaling
49 ------------------
49 ------------------
50
50
51 Scaling horizontally means running several Kallithea instances (also known as
51 Scaling horizontally means running several Kallithea instances (also known as
52 worker processes) and let them share the load. That is essential to serve other
52 worker processes) and let them share the load. That is essential to serve other
53 users while processing a long-running request from a user. Usually, the
53 users while processing a long-running request from a user. Usually, the
54 bottleneck on a Kallithea server is not CPU but I/O speed - especially network
54 bottleneck on a Kallithea server is not CPU but I/O speed - especially network
55 speed. It is thus a good idea to run multiple worker processes on one server.
55 speed. It is thus a good idea to run multiple worker processes on one server.
56
56
57 .. note::
57 .. note::
58
58
59 Kallithea and the embedded Mercurial backend are not thread-safe. Each
59 Kallithea and the embedded Mercurial backend are not thread-safe. Each
60 worker process must thus be single-threaded.
60 worker process must thus be single-threaded.
61
61
62 Web servers can usually launch multiple worker processes - for example ``mod_wsgi`` with the
62 Web servers can usually launch multiple worker processes - for example ``mod_wsgi`` with the
63 ``WSGIDaemonProcess`` ``processes`` parameter or ``uWSGI`` or ``gunicorn`` with
63 ``WSGIDaemonProcess`` ``processes`` parameter or ``uWSGI`` or ``gunicorn`` with
64 their ``workers`` setting.
64 their ``workers`` setting.
65
65
66 Kallithea can also be scaled horizontally across multiple machines.
66 Kallithea can also be scaled horizontally across multiple machines.
67 In order to scale horizontally on multiple machines, you need to do the
67 In order to scale horizontally on multiple machines, you need to do the
68 following:
68 following:
69
69
70 - Each instance's ``data`` storage needs to be configured to be stored on a
70 - Each instance's ``data`` storage needs to be configured to be stored on a
71 shared disk storage, preferably together with repositories. This ``data``
71 shared disk storage, preferably together with repositories. This ``data``
72 dir contains template caches, sessions, whoosh index and is used for
72 dir contains template caches, sessions, whoosh index and is used for
73 task locking (so it is safe across multiple instances). Set the
73 task locking (so it is safe across multiple instances). Set the
74 ``cache_dir``, ``index_dir``, ``beaker.cache.data_dir``, ``beaker.cache.lock_dir``
74 ``cache_dir``, ``index_dir``, ``beaker.cache.data_dir``, ``beaker.cache.lock_dir``
75 variables in each .ini file to a shared location across Kallithea instances
75 variables in each .ini file to a shared location across Kallithea instances
76 - If using several Celery instances,
76 - If using several Celery instances,
77 the message broker should be common to all of them (e.g., one
77 the message broker should be common to all of them (e.g., one
78 shared RabbitMQ server)
78 shared RabbitMQ server)
79 - Load balance using round robin or IP hash, recommended is writing LB rules
79 - Load balance using round robin or IP hash, recommended is writing LB rules
80 that will separate regular user traffic from automated processes like CI
80 that will separate regular user traffic from automated processes like CI
81 servers or build bots.
81 servers or build bots.
82
82
83
83
84 Serve static files directly from the web server
84 Serve static files directly from the web server
85 -----------------------------------------------
85 -----------------------------------------------
86
86
87 With the default ``static_files`` ini setting, the Kallithea WSGI application
87 With the default ``static_files`` ini setting, the Kallithea WSGI application
88 will take care of serving the static files from ``kallithea/public/`` at the
88 will take care of serving the static files from ``kallithea/public/`` at the
89 root of the application URL.
89 root of the application URL.
90
90
91 The actual serving of the static files is very fast and unlikely to be a
91 The actual serving of the static files is very fast and unlikely to be a
92 problem in a Kallithea setup - the responses generated by Kallithea from
92 problem in a Kallithea setup - the responses generated by Kallithea from
93 database and repository content will take significantly more time and
93 database and repository content will take significantly more time and
94 resources.
94 resources.
95
95
96 To serve static files from the web server, use something like this Apache config
96 To serve static files from the web server, use something like this Apache config
97 snippet::
97 snippet::
98
98
99 Alias /images/ /srv/kallithea/kallithea/kallithea/public/images/
99 Alias /images/ /srv/kallithea/kallithea/kallithea/public/images/
100 Alias /css/ /srv/kallithea/kallithea/kallithea/public/css/
100 Alias /css/ /srv/kallithea/kallithea/kallithea/public/css/
101 Alias /js/ /srv/kallithea/kallithea/kallithea/public/js/
101 Alias /js/ /srv/kallithea/kallithea/kallithea/public/js/
102 Alias /codemirror/ /srv/kallithea/kallithea/kallithea/public/codemirror/
102 Alias /codemirror/ /srv/kallithea/kallithea/kallithea/public/codemirror/
103 Alias /fontello/ /srv/kallithea/kallithea/kallithea/public/fontello/
103 Alias /fontello/ /srv/kallithea/kallithea/kallithea/public/fontello/
104
104
105 Then disable serving of static files in the ``.ini`` ``app:main`` section::
105 Then disable serving of static files in the ``.ini`` ``app:main`` section::
106
106
107 static_files = false
107 static_files = false
108
108
109 If using Kallithea installed as a package, you should be able to find the files
109 If using Kallithea installed as a package, you should be able to find the files
110 under ``site-packages/kallithea``, either in your Python installation or in your
110 under ``site-packages/kallithea``, either in your Python installation or in your
111 virtualenv. When upgrading, make sure to update the web server configuration
111 virtualenv. When upgrading, make sure to update the web server configuration
112 too if necessary.
112 too if necessary.
113
113
114 It might also be possible to improve performance by configuring the web server
114 It might also be possible to improve performance by configuring the web server
115 to compress responses (served from static files or generated by Kallithea) when
115 to compress responses (served from static files or generated by Kallithea) when
116 serving them. That might also imply buffering of responses - that is more
116 serving them. That might also imply buffering of responses - that is more
117 likely to be a problem; large responses (clones or pulls) will have to be fully
117 likely to be a problem; large responses (clones or pulls) will have to be fully
118 processed and spooled to disk or memory before the client will see any
118 processed and spooled to disk or memory before the client will see any
119 response. See the documentation for your web server.
119 response. See the documentation for your web server.
120
120
121
121
122 .. _SQLAlchemyGrate: https://github.com/shazow/sqlalchemygrate
122 .. _SQLAlchemyGrate: https://github.com/shazow/sqlalchemygrate
123 .. _mod_wsgi: https://modwsgi.readthedocs.io/
123 .. _mod_wsgi: https://modwsgi.readthedocs.io/
124 .. _uWSGI: https://uwsgi-docs.readthedocs.io/
124 .. _uWSGI: https://uwsgi-docs.readthedocs.io/
125 .. _gunicorn: http://pypi.python.org/pypi/gunicorn
125 .. _gunicorn: http://pypi.python.org/pypi/gunicorn
General Comments 0
You need to be logged in to leave comments. Login now