##// END OF EJS Templates
docs: tweak formatting of the performance page - replace the odd numbered list with sections
Mads Kiilerich -
r6337:d02c715e default
parent child Browse files
Show More
@@ -1,99 +1,112 b''
1 1 .. _performance:
2 2
3 3 ================================
4 4 Optimizing Kallithea performance
5 5 ================================
6 6
7 When serving a large amount of big repositories, Kallithea can start
8 performing slower than expected. Because of the demanding nature of handling large
9 amounts of data from version control systems, here are some tips on how to get
10 the best performance.
7 When serving a large amount of big repositories, Kallithea can start performing
8 slower than expected. Because of the demanding nature of handling large amounts
9 of data from version control systems, here are some tips on how to get the best
10 performance.
11 11
12 Follow these few steps to improve performance of Kallithea system.
12
13 Fast storage
14 ------------
13 15
14 1. Kallithea is often I/O bound, and hence a fast disk (SSD/SAN) is
15 usually more important than a fast CPU.
16 Kallithea is often I/O bound, and hence a fast disk (SSD/SAN) and plenty of RAM
17 is usually more important than a fast CPU.
16 18
17 2. Increase cache
19
20 Caching
21 -------
18 22
19 Tweak beaker cache settings in the ini file. The actual effect of that
20 is questionable.
23 Tweak beaker cache settings in the ini file. The actual effect of that is
24 questionable.
21 25
22 3. Switch from SQLite to PostgreSQL or MySQL
26
27 Database
28 --------
23 29
24 30 SQLite is a good option when having a small load on the system. But due to
25 31 locking issues with SQLite, it is not recommended to use it for larger
26 deployments. Switching to MySQL or PostgreSQL will result in an immediate
27 performance increase. A tool like SQLAlchemyGrate_ can be used for
28 migrating to another database platform.
32 deployments.
33
34 Switching to MySQL or PostgreSQL will result in an immediate performance
35 increase. A tool like SQLAlchemyGrate_ can be used for migrating to another
36 database platform.
37
29 38
30 4. Scale Kallithea horizontally
39 Horizontal scaling
40 ------------------
31 41
32 Scaling horizontally can give huge performance benefits when dealing with
33 large amounts of traffic (many users, CI servers, etc.). Kallithea can be
34 scaled horizontally on one (recommended) or multiple machines.
42 Scaling horizontally means running several Kallithea instances and let them
43 share the load. That can give huge performance benefits when dealing with large
44 amounts of traffic (many users, CI servers, etc.). Kallithea can be scaled
45 horizontally on one (recommended) or multiple machines.
35 46
36 47 It is generally possible to run WSGI applications multithreaded, so that
37 several HTTP requests are served from the same Python process at once. That
38 can in principle give better utilization of internal caches and less
39 process overhead.
48 several HTTP requests are served from the same Python process at once. That can
49 in principle give better utilization of internal caches and less process
50 overhead.
40 51
41 One danger of running multithreaded is that program execution becomes much
42 more complex; programs must be written to consider all combinations of
43 events and problems might depend on timing and be impossible to reproduce.
52 One danger of running multithreaded is that program execution becomes much more
53 complex; programs must be written to consider all combinations of events and
54 problems might depend on timing and be impossible to reproduce.
44 55
45 56 Kallithea can't promise to be thread-safe, just like the embedded Mercurial
46 57 backend doesn't make any strong promises when used as Kallithea uses it.
47 58 Instead, we recommend scaling by using multiple server processes.
48 59
49 60 Web servers with multiple worker processes (such as ``mod_wsgi`` with the
50 61 ``WSGIDaemonProcess`` ``processes`` parameter) will work out of the box.
51 62
52 63 In order to scale horizontally on multiple machines, you need to do the
53 64 following:
54 65
55 66 - Each instance's ``data`` storage needs to be configured to be stored on a
56 67 shared disk storage, preferably together with repositories. This ``data``
57 68 dir contains template caches, sessions, whoosh index and is used for
58 69 task locking (so it is safe across multiple instances). Set the
59 70 ``cache_dir``, ``index_dir``, ``beaker.cache.data_dir``, ``beaker.cache.lock_dir``
60 71 variables in each .ini file to a shared location across Kallithea instances
61 72 - If using several Celery instances,
62 73 the message broker should be common to all of them (e.g., one
63 74 shared RabbitMQ server)
64 75 - Load balance using round robin or IP hash, recommended is writing LB rules
65 76 that will separate regular user traffic from automated processes like CI
66 77 servers or build bots.
67 78
68 5. Serve static files directly from the web server
79
80 Serve static files directly from the web server
81 -----------------------------------------------
69 82
70 83 With the default ``static_files`` ini setting, the Kallithea WSGI application
71 84 will take care of serving the static files found in ``kallithea/public`` from
72 85 the root of the application URL. While doing that, it will currently also
73 86 apply buffering and compression of all the responses it is serving.
74 87
75 88 The actual serving of the static files is unlikely to be a problem in a
76 89 Kallithea setup. The buffering of responses is more likely to be a problem;
77 90 large responses (clones or pulls) will have to be fully processed and spooled
78 91 to disk or memory before the client will see any response.
79 92
80 93 To serve static files from the web server, use something like this Apache config
81 94 snippet::
82 95
83 96 Alias /images/ /srv/kallithea/kallithea/kallithea/public/images/
84 97 Alias /css/ /srv/kallithea/kallithea/kallithea/public/css/
85 98 Alias /js/ /srv/kallithea/kallithea/kallithea/public/js/
86 99 Alias /codemirror/ /srv/kallithea/kallithea/kallithea/public/codemirror/
87 100 Alias /fontello/ /srv/kallithea/kallithea/kallithea/public/fontello/
88 101
89 102 Then disable serving of static files in the ``.ini`` ``app:main`` section::
90 103
91 104 static_files = false
92 105
93 106 If using Kallithea installed as a package, you should be able to find the files
94 107 under site-packages/kallithea, either in your Python installation or in your
95 108 virtualenv. When upgrading, make sure to update the web server configuration
96 109 too if necessary.
97 110
98 111
99 112 .. _SQLAlchemyGrate: https://github.com/shazow/sqlalchemygrate
General Comments 0
You need to be logged in to leave comments. Login now