backend: add TurboGears2 DebugBar support

DebugBar is a toolbar overlayed over the Kallithea web interface, allowing
you to see:

* timing information of the current request, including profiling information
* request data, including GET data, POST data, cookies, headers and
  environment variables
* a list of executed database queries, including timing and result values

Enable it automatically when in debug mode and all requirements are met.
Add documentation.

docs/contributing.rst

@@ -1,227 +1,247 b''
 .. _contributing:
 
 =========================
 Contributing to Kallithea
 =========================
 
 Kallithea is developed and maintained by its users. Please join us and scratch
 your own itch.
 
 
 Infrastructure
 --------------
 
 The main repository is hosted on Our Own Kallithea (aka OOK) at
 https://kallithea-scm.org/repos/kallithea/, our self-hosted instance
 of Kallithea.
 
 For now, we use Bitbucket_ for `pull requests`_ and `issue tracking`_. The
 issue tracker is for tracking bugs, not for support, discussion, or ideas --
 please use the `mailing list`_ or :ref:`IRC <readme>` to reach the community.
 
 We use Weblate_ to translate the user interface messages into languages other
 than English. Join our project on `Hosted Weblate`_ to help us.
 To register, you can use your Bitbucket or GitHub account. See :ref:`translations`
 for more details.
 
 
 Getting started
 ---------------
 
 To get started with development::
 
         hg clone https://kallithea-scm.org/repos/kallithea
         cd kallithea
         virtualenv ../kallithea-venv
         source ../kallithea-venv/bin/activate
         pip install --upgrade pip setuptools
         pip install -e .
         gearbox make-config my.ini
         gearbox setup-db -c my.ini --user=user --email=[e-mail] --password=password --repos=/tmp
         gearbox serve -c my.ini --reload &
         firefox http://127.0.0.1:5000/
 
 You can also start out by forking https://bitbucket.org/conservancy/kallithea
 on Bitbucket_ and create a local clone of your own fork.
 
 
 Running tests
 -------------
 
 After finishing your changes make sure all tests pass cleanly. Install the test
 dependencies, then run the testsuite by invoking ``py.test`` from the
 project root::
 
     pip install -r dev_requirements.txt
     py.test
 
 Note that testing on Python 2.6 also requires ``unittest2``.
 
 You can also use ``tox`` to run the tests with all supported Python versions
 (currently Python 2.6--2.7).
 
 When running tests, Kallithea uses `kallithea/tests/test.ini` and populates the
 SQLite database specified there.
 
 It is possible to avoid recreating the full test database on each invocation of
 the tests, thus eliminating the initial delay. To achieve this, run the tests as::
 
     gearbox serve -c kallithea/tests/test.ini --pid-file=test.pid --daemon
     KALLITHEA_WHOOSH_TEST_DISABLE=1 KALLITHEA_NO_TMP_PATH=1 py.test
     kill -9 $(cat test.pid)
 
 In these commands, the following variables are used::
 
     KALLITHEA_WHOOSH_TEST_DISABLE=1 - skip whoosh index building and tests
     KALLITHEA_NO_TMP_PATH=1 - disable new temp path for tests, used mostly for testing_vcs_operations
 
 You can run individual tests by specifying their path as argument to py.test.
 py.test also has many more options, see `py.test -h`. Some useful options
 are::
 
     -k EXPRESSION         only run tests which match the given substring
                           expression. An expression is a python evaluable
                           expression where all names are substring-matched
                           against test names and their parent classes. Example:
     -x, --exitfirst       exit instantly on first error or failed test.
     --lf                  rerun only the tests that failed at the last run (or
                           all if none failed)
     --ff                  run all tests but run the last failures first. This
                           may re-order tests and thus lead to repeated fixture
                           setup/teardown
     --pdb                 start the interactive Python debugger on errors.
     -s, --capture=no      don't capture stdout (any stdout output will be
                           printed immediately)
 
 
 Contribution guidelines
 -----------------------
 
 Kallithea is GPLv3 and we assume all contributions are made by the
 committer/contributor and under GPLv3 unless explicitly stated. We do care a
 lot about preservation of copyright and license information for existing code
 that is brought into the project.
 
 Contributions will be accepted in most formats -- such as pull requests on
 Bitbucket, something hosted on your own Kallithea instance, or patches sent by
 email to the `kallithea-general`_ mailing list.
 
 When contributing via Bitbucket, please make your fork of
 https://bitbucket.org/conservancy/kallithea/ `non-publishing`_ -- it is one of
 the settings on "Repository details" page. This ensures your commits are in
 "draft" phase and makes it easier for you to address feedback and for project
 maintainers to integrate your changes.
 
 .. _non-publishing: https://www.mercurial-scm.org/wiki/Phases#Publishing_Repository
 
 Make sure to test your changes both manually and with the automatic tests
 before posting.
 
 We care about quality and review and keeping a clean repository history. We
 might give feedback that requests polishing contributions until they are
 "perfect". We might also rebase and collapse and make minor adjustments to your
 changes when we apply them.
 
 We try to make sure we have consensus on the direction the project is taking.
 Everything non-sensitive should be discussed in public -- preferably on the
 mailing list.  We aim at having all non-trivial changes reviewed by at least
 one other core developer before pushing. Obvious non-controversial changes will
 be handled more casually.
 
 For now we just have one official branch ("default") and will keep it so stable
 that it can be (and is) used in production. Experimental changes should live
 elsewhere (for example in a pull request) until they are ready.
 
 
 Coding guidelines
 -----------------
 
 We don't have a formal coding/formatting standard. We are currently using a mix
 of Mercurial's (https://www.mercurial-scm.org/wiki/CodingStyle), pep8, and
 consistency with existing code. Run ``scripts/run-all-cleanup`` before
 committing to ensure some basic code formatting consistency.
 
 We support both Python 2.6.x and 2.7.x and nothing else. For now we don't care
 about Python 3 compatibility.
 
 We try to support the most common modern web browsers. IE9 is still supported
 to the extent it is feasible, IE8 is not.
 
 We primarily support Linux and OS X on the server side but Windows should also work.
 
 HTML templates should use 2 spaces for indentation ... but be pragmatic. We
 should use templates cleverly and avoid duplication. We should use reasonable
 semantic markup with element classes and IDs that can be used for styling and testing.
 We should only use inline styles in places where it really is semantic (such as
 ``display: none``).
 
 JavaScript must use ``;`` between/after statements. Indentation 4 spaces. Inline
 multiline functions should be indented two levels -- one for the ``()`` and one for
 ``{}``.
 Variables holding jQuery objects should be named with a leading ``$``.
 
 Commit messages should have a leading short line summarizing the changes. For
 bug fixes, put ``(Issue #123)`` at the end of this line.
 
 Use American English grammar and spelling overall. Use `English title case`_ for
 page titles, button labels, headers, and 'labels' for fields in forms.
 
 .. _English title case: https://en.wikipedia.org/wiki/Capitalization#Title_case
 
 Template helpers (that is, everything in ``kallithea.lib.helpers``)
 should only be referenced from templates. If you need to call a
 helper from the Python code, consider moving the function somewhere
 else (e.g. to the model).
 
 Notes on the SQLAlchemy session
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Each HTTP request runs inside an independent SQLAlchemy session (as well
 as in an independent database transaction). Database model objects
 (almost) always belong to a particular SQLAlchemy session, which means
 that SQLAlchemy will ensure that they're kept in sync with the database
 (but also means that they cannot be shared across requests).
 
 Objects can be added to the session using ``Session().add``, but this is
 rarely needed:
 
 * When creating a database object by calling the constructor directly,
   it must explicitly be added to the session.
 
 * When creating an object using a factory function (like
   ``create_repo``), the returned object has already (by convention)
   been added to the session, and should not be added again.
 
 * When getting an object from the session (via ``Session().query`` or
   any of the utility functions that look up objects in the database),
   it's already part of the session, and should not be added again.
   SQLAlchemy monitors attribute modifications automatically for all
   objects it knows about and syncs them to the database.
 
 SQLAlchemy also flushes changes to the database automatically; manually
 calling ``Session().flush`` is usually only necessary when the Python
 code needs the database to assign an "auto-increment" primary key ID to
 a freshly created model object (before flushing, the ID attribute will
 be ``None``).
 
+TurboGears2 DebugBar
+^^^^^^^^^^^^^^^^^^^^
+
+It is possible to enable the TurboGears2-provided DebugBar_, a toolbar overlayed
+over the Kallithea web interface, allowing you to see:
+
+* timing information of the current request, including profiling information
+* request data, including GET data, POST data, cookies, headers and environment
+  variables
+* a list of executed database queries, including timing and result values
+
+DebugBar is only activated when ``debug = true`` is set in the configuration
+file. This is important, because the DebugBar toolbar will be visible for all
+users, and allow them to see information they should not be allowed to see. Like
+is anyway the case for ``debug = true``, do not use this in production!
+
+To enable DebugBar, install ``tgext.debugbar`` and ``kajiki`` (typically via
+``pip``) and restart Kallithea (in debug mode).
+
 
 "Roadmap"
 ---------
 
 We do not have a road map but are waiting for your contributions. Refer to the
 wiki_ for some ideas of places we might want to go -- contributions in these
 areas are very welcome.
 
 
 Thank you for your contribution!
 --------------------------------
 
 
 .. _Weblate: http://weblate.org/
 .. _issue tracking: https://bitbucket.org/conservancy/kallithea/issues?status=new&status=open
 .. _pull requests: https://bitbucket.org/conservancy/kallithea/pull-requests
 .. _bitbucket: http://bitbucket.org/
 .. _mailing list: http://lists.sfconservancy.org/mailman/listinfo/kallithea-general
 .. _kallithea-general: http://lists.sfconservancy.org/mailman/listinfo/kallithea-general
 .. _Hosted Weblate: https://hosted.weblate.org/projects/kallithea/kallithea/
 .. _wiki: https://bitbucket.org/conservancy/kallithea/wiki/Home
+.. _DebugBar: https://github.com/TurboGears/tgext.debugbar

kallithea/config/app_cfg.py

@@ -1,175 +1,190 b''
 # -*- coding: utf-8 -*-
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 """
 Global configuration file for TurboGears2 specific settings in Kallithea.
 
 This file complements the .ini file.
 """
 
 import platform
 import os, sys
 
 import tg
 from tg import hooks
 from tg.configuration import AppConfig
 from tg.support.converters import asbool
 
 from kallithea.lib.middleware.https_fixup import HttpsFixup
 from kallithea.lib.middleware.simplegit import SimpleGit
 from kallithea.lib.middleware.simplehg import SimpleHg
 from kallithea.config.routing import make_map
 from kallithea.lib.auth import set_available_permissions
 from kallithea.lib.db_manage import DbManage
 from kallithea.lib.utils import load_rcextensions, make_ui, set_app_settings, set_vcs_config, \
     set_indexer_config, check_git_version, repo2db_mapper
 from kallithea.lib.utils2 import str2bool
 from kallithea.model.scm import ScmModel
 
 import formencode
 import kallithea
 
 
 class KallitheaAppConfig(AppConfig):
     # Note: AppConfig has a misleading name, as it's not the application
     # configuration, but the application configurator. The AppConfig values are
     # used as a template to create the actual configuration, which might
     # overwrite or extend the one provided by the configurator template.
 
     # To make it clear, AppConfig creates the config and sets into it the same
     # values that AppConfig itself has. Then the values from the config file and
     # gearbox options are loaded and merged into the configuration. Then an
     # after_init_config(conf) method of AppConfig is called for any change that
     # might depend on options provided by configuration files.
 
     def __init__(self):
         super(KallitheaAppConfig, self).__init__()
 
         self['package'] = kallithea
 
         self['prefer_toscawidgets2'] = False
         self['use_toscawidgets'] = False
 
         self['renderers'] = []
 
         # Enable json in expose
         self['renderers'].append('json')
 
         # Configure template rendering
         self['renderers'].append('mako')
         self['default_renderer'] = 'mako'
         self['use_dotted_templatenames'] = False
 
         # Configure Sessions, store data as JSON to avoid pickle security issues
         self['session.enabled'] = True
         self['session.data_serializer'] = 'json'
 
         # Configure the base SQLALchemy Setup
         self['use_sqlalchemy'] = True
         self['model'] = kallithea.model.base
         self['DBSession'] = kallithea.model.meta.Session
 
         # Configure App without an authentication backend.
         self['auth_backend'] = None
 
         # Use custom error page for these errors. By default, Turbogears2 does not add
         # 400 in this list.
         # Explicitly listing all is considered more robust than appending to defaults,
         # in light of possible future framework changes.
         self['errorpage.status_codes'] = [400, 401, 403, 404]
 
         # Disable transaction manager -- currently Kallithea takes care of transactions itself
         self['tm.enabled'] = False
 
 base_config = KallitheaAppConfig()
 
 # TODO still needed as long as we use pylonslib
 sys.modules['pylons'] = tg
 
+# DebugBar, a debug toolbar for TurboGears2.
+# (https://github.com/TurboGears/tgext.debugbar)
+# To enable it, install 'tgext.debugbar' and 'kajiki', and run Kallithea with
+# 'debug = true' (not in production!)
+# See the Kallithea documentation for more information.
+try:
+    from tgext.debugbar import enable_debugbar
+    import kajiki # only to check its existence
+except ImportError:
+    pass
+else:
+    base_config['renderers'].append('kajiki')
+    enable_debugbar(base_config)
+
+
 def setup_configuration(app):
     config = app.config
 
     # store some globals into kallithea
     kallithea.CELERY_ON = str2bool(config['app_conf'].get('use_celery'))
     kallithea.CELERY_EAGER = str2bool(config['app_conf'].get('celery.always.eager'))
     kallithea.CONFIG = config
 
     # Provide routes mapper to the RoutedController
     root_controller = app.find_controller('root')
     root_controller.mapper = config['routes.map'] = make_map(config)
 
     load_rcextensions(root_path=config['here'])
 
     # FIXME move test setup code out of here
     test = os.path.split(config['__file__'])[-1] == 'test.ini'
     if test:
         test_env = not int(os.environ.get('KALLITHEA_NO_TMP_PATH', 0))
         test_index = not int(os.environ.get('KALLITHEA_WHOOSH_TEST_DISABLE', 0))
         if os.environ.get('TEST_DB'):
             # swap config if we pass environment variable
             config['sqlalchemy.url'] = os.environ.get('TEST_DB')
 
         from kallithea.tests.fixture import create_test_env, create_test_index
         from kallithea.tests.base import TESTS_TMP_PATH
         #set KALLITHEA_NO_TMP_PATH=1 to disable re-creating the database and
         #test repos
         if test_env:
             create_test_env(TESTS_TMP_PATH, config)
         #set KALLITHEA_WHOOSH_TEST_DISABLE=1 to disable whoosh index during tests
         if test_index:
             create_test_index(TESTS_TMP_PATH, config, True)
 
     set_available_permissions(config)
     repos_path = make_ui('db').configitems('paths')[0][1]
     config['base_path'] = repos_path
     set_app_settings(config)
 
     instance_id = kallithea.CONFIG.get('instance_id', '*')
     if instance_id == '*':
         instance_id = '%s-%s' % (platform.uname()[1], os.getpid())
         kallithea.CONFIG['instance_id'] = instance_id
 
     # update kallithea.CONFIG with the meanwhile changed 'config'
     kallithea.CONFIG.update(config)
 
     # configure vcs and indexer libraries (they are supposed to be independent
     # as much as possible and thus avoid importing tg.config or
     # kallithea.CONFIG).
     set_vcs_config(kallithea.CONFIG)
     set_indexer_config(kallithea.CONFIG)
 
     check_git_version()
 
     if str2bool(config.get('initial_repo_scan', True)):
         repo2db_mapper(ScmModel().repo_scan(repos_path),
                        remove_obsolete=False, install_git_hooks=False)
 
     formencode.api.set_stdtranslation(languages=[config.get('lang')])
 
 hooks.register('configure_new_app', setup_configuration)
 
 
 def setup_application(app):
     config = app.config
 
     # we want our low level middleware to get to the request ASAP. We don't
     # need any stack middleware in them - especially no StatusCodeRedirect buffering
     app = SimpleHg(app, config)
     app = SimpleGit(app, config)
 
     # Enable https redirects based on HTTP_X_URL_SCHEME set by proxy
     if any(asbool(config.get(x)) for x in ['https_fixup', 'force_https', 'use_htsts']):
         app = HttpsFixup(app, config)
     return app
 
 hooks.register('before_config', setup_application)