changes for release 1.1.5

README.rst

@@ -1,131 +1,132 b''
 
 =================================================
 Welcome to RhodeCode (RhodiumCode) documentation!
 =================================================
 
 ``RhodeCode`` (formerly hg-app) is a Pylons framework based Mercurial repository 
 browser/management tool with a built in push/pull server and full text search.
 It works on http/https and has a built in permission/authentication system with 
 the ability to authenticate via LDAP.
 
-RhodeCode is similar in some respects to github or bitbucket, 
-however RhodeCode can be run as standalone hosted application on your own server.  It is open source 
-and donation ware and focuses more on providing a customized, self administered 
-interface for Mercurial(and soon GIT) repositories. RhodeCode is powered by a vcs_ 
-library that Lukasz Balcerzak and I created to handle multiple different version 
-control systems.
+RhodeCode is similar in some respects to github or bitbucket_, 
+however RhodeCode can be run as standalone hosted application on your own server.  
+It is open source and donation ware and focuses more on providing a customized, 
+self administered interface for Mercurial(and soon GIT) repositories. 
+RhodeCode is powered by a vcs_ library that Lukasz Balcerzak and I created to 
+handle multiple different version control systems.
 
 RhodeCode uses `Semantic Versioning <http://semver.org/>`_
 
 RhodeCode demo
 --------------
 
-http://hg.python-works.com
+http://demo.rhodecode.org
 
 The default access is anonymous but you can login to an administrative account
 using the following credentials:
 
 - username: demo
 - password: demo
 
 Source code
 -----------
 
-The latest source for RhodeCode can be obtained from my own RhodeCode instance
-https://rhodecode.org 
+The latest source for RhodeCode can be obtained from official RhodeCode instance
+https://hg.rhodecode.org 
 
 Rarely updated source code and issue tracker is available at bitbcuket
 http://bitbucket.org/marcinkuzminski/rhodecode
 
 Installation
 ------------
 
 Please visit http://packages.python.org/RhodeCode/installation.html
 
 
 RhodeCode Features
 ------------------
 
 - Has it's own middleware to handle mercurial_ protocol requests. 
   Each request can be logged and authenticated.
 - Runs on threads unlike hgweb. You can make multiple pulls/pushes simultaneous. Supports http/https 
   and LDAP
 - Full permissions (private/read/write/admin) and authentication per project. 
   One account for web interface and mercurial_ push/pull/clone operations.
 - Mako templates let's you customize the look and feel of the application.
 - Beautiful diffs, annotations and source code browsing all colored by pygments.
 - Mercurial_ branch graph and yui-flot powered graphs with zooming and statistics
 - Admin interface with user/permission management. Admin activity journal, logs
   pulls, pushes, forks, registrations and other actions made by all users.
 - Server side forks. It is possible to fork a project and modify it freely without
   breaking the main repository.
 - Full text search powered by Whoosh on the source files, and file names.
   Build in indexing daemons, with optional incremental index build
   (no external search servers required all in one application)
 - Setup project descriptions and info inside built in db for easy, non 
   file-system operations
 - Intelligent cache with invalidation after push or project change, provides high 
   performance and always up to date data.
 - Rss / atom feeds, gravatar support, download sources as zip/tar/gz
 - Async tasks for speed and performance using celery_ (works without them too)  
 - Backup scripts can do backup of whole app and send it over scp to desired 
   location 
 - Based on pylons / sqlalchemy / sqlite / whoosh / vcs
 
 
 .. include:: ./docs/screenshots.rst
     
     
 Incoming / Plans
 ----------------
 
 - Project grouping
 - User groups/teams
 - SSH based authentication with server side key management
 - Code review (probably based on hg-review)
 - Full git_ support, with push/pull server (currently in beta tests)
 - Redmine integration
 - Public accessible activity feeds
 - Commit based built in wiki system
 - Clone points and cloning from remote repositories into RhodeCode
 - More statistics and graph (global annotation + some more statistics)
 - Other advancements as development continues (or you can of course make additions and or requests)
 
 License
 -------
 
 ``RhodeCode`` is released under the GPL_ license.
 
 
 Mailing group Q&A
 -----------------
 
 Join the `Google group <http://groups.google.com/group/rhodecode>`_
 
 Open an issue at `issue tracker <http://bitbucket.org/marcinkuzminski/rhodecode/issues>`_
 
 Join #rhodecode on FreeNode (irc.freenode.net)
 or use http://webchat.freenode.net/?channels=rhodecode for web access to irc.
 
 Online documentation
 --------------------
 
 Online documentation for the current version of RhodeCode is available at
 http://packages.python.org/RhodeCode/.
 You may also build the documentation for yourself - go into ``docs/`` and run::
 
    make html
 
 (You need to have sphinx installed to build the documentation. If you don't
 have sphinx installed you can install it via the command: ``easy_install sphinx``)
  
 .. _virtualenv: http://pypi.python.org/pypi/virtualenv
 .. _python: http://www.python.org/
 .. _django: http://www.djangoproject.com/
 .. _mercurial: http://mercurial.selenic.com/
+.. _bitbucket: http://bitbucket.org/
 .. _subversion: http://subversion.tigris.org/
 .. _git: http://git-scm.com/
 .. _celery: http://celeryproject.org/
 .. _Sphinx: http://sphinx.pocoo.org/
 .. _GPL: http://www.gnu.org/licenses/gpl.html
 .. _vcs: http://pypi.python.org/pypi/vcs
  No newline at end of file

docs/changelog.rst

@@ -1,191 +1,213 b''
 .. _changelog:
 
 Changelog
 =========
 
+
+1.1.5 (**2011-03-1X**)
+======================
+
+news
+----
+
+- basic windows support, by exchanging pybcrypt into sha256 for windows only
+  highly inspired by idea of mantis406
+
+fixes
+-----
+
+- fixed sorting by author in main page
+- fixed crashes with diffs on binary files
+- fixed #131 problem with boolean values for LDAP
+- fixed #122 mysql problems thanks to striker69 
+- fixed problem with errors on calling raw/raw_files/annotate functions 
+  with unknown revisions
+- fixed returned rawfiles attachment names with international character
+- cleaned out docs, big thanks to Jason Harris
+
 1.1.4 (**2011-02-19**)
 ======================
 
 news
 ----
 
 fixes
 -----
 
 - fixed formencode import problem on settings page, that caused server crash
   when that page was accessed as first after server start
 - journal fixes
 - fixed option to access repository just by entering http://server/<repo_name> 
 
 
 1.1.3 (**2011-02-16**)
 ======================
 
 news
 ----
 
 - implemented #102 allowing the '.' character in username
 - added option to access repository just by entering http://server/<repo_name>
 - celery task ignores result for better performance
 
 fixes
 -----
 
 - fixed ehlo command and non auth mail servers on smtp_lib. Thanks to 
   apollo13 and Johan Walles
 - small fixes in journal
 - fixed problems with getting setting for celery from .ini files
 - registration, password reset and login boxes share the same title as main 
   application now
 - fixed #113: to high permissions to fork repository
 - fixed problem with '[' chars in commit messages in journal
 - removed issue with space inside renamed repository after deletion
 - db transaction fixes when filesystem repository creation failed
 - fixed #106 relation issues on databases different than sqlite
 - fixed static files paths links to use of url() method
 
 1.1.2 (**2011-01-12**)
 ======================
 
 news
 ----
 
 
 fixes
 -----
 
 - fixes #98 protection against float division of percentage stats
 - fixed graph bug
 - forced webhelpers version since it was making troubles during installation 
 
 1.1.1 (**2011-01-06**)
 ======================
  
 news
 ----
 
 - added force https option into ini files for easier https usage (no need to
   set server headers with this options)
 - small css updates
 
 fixes
 -----
 
 - fixed #96 redirect loop on files view on repositories without changesets
 - fixed #97 unicode string passed into server header in special cases (mod_wsgi)
   and server crashed with errors
 - fixed large tooltips problems on main page
 - fixed #92 whoosh indexer is more error proof
 
 1.1.0 (**2010-12-18**)
 ======================
 
 news
 ----
 
 - rewrite of internals for vcs >=0.1.10
 - uses mercurial 1.7 with dotencode disabled for maintaining compatibility 
   with older clients
 - anonymous access, authentication via ldap
 - performance upgrade for cached repos list - each repository has it's own 
   cache that's invalidated when needed.
 - performance upgrades on repositories with large amount of commits (20K+)
 - main page quick filter for filtering repositories
 - user dashboards with ability to follow chosen repositories actions
 - sends email to admin on new user registration
 - added cache/statistics reset options into repository settings
 - more detailed action logger (based on hooks) with pushed changesets lists
   and options to disable those hooks from admin panel
 - introduced new enhanced changelog for merges that shows more accurate results
 - new improved and faster code stats (based on pygments lexers mapping tables, 
   showing up to 10 trending sources for each repository. Additionally stats
   can be disabled in repository settings.
 - gui optimizations, fixed application width to 1024px
 - added cut off (for large files/changesets) limit into config files
 - whoosh, celeryd, upgrade moved to paster command
 - other than sqlite database backends can be used
 
 fixes
 -----
 
 - fixes #61 forked repo was showing only after cache expired
 - fixes #76 no confirmation on user deletes
 - fixes #66 Name field misspelled
 - fixes #72 block user removal when he owns repositories
 - fixes #69 added password confirmation fields
 - fixes #87 RhodeCode crashes occasionally on updating repository owner
 - fixes #82 broken annotations on files with more than 1 blank line at the end
 - a lot of fixes and tweaks for file browser
 - fixed detached session issues
 - fixed when user had no repos he would see all repos listed in my account
 - fixed ui() instance bug when global hgrc settings was loaded for server 
   instance and all hgrc options were merged with our db ui() object
 - numerous small bugfixes
  
 (special thanks for TkSoh for detailed feedback)
 
 
 1.0.2 (**2010-11-12**)
 ======================
 
 news
 ----
 
 - tested under python2.7
 - bumped sqlalchemy and celery versions
 
 fixes
 -----
 
 - fixed #59 missing graph.js
 - fixed repo_size crash when repository had broken symlinks
 - fixed python2.5 crashes.
 
 
 1.0.1 (**2010-11-10**)
 ======================
 
 news
 ----
 
 - small css updated
 
 fixes
 -----
 
 - fixed #53 python2.5 incompatible enumerate calls
 - fixed #52 disable mercurial extension for web
 - fixed #51 deleting repositories don't delete it's dependent objects
 
 
 1.0.0 (**2010-11-02**)
 ======================
 
 - security bugfix simplehg wasn't checking for permissions on commands
   other than pull or push.
 - fixed doubled messages after push or pull in admin journal
 - templating and css corrections, fixed repo switcher on chrome, updated titles
 - admin menu accessible from options menu on repository view
 - permissions cached queries
 
 1.0.0rc4  (**2010-10-12**)
 ==========================
 
 - fixed python2.5 missing simplejson imports (thanks to Jens Bäckman)
 - removed cache_manager settings from sqlalchemy meta
 - added sqlalchemy cache settings to ini files
 - validated password length and added second try of failure on paster setup-app
 - fixed setup database destroy prompt even when there was no db
 
 
 1.0.0rc3 (**2010-10-11**)
 =========================
 
 - fixed i18n during installation.
 
 1.0.0rc2 (**2010-10-11**)
 =========================
 
 - Disabled dirsize in file browser, it's causing nasty bug when dir renames 
   occure. After vcs is fixed it'll be put back again.
 - templating/css rewrites, optimized css.
  No newline at end of file

docs/contributing.rst

@@ -1,19 +1,25 b''
 .. _contributing:
 
 Contributing to RhodeCode
 =========================
 
 If you would like to contribute to RhodeCode, please contact me, any help is
 greatly appreciated!
 
 Could I request that you make your source contributions by first forking the
-RhodeCode repository on bitbucket
+RhodeCode repository on bitbucket_
 https://bitbucket.org/marcinkuzminski/rhodecode and then make your changes to
-your forked repository. Finally, when you are finished making a change, please
-send me a pull request.
+your forked repository. Please post all fixes into **BETA** branch since your 
+fix might be already fixed there and i try to merge all fixes from beta into
+stable, and not the other way. Finally, when you are finished making a change, 
+please send me a pull request.
 
 To run RhodeCode in a development version you always need to install the tip
 version of RhodeCode and the VCS library.
 
 | Thank you for any contributions!
-|  Marcin
  No newline at end of file
+|  Marcin
+
+
+
+.. _bitbucket: http://bitbucket.org/

docs/index.rst

@@ -1,56 +1,57 b''
 .. _index:
 
 .. include:: ./../README.rst
 
 Documentation
 -------------
 
 **Installation:**
 
 .. toctree::
    :maxdepth: 1
 
    installation
    setup
    upgrade
    
 **Usage**
 
 .. toctree::
    :maxdepth: 1
 
    enable_git
    statistics
    
 **Develop**
 
 .. toctree::
    :maxdepth: 1
    
    contributing
    changelog
 
 **API**
 
 .. toctree::
    :maxdepth: 2
 
    api/index
    
 
 Other topics
 ------------
 
 * :ref:`genindex`
 * :ref:`search`
 
 .. _virtualenv: http://pypi.python.org/pypi/virtualenv
 .. _python: http://www.python.org/
 .. _django: http://www.djangoproject.com/
 .. _mercurial: http://mercurial.selenic.com/
+.. _bitbucket: http://bitbucket.org/
 .. _subversion: http://subversion.tigris.org/
 .. _git: http://git-scm.com/
 .. _celery: http://celeryproject.org/
 .. _Sphinx: http://sphinx.pocoo.org/
 .. _GPL: http://www.gnu.org/licenses/gpl.html
 .. _vcs: http://pypi.python.org/pypi/vcs

docs/installation.rst

@@ -1,105 +1,106 b''
 .. _installation:
 
 Installation
 ============
 
 ``RhodeCode`` is written entirely in Python. In order to gain maximum performance
 there are some third-party you must install. When RhodeCode is used 
 together with celery you have to install some kind of message broker,
 recommended one is rabbitmq_ to make the async tasks work.
 
 Of course RhodeCode works in sync mode also and then you do not have to install
-any third party applications. However, using Celery_ will give you a large speed improvement when using
-many big repositories. If you plan to use RhodeCode for say 7 to 10 small repositories, RhodeCode
-will perform perfectly well without celery running.
+any third party applications. However, using Celery_ will give you a large 
+speed improvement when using many big repositories. If you plan to use 
+RhodeCode for say 7 to 10 small repositories, RhodeCode will perform perfectly 
+well without celery running.
    
-If you make the decision to run RhodeCode with celery make sure you run celeryd using paster
-and message broker together with the application.   
+If you make the decision to run RhodeCode with celery make sure you run 
+celeryd using paster and message broker together with the application.   
 
 Installing RhodeCode from Cheese Shop
 -------------------------------------
 
 Rhodecode requires python version 2.5 or higher.
 
 The easiest way to install ``rhodecode`` is to run::
 
     easy_install rhodecode
 
 Or::
 
     pip install rhodecode
 
 If you prefer to install RhodeCode manually simply grab latest release from
 http://pypi.python.org/pypi/rhodecode, decompress the archive and run::
 
     python setup.py install
 
 
 Step by step installation example
 ---------------------------------
 
 
 - Assuming you have installed virtualenv_ create a new virtual environment using virtualenv:: 
 
     virtualenv --no-site-packages /var/www/rhodecode-venv
 
 
 .. note:: Using ``--no-site-packages`` when generating your
    virtualenv is **very important**. This flag provides the necessary
    isolation for running the set of packages required by
    RhodeCode.  If you do not specify ``--no-site-packages``,
    it's possible that RhodeCode will not install properly into
    the virtualenv, or, even if it does, may not run properly,
    depending on the packages you've already got installed into your
    Python's "main" site-packages dir.
 
 
 - this will install new virtualenv_ into `/var/www/rhodecode-venv`. 
 - Activate the virtualenv_ by running::
 
     source /var/www/rhodecode-venv/bin/activate
 
 .. note:: If you're using UNIX, *do not* use ``sudo`` to run the
    ``virtualenv`` script.  It's perfectly acceptable (and desirable)
    to create a virtualenv as a normal user.
      
 - Make a folder for rhodecode somewhere on the filesystem for example::
 
     mkdir /var/www/rhodecode
   
     
 - Run this command to install rhodecode::
 
     easy_install rhodecode 
 
 - This will install rhodecode together with pylons and all other required python
   libraries
 
 Requirements for Celery (optional)
 ----------------------------------
 
 .. note::
    Installing message broker and using celery is optional, RhodeCode will
    work perfectly fine without them.
 
 
 **Message Broker** 
 
 - preferred is `RabbitMq <http://www.rabbitmq.com/>`_
 - A possible alternative is `Redis <http://code.google.com/p/redis/>`_
 
 For installation instructions you can visit: 
 http://ask.github.com/celery/getting-started/index.html.
 This is a very nice tutorial on how to start using celery_ with rabbitmq_
 
 
 You can now proceed to :ref:`setup`
 -----------------------------------
 
 
 
 .. _virtualenv: http://pypi.python.org/pypi/virtualenv  
 .. _python: http://www.python.org/
 .. _mercurial: http://mercurial.selenic.com/
 .. _celery: http://celeryproject.org/
 .. _rabbitmq: http://www.rabbitmq.com/
  No newline at end of file

docs/setup.rst

@@ -1,384 +1,384 b''
 .. _setup:
 
 Setup
 =====
 
 
 Setting up RhodeCode
 --------------------------
 
 First, you will need to create a RhodeCode configuration file. Run the following
 command to do this::
  
  paster make-config RhodeCode production.ini
 
 - This will create the file `production.ini` in the current directory. This
   configuration file contains the various settings for RhodeCode, e.g proxy port,
   email settings, usage of static files, cache, celery settings and logging.
 
 
 Next, you need to create the databases used by RhodeCode. I recommend that you
 use sqlite (default) or postgresql. If you choose a database other than the
 default ensure you properly adjust the db url in your production.ini
 configuration file to use this other database. Create the databases by running
 the following command::
 
  paster setup-app production.ini
 
 This will prompt you for a "root" path. This "root" path is the location where
 RhodeCode will store all of its repositories on the current machine. After
 entering this "root" path ``setup-app`` will also prompt you for a username and password
 for the initial admin account which ``setup-app`` sets up for you.
 
 - The ``setup-app`` command will create all of the needed tables and an admin
   account. When choosing a root path you can either use a new empty location, or a
   location which already contains existing repositories. If you choose a location
   which contains existing repositories RhodeCode will simply add all of the
   repositories at the chosen location to it's database. (Note: make sure you
   specify the correct path to the root).
 - Note: the given path for mercurial_ repositories **must** be write accessible
   for the application. It's very important since the RhodeCode web interface will
   work without write access, but when trying to do a push it will eventually fail
   with permission denied errors unless it has write access.
 
 You are now ready to use RhodeCode, to run it simply execute::
  
  paster serve production.ini
  
 - This command runs the RhodeCode server. The web app should be available at the 
   127.0.0.1:5000. This ip and port is configurable via the production.ini 
   file created in previous step
 - Use the admin account you created above when running ``setup-app`` to login to the web app.
 - The default permissions on each repository is read, and the owner is admin. 
   Remember to update these if needed.
 - In the admin panel you can toggle ldap, anonymous, permissions settings. As
   well as edit more advanced options on users and repositories
 
 Try copying your own mercurial repository into the "root" directory you are
 using, then from within the RhodeCode web application choose Admin >
 repositories. Then choose Add New Repository. Add the repository you copied into
 the root. Test that you can browse your repository from within RhodeCode and then
 try cloning your repository from RhodeCode with::
 
   hg clone http://127.0.0.1:5000/<repository name>
 
 where *repository name* is replaced by the name of your repository.
 
 Using RhodeCode with SSH
 ------------------------
 
 RhodeCode currently only hosts repositories using http and https. (The addition of
 ssh hosting is a planned future feature.) However you can easily use ssh in
 parallel with RhodeCode. (Repository access via ssh is a standard "out of
 the box" feature of mercurial_ and you can use this to access any of the
 repositories that RhodeCode is hosting. See PublishingRepositories_)
 
 RhodeCode repository structures are kept in directories with the same name 
 as the project. When using repository groups, each group is a subdirectory.
 This allows you to easily use ssh for accessing repositories.
 
 In order to use ssh you need to make sure that your web-server and the users login
 accounts have the correct permissions set on the appropriate directories. (Note
 that these permissions are independent of any permissions you have set up using
 the RhodeCode web interface.)
 
 If your main directory (the same as set in RhodeCode settings) is for example
 set to **/home/hg** and the repository you are using is named `rhodecode`, then
 to clone via ssh you should run::
 
     hg clone ssh://[e-mail]/home/hg/rhodecode
 
 Using other external tools such as mercurial-server_ or using ssh key based
 authentication is fully supported.
 
 Note: In an advanced setup, in order for your ssh access to use the same
 permissions as set up via the RhodeCode web interface, you can create an
 authentication hook to connect to the rhodecode db and runs check functions for
 permissions against that.
 
 
     
 Setting up Whoosh full text search
 ----------------------------------
 
 Starting from version 1.1 the whoosh index can be build by using the paster
 command ``make-index``. To use ``make-index`` you must specify the configuration
 file that stores the location of the index, and the location of the repositories
 (`--repo-location`).
 
 You may optionally pass the option `-f` to enable a full index rebuild. Without
 the `-f` option, indexing will run always in "incremental" mode.
 
 For an incremental index build use::
 
 	paster make-index production.ini --repo-location=<location for repos> 
 
 For a full index rebuild use::
 
 	paster make-index production.ini -f --repo-location=<location for repos>
 
 - For full text search you can either put crontab entry for
 
 In order to do periodical index builds and keep your index always up to date.
 It's recommended to do a crontab entry for incremental indexing. 
 An example entry might look like this::
  
  /path/to/python/bin/paster /path/to/rhodecode/production.ini --repo-location=<location for repos> 
   
 When using incremental mode (the default) whoosh will check the last
 modification date of each file and add it to be reindexed if a newer file is
 available. The indexing daemon checks for any removed files and removes them
 from index.
 
 If you want to rebuild index from scratch, you can use the `-f` flag as above,
 or in the admin panel you can check `build from scratch` flag.
 
 
 Setting up LDAP support
 -----------------------
 
 RhodeCode starting from version 1.1 supports ldap authentication. In order
 to use LDAP, you have to install the python-ldap_ package. This package is available
 via pypi, so you can install it by running
 
 ::
 
  easy_install python-ldap
  
 ::
 
  pip install python-ldap
 
 .. note::
    python-ldap requires some certain libs on your system, so before installing 
    it check that you have at least `openldap`, and `sasl` libraries.
 
 ldap settings are located in admin->ldap section,
 
 Here's a typical ldap setup::
 
  Enable ldap  = checked                 #controls if ldap access is enabled
  Host         = host.domain.org         #actual ldap server to connect
  Port         = 389 or 689 for ldaps    #ldap server ports
  Enable LDAPS = unchecked               #enable disable ldaps
  Account      = <account>               #access for ldap server(if required)
  Password     = <password>              #password for ldap server(if required)
  Base DN      = uid=%(user)s,CN=users,DC=host,DC=domain,DC=org
  
 
 `Account` and `Password` are optional, and used for two-phase ldap 
 authentication so those are credentials to access your ldap, if it doesn't 
 support anonymous search/user lookups. 
 
 Base DN must have the %(user)s template inside, it's a place holder where your uid
 used to login would go. It allows admins to specify non-standard schema for the
 uid variable.
 
 If all of the data is correctly entered, and `python-ldap` is properly
 installed, then users should be granted access to RhodeCode with ldap accounts.
 When logging in the first time a special ldap account is created inside
 RhodeCode, so you can control the permissions even on ldap users. If such users
 already exist in the RhodeCode database, then the ldap user with the same
 username would be not be able to access RhodeCode.
 
 If you have problems with ldap access and believe you have correctly entered the
 required information then proceed by investigating the RhodeCode logs. Any
 error messages sent from ldap will be saved there.
 
 
 
 Setting Up Celery
 -----------------
 
 Since version 1.1 celery is configured by the rhodecode ini configuration files.
 Simply set use_celery=true in the ini file then add / change the configuration 
 variables inside the ini file.
 
 Remember that the ini files use the format with '.' not with '_' like celery.
 So for example setting `BROKER_HOST` in celery means setting `broker.host` in
 the config file.
 
 In order to start using celery run::
 
  paster celeryd <configfile.ini>
 
 
 .. note::
    Make sure you run this command from the same virtualenv, and with the same user
    that rhodecode runs.
    
 HTTPS support
 -------------
 
 There are two ways to enable https:
 
 - Set HTTP_X_URL_SCHEME in your http server headers, than rhodecode will
   recognize this headers and make proper https redirections
 - Alternatively, set `force_https = true` in the ini configuration to force using
   https, no headers are needed than to enable https
 
 
 Nginx virtual host example
 --------------------------
 
 Sample config for nginx using proxy::
 
     server {
        listen          80;
        server_name     hg.myserver.com;
        access_log      /var/log/nginx/rhodecode.access.log;
        error_log       /var/log/nginx/rhodecode.error.log;
        location / {
                root /var/www/rhodecode/rhodecode/public/;
                if (!-f $request_filename){
                    proxy_pass      http://127.0.0.1:5000;
                }
                #this is important if you want to use https !!!
                proxy_set_header X-Url-Scheme $scheme;
                include         /etc/nginx/proxy.conf;  
        }
     }  
   
 Here's the proxy.conf. It's tuned so it will not timeout on long
 pushes or large pushes::
 
     proxy_redirect              off;
     proxy_set_header            Host $host;
     proxy_set_header            X-Host $http_host;
     proxy_set_header            X-Real-IP $remote_addr;
     proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;
     proxy_set_header            Proxy-host $proxy_host;
     client_max_body_size        400m;
     client_body_buffer_size     128k;
     proxy_buffering             off;
     proxy_connect_timeout       3600;
     proxy_send_timeout          3600;
     proxy_read_timeout          3600;
     proxy_buffer_size           16k;
     proxy_buffers               4 16k;
     proxy_busy_buffers_size     64k;
     proxy_temp_file_write_size  64k;
  
 Also, when using root path with nginx you might set the static files to false
 in the production.ini file::
 
     [app:main]
       use = egg:rhodecode
       full_stack = true
       static_files = false
       lang=en
       cache_dir = %(here)s/data
 
 In order to not have the statics served by the application. This improves speed.
 
 
 Apache virtual host example
 ---------------------------
 
 Here is a sample configuration file for apache using proxy::
 
     <VirtualHost *:80>
             ServerName hg.myserver.com
             ServerAlias hg.myserver.com
     
             <Proxy *>
               Order allow,deny
               Allow from all
             </Proxy>
     
             #important !
             #Directive to properly generate url (clone url) for pylons
             ProxyPreserveHost On
     
             #rhodecode instance
             ProxyPass / http://127.0.0.1:5000/
             ProxyPassReverse / http://127.0.0.1:5000/
             
             #to enable https use line below
             #SetEnvIf X-Url-Scheme https HTTPS=1
             
     </VirtualHost> 
 
 
 Additional tutorial
 http://wiki.pylonshq.com/display/pylonscookbook/Apache+as+a+reverse+proxy+for+Pylons
 
 
 Apache as subdirectory
 ----------------------
 
 
 Apache subdirectory part::
 
     <Location /rhodecode>
       ProxyPass http://127.0.0.1:59542/rhodecode
       ProxyPassReverse http://127.0.0.1:59542/rhodecode
       SetEnvIf X-Url-Scheme https HTTPS=1
     </Location> 
 
 Besides the regular apache setup you will need to add the following to your .ini file::
 
     filter-with = proxy-prefix
 
 Add the following at the end of the .ini file::
 
     [filter:proxy-prefix]
     use = egg:PasteDeploy#prefix
     prefix = /<someprefix> 
 
 
 Apache's example FCGI config
 ----------------------------
 
 TODO !
 
 Other configuration files
 -------------------------
 
 Some example init.d scripts can be found here, for debian and gentoo:
 
 https://rhodeocode.org/rhodecode/files/tip/init.d
 
 
 Troubleshooting
 ---------------
 
 :Q: **Missing static files?**
 :A: Make sure either to set the `static_files = true` in the .ini file or
    double check the root path for your http setup. It should point to 
    for example:
    /home/my-virtual-python/lib/python2.6/site-packages/rhodecode/public
-
-|
+   
+| 
 
 :Q: **Can't install celery/rabbitmq**
 :A: Don't worry RhodeCode works without them too. No extra setup is required.
 
 |
-
+ 
 :Q: **Long lasting push timeouts?**
 :A: Make sure you set a longer timeouts in your proxy/fcgi settings, timeouts
     are caused by https server and not RhodeCode.
-
-|
+    
+| 
 
 :Q: **Large pushes timeouts?**
 :A: Make sure you set a proper max_body_size for the http server.
 
 |
 
 :Q: **Apache doesn't pass basicAuth on pull/push?**
 :A: Make sure you added `WSGIPassAuthorization true`.
 
 For further questions search the `Issues tracker`_, or post a message in the `google group rhodecode`_
 
 .. _virtualenv: http://pypi.python.org/pypi/virtualenv
 .. _python: http://www.python.org/
 .. _mercurial: http://mercurial.selenic.com/
 .. _celery: http://celeryproject.org/
 .. _rabbitmq: http://www.rabbitmq.com/
 .. _python-ldap: http://www.python-ldap.org/
 .. _mercurial-server: http://www.lshift.net/mercurial-server.html
 .. _PublishingRepositories: http://mercurial.selenic.com/wiki/PublishingRepositories
 .. _Issues tracker: https://bitbucket.org/marcinkuzminski/rhodecode/issues
 .. _google group rhodecode: http://groups.google.com/group/rhodecode
  No newline at end of file

docs/upgrade.rst

@@ -1,51 +1,53 b''
 .. _upgrade:
 
 Upgrade
 =======
 
 Upgrading from Cheese Shop
 --------------------------
 
 .. note::
-   Firstly, it is recommended that you **always** perform a database backup before doing an upgrade.
+   Firstly, it is recommended that you **always** perform a database backup 
+   before doing an upgrade.
 
 The easiest way to upgrade ``rhodecode`` is to run::
 
  easy_install -U rhodecode
 
 Or::
 
  pip install --upgrade rhodecode
 
 
 Then make sure you run the following command from the installation directory::
  
  paster make-config RhodeCode production.ini
  
 This will display any changes made by the new version of RhodeCode to your
 current configuration. It will try to perform an automerge. It's always better
-to make a backup of your configuration file before hand and recheck the content after the automerge.
+to make a backup of your configuration file before hand and recheck the 
+content after the automerge.
 
 .. note::
    The next steps only apply to upgrading from non bugfix releases eg. from
    any minor or major releases. Bugfix releases (eg. 1.1.2->1.1.3) will 
    not have any database schema changes or whoosh library updates.
 
-It is also recommended that you rebuild the whoosh index after upgrading since the new whoosh 
-version could introduce some incompatible index changes.
+It is also recommended that you rebuild the whoosh index after upgrading since 
+the new whoosh version could introduce some incompatible index changes.
 
 
 The final step is to upgrade the database. To do this simply run::
 
     paster upgrade-db production.ini
  
 This will upgrade the schema and update some of the defaults in the database,
-and will always recheck the settings of the application, if there are no new options
-that need to be set.
+and will always recheck the settings of the application, if there are no new 
+options that need to be set.
 
 
 .. _virtualenv: http://pypi.python.org/pypi/virtualenv  
 .. _python: http://www.python.org/
 .. _mercurial: http://mercurial.selenic.com/
 .. _celery: http://celeryproject.org/
 .. _rabbitmq: http://www.rabbitmq.com/
  No newline at end of file

rhodecode/__init__.py

@@ -1,52 +1,53 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.__init__
     ~~~~~~~~~~~~~~~~~~
 
     RhodeCode, a web based repository management based on pylons
     versioning implementation: http://semver.org/
 
     :created_on: Apr 9, 2010
     :author: marcink
     :copyright: (C) 2009-2011 Marcin Kuzminski <[e-mail]>    
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
-
+import platform
 
-VERSION = (1, 1, 4)
+VERSION = (1, 1, 5)
 __version__ = '.'.join((str(each) for each in VERSION[:4]))
 __dbversion__ = 2 #defines current db version for migrations
+__platform__ = platform.system()
 
 try:
     from rhodecode.lib.utils import get_current_revision
     _rev = get_current_revision()
 except ImportError:
     #this is needed when doing some setup.py operations
     _rev = False
 
 if len(VERSION) > 3 and _rev:
     __version__ += ' [rev:%s]' % _rev[0]
 
 def get_version():
     """Returns shorter version (digit parts only) as string."""
 
     return '.'.join((str(each) for each in VERSION[:3]))
 
 BACKENDS = {
     'hg': 'Mercurial repository',
    #'git': 'Git repository',
 }

rhodecode/config/deployment.ini_tmpl

@@ -1,218 +1,218 b''
 ################################################################################
 ################################################################################
 # RhodeCode - Pylons environment configuration                                 #
 #                                                                              # 
 # The %(here)s variable will be replaced with the parent directory of this file#
 ################################################################################
 
 [DEFAULT]
 debug = true
 ################################################################################
 ## Uncomment and replace with the address which should receive                ## 
 ## any error reports after application crash                                  ##
 ## Additionally those settings will be used by RhodeCode mailing system       ##
 ################################################################################
 #email_to = admin@localhost
 #error_email_from = paste_error@localhost
 #app_email_from = rhodecode-noreply@localhost
 #error_message =
 
 #smtp_server = mail.server.com
 #smtp_username = 
 #smtp_password = 
 #smtp_port = 
 #smtp_use_tls = false
 #smtp_use_ssl = true
 
 [server:main]
 ##nr of threads to spawn
 threadpool_workers = 5
 
 ##max request before thread respawn
 threadpool_max_requests = 10
 
 ##option to use threads of process
 use_threadpool = true
 
 use = egg:Paste#http
 host = 127.0.0.1
 port = 5000
 
 [app:main]
 use = egg:rhodecode
 full_stack = true
 static_files = true
 lang=en
 cache_dir = %(here)s/data
 index_dir = %(here)s/data/index
 app_instance_uuid = ${app_instance_uuid}
 cut_off_limit = 256000
 force_https = false 
 
 ####################################
 ###        CELERY CONFIG        ####
 ####################################
 use_celery = false
 broker.host = localhost
 broker.vhost = rabbitmqhost
 broker.port = 5672
 broker.user = rabbitmq
 broker.password = qweqwe
 
 celery.imports = rhodecode.lib.celerylib.tasks
 
 celery.result.backend = amqp
 celery.result.dburi = amqp://
 celery.result.serialier = json
 
 #celery.send.task.error.emails = true
 #celery.amqp.task.result.expires = 18000
 
 celeryd.concurrency = 2
 #celeryd.log.file = celeryd.log
 celeryd.log.level = debug
-celeryd.max.tasks.per.child = 3
+celeryd.max.tasks.per.child = 1
 
 #tasks will never be sent to the queue, but executed locally instead.
 celery.always.eager = false
 
 ####################################
 ###         BEAKER CACHE        ####
 ####################################
 beaker.cache.data_dir=%(here)s/data/cache/data
 beaker.cache.lock_dir=%(here)s/data/cache/lock
 
 beaker.cache.regions=super_short_term,short_term,long_term,sql_cache_short,sql_cache_med,sql_cache_long
 
 beaker.cache.super_short_term.type=memory
 beaker.cache.super_short_term.expire=10
 
 beaker.cache.short_term.type=memory
 beaker.cache.short_term.expire=60
 
 beaker.cache.long_term.type=memory
 beaker.cache.long_term.expire=36000
 
 beaker.cache.sql_cache_short.type=memory
 beaker.cache.sql_cache_short.expire=10
 
 beaker.cache.sql_cache_med.type=memory
 beaker.cache.sql_cache_med.expire=360
 
 beaker.cache.sql_cache_long.type=file
 beaker.cache.sql_cache_long.expire=3600
 
 ####################################
 ###       BEAKER SESSION        ####
 ####################################
 ## Type of storage used for the session, current types are 
 ## dbm, file, memcached, database, and memory. 
 ## The storage uses the Container API 
 ##that is also used by the cache system.
 beaker.session.type = file
 
 beaker.session.key = rhodecode
 beaker.session.secret = ${app_instance_secret}
 beaker.session.timeout = 36000
 
 ##auto save the session to not to use .save()
 beaker.session.auto = False
 
 ##true exire at browser close
 #beaker.session.cookie_expires = 3600
 
     
 ################################################################################
 ## WARNING: *THE LINE BELOW MUST BE UNCOMMENTED ON A PRODUCTION ENVIRONMENT*  ##
 ## Debug mode will enable the interactive debugging tool, allowing ANYONE to  ##
 ## execute malicious code after an exception is raised.                       ##
 ################################################################################
 set debug = false
 
 ##################################
 ###       LOGVIEW CONFIG       ###
 ##################################
 logview.sqlalchemy = #faa
 logview.pylons.templating = #bfb
 logview.pylons.util = #eee
 
 #########################################################
 ### DB CONFIGS - EACH DB WILL HAVE IT'S OWN CONFIG    ###
 #########################################################
 sqlalchemy.db1.url = sqlite:///%(here)s/rhodecode.db
 #sqlalchemy.db1.echo = False
 #sqlalchemy.db1.pool_recycle = 3600
 sqlalchemy.convert_unicode = true
 
 ################################
 ### LOGGING CONFIGURATION   ####
 ################################
 [loggers]
 keys = root, routes, rhodecode, sqlalchemy,beaker,templates
 
 [handlers]
 keys = console
 
 [formatters]
 keys = generic,color_formatter
 
 #############
 ## LOGGERS ##
 #############
 [logger_root]
 level = INFO
 handlers = console
 
 [logger_routes]
 level = INFO
 handlers = console
 qualname = routes.middleware
 # "level = DEBUG" logs the route matched and routing variables.
 propagate = 0
 
 [logger_beaker]
 level = ERROR
 handlers = console
 qualname = beaker.container
 propagate = 0
 
 [logger_templates]
 level = INFO
 handlers = console
 qualname = pylons.templating
 propagate = 0
 
 [logger_rhodecode]
 level = DEBUG
 handlers = console
 qualname = rhodecode
 propagate = 0
 
 [logger_sqlalchemy]
 level = ERROR
 handlers = console
 qualname = sqlalchemy.engine
 propagate = 0
 
 ##############
 ## HANDLERS ##
 ##############
 
 [handler_console]
 class = StreamHandler
 args = (sys.stderr,)
 level = NOTSET
 formatter = color_formatter
 
 ################
 ## FORMATTERS ##
 ################
 
 [formatter_generic]
 format = %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
 
 [formatter_color_formatter]
 class=rhodecode.lib.colored_formatter.ColorFormatter
 format= %(asctime)s.%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s
 datefmt = %Y-%m-%d %H:%M:%S
  No newline at end of file

rhodecode/controllers/admin/settings.py

@@ -1,340 +1,340 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.controllers.admin.settings
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     
     settings controller for rhodecode admin
         
     :created_on: Jul 14, 2010
     :author: marcink
     :copyright: (C) 2009-2010 Marcin Kuzminski <[e-mail]>    
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 
 from formencode import htmlfill
 from pylons import request, session, tmpl_context as c, url, app_globals as g, \
     config
 from pylons.controllers.util import abort, redirect
 from pylons.i18n.translation import _
 from rhodecode.lib import helpers as h
 from rhodecode.lib.auth import LoginRequired, HasPermissionAllDecorator, \
     HasPermissionAnyDecorator, NotAnonymous
 from rhodecode.lib.base import BaseController, render
 from rhodecode.lib.celerylib import tasks, run_task
 from rhodecode.lib.utils import repo2db_mapper, invalidate_cache, \
     set_rhodecode_config
 from rhodecode.model.db import RhodeCodeUi, Repository
 from rhodecode.model.forms import UserForm, ApplicationSettingsForm, \
     ApplicationUiSettingsForm
 from rhodecode.model.scm import ScmModel
 from rhodecode.model.settings import SettingsModel
 from rhodecode.model.user import UserModel
 from sqlalchemy import func
 import formencode
 import logging
 import traceback
 
 log = logging.getLogger(__name__)
 
 
 class SettingsController(BaseController):
     """REST Controller styled on the Atom Publishing Protocol"""
     # To properly map this controller, ensure your config/routing.py
     # file has a resource setup:
     #     map.resource('setting', 'settings', controller='admin/settings', 
     #         path_prefix='/admin', name_prefix='admin_')
 
 
     @LoginRequired()
     def __before__(self):
         c.admin_user = session.get('admin_user')
         c.admin_username = session.get('admin_username')
         super(SettingsController, self).__before__()
 
 
     @HasPermissionAllDecorator('hg.admin')
     def index(self, format='html'):
         """GET /admin/settings: All items in the collection"""
         # url('admin_settings')
 
         defaults = SettingsModel().get_app_settings()
         defaults.update(self.get_hg_ui_settings())
         return htmlfill.render(
             render('admin/settings/settings.html'),
             defaults=defaults,
             encoding="UTF-8",
             force_defaults=False
         )
 
     @HasPermissionAllDecorator('hg.admin')
     def create(self):
         """POST /admin/settings: Create a new item"""
         # url('admin_settings')
 
     @HasPermissionAllDecorator('hg.admin')
     def new(self, format='html'):
         """GET /admin/settings/new: Form to create a new item"""
         # url('admin_new_setting')
 
     @HasPermissionAllDecorator('hg.admin')
     def update(self, setting_id):
         """PUT /admin/settings/setting_id: Update an existing item"""
         # Forms posted to this method should contain a hidden field:
         #    <input type="hidden" name="_method" value="PUT" />
         # Or using helpers:
         #    h.form(url('admin_setting', setting_id=ID),
         #           method='put')
         # url('admin_setting', setting_id=ID)
         if setting_id == 'mapping':
             rm_obsolete = request.POST.get('destroy', False)
             log.debug('Rescanning directories with destroy=%s', rm_obsolete)
 
             initial = ScmModel().repo_scan(g.paths[0][1], g.baseui)
             for repo_name in initial.keys():
                 invalidate_cache('get_repo_cached_%s' % repo_name)
 
             repo2db_mapper(initial, rm_obsolete)
 
             h.flash(_('Repositories successfully rescanned'), category='success')
 
         if setting_id == 'whoosh':
             repo_location = self.get_hg_ui_settings()['paths_root_path']
             full_index = request.POST.get('full_index', False)
             task = run_task(tasks.whoosh_index, repo_location, full_index)
 
             h.flash(_('Whoosh reindex task scheduled'), category='success')
         if setting_id == 'global':
 
             application_form = ApplicationSettingsForm()()
             try:
                 form_result = application_form.to_python(dict(request.POST))
                 settings_model = SettingsModel()
                 try:
                     hgsettings1 = settings_model.get('title')
                     hgsettings1.app_settings_value = form_result['rhodecode_title']
 
                     hgsettings2 = settings_model.get('realm')
                     hgsettings2.app_settings_value = form_result['rhodecode_realm']
 
 
                     self.sa.add(hgsettings1)
                     self.sa.add(hgsettings2)
                     self.sa.commit()
                     set_rhodecode_config(config)
                     h.flash(_('Updated application settings'),
                             category='success')
 
                 except:
                     log.error(traceback.format_exc())
-                    h.flash(_('error occurred during updating application settings'),
-                            category='error')
+                    h.flash(_('error occurred during updating'
+                              ' application settings'), category='error')
 
                     self.sa.rollback()
 
 
             except formencode.Invalid, errors:
                 return htmlfill.render(
                      render('admin/settings/settings.html'),
                      defaults=errors.value,
                      errors=errors.error_dict or {},
                      prefix_error=False,
                      encoding="UTF-8")
 
         if setting_id == 'mercurial':
             application_form = ApplicationUiSettingsForm()()
             try:
                 form_result = application_form.to_python(dict(request.POST))
 
                 try:
 
                     hgsettings1 = self.sa.query(RhodeCodeUi)\
                     .filter(RhodeCodeUi.ui_key == 'push_ssl').one()
                     hgsettings1.ui_value = form_result['web_push_ssl']
 
                     hgsettings2 = self.sa.query(RhodeCodeUi)\
                     .filter(RhodeCodeUi.ui_key == '/').one()
                     hgsettings2.ui_value = form_result['paths_root_path']
 
 
                     #HOOKS
                     hgsettings3 = self.sa.query(RhodeCodeUi)\
                     .filter(RhodeCodeUi.ui_key == 'changegroup.update').one()
                     hgsettings3.ui_active = bool(form_result['hooks_changegroup_update'])
 
                     hgsettings4 = self.sa.query(RhodeCodeUi)\
                     .filter(RhodeCodeUi.ui_key == 'changegroup.repo_size').one()
                     hgsettings4.ui_active = bool(form_result['hooks_changegroup_repo_size'])
 
                     hgsettings5 = self.sa.query(RhodeCodeUi)\
                     .filter(RhodeCodeUi.ui_key == 'pretxnchangegroup.push_logger').one()
                     hgsettings5.ui_active = bool(form_result['hooks_pretxnchangegroup_push_logger'])
 
                     hgsettings6 = self.sa.query(RhodeCodeUi)\
                     .filter(RhodeCodeUi.ui_key == 'preoutgoing.pull_logger').one()
                     hgsettings6.ui_active = bool(form_result['hooks_preoutgoing_pull_logger'])
 
 
                     self.sa.add(hgsettings1)
                     self.sa.add(hgsettings2)
                     self.sa.add(hgsettings3)
                     self.sa.add(hgsettings4)
                     self.sa.add(hgsettings5)
                     self.sa.add(hgsettings6)
                     self.sa.commit()
 
                     h.flash(_('Updated mercurial settings'),
                             category='success')
 
                 except:
                     log.error(traceback.format_exc())
                     h.flash(_('error occurred during updating application settings'),
                             category='error')
 
                     self.sa.rollback()
 
 
             except formencode.Invalid, errors:
                 return htmlfill.render(
                      render('admin/settings/settings.html'),
                      defaults=errors.value,
                      errors=errors.error_dict or {},
                      prefix_error=False,
                      encoding="UTF-8")
 
 
 
         return redirect(url('admin_settings'))
 
     @HasPermissionAllDecorator('hg.admin')
     def delete(self, setting_id):
         """DELETE /admin/settings/setting_id: Delete an existing item"""
         # Forms posted to this method should contain a hidden field:
         #    <input type="hidden" name="_method" value="DELETE" />
         # Or using helpers:
         #    h.form(url('admin_setting', setting_id=ID),
         #           method='delete')
         # url('admin_setting', setting_id=ID)
 
     @HasPermissionAllDecorator('hg.admin')
     def show(self, setting_id, format='html'):
         """GET /admin/settings/setting_id: Show a specific item"""
         # url('admin_setting', setting_id=ID)
 
     @HasPermissionAllDecorator('hg.admin')
     def edit(self, setting_id, format='html'):
         """GET /admin/settings/setting_id/edit: Form to edit an existing item"""
         # url('admin_edit_setting', setting_id=ID)
 
     @NotAnonymous()
     def my_account(self):
         """
         GET /_admin/my_account Displays info about my account 
         """
         # url('admin_settings_my_account')
 
         c.user = UserModel().get(c.rhodecode_user.user_id, cache=False)
         all_repos = self.sa.query(Repository)\
             .filter(Repository.user_id == c.user.user_id)\
             .order_by(func.lower(Repository.repo_name))\
             .all()
 
         c.user_repos = ScmModel().get_repos(all_repos)
 
         if c.user.username == 'default':
             h.flash(_("You can't edit this user since it's"
               " crucial for entire application"), category='warning')
             return redirect(url('users'))
 
         defaults = c.user.get_dict()
         return htmlfill.render(
             render('admin/users/user_edit_my_account.html'),
             defaults=defaults,
             encoding="UTF-8",
             force_defaults=False
         )
 
     def my_account_update(self):
         """PUT /_admin/my_account_update: Update an existing item"""
         # Forms posted to this method should contain a hidden field:
         #    <input type="hidden" name="_method" value="PUT" />
         # Or using helpers:
         #    h.form(url('admin_settings_my_account_update'),
         #           method='put')
         # url('admin_settings_my_account_update', id=ID)
         user_model = UserModel()
         uid = c.rhodecode_user.user_id
         _form = UserForm(edit=True, old_data={'user_id':uid,
                                               'email':c.rhodecode_user.email})()
         form_result = {}
         try:
             form_result = _form.to_python(dict(request.POST))
             user_model.update_my_account(uid, form_result)
             h.flash(_('Your account was updated successfully'),
                     category='success')
 
         except formencode.Invalid, errors:
             c.user = user_model.get(c.rhodecode_user.user_id, cache=False)
             c.user = UserModel().get(c.rhodecode_user.user_id, cache=False)
             all_repos = self.sa.query(Repository)\
                 .filter(Repository.user_id == c.user.user_id)\
                 .order_by(func.lower(Repository.repo_name))\
                 .all()
             c.user_repos = ScmModel().get_repos(all_repos)
 
             return htmlfill.render(
                 render('admin/users/user_edit_my_account.html'),
                 defaults=errors.value,
                 errors=errors.error_dict or {},
                 prefix_error=False,
                 encoding="UTF-8")
         except Exception:
             log.error(traceback.format_exc())
             h.flash(_('error occurred during update of user %s') \
                     % form_result.get('username'), category='error')
 
         return redirect(url('my_account'))
 
     @NotAnonymous()
     @HasPermissionAnyDecorator('hg.admin', 'hg.create.repository')
     def create_repository(self):
         """GET /_admin/create_repository: Form to create a new item"""
         new_repo = request.GET.get('repo', '')
         c.new_repo = h.repo_name_slug(new_repo)
 
         return render('admin/repos/repo_add_create_repository.html')
 
     def get_hg_ui_settings(self):
         ret = self.sa.query(RhodeCodeUi).all()
 
         if not ret:
             raise Exception('Could not get application ui settings !')
         settings = {}
         for each in ret:
             k = each.ui_key
             v = each.ui_value
             if k == '/':
                 k = 'root_path'
 
             if k.find('.') != -1:
                 k = k.replace('.', '_')
 
             if each.ui_section == 'hooks':
                 v = each.ui_active
 
             settings[each.ui_section + '_' + k] = v
 
         return settings

rhodecode/controllers/files.py

@@ -1,258 +1,278 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.controllers.files
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
     Files controller for RhodeCode
     
     :created_on: Apr 21, 2010
     :author: marcink
-    :copyright: (C) 2009-2010 Marcin Kuzminski <[e-mail]>    
+    :copyright: (C) 2009-2011 Marcin Kuzminski <[e-mail]>    
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 import tempfile
 import logging
 import rhodecode.lib.helpers as h
 
 from mercurial import archival
 
 from pylons import request, response, session, tmpl_context as c, url
 from pylons.i18n.translation import _
 from pylons.controllers.util import redirect
 
 from rhodecode.lib.auth import LoginRequired, HasRepoPermissionAnyDecorator
 from rhodecode.lib.base import BaseController, render
 from rhodecode.lib.utils import EmptyChangeset
 from rhodecode.model.scm import ScmModel
 
 from vcs.exceptions import RepositoryError, ChangesetError, \
     ChangesetDoesNotExistError, EmptyRepositoryError
 from vcs.nodes import FileNode
 from vcs.utils import diffs as differ
 
 log = logging.getLogger(__name__)
 
 class FilesController(BaseController):
 
     @LoginRequired()
     @HasRepoPermissionAnyDecorator('repository.read', 'repository.write',
                                    'repository.admin')
     def __before__(self):
         super(FilesController, self).__before__()
         c.cut_off_limit = self.cut_off_limit
 
+    def __get_cs_or_redirect(self, rev, repo_name):
+        """
+        Safe way to get changeset if error occur it redirects to tip with
+        proper message
+        
+        :param rev: revision to fetch
+        :param repo_name: repo name to redirect after
+        """
+
+        _repo = ScmModel().get_repo(c.repo_name)
+        try:
+            return _repo.get_changeset(rev)
+        except EmptyRepositoryError, e:
+            h.flash(_('There are no files yet'), category='warning')
+            redirect(h.url('summary_home', repo_name=repo_name))
+
+        except RepositoryError, e:
+            h.flash(str(e), category='warning')
+            redirect(h.url('files_home', repo_name=repo_name, revision='tip'))
+
     def index(self, repo_name, revision, f_path):
-        hg_model = ScmModel()
-        c.repo = hg_model.get_repo(c.repo_name)
+        cs = self.__get_cs_or_redirect(revision, repo_name)
+        c.repo = ScmModel().get_repo(c.repo_name)
+
         revision = request.POST.get('at_rev', None) or revision
 
         def get_next_rev(cur):
             max_rev = len(c.repo.revisions) - 1
             r = cur + 1
             if r > max_rev:
                 r = max_rev
             return r
 
         def get_prev_rev(cur):
             r = cur - 1
             return r
 
         c.f_path = f_path
+        c.changeset = cs
+        cur_rev = c.changeset.revision
+        prev_rev = c.repo.get_changeset(get_prev_rev(cur_rev)).raw_id
+        next_rev = c.repo.get_changeset(get_next_rev(cur_rev)).raw_id
 
+        c.url_prev = url('files_home', repo_name=c.repo_name,
+                         revision=prev_rev, f_path=f_path)
+        c.url_next = url('files_home', repo_name=c.repo_name,
+                     revision=next_rev, f_path=f_path)
 
         try:
-            c.changeset = c.repo.get_changeset(revision)
-            cur_rev = c.changeset.revision
-            prev_rev = c.repo.get_changeset(get_prev_rev(cur_rev)).raw_id
-            next_rev = c.repo.get_changeset(get_next_rev(cur_rev)).raw_id
-
-            c.url_prev = url('files_home', repo_name=c.repo_name,
-                             revision=prev_rev, f_path=f_path)
-            c.url_next = url('files_home', repo_name=c.repo_name,
-                         revision=next_rev, f_path=f_path)
-
-            try:
-                c.files_list = c.changeset.get_node(f_path)
-                c.file_history = self._get_history(c.repo, c.files_list, f_path)
-            except RepositoryError, e:
-                h.flash(str(e), category='warning')
-                redirect(h.url('files_home', repo_name=repo_name, revision=revision))
-
-        except EmptyRepositoryError, e:
-            h.flash(_('There are no files yet'), category='warning')
-            redirect(h.url('summary_home', repo_name=repo_name))
-
+            c.files_list = c.changeset.get_node(f_path)
+            c.file_history = self._get_history(c.repo, c.files_list, f_path)
         except RepositoryError, e:
             h.flash(str(e), category='warning')
-            redirect(h.url('files_home', repo_name=repo_name, revision='tip'))
-
+            redirect(h.url('files_home', repo_name=repo_name,
+                           revision=revision))
 
 
         return render('files/files.html')
 
     def rawfile(self, repo_name, revision, f_path):
-        hg_model = ScmModel()
-        c.repo = hg_model.get_repo(c.repo_name)
-        file_node = c.repo.get_changeset(revision).get_node(f_path)
+        cs = self.__get_cs_or_redirect(revision, repo_name)
+        try:
+            file_node = cs.get_node(f_path)
+        except RepositoryError, e:
+            h.flash(str(e), category='warning')
+            redirect(h.url('files_home', repo_name=repo_name,
+                           revision=cs.raw_id))
+
+        fname = f_path.split('/')[-1].encode('utf8', 'replace')
+
+        response.content_disposition = 'attachment; filename=%s' % fname
         response.content_type = file_node.mimetype
-        response.content_disposition = 'attachment; filename=%s' \
-                                                    % f_path.split('/')[-1]
         return file_node.content
 
     def raw(self, repo_name, revision, f_path):
-        hg_model = ScmModel()
-        c.repo = hg_model.get_repo(c.repo_name)
-        file_node = c.repo.get_changeset(revision).get_node(f_path)
+        cs = self.__get_cs_or_redirect(revision, repo_name)
+        try:
+            file_node = cs.get_node(f_path)
+        except RepositoryError, e:
+            h.flash(str(e), category='warning')
+            redirect(h.url('files_home', repo_name=repo_name,
+                           revision=cs.raw_id))
+
         response.content_type = 'text/plain'
-
         return file_node.content
 
     def annotate(self, repo_name, revision, f_path):
-        hg_model = ScmModel()
-        c.repo = hg_model.get_repo(c.repo_name)
-
+        cs = self.__get_cs_or_redirect(revision, repo_name)
         try:
-            c.cs = c.repo.get_changeset(revision)
-            c.file = c.cs.get_node(f_path)
+            c.file = cs.get_node(f_path)
         except RepositoryError, e:
             h.flash(str(e), category='warning')
-            redirect(h.url('files_home', repo_name=repo_name, revision=revision))
+            redirect(h.url('files_home', repo_name=repo_name, revision=cs.raw_id))
 
-        c.file_history = self._get_history(c.repo, c.file, f_path)
-
+        c.file_history = self._get_history(ScmModel().get_repo(c.repo_name), c.file, f_path)
+        c.cs = cs
         c.f_path = f_path
 
         return render('files/files_annotate.html')
 
     def archivefile(self, repo_name, revision, fileformat):
         archive_specs = {
           '.tar.bz2': ('application/x-tar', 'tbz2'),
           '.tar.gz': ('application/x-tar', 'tgz'),
           '.zip': ('application/zip', 'zip'),
         }
         if not archive_specs.has_key(fileformat):
             return 'Unknown archive type %s' % fileformat
 
         def read_in_chunks(file_object, chunk_size=1024 * 40):
             """Lazy function (generator) to read a file piece by piece.
             Default chunk size: 40k."""
             while True:
                 data = file_object.read(chunk_size)
                 if not data:
                     break
                 yield data
 
         archive = tempfile.TemporaryFile()
         repo = ScmModel().get_repo(repo_name).repo
         fname = '%s-%s%s' % (repo_name, revision, fileformat)
         archival.archive(repo, archive, revision, archive_specs[fileformat][1],
                          prefix='%s-%s' % (repo_name, revision))
         response.content_type = archive_specs[fileformat][0]
         response.content_disposition = 'attachment; filename=%s' % fname
         archive.seek(0)
         return read_in_chunks(archive)
 
     def diff(self, repo_name, f_path):
         hg_model = ScmModel()
         diff1 = request.GET.get('diff1')
         diff2 = request.GET.get('diff2')
         c.action = request.GET.get('diff')
         c.no_changes = diff1 == diff2
         c.f_path = f_path
         c.repo = hg_model.get_repo(c.repo_name)
 
         try:
             if diff1 not in ['', None, 'None', '0' * 12, '0' * 40]:
                 c.changeset_1 = c.repo.get_changeset(diff1)
                 node1 = c.changeset_1.get_node(f_path)
             else:
                 c.changeset_1 = EmptyChangeset()
                 node1 = FileNode('.', '', changeset=c.changeset_1)
 
             if diff2 not in ['', None, 'None', '0' * 12, '0' * 40]:
                 c.changeset_2 = c.repo.get_changeset(diff2)
                 node2 = c.changeset_2.get_node(f_path)
             else:
                 c.changeset_2 = EmptyChangeset()
                 node2 = FileNode('.', '', changeset=c.changeset_2)
         except RepositoryError:
             return redirect(url('files_home',
                                 repo_name=c.repo_name, f_path=f_path))
 
         f_udiff = differ.get_udiff(node1, node2)
         diff = differ.DiffProcessor(f_udiff)
 
         if c.action == 'download':
             diff_name = '%s_vs_%s.diff' % (diff1, diff2)
             response.content_type = 'text/plain'
             response.content_disposition = 'attachment; filename=%s' \
                                                     % diff_name
+            if node1.is_binary or node2.is_binary:
+                return _('binary file changed')
             return diff.raw_diff()
 
         elif c.action == 'raw':
             response.content_type = 'text/plain'
+            if node1.is_binary or node2.is_binary:
+                return _('binary file changed')
             return diff.raw_diff()
 
         elif c.action == 'diff':
             if node1.size > self.cut_off_limit or node2.size > self.cut_off_limit:
                 c.cur_diff = _('Diff is to big to display')
+            elif node1.is_binary or node2.is_binary:
+                c.cur_diff = _('Binary file')
             else:
                 c.cur_diff = diff.as_html()
         else:
             #default option
             if node1.size > self.cut_off_limit or node2.size > self.cut_off_limit:
                 c.cur_diff = _('Diff is to big to display')
+            elif node1.is_binary or node2.is_binary:
+                c.cur_diff = _('Binary file')
             else:
                 c.cur_diff = diff.as_html()
 
-        if not c.cur_diff: c.no_changes = True
+        if not c.cur_diff:
+            c.no_changes = True
         return render('files/file_diff.html')
 
     def _get_history(self, repo, node, f_path):
         from vcs.nodes import NodeKind
         if not node.kind is NodeKind.FILE:
             return []
         changesets = node.history
         hist_l = []
 
         changesets_group = ([], _("Changesets"))
         branches_group = ([], _("Branches"))
         tags_group = ([], _("Tags"))
 
         for chs in changesets:
             n_desc = 'r%s:%s' % (chs.revision, chs.short_id)
             changesets_group[0].append((chs.raw_id, n_desc,))
 
         hist_l.append(changesets_group)
 
         for name, chs in c.repository_branches.items():
             #chs = chs.split(':')[-1]
             branches_group[0].append((chs, name),)
         hist_l.append(branches_group)
 
         for name, chs in c.repository_tags.items():
             #chs = chs.split(':')[-1]
             tags_group[0].append((chs, name),)
         hist_l.append(tags_group)
 
         return hist_l
-
-#                [
-#                 ([("u1", "User1"), ("u2", "User2")], "Users"),
-#                 ([("g1", "Group1"), ("g2", "Group2")], "Groups")
-#                 ]
-

rhodecode/controllers/home.py

@@ -1,66 +1,66 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.controllers.home
     ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
     Home controller for Rhodecode
     
     :created_on: Feb 18, 2010
     :author: marcink
     :copyright: (C) 2009-2010 Marcin Kuzminski <[e-mail]>    
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 
 import logging
 from operator import itemgetter
 
 from pylons import tmpl_context as c, request
 
 from rhodecode.lib.auth import LoginRequired
 from rhodecode.lib.base import BaseController, render
 from rhodecode.model.scm import ScmModel
 
 log = logging.getLogger(__name__)
 
 class HomeController(BaseController):
 
     @LoginRequired()
     def __before__(self):
         super(HomeController, self).__before__()
 
     def index(self):
-        sortables = ['name', 'description', 'last_change', 'tip', 'contact']
+        sortables = ['name', 'description', 'last_change', 'tip', 'owner']
         current_sort = request.GET.get('sort', 'name')
         current_sort_slug = current_sort.replace('-', '')
 
         if current_sort_slug not in sortables:
             c.sort_by = 'name'
             current_sort_slug = c.sort_by
         else:
             c.sort_by = current_sort
         c.sort_slug = current_sort_slug
         cached_repo_list = ScmModel().get_repos()
 
         sort_key = current_sort_slug + '_sort'
         if c.sort_by.startswith('-'):
             c.repos_list = sorted(cached_repo_list, key=itemgetter(sort_key),
                                   reverse=True)
         else:
             c.repos_list = sorted(cached_repo_list, key=itemgetter(sort_key),
                                   reverse=False)
 
         return render('/index.html')

rhodecode/controllers/journal.py

@@ -1,97 +1,100 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.controllers.journal
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
     Journal controller for pylons
     
     :created_on: Nov 21, 2010
     :author: marcink
     :copyright: (C) 2009-2010 Marcin Kuzminski <[e-mail]>    
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 
 import logging
-from sqlalchemy import or_
+import traceback
 
 from pylons import request, response, session, tmpl_context as c, url
+from paste.httpexceptions import HTTPInternalServerError, HTTPBadRequest
+
+from sqlalchemy import or_
 
 from rhodecode.lib.auth import LoginRequired, NotAnonymous
 from rhodecode.lib.base import BaseController, render
 from rhodecode.lib.helpers import get_token
 from rhodecode.model.db import UserLog, UserFollowing
 from rhodecode.model.scm import ScmModel
 
-from paste.httpexceptions import HTTPInternalServerError
-
 log = logging.getLogger(__name__)
 
 class JournalController(BaseController):
 
 
     @LoginRequired()
     @NotAnonymous()
     def __before__(self):
         super(JournalController, self).__before__()
 
     def index(self):
         # Return a rendered template
 
         c.following = self.sa.query(UserFollowing)\
             .filter(UserFollowing.user_id == c.rhodecode_user.user_id).all()
 
         repo_ids = [x.follows_repository.repo_id for x in c.following
                     if x.follows_repository is not None]
         user_ids = [x.follows_user.user_id for x in c.following
                     if x.follows_user is not None]
 
         c.journal = self.sa.query(UserLog)\
             .filter(or_(
                         UserLog.repository_id.in_(repo_ids),
                         UserLog.user_id.in_(user_ids),
                         ))\
             .order_by(UserLog.action_date.desc())\
             .limit(20)\
             .all()
         return render('/journal.html')
 
     def toggle_following(self):
 
         if request.POST.get('auth_token') == get_token():
             scm_model = ScmModel()
 
             user_id = request.POST.get('follows_user_id')
             if user_id:
                 try:
                     scm_model.toggle_following_user(user_id,
                                                     c.rhodecode_user.user_id)
                     return 'ok'
                 except:
+                    log.error(traceback.format_exc())
                     raise HTTPInternalServerError()
 
             repo_id = request.POST.get('follows_repo_id')
             if repo_id:
                 try:
                     scm_model.toggle_following_repo(repo_id,
                                                     c.rhodecode_user.user_id)
                     return 'ok'
                 except:
+                    log.error(traceback.format_exc())
                     raise HTTPInternalServerError()
 
 
 
-        raise HTTPInternalServerError()
+        raise HTTPBadRequest()

rhodecode/lib/__init__.py

@@ -1,29 +1,46 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.lib.__init__
     ~~~~~~~~~~~~~~~~~~~~~~~
 
     Some simple helper functions
     
     :created_on: Jan 5, 2011
     :author: marcink
     :copyright: (C) 2009-2010 Marcin Kuzminski <[e-mail]>    
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 
 def str2bool(v):
-    return v.lower() in ["yes", "true", "t", "1"] if v else None
+    if isinstance(v, (str, unicode)):
+        obj = v.strip().lower()
+        if obj in ['true', 'yes', 'on', 'y', 't', '1']:
+            return True
+        elif obj in ['false', 'no', 'off', 'n', 'f', '0']:
+            return False
+        else:
+            raise ValueError("String is not true/false: %r" % obj)
+    return bool(obj)
+
+def generate_api_key(username, salt=None):
+    from tempfile import _RandomNameSequence
+    import hashlib
+
+    if salt is None:
+        salt = _RandomNameSequence().next()
+
+    return hashlib.sha1(username + salt).hexdigest()

rhodecode/lib/auth.py

@@ -1,569 +1,614 b''
-#!/usr/bin/env python
-# encoding: utf-8
-# authentication and permission libraries
-# Copyright (C) 2009-2010 Marcin Kuzminski <[e-mail]>
-#
+# -*- coding: utf-8 -*-
+"""
+    rhodecode.lib.auth
+    ~~~~~~~~~~~~~~~~~~
+    
+    authentication and permission libraries
+    
+    :created_on: Apr 4, 2010
+    :copyright: (c) 2010 by marcink.
+    :license: LICENSE_NAME, see LICENSE_FILE for more details.
+"""
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
-"""
-Created on April 4, 2010
 
-@author: marcink
-"""
+import random
+import logging
+import traceback
+
+from decorator import decorator
+
 from pylons import config, session, url, request
 from pylons.controllers.util import abort, redirect
-from rhodecode.lib.exceptions import *
+from pylons.i18n.translation import _
+
+from rhodecode import __platform__
+
+if __platform__ == 'Windows':
+    from hashlib import sha256
+if __platform__ in ('Linux', 'Darwin'):
+    import bcrypt
+
+from rhodecode.lib import str2bool
+from rhodecode.lib.exceptions import LdapPasswordError, LdapUsernameError
 from rhodecode.lib.utils import get_repo_slug
 from rhodecode.lib.auth_ldap import AuthLdap
+
 from rhodecode.model import meta
 from rhodecode.model.user import UserModel
-from rhodecode.model.caching_query import FromCache
-from rhodecode.model.db import User, RepoToPerm, Repository, Permission, \
-    UserToPerm
-import bcrypt
-from decorator import decorator
-import logging
-import random
-import traceback
+from rhodecode.model.db import Permission, RepoToPerm, Repository, \
+    User, UserToPerm
+
 
 log = logging.getLogger(__name__)
 
 class PasswordGenerator(object):
     """This is a simple class for generating password from
         different sets of characters
         usage:
         passwd_gen = PasswordGenerator()
         #print 8-letter password containing only big and small letters of alphabet
         print passwd_gen.gen_password(8, passwd_gen.ALPHABETS_BIG_SMALL)        
     """
     ALPHABETS_NUM = r'''1234567890'''#[0]
     ALPHABETS_SMALL = r'''qwertyuiopasdfghjklzxcvbnm'''#[1]
     ALPHABETS_BIG = r'''QWERTYUIOPASDFGHJKLZXCVBNM'''#[2]
     ALPHABETS_SPECIAL = r'''`-=[]\;',./~!@#$%^&*()_+{}|:"<>?'''    #[3]
     ALPHABETS_FULL = ALPHABETS_BIG + ALPHABETS_SMALL + ALPHABETS_NUM + ALPHABETS_SPECIAL#[4]
     ALPHABETS_ALPHANUM = ALPHABETS_BIG + ALPHABETS_SMALL + ALPHABETS_NUM#[5]
     ALPHABETS_BIG_SMALL = ALPHABETS_BIG + ALPHABETS_SMALL
     ALPHABETS_ALPHANUM_BIG = ALPHABETS_BIG + ALPHABETS_NUM#[6]
     ALPHABETS_ALPHANUM_SMALL = ALPHABETS_SMALL + ALPHABETS_NUM#[7]
 
     def __init__(self, passwd=''):
         self.passwd = passwd
 
     def gen_password(self, len, type):
         self.passwd = ''.join([random.choice(type) for _ in xrange(len)])
         return self.passwd
 
+class RhodeCodeCrypto(object):
+
+    @classmethod
+    def hash_string(cls, str_):
+        """
+        Cryptographic function used for password hashing based on pybcrypt
+        or pycrypto in windows
+        
+        :param password: password to hash
+        """
+        if __platform__ == 'Windows':
+            return sha256(str_).hexdigest()
+        elif __platform__ in ('Linux', 'Darwin'):
+            return bcrypt.hashpw(str_, bcrypt.gensalt(10))
+        else:
+            raise Exception('Unknown or unsupported platform %s' % __platform__)
+
+    @classmethod
+    def hash_check(cls, password, hashed):
+        """
+        Checks matching password with it's hashed value, runs different
+        implementation based on platform it runs on
+        
+        :param password: password
+        :param hashed: password in hashed form
+        """
+
+        if __platform__ == 'Windows':
+            return sha256(password).hexdigest() == hashed
+        elif __platform__ in ('Linux', 'Darwin'):
+            return bcrypt.hashpw(password, hashed) == hashed
+        else:
+            raise Exception('Unknown or unsupported platform %s' % __platform__)
+
 
 def get_crypt_password(password):
-    """Cryptographic function used for password hashing based on sha1
-    :param password: password to hash
-    """
-    return bcrypt.hashpw(password, bcrypt.gensalt(10))
+    return RhodeCodeCrypto.hash_string(password)
 
 def check_password(password, hashed):
-    return bcrypt.hashpw(password, hashed) == hashed
+    return RhodeCodeCrypto.hash_check(password, hashed)
 
 def authfunc(environ, username, password):
     """
     Dummy authentication function used in Mercurial/Git/ and access control,
     
     :param environ: needed only for using in Basic auth
     """
     return authenticate(username, password)
 
 
 def authenticate(username, password):
     """
     Authentication function used for access control,
     firstly checks for db authentication then if ldap is enabled for ldap
     authentication, also creates ldap user if not in database
     
     :param username: username
     :param password: password
     """
     user_model = UserModel()
     user = user_model.get_by_username(username, cache=False)
 
     log.debug('Authenticating user using RhodeCode account')
     if user is not None and user.is_ldap is False:
         if user.active:
 
             if user.username == 'default' and user.active:
                 log.info('user %s authenticated correctly as anonymous user',
                          username)
                 return True
 
             elif user.username == username and check_password(password, user.password):
                 log.info('user %s authenticated correctly', username)
                 return True
         else:
             log.warning('user %s is disabled', username)
 
     else:
         log.debug('Regular authentication failed')
         user_obj = user_model.get_by_username(username, cache=False,
                                             case_insensitive=True)
 
         if user_obj is not None and user_obj.is_ldap is False:
             log.debug('this user already exists as non ldap')
             return False
 
         from rhodecode.model.settings import SettingsModel
         ldap_settings = SettingsModel().get_ldap_settings()
 
         #======================================================================
         # FALLBACK TO LDAP AUTH IN ENABLE                
         #======================================================================
-        if ldap_settings.get('ldap_active', False):
+        if str2bool(ldap_settings.get('ldap_active')):
             log.debug("Authenticating user using ldap")
             kwargs = {
                   'server':ldap_settings.get('ldap_host', ''),
                   'base_dn':ldap_settings.get('ldap_base_dn', ''),
                   'port':ldap_settings.get('ldap_port'),
                   'bind_dn':ldap_settings.get('ldap_dn_user'),
                   'bind_pass':ldap_settings.get('ldap_dn_pass'),
-                  'use_ldaps':ldap_settings.get('ldap_ldaps'),
+                  'use_ldaps':str2bool(ldap_settings.get('ldap_ldaps')),
                   'ldap_version':3,
                   }
             log.debug('Checking for ldap authentication')
             try:
                 aldap = AuthLdap(**kwargs)
                 res = aldap.authenticate_ldap(username, password)
                 log.debug('Got ldap response %s', res)
 
                 if user_model.create_ldap(username, password):
                     log.info('created new ldap user')
 
                 return True
             except (LdapUsernameError, LdapPasswordError,):
                 pass
             except (Exception,):
                 log.error(traceback.format_exc())
                 pass
     return False
 
 class  AuthUser(object):
     """
     A simple object that handles a mercurial username for authentication
     """
     def __init__(self):
         self.username = 'None'
         self.name = ''
         self.lastname = ''
         self.email = ''
         self.user_id = None
         self.is_authenticated = False
         self.is_admin = False
         self.permissions = {}
 
     def __repr__(self):
         return "<AuthUser('id:%s:%s')>" % (self.user_id, self.username)
 
 def set_available_permissions(config):
     """
     This function will propagate pylons globals with all available defined
     permission given in db. We don't wannt to check each time from db for new 
     permissions since adding a new permission also requires application restart
     ie. to decorate new views with the newly created permission
     :param config:
     """
     log.info('getting information about all available permissions')
     try:
         sa = meta.Session()
         all_perms = sa.query(Permission).all()
     except:
         pass
     finally:
         meta.Session.remove()
 
     config['available_permissions'] = [x.permission_name for x in all_perms]
 
 def set_base_path(config):
     config['base_path'] = config['pylons.app_globals'].base_path
 
 
 def fill_perms(user):
     """
     Fills user permission attribute with permissions taken from database
     :param user:
     """
 
     sa = meta.Session()
     user.permissions['repositories'] = {}
     user.permissions['global'] = set()
 
     #===========================================================================
     # fetch default permissions
     #===========================================================================
     default_user = UserModel().get_by_username('default', cache=True)
 
     default_perms = sa.query(RepoToPerm, Repository, Permission)\
         .join((Repository, RepoToPerm.repository_id == Repository.repo_id))\
         .join((Permission, RepoToPerm.permission_id == Permission.permission_id))\
         .filter(RepoToPerm.user == default_user).all()
 
     if user.is_admin:
         #=======================================================================
         # #admin have all default rights set to admin        
         #=======================================================================
         user.permissions['global'].add('hg.admin')
 
         for perm in default_perms:
             p = 'repository.admin'
             user.permissions['repositories'][perm.RepoToPerm.repository.repo_name] = p
 
     else:
         #=======================================================================
         # set default permissions
         #=======================================================================
 
         #default global
         default_global_perms = sa.query(UserToPerm)\
             .filter(UserToPerm.user == sa.query(User)\
                    .filter(User.username == 'default').one())
 
         for perm in default_global_perms:
             user.permissions['global'].add(perm.permission.permission_name)
 
         #default repositories
         for perm in default_perms:
             if perm.Repository.private and not perm.Repository.user_id == user.user_id:
                 #disable defaults for private repos,
                 p = 'repository.none'
             elif perm.Repository.user_id == user.user_id:
                 #set admin if owner
                 p = 'repository.admin'
             else:
                 p = perm.Permission.permission_name
 
             user.permissions['repositories'][perm.RepoToPerm.repository.repo_name] = p
 
         #=======================================================================
         # #overwrite default with user permissions if any
         #=======================================================================
         user_perms = sa.query(RepoToPerm, Permission, Repository)\
             .join((Repository, RepoToPerm.repository_id == Repository.repo_id))\
             .join((Permission, RepoToPerm.permission_id == Permission.permission_id))\
             .filter(RepoToPerm.user_id == user.user_id).all()
 
         for perm in user_perms:
             if perm.Repository.user_id == user.user_id:#set admin if owner
                 p = 'repository.admin'
             else:
                 p = perm.Permission.permission_name
             user.permissions['repositories'][perm.RepoToPerm.repository.repo_name] = p
     meta.Session.remove()
     return user
 
 def get_user(session):
     """
     Gets user from session, and wraps permissions into user
     :param session:
     """
     user = session.get('rhodecode_user', AuthUser())
     #if the user is not logged in we check for anonymous access
     #if user is logged and it's a default user check if we still have anonymous
     #access enabled
     if user.user_id is None or user.username == 'default':
         anonymous_user = UserModel().get_by_username('default', cache=True)
         if anonymous_user.active is True:
             #then we set this user is logged in
             user.is_authenticated = True
             user.user_id = anonymous_user.user_id
         else:
             user.is_authenticated = False
 
     if user.is_authenticated:
         user = UserModel().fill_data(user)
 
     user = fill_perms(user)
     session['rhodecode_user'] = user
     session.save()
     return user
 
 #===============================================================================
 # CHECK DECORATORS
 #===============================================================================
 class LoginRequired(object):
     """Must be logged in to execute this function else 
     redirect to login page"""
 
     def __call__(self, func):
         return decorator(self.__wrapper, func)
 
     def __wrapper(self, func, *fargs, **fkwargs):
         user = session.get('rhodecode_user', AuthUser())
         log.debug('Checking login required for user:%s', user.username)
         if user.is_authenticated:
             log.debug('user %s is authenticated', user.username)
             return func(*fargs, **fkwargs)
         else:
             log.warn('user %s not authenticated', user.username)
 
             p = ''
             if request.environ.get('SCRIPT_NAME') != '/':
                 p += request.environ.get('SCRIPT_NAME')
 
             p += request.environ.get('PATH_INFO')
             if request.environ.get('QUERY_STRING'):
                 p += '?' + request.environ.get('QUERY_STRING')
 
             log.debug('redirecting to login page with %s', p)
             return redirect(url('login_home', came_from=p))
 
 class NotAnonymous(object):
     """Must be logged in to execute this function else 
     redirect to login page"""
 
     def __call__(self, func):
         return decorator(self.__wrapper, func)
 
     def __wrapper(self, func, *fargs, **fkwargs):
         user = session.get('rhodecode_user', AuthUser())
         log.debug('Checking if user is not anonymous')
 
         anonymous = user.username == 'default'
 
         if anonymous:
             p = ''
             if request.environ.get('SCRIPT_NAME') != '/':
                 p += request.environ.get('SCRIPT_NAME')
 
             p += request.environ.get('PATH_INFO')
             if request.environ.get('QUERY_STRING'):
                 p += '?' + request.environ.get('QUERY_STRING')
             return redirect(url('login_home', came_from=p))
         else:
             return func(*fargs, **fkwargs)
 
 class PermsDecorator(object):
     """Base class for decorators"""
 
     def __init__(self, *required_perms):
         available_perms = config['available_permissions']
         for perm in required_perms:
             if perm not in available_perms:
                 raise Exception("'%s' permission is not defined" % perm)
         self.required_perms = set(required_perms)
         self.user_perms = None
 
     def __call__(self, func):
         return decorator(self.__wrapper, func)
 
 
     def __wrapper(self, func, *fargs, **fkwargs):
 #        _wrapper.__name__ = func.__name__
 #        _wrapper.__dict__.update(func.__dict__)
 #        _wrapper.__doc__ = func.__doc__
         self.user = session.get('rhodecode_user', AuthUser())
         self.user_perms = self.user.permissions
         log.debug('checking %s permissions %s for %s %s',
            self.__class__.__name__, self.required_perms, func.__name__,
                self.user)
 
         if self.check_permissions():
             log.debug('Permission granted for %s %s', func.__name__, self.user)
 
             return func(*fargs, **fkwargs)
 
         else:
             log.warning('Permission denied for %s %s', func.__name__, self.user)
             #redirect with forbidden ret code
             return abort(403)
 
 
 
     def check_permissions(self):
         """Dummy function for overriding"""
         raise Exception('You have to write this function in child class')
 
 class HasPermissionAllDecorator(PermsDecorator):
     """Checks for access permission for all given predicates. All of them 
     have to be meet in order to fulfill the request
     """
 
     def check_permissions(self):
         if self.required_perms.issubset(self.user_perms.get('global')):
             return True
         return False
 
 
 class HasPermissionAnyDecorator(PermsDecorator):
     """Checks for access permission for any of given predicates. In order to 
     fulfill the request any of predicates must be meet
     """
 
     def check_permissions(self):
         if self.required_perms.intersection(self.user_perms.get('global')):
             return True
         return False
 
 class HasRepoPermissionAllDecorator(PermsDecorator):
     """Checks for access permission for all given predicates for specific 
     repository. All of them have to be meet in order to fulfill the request
     """
 
     def check_permissions(self):
         repo_name = get_repo_slug(request)
         try:
             user_perms = set([self.user_perms['repositories'][repo_name]])
         except KeyError:
             return False
         if self.required_perms.issubset(user_perms):
             return True
         return False
 
 
 class HasRepoPermissionAnyDecorator(PermsDecorator):
     """Checks for access permission for any of given predicates for specific 
     repository. In order to fulfill the request any of predicates must be meet
     """
 
     def check_permissions(self):
         repo_name = get_repo_slug(request)
 
         try:
             user_perms = set([self.user_perms['repositories'][repo_name]])
         except KeyError:
             return False
         if self.required_perms.intersection(user_perms):
             return True
         return False
 #===============================================================================
 # CHECK FUNCTIONS
 #===============================================================================
 
 class PermsFunction(object):
     """Base function for other check functions"""
 
     def __init__(self, *perms):
         available_perms = config['available_permissions']
 
         for perm in perms:
             if perm not in available_perms:
                 raise Exception("'%s' permission in not defined" % perm)
         self.required_perms = set(perms)
         self.user_perms = None
         self.granted_for = ''
         self.repo_name = None
 
     def __call__(self, check_Location=''):
         user = session.get('rhodecode_user', False)
         if not user:
             return False
         self.user_perms = user.permissions
         self.granted_for = user.username
         log.debug('checking %s %s %s', self.__class__.__name__,
                   self.required_perms, user)
 
         if self.check_permissions():
             log.debug('Permission granted for %s @ %s %s', self.granted_for,
                       check_Location, user)
             return True
 
         else:
             log.warning('Permission denied for %s @ %s %s', self.granted_for,
                         check_Location, user)
             return False
 
     def check_permissions(self):
         """Dummy function for overriding"""
         raise Exception('You have to write this function in child class')
 
 class HasPermissionAll(PermsFunction):
     def check_permissions(self):
         if self.required_perms.issubset(self.user_perms.get('global')):
             return True
         return False
 
 class HasPermissionAny(PermsFunction):
     def check_permissions(self):
         if self.required_perms.intersection(self.user_perms.get('global')):
             return True
         return False
 
 class HasRepoPermissionAll(PermsFunction):
 
     def __call__(self, repo_name=None, check_Location=''):
         self.repo_name = repo_name
         return super(HasRepoPermissionAll, self).__call__(check_Location)
 
     def check_permissions(self):
         if not self.repo_name:
             self.repo_name = get_repo_slug(request)
 
         try:
             self.user_perms = set([self.user_perms['repositories']\
                                    [self.repo_name]])
         except KeyError:
             return False
         self.granted_for = self.repo_name
         if self.required_perms.issubset(self.user_perms):
             return True
         return False
 
 class HasRepoPermissionAny(PermsFunction):
 
     def __call__(self, repo_name=None, check_Location=''):
         self.repo_name = repo_name
         return super(HasRepoPermissionAny, self).__call__(check_Location)
 
     def check_permissions(self):
         if not self.repo_name:
             self.repo_name = get_repo_slug(request)
 
         try:
             self.user_perms = set([self.user_perms['repositories']\
                                    [self.repo_name]])
         except KeyError:
             return False
         self.granted_for = self.repo_name
         if self.required_perms.intersection(self.user_perms):
             return True
         return False
 
 #===============================================================================
 # SPECIAL VERSION TO HANDLE MIDDLEWARE AUTH
 #===============================================================================
 
 class HasPermissionAnyMiddleware(object):
     def __init__(self, *perms):
         self.required_perms = set(perms)
 
     def __call__(self, user, repo_name):
         usr = AuthUser()
         usr.user_id = user.user_id
         usr.username = user.username
         usr.is_admin = user.admin
 
         try:
             self.user_perms = set([fill_perms(usr)\
                                    .permissions['repositories'][repo_name]])
         except:
             self.user_perms = set()
         self.granted_for = ''
         self.username = user.username
         self.repo_name = repo_name
         return self.check_permissions()
 
     def check_permissions(self):
         log.debug('checking mercurial protocol '
                   'permissions for user:%s repository:%s',
                                                 self.username, self.repo_name)
         if self.required_perms.intersection(self.user_perms):
             log.debug('permission granted')
             return True
         log.debug('permission denied')
         return False

rhodecode/lib/auth_ldap.py

@@ -1,104 +1,104 b''
 #!/usr/bin/env python
 # encoding: utf-8
 # ldap authentication lib
-# Copyright (C) 2009-2010 Marcin Kuzminski <[e-mail]>
+# Copyright (C) 2009-2011 Marcin Kuzminski <[e-mail]>
 #
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 """
 Created on Nov 17, 2010
 
 @author: marcink
 """
 
 from rhodecode.lib.exceptions import *
 import logging
 
 log = logging.getLogger(__name__)
 
 try:
     import ldap
 except ImportError:
     pass
 
 class AuthLdap(object):
 
     def __init__(self, server, base_dn, port=389, bind_dn='', bind_pass='',
                  use_ldaps=False, ldap_version=3):
         self.ldap_version = ldap_version
         if use_ldaps:
             port = port or 689
         self.LDAP_USE_LDAPS = use_ldaps
         self.LDAP_SERVER_ADDRESS = server
         self.LDAP_SERVER_PORT = port
 
         #USE FOR READ ONLY BIND TO LDAP SERVER
         self.LDAP_BIND_DN = bind_dn
         self.LDAP_BIND_PASS = bind_pass
 
         ldap_server_type = 'ldap'
         if self.LDAP_USE_LDAPS:ldap_server_type = ldap_server_type + 's'
         self.LDAP_SERVER = "%s://%s:%s" % (ldap_server_type,
                                                self.LDAP_SERVER_ADDRESS,
                                                self.LDAP_SERVER_PORT)
 
         self.BASE_DN = base_dn
 
     def authenticate_ldap(self, username, password):
         """Authenticate a user via LDAP and return his/her LDAP properties.
     
         Raises AuthenticationError if the credentials are rejected, or
         EnvironmentError if the LDAP server can't be reached.
         
         :param username: username
         :param password: password
         """
 
         from rhodecode.lib.helpers import chop_at
 
         uid = chop_at(username, "@%s" % self.LDAP_SERVER_ADDRESS)
 
         if "," in username:
             raise LdapUsernameError("invalid character in username: ,")
         try:
             ldap.set_option(ldap.OPT_X_TLS_CACERTDIR, '/etc/openldap/cacerts')
             ldap.set_option(ldap.OPT_NETWORK_TIMEOUT, 10)
             server = ldap.initialize(self.LDAP_SERVER)
             if self.ldap_version == 2:
                 server.protocol = ldap.VERSION2
             else:
                 server.protocol = ldap.VERSION3
 
             if self.LDAP_BIND_DN and self.LDAP_BIND_PASS:
                 server.simple_bind_s(self.LDAP_BIND_DN, self.LDAP_BIND_PASS)
 
             dn = self.BASE_DN % {'user':uid}
             log.debug("Authenticating %r at %s", dn, self.LDAP_SERVER)
             server.simple_bind_s(dn, password)
 
             properties = server.search_s(dn, ldap.SCOPE_SUBTREE)
             if not properties:
                 raise ldap.NO_SUCH_OBJECT()
         except ldap.NO_SUCH_OBJECT, e:
             log.debug("LDAP says no such user '%s' (%s)", uid, username)
             raise LdapUsernameError()
         except ldap.INVALID_CREDENTIALS, e:
             log.debug("LDAP rejected password for user '%s' (%s)", uid, username)
             raise LdapPasswordError()
         except ldap.SERVER_DOWN, e:
             raise LdapConnectionError("LDAP can't access authentication server")
 
         return properties[0]
 

rhodecode/lib/db_manage.py

@@ -1,493 +1,477 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.lib.db_manage
     ~~~~~~~~~~~~~~~~~~~~~~~
 
     Database creation, and setup module for RhodeCode. Used for creation
     of database as well as for migration operations
     
     :created_on: Apr 10, 2010
     :author: marcink
     :copyright: (C) 2009-2011 Marcin Kuzminski <[e-mail]>    
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 
 import os
 import sys
 import uuid
 import logging
 from os.path import dirname as dn, join as jn
 
 from rhodecode import __dbversion__
 from rhodecode.model import meta
 
 from rhodecode.lib.auth import get_crypt_password
 from rhodecode.lib.utils import ask_ok
 from rhodecode.model import init_model
 from rhodecode.model.db import User, Permission, RhodeCodeUi, RhodeCodeSettings, \
     UserToPerm, DbMigrateVersion
 
 from sqlalchemy.engine import create_engine
 
 log = logging.getLogger(__name__)
 
 class DbManage(object):
     def __init__(self, log_sql, dbconf, root, tests=False):
         self.dbname = dbconf.split('/')[-1]
         self.tests = tests
         self.root = root
         self.dburi = dbconf
         self.log_sql = log_sql
         self.db_exists = False
         self.init_db()
 
     def init_db(self):
         engine = create_engine(self.dburi, echo=self.log_sql)
         init_model(engine)
         self.sa = meta.Session()
 
-    def check_for_db(self, override):
-        db_path = jn(self.root, self.dbname)
-        if self.dburi.startswith('sqlite'):
-            log.info('checking for existing db in %s', db_path)
-            if os.path.isfile(db_path):
-
-                self.db_exists = True
-                if not override:
-                    raise Exception('database already exists')
-            return 'sqlite'
-        if self.dburi.startswith('postgresql'):
-            self.db_exists = True
-            return 'postgresql'
-
-
     def create_tables(self, override=False):
         """Create a auth database
         """
 
-        db_type = self.check_for_db(override)
-        if self.db_exists:
-            log.info("database exist and it's going to be destroyed")
-            if self.tests:
-                destroy = True
-            else:
-                destroy = ask_ok('Are you sure to destroy old database ? [y/n]')
-            if not destroy:
-                sys.exit()
-            if self.db_exists and destroy:
-                if db_type == 'sqlite':
-                    os.remove(jn(self.root, self.dbname))
-                if db_type == 'postgresql':
-                    meta.Base.metadata.drop_all()
+        log.info("Any existing database is going to be destroyed")
+        if self.tests:
+            destroy = True
+        else:
+            destroy = ask_ok('Are you sure to destroy old database ? [y/n]')
+        if not destroy:
+            sys.exit()
+        if destroy:
+            meta.Base.metadata.drop_all()
 
         checkfirst = not override
         meta.Base.metadata.create_all(checkfirst=checkfirst)
         log.info('Created tables for %s', self.dbname)
 
 
 
     def set_db_version(self):
         try:
             ver = DbMigrateVersion()
             ver.version = __dbversion__
             ver.repository_id = 'rhodecode_db_migrations'
             ver.repository_path = 'versions'
             self.sa.add(ver)
             self.sa.commit()
         except:
             self.sa.rollback()
             raise
         log.info('db version set to: %s', __dbversion__)
 
 
     def upgrade(self):
         """Upgrades given database schema to given revision following 
         all needed steps, to perform the upgrade
         
         :param revision: revision to upgrade to
         """
 
         from rhodecode.lib.dbmigrate.migrate.versioning import api
         from rhodecode.lib.dbmigrate.migrate.exceptions import \
             DatabaseNotControlledError
 
         upgrade = ask_ok('You are about to perform database upgrade, make '
                          'sure You backed up your database before. '
                          'Continue ? [y/n]')
         if not upgrade:
             sys.exit('Nothing done')
 
         repository_path = jn(dn(dn(dn(os.path.realpath(__file__)))),
                              'rhodecode/lib/dbmigrate')
         db_uri = self.dburi
 
         try:
             curr_version = api.db_version(db_uri, repository_path)
             msg = ('Found current database under version'
                  ' control with version %s' % curr_version)
 
         except (RuntimeError, DatabaseNotControlledError), e:
             curr_version = 1
             msg = ('Current database is not under version control. Setting'
                    ' as version %s' % curr_version)
             api.version_control(db_uri, repository_path, curr_version)
 
         print (msg)
 
         if curr_version == __dbversion__:
             sys.exit('This database is already at the newest version')
 
         #======================================================================
         # UPGRADE STEPS
         #======================================================================
         class UpgradeSteps(object):
             """Those steps follow schema versions so for example schema 
             for example schema with seq 002 == step_2 and so on.
             """
 
             def __init__(self, klass):
                 self.klass = klass
 
             def step_0(self):
                 #step 0 is the schema upgrade, and than follow proper upgrades
                 print ('attempting to do database upgrade to version %s' \
                                 % __dbversion__)
                 api.upgrade(db_uri, repository_path, __dbversion__)
                 print ('Schema upgrade completed')
 
             def step_1(self):
                 pass
 
             def step_2(self):
                 print ('Patching repo paths for newer version of RhodeCode')
                 self.klass.fix_repo_paths()
 
                 print ('Patching default user of RhodeCode')
                 self.klass.fix_default_user()
 
                 log.info('Changing ui settings')
                 self.klass.create_ui_settings()
 
 
         upgrade_steps = [0] + range(curr_version + 1, __dbversion__ + 1)
 
         #CALL THE PROPER ORDER OF STEPS TO PERFORM FULL UPGRADE
         for step in upgrade_steps:
             print ('performing upgrade step %s' % step)
             callable = getattr(UpgradeSteps(self), 'step_%s' % step)()
 
 
 
     def fix_repo_paths(self):
         """Fixes a old rhodecode version path into new one without a '*'
         """
 
         paths = self.sa.query(RhodeCodeUi)\
                 .filter(RhodeCodeUi.ui_key == '/')\
                 .scalar()
 
         paths.ui_value = paths.ui_value.replace('*', '')
 
         try:
             self.sa.add(paths)
             self.sa.commit()
         except:
             self.sa.rollback()
             raise
 
     def fix_default_user(self):
         """Fixes a old default user with some 'nicer' default values,
         used mostly for anonymous access
         """
         def_user = self.sa.query(User)\
                 .filter(User.username == 'default')\
                 .one()
 
         def_user.name = 'Anonymous'
         def_user.lastname = 'User'
         def_user.email = '[e-mail]'
 
         try:
             self.sa.add(def_user)
             self.sa.commit()
         except:
             self.sa.rollback()
             raise
 
 
 
     def admin_prompt(self, second=False):
         if not self.tests:
             import getpass
 
 
             def get_password():
                 password = getpass.getpass('Specify admin password (min 6 chars):')
                 confirm = getpass.getpass('Confirm password:')
 
                 if password != confirm:
                     log.error('passwords mismatch')
                     return False
                 if len(password) < 6:
                     log.error('password is to short use at least 6 characters')
                     return False
 
                 return password
 
             username = raw_input('Specify admin username:')
 
             password = get_password()
             if not password:
                 #second try
                 password = get_password()
                 if not password:
                     sys.exit()
 
             email = raw_input('Specify admin email:')
             self.create_user(username, password, email, True)
         else:
             log.info('creating admin and regular test users')
             self.create_user('test_admin', 'test12', '[e-mail]', True)
             self.create_user('test_regular', 'test12', '[e-mail]', False)
             self.create_user('test_regular2', 'test12', '[e-mail]', False)
 
     def create_ui_settings(self):
         """Creates ui settings, fills out hooks
         and disables dotencode
         
         """
         #HOOKS
         hooks1_key = 'changegroup.update'
         hooks1_ = self.sa.query(RhodeCodeUi)\
             .filter(RhodeCodeUi.ui_key == hooks1_key).scalar()
 
         hooks1 = RhodeCodeUi() if hooks1_ is None else hooks1_
         hooks1.ui_section = 'hooks'
         hooks1.ui_key = hooks1_key
         hooks1.ui_value = 'hg update >&2'
         hooks1.ui_active = False
 
         hooks2_key = 'changegroup.repo_size'
         hooks2_ = self.sa.query(RhodeCodeUi)\
             .filter(RhodeCodeUi.ui_key == hooks2_key).scalar()
 
         hooks2 = RhodeCodeUi() if hooks2_ is None else hooks2_
         hooks2.ui_section = 'hooks'
         hooks2.ui_key = hooks2_key
         hooks2.ui_value = 'python:rhodecode.lib.hooks.repo_size'
 
         hooks3 = RhodeCodeUi()
         hooks3.ui_section = 'hooks'
         hooks3.ui_key = 'pretxnchangegroup.push_logger'
         hooks3.ui_value = 'python:rhodecode.lib.hooks.log_push_action'
 
         hooks4 = RhodeCodeUi()
         hooks4.ui_section = 'hooks'
         hooks4.ui_key = 'preoutgoing.pull_logger'
         hooks4.ui_value = 'python:rhodecode.lib.hooks.log_pull_action'
 
         #For mercurial 1.7 set backward comapatibility with format
         dotencode_disable = RhodeCodeUi()
         dotencode_disable.ui_section = 'format'
         dotencode_disable.ui_key = 'dotencode'
         dotencode_disable.ui_value = 'false'
 
         try:
             self.sa.add(hooks1)
             self.sa.add(hooks2)
             self.sa.add(hooks3)
             self.sa.add(hooks4)
             self.sa.add(dotencode_disable)
             self.sa.commit()
         except:
             self.sa.rollback()
             raise
 
 
     def create_ldap_options(self):
         """Creates ldap settings"""
 
         try:
-            for k in ['ldap_active', 'ldap_host', 'ldap_port', 'ldap_ldaps',
-                      'ldap_dn_user', 'ldap_dn_pass', 'ldap_base_dn']:
+            for k, v in [('ldap_active', 'false'),
+                        ('ldap_host', ''),
+                        ('ldap_port', '389'),
+                        ('ldap_ldaps', 'false'),
+                        ('ldap_dn_user', ''), ('ldap_dn_pass', ''),
+                        ('ldap_base_dn', '')]:
 
-                setting = RhodeCodeSettings(k, '')
+                setting = RhodeCodeSettings(k, v)
                 self.sa.add(setting)
             self.sa.commit()
         except:
             self.sa.rollback()
             raise
 
     def config_prompt(self, test_repo_path=''):
         log.info('Setting up repositories config')
 
         if not self.tests and not test_repo_path:
             path = raw_input('Specify valid full path to your repositories'
                         ' you can change this later in application settings:')
         else:
             path = test_repo_path
 
         if not os.path.isdir(path):
             log.error('You entered wrong path: %s', path)
             sys.exit()
 
         self.create_ui_settings()
 
         #HG UI OPTIONS
         web1 = RhodeCodeUi()
         web1.ui_section = 'web'
         web1.ui_key = 'push_ssl'
         web1.ui_value = 'false'
 
         web2 = RhodeCodeUi()
         web2.ui_section = 'web'
         web2.ui_key = 'allow_archive'
         web2.ui_value = 'gz zip bz2'
 
         web3 = RhodeCodeUi()
         web3.ui_section = 'web'
         web3.ui_key = 'allow_push'
         web3.ui_value = '*'
 
         web4 = RhodeCodeUi()
         web4.ui_section = 'web'
         web4.ui_key = 'baseurl'
         web4.ui_value = '/'
 
         paths = RhodeCodeUi()
         paths.ui_section = 'paths'
         paths.ui_key = '/'
         paths.ui_value = path
 
 
         hgsettings1 = RhodeCodeSettings('realm', 'RhodeCode authentication')
         hgsettings2 = RhodeCodeSettings('title', 'RhodeCode')
 
 
         try:
             self.sa.add(web1)
             self.sa.add(web2)
             self.sa.add(web3)
             self.sa.add(web4)
             self.sa.add(paths)
             self.sa.add(hgsettings1)
             self.sa.add(hgsettings2)
 
             self.sa.commit()
         except:
             self.sa.rollback()
             raise
 
         self.create_ldap_options()
 
         log.info('created ui config')
 
     def create_user(self, username, password, email='', admin=False):
         log.info('creating administrator user %s', username)
         new_user = User()
         new_user.username = username
         new_user.password = get_crypt_password(password)
         new_user.name = 'RhodeCode'
         new_user.lastname = 'Admin'
         new_user.email = email
         new_user.admin = admin
         new_user.active = True
 
         try:
             self.sa.add(new_user)
             self.sa.commit()
         except:
             self.sa.rollback()
             raise
 
     def create_default_user(self):
         log.info('creating default user')
         #create default user for handling default permissions.
         def_user = User()
         def_user.username = 'default'
         def_user.password = get_crypt_password(str(uuid.uuid1())[:8])
         def_user.name = 'Anonymous'
         def_user.lastname = 'User'
         def_user.email = '[e-mail]'
         def_user.admin = False
         def_user.active = False
         try:
             self.sa.add(def_user)
             self.sa.commit()
         except:
             self.sa.rollback()
             raise
 
     def create_permissions(self):
         #module.(access|create|change|delete)_[name]
         #module.(read|write|owner)
         perms = [('repository.none', 'Repository no access'),
                  ('repository.read', 'Repository read access'),
                  ('repository.write', 'Repository write access'),
                  ('repository.admin', 'Repository admin access'),
                  ('hg.admin', 'Hg Administrator'),
                  ('hg.create.repository', 'Repository create'),
                  ('hg.create.none', 'Repository creation disabled'),
                  ('hg.register.none', 'Register disabled'),
                  ('hg.register.manual_activate', 'Register new user with rhodecode without manual activation'),
                  ('hg.register.auto_activate', 'Register new user with rhodecode without auto activation'),
                 ]
 
         for p in perms:
             new_perm = Permission()
             new_perm.permission_name = p[0]
             new_perm.permission_longname = p[1]
             try:
                 self.sa.add(new_perm)
                 self.sa.commit()
             except:
                 self.sa.rollback()
                 raise
 
     def populate_default_permissions(self):
         log.info('creating default user permissions')
 
         default_user = self.sa.query(User)\
         .filter(User.username == 'default').scalar()
 
         reg_perm = UserToPerm()
         reg_perm.user = default_user
         reg_perm.permission = self.sa.query(Permission)\
         .filter(Permission.permission_name == 'hg.register.manual_activate')\
         .scalar()
 
         create_repo_perm = UserToPerm()
         create_repo_perm.user = default_user
         create_repo_perm.permission = self.sa.query(Permission)\
         .filter(Permission.permission_name == 'hg.create.repository')\
         .scalar()
 
         default_repo_perm = UserToPerm()
         default_repo_perm.user = default_user
         default_repo_perm.permission = self.sa.query(Permission)\
         .filter(Permission.permission_name == 'repository.read')\
         .scalar()
 
         try:
             self.sa.add(reg_perm)
             self.sa.add(create_repo_perm)
             self.sa.add(default_repo_perm)
             self.sa.commit()
         except:
             self.sa.rollback()
             raise
 

rhodecode/lib/helpers.py

@@ -1,555 +1,557 b''
 """Helper functions
 
 Consists of functions to typically be used within templates, but also
 available to Controllers. This module is available to both as 'h'.
 """
 import random
 import hashlib
 from pygments.formatters import HtmlFormatter
 from pygments import highlight as code_highlight
 from pylons import url, app_globals as g
 from pylons.i18n.translation import _, ungettext
 from vcs.utils.annotate import annotate_highlight
 from webhelpers.html import literal, HTML, escape
 from webhelpers.html.tools import *
 from webhelpers.html.builder import make_tag
 from webhelpers.html.tags import auto_discovery_link, checkbox, css_classes, \
     end_form, file, form, hidden, image, javascript_link, link_to, link_to_if, \
     link_to_unless, ol, required_legend, select, stylesheet_link, submit, text, \
     password, textarea, title, ul, xml_declaration, radio
 from webhelpers.html.tools import auto_link, button_to, highlight, js_obfuscate, \
     mail_to, strip_links, strip_tags, tag_re
 from webhelpers.number import format_byte_size, format_bit_size
 from webhelpers.pylonslib import Flash as _Flash
 from webhelpers.pylonslib.secure_form import secure_form
 from webhelpers.text import chop_at, collapse, convert_accented_entities, \
     convert_misc_entities, lchop, plural, rchop, remove_formatting, \
     replace_whitespace, urlify, truncate, wrap_paragraphs
 from webhelpers.date import time_ago_in_words
 
 from webhelpers.html.tags import _set_input_attrs, _set_id_attr, \
     convert_boolean_attrs, NotGiven
 
 def _reset(name, value=None, id=NotGiven, type="reset", **attrs):
     _set_input_attrs(attrs, type, name, value)
     _set_id_attr(attrs, id, name)
     convert_boolean_attrs(attrs, ["disabled"])
     return HTML.input(**attrs)
 
 reset = _reset
 
 
 def get_token():
     """Return the current authentication token, creating one if one doesn't
     already exist.
     """
     token_key = "_authentication_token"
     from pylons import session
     if not token_key in session:
         try:
             token = hashlib.sha1(str(random.getrandbits(128))).hexdigest()
         except AttributeError: # Python < 2.4
             token = hashlib.sha1(str(random.randrange(2 ** 128))).hexdigest()
         session[token_key] = token
         if hasattr(session, 'save'):
             session.save()
     return session[token_key]
 
 
 #Custom helpers here :)
 class _Link(object):
     '''
     Make a url based on label and url with help of url_for
     :param label:name of link    if not defined url is used
     :param url: the url for link
     '''
 
     def __call__(self, label='', *url_, **urlargs):
         if label is None or '':
             label = url
         link_fn = link_to(label, url(*url_, **urlargs))
         return link_fn
 
 link = _Link()
 
 class _GetError(object):
 
     def __call__(self, field_name, form_errors):
         tmpl = """<span class="error_msg">%s</span>"""
         if form_errors and form_errors.has_key(field_name):
             return literal(tmpl % form_errors.get(field_name))
 
 get_error = _GetError()
 
 def recursive_replace(str, replace=' '):
     """
     Recursive replace of given sign to just one instance
     :param str: given string
     :param replace:char to find and replace multiple instances
         
     Examples::
     >>> recursive_replace("Mighty---Mighty-Bo--sstones",'-')
     'Mighty-Mighty-Bo-sstones'
     """
 
     if str.find(replace * 2) == -1:
         return str
     else:
         str = str.replace(replace * 2, replace)
         return recursive_replace(str, replace)
 
 class _ToolTip(object):
 
     def __call__(self, tooltip_title, trim_at=50):
         """
         Special function just to wrap our text into nice formatted autowrapped
         text
         :param tooltip_title:
         """
 
         return wrap_paragraphs(escape(tooltip_title), trim_at)\
                        .replace('\n', '<br/>')
 
     def activate(self):
         """
         Adds tooltip mechanism to the given Html all tooltips have to have 
         set class tooltip and set attribute tooltip_title.
         Then a tooltip will be generated based on that
         All with yui js tooltip
         """
 
         js = '''
         YAHOO.util.Event.onDOMReady(function(){
             function toolTipsId(){
                 var ids = [];
                 var tts = YAHOO.util.Dom.getElementsByClassName('tooltip');
                 
                 for (var i = 0; i < tts.length; i++) {
                     //if element doesn't not have and id autogenerate one for tooltip
                     
                     if (!tts[i].id){
                         tts[i].id='tt'+i*100;
                     }
                     ids.push(tts[i].id);
                 }
                 return ids        
             };
             var myToolTips = new YAHOO.widget.Tooltip("tooltip", { 
                 context: toolTipsId(),
                 monitorresize:false,
                 xyoffset :[0,0],
                 autodismissdelay:300000,
                 hidedelay:5,
                 showdelay:20,
             });
             
             //Mouse Over event disabled for new repositories since they don't
             //have last commit message
             myToolTips.contextMouseOverEvent.subscribe(
                 function(type, args) {
                     var context = args[0];
                     var txt = context.getAttribute('tooltip_title');
                     if(txt){                                       
                         return true;
                     }
                     else{
                         return false;
                     }
                 });
             
                             
             // Set the text for the tooltip just before we display it. Lazy method
             myToolTips.contextTriggerEvent.subscribe( 
                  function(type, args) { 
 
                  
                         var context = args[0]; 
                         
                         var txt = context.getAttribute('tooltip_title');
                         this.cfg.setProperty("text", txt);
                         
                         
                         // positioning of tooltip
                         var tt_w = this.element.clientWidth;
                         var tt_h = this.element.clientHeight;
                         
                         var context_w = context.offsetWidth;
                         var context_h = context.offsetHeight;
                         
                         var pos_x = YAHOO.util.Dom.getX(context);
                         var pos_y = YAHOO.util.Dom.getY(context);
 
                         var display_strategy = 'top';
                         var xy_pos = [0,0];
                         switch (display_strategy){
                         
                             case 'top':
                                 var cur_x = (pos_x+context_w/2)-(tt_w/2);
                                 var cur_y = (pos_y-tt_h-4);
                                 xy_pos = [cur_x,cur_y];                                
                                 break;
                             case 'bottom':
                                 var cur_x = (pos_x+context_w/2)-(tt_w/2);
                                 var cur_y = pos_y+context_h+4;
                                 xy_pos = [cur_x,cur_y];                                
                                 break;
                             case 'left':
                                 var cur_x = (pos_x-tt_w-4);
                                 var cur_y = pos_y-((tt_h/2)-context_h/2);
                                 xy_pos = [cur_x,cur_y];                                
                                 break;
                             case 'right':
                                 var cur_x = (pos_x+context_w+4);
                                 var cur_y = pos_y-((tt_h/2)-context_h/2);
                                 xy_pos = [cur_x,cur_y];                                
                                 break;
                              default:
                                 var cur_x = (pos_x+context_w/2)-(tt_w/2);
                                 var cur_y = pos_y-tt_h-4;
                                 xy_pos = [cur_x,cur_y];                                
                                 break;                             
                                  
                         }
 
                         this.cfg.setProperty("xy",xy_pos);
 
                   });
                   
             //Mouse out 
             myToolTips.contextMouseOutEvent.subscribe(
                 function(type, args) {
                     var context = args[0];
                     
                 });
         });
         '''
         return literal(js)
 
 tooltip = _ToolTip()
 
 class _FilesBreadCrumbs(object):
 
     def __call__(self, repo_name, rev, paths):
+        if isinstance(paths, str):
+            paths = paths.decode('utf-8', 'replace')
         url_l = [link_to(repo_name, url('files_home',
                                         repo_name=repo_name,
                                         revision=rev, f_path=''))]
         paths_l = paths.split('/')
 
         for cnt, p in enumerate(paths_l):
             if p != '':
                 url_l.append(link_to(p, url('files_home',
                                             repo_name=repo_name,
                                             revision=rev,
                                             f_path='/'.join(paths_l[:cnt + 1]))))
 
         return literal('/'.join(url_l))
 
 files_breadcrumbs = _FilesBreadCrumbs()
 class CodeHtmlFormatter(HtmlFormatter):
 
     def wrap(self, source, outfile):
         return self._wrap_div(self._wrap_pre(self._wrap_code(source)))
 
     def _wrap_code(self, source):
         for cnt, it in enumerate(source):
             i, t = it
             t = '<div id="#S-%s">%s</div>' % (cnt + 1, t)
             yield i, t
 def pygmentize(filenode, **kwargs):
     """
     pygmentize function using pygments
     :param filenode:
     """
     return literal(code_highlight(filenode.content,
                                   filenode.lexer, CodeHtmlFormatter(**kwargs)))
 
 def pygmentize_annotation(filenode, **kwargs):
     """
     pygmentize function for annotation
     :param filenode:
     """
 
     color_dict = {}
     def gen_color():
         """generator for getting 10k of evenly distibuted colors using hsv color
         and golden ratio.
         """
         import colorsys
         n = 10000
         golden_ratio = 0.618033988749895
         h = 0.22717784590367374
         #generate 10k nice web friendly colors in the same order
         for c in xrange(n):
             h += golden_ratio
             h %= 1
             HSV_tuple = [h, 0.95, 0.95]
             RGB_tuple = colorsys.hsv_to_rgb(*HSV_tuple)
             yield map(lambda x:str(int(x * 256)), RGB_tuple)
 
     cgenerator = gen_color()
 
     def get_color_string(cs):
         if color_dict.has_key(cs):
             col = color_dict[cs]
         else:
             col = color_dict[cs] = cgenerator.next()
         return "color: rgb(%s)! important;" % (', '.join(col))
 
     def url_func(changeset):
         tooltip_html = "<div style='font-size:0.8em'><b>Author:</b>" + \
         " %s<br/><b>Date:</b> %s</b><br/><b>Message:</b> %s<br/></div>"
 
         tooltip_html = tooltip_html % (changeset.author,
                                                changeset.date,
                                                tooltip(changeset.message))
         lnk_format = '%5s:%s' % ('r%s' % changeset.revision,
                                  short_id(changeset.raw_id))
         uri = link_to(
                 lnk_format,
                 url('changeset_home', repo_name=changeset.repository.name,
                     revision=changeset.raw_id),
                 style=get_color_string(changeset.raw_id),
                 class_='tooltip',
                 tooltip_title=tooltip_html
               )
 
         uri += '\n'
         return uri
     return literal(annotate_highlight(filenode, url_func, **kwargs))
 
 def repo_name_slug(value):
     """Return slug of name of repository
     This function is called on each creation/modification
     of repository to prevent bad names in repo
     """
     slug = remove_formatting(value)
     slug = strip_tags(slug)
 
     for c in """=[]\;'"<>,/~!@#$%^&*()+{}|: """:
         slug = slug.replace(c, '-')
     slug = recursive_replace(slug, '-')
     slug = collapse(slug, '-')
     return slug
 
 def get_changeset_safe(repo, rev):
     from vcs.backends.base import BaseRepository
     from vcs.exceptions import RepositoryError
     if not isinstance(repo, BaseRepository):
         raise Exception('You must pass an Repository '
                         'object as first argument got %s', type(repo))
 
     try:
         cs = repo.get_changeset(rev)
     except RepositoryError:
         from rhodecode.lib.utils import EmptyChangeset
         cs = EmptyChangeset()
     return cs
 
 
 flash = _Flash()
 
 
 #==============================================================================
 # MERCURIAL FILTERS available via h.
 #==============================================================================
 from mercurial import util
 from mercurial.templatefilters import person as _person
 
 
 
 def _age(curdate):
     """turns a datetime into an age string."""
 
     if not curdate:
         return ''
 
     from datetime import timedelta, datetime
 
     agescales = [("year", 3600 * 24 * 365),
                  ("month", 3600 * 24 * 30),
                  ("day", 3600 * 24),
                  ("hour", 3600),
                  ("minute", 60),
                  ("second", 1), ]
 
     age = datetime.now() - curdate
     age_seconds = (age.days * agescales[2][1]) + age.seconds
     pos = 1
     for scale in agescales:
         if scale[1] <= age_seconds:
             if pos == 6:pos = 5
             return time_ago_in_words(curdate, agescales[pos][0]) + ' ' + _('ago')
         pos += 1
 
     return _('just now')
 
 age = lambda  x:_age(x)
 capitalize = lambda x: x.capitalize()
 email = util.email
 email_or_none = lambda x: util.email(x) if util.email(x) != x else None
 person = lambda x: _person(x)
 short_id = lambda x: x[:12]
 
 
 def bool2icon(value):
     """
     Returns True/False values represented as small html image of true/false
     icons
     :param value: bool value
     """
 
     if value is True:
         return HTML.tag('img', src=url("/images/icons/accept.png"), alt=_('True'))
 
     if value is False:
         return HTML.tag('img', src=url("/images/icons/cancel.png"), alt=_('False'))
 
     return value
 
 
 def action_parser(user_log):
     """
     This helper will map the specified string action into translated
     fancy names with icons and links
     
     @param action:
     """
     action = user_log.action
     action_params = ' '
 
     x = action.split(':')
 
     if len(x) > 1:
         action, action_params = x
 
     def get_cs_links():
         revs_limit = 5
         revs = action_params.split(',')
         cs_links = " " + ', '.join ([link(rev,
                 url('changeset_home',
                 repo_name=user_log.repository.repo_name,
                 revision=rev)) for rev in revs[:revs_limit] ])
         if len(revs) > revs_limit:
             uniq_id = revs[0]
             html_tmpl = ('<span> %s '
             '<a class="show_more" id="_%s" href="#">%s</a> '
             '%s</span>')
             cs_links += html_tmpl % (_('and'), uniq_id, _('%s more') \
                                         % (len(revs) - revs_limit),
                                         _('revisions'))
 
             html_tmpl = '<span id="%s" style="display:none"> %s </span>'
             cs_links += html_tmpl % (uniq_id, ', '.join([link(rev,
                 url('changeset_home',
                 repo_name=user_log.repository.repo_name,
                 revision=rev)) for rev in revs[revs_limit:] ]))
 
         return cs_links
 
     def get_fork_name():
         repo_name = action_params
         return str(link_to(action_params, url('summary_home',
                                           repo_name=repo_name,)))
 
     map = {'user_deleted_repo':(_('[deleted] repository'), None),
            'user_created_repo':(_('[created] repository'), None),
            'user_forked_repo':(_('[forked] repository as'), get_fork_name),
            'user_updated_repo':(_('[updated] repository'), None),
            'admin_deleted_repo':(_('[delete] repository'), None),
            'admin_created_repo':(_('[created] repository'), None),
            'admin_forked_repo':(_('[forked] repository'), None),
            'admin_updated_repo':(_('[updated] repository'), None),
            'push':(_('[pushed] '), get_cs_links),
            'pull':(_('[pulled] '), None),
            'started_following_repo':(_('[started following] repository'), None),
            'stopped_following_repo':(_('[stopped following] repository'), None),
             }
 
     action_str = map.get(action, action)
     action = action_str[0].replace('[', '<span class="journal_highlight">')\
                    .replace(']', '</span>')
     action_params_func = lambda :""
 
     if action_str[1] is not None:
         action_params_func = action_str[1]
 
     return literal(action + " " + action_params_func())
 
 def action_parser_icon(user_log):
     action = user_log.action
     action_params = None
     x = action.split(':')
 
     if len(x) > 1:
         action, action_params = x
 
-    tmpl = """<img src="%s/%s" alt="%s"/>"""
+    tmpl = """<img src="%s%s" alt="%s"/>"""
     map = {'user_deleted_repo':'database_delete.png',
            'user_created_repo':'database_add.png',
            'user_forked_repo':'arrow_divide.png',
            'user_updated_repo':'database_edit.png',
            'admin_deleted_repo':'database_delete.png',
            'admin_created_repo':'database_add.png',
            'admin_forked_repo':'arrow_divide.png',
            'admin_updated_repo':'database_edit.png',
            'push':'script_add.png',
            'pull':'down_16.png',
            'started_following_repo':'heart_add.png',
            'stopped_following_repo':'heart_delete.png',
             }
     return literal(tmpl % ((url('/images/icons/')),
                            map.get(action, action), action))
 
 
 #==============================================================================
 # PERMS
 #==============================================================================
 from rhodecode.lib.auth import HasPermissionAny, HasPermissionAll, \
 HasRepoPermissionAny, HasRepoPermissionAll
 
 #==============================================================================
 # GRAVATAR URL
 #==============================================================================
 import hashlib
 import urllib
 from pylons import request
 
 def gravatar_url(email_address, size=30):
     ssl_enabled = 'https' == request.environ.get('wsgi.url_scheme')
     default = 'identicon'
     baseurl_nossl = "http://www.gravatar.com/avatar/"
     baseurl_ssl = "https://secure.gravatar.com/avatar/"
     baseurl = baseurl_ssl if ssl_enabled else baseurl_nossl
 
 
     # construct the url
     gravatar_url = baseurl + hashlib.md5(email_address.lower()).hexdigest() + "?"
     gravatar_url += urllib.urlencode({'d':default, 's':str(size)})
 
     return gravatar_url
 
 def safe_unicode(str):
     """safe unicode function. In case of UnicodeDecode error we try to return
     unicode with errors replace, if this failes we return unicode with 
     string_escape decoding """
 
     try:
         u_str = unicode(str)
     except UnicodeDecodeError:
         try:
             u_str = unicode(str, 'utf-8', 'replace')
         except UnicodeDecodeError:
             #incase we have a decode error just represent as byte string
             u_str = unicode(str(str).encode('string_escape'))
 
     return u_str
 
 def changed_tooltip(nodes):
     if nodes:
         pref = ': <br/> '
         suf = ''
         if len(nodes) > 30:
             suf = '<br/>' + _(' and %s more') % (len(nodes) - 30)
-        return literal(pref + '<br/> '.join([x.path for x in nodes[:30]]) + suf)
+        return literal(pref + '<br/> '.join([x.path.decode('utf-8', 'replace') for x in nodes[:30]]) + suf)
     else:
         return ': ' + _('No Files')

rhodecode/model/db.py

@@ -1,271 +1,249 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.model.db
     ~~~~~~~~~~~~~~~~~~
     
     Database Models for RhodeCode
     
     :created_on: Apr 08, 2010
     :author: marcink
     :copyright: (C) 2009-2011 Marcin Kuzminski <[e-mail]>    
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 import logging
 import datetime
 from datetime import date
 
 from sqlalchemy import *
 from sqlalchemy.exc import DatabaseError
-from sqlalchemy.orm import relationship, backref, class_mapper
-from sqlalchemy.orm.session import Session
+from sqlalchemy.orm import relationship, backref
+from sqlalchemy.orm.interfaces import MapperExtension
 
-from rhodecode.model.meta import Base
+from rhodecode.model.meta import Base, Session
 
 log = logging.getLogger(__name__)
 
-class BaseModel(object):
 
-    @classmethod
-    def _get_keys(cls):
-        """return column names for this model """
-        return class_mapper(cls).c.keys()
-
-    def get_dict(self):
-        """return dict with keys and values corresponding 
-        to this model data """
-
-        d = {}
-        for k in self._get_keys():
-            d[k] = getattr(self, k)
-        return d
-
-    def get_appstruct(self):
-        """return list with keys and values tupples corresponding 
-        to this model data """
-
-        l = []
-        for k in self._get_keys():
-            l.append((k, getattr(self, k),))
-        return l
-
-    def populate_obj(self, populate_dict):
-        """populate model with data from given populate_dict"""
-
-        for k in self._get_keys():
-            if k in populate_dict:
-                setattr(self, k, populate_dict[k])
-
-class RhodeCodeSettings(Base, BaseModel):
+class RhodeCodeSettings(Base):
     __tablename__ = 'rhodecode_settings'
     __table_args__ = (UniqueConstraint('app_settings_name'), {'useexisting':True})
     app_settings_id = Column("app_settings_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
-    app_settings_name = Column("app_settings_name", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
-    app_settings_value = Column("app_settings_value", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    app_settings_name = Column("app_settings_name", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    app_settings_value = Column("app_settings_value", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
 
     def __init__(self, k='', v=''):
         self.app_settings_name = k
         self.app_settings_value = v
 
     def __repr__(self):
         return "<%s('%s:%s')>" % (self.__class__.__name__,
                                   self.app_settings_name, self.app_settings_value)
 
-class RhodeCodeUi(Base, BaseModel):
+class RhodeCodeUi(Base):
     __tablename__ = 'rhodecode_ui'
     __table_args__ = {'useexisting':True}
     ui_id = Column("ui_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
-    ui_section = Column("ui_section", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
-    ui_key = Column("ui_key", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
-    ui_value = Column("ui_value", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    ui_section = Column("ui_section", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    ui_key = Column("ui_key", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    ui_value = Column("ui_value", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     ui_active = Column("ui_active", Boolean(), nullable=True, unique=None, default=True)
 
 
-class User(Base, BaseModel):
+class User(Base):
     __tablename__ = 'users'
     __table_args__ = (UniqueConstraint('username'), UniqueConstraint('email'), {'useexisting':True})
     user_id = Column("user_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
-    username = Column("username", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
-    password = Column("password", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    username = Column("username", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    password = Column("password", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     active = Column("active", Boolean(), nullable=True, unique=None, default=None)
     admin = Column("admin", Boolean(), nullable=True, unique=None, default=False)
-    name = Column("name", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
-    lastname = Column("lastname", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
-    email = Column("email", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    name = Column("name", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    lastname = Column("lastname", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    email = Column("email", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     last_login = Column("last_login", DateTime(timezone=False), nullable=True, unique=None, default=None)
     is_ldap = Column("is_ldap", Boolean(), nullable=False, unique=None, default=False)
 
     user_log = relationship('UserLog', cascade='all')
     user_perms = relationship('UserToPerm', primaryjoin="User.user_id==UserToPerm.user_id", cascade='all')
 
     repositories = relationship('Repository')
     user_followers = relationship('UserFollowing', primaryjoin='UserFollowing.follows_user_id==User.user_id', cascade='all')
 
     @property
     def full_contact(self):
         return '%s %s <%s>' % (self.name, self.lastname, self.email)
 
+    @property
+    def short_contact(self):
+        return '%s %s' % (self.name, self.lastname)
+
 
     @property
     def is_admin(self):
         return self.admin
 
     def __repr__(self):
         return "<%s('id:%s:%s')>" % (self.__class__.__name__,
                                      self.user_id, self.username)
 
+    @classmethod
+    def by_username(cls, username):
+        return Session.query(cls).filter(cls.username == username).one()
+
+
     def update_lastlogin(self):
         """Update user lastlogin"""
 
         try:
             session = Session.object_session(self)
             self.last_login = datetime.datetime.now()
             session.add(self)
             session.commit()
             log.debug('updated user %s lastlogin', self.username)
         except (DatabaseError,):
             session.rollback()
 
 
-class UserLog(Base, BaseModel):
+class UserLog(Base):
     __tablename__ = 'user_logs'
     __table_args__ = {'useexisting':True}
     user_log_id = Column("user_log_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
     repository_id = Column("repository_id", Integer(length=None, convert_unicode=False, assert_unicode=None), ForeignKey('repositories.repo_id'), nullable=False, unique=None, default=None)
-    repository_name = Column("repository_name", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
-    user_ip = Column("user_ip", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
-    action = Column("action", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    repository_name = Column("repository_name", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    user_ip = Column("user_ip", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    action = Column("action", UnicodeText(length=1200000, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     action_date = Column("action_date", DateTime(timezone=False), nullable=True, unique=None, default=None)
 
     @property
     def action_as_day(self):
         return date(*self.action_date.timetuple()[:3])
 
     user = relationship('User')
     repository = relationship('Repository')
 
-class Repository(Base, BaseModel):
+class Repository(Base):
     __tablename__ = 'repositories'
     __table_args__ = (UniqueConstraint('repo_name'), {'useexisting':True},)
     repo_id = Column("repo_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
-    repo_name = Column("repo_name", String(length=None, convert_unicode=False, assert_unicode=None), nullable=False, unique=True, default=None)
-    repo_type = Column("repo_type", String(length=None, convert_unicode=False, assert_unicode=None), nullable=False, unique=False, default='hg')
+    repo_name = Column("repo_name", String(length=255, convert_unicode=False, assert_unicode=None), nullable=False, unique=True, default=None)
+    repo_type = Column("repo_type", String(length=255, convert_unicode=False, assert_unicode=None), nullable=False, unique=False, default='hg')
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=False, default=None)
     private = Column("private", Boolean(), nullable=True, unique=None, default=None)
     enable_statistics = Column("statistics", Boolean(), nullable=True, unique=None, default=True)
-    description = Column("description", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    description = Column("description", String(length=10000, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     fork_id = Column("fork_id", Integer(), ForeignKey('repositories.repo_id'), nullable=True, unique=False, default=None)
 
     user = relationship('User')
     fork = relationship('Repository', remote_side=repo_id)
     repo_to_perm = relationship('RepoToPerm', cascade='all')
     stats = relationship('Statistics', cascade='all', uselist=False)
 
     repo_followers = relationship('UserFollowing', primaryjoin='UserFollowing.follows_repo_id==Repository.repo_id', cascade='all')
 
     logs = relationship('UserLog', cascade='all')
-    
+
     def __repr__(self):
         return "<%s('%s:%s')>" % (self.__class__.__name__,
                                   self.repo_id, self.repo_name)
 
-class Permission(Base, BaseModel):
+class Permission(Base):
     __tablename__ = 'permissions'
     __table_args__ = {'useexisting':True}
     permission_id = Column("permission_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
-    permission_name = Column("permission_name", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
-    permission_longname = Column("permission_longname", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    permission_name = Column("permission_name", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    permission_longname = Column("permission_longname", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
 
     def __repr__(self):
         return "<%s('%s:%s')>" % (self.__class__.__name__,
                                   self.permission_id, self.permission_name)
 
-class RepoToPerm(Base, BaseModel):
+class RepoToPerm(Base):
     __tablename__ = 'repo_to_perm'
     __table_args__ = (UniqueConstraint('user_id', 'repository_id'), {'useexisting':True})
     repo_to_perm_id = Column("repo_to_perm_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
     permission_id = Column("permission_id", Integer(), ForeignKey('permissions.permission_id'), nullable=False, unique=None, default=None)
     repository_id = Column("repository_id", Integer(), ForeignKey('repositories.repo_id'), nullable=False, unique=None, default=None)
 
     user = relationship('User')
     permission = relationship('Permission')
     repository = relationship('Repository')
 
-class UserToPerm(Base, BaseModel):
+class UserToPerm(Base):
     __tablename__ = 'user_to_perm'
     __table_args__ = (UniqueConstraint('user_id', 'permission_id'), {'useexisting':True})
     user_to_perm_id = Column("user_to_perm_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
     permission_id = Column("permission_id", Integer(), ForeignKey('permissions.permission_id'), nullable=False, unique=None, default=None)
 
     user = relationship('User')
     permission = relationship('Permission')
 
-class Statistics(Base, BaseModel):
+class Statistics(Base):
     __tablename__ = 'statistics'
     __table_args__ = (UniqueConstraint('repository_id'), {'useexisting':True})
     stat_id = Column("stat_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
-    repository_id = Column("repository_id", Integer(), ForeignKey(u'repositories.repo_id'), nullable=False, unique=True, default=None)
+    repository_id = Column("repository_id", Integer(), ForeignKey('repositories.repo_id'), nullable=False, unique=True, default=None)
     stat_on_revision = Column("stat_on_revision", Integer(), nullable=False)
-    commit_activity = Column("commit_activity", LargeBinary(), nullable=False)#JSON data
+    commit_activity = Column("commit_activity", LargeBinary(1000000), nullable=False)#JSON data
     commit_activity_combined = Column("commit_activity_combined", LargeBinary(), nullable=False)#JSON data
-    languages = Column("languages", LargeBinary(), nullable=False)#JSON data
+    languages = Column("languages", LargeBinary(1000000), nullable=False)#JSON data
 
     repository = relationship('Repository', single_parent=True)
 
-class UserFollowing(Base, BaseModel):
+class UserFollowing(Base):
     __tablename__ = 'user_followings'
     __table_args__ = (UniqueConstraint('user_id', 'follows_repository_id'),
                       UniqueConstraint('user_id', 'follows_user_id')
                       , {'useexisting':True})
 
     user_following_id = Column("user_following_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
     follows_repo_id = Column("follows_repository_id", Integer(), ForeignKey('repositories.repo_id'), nullable=True, unique=None, default=None)
     follows_user_id = Column("follows_user_id", Integer(), ForeignKey('users.user_id'), nullable=True, unique=None, default=None)
 
     user = relationship('User', primaryjoin='User.user_id==UserFollowing.user_id')
 
     follows_user = relationship('User', primaryjoin='User.user_id==UserFollowing.follows_user_id')
     follows_repository = relationship('Repository', order_by='Repository.repo_name')
 
-class CacheInvalidation(Base, BaseModel):
+class CacheInvalidation(Base):
     __tablename__ = 'cache_invalidation'
     __table_args__ = (UniqueConstraint('cache_key'), {'useexisting':True})
     cache_id = Column("cache_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
-    cache_key = Column("cache_key", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
-    cache_args = Column("cache_args", String(length=None, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    cache_key = Column("cache_key", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
+    cache_args = Column("cache_args", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     cache_active = Column("cache_active", Boolean(), nullable=True, unique=None, default=False)
 
 
     def __init__(self, cache_key, cache_args=''):
         self.cache_key = cache_key
         self.cache_args = cache_args
         self.cache_active = False
 
     def __repr__(self):
         return "<%s('%s:%s')>" % (self.__class__.__name__,
                                   self.cache_id, self.cache_key)
 
-class DbMigrateVersion(Base, BaseModel):
+class DbMigrateVersion(Base):
     __tablename__ = 'db_migrate_version'
     __table_args__ = {'useexisting':True}
-    repository_id = Column('repository_id', String(250), primary_key=True)
+    repository_id = Column('repository_id', String(255), primary_key=True)
     repository_path = Column('repository_path', Text)
     version = Column('version', Integer)
 

rhodecode/model/meta.py

@@ -1,26 +1,70 b''
 """SQLAlchemy Metadata and Session object"""
 from sqlalchemy.ext.declarative import declarative_base
-from sqlalchemy.orm import scoped_session, sessionmaker
+from sqlalchemy.orm import scoped_session, sessionmaker, class_mapper
+from beaker import cache
+
 from rhodecode.model import caching_query
-from beaker import cache
+
 
 # Beaker CacheManager.  A home base for cache configurations.
 cache_manager = cache.CacheManager()
 
 __all__ = ['Base', 'Session']
 #
 # SQLAlchemy session manager. Updated by model.init_model()
 #
 Session = scoped_session(
                 sessionmaker(
                     query_cls=caching_query.query_callable(cache_manager)
                 )
           )
 
+class BaseModel(object):
+    """Base Model for all classess
+    
+    """
+
+    @classmethod
+    def _get_keys(cls):
+        """return column names for this model """
+        return class_mapper(cls).c.keys()
+
+    def get_dict(self):
+        """return dict with keys and values corresponding 
+        to this model data """
+
+        d = {}
+        for k in self._get_keys():
+            d[k] = getattr(self, k)
+        return d
+
+    def get_appstruct(self):
+        """return list with keys and values tupples corresponding 
+        to this model data """
+
+        l = []
+        for k in self._get_keys():
+            l.append((k, getattr(self, k),))
+        return l
+
+    def populate_obj(self, populate_dict):
+        """populate model with data from given populate_dict"""
+
+        for k in self._get_keys():
+            if k in populate_dict:
+                setattr(self, k, populate_dict[k])
+
+    @classmethod
+    def query(cls):
+        return Session.query(cls)
+
+    @classmethod
+    def get(cls, id_):
+        return Session.query(cls).get(id_)
+
+
 # The declarative Base
-Base = declarative_base()
-#For another db...
-#Base2 = declarative_base()
+Base = declarative_base(cls=BaseModel)
 
 #to use cache use this in query
 #.options(FromCache("sqlalchemy_cache_type", "cachekey"))

rhodecode/model/scm.py

@@ -1,377 +1,378 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.model.scm
     ~~~~~~~~~~~~~~~~~~~
 
     Scm model for RhodeCode
 
     :created_on: Apr 9, 2010
     :author: marcink
     :copyright: (C) 2009-2010 Marcin Kuzminski <[e-mail]>    
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software; you can redistribute it and/or
 # modify it under the terms of the GNU General Public License
 # as published by the Free Software Foundation; version 2
 # of the License or (at your opinion) any later version of the license.
 # 
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 # 
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software
 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
 # MA  02110-1301, USA.
 import os
 import time
 import traceback
 import logging
 
 from vcs import get_backend
 from vcs.utils.helpers import get_scm
 from vcs.exceptions import RepositoryError, VCSError
 from vcs.utils.lazy import LazyProperty
 
 from mercurial import ui
 
 from beaker.cache import cache_region, region_invalidate
 
 from rhodecode import BACKENDS
 from rhodecode.lib import helpers as h
 from rhodecode.lib.auth import HasRepoPermissionAny
 from rhodecode.lib.utils import get_repos, make_ui, action_logger
 from rhodecode.model import BaseModel
 from rhodecode.model.user import UserModel
 
 from rhodecode.model.db import Repository, RhodeCodeUi, CacheInvalidation, \
     UserFollowing, UserLog
 from rhodecode.model.caching_query import FromCache
 
 from sqlalchemy.orm import joinedload
 from sqlalchemy.orm.session import make_transient
 from sqlalchemy.exc import DatabaseError
 
 log = logging.getLogger(__name__)
 
 
 class UserTemp(object):
     def __init__(self, user_id):
         self.user_id = user_id
 class RepoTemp(object):
     def __init__(self, repo_id):
         self.repo_id = repo_id
 
 class ScmModel(BaseModel):
     """Generic Scm Model
     """
 
     @LazyProperty
     def repos_path(self):
         """Get's the repositories root path from database
         """
 
         q = self.sa.query(RhodeCodeUi).filter(RhodeCodeUi.ui_key == '/').one()
 
         return q.ui_value
 
     def repo_scan(self, repos_path, baseui):
         """Listing of repositories in given path. This path should not be a 
         repository itself. Return a dictionary of repository objects
         
         :param repos_path: path to directory containing repositories
         :param baseui: baseui instance to instantiate MercurialRepostitory with
         """
 
         log.info('scanning for repositories in %s', repos_path)
 
         if not isinstance(baseui, ui.ui):
             baseui = make_ui('db')
         repos_list = {}
 
         for name, path in get_repos(repos_path):
             try:
                 if repos_list.has_key(name):
                     raise RepositoryError('Duplicate repository name %s '
                                     'found in %s' % (name, path))
                 else:
 
                     klass = get_backend(path[0])
 
                     if path[0] == 'hg' and path[0] in BACKENDS.keys():
                         repos_list[name] = klass(path[1], baseui=baseui)
 
                     if path[0] == 'git' and path[0] in BACKENDS.keys():
                         repos_list[name] = klass(path[1])
             except OSError:
                 continue
 
         return repos_list
 
     def get_repos(self, all_repos=None):
         """Get all repos from db and for each repo create it's backend instance.
         and fill that backed with information from database
         
         :param all_repos: give specific repositories list, good for filtering
         """
 
         if all_repos is None:
             all_repos = self.sa.query(Repository)\
                 .order_by(Repository.repo_name).all()
 
         #get the repositories that should be invalidated
         invalidation_list = [str(x.cache_key) for x in \
                              self.sa.query(CacheInvalidation.cache_key)\
                              .filter(CacheInvalidation.cache_active == False)\
                              .all()]
 
         for r in all_repos:
 
             repo = self.get(r.repo_name, invalidation_list)
 
             if repo is not None:
                 last_change = repo.last_change
                 tip = h.get_changeset_safe(repo, 'tip')
 
                 tmp_d = {}
                 tmp_d['name'] = repo.name
                 tmp_d['name_sort'] = tmp_d['name'].lower()
                 tmp_d['description'] = repo.dbrepo.description
                 tmp_d['description_sort'] = tmp_d['description']
                 tmp_d['last_change'] = last_change
                 tmp_d['last_change_sort'] = time.mktime(last_change.timetuple())
                 tmp_d['tip'] = tip.raw_id
                 tmp_d['tip_sort'] = tip.revision
                 tmp_d['rev'] = tip.revision
                 tmp_d['contact'] = repo.dbrepo.user.full_contact
                 tmp_d['contact_sort'] = tmp_d['contact']
+                tmp_d['owner_sort'] = tmp_d['contact']
                 tmp_d['repo_archives'] = list(repo._get_archives())
                 tmp_d['last_msg'] = tip.message
                 tmp_d['repo'] = repo
                 yield tmp_d
 
     def get_repo(self, repo_name):
         return self.get(repo_name)
 
     def get(self, repo_name, invalidation_list=None):
         """Get's repository from given name, creates BackendInstance and
         propagates it's data from database with all additional information
         
         :param repo_name:
         :param invalidation_list: if a invalidation list is given the get
             method should not manually check if this repository needs 
             invalidation and just invalidate the repositories in list
             
         """
         if not HasRepoPermissionAny('repository.read', 'repository.write',
                             'repository.admin')(repo_name, 'get repo check'):
             return
 
         #======================================================================
         # CACHE FUNCTION
         #======================================================================
         @cache_region('long_term')
         def _get_repo(repo_name):
 
             repo_path = os.path.join(self.repos_path, repo_name)
 
             try:
                 alias = get_scm(repo_path)[0]
 
                 log.debug('Creating instance of %s repository', alias)
                 backend = get_backend(alias)
             except VCSError:
                 log.error(traceback.format_exc())
                 return
 
             if alias == 'hg':
                 from pylons import app_globals as g
                 repo = backend(repo_path, create=False, baseui=g.baseui)
                 #skip hidden web repository
                 if repo._get_hidden():
                     return
             else:
                 repo = backend(repo_path, create=False)
 
             dbrepo = self.sa.query(Repository)\
                 .options(joinedload(Repository.fork))\
                 .options(joinedload(Repository.user))\
                 .filter(Repository.repo_name == repo_name)\
                 .scalar()
 
             make_transient(dbrepo)
             if dbrepo.user:
                 make_transient(dbrepo.user)
             if dbrepo.fork:
                 make_transient(dbrepo.fork)
 
             repo.dbrepo = dbrepo
             return repo
 
         pre_invalidate = True
         if invalidation_list is not None:
             pre_invalidate = repo_name in invalidation_list
 
         if pre_invalidate:
             invalidate = self._should_invalidate(repo_name)
 
             if invalidate:
                 log.info('invalidating cache for repository %s', repo_name)
                 region_invalidate(_get_repo, None, repo_name)
                 self._mark_invalidated(invalidate)
 
         return _get_repo(repo_name)
 
 
 
     def mark_for_invalidation(self, repo_name):
         """Puts cache invalidation task into db for 
         further global cache invalidation
         
         :param repo_name: this repo that should invalidation take place
         """
 
         log.debug('marking %s for invalidation', repo_name)
         cache = self.sa.query(CacheInvalidation)\
             .filter(CacheInvalidation.cache_key == repo_name).scalar()
 
         if cache:
             #mark this cache as inactive
             cache.cache_active = False
         else:
             log.debug('cache key not found in invalidation db -> creating one')
             cache = CacheInvalidation(repo_name)
 
         try:
             self.sa.add(cache)
             self.sa.commit()
         except (DatabaseError,):
             log.error(traceback.format_exc())
             self.sa.rollback()
 
 
     def toggle_following_repo(self, follow_repo_id, user_id):
 
         f = self.sa.query(UserFollowing)\
             .filter(UserFollowing.follows_repo_id == follow_repo_id)\
             .filter(UserFollowing.user_id == user_id).scalar()
 
         if f is not None:
 
             try:
                 self.sa.delete(f)
                 self.sa.commit()
                 action_logger(UserTemp(user_id),
                               'stopped_following_repo',
                               RepoTemp(follow_repo_id))
                 return
             except:
                 log.error(traceback.format_exc())
                 self.sa.rollback()
                 raise
 
 
         try:
             f = UserFollowing()
             f.user_id = user_id
             f.follows_repo_id = follow_repo_id
             self.sa.add(f)
             self.sa.commit()
             action_logger(UserTemp(user_id),
                           'started_following_repo',
                           RepoTemp(follow_repo_id))
         except:
             log.error(traceback.format_exc())
             self.sa.rollback()
             raise
 
     def toggle_following_user(self, follow_user_id , user_id):
         f = self.sa.query(UserFollowing)\
             .filter(UserFollowing.follows_user_id == follow_user_id)\
             .filter(UserFollowing.user_id == user_id).scalar()
 
         if f is not None:
             try:
                 self.sa.delete(f)
                 self.sa.commit()
                 return
             except:
                 log.error(traceback.format_exc())
                 self.sa.rollback()
                 raise
 
         try:
             f = UserFollowing()
             f.user_id = user_id
             f.follows_user_id = follow_user_id
             self.sa.add(f)
             self.sa.commit()
         except:
             log.error(traceback.format_exc())
             self.sa.rollback()
             raise
 
     def is_following_repo(self, repo_name, user_id):
         r = self.sa.query(Repository)\
             .filter(Repository.repo_name == repo_name).scalar()
 
         f = self.sa.query(UserFollowing)\
             .filter(UserFollowing.follows_repository == r)\
             .filter(UserFollowing.user_id == user_id).scalar()
 
         return f is not None
 
     def is_following_user(self, username, user_id):
         u = UserModel(self.sa).get_by_username(username)
 
         f = self.sa.query(UserFollowing)\
             .filter(UserFollowing.follows_user == u)\
             .filter(UserFollowing.user_id == user_id).scalar()
 
         return f is not None
 
     def get_followers(self, repo_id):
         return self.sa.query(UserFollowing)\
                 .filter(UserFollowing.follows_repo_id == repo_id).count()
 
     def get_forks(self, repo_id):
         return self.sa.query(Repository)\
                 .filter(Repository.fork_id == repo_id).count()
 
 
     def get_unread_journal(self):
         return self.sa.query(UserLog).count()
 
 
     def _should_invalidate(self, repo_name):
         """Looks up database for invalidation signals for this repo_name
         
         :param repo_name:
         """
 
         ret = self.sa.query(CacheInvalidation)\
             .options(FromCache('sql_cache_short',
                            'get_invalidation_%s' % repo_name))\
             .filter(CacheInvalidation.cache_key == repo_name)\
             .filter(CacheInvalidation.cache_active == False)\
             .scalar()
 
         return ret
 
     def _mark_invalidated(self, cache_key):
         """ Marks all occurences of cache to invaldation as already invalidated
         
         :param cache_key:
         """
 
         if cache_key:
             log.debug('marking %s as already invalidated', cache_key)
         try:
             cache_key.cache_active = True
             self.sa.add(cache_key)
             self.sa.commit()
         except (DatabaseError,):
             log.error(traceback.format_exc())
             self.sa.rollback()
 

setup.py

@@ -1,104 +1,112 b''
 import sys
+from rhodecode import get_version
+from rhodecode import __platform__
+
 py_version = sys.version_info
 
-from rhodecode import get_version
-
 requirements = [
         "Pylons==1.0.0",
         "WebHelpers==1.2",
         "SQLAlchemy==0.6.6",
-        "Mako==0.3.6",
-        "vcs==0.1.10",
-        "pygments==1.3.1",
+        "Mako==0.4.0",
+        "vcs==0.1.11",
+        "pygments==1.4.0",
         "mercurial==1.7.5",
         "whoosh==1.3.4",
-        "celery==2.1.4",
-        "py-bcrypt",
+        "celery==2.2.4",
         "babel",
     ]
 
 classifiers = ['Development Status :: 5 - Production/Stable',
                'Environment :: Web Environment',
                'Framework :: Pylons',
                'Intended Audience :: Developers',
                'License :: OSI Approved :: BSD License',
                'Operating System :: OS Independent',
                'Programming Language :: Python', ]
 
-if sys.version_info < (2, 6):
+if py_version < (2, 6):
     requirements.append("simplejson")
     requirements.append("pysqlite")
 
+if __platform__ in ('Linux', 'Darwin'):
+    requirements.append("py-bcrypt")
+
+
 #additional files from project that goes somewhere in the filesystem
 #relative to sys.prefix
 data_files = []
 
 #additional files that goes into package itself
 package_data = {'rhodecode': ['i18n/*/LC_MESSAGES/*.mo', ], }
 
 description = ('Mercurial repository browser/management with '
                'build in push/pull server and full text search')
+keywords = ' '.join (['rhodecode', 'rhodiumcode', 'mercurial', 'git',
+                      'repository management', 'hgweb replacement'
+                      'hgwebdir', 'gitweb replacement', 'serving hgweb',
+                     ])
 #long description
 try:
     readme_file = 'README.rst'
     changelog_file = 'docs/changelog.rst'
     long_description = open(readme_file).read() + '\n\n' + \
         open(changelog_file).read()
 
 except IOError, err:
     sys.stderr.write("[WARNING] Cannot find file specified as "
         "long_description (%s)\n or changelog (%s) skipping that file" \
             % (readme_file, changelog_file))
     long_description = description
 
 
 try:
     from setuptools import setup, find_packages
 except ImportError:
     from ez_setup import use_setuptools
     use_setuptools()
     from setuptools import setup, find_packages
 #packages
 packages = find_packages(exclude=['ez_setup'])
 
 setup(
     name='RhodeCode',
     version=get_version(),
     description=description,
     long_description=long_description,
-    keywords='rhodiumcode mercurial web hgwebdir gitweb git replacement serving hgweb rhodecode',
+    keywords=keywords,
     license='BSD',
     author='Marcin Kuzminski',
     author_email='[e-mail]',
     url='http://hg.python-works.com',
     install_requires=requirements,
     classifiers=classifiers,
     setup_requires=["PasteScript>=1.6.3"],
     data_files=data_files,
     packages=packages,
     include_package_data=True,
     test_suite='nose.collector',
     package_data=package_data,
     message_extractors={'rhodecode': [
             ('**.py', 'python', None),
             ('templates/**.mako', 'mako', {'input_encoding': 'utf-8'}),
             ('public/**', 'ignore', None)]},
     zip_safe=False,
     paster_plugins=['PasteScript', 'Pylons'],
     entry_points="""
     [paste.app_factory]
     main = rhodecode.config.middleware:make_app
 
     [paste.app_install]
     main = pylons.util:PylonsInstaller
 
     [paste.global_paster_command]
     make-index = rhodecode.lib.indexers:MakeIndex
     upgrade-db = rhodecode.lib.dbmigrate:UpgradeDb
     celeryd=rhodecode.lib.celerypylons.commands:CeleryDaemonCommand
     celerybeat=rhodecode.lib.celerypylons.commands:CeleryBeatCommand
     camqadm=rhodecode.lib.celerypylons.commands:CAMQPAdminCommand
     celeryev=rhodecode.lib.celerypylons.commands:CeleryEventCommand
               
     """,
 )