merge with beta

CONTRIBUTORS

@@ -1,18 +1,19 b''
 List of contributors to RhodeCode project:
     Marcin Kuźmiński <[e-mail]>
     Lukasz Balcerzak <[e-mail]>
     Jason Harris <[e-mail]>
     Thayne Harbaugh  <[e-mail]>
     cejones
     Thomas Waldmann <[e-mail]>
     Lorenzo M. Catucci <[e-mail]>
     Dmitri Kuznetsov
     Jared Bunting <[e-mail]>
     Steve Romanow <[e-mail]>
     Augosto Hermann <[e-mail]>    
     Ankit Solanki <[e-mail]>    
     Liad Shani <[e-mail]>
     Les Peabody <[e-mail]>
     Jonas Oberschweiber <[e-mail]>
     Matt Zuba <[e-mail]>
-    Aras Pranckevicius <[e-mail]>
  No newline at end of file
+    Aras Pranckevicius <[e-mail]>
+    Tony Bussieres <[e-mail]>

docs/changelog.rst

@@ -1,516 +1,535 b''
 .. _changelog:
 
 Changelog
 =========
 
 
+1.3.2 (**2012-02-28**)
+----------------------
+
+news
+++++
+
+
+fixes
++++++
+
+- fixed git protocol issues with repos-groups
+- fixed git remote repos validator that prevented from cloning remote git repos
+- fixes #370 ending slashes fixes for repo and groups
+- fixes #368 improved git-protocol detection to handle other clients
+- fixes #366 When Setting Repository Group To Blank Repo Group Wont Be 
+  Moved To Root
+- fixes #371 fixed issues with beaker/sqlalchemy and non-ascii cache keys 
+- fixed #373 missing cascade drop on user_group_to_perm table
+
 1.3.1 (**2012-02-27**)
 ----------------------
 
 news
 ++++
 
 
 fixes
 +++++
 
 - redirection loop occurs when remember-me wasn't checked during login
 - fixes issues with git blob history generation 
 - don't fetch branch for git in file history dropdown. Causes unneeded slowness
 
 1.3.0 (**2012-02-26**)
 ----------------------
 
 news
 ++++
 
 - code review, inspired by github code-comments 
 - #215 rst and markdown README files support
 - #252 Container-based and proxy pass-through authentication support
 - #44 branch browser. Filtering of changelog by branches
 - mercurial bookmarks support
 - new hover top menu, optimized to add maximum size for important views
 - configurable clone url template with possibility to specify  protocol like 
   ssh:// or http:// and also manually alter other parts of clone_url.
 - enabled largefiles extension by default
 - optimized summary file pages and saved a lot of unused space in them
 - #239 option to manually mark repository as fork
 - #320 mapping of commit authors to RhodeCode users
 - #304 hashes are displayed using monospace font    
 - diff configuration, toggle white lines and context lines
 - #307 configurable diffs, whitespace toggle, increasing context lines
 - sorting on branches, tags and bookmarks using YUI datatable
 - improved file filter on files page
 - implements #330 api method for listing nodes ar particular revision
 - #73 added linking issues in commit messages to chosen issue tracker url
   based on user defined regular expression
 - added linking of changesets in commit messages  
 - new compact changelog with expandable commit messages
 - firstname and lastname are optional in user creation
 - #348 added post-create repository hook
 - #212 global encoding settings is now configurable from .ini files 
 - #227 added repository groups permissions
 - markdown gets codehilite extensions
 - new API methods, delete_repositories, grante/revoke permissions for groups 
   and repos
   
     
 fixes
 +++++
 
 - rewrote dbsession management for atomic operations, and better error handling
 - fixed sorting of repo tables
 - #326 escape of special html entities in diffs
 - normalized user_name => username in api attributes
 - fixes #298 ldap created users with mixed case emails created conflicts 
   on saving a form
 - fixes issue when owner of a repo couldn't revoke permissions for users 
   and groups
 - fixes #271 rare JSON serialization problem with statistics
 - fixes #337 missing validation check for conflicting names of a group with a
   repositories group
 - #340 fixed session problem for mysql and celery tasks
 - fixed #331 RhodeCode mangles repository names if the a repository group 
   contains the "full path" to the repositories
 - #355 RhodeCode doesn't store encrypted LDAP passwords
 
 1.2.5 (**2012-01-28**)
 ----------------------
 
 news
 ++++
 
 fixes
 +++++
 
 - #340 Celery complains about MySQL server gone away, added session cleanup
   for celery tasks
 - #341 "scanning for repositories in None" log message during Rescan was missing
   a parameter
 - fixed creating archives with subrepos. Some hooks were triggered during that
   operation leading to crash.
 - fixed missing email in account page.
 - Reverted Mercurial to 2.0.1 for windows due to bug in Mercurial that makes
   forking on windows impossible 
 
 1.2.4 (**2012-01-19**)
 ----------------------
 
 news
 ++++
 
 - RhodeCode is bundled with mercurial series 2.0.X by default, with
   full support to largefiles extension. Enabled by default in new installations
 - #329 Ability to Add/Remove Groups to/from a Repository via AP
 - added requires.txt file with requirements
      
 fixes
 +++++
 
 - fixes db session issues with celery when emailing admins
 - #331 RhodeCode mangles repository names if the a repository group 
   contains the "full path" to the repositories
 - #298 Conflicting e-mail addresses for LDAP and RhodeCode users
 - DB session cleanup after hg protocol operations, fixes issues with
   `mysql has gone away` errors
 - #333 doc fixes for get_repo api function
 - #271 rare JSON serialization problem with statistics enabled
 - #337 Fixes issues with validation of repository name conflicting with 
   a group name. A proper message is now displayed.
 - #292 made ldap_dn in user edit readonly, to get rid of confusion that field
   doesn't work   
 - #316 fixes issues with web description in hgrc files 
 
 1.2.3 (**2011-11-02**)
 ----------------------
 
 news
 ++++
 
 - added option to manage repos group for non admin users
 - added following API methods for get_users, create_user, get_users_groups, 
   get_users_group, create_users_group, add_user_to_users_groups, get_repos, 
   get_repo, create_repo, add_user_to_repo
 - implements #237 added password confirmation for my account 
   and admin edit user.
 - implements #291 email notification for global events are now sent to all
   administrator users, and global config email.
      
 fixes
 +++++
 
 - added option for passing auth method for smtp mailer
 - #276 issue with adding a single user with id>10 to usergroups
 - #277 fixes windows LDAP settings in which missing values breaks the ldap auth 
 - #288 fixes managing of repos in a group for non admin user
 
 1.2.2 (**2011-10-17**)
 ----------------------
 
 news
 ++++
 
 - #226 repo groups are available by path instead of numerical id
  
 fixes
 +++++
 
 - #259 Groups with the same name but with different parent group
 - #260 Put repo in group, then move group to another group -> repo becomes unavailable
 - #258 RhodeCode 1.2 assumes egg folder is writable (lockfiles problems)
 - #265 ldap save fails sometimes on converting attributes to booleans, 
   added getter and setter into model that will prevent from this on db model level
 - fixed problems with timestamps issues #251 and #213
 - fixes #266 RhodeCode allows to create repo with the same name and in 
   the same parent as group
 - fixes #245 Rescan of the repositories on Windows
 - fixes #248 cannot edit repos inside a group on windows
 - fixes #219 forking problems on windows
 
 1.2.1 (**2011-10-08**)
 ----------------------
 
 news
 ++++
 
 
 fixes
 +++++
 
 - fixed problems with basic auth and push problems 
 - gui fixes
 - fixed logger
 
 1.2.0 (**2011-10-07**)
 ----------------------
 
 news
 ++++
 
 - implemented #47 repository groups
 - implemented #89 Can setup google analytics code from settings menu
 - implemented #91 added nicer looking archive urls with more download options
   like tags, branches
 - implemented #44 into file browsing, and added follow branch option
 - implemented #84 downloads can be enabled/disabled for each repository
 - anonymous repository can be cloned without having to pass default:default
   into clone url
 - fixed #90 whoosh indexer can index chooses repositories passed in command 
   line
 - extended journal with day aggregates and paging
 - implemented #107 source code lines highlight ranges
 - implemented #93 customizable changelog on combined revision ranges - 
   equivalent of githubs compare view 
 - implemented #108 extended and more powerful LDAP configuration
 - implemented #56 users groups
 - major code rewrites optimized codes for speed and memory usage
 - raw and diff downloads are now in git format
 - setup command checks for write access to given path
 - fixed many issues with international characters and unicode. It uses utf8
   decode with replace to provide less errors even with non utf8 encoded strings
 - #125 added API KEY access to feeds
 - #109 Repository can be created from external Mercurial link (aka. remote 
   repository, and manually updated (via pull) from admin panel
 - beta git support - push/pull server + basic view for git repos
 - added followers page and forks page
 - server side file creation (with binary file upload interface) 
   and edition with commits powered by codemirror 
 - #111 file browser file finder, quick lookup files on whole file tree 
 - added quick login sliding menu into main page
 - changelog uses lazy loading of affected files details, in some scenarios 
   this can improve speed of changelog page dramatically especially for 
   larger repositories.
 - implements #214 added support for downloading subrepos in download menu.
 - Added basic API for direct operations on rhodecode via JSON
 - Implemented advanced hook management
 
 fixes
 +++++
 
 - fixed file browser bug, when switching into given form revision the url was 
   not changing
 - fixed propagation to error controller on simplehg and simplegit middlewares
 - fixed error when trying to make a download on empty repository
 - fixed problem with '[' chars in commit messages in journal
 - fixed #99 Unicode errors, on file node paths with non utf-8 characters
 - journal fork fixes
 - removed issue with space inside renamed repository after deletion
 - fixed strange issue on formencode imports
 - fixed #126 Deleting repository on Windows, rename used incompatible chars. 
 - #150 fixes for errors on repositories mapped in db but corrupted in 
   filesystem
 - fixed problem with ascendant characters in realm #181
 - fixed problem with sqlite file based database connection pool
 - whoosh indexer and code stats share the same dynamic extensions map
 - fixes #188 - relationship delete of repo_to_perm entry on user removal
 - fixes issue #189 Trending source files shows "show more" when no more exist
 - fixes issue #197 Relative paths for pidlocks
 - fixes issue #198 password will require only 3 chars now for login form
 - fixes issue #199 wrong redirection for non admin users after creating a repository
 - fixes issues #202, bad db constraint made impossible to attach same group 
   more than one time. Affects only mysql/postgres
 - fixes #218 os.kill patch for windows was missing sig param
 - improved rendering of dag (they are not trimmed anymore when number of 
   heads exceeds 5)
     
 1.1.8 (**2011-04-12**)
 ----------------------
 
 news
 ++++
 
 - improved windows support
 
 fixes
 +++++
 
 - fixed #140 freeze of python dateutil library, since new version is python2.x
   incompatible
 - setup-app will check for write permission in given path
 - cleaned up license info issue #149
 - fixes for issues #137,#116 and problems with unicode and accented characters.
 - fixes crashes on gravatar, when passed in email as unicode
 - fixed tooltip flickering problems
 - fixed came_from redirection on windows
 - fixed logging modules, and sql formatters
 - windows fixes for os.kill issue #133
 - fixes path splitting for windows issues #148
 - fixed issue #143 wrong import on migration to 1.1.X
 - fixed problems with displaying binary files, thanks to Thomas Waldmann
 - removed name from archive files since it's breaking ui for long repo names
 - fixed issue with archive headers sent to browser, thanks to Thomas Waldmann
 - fixed compatibility for 1024px displays, and larger dpi settings, thanks to 
   Thomas Waldmann
 - fixed issue #166 summary pager was skipping 10 revisions on second page
 
 
 1.1.7 (**2011-03-23**)
 ----------------------
 
 news
 ++++
 
 fixes
 +++++
 
 - fixed (again) #136 installation support for FreeBSD
 
 
 1.1.6 (**2011-03-21**)
 ----------------------
 
 news
 ++++
 
 fixes
 +++++
 
 - fixed #136 installation support for FreeBSD
 - RhodeCode will check for python version during installation
 
 1.1.5 (**2011-03-17**)
 ----------------------
 
 news
 ++++
 
 - basic windows support, by exchanging pybcrypt into sha256 for windows only
   highly inspired by idea of mantis406
 
 fixes
 +++++
 
 - fixed sorting by author in main page
 - fixed crashes with diffs on binary files
 - fixed #131 problem with boolean values for LDAP
 - fixed #122 mysql problems thanks to striker69 
 - fixed problem with errors on calling raw/raw_files/annotate functions 
   with unknown revisions
 - fixed returned rawfiles attachment names with international character
 - cleaned out docs, big thanks to Jason Harris
 
 1.1.4 (**2011-02-19**)
 ----------------------
 
 news
 ++++
 
 fixes
 +++++
 
 - fixed formencode import problem on settings page, that caused server crash
   when that page was accessed as first after server start
 - journal fixes
 - fixed option to access repository just by entering http://server/<repo_name> 
 
 1.1.3 (**2011-02-16**)
 ----------------------
 
 news
 ++++
 
 - implemented #102 allowing the '.' character in username
 - added option to access repository just by entering http://server/<repo_name>
 - celery task ignores result for better performance
 
 fixes
 +++++
 
 - fixed ehlo command and non auth mail servers on smtp_lib. Thanks to 
   apollo13 and Johan Walles
 - small fixes in journal
 - fixed problems with getting setting for celery from .ini files
 - registration, password reset and login boxes share the same title as main 
   application now
 - fixed #113: to high permissions to fork repository
 - fixed problem with '[' chars in commit messages in journal
 - removed issue with space inside renamed repository after deletion
 - db transaction fixes when filesystem repository creation failed
 - fixed #106 relation issues on databases different than sqlite
 - fixed static files paths links to use of url() method
 
 1.1.2 (**2011-01-12**)
 ----------------------
 
 news
 ++++
 
 
 fixes
 +++++
 
 - fixes #98 protection against float division of percentage stats
 - fixed graph bug
 - forced webhelpers version since it was making troubles during installation 
 
 1.1.1 (**2011-01-06**)
 ----------------------
  
 news
 ++++
 
 - added force https option into ini files for easier https usage (no need to
   set server headers with this options)
 - small css updates
 
 fixes
 +++++
 
 - fixed #96 redirect loop on files view on repositories without changesets
 - fixed #97 unicode string passed into server header in special cases (mod_wsgi)
   and server crashed with errors
 - fixed large tooltips problems on main page
 - fixed #92 whoosh indexer is more error proof
 
 1.1.0 (**2010-12-18**)
 ----------------------
 
 news
 ++++
 
 - rewrite of internals for vcs >=0.1.10
 - uses mercurial 1.7 with dotencode disabled for maintaining compatibility 
   with older clients
 - anonymous access, authentication via ldap
 - performance upgrade for cached repos list - each repository has its own 
   cache that's invalidated when needed.
 - performance upgrades on repositories with large amount of commits (20K+)
 - main page quick filter for filtering repositories
 - user dashboards with ability to follow chosen repositories actions
 - sends email to admin on new user registration
 - added cache/statistics reset options into repository settings
 - more detailed action logger (based on hooks) with pushed changesets lists
   and options to disable those hooks from admin panel
 - introduced new enhanced changelog for merges that shows more accurate results
 - new improved and faster code stats (based on pygments lexers mapping tables, 
   showing up to 10 trending sources for each repository. Additionally stats
   can be disabled in repository settings.
 - gui optimizations, fixed application width to 1024px
 - added cut off (for large files/changesets) limit into config files
 - whoosh, celeryd, upgrade moved to paster command
 - other than sqlite database backends can be used
 
 fixes
 +++++
 
 - fixes #61 forked repo was showing only after cache expired
 - fixes #76 no confirmation on user deletes
 - fixes #66 Name field misspelled
 - fixes #72 block user removal when he owns repositories
 - fixes #69 added password confirmation fields
 - fixes #87 RhodeCode crashes occasionally on updating repository owner
 - fixes #82 broken annotations on files with more than 1 blank line at the end
 - a lot of fixes and tweaks for file browser
 - fixed detached session issues
 - fixed when user had no repos he would see all repos listed in my account
 - fixed ui() instance bug when global hgrc settings was loaded for server 
   instance and all hgrc options were merged with our db ui() object
 - numerous small bugfixes
  
 (special thanks for TkSoh for detailed feedback)
 
 
 1.0.2 (**2010-11-12**)
 ----------------------
 
 news
 ++++
 
 - tested under python2.7
 - bumped sqlalchemy and celery versions
 
 fixes
 +++++
 
 - fixed #59 missing graph.js
 - fixed repo_size crash when repository had broken symlinks
 - fixed python2.5 crashes.
 
 
 1.0.1 (**2010-11-10**)
 ----------------------
 
 news
 ++++
 
 - small css updated
 
 fixes
 +++++
 
 - fixed #53 python2.5 incompatible enumerate calls
 - fixed #52 disable mercurial extension for web
 - fixed #51 deleting repositories don't delete it's dependent objects
 
 
 1.0.0 (**2010-11-02**)
 ----------------------
 
 - security bugfix simplehg wasn't checking for permissions on commands
   other than pull or push.
 - fixed doubled messages after push or pull in admin journal
 - templating and css corrections, fixed repo switcher on chrome, updated titles
 - admin menu accessible from options menu on repository view
 - permissions cached queries
 
 1.0.0rc4  (**2010-10-12**)
 --------------------------
 
 - fixed python2.5 missing simplejson imports (thanks to Jens Bäckman)
 - removed cache_manager settings from sqlalchemy meta
 - added sqlalchemy cache settings to ini files
 - validated password length and added second try of failure on paster setup-app
 - fixed setup database destroy prompt even when there was no db
 
 
 1.0.0rc3 (**2010-10-11**)
 -------------------------
 
 - fixed i18n during installation.
 
 1.0.0rc2 (**2010-10-11**)
 -------------------------
 
 - Disabled dirsize in file browser, it's causing nasty bug when dir renames 
   occure. After vcs is fixed it'll be put back again.
 - templating/css rewrites, optimized css.
  No newline at end of file

docs/theme/nature/layout.html

@@ -1,18 +1,18 b''
 {% extends "basic/layout.html" %}
 
 {% block sidebarlogo %}
 <h3>Support RhodeCode development.</h3>
 <div style="text-align:center">
 	<form action="https://www.paypal.com/cgi-bin/webscr" method="post">
 	<input type="hidden" name="cmd" value="_s-xclick">
 	<input type="hidden" name="hosted_button_id" value="8U2LLRPLBKWDU">
 	<input style="border:0px !important" type="image" src="https://www.paypal.com/en_US/i/btn/btn_donate_SM.gif"
 	border="0" name="submit" alt="PayPal - The safer, easier way to pay online!">
 	<img alt="" border="0" src="https://www.paypal.com/en_US/i/scr/pixel.gif" width="1" height="1">
 	</form>
     <div style="padding:5px">
      <a href="http://flattr.com/thing/167489/RhodeCode" target="_blank">
      <img src="http://api.flattr.com/button/flattr-badge-large.png" alt="Flattr this" title="Flattr this" border="0" /></a>
-   </div>  
+   </div>
 </div>
 {% endblock %}}

rhodecode/controllers/admin/repos_groups.py

@@ -1,313 +1,318 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.controllers.admin.repos_groups
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
     Repositories groups controller for RhodeCode
 
     :created_on: Mar 23, 2010
     :author: marcink
     :copyright: (C) 2010-2012 Marcin Kuzminski <[e-mail]>
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import logging
 import traceback
 import formencode
 
 from formencode import htmlfill
 
 from pylons import request, tmpl_context as c, url
 from pylons.controllers.util import redirect
 from pylons.i18n.translation import _
 
 from sqlalchemy.exc import IntegrityError
 
 from rhodecode.lib import helpers as h
 from rhodecode.lib.auth import LoginRequired, HasPermissionAnyDecorator,\
     HasReposGroupPermissionAnyDecorator
 from rhodecode.lib.base import BaseController, render
 from rhodecode.model.db import RepoGroup
 from rhodecode.model.repos_group import ReposGroupModel
 from rhodecode.model.forms import ReposGroupForm
 from rhodecode.model.meta import Session
 from rhodecode.model.repo import RepoModel
 from webob.exc import HTTPInternalServerError
 
 log = logging.getLogger(__name__)
 
 
 class ReposGroupsController(BaseController):
     """REST Controller styled on the Atom Publishing Protocol"""
     # To properly map this controller, ensure your config/routing.py
     # file has a resource setup:
     #     map.resource('repos_group', 'repos_groups')
 
     @LoginRequired()
     def __before__(self):
         super(ReposGroupsController, self).__before__()
 
     def __load_defaults(self):
         c.repo_groups = RepoGroup.groups_choices()
         c.repo_groups_choices = map(lambda k: unicode(k[0]), c.repo_groups)
 
         repo_model = RepoModel()
         c.users_array = repo_model.get_users_js()
         c.users_groups_array = repo_model.get_users_groups_js()
 
     def __load_data(self, group_id):
         """
         Load defaults settings for edit, and update
 
         :param group_id:
         """
         self.__load_defaults()
 
         repo_group = RepoGroup.get(group_id)
 
         data = repo_group.get_dict()
 
         data['group_name'] = repo_group.name
 
         # fill repository users
         for p in repo_group.repo_group_to_perm:
             data.update({'u_perm_%s' % p.user.username:
                              p.permission.permission_name})
 
         # fill repository groups
         for p in repo_group.users_group_to_perm:
             data.update({'g_perm_%s' % p.users_group.users_group_name:
                              p.permission.permission_name})
 
         return data
 
     @HasPermissionAnyDecorator('hg.admin')
     def index(self, format='html'):
         """GET /repos_groups: All items in the collection"""
         # url('repos_groups')
         sk = lambda g: g.parents[0].group_name if g.parents else g.group_name
         c.groups = sorted(RepoGroup.query().all(), key=sk)
         return render('admin/repos_groups/repos_groups_show.html')
 
     @HasPermissionAnyDecorator('hg.admin')
     def create(self):
         """POST /repos_groups: Create a new item"""
         # url('repos_groups')
         self.__load_defaults()
         repos_group_form = ReposGroupForm(available_groups =
                                           c.repo_groups_choices)()
         try:
             form_result = repos_group_form.to_python(dict(request.POST))
             ReposGroupModel().create(
                     group_name=form_result['group_name'],
                     group_description=form_result['group_description'],
                     parent=form_result['group_parent_id']
             )
             Session.commit()
             h.flash(_('created repos group %s') \
                     % form_result['group_name'], category='success')
             #TODO: in futureaction_logger(, '', '', '', self.sa)
         except formencode.Invalid, errors:
 
             return htmlfill.render(
                 render('admin/repos_groups/repos_groups_add.html'),
                 defaults=errors.value,
                 errors=errors.error_dict or {},
                 prefix_error=False,
                 encoding="UTF-8")
         except Exception:
             log.error(traceback.format_exc())
             h.flash(_('error occurred during creation of repos group %s') \
                     % request.POST.get('group_name'), category='error')
 
         return redirect(url('repos_groups'))
 
     @HasPermissionAnyDecorator('hg.admin')
     def new(self, format='html'):
         """GET /repos_groups/new: Form to create a new item"""
         # url('new_repos_group')
         self.__load_defaults()
         return render('admin/repos_groups/repos_groups_add.html')
 
     @HasPermissionAnyDecorator('hg.admin')
     def update(self, id):
         """PUT /repos_groups/id: Update an existing item"""
         # Forms posted to this method should contain a hidden field:
         #    <input type="hidden" name="_method" value="PUT" />
         # Or using helpers:
         #    h.form(url('repos_group', id=ID),
         #           method='put')
         # url('repos_group', id=ID)
 
         self.__load_defaults()
         c.repos_group = RepoGroup.get(id)
 
         repos_group_form = ReposGroupForm(
             edit=True,
             old_data=c.repos_group.get_dict(),
             available_groups=c.repo_groups_choices
         )()
         try:
             form_result = repos_group_form.to_python(dict(request.POST))
             ReposGroupModel().update(id, form_result)
             Session.commit()
             h.flash(_('updated repos group %s') \
                     % form_result['group_name'], category='success')
             #TODO: in futureaction_logger(, '', '', '', self.sa)
         except formencode.Invalid, errors:
 
             return htmlfill.render(
                 render('admin/repos_groups/repos_groups_edit.html'),
                 defaults=errors.value,
                 errors=errors.error_dict or {},
                 prefix_error=False,
                 encoding="UTF-8")
         except Exception:
             log.error(traceback.format_exc())
             h.flash(_('error occurred during update of repos group %s') \
                     % request.POST.get('group_name'), category='error')
 
         return redirect(url('repos_groups'))
 
     @HasPermissionAnyDecorator('hg.admin')
     def delete(self, id):
         """DELETE /repos_groups/id: Delete an existing item"""
         # Forms posted to this method should contain a hidden field:
         #    <input type="hidden" name="_method" value="DELETE" />
         # Or using helpers:
         #    h.form(url('repos_group', id=ID),
         #           method='delete')
         # url('repos_group', id=ID)
 
         gr = RepoGroup.get(id)
         repos = gr.repositories.all()
         if repos:
             h.flash(_('This group contains %s repositores and cannot be '
                       'deleted' % len(repos)),
                     category='error')
             return redirect(url('repos_groups'))
 
         try:
             ReposGroupModel().delete(id)
             Session.commit()
             h.flash(_('removed repos group %s' % gr.group_name), category='success')
             #TODO: in future action_logger(, '', '', '', self.sa)
         except IntegrityError, e:
             if e.message.find('groups_group_parent_id_fkey') != -1:
                 log.error(traceback.format_exc())
                 h.flash(_('Cannot delete this group it still contains '
                           'subgroups'),
                         category='warning')
             else:
                 log.error(traceback.format_exc())
                 h.flash(_('error occurred during deletion of repos '
                           'group %s' % gr.group_name), category='error')
 
         except Exception:
             log.error(traceback.format_exc())
             h.flash(_('error occurred during deletion of repos '
                       'group %s' % gr.group_name), category='error')
 
         return redirect(url('repos_groups'))
 
     @HasReposGroupPermissionAnyDecorator('group.admin')
     def delete_repos_group_user_perm(self, group_name):
         """
         DELETE an existing repositories group permission user
 
         :param group_name:
         """
 
         try:
             ReposGroupModel().revoke_user_permission(
                 repos_group=group_name, user=request.POST['user_id']
             )
             Session.commit()
         except Exception:
             log.error(traceback.format_exc())
             h.flash(_('An error occurred during deletion of group user'),
                     category='error')
             raise HTTPInternalServerError()
 
     @HasReposGroupPermissionAnyDecorator('group.admin')
     def delete_repos_group_users_group_perm(self, group_name):
         """
         DELETE an existing repositories group permission users group
 
         :param group_name:
         """
 
         try:
             ReposGroupModel().revoke_users_group_permission(
                 repos_group=group_name,
                 group_name=request.POST['users_group_id']
             )
             Session.commit()
         except Exception:
             log.error(traceback.format_exc())
             h.flash(_('An error occurred during deletion of group'
                       ' users groups'),
                     category='error')
             raise HTTPInternalServerError()
 
     def show_by_name(self, group_name):
+        """
+        This is a proxy that does a lookup group_name -> id, and shows
+        the group by id view instead
+        """
+        group_name = group_name.rstrip('/')
         id_ = RepoGroup.get_by_group_name(group_name).group_id
         return self.show(id_)
 
     @HasReposGroupPermissionAnyDecorator('group.read', 'group.write',
                                          'group.admin')
     def show(self, id, format='html'):
         """GET /repos_groups/id: Show a specific item"""
         # url('repos_group', id=ID)
 
         c.group = RepoGroup.get(id)
 
         if c.group:
             c.group_repos = c.group.repositories.all()
         else:
             return redirect(url('home'))
 
         #overwrite our cached list with current filter
         gr_filter = c.group_repos
         c.cached_repo_list = self.scm_model.get_repos(all_repos=gr_filter)
 
         c.repos_list = c.cached_repo_list
 
         c.repo_cnt = 0
 
         c.groups = self.sa.query(RepoGroup).order_by(RepoGroup.group_name)\
             .filter(RepoGroup.group_parent_id == id).all()
 
         return render('admin/repos_groups/repos_groups.html')
 
     @HasPermissionAnyDecorator('hg.admin')
     def edit(self, id, format='html'):
         """GET /repos_groups/id/edit: Form to edit an existing item"""
         # url('edit_repos_group', id=ID)
 
         id_ = int(id)
 
         c.repos_group = RepoGroup.get(id_)
         defaults = self.__load_data(id_)
 
         # we need to exclude this group from the group list for editing
         c.repo_groups = filter(lambda x: x[0] != id_, c.repo_groups)
 
         return htmlfill.render(
             render('admin/repos_groups/repos_groups_edit.html'),
             defaults=defaults,
             encoding="UTF-8",
             force_defaults=False
         )

rhodecode/controllers/admin/users_groups.py

@@ -1,226 +1,227 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.controllers.admin.users_groups
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
     Users Groups crud controller for pylons
 
     :created_on: Jan 25, 2011
     :author: marcink
     :copyright: (C) 2010-2012 Marcin Kuzminski <[e-mail]>
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import logging
 import traceback
 import formencode
 
 from formencode import htmlfill
 from pylons import request, session, tmpl_context as c, url, config
 from pylons.controllers.util import abort, redirect
 from pylons.i18n.translation import _
 
 from rhodecode.lib.exceptions import UsersGroupsAssignedException
 from rhodecode.lib import helpers as h, safe_unicode
 from rhodecode.lib.auth import LoginRequired, HasPermissionAllDecorator
 from rhodecode.lib.base import BaseController, render
 
 from rhodecode.model.users_group import UsersGroupModel
 
 from rhodecode.model.db import User, UsersGroup, Permission, UsersGroupToPerm
 from rhodecode.model.forms import UsersGroupForm
 from rhodecode.model.meta import Session
 
 log = logging.getLogger(__name__)
 
 
 class UsersGroupsController(BaseController):
     """REST Controller styled on the Atom Publishing Protocol"""
     # To properly map this controller, ensure your config/routing.py
     # file has a resource setup:
     #     map.resource('users_group', 'users_groups')
 
     @LoginRequired()
     @HasPermissionAllDecorator('hg.admin')
     def __before__(self):
         c.admin_user = session.get('admin_user')
         c.admin_username = session.get('admin_username')
         super(UsersGroupsController, self).__before__()
         c.available_permissions = config['available_permissions']
 
     def index(self, format='html'):
         """GET /users_groups: All items in the collection"""
         # url('users_groups')
         c.users_groups_list = self.sa.query(UsersGroup).all()
         return render('admin/users_groups/users_groups.html')
 
     def create(self):
         """POST /users_groups: Create a new item"""
         # url('users_groups')
 
         users_group_form = UsersGroupForm()()
         try:
             form_result = users_group_form.to_python(dict(request.POST))
             UsersGroupModel().create(name=form_result['users_group_name'],
                                      active=form_result['users_group_active'])
             h.flash(_('created users group %s') \
                     % form_result['users_group_name'], category='success')
             #action_logger(self.rhodecode_user, 'new_user', '', '', self.sa)
             Session.commit()
         except formencode.Invalid, errors:
             return htmlfill.render(
                 render('admin/users_groups/users_group_add.html'),
                 defaults=errors.value,
                 errors=errors.error_dict or {},
                 prefix_error=False,
                 encoding="UTF-8")
         except Exception:
             log.error(traceback.format_exc())
             h.flash(_('error occurred during creation of users group %s') \
                     % request.POST.get('users_group_name'), category='error')
 
         return redirect(url('users_groups'))
 
     def new(self, format='html'):
         """GET /users_groups/new: Form to create a new item"""
         # url('new_users_group')
         return render('admin/users_groups/users_group_add.html')
 
     def update(self, id):
         """PUT /users_groups/id: Update an existing item"""
         # Forms posted to this method should contain a hidden field:
         #    <input type="hidden" name="_method" value="PUT" />
         # Or using helpers:
         #    h.form(url('users_group', id=ID),
         #           method='put')
         # url('users_group', id=ID)
 
         c.users_group = UsersGroup.get(id)
         c.group_members_obj = [x.user for x in c.users_group.members]
         c.group_members = [(x.user_id, x.username) for x in
                            c.group_members_obj]
 
         c.available_members = [(x.user_id, x.username) for x in
                                self.sa.query(User).all()]
 
         available_members = [safe_unicode(x[0]) for x in c.available_members]
 
         users_group_form = UsersGroupForm(edit=True,
                                           old_data=c.users_group.get_dict(),
                                           available_members=available_members)()
 
         try:
             form_result = users_group_form.to_python(request.POST)
             UsersGroupModel().update(c.users_group, form_result)
             h.flash(_('updated users group %s') \
                         % form_result['users_group_name'],
                     category='success')
             #action_logger(self.rhodecode_user, 'new_user', '', '', self.sa)
             Session.commit()
         except formencode.Invalid, errors:
             e = errors.error_dict or {}
 
             perm = Permission.get_by_key('hg.create.repository')
             e.update({'create_repo_perm':
                          UsersGroupModel().has_perm(id, perm)})
 
             return htmlfill.render(
                 render('admin/users_groups/users_group_edit.html'),
                 defaults=errors.value,
                 errors=e,
                 prefix_error=False,
                 encoding="UTF-8")
         except Exception:
             log.error(traceback.format_exc())
             h.flash(_('error occurred during update of users group %s') \
                     % request.POST.get('users_group_name'), category='error')
 
         return redirect(url('users_groups'))
 
     def delete(self, id):
         """DELETE /users_groups/id: Delete an existing item"""
         # Forms posted to this method should contain a hidden field:
         #    <input type="hidden" name="_method" value="DELETE" />
         # Or using helpers:
         #    h.form(url('users_group', id=ID),
         #           method='delete')
         # url('users_group', id=ID)
 
         try:
             UsersGroupModel().delete(id)
+            Session.commit()
             h.flash(_('successfully deleted users group'), category='success')
-            Session.commit()
         except UsersGroupsAssignedException, e:
             h.flash(e, category='error')
         except Exception:
+            log.error(traceback.format_exc())
             h.flash(_('An error occurred during deletion of users group'),
                     category='error')
         return redirect(url('users_groups'))
 
     def show(self, id, format='html'):
         """GET /users_groups/id: Show a specific item"""
         # url('users_group', id=ID)
 
     def edit(self, id, format='html'):
         """GET /users_groups/id/edit: Form to edit an existing item"""
         # url('edit_users_group', id=ID)
 
         c.users_group = self.sa.query(UsersGroup).get(id)
         if not c.users_group:
             return redirect(url('users_groups'))
 
         c.users_group.permissions = {}
         c.group_members_obj = [x.user for x in c.users_group.members]
         c.group_members = [(x.user_id, x.username) for x in
                            c.group_members_obj]
         c.available_members = [(x.user_id, x.username) for x in
                                self.sa.query(User).all()]
         defaults = c.users_group.get_dict()
         perm = Permission.get_by_key('hg.create.repository')
         defaults.update({'create_repo_perm':
                          UsersGroupModel().has_perm(c.users_group, perm)})
         return htmlfill.render(
             render('admin/users_groups/users_group_edit.html'),
             defaults=defaults,
             encoding="UTF-8",
             force_defaults=False
         )
 
     def update_perm(self, id):
         """PUT /users_perm/id: Update an existing item"""
         # url('users_group_perm', id=ID, method='put')
 
         grant_perm = request.POST.get('create_repo_perm', False)
 
         if grant_perm:
             perm = Permission.get_by_key('hg.create.none')
             UsersGroupModel().revoke_perm(id, perm)
 
             perm = Permission.get_by_key('hg.create.repository')
             UsersGroupModel().grant_perm(id, perm)
             h.flash(_("Granted 'repository create' permission to user"),
                     category='success')
 
             Session.commit()
         else:
             perm = Permission.get_by_key('hg.create.repository')
             UsersGroupModel().revoke_perm(id, perm)
 
             perm = Permission.get_by_key('hg.create.none')
             UsersGroupModel().grant_perm(id, perm)
             h.flash(_("Revoked 'repository create' permission to user"),
                     category='success')
             Session.commit()
         return redirect(url('edit_users_group', id=id))

rhodecode/lib/__init__.py

@@ -1,465 +1,465 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.lib.__init__
     ~~~~~~~~~~~~~~~~~~~~~~~
 
     Some simple helper functions
 
     :created_on: Jan 5, 2011
     :author: marcink
     :copyright: (C) 2011-2012 Marcin Kuzminski <[e-mail]>
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import os
 import re
 from rhodecode.lib.vcs.utils.lazy import LazyProperty
 
 
 def __get_lem():
     from pygments import lexers
     from string import lower
     from collections import defaultdict
 
     d = defaultdict(lambda: [])
 
     def __clean(s):
         s = s.lstrip('*')
         s = s.lstrip('.')
 
         if s.find('[') != -1:
             exts = []
             start, stop = s.find('['), s.find(']')
 
             for suffix in s[start + 1:stop]:
                 exts.append(s[:s.find('[')] + suffix)
             return map(lower, exts)
         else:
             return map(lower, [s])
 
     for lx, t in sorted(lexers.LEXERS.items()):
         m = map(__clean, t[-2])
         if m:
             m = reduce(lambda x, y: x + y, m)
             for ext in m:
                 desc = lx.replace('Lexer', '')
                 d[ext].append(desc)
 
     return dict(d)
 
 # language map is also used by whoosh indexer, which for those specified
 # extensions will index it's content
 LANGUAGES_EXTENSIONS_MAP = __get_lem()
 
 # Additional mappings that are not present in the pygments lexers
 # NOTE: that this will overide any mappings in LANGUAGES_EXTENSIONS_MAP
 ADDITIONAL_MAPPINGS = {'xaml': 'XAML'}
 
 LANGUAGES_EXTENSIONS_MAP.update(ADDITIONAL_MAPPINGS)
 
 # list of readme files to search in file tree and display in summary
 # attached weights defines the search  order lower is first
 ALL_READMES = [
     ('readme', 0), ('README', 0), ('Readme', 0),
     ('doc/readme', 1), ('doc/README', 1), ('doc/Readme', 1),
     ('Docs/readme', 2), ('Docs/README', 2), ('Docs/Readme', 2),
     ('DOCS/readme', 2), ('DOCS/README', 2), ('DOCS/Readme', 2),
     ('docs/readme', 2), ('docs/README', 2), ('docs/Readme', 2),
 ]
 
 # extension together with weights to search lower is first
 RST_EXTS = [
     ('', 0), ('.rst', 1), ('.rest', 1),
     ('.RST', 2), ('.REST', 2),
     ('.txt', 3), ('.TXT', 3)
 ]
 
 MARKDOWN_EXTS = [
     ('.md', 1), ('.MD', 1),
     ('.mkdn', 2), ('.MKDN', 2),
     ('.mdown', 3), ('.MDOWN', 3),
     ('.markdown', 4), ('.MARKDOWN', 4)
 ]
 
 PLAIN_EXTS = [('.text', 2), ('.TEXT', 2)]
 
 ALL_EXTS = MARKDOWN_EXTS + RST_EXTS + PLAIN_EXTS
 
 
 def str2bool(_str):
     """
     returs True/False value from given string, it tries to translate the
     string into boolean
 
     :param _str: string value to translate into boolean
     :rtype: boolean
     :returns: boolean from given string
     """
     if _str is None:
         return False
     if _str in (True, False):
         return _str
     _str = str(_str).strip().lower()
     return _str in ('t', 'true', 'y', 'yes', 'on', '1')
 
 
 def convert_line_endings(line, mode):
     """
     Converts a given line  "line end" accordingly to given mode
 
     Available modes are::
         0 - Unix
         1 - Mac
         2 - DOS
 
     :param line: given line to convert
     :param mode: mode to convert to
     :rtype: str
     :return: converted line according to mode
     """
     from string import replace
 
     if mode == 0:
             line = replace(line, '\r\n', '\n')
             line = replace(line, '\r', '\n')
     elif mode == 1:
             line = replace(line, '\r\n', '\r')
             line = replace(line, '\n', '\r')
     elif mode == 2:
             line = re.sub("\r(?!\n)|(?<!\r)\n", "\r\n", line)
     return line
 
 
 def detect_mode(line, default):
     """
     Detects line break for given line, if line break couldn't be found
     given default value is returned
 
     :param line: str line
     :param default: default
     :rtype: int
     :return: value of line end on of 0 - Unix, 1 - Mac, 2 - DOS
     """
     if line.endswith('\r\n'):
         return 2
     elif line.endswith('\n'):
         return 0
     elif line.endswith('\r'):
         return 1
     else:
         return default
 
 
 def generate_api_key(username, salt=None):
     """
     Generates unique API key for given username, if salt is not given
     it'll be generated from some random string
 
     :param username: username as string
     :param salt: salt to hash generate KEY
     :rtype: str
     :returns: sha1 hash from username+salt
     """
     from tempfile import _RandomNameSequence
     import hashlib
 
     if salt is None:
         salt = _RandomNameSequence().next()
 
     return hashlib.sha1(username + salt).hexdigest()
 
 
 def safe_unicode(str_, from_encoding=None):
     """
     safe unicode function. Does few trick to turn str_ into unicode
 
     In case of UnicodeDecode error we try to return it with encoding detected
     by chardet library if it fails fallback to unicode with errors replaced
 
     :param str_: string to decode
     :rtype: unicode
     :returns: unicode object
     """
     if isinstance(str_, unicode):
         return str_
 
     if not from_encoding:
         import rhodecode
         DEFAULT_ENCODING = rhodecode.CONFIG.get('default_encoding','utf8')
         from_encoding = DEFAULT_ENCODING
 
     try:
         return unicode(str_)
     except UnicodeDecodeError:
         pass
 
     try:
         return unicode(str_, from_encoding)
     except UnicodeDecodeError:
         pass
 
     try:
         import chardet
         encoding = chardet.detect(str_)['encoding']
         if encoding is None:
             raise Exception()
         return str_.decode(encoding)
     except (ImportError, UnicodeDecodeError, Exception):
         return unicode(str_, from_encoding, 'replace')
 
 
 def safe_str(unicode_, to_encoding=None):
     """
     safe str function. Does few trick to turn unicode_ into string
 
     In case of UnicodeEncodeError we try to return it with encoding detected
     by chardet library if it fails fallback to string with errors replaced
 
     :param unicode_: unicode to encode
     :rtype: str
     :returns: str object
     """
-    
+
     # if it's not basestr cast to str
     if not isinstance(unicode_, basestring):
         return str(unicode_)
 
     if isinstance(unicode_, str):
         return unicode_
 
     if not to_encoding:
         import rhodecode
         DEFAULT_ENCODING = rhodecode.CONFIG.get('default_encoding','utf8')
         to_encoding = DEFAULT_ENCODING
 
     try:
         return unicode_.encode(to_encoding)
     except UnicodeEncodeError:
         pass
 
     try:
         import chardet
         encoding = chardet.detect(unicode_)['encoding']
         print encoding
         if encoding is None:
             raise UnicodeEncodeError()
 
         return unicode_.encode(encoding)
     except (ImportError, UnicodeEncodeError):
         return unicode_.encode(to_encoding, 'replace')
 
     return safe_str
 
 
 def engine_from_config(configuration, prefix='sqlalchemy.', **kwargs):
     """
     Custom engine_from_config functions that makes sure we use NullPool for
     file based sqlite databases. This prevents errors on sqlite. This only
     applies to sqlalchemy versions < 0.7.0
 
     """
     import sqlalchemy
     from sqlalchemy import engine_from_config as efc
     import logging
 
     if int(sqlalchemy.__version__.split('.')[1]) < 7:
 
         # This solution should work for sqlalchemy < 0.7.0, and should use
         # proxy=TimerProxy() for execution time profiling
 
         from sqlalchemy.pool import NullPool
         url = configuration[prefix + 'url']
 
         if url.startswith('sqlite'):
             kwargs.update({'poolclass': NullPool})
         return efc(configuration, prefix, **kwargs)
     else:
         import time
         from sqlalchemy import event
         from sqlalchemy.engine import Engine
 
         log = logging.getLogger('sqlalchemy.engine')
         BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE = xrange(30, 38)
         engine = efc(configuration, prefix, **kwargs)
 
         def color_sql(sql):
             COLOR_SEQ = "\033[1;%dm"
             COLOR_SQL = YELLOW
             normal = '\x1b[0m'
             return ''.join([COLOR_SEQ % COLOR_SQL, sql, normal])
 
         if configuration['debug']:
             #attach events only for debug configuration
 
             def before_cursor_execute(conn, cursor, statement,
                                     parameters, context, executemany):
                 context._query_start_time = time.time()
                 log.info(color_sql(">>>>> STARTING QUERY >>>>>"))
 
 
             def after_cursor_execute(conn, cursor, statement,
                                     parameters, context, executemany):
                 total = time.time() - context._query_start_time
                 log.info(color_sql("<<<<< TOTAL TIME: %f <<<<<" % total))
 
             event.listen(engine, "before_cursor_execute",
                          before_cursor_execute)
             event.listen(engine, "after_cursor_execute",
                          after_cursor_execute)
 
     return engine
 
 
 def age(curdate):
     """
     turns a datetime into an age string.
 
     :param curdate: datetime object
     :rtype: unicode
     :returns: unicode words describing age
     """
 
     from datetime import datetime
     from webhelpers.date import time_ago_in_words
 
     _ = lambda s: s
 
     if not curdate:
         return ''
 
     agescales = [(_(u"year"), 3600 * 24 * 365),
                  (_(u"month"), 3600 * 24 * 30),
                  (_(u"day"), 3600 * 24),
                  (_(u"hour"), 3600),
                  (_(u"minute"), 60),
                  (_(u"second"), 1), ]
 
     age = datetime.now() - curdate
     age_seconds = (age.days * agescales[2][1]) + age.seconds
     pos = 1
     for scale in agescales:
         if scale[1] <= age_seconds:
             if pos == 6:
                 pos = 5
             return '%s %s' % (time_ago_in_words(curdate,
                                                 agescales[pos][0]), _('ago'))
         pos += 1
 
     return _(u'just now')
 
 
 def uri_filter(uri):
     """
     Removes user:password from given url string
 
     :param uri:
     :rtype: unicode
     :returns: filtered list of strings
     """
     if not uri:
         return ''
 
     proto = ''
 
     for pat in ('https://', 'http://'):
         if uri.startswith(pat):
             uri = uri[len(pat):]
             proto = pat
             break
 
     # remove passwords and username
     uri = uri[uri.find('@') + 1:]
 
     # get the port
     cred_pos = uri.find(':')
     if cred_pos == -1:
         host, port = uri, None
     else:
         host, port = uri[:cred_pos], uri[cred_pos + 1:]
 
     return filter(None, [proto, host, port])
 
 
 def credentials_filter(uri):
     """
     Returns a url with removed credentials
 
     :param uri:
     """
 
     uri = uri_filter(uri)
     #check if we have port
     if len(uri) > 2 and uri[2]:
         uri[2] = ':' + uri[2]
 
     return ''.join(uri)
 
 
 def get_changeset_safe(repo, rev):
     """
     Safe version of get_changeset if this changeset doesn't exists for a
     repo it returns a Dummy one instead
 
     :param repo:
     :param rev:
     """
     from rhodecode.lib.vcs.backends.base import BaseRepository
     from rhodecode.lib.vcs.exceptions import RepositoryError
     if not isinstance(repo, BaseRepository):
         raise Exception('You must pass an Repository '
                         'object as first argument got %s', type(repo))
 
     try:
         cs = repo.get_changeset(rev)
     except RepositoryError:
         from rhodecode.lib.utils import EmptyChangeset
         cs = EmptyChangeset(requested_revision=rev)
     return cs
 
 
 def get_current_revision(quiet=False):
     """
     Returns tuple of (number, id) from repository containing this package
     or None if repository could not be found.
 
     :param quiet: prints error for fetching revision if True
     """
 
     try:
         from rhodecode.lib.vcs import get_repo
         from rhodecode.lib.vcs.utils.helpers import get_scm
         repopath = os.path.join(os.path.dirname(__file__), '..', '..')
         scm = get_scm(repopath)[0]
         repo = get_repo(path=repopath, alias=scm)
         tip = repo.get_changeset()
         return (tip.revision, tip.short_id)
     except Exception, err:
         if not quiet:
             print ("Cannot retrieve rhodecode's revision. Original error "
                    "was: %s" % err)
         return None
 
 
 def extract_mentioned_users(s):
     """
     Returns unique usernames from given string s that have @mention
 
     :param s: string to get mentions
     """
     usrs = {}
     for username in re.findall(r'(?:^@|\s@)(\w+)', s):
         usrs[username] = username
 
     return sorted(usrs.keys())

rhodecode/lib/caching_query.py

@@ -1,299 +1,301 b''
 """caching_query.py
 
 Represent persistence structures which allow the usage of
 Beaker caching with SQLAlchemy.
 
 The three new concepts introduced here are:
 
  * CachingQuery - a Query subclass that caches and
    retrieves results in/from Beaker.
  * FromCache - a query option that establishes caching
    parameters on a Query
  * RelationshipCache - a variant of FromCache which is specific
    to a query invoked during a lazy load.
  * _params_from_query - extracts value parameters from
    a Query.
 
 The rest of what's here are standard SQLAlchemy and
 Beaker constructs.
 
 """
 import beaker
 from beaker.exceptions import BeakerException
 
 from sqlalchemy.orm.interfaces import MapperOption
 from sqlalchemy.orm.query import Query
 from sqlalchemy.sql import visitors
+from rhodecode.lib import safe_str
 
 
 class CachingQuery(Query):
     """A Query subclass which optionally loads full results from a Beaker
     cache region.
 
     The CachingQuery stores additional state that allows it to consult
     a Beaker cache before accessing the database:
 
     * A "region", which is a cache region argument passed to a
       Beaker CacheManager, specifies a particular cache configuration
       (including backend implementation, expiration times, etc.)
     * A "namespace", which is a qualifying name that identifies a
       group of keys within the cache.  A query that filters on a name
       might use the name "by_name", a query that filters on a date range
       to a joined table might use the name "related_date_range".
 
     When the above state is present, a Beaker cache is retrieved.
 
     The "namespace" name is first concatenated with
     a string composed of the individual entities and columns the Query
     requests, i.e. such as ``Query(User.id, User.name)``.
 
     The Beaker cache is then loaded from the cache manager based
     on the region and composed namespace.  The key within the cache
     itself is then constructed against the bind parameters specified
     by this query, which are usually literals defined in the
     WHERE clause.
 
     The FromCache and RelationshipCache mapper options below represent
     the "public" method of configuring this state upon the CachingQuery.
 
     """
 
     def __init__(self, manager, *args, **kw):
         self.cache_manager = manager
         Query.__init__(self, *args, **kw)
 
     def __iter__(self):
         """override __iter__ to pull results from Beaker
            if particular attributes have been configured.
 
            Note that this approach does *not* detach the loaded objects from
            the current session. If the cache backend is an in-process cache
            (like "memory") and lives beyond the scope of the current session's
            transaction, those objects may be expired. The method here can be
            modified to first expunge() each loaded item from the current
            session before returning the list of items, so that the items
            in the cache are not the same ones in the current Session.
 
         """
         if hasattr(self, '_cache_parameters'):
             return self.get_value(createfunc=lambda:
                                   list(Query.__iter__(self)))
         else:
             return Query.__iter__(self)
 
     def invalidate(self):
         """Invalidate the value represented by this Query."""
 
         cache, cache_key = _get_cache_parameters(self)
         cache.remove(cache_key)
 
     def get_value(self, merge=True, createfunc=None):
         """Return the value from the cache for this query.
 
         Raise KeyError if no value present and no
         createfunc specified.
 
         """
         cache, cache_key = _get_cache_parameters(self)
         ret = cache.get_value(cache_key, createfunc=createfunc)
         if merge:
             ret = self.merge_result(ret, load=False)
         return ret
 
     def set_value(self, value):
         """Set the value in the cache for this query."""
 
         cache, cache_key = _get_cache_parameters(self)
         cache.put(cache_key, value)
 
 
 def query_callable(manager, query_cls=CachingQuery):
     def query(*arg, **kw):
         return query_cls(manager, *arg, **kw)
     return query
 
 
 def get_cache_region(name, region):
     if region not in beaker.cache.cache_regions:
         raise BeakerException('Cache region `%s` not configured '
             'Check if proper cache settings are in the .ini files' % region)
     kw = beaker.cache.cache_regions[region]
     return beaker.cache.Cache._get_cache(name, kw)
 
 
 def _get_cache_parameters(query):
     """For a query with cache_region and cache_namespace configured,
     return the correspoinding Cache instance and cache key, based
     on this query's current criterion and parameter values.
 
     """
     if not hasattr(query, '_cache_parameters'):
         raise ValueError("This Query does not have caching "
                          "parameters configured.")
 
     region, namespace, cache_key = query._cache_parameters
 
     namespace = _namespace_from_query(namespace, query)
 
     if cache_key is None:
         # cache key - the value arguments from this query's parameters.
-        args = [str(x) for x in _params_from_query(query)]
-        args.extend(filter(lambda k:k not in ['None', None, u'None'],
+        args = [safe_str(x) for x in _params_from_query(query)]
+        args.extend(filter(lambda k: k not in ['None', None, u'None'],
                            [str(query._limit), str(query._offset)]))
+
         cache_key = " ".join(args)
 
     if cache_key is None:
         raise Exception('Cache key cannot be None')
 
     # get cache
     #cache = query.cache_manager.get_cache_region(namespace, region)
     cache = get_cache_region(namespace, region)
     # optional - hash the cache_key too for consistent length
     # import uuid
     # cache_key= str(uuid.uuid5(uuid.NAMESPACE_DNS, cache_key))
 
     return cache, cache_key
 
 
 def _namespace_from_query(namespace, query):
     # cache namespace - the token handed in by the
     # option + class we're querying against
     namespace = " ".join([namespace] + [str(x) for x in query._entities])
 
     # memcached wants this
     namespace = namespace.replace(' ', '_')
 
     return namespace
 
 
 def _set_cache_parameters(query, region, namespace, cache_key):
 
     if hasattr(query, '_cache_parameters'):
         region, namespace, cache_key = query._cache_parameters
         raise ValueError("This query is already configured "
                         "for region %r namespace %r" %
                         (region, namespace)
                     )
     query._cache_parameters = region, namespace, cache_key
 
 
 class FromCache(MapperOption):
     """Specifies that a Query should load results from a cache."""
 
     propagate_to_loaders = False
 
     def __init__(self, region, namespace, cache_key=None):
         """Construct a new FromCache.
 
         :param region: the cache region.  Should be a
         region configured in the Beaker CacheManager.
 
         :param namespace: the cache namespace.  Should
         be a name uniquely describing the target Query's
         lexical structure.
 
         :param cache_key: optional.  A string cache key
         that will serve as the key to the query.   Use this
         if your query has a huge amount of parameters (such
         as when using in_()) which correspond more simply to
         some other identifier.
 
         """
         self.region = region
         self.namespace = namespace
         self.cache_key = cache_key
 
     def process_query(self, query):
         """Process a Query during normal loading operation."""
 
         _set_cache_parameters(query, self.region, self.namespace,
                               self.cache_key)
 
 
 class RelationshipCache(MapperOption):
     """Specifies that a Query as called within a "lazy load"
        should load results from a cache."""
 
     propagate_to_loaders = True
 
     def __init__(self, region, namespace, attribute):
         """Construct a new RelationshipCache.
 
         :param region: the cache region.  Should be a
         region configured in the Beaker CacheManager.
 
         :param namespace: the cache namespace.  Should
         be a name uniquely describing the target Query's
         lexical structure.
 
         :param attribute: A Class.attribute which
         indicates a particular class relationship() whose
         lazy loader should be pulled from the cache.
 
         """
         self.region = region
         self.namespace = namespace
         self._relationship_options = {
             (attribute.property.parent.class_, attribute.property.key): self
         }
 
     def process_query_conditionally(self, query):
         """Process a Query that is used within a lazy loader.
 
         (the process_query_conditionally() method is a SQLAlchemy
         hook invoked only within lazyload.)
 
         """
         if query._current_path:
             mapper, key = query._current_path[-2:]
 
             for cls in mapper.class_.__mro__:
                 if (cls, key) in self._relationship_options:
                     relationship_option = \
                         self._relationship_options[(cls, key)]
                     _set_cache_parameters(
                             query,
                             relationship_option.region,
                             relationship_option.namespace,
                             None)
 
     def and_(self, option):
         """Chain another RelationshipCache option to this one.
 
         While many RelationshipCache objects can be specified on a single
         Query separately, chaining them together allows for a more efficient
         lookup during load.
 
         """
         self._relationship_options.update(option._relationship_options)
         return self
 
 
 def _params_from_query(query):
     """Pull the bind parameter values from a query.
 
     This takes into account any scalar attribute bindparam set up.
 
     E.g. params_from_query(query.filter(Cls.foo==5).filter(Cls.bar==7)))
     would return [5, 7].
 
     """
     v = []
     def visit_bindparam(bind):
 
         if bind.key in query._params:
             value = query._params[bind.key]
         elif bind.callable:
             # lazyloader may dig a callable in here, intended
             # to late-evaluate params after autoflush is called.
             # convert to a scalar value.
             value = bind.callable()
         else:
             value = bind.value
 
         v.append(value)
     if query._criterion is not None:
         visitors.traverse(query._criterion, {}, {'bindparam':visit_bindparam})
     for f in query._from_obj:
         visitors.traverse(f, {}, {'bindparam':visit_bindparam})
     return v

rhodecode/lib/middleware/https_fixup.py

@@ -1,54 +1,62 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.lib.middleware.https_fixup
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
     middleware to handle https correctly
 
     :created_on: May 23, 2010
     :author: marcink
     :copyright: (C) 2010-2012 Marcin Kuzminski <[e-mail]>
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 from rhodecode.lib import str2bool
 
 
 class HttpsFixup(object):
 
     def __init__(self, app, config):
         self.application = app
         self.config = config
 
     def __call__(self, environ, start_response):
         self.__fixup(environ)
         return self.application(environ, start_response)
 
     def __fixup(self, environ):
         """
         Function to fixup the environ as needed. In order to use this
         middleware you should set this header inside your
         proxy ie. nginx, apache etc.
         """
-        proto = environ.get('HTTP_X_URL_SCHEME')
 
         if str2bool(self.config.get('force_https')):
             proto = 'https'
-
+        else:
+            if 'HTTP_X_URL_SCHEME' in environ:
+                proto = environ.get('HTTP_X_URL_SCHEME')
+            elif 'HTTP_X_FORWARDED_SCHEME' in environ:
+                proto = environ.get('HTTP_X_FORWARDED_SCHEME')
+            elif 'HTTP_X_FORWARDED_PROTO' in environ:
+                proto = environ.get('HTTP_X_FORWARDED_PROTO')
+            else:
+                proto = 'http'
         if proto == 'https':
             environ['wsgi.url_scheme'] = proto
         else:
             environ['wsgi.url_scheme'] = 'http'
+
         return None

rhodecode/lib/middleware/simplegit.py

@@ -1,248 +1,247 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.lib.middleware.simplegit
     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
     SimpleGit middleware for handling git protocol request (push/clone etc.)
     It's implemented with basic auth function
 
     :created_on: Apr 28, 2010
     :author: marcink
     :copyright: (C) 2010-2012 Marcin Kuzminski <[e-mail]>
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import os
+import re
 import logging
 import traceback
 
 from dulwich import server as dulserver
 
 
 class SimpleGitUploadPackHandler(dulserver.UploadPackHandler):
 
     def handle(self):
         write = lambda x: self.proto.write_sideband(1, x)
 
         graph_walker = dulserver.ProtocolGraphWalker(self,
                                                      self.repo.object_store,
                                                      self.repo.get_peeled)
         objects_iter = self.repo.fetch_objects(
           graph_walker.determine_wants, graph_walker, self.progress,
           get_tagged=self.get_tagged)
 
         # Do they want any objects?
         if objects_iter is None or len(objects_iter) == 0:
             return
 
         self.progress("counting objects: %d, done.\n" % len(objects_iter))
         dulserver.write_pack_objects(dulserver.ProtocolFile(None, write),
                                   objects_iter, len(objects_iter))
         messages = []
         messages.append('thank you for using rhodecode')
 
         for msg in messages:
             self.progress(msg + "\n")
         # we are done
         self.proto.write("0000")
 
 dulserver.DEFAULT_HANDLERS = {
   'git-upload-pack': SimpleGitUploadPackHandler,
   'git-receive-pack': dulserver.ReceivePackHandler,
 }
 
 from dulwich.repo import Repo
 from dulwich.web import HTTPGitApplication
 
 from paste.httpheaders import REMOTE_USER, AUTH_TYPE
 
 from rhodecode.lib import safe_str
 from rhodecode.lib.base import BaseVCSController
 from rhodecode.lib.auth import get_container_username
 from rhodecode.lib.utils import is_valid_repo
 from rhodecode.model.db import User
 
 from webob.exc import HTTPNotFound, HTTPForbidden, HTTPInternalServerError
 
 log = logging.getLogger(__name__)
 
 
-def is_git(environ):
-    """Returns True if request's target is git server.
-    ``HTTP_USER_AGENT`` would then have git client version given.
+GIT_PROTO_PAT = re.compile(r'^/(.+)/(info/refs|git-upload-pack|git-receive-pack)')
+
 
-    :param environ:
-    """
-    http_user_agent = environ.get('HTTP_USER_AGENT')
-    if http_user_agent and http_user_agent.startswith('git'):
-        return True
-    return False
+def is_git(environ):
+    path_info = environ['PATH_INFO']
+    isgit_path = GIT_PROTO_PAT.match(path_info)
+    log.debug('is a git path %s pathinfo : %s' % (isgit_path, path_info))
+    return isgit_path
 
 
 class SimpleGit(BaseVCSController):
 
     def _handle_request(self, environ, start_response):
+
         if not is_git(environ):
             return self.application(environ, start_response)
 
         proxy_key = 'HTTP_X_REAL_IP'
         def_key = 'REMOTE_ADDR'
         ipaddr = environ.get(proxy_key, environ.get(def_key, '0.0.0.0'))
         username = None
         # skip passing error to error controller
         environ['pylons.status_code_redirect'] = True
 
         #======================================================================
         # EXTRACT REPOSITORY NAME FROM ENV
         #======================================================================
         try:
             repo_name = self.__get_repository(environ)
             log.debug('Extracted repo name is %s' % repo_name)
         except:
             return HTTPInternalServerError()(environ, start_response)
 
         #======================================================================
         # GET ACTION PULL or PUSH
         #======================================================================
         action = self.__get_action(environ)
 
         #======================================================================
         # CHECK ANONYMOUS PERMISSION
         #======================================================================
         if action in ['pull', 'push']:
             anonymous_user = self.__get_user('default')
             username = anonymous_user.username
             anonymous_perm = self._check_permission(action, anonymous_user,
                                                     repo_name)
 
             if anonymous_perm is not True or anonymous_user.active is False:
                 if anonymous_perm is not True:
                     log.debug('Not enough credentials to access this '
                               'repository as anonymous user')
                 if anonymous_user.active is False:
                     log.debug('Anonymous access is disabled, running '
                               'authentication')
                 #==============================================================
                 # DEFAULT PERM FAILED OR ANONYMOUS ACCESS IS DISABLED SO WE
                 # NEED TO AUTHENTICATE AND ASK FOR AUTH USER PERMISSIONS
                 #==============================================================
 
                 # Attempting to retrieve username from the container
                 username = get_container_username(environ, self.config)
 
                 # If not authenticated by the container, running basic auth
                 if not username:
                     self.authenticate.realm = \
                         safe_str(self.config['rhodecode_realm'])
                     result = self.authenticate(environ)
                     if isinstance(result, str):
                         AUTH_TYPE.update(environ, 'basic')
                         REMOTE_USER.update(environ, result)
                         username = result
                     else:
                         return result.wsgi_application(environ, start_response)
 
                 #==============================================================
                 # CHECK PERMISSIONS FOR THIS REQUEST USING GIVEN USERNAME
                 #==============================================================
                 if action in ['pull', 'push']:
                     try:
                         user = self.__get_user(username)
                         if user is None or not user.active:
                             return HTTPForbidden()(environ, start_response)
                         username = user.username
                     except:
                         log.error(traceback.format_exc())
                         return HTTPInternalServerError()(environ,
                                                          start_response)
 
                     #check permissions for this repository
                     perm = self._check_permission(action, user,
                                                    repo_name)
                     if perm is not True:
                         return HTTPForbidden()(environ, start_response)
 
         #===================================================================
         # GIT REQUEST HANDLING
         #===================================================================
 
         repo_path = safe_str(os.path.join(self.basepath, repo_name))
         log.debug('Repository path is %s' % repo_path)
 
         # quick check if that dir exists...
         if is_valid_repo(repo_name, self.basepath) is False:
             return HTTPNotFound()(environ, start_response)
 
         try:
             #invalidate cache on push
             if action == 'push':
                 self._invalidate_cache(repo_name)
             log.info('%s action on GIT repo "%s"' % (action, repo_name))
             app = self.__make_app(repo_name, repo_path)
             return app(environ, start_response)
         except Exception:
             log.error(traceback.format_exc())
             return HTTPInternalServerError()(environ, start_response)
 
     def __make_app(self, repo_name, repo_path):
         """
         Make an wsgi application using dulserver
 
         :param repo_name: name of the repository
         :param repo_path: full path to the repository
         """
 
         _d = {'/' + repo_name: Repo(repo_path)}
         backend = dulserver.DictBackend(_d)
         gitserve = HTTPGitApplication(backend)
 
         return gitserve
 
     def __get_repository(self, environ):
         """
         Get's repository name out of PATH_INFO header
 
         :param environ: environ where PATH_INFO is stored
         """
         try:
             environ['PATH_INFO'] = self._get_by_id(environ['PATH_INFO'])
-            repo_name = '/'.join(environ['PATH_INFO'].split('/')[1:])
-            if repo_name.endswith('/'):
-                repo_name = repo_name.rstrip('/')
+            repo_name = GIT_PROTO_PAT.match(environ['PATH_INFO']).group(1)
         except:
             log.error(traceback.format_exc())
             raise
-        repo_name = repo_name.split('/')[0]
+
         return repo_name
 
     def __get_user(self, username):
         return User.get_by_username(username)
 
     def __get_action(self, environ):
         """Maps git request commands into a pull or push command.
 
         :param environ:
         """
         service = environ['QUERY_STRING'].split('=')
         if len(service) > 1:
             service_cmd = service[1]
-            mapping = {'git-receive-pack': 'push',
-                       'git-upload-pack': 'pull',
-                       }
+            mapping = {
+                'git-receive-pack': 'push',
+                'git-upload-pack': 'pull',
+            }
 
             return mapping.get(service_cmd,
                                service_cmd if service_cmd else 'other')
         else:
             return 'other'

rhodecode/lib/utils.py

@@ -1,616 +1,622 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.lib.utils
     ~~~~~~~~~~~~~~~~~~~
 
     Utilities library for RhodeCode
 
     :created_on: Apr 18, 2010
     :author: marcink
     :copyright: (C) 2010-2012 Marcin Kuzminski <[e-mail]>
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import os
 import logging
 import datetime
 import traceback
 import paste
 import beaker
 import tarfile
 import shutil
 from os.path import abspath
 from os.path import dirname as dn, join as jn
 
 from paste.script.command import Command, BadCommand
 
 from mercurial import ui, config
 
 from webhelpers.text import collapse, remove_formatting, strip_tags
 
 from rhodecode.lib.vcs import get_backend
 from rhodecode.lib.vcs.backends.base import BaseChangeset
 from rhodecode.lib.vcs.utils.lazy import LazyProperty
 from rhodecode.lib.vcs.utils.helpers import get_scm
 from rhodecode.lib.vcs.exceptions import VCSError
 
 from rhodecode.lib.caching_query import FromCache
 
 from rhodecode.model import meta
 from rhodecode.model.db import Repository, User, RhodeCodeUi, \
     UserLog, RepoGroup, RhodeCodeSetting, UserRepoGroupToPerm
 from rhodecode.model.meta import Session
 from rhodecode.model.repos_group import ReposGroupModel
 
 log = logging.getLogger(__name__)
 
 
 def recursive_replace(str_, replace=' '):
     """Recursive replace of given sign to just one instance
 
     :param str_: given string
     :param replace: char to find and replace multiple instances
 
     Examples::
     >>> recursive_replace("Mighty---Mighty-Bo--sstones",'-')
     'Mighty-Mighty-Bo-sstones'
     """
 
     if str_.find(replace * 2) == -1:
         return str_
     else:
         str_ = str_.replace(replace * 2, replace)
         return recursive_replace(str_, replace)
 
 
 def repo_name_slug(value):
     """Return slug of name of repository
     This function is called on each creation/modification
     of repository to prevent bad names in repo
     """
 
     slug = remove_formatting(value)
     slug = strip_tags(slug)
 
     for c in """=[]\;'"<>,/~!@#$%^&*()+{}|: """:
         slug = slug.replace(c, '-')
     slug = recursive_replace(slug, '-')
     slug = collapse(slug, '-')
     return slug
 
 
 def get_repo_slug(request):
-    return request.environ['pylons.routes_dict'].get('repo_name')
+    _repo = request.environ['pylons.routes_dict'].get('repo_name')
+    if _repo:
+        _repo = _repo.rstrip('/')
+    return _repo
 
 
 def get_repos_group_slug(request):
-    return request.environ['pylons.routes_dict'].get('group_name')
+    _group = request.environ['pylons.routes_dict'].get('group_name')
+    if _group:
+        _group = _group.rstrip('/')
+    return _group
 
 
 def action_logger(user, action, repo, ipaddr='', sa=None, commit=False):
     """
     Action logger for various actions made by users
 
     :param user: user that made this action, can be a unique username string or
         object containing user_id attribute
     :param action: action to log, should be on of predefined unique actions for
         easy translations
     :param repo: string name of repository or object containing repo_id,
         that action was made on
     :param ipaddr: optional ip address from what the action was made
     :param sa: optional sqlalchemy session
 
     """
 
     if not sa:
         sa = meta.Session
 
     try:
         if hasattr(user, 'user_id'):
             user_obj = user
         elif isinstance(user, basestring):
             user_obj = User.get_by_username(user)
         else:
             raise Exception('You have to provide user object or username')
 
         if hasattr(repo, 'repo_id'):
             repo_obj = Repository.get(repo.repo_id)
             repo_name = repo_obj.repo_name
         elif  isinstance(repo, basestring):
             repo_name = repo.lstrip('/')
             repo_obj = Repository.get_by_repo_name(repo_name)
         else:
             raise Exception('You have to provide repository to action logger')
 
         user_log = UserLog()
         user_log.user_id = user_obj.user_id
         user_log.action = action
 
         user_log.repository_id = repo_obj.repo_id
         user_log.repository_name = repo_name
 
         user_log.action_date = datetime.datetime.now()
         user_log.user_ip = ipaddr
         sa.add(user_log)
 
         log.info('Adding user %s, action %s on %s' % (user_obj, action, repo))
         if commit:
             sa.commit()
     except:
         log.error(traceback.format_exc())
         raise
 
 
 def get_repos(path, recursive=False):
     """
     Scans given path for repos and return (name,(type,path)) tuple
 
     :param path: path to scan for repositories
     :param recursive: recursive search and return names with subdirs in front
     """
 
     # remove ending slash for better results
     path = path.rstrip(os.sep)
 
     def _get_repos(p):
         if not os.access(p, os.W_OK):
             return
         for dirpath in os.listdir(p):
             if os.path.isfile(os.path.join(p, dirpath)):
                 continue
             cur_path = os.path.join(p, dirpath)
             try:
                 scm_info = get_scm(cur_path)
                 yield scm_info[1].split(path, 1)[-1].lstrip(os.sep), scm_info
             except VCSError:
                 if not recursive:
                     continue
                 #check if this dir containts other repos for recursive scan
                 rec_path = os.path.join(p, dirpath)
                 if os.path.isdir(rec_path):
                     for inner_scm in _get_repos(rec_path):
                         yield inner_scm
 
     return _get_repos(path)
 
 
 def is_valid_repo(repo_name, base_path):
     """
     Returns True if given path is a valid repository False otherwise
     :param repo_name:
     :param base_path:
 
     :return True: if given path is a valid repository
     """
     full_path = os.path.join(base_path, repo_name)
 
     try:
         get_scm(full_path)
         return True
     except VCSError:
         return False
 
 
 def is_valid_repos_group(repos_group_name, base_path):
     """
     Returns True if given path is a repos group False otherwise
 
     :param repo_name:
     :param base_path:
     """
     full_path = os.path.join(base_path, repos_group_name)
 
     # check if it's not a repo
     if is_valid_repo(repos_group_name, base_path):
         return False
 
     # check if it's a valid path
     if os.path.isdir(full_path):
         return True
 
     return False
 
 
 def ask_ok(prompt, retries=4, complaint='Yes or no, please!'):
     while True:
         ok = raw_input(prompt)
         if ok in ('y', 'ye', 'yes'):
             return True
         if ok in ('n', 'no', 'nop', 'nope'):
             return False
         retries = retries - 1
         if retries < 0:
             raise IOError
         print complaint
 
 #propagated from mercurial documentation
 ui_sections = ['alias', 'auth',
                 'decode/encode', 'defaults',
                 'diff', 'email',
                 'extensions', 'format',
                 'merge-patterns', 'merge-tools',
                 'hooks', 'http_proxy',
                 'smtp', 'patch',
                 'paths', 'profiling',
                 'server', 'trusted',
                 'ui', 'web', ]
 
 
 def make_ui(read_from='file', path=None, checkpaths=True):
     """A function that will read python rc files or database
     and make an mercurial ui object from read options
 
     :param path: path to mercurial config file
     :param checkpaths: check the path
     :param read_from: read from 'file' or 'db'
     """
 
     baseui = ui.ui()
 
     # clean the baseui object
     baseui._ocfg = config.config()
     baseui._ucfg = config.config()
     baseui._tcfg = config.config()
 
     if read_from == 'file':
         if not os.path.isfile(path):
             log.debug('hgrc file is not present at %s skipping...' % path)
             return False
         log.debug('reading hgrc from %s' % path)
         cfg = config.config()
         cfg.read(path)
         for section in ui_sections:
             for k, v in cfg.items(section):
                 log.debug('settings ui from file[%s]%s:%s' % (section, k, v))
                 baseui.setconfig(section, k, v)
 
     elif read_from == 'db':
         sa = meta.Session
         ret = sa.query(RhodeCodeUi)\
             .options(FromCache("sql_cache_short", "get_hg_ui_settings"))\
             .all()
 
         hg_ui = ret
         for ui_ in hg_ui:
             if ui_.ui_active:
                 log.debug('settings ui from db[%s]%s:%s', ui_.ui_section,
                           ui_.ui_key, ui_.ui_value)
                 baseui.setconfig(ui_.ui_section, ui_.ui_key, ui_.ui_value)
 
         meta.Session.remove()
     return baseui
 
 
 def set_rhodecode_config(config):
     """
     Updates pylons config with new settings from database
 
     :param config:
     """
     hgsettings = RhodeCodeSetting.get_app_settings()
 
     for k, v in hgsettings.items():
         config[k] = v
 
 
 def invalidate_cache(cache_key, *args):
     """
     Puts cache invalidation task into db for
     further global cache invalidation
     """
 
     from rhodecode.model.scm import ScmModel
 
     if cache_key.startswith('get_repo_cached_'):
         name = cache_key.split('get_repo_cached_')[-1]
         ScmModel().mark_for_invalidation(name)
 
 
 class EmptyChangeset(BaseChangeset):
     """
     An dummy empty changeset. It's possible to pass hash when creating
     an EmptyChangeset
     """
 
     def __init__(self, cs='0' * 40, repo=None, requested_revision=None,
                  alias=None):
         self._empty_cs = cs
         self.revision = -1
         self.message = ''
         self.author = ''
         self.date = ''
         self.repository = repo
         self.requested_revision = requested_revision
         self.alias = alias
 
     @LazyProperty
     def raw_id(self):
         """
         Returns raw string identifying this changeset, useful for web
         representation.
         """
 
         return self._empty_cs
 
     @LazyProperty
     def branch(self):
         return get_backend(self.alias).DEFAULT_BRANCH_NAME
 
     @LazyProperty
     def short_id(self):
         return self.raw_id[:12]
 
     def get_file_changeset(self, path):
         return self
 
     def get_file_content(self, path):
         return u''
 
     def get_file_size(self, path):
         return 0
 
 
 def map_groups(groups):
     """
     Checks for groups existence, and creates groups structures.
     It returns last group in structure
 
     :param groups: list of groups structure
     """
     sa = meta.Session
 
     parent = None
     group = None
 
     # last element is repo in nested groups structure
     groups = groups[:-1]
     rgm = ReposGroupModel(sa)
     for lvl, group_name in enumerate(groups):
         group_name = '/'.join(groups[:lvl] + [group_name])
         group = RepoGroup.get_by_group_name(group_name)
         desc = '%s group' % group_name
 
 #        # WTF that doesn't work !?
 #        if group is None:
 #            group = rgm.create(group_name, desc, parent, just_db=True)
 #            sa.commit()
 
         if group is None:
             log.debug('creating group level: %s group_name: %s' % (lvl, group_name))
             group = RepoGroup(group_name, parent)
             group.group_description = desc
             sa.add(group)
             rgm._create_default_perms(group)
             sa.commit()
         parent = group
     return group
 
 
 def repo2db_mapper(initial_repo_list, remove_obsolete=False):
     """
     maps all repos given in initial_repo_list, non existing repositories
     are created, if remove_obsolete is True it also check for db entries
     that are not in initial_repo_list and removes them.
 
     :param initial_repo_list: list of repositories found by scanning methods
     :param remove_obsolete: check for obsolete entries in database
     """
     from rhodecode.model.repo import RepoModel
     sa = meta.Session
     rm = RepoModel()
     user = sa.query(User).filter(User.admin == True).first()
     if user is None:
         raise Exception('Missing administrative account !')
     added = []
 
     for name, repo in initial_repo_list.items():
         group = map_groups(name.split(Repository.url_sep()))
         if not rm.get_by_repo_name(name, cache=False):
             log.info('repository %s not found creating default' % name)
             added.append(name)
             form_data = {
              'repo_name': name,
              'repo_name_full': name,
              'repo_type': repo.alias,
              'description': repo.description \
                 if repo.description != 'unknown' else '%s repository' % name,
              'private': False,
              'group_id': getattr(group, 'group_id', None)
             }
             rm.create(form_data, user, just_db=True)
     sa.commit()
     removed = []
     if remove_obsolete:
         #remove from database those repositories that are not in the filesystem
         for repo in sa.query(Repository).all():
             if repo.repo_name not in initial_repo_list.keys():
                 removed.append(repo.repo_name)
                 sa.delete(repo)
                 sa.commit()
 
     return added, removed
 
 
 # set cache regions for beaker so celery can utilise it
 def add_cache(settings):
     cache_settings = {'regions': None}
     for key in settings.keys():
         for prefix in ['beaker.cache.', 'cache.']:
             if key.startswith(prefix):
                 name = key.split(prefix)[1].strip()
                 cache_settings[name] = settings[key].strip()
     if cache_settings['regions']:
         for region in cache_settings['regions'].split(','):
             region = region.strip()
             region_settings = {}
             for key, value in cache_settings.items():
                 if key.startswith(region):
                     region_settings[key.split('.')[1]] = value
             region_settings['expire'] = int(region_settings.get('expire',
                                                                 60))
             region_settings.setdefault('lock_dir',
                                        cache_settings.get('lock_dir'))
             region_settings.setdefault('data_dir',
                                        cache_settings.get('data_dir'))
 
             if 'type' not in region_settings:
                 region_settings['type'] = cache_settings.get('type',
                                                              'memory')
             beaker.cache.cache_regions[region] = region_settings
 
 
 #==============================================================================
 # TEST FUNCTIONS AND CREATORS
 #==============================================================================
 def create_test_index(repo_location, config, full_index):
     """
     Makes default test index
 
     :param config: test config
     :param full_index:
     """
 
     from rhodecode.lib.indexers.daemon import WhooshIndexingDaemon
     from rhodecode.lib.pidlock import DaemonLock, LockHeld
 
     repo_location = repo_location
 
     index_location = os.path.join(config['app_conf']['index_dir'])
     if not os.path.exists(index_location):
         os.makedirs(index_location)
 
     try:
         l = DaemonLock(file_=jn(dn(index_location), 'make_index.lock'))
         WhooshIndexingDaemon(index_location=index_location,
                              repo_location=repo_location)\
             .run(full_index=full_index)
         l.release()
     except LockHeld:
         pass
 
 
 def create_test_env(repos_test_path, config):
     """
     Makes a fresh database and
     install test repository into tmp dir
     """
     from rhodecode.lib.db_manage import DbManage
     from rhodecode.tests import HG_REPO, TESTS_TMP_PATH
 
     # PART ONE create db
     dbconf = config['sqlalchemy.db1.url']
     log.debug('making test db %s' % dbconf)
 
     # create test dir if it doesn't exist
     if not os.path.isdir(repos_test_path):
         log.debug('Creating testdir %s' % repos_test_path)
         os.makedirs(repos_test_path)
 
     dbmanage = DbManage(log_sql=True, dbconf=dbconf, root=config['here'],
                         tests=True)
     dbmanage.create_tables(override=True)
     dbmanage.create_settings(dbmanage.config_prompt(repos_test_path))
     dbmanage.create_default_user()
     dbmanage.admin_prompt()
     dbmanage.create_permissions()
     dbmanage.populate_default_permissions()
     Session.commit()
     # PART TWO make test repo
     log.debug('making test vcs repositories')
 
     idx_path = config['app_conf']['index_dir']
     data_path = config['app_conf']['cache_dir']
 
     #clean index and data
     if idx_path and os.path.exists(idx_path):
         log.debug('remove %s' % idx_path)
         shutil.rmtree(idx_path)
 
     if data_path and os.path.exists(data_path):
         log.debug('remove %s' % data_path)
         shutil.rmtree(data_path)
 
     #CREATE DEFAULT HG REPOSITORY
     cur_dir = dn(dn(abspath(__file__)))
     tar = tarfile.open(jn(cur_dir, 'tests', "vcs_test_hg.tar.gz"))
     tar.extractall(jn(TESTS_TMP_PATH, HG_REPO))
     tar.close()
 
 
 #==============================================================================
 # PASTER COMMANDS
 #==============================================================================
 class BasePasterCommand(Command):
     """
     Abstract Base Class for paster commands.
 
     The celery commands are somewhat aggressive about loading
     celery.conf, and since our module sets the `CELERY_LOADER`
     environment variable to our loader, we have to bootstrap a bit and
     make sure we've had a chance to load the pylons config off of the
     command line, otherwise everything fails.
     """
     min_args = 1
     min_args_error = "Please provide a paster config file as an argument."
     takes_config_file = 1
     requires_config_file = True
 
     def notify_msg(self, msg, log=False):
         """Make a notification to user, additionally if logger is passed
         it logs this action using given logger
 
         :param msg: message that will be printed to user
         :param log: logging instance, to use to additionally log this message
 
         """
         if log and isinstance(log, logging):
             log(msg)
 
     def run(self, args):
         """
         Overrides Command.run
 
         Checks for a config file argument and loads it.
         """
         if len(args) < self.min_args:
             raise BadCommand(
                 self.min_args_error % {'min_args': self.min_args,
                                        'actual_args': len(args)})
 
         # Decrement because we're going to lob off the first argument.
         # @@ This is hacky
         self.min_args -= 1
         self.bootstrap_config(args[0])
         self.update_parser()
         return super(BasePasterCommand, self).run(args[1:])
 
     def update_parser(self):
         """
         Abstract method.  Allows for the class's parser to be updated
         before the superclass's `run` method is called.  Necessary to
         allow options/arguments to be passed through to the underlying
         celery command.
         """
         raise NotImplementedError("Abstract Method.")
 
     def bootstrap_config(self, conf):
         """
         Loads the pylons configuration.
         """
         from pylons import config as pylonsconfig
 
         path_to_ini_file = os.path.realpath(conf)
         conf = paste.deploy.appconfig('config:' + path_to_ini_file)
         pylonsconfig.init_app(conf.global_conf, conf.local_conf)

rhodecode/model/db.py

@@ -1,1203 +1,1216 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.model.db
     ~~~~~~~~~~~~~~~~~~
 
     Database Models for RhodeCode
 
     :created_on: Apr 08, 2010
     :author: marcink
     :copyright: (C) 2010-2012 Marcin Kuzminski <[e-mail]>
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import os
 import logging
 import datetime
 import traceback
 from collections import defaultdict
 
 from sqlalchemy import *
 from sqlalchemy.ext.hybrid import hybrid_property
 from sqlalchemy.orm import relationship, joinedload, class_mapper, validates
 from beaker.cache import cache_region, region_invalidate
 
 from rhodecode.lib.vcs import get_backend
 from rhodecode.lib.vcs.utils.helpers import get_scm
 from rhodecode.lib.vcs.exceptions import VCSError
 from rhodecode.lib.vcs.utils.lazy import LazyProperty
 
 from rhodecode.lib import str2bool, safe_str, get_changeset_safe, safe_unicode
 from rhodecode.lib.compat import json
 from rhodecode.lib.caching_query import FromCache
 
 from rhodecode.model.meta import Base, Session
+import hashlib
 
 
 log = logging.getLogger(__name__)
 
 #==============================================================================
 # BASE CLASSES
 #==============================================================================
 
+_hash_key = lambda k: hashlib.md5(safe_str(k)).hexdigest()
+
 
 class ModelSerializer(json.JSONEncoder):
     """
     Simple Serializer for JSON,
 
     usage::
 
         to make object customized for serialization implement a __json__
         method that will return a dict for serialization into json
 
     example::
 
         class Task(object):
 
             def __init__(self, name, value):
                 self.name = name
                 self.value = value
 
             def __json__(self):
                 return dict(name=self.name,
                             value=self.value)
 
     """
 
     def default(self, obj):
 
         if hasattr(obj, '__json__'):
             return obj.__json__()
         else:
             return json.JSONEncoder.default(self, obj)
 
 
 class BaseModel(object):
     """
     Base Model for all classess
     """
 
     @classmethod
     def _get_keys(cls):
         """return column names for this model """
         return class_mapper(cls).c.keys()
 
     def get_dict(self):
         """
         return dict with keys and values corresponding
         to this model data """
 
         d = {}
         for k in self._get_keys():
             d[k] = getattr(self, k)
 
         # also use __json__() if present to get additional fields
         for k, val in getattr(self, '__json__', lambda: {})().iteritems():
             d[k] = val
         return d
 
     def get_appstruct(self):
         """return list with keys and values tupples corresponding
         to this model data """
 
         l = []
         for k in self._get_keys():
             l.append((k, getattr(self, k),))
         return l
 
     def populate_obj(self, populate_dict):
         """populate model with data from given populate_dict"""
 
         for k in self._get_keys():
             if k in populate_dict:
                 setattr(self, k, populate_dict[k])
 
     @classmethod
     def query(cls):
         return Session.query(cls)
 
     @classmethod
     def get(cls, id_):
         if id_:
             return cls.query().get(id_)
 
     @classmethod
     def getAll(cls):
         return cls.query().all()
 
     @classmethod
     def delete(cls, id_):
         obj = cls.query().get(id_)
         Session.delete(obj)
 
 
 class RhodeCodeSetting(Base, BaseModel):
     __tablename__ = 'rhodecode_settings'
     __table_args__ = (
         UniqueConstraint('app_settings_name'),
         {'extend_existing': True}
     )
     app_settings_id = Column("app_settings_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     app_settings_name = Column("app_settings_name", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     _app_settings_value = Column("app_settings_value", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
 
     def __init__(self, k='', v=''):
         self.app_settings_name = k
         self.app_settings_value = v
 
     @validates('_app_settings_value')
     def validate_settings_value(self, key, val):
         assert type(val) == unicode
         return val
 
     @hybrid_property
     def app_settings_value(self):
         v = self._app_settings_value
         if self.app_settings_name == 'ldap_active':
             v = str2bool(v)
         return v
 
     @app_settings_value.setter
     def app_settings_value(self, val):
         """
         Setter that will always make sure we use unicode in app_settings_value
 
         :param val:
         """
         self._app_settings_value = safe_unicode(val)
 
     def __repr__(self):
         return "<%s('%s:%s')>" % (
             self.__class__.__name__,
             self.app_settings_name, self.app_settings_value
         )
 
     @classmethod
     def get_by_name(cls, ldap_key):
         return cls.query()\
             .filter(cls.app_settings_name == ldap_key).scalar()
 
     @classmethod
     def get_app_settings(cls, cache=False):
 
         ret = cls.query()
 
         if cache:
             ret = ret.options(FromCache("sql_cache_short", "get_hg_settings"))
 
         if not ret:
             raise Exception('Could not get application settings !')
         settings = {}
         for each in ret:
             settings['rhodecode_' + each.app_settings_name] = \
                 each.app_settings_value
 
         return settings
 
     @classmethod
     def get_ldap_settings(cls, cache=False):
         ret = cls.query()\
                 .filter(cls.app_settings_name.startswith('ldap_')).all()
         fd = {}
         for row in ret:
             fd.update({row.app_settings_name:row.app_settings_value})
 
         return fd
 
 
 class RhodeCodeUi(Base, BaseModel):
     __tablename__ = 'rhodecode_ui'
     __table_args__ = (
         UniqueConstraint('ui_key'),
         {'extend_existing': True}
     )
 
     HOOK_UPDATE = 'changegroup.update'
     HOOK_REPO_SIZE = 'changegroup.repo_size'
     HOOK_PUSH = 'pretxnchangegroup.push_logger'
     HOOK_PULL = 'preoutgoing.pull_logger'
 
     ui_id = Column("ui_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     ui_section = Column("ui_section", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     ui_key = Column("ui_key", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     ui_value = Column("ui_value", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     ui_active = Column("ui_active", Boolean(), nullable=True, unique=None, default=True)
 
     @classmethod
     def get_by_key(cls, key):
         return cls.query().filter(cls.ui_key == key)
 
     @classmethod
     def get_builtin_hooks(cls):
         q = cls.query()
         q = q.filter(cls.ui_key.in_([cls.HOOK_UPDATE,
                                     cls.HOOK_REPO_SIZE,
                                     cls.HOOK_PUSH, cls.HOOK_PULL]))
         return q.all()
 
     @classmethod
     def get_custom_hooks(cls):
         q = cls.query()
         q = q.filter(~cls.ui_key.in_([cls.HOOK_UPDATE,
                                     cls.HOOK_REPO_SIZE,
                                     cls.HOOK_PUSH, cls.HOOK_PULL]))
         q = q.filter(cls.ui_section == 'hooks')
         return q.all()
 
     @classmethod
     def create_or_update_hook(cls, key, val):
         new_ui = cls.get_by_key(key).scalar() or cls()
         new_ui.ui_section = 'hooks'
         new_ui.ui_active = True
         new_ui.ui_key = key
         new_ui.ui_value = val
 
         Session.add(new_ui)
 
 
 class User(Base, BaseModel):
     __tablename__ = 'users'
     __table_args__ = (
         UniqueConstraint('username'), UniqueConstraint('email'),
         {'extend_existing': True}
     )
     user_id = Column("user_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     username = Column("username", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     password = Column("password", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     active = Column("active", Boolean(), nullable=True, unique=None, default=None)
     admin = Column("admin", Boolean(), nullable=True, unique=None, default=False)
     name = Column("name", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     lastname = Column("lastname", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     _email = Column("email", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     last_login = Column("last_login", DateTime(timezone=False), nullable=True, unique=None, default=None)
     ldap_dn = Column("ldap_dn", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     api_key = Column("api_key", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
 
     user_log = relationship('UserLog', cascade='all')
     user_perms = relationship('UserToPerm', primaryjoin="User.user_id==UserToPerm.user_id", cascade='all')
 
     repositories = relationship('Repository')
     user_followers = relationship('UserFollowing', primaryjoin='UserFollowing.follows_user_id==User.user_id', cascade='all')
     repo_to_perm = relationship('UserRepoToPerm', primaryjoin='UserRepoToPerm.user_id==User.user_id', cascade='all')
 
     group_member = relationship('UsersGroupMember', cascade='all')
 
     notifications = relationship('UserNotification',)
 
     @hybrid_property
     def email(self):
         return self._email
 
     @email.setter
     def email(self, val):
         self._email = val.lower() if val else None
 
     @property
     def full_name(self):
         return '%s %s' % (self.name, self.lastname)
 
     @property
     def full_name_or_username(self):
         return ('%s %s' % (self.name, self.lastname)
                 if (self.name and self.lastname) else self.username)
 
     @property
     def full_contact(self):
         return '%s %s <%s>' % (self.name, self.lastname, self.email)
 
     @property
     def short_contact(self):
         return '%s %s' % (self.name, self.lastname)
 
     @property
     def is_admin(self):
         return self.admin
 
     def __repr__(self):
         return "<%s('id:%s:%s')>" % (self.__class__.__name__,
                                      self.user_id, self.username)
 
     @classmethod
     def get_by_username(cls, username, case_insensitive=False, cache=False):
         if case_insensitive:
             q = cls.query().filter(cls.username.ilike(username))
         else:
             q = cls.query().filter(cls.username == username)
 
         if cache:
-            q = q.options(FromCache("sql_cache_short",
-                                    "get_user_%s" % username))
+            q = q.options(FromCache(
+                            "sql_cache_short",
+                            "get_user_%s" % _hash_key(username)
+                          )
+            )
         return q.scalar()
 
     @classmethod
     def get_by_api_key(cls, api_key, cache=False):
         q = cls.query().filter(cls.api_key == api_key)
 
         if cache:
             q = q.options(FromCache("sql_cache_short",
                                     "get_api_key_%s" % api_key))
         return q.scalar()
 
     @classmethod
     def get_by_email(cls, email, case_insensitive=False, cache=False):
         if case_insensitive:
             q = cls.query().filter(cls.email.ilike(email))
         else:
             q = cls.query().filter(cls.email == email)
 
         if cache:
             q = q.options(FromCache("sql_cache_short",
                                     "get_api_key_%s" % email))
         return q.scalar()
 
     def update_lastlogin(self):
         """Update user lastlogin"""
         self.last_login = datetime.datetime.now()
         Session.add(self)
         log.debug('updated user %s lastlogin' % self.username)
 
     def __json__(self):
         return dict(
             email=self.email,
             full_name=self.full_name,
             full_name_or_username=self.full_name_or_username,
             short_contact=self.short_contact,
             full_contact=self.full_contact
         )
 
 
 class UserLog(Base, BaseModel):
     __tablename__ = 'user_logs'
     __table_args__ = {'extend_existing': True}
     user_log_id = Column("user_log_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
     repository_id = Column("repository_id", Integer(), ForeignKey('repositories.repo_id'), nullable=True)
     repository_name = Column("repository_name", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     user_ip = Column("user_ip", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     action = Column("action", UnicodeText(length=1200000, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     action_date = Column("action_date", DateTime(timezone=False), nullable=True, unique=None, default=None)
 
     @property
     def action_as_day(self):
         return datetime.date(*self.action_date.timetuple()[:3])
 
     user = relationship('User')
-    repository = relationship('Repository',cascade='')
+    repository = relationship('Repository', cascade='')
 
 
 class UsersGroup(Base, BaseModel):
     __tablename__ = 'users_groups'
     __table_args__ = {'extend_existing': True}
 
     users_group_id = Column("users_group_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     users_group_name = Column("users_group_name", String(length=255, convert_unicode=False, assert_unicode=None), nullable=False, unique=True, default=None)
     users_group_active = Column("users_group_active", Boolean(), nullable=True, unique=None, default=None)
 
     members = relationship('UsersGroupMember', cascade="all, delete, delete-orphan", lazy="joined")
+    users_group_to_perm = relationship('UsersGroupToPerm', cascade='all')
 
     def __repr__(self):
         return '<userGroup(%s)>' % (self.users_group_name)
 
     @classmethod
     def get_by_group_name(cls, group_name, cache=False,
                           case_insensitive=False):
         if case_insensitive:
             q = cls.query().filter(cls.users_group_name.ilike(group_name))
         else:
             q = cls.query().filter(cls.users_group_name == group_name)
         if cache:
-            q = q.options(FromCache("sql_cache_short",
-                                    "get_user_%s" % group_name))
+            q = q.options(FromCache(
+                            "sql_cache_short",
+                            "get_user_%s" % _hash_key(group_name)
+                          )
+            )
         return q.scalar()
 
     @classmethod
     def get(cls, users_group_id, cache=False):
         users_group = cls.query()
         if cache:
             users_group = users_group.options(FromCache("sql_cache_short",
                                     "get_users_group_%s" % users_group_id))
         return users_group.get(users_group_id)
 
 
 class UsersGroupMember(Base, BaseModel):
     __tablename__ = 'users_groups_members'
     __table_args__ = {'extend_existing': True}
 
     users_group_member_id = Column("users_group_member_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     users_group_id = Column("users_group_id", Integer(), ForeignKey('users_groups.users_group_id'), nullable=False, unique=None, default=None)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
 
     user = relationship('User', lazy='joined')
     users_group = relationship('UsersGroup')
 
     def __init__(self, gr_id='', u_id=''):
         self.users_group_id = gr_id
         self.user_id = u_id
 
 
 class Repository(Base, BaseModel):
     __tablename__ = 'repositories'
     __table_args__ = (
         UniqueConstraint('repo_name'),
         {'extend_existing': True},
     )
 
     repo_id = Column("repo_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     repo_name = Column("repo_name", String(length=255, convert_unicode=False, assert_unicode=None), nullable=False, unique=True, default=None)
     clone_uri = Column("clone_uri", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=False, default=None)
     repo_type = Column("repo_type", String(length=255, convert_unicode=False, assert_unicode=None), nullable=False, unique=False, default='hg')
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=False, default=None)
     private = Column("private", Boolean(), nullable=True, unique=None, default=None)
     enable_statistics = Column("statistics", Boolean(), nullable=True, unique=None, default=True)
     enable_downloads = Column("downloads", Boolean(), nullable=True, unique=None, default=True)
     description = Column("description", String(length=10000, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     created_on = Column('created_on', DateTime(timezone=False), nullable=True, unique=None, default=datetime.datetime.now)
 
     fork_id = Column("fork_id", Integer(), ForeignKey('repositories.repo_id'), nullable=True, unique=False, default=None)
     group_id = Column("group_id", Integer(), ForeignKey('groups.group_id'), nullable=True, unique=False, default=None)
 
     user = relationship('User')
     fork = relationship('Repository', remote_side=repo_id)
     group = relationship('RepoGroup')
     repo_to_perm = relationship('UserRepoToPerm', cascade='all', order_by='UserRepoToPerm.repo_to_perm_id')
     users_group_to_perm = relationship('UsersGroupRepoToPerm', cascade='all')
     stats = relationship('Statistics', cascade='all', uselist=False)
 
     followers = relationship('UserFollowing', primaryjoin='UserFollowing.follows_repo_id==Repository.repo_id', cascade='all')
 
     logs = relationship('UserLog')
 
     def __repr__(self):
         return "<%s('%s:%s')>" % (self.__class__.__name__,
                                   self.repo_id, self.repo_name)
 
     @classmethod
     def url_sep(cls):
         return '/'
 
     @classmethod
     def get_by_repo_name(cls, repo_name):
         q = Session.query(cls).filter(cls.repo_name == repo_name)
         q = q.options(joinedload(Repository.fork))\
                 .options(joinedload(Repository.user))\
                 .options(joinedload(Repository.group))
         return q.scalar()
 
     @classmethod
     def get_repo_forks(cls, repo_id):
         return cls.query().filter(Repository.fork_id == repo_id)
 
     @classmethod
     def base_path(cls):
         """
         Returns base path when all repos are stored
 
         :param cls:
         """
         q = Session.query(RhodeCodeUi)\
             .filter(RhodeCodeUi.ui_key == cls.url_sep())
         q = q.options(FromCache("sql_cache_short", "repository_repo_path"))
         return q.one().ui_value
 
     @property
     def just_name(self):
         return self.repo_name.split(Repository.url_sep())[-1]
 
     @property
     def groups_with_parents(self):
         groups = []
         if self.group is None:
             return groups
 
         cur_gr = self.group
         groups.insert(0, cur_gr)
         while 1:
             gr = getattr(cur_gr, 'parent_group', None)
             cur_gr = cur_gr.parent_group
             if gr is None:
                 break
             groups.insert(0, gr)
 
         return groups
 
     @property
     def groups_and_repo(self):
         return self.groups_with_parents, self.just_name
 
     @LazyProperty
     def repo_path(self):
         """
         Returns base full path for that repository means where it actually
         exists on a filesystem
         """
         q = Session.query(RhodeCodeUi).filter(RhodeCodeUi.ui_key ==
                                               Repository.url_sep())
         q = q.options(FromCache("sql_cache_short", "repository_repo_path"))
         return q.one().ui_value
 
     @property
     def repo_full_path(self):
         p = [self.repo_path]
         # we need to split the name by / since this is how we store the
         # names in the database, but that eventually needs to be converted
         # into a valid system path
         p += self.repo_name.split(Repository.url_sep())
         return os.path.join(*p)
 
     def get_new_name(self, repo_name):
         """
         returns new full repository name based on assigned group and new new
 
         :param group_name:
         """
         path_prefix = self.group.full_path_splitted if self.group else []
         return Repository.url_sep().join(path_prefix + [repo_name])
 
     @property
     def _ui(self):
         """
         Creates an db based ui object for this repository
         """
         from mercurial import ui
         from mercurial import config
         baseui = ui.ui()
 
         #clean the baseui object
         baseui._ocfg = config.config()
         baseui._ucfg = config.config()
         baseui._tcfg = config.config()
 
         ret = RhodeCodeUi.query()\
             .options(FromCache("sql_cache_short", "repository_repo_ui")).all()
 
         hg_ui = ret
         for ui_ in hg_ui:
             if ui_.ui_active:
                 log.debug('settings ui from db[%s]%s:%s', ui_.ui_section,
                           ui_.ui_key, ui_.ui_value)
                 baseui.setconfig(ui_.ui_section, ui_.ui_key, ui_.ui_value)
 
         return baseui
 
     @classmethod
     def is_valid(cls, repo_name):
         """
         returns True if given repo name is a valid filesystem repository
 
         :param cls:
         :param repo_name:
         """
         from rhodecode.lib.utils import is_valid_repo
 
         return is_valid_repo(repo_name, cls.base_path())
 
     #==========================================================================
     # SCM PROPERTIES
     #==========================================================================
 
     def get_changeset(self, rev):
         return get_changeset_safe(self.scm_instance, rev)
 
     @property
     def tip(self):
         return self.get_changeset('tip')
 
     @property
     def author(self):
         return self.tip.author
 
     @property
     def last_change(self):
         return self.scm_instance.last_change
 
     def comments(self, revisions=None):
         """
         Returns comments for this repository grouped by revisions
 
         :param revisions: filter query by revisions only
         """
         cmts = ChangesetComment.query()\
             .filter(ChangesetComment.repo == self)
         if revisions:
             cmts = cmts.filter(ChangesetComment.revision.in_(revisions))
         grouped = defaultdict(list)
         for cmt in cmts.all():
             grouped[cmt.revision].append(cmt)
         return grouped
 
     #==========================================================================
     # SCM CACHE INSTANCE
     #==========================================================================
 
     @property
     def invalidate(self):
         return CacheInvalidation.invalidate(self.repo_name)
 
     def set_invalidate(self):
         """
         set a cache for invalidation for this instance
         """
         CacheInvalidation.set_invalidate(self.repo_name)
 
     @LazyProperty
     def scm_instance(self):
         return self.__get_instance()
 
     @property
     def scm_instance_cached(self):
         @cache_region('long_term')
         def _c(repo_name):
             return self.__get_instance()
         rn = self.repo_name
         log.debug('Getting cached instance of repo')
         inv = self.invalidate
         if inv is not None:
             region_invalidate(_c, None, rn)
             # update our cache
             CacheInvalidation.set_valid(inv.cache_key)
         return _c(rn)
 
     def __get_instance(self):
         repo_full_path = self.repo_full_path
         try:
             alias = get_scm(repo_full_path)[0]
             log.debug('Creating instance of %s repository' % alias)
             backend = get_backend(alias)
         except VCSError:
             log.error(traceback.format_exc())
             log.error('Perhaps this repository is in db and not in '
                       'filesystem run rescan repositories with '
                       '"destroy old data " option from admin panel')
             return
 
         if alias == 'hg':
 
             repo = backend(safe_str(repo_full_path), create=False,
                            baseui=self._ui)
             # skip hidden web repository
             if repo._get_hidden():
                 return
         else:
             repo = backend(repo_full_path, create=False)
 
         return repo
 
 
 class RepoGroup(Base, BaseModel):
     __tablename__ = 'groups'
     __table_args__ = (
         UniqueConstraint('group_name', 'group_parent_id'),
         CheckConstraint('group_id != group_parent_id'),
         {'extend_existing': True},
     )
     __mapper_args__ = {'order_by': 'group_name'}
 
     group_id = Column("group_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     group_name = Column("group_name", String(length=255, convert_unicode=False, assert_unicode=None), nullable=False, unique=True, default=None)
     group_parent_id = Column("group_parent_id", Integer(), ForeignKey('groups.group_id'), nullable=True, unique=None, default=None)
     group_description = Column("group_description", String(length=10000, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
 
     repo_group_to_perm = relationship('UserRepoGroupToPerm', cascade='all', order_by='UserRepoGroupToPerm.group_to_perm_id')
     users_group_to_perm = relationship('UsersGroupRepoGroupToPerm', cascade='all')
 
     parent_group = relationship('RepoGroup', remote_side=group_id)
 
     def __init__(self, group_name='', parent_group=None):
         self.group_name = group_name
         self.parent_group = parent_group
 
     def __repr__(self):
         return "<%s('%s:%s')>" % (self.__class__.__name__, self.group_id,
                                   self.group_name)
 
     @classmethod
     def groups_choices(cls):
         from webhelpers.html import literal as _literal
         repo_groups = [('', '')]
         sep = ' &raquo; '
         _name = lambda k: _literal(sep.join(k))
 
         repo_groups.extend([(x.group_id, _name(x.full_path_splitted))
                               for x in cls.query().all()])
 
         repo_groups = sorted(repo_groups, key=lambda t: t[1].split(sep)[0])
         return repo_groups
 
     @classmethod
     def url_sep(cls):
         return '/'
 
     @classmethod
     def get_by_group_name(cls, group_name, cache=False, case_insensitive=False):
         if case_insensitive:
             gr = cls.query()\
                 .filter(cls.group_name.ilike(group_name))
         else:
             gr = cls.query()\
                 .filter(cls.group_name == group_name)
         if cache:
-            gr = gr.options(FromCache("sql_cache_short",
-                                          "get_group_%s" % group_name))
+            gr = gr.options(FromCache(
+                            "sql_cache_short",
+                            "get_group_%s" % _hash_key(group_name)
+                            )
+            )
         return gr.scalar()
 
     @property
     def parents(self):
         parents_recursion_limit = 5
         groups = []
         if self.parent_group is None:
             return groups
         cur_gr = self.parent_group
         groups.insert(0, cur_gr)
         cnt = 0
         while 1:
             cnt += 1
             gr = getattr(cur_gr, 'parent_group', None)
             cur_gr = cur_gr.parent_group
             if gr is None:
                 break
             if cnt == parents_recursion_limit:
                 # this will prevent accidental infinit loops
                 log.error('group nested more than %s' %
                           parents_recursion_limit)
                 break
 
             groups.insert(0, gr)
         return groups
 
     @property
     def children(self):
         return RepoGroup.query().filter(RepoGroup.parent_group == self)
 
     @property
     def name(self):
         return self.group_name.split(RepoGroup.url_sep())[-1]
 
     @property
     def full_path(self):
         return self.group_name
 
     @property
     def full_path_splitted(self):
         return self.group_name.split(RepoGroup.url_sep())
 
     @property
     def repositories(self):
         return Repository.query().filter(Repository.group == self)
 
     @property
     def repositories_recursive_count(self):
         cnt = self.repositories.count()
 
         def children_count(group):
             cnt = 0
             for child in group.children:
                 cnt += child.repositories.count()
                 cnt += children_count(child)
             return cnt
 
         return cnt + children_count(self)
 
     def get_new_name(self, group_name):
         """
         returns new full group name based on parent and new name
 
         :param group_name:
         """
         path_prefix = (self.parent_group.full_path_splitted if
                        self.parent_group else [])
         return RepoGroup.url_sep().join(path_prefix + [group_name])
 
 
 class Permission(Base, BaseModel):
     __tablename__ = 'permissions'
     __table_args__ = {'extend_existing': True}
     permission_id = Column("permission_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     permission_name = Column("permission_name", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     permission_longname = Column("permission_longname", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
 
     def __repr__(self):
         return "<%s('%s:%s')>" % (
             self.__class__.__name__, self.permission_id, self.permission_name
         )
 
     @classmethod
     def get_by_key(cls, key):
         return cls.query().filter(cls.permission_name == key).scalar()
 
     @classmethod
     def get_default_perms(cls, default_user_id):
         q = Session.query(UserRepoToPerm, Repository, cls)\
          .join((Repository, UserRepoToPerm.repository_id == Repository.repo_id))\
          .join((cls, UserRepoToPerm.permission_id == cls.permission_id))\
          .filter(UserRepoToPerm.user_id == default_user_id)
 
         return q.all()
 
     @classmethod
     def get_default_group_perms(cls, default_user_id):
         q = Session.query(UserRepoGroupToPerm, RepoGroup, cls)\
          .join((RepoGroup, UserRepoGroupToPerm.group_id == RepoGroup.group_id))\
          .join((cls, UserRepoGroupToPerm.permission_id == cls.permission_id))\
          .filter(UserRepoGroupToPerm.user_id == default_user_id)
 
         return q.all()
 
 
 class UserRepoToPerm(Base, BaseModel):
     __tablename__ = 'repo_to_perm'
     __table_args__ = (
         UniqueConstraint('user_id', 'repository_id', 'permission_id'),
         {'extend_existing': True}
     )
     repo_to_perm_id = Column("repo_to_perm_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
     permission_id = Column("permission_id", Integer(), ForeignKey('permissions.permission_id'), nullable=False, unique=None, default=None)
     repository_id = Column("repository_id", Integer(), ForeignKey('repositories.repo_id'), nullable=False, unique=None, default=None)
 
     user = relationship('User')
     repository = relationship('Repository')
     permission = relationship('Permission')
 
     @classmethod
     def create(cls, user, repository, permission):
         n = cls()
         n.user = user
         n.repository = repository
         n.permission = permission
         Session.add(n)
         return n
 
     def __repr__(self):
         return '<user:%s => %s >' % (self.user, self.repository)
 
 
 class UserToPerm(Base, BaseModel):
     __tablename__ = 'user_to_perm'
     __table_args__ = (
         UniqueConstraint('user_id', 'permission_id'),
         {'extend_existing': True}
     )
     user_to_perm_id = Column("user_to_perm_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
     permission_id = Column("permission_id", Integer(), ForeignKey('permissions.permission_id'), nullable=False, unique=None, default=None)
 
     user = relationship('User')
     permission = relationship('Permission', lazy='joined')
 
 
 class UsersGroupRepoToPerm(Base, BaseModel):
     __tablename__ = 'users_group_repo_to_perm'
     __table_args__ = (
         UniqueConstraint('repository_id', 'users_group_id', 'permission_id'),
         {'extend_existing': True}
     )
     users_group_to_perm_id = Column("users_group_to_perm_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     users_group_id = Column("users_group_id", Integer(), ForeignKey('users_groups.users_group_id'), nullable=False, unique=None, default=None)
     permission_id = Column("permission_id", Integer(), ForeignKey('permissions.permission_id'), nullable=False, unique=None, default=None)
     repository_id = Column("repository_id", Integer(), ForeignKey('repositories.repo_id'), nullable=False, unique=None, default=None)
 
     users_group = relationship('UsersGroup')
     permission = relationship('Permission')
     repository = relationship('Repository')
 
     @classmethod
     def create(cls, users_group, repository, permission):
         n = cls()
         n.users_group = users_group
         n.repository = repository
         n.permission = permission
         Session.add(n)
         return n
 
     def __repr__(self):
         return '<userGroup:%s => %s >' % (self.users_group, self.repository)
 
 
 class UsersGroupToPerm(Base, BaseModel):
     __tablename__ = 'users_group_to_perm'
     __table_args__ = (
         UniqueConstraint('users_group_id', 'permission_id',),
         {'extend_existing': True}
     )
     users_group_to_perm_id = Column("users_group_to_perm_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     users_group_id = Column("users_group_id", Integer(), ForeignKey('users_groups.users_group_id'), nullable=False, unique=None, default=None)
     permission_id = Column("permission_id", Integer(), ForeignKey('permissions.permission_id'), nullable=False, unique=None, default=None)
 
     users_group = relationship('UsersGroup')
     permission = relationship('Permission')
 
 
 class UserRepoGroupToPerm(Base, BaseModel):
     __tablename__ = 'user_repo_group_to_perm'
     __table_args__ = (
         UniqueConstraint('user_id', 'group_id', 'permission_id'),
         {'extend_existing': True}
     )
 
     group_to_perm_id = Column("group_to_perm_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
     group_id = Column("group_id", Integer(), ForeignKey('groups.group_id'), nullable=False, unique=None, default=None)
     permission_id = Column("permission_id", Integer(), ForeignKey('permissions.permission_id'), nullable=False, unique=None, default=None)
 
     user = relationship('User')
     group = relationship('RepoGroup')
     permission = relationship('Permission')
 
 
 class UsersGroupRepoGroupToPerm(Base, BaseModel):
     __tablename__ = 'users_group_repo_group_to_perm'
     __table_args__ = (
         UniqueConstraint('users_group_id', 'group_id'),
         {'extend_existing': True}
     )
 
     users_group_repo_group_to_perm_id = Column("users_group_repo_group_to_perm_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     users_group_id = Column("users_group_id", Integer(), ForeignKey('users_groups.users_group_id'), nullable=False, unique=None, default=None)
     group_id = Column("group_id", Integer(), ForeignKey('groups.group_id'), nullable=False, unique=None, default=None)
     permission_id = Column("permission_id", Integer(), ForeignKey('permissions.permission_id'), nullable=False, unique=None, default=None)
 
     users_group = relationship('UsersGroup')
     permission = relationship('Permission')
     group = relationship('RepoGroup')
 
 
 class Statistics(Base, BaseModel):
     __tablename__ = 'statistics'
     __table_args__ = (UniqueConstraint('repository_id'), {'extend_existing': True})
     stat_id = Column("stat_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     repository_id = Column("repository_id", Integer(), ForeignKey('repositories.repo_id'), nullable=False, unique=True, default=None)
     stat_on_revision = Column("stat_on_revision", Integer(), nullable=False)
     commit_activity = Column("commit_activity", LargeBinary(1000000), nullable=False)#JSON data
     commit_activity_combined = Column("commit_activity_combined", LargeBinary(), nullable=False)#JSON data
     languages = Column("languages", LargeBinary(1000000), nullable=False)#JSON data
 
     repository = relationship('Repository', single_parent=True)
 
 
 class UserFollowing(Base, BaseModel):
     __tablename__ = 'user_followings'
     __table_args__ = (
         UniqueConstraint('user_id', 'follows_repository_id'),
         UniqueConstraint('user_id', 'follows_user_id'),
         {'extend_existing': True}
     )
 
     user_following_id = Column("user_following_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     user_id = Column("user_id", Integer(), ForeignKey('users.user_id'), nullable=False, unique=None, default=None)
     follows_repo_id = Column("follows_repository_id", Integer(), ForeignKey('repositories.repo_id'), nullable=True, unique=None, default=None)
     follows_user_id = Column("follows_user_id", Integer(), ForeignKey('users.user_id'), nullable=True, unique=None, default=None)
     follows_from = Column('follows_from', DateTime(timezone=False), nullable=True, unique=None, default=datetime.datetime.now)
 
     user = relationship('User', primaryjoin='User.user_id==UserFollowing.user_id')
 
     follows_user = relationship('User', primaryjoin='User.user_id==UserFollowing.follows_user_id')
     follows_repository = relationship('Repository', order_by='Repository.repo_name')
 
     @classmethod
     def get_repo_followers(cls, repo_id):
         return cls.query().filter(cls.follows_repo_id == repo_id)
 
 
 class CacheInvalidation(Base, BaseModel):
     __tablename__ = 'cache_invalidation'
     __table_args__ = (UniqueConstraint('cache_key'), {'extend_existing': True})
     cache_id = Column("cache_id", Integer(), nullable=False, unique=True, default=None, primary_key=True)
     cache_key = Column("cache_key", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     cache_args = Column("cache_args", String(length=255, convert_unicode=False, assert_unicode=None), nullable=True, unique=None, default=None)
     cache_active = Column("cache_active", Boolean(), nullable=True, unique=None, default=False)
 
     def __init__(self, cache_key, cache_args=''):
         self.cache_key = cache_key
         self.cache_args = cache_args
         self.cache_active = False
 
     def __repr__(self):
         return "<%s('%s:%s')>" % (self.__class__.__name__,
                                   self.cache_id, self.cache_key)
 
     @classmethod
     def _get_key(cls, key):
         """
         Wrapper for generating a key
 
         :param key:
         """
         import rhodecode
         prefix = ''
         iid = rhodecode.CONFIG.get('instance_id')
         if iid:
-            prefix = iid 
+            prefix = iid
         return "%s%s" % (prefix, key)
 
     @classmethod
     def get_by_key(cls, key):
         return cls.query().filter(cls.cache_key == key).scalar()
 
     @classmethod
     def invalidate(cls, key):
         """
         Returns Invalidation object if this given key should be invalidated
         None otherwise. `cache_active = False` means that this cache
         state is not valid and needs to be invalidated
 
         :param key:
         """
         return cls.query()\
                 .filter(CacheInvalidation.cache_key == key)\
                 .filter(CacheInvalidation.cache_active == False)\
                 .scalar()
 
     @classmethod
     def set_invalidate(cls, key):
         """
         Mark this Cache key for invalidation
 
         :param key:
         """
 
         log.debug('marking %s for invalidation' % key)
         inv_obj = Session.query(cls)\
             .filter(cls.cache_key == key).scalar()
         if inv_obj:
             inv_obj.cache_active = False
         else:
             log.debug('cache key not found in invalidation db -> creating one')
             inv_obj = CacheInvalidation(key)
 
         try:
             Session.add(inv_obj)
             Session.commit()
         except Exception:
             log.error(traceback.format_exc())
             Session.rollback()
 
     @classmethod
     def set_valid(cls, key):
         """
         Mark this cache key as active and currently cached
 
         :param key:
         """
         inv_obj = cls.get_by_key(key)
         inv_obj.cache_active = True
         Session.add(inv_obj)
         Session.commit()
 
 
 class ChangesetComment(Base, BaseModel):
     __tablename__ = 'changeset_comments'
     __table_args__ = ({'extend_existing': True},)
     comment_id = Column('comment_id', Integer(), nullable=False, primary_key=True)
     repo_id = Column('repo_id', Integer(), ForeignKey('repositories.repo_id'), nullable=False)
     revision = Column('revision', String(40), nullable=False)
     line_no = Column('line_no', Unicode(10), nullable=True)
     f_path = Column('f_path', Unicode(1000), nullable=True)
     user_id = Column('user_id', Integer(), ForeignKey('users.user_id'), nullable=False)
     text = Column('text', Unicode(25000), nullable=False)
     modified_at = Column('modified_at', DateTime(), nullable=False, default=datetime.datetime.now)
 
     author = relationship('User', lazy='joined')
     repo = relationship('Repository')
 
     @classmethod
     def get_users(cls, revision):
         """
         Returns user associated with this changesetComment. ie those
         who actually commented
 
         :param cls:
         :param revision:
         """
         return Session.query(User)\
                 .filter(cls.revision == revision)\
                 .join(ChangesetComment.author).all()
 
 
 class Notification(Base, BaseModel):
     __tablename__ = 'notifications'
     __table_args__ = ({'extend_existing': True},)
 
     TYPE_CHANGESET_COMMENT = u'cs_comment'
     TYPE_MESSAGE = u'message'
     TYPE_MENTION = u'mention'
     TYPE_REGISTRATION = u'registration'
 
     notification_id = Column('notification_id', Integer(), nullable=False, primary_key=True)
     subject = Column('subject', Unicode(512), nullable=True)
     body = Column('body', Unicode(50000), nullable=True)
     created_by = Column("created_by", Integer(), ForeignKey('users.user_id'), nullable=True)
     created_on = Column('created_on', DateTime(timezone=False), nullable=False, default=datetime.datetime.now)
     type_ = Column('type', Unicode(256))
 
     created_by_user = relationship('User')
     notifications_to_users = relationship('UserNotification', lazy='joined',
                                           cascade="all, delete, delete-orphan")
 
     @property
     def recipients(self):
         return [x.user for x in UserNotification.query()\
                 .filter(UserNotification.notification == self).all()]
 
     @classmethod
     def create(cls, created_by, subject, body, recipients, type_=None):
         if type_ is None:
             type_ = Notification.TYPE_MESSAGE
 
         notification = cls()
         notification.created_by_user = created_by
         notification.subject = subject
         notification.body = body
         notification.type_ = type_
         notification.created_on = datetime.datetime.now()
 
         for u in recipients:
             assoc = UserNotification()
             assoc.notification = notification
             u.notifications.append(assoc)
         Session.add(notification)
         return notification
 
     @property
     def description(self):
         from rhodecode.model.notification import NotificationModel
         return NotificationModel().make_description(self)
 
 
 class UserNotification(Base, BaseModel):
     __tablename__ = 'user_to_notification'
     __table_args__ = (
         UniqueConstraint('user_id', 'notification_id'),
         {'extend_existing': True}
     )
     user_id = Column('user_id', Integer(), ForeignKey('users.user_id'), primary_key=True)
     notification_id = Column("notification_id", Integer(), ForeignKey('notifications.notification_id'), primary_key=True)
     read = Column('read', Boolean, default=False)
     sent_on = Column('sent_on', DateTime(timezone=False), nullable=True, unique=None)
 
     user = relationship('User', lazy="joined")
     notification = relationship('Notification', lazy="joined",
                                 order_by=lambda: Notification.created_on.desc(),)
 
     def mark_as_read(self):
         self.read = True
         Session.add(self)
 
 
 class DbMigrateVersion(Base, BaseModel):
     __tablename__ = 'db_migrate_version'
     __table_args__ = {'extend_existing': True}
     repository_id = Column('repository_id', String(250), primary_key=True)
     repository_path = Column('repository_path', Text)
     version = Column('version', Integer)

rhodecode/model/forms.py

@@ -1,758 +1,773 b''
 """ this is forms validation classes
 http://formencode.org/module-formencode.validators.html
 for list off all availible validators
 
 we can create our own validators
 
 The table below outlines the options which can be used in a schema in addition to the validators themselves
 pre_validators          []     These validators will be applied before the schema
 chained_validators      []     These validators will be applied after the schema
 allow_extra_fields      False     If True, then it is not an error when keys that aren't associated with a validator are present
 filter_extra_fields     False     If True, then keys that aren't associated with a validator are removed
 if_key_missing          NoDefault If this is given, then any keys that aren't available but are expected will be replaced with this value (and then validated). This does not override a present .if_missing attribute on validators. NoDefault is a special FormEncode class to mean that no default values has been specified and therefore missing keys shouldn't take a default value.
 ignore_key_missing      False     If True, then missing keys will be missing in the result, if the validator doesn't have .if_missing on it already
 
 
 <name> = formencode.validators.<name of validator>
 <name> must equal form name
 list=[1,2,3,4,5]
 for SELECT use formencode.All(OneOf(list), Int())
 
 """
 import os
 import re
 import logging
 import traceback
 
 import formencode
 from formencode import All
 from formencode.validators import UnicodeString, OneOf, Int, Number, Regex, \
     Email, Bool, StringBoolean, Set
 
 from pylons.i18n.translation import _
 from webhelpers.pylonslib.secure_form import authentication_token
 
 from rhodecode.config.routing import ADMIN_PREFIX
 from rhodecode.lib.utils import repo_name_slug
 from rhodecode.lib.auth import authenticate, get_crypt_password
 from rhodecode.lib.exceptions import LdapImportError
 from rhodecode.model.db import User, UsersGroup, RepoGroup, Repository
 from rhodecode import BACKENDS
 
 log = logging.getLogger(__name__)
 
 
 #this is needed to translate the messages using _() in validators
 class State_obj(object):
     _ = staticmethod(_)
 
 
 #==============================================================================
 # VALIDATORS
 #==============================================================================
 class ValidAuthToken(formencode.validators.FancyValidator):
     messages = {'invalid_token': _('Token mismatch')}
 
     def validate_python(self, value, state):
 
         if value != authentication_token():
             raise formencode.Invalid(
                 self.message('invalid_token',
                              state, search_number=value),
                 value,
                 state
             )
 
 
 def ValidUsername(edit, old_data):
     class _ValidUsername(formencode.validators.FancyValidator):
 
         def validate_python(self, value, state):
             if value in ['default', 'new_user']:
                 raise formencode.Invalid(_('Invalid username'), value, state)
             #check if user is unique
             old_un = None
             if edit:
                 old_un = User.get(old_data.get('user_id')).username
 
             if old_un != value or not edit:
                 if User.get_by_username(value, case_insensitive=True):
                     raise formencode.Invalid(_('This username already '
                                                'exists') , value, state)
 
             if re.match(r'^[a-zA-Z0-9]{1}[a-zA-Z0-9\-\_\.]+$', value) is None:
                 raise formencode.Invalid(
                     _('Username may only contain alphanumeric characters '
                       'underscores, periods or dashes and must begin with '
                       'alphanumeric character'),
                     value,
                     state
                 )
 
     return _ValidUsername
 
 
 def ValidUsersGroup(edit, old_data):
 
     class _ValidUsersGroup(formencode.validators.FancyValidator):
 
         def validate_python(self, value, state):
             if value in ['default']:
                 raise formencode.Invalid(_('Invalid group name'), value, state)
             #check if group is unique
             old_ugname = None
             if edit:
                 old_ugname = UsersGroup.get(
                             old_data.get('users_group_id')).users_group_name
 
             if old_ugname != value or not edit:
                 if UsersGroup.get_by_group_name(value, cache=False,
                                                case_insensitive=True):
                     raise formencode.Invalid(_('This users group '
                                                'already exists'), value,
                                              state)
 
             if re.match(r'^[a-zA-Z0-9]{1}[a-zA-Z0-9\-\_\.]+$', value) is None:
                 raise formencode.Invalid(
                     _('RepoGroup name may only contain  alphanumeric characters '
                       'underscores, periods or dashes and must begin with '
                       'alphanumeric character'),
                     value,
                     state
                 )
 
     return _ValidUsersGroup
 
 
 def ValidReposGroup(edit, old_data):
     class _ValidReposGroup(formencode.validators.FancyValidator):
 
         def validate_python(self, value, state):
             # TODO WRITE VALIDATIONS
             group_name = value.get('group_name')
             group_parent_id = value.get('group_parent_id')
 
             # slugify repo group just in case :)
             slug = repo_name_slug(group_name)
 
             # check for parent of self
             parent_of_self = lambda: (
                 old_data['group_id'] == int(group_parent_id)
                 if group_parent_id else False
             )
             if edit and parent_of_self():
                     e_dict = {
                         'group_parent_id': _('Cannot assign this group as parent')
                     }
                     raise formencode.Invalid('', value, state,
                                              error_dict=e_dict)
 
             old_gname = None
             if edit:
                 old_gname = RepoGroup.get(old_data.get('group_id')).group_name
 
             if old_gname != group_name or not edit:
 
                 # check group
                 gr = RepoGroup.query()\
                       .filter(RepoGroup.group_name == slug)\
                       .filter(RepoGroup.group_parent_id == group_parent_id)\
                       .scalar()
 
                 if gr:
                     e_dict = {
                         'group_name': _('This group already exists')
                     }
                     raise formencode.Invalid('', value, state,
                                              error_dict=e_dict)
 
                 # check for same repo
                 repo = Repository.query()\
                       .filter(Repository.repo_name == slug)\
                       .scalar()
 
                 if repo:
                     e_dict = {
                         'group_name': _('Repository with this name already exists')
                     }
                     raise formencode.Invalid('', value, state,
                                              error_dict=e_dict)
 
     return _ValidReposGroup
 
 
 class ValidPassword(formencode.validators.FancyValidator):
 
     def to_python(self, value, state):
 
         if not value:
             return
 
         if value.get('password'):
             try:
                 value['password'] = get_crypt_password(value['password'])
             except UnicodeEncodeError:
                 e_dict = {'password': _('Invalid characters in password')}
                 raise formencode.Invalid('', value, state, error_dict=e_dict)
 
         if value.get('password_confirmation'):
             try:
                 value['password_confirmation'] = \
                     get_crypt_password(value['password_confirmation'])
             except UnicodeEncodeError:
                 e_dict = {
                     'password_confirmation': _('Invalid characters in password')
                 }
                 raise formencode.Invalid('', value, state, error_dict=e_dict)
 
         if value.get('new_password'):
             try:
                 value['new_password'] = \
                     get_crypt_password(value['new_password'])
             except UnicodeEncodeError:
                 e_dict = {'new_password': _('Invalid characters in password')}
                 raise formencode.Invalid('', value, state, error_dict=e_dict)
 
         return value
 
 
 class ValidPasswordsMatch(formencode.validators.FancyValidator):
 
     def validate_python(self, value, state):
 
         pass_val = value.get('password') or value.get('new_password')
         if pass_val != value['password_confirmation']:
             e_dict = {'password_confirmation':
                    _('Passwords do not match')}
             raise formencode.Invalid('', value, state, error_dict=e_dict)
 
 
 class ValidAuth(formencode.validators.FancyValidator):
     messages = {
         'invalid_password':_('invalid password'),
         'invalid_login':_('invalid user name'),
         'disabled_account':_('Your account is disabled')
     }
 
     # error mapping
     e_dict = {'username': messages['invalid_login'],
               'password': messages['invalid_password']}
     e_dict_disable = {'username': messages['disabled_account']}
 
     def validate_python(self, value, state):
         password = value['password']
         username = value['username']
         user = User.get_by_username(username)
 
         if authenticate(username, password):
             return value
         else:
             if user and user.active is False:
                 log.warning('user %s is disabled' % username)
                 raise formencode.Invalid(
                     self.message('disabled_account',
                     state=State_obj),
                     value, state,
                     error_dict=self.e_dict_disable
                 )
             else:
                 log.warning('user %s failed to authenticate' % username)
                 raise formencode.Invalid(
                     self.message('invalid_password',
                     state=State_obj), value, state,
                     error_dict=self.e_dict
                 )
 
 
 class ValidRepoUser(formencode.validators.FancyValidator):
 
     def to_python(self, value, state):
         try:
             User.query().filter(User.active == True)\
                 .filter(User.username == value).one()
         except Exception:
             raise formencode.Invalid(_('This username is not valid'),
                                      value, state)
         return value
 
 
 def ValidRepoName(edit, old_data):
     class _ValidRepoName(formencode.validators.FancyValidator):
         def to_python(self, value, state):
 
             repo_name = value.get('repo_name')
 
             slug = repo_name_slug(repo_name)
             if slug in [ADMIN_PREFIX, '']:
                 e_dict = {'repo_name': _('This repository name is disallowed')}
                 raise formencode.Invalid('', value, state, error_dict=e_dict)
 
             if value.get('repo_group'):
                 gr = RepoGroup.get(value.get('repo_group'))
                 group_path = gr.full_path
                 # value needs to be aware of group name in order to check
                 # db key This is an actual just the name to store in the
                 # database
                 repo_name_full = group_path + RepoGroup.url_sep() + repo_name
 
             else:
                 group_path = ''
                 repo_name_full = repo_name
 
             value['repo_name_full'] = repo_name_full
             rename = old_data.get('repo_name') != repo_name_full
             create = not edit
             if  rename or create:
 
                 if group_path != '':
                     if Repository.get_by_repo_name(repo_name_full):
                         e_dict = {
                             'repo_name': _('This repository already exists in '
                                            'a group "%s"') % gr.group_name
                         }
                         raise formencode.Invalid('', value, state,
                                                  error_dict=e_dict)
                 elif RepoGroup.get_by_group_name(repo_name_full):
                         e_dict = {
                             'repo_name': _('There is a group with this name '
                                            'already "%s"') % repo_name_full
                         }
                         raise formencode.Invalid('', value, state,
                                                  error_dict=e_dict)
 
                 elif Repository.get_by_repo_name(repo_name_full):
                         e_dict = {'repo_name': _('This repository '
                                                 'already exists')}
                         raise formencode.Invalid('', value, state,
                                                  error_dict=e_dict)
 
             return value
 
     return _ValidRepoName
 
 
 def ValidForkName(*args, **kwargs):
     return ValidRepoName(*args, **kwargs)
 
 
 def SlugifyName():
     class _SlugifyName(formencode.validators.FancyValidator):
 
         def to_python(self, value, state):
             return repo_name_slug(value)
 
     return _SlugifyName
 
 
 def ValidCloneUri():
-    from mercurial.httprepo import httprepository, httpsrepository
     from rhodecode.lib.utils import make_ui
 
+    def url_handler(repo_type, url, proto, ui=None):
+        if repo_type == 'hg':
+            from mercurial.httprepo import httprepository, httpsrepository
+            if proto == 'https':
+                httpsrepository(make_ui('db'), url).capabilities
+            elif proto == 'http':
+                httprepository(make_ui('db'), url).capabilities
+        elif repo_type == 'git':
+            #TODO: write a git url validator
+            pass
+
     class _ValidCloneUri(formencode.validators.FancyValidator):
 
         def to_python(self, value, state):
-            if not value:
+
+            repo_type = value.get('repo_type')
+            url = value.get('clone_uri')
+            e_dict = {'clone_uri': _('invalid clone url')}
+
+            if not url:
                 pass
-            elif value.startswith('https'):
+            elif url.startswith('https'):
                 try:
-                    httpsrepository(make_ui('db'), value).capabilities
+                    url_handler(repo_type, url, 'https', make_ui('db'))
                 except Exception:
                     log.error(traceback.format_exc())
-                    raise formencode.Invalid(_('invalid clone url'), value,
-                                             state)
-            elif value.startswith('http'):
+                    raise formencode.Invalid('', value, state, error_dict=e_dict)
+            elif url.startswith('http'):
                 try:
-                    httprepository(make_ui('db'), value).capabilities
+                    url_handler(repo_type, url, 'http', make_ui('db'))
                 except Exception:
                     log.error(traceback.format_exc())
-                    raise formencode.Invalid(_('invalid clone url'), value,
-                                             state)
+                    raise formencode.Invalid('', value, state, error_dict=e_dict)
             else:
-                raise formencode.Invalid(_('Invalid clone url, provide a '
-                                           'valid clone http\s url'), value,
-                                         state)
+                e_dict = {'clone_uri': _('Invalid clone url, provide a '
+                                         'valid clone http\s url')}
+                raise formencode.Invalid('', value, state, error_dict=e_dict)
+
             return value
 
     return _ValidCloneUri
 
 
 def ValidForkType(old_data):
     class _ValidForkType(formencode.validators.FancyValidator):
 
         def to_python(self, value, state):
             if old_data['repo_type'] != value:
                 raise formencode.Invalid(_('Fork have to be the same '
                                            'type as original'), value, state)
 
             return value
     return _ValidForkType
 
 
 def ValidPerms(type_='repo'):
     if type_ == 'group':
         EMPTY_PERM = 'group.none'
     elif type_ == 'repo':
         EMPTY_PERM = 'repository.none'
 
     class _ValidPerms(formencode.validators.FancyValidator):
         messages = {
             'perm_new_member_name':
                 _('This username or users group name is not valid')
         }
 
         def to_python(self, value, state):
             perms_update = []
             perms_new = []
             # build a list of permission to update and new permission to create
             for k, v in value.items():
                 # means new added member to permissions
                 if k.startswith('perm_new_member'):
                     new_perm = value.get('perm_new_member', False)
                     new_member = value.get('perm_new_member_name', False)
                     new_type = value.get('perm_new_member_type')
 
                     if new_member and new_perm:
                         if (new_member, new_perm, new_type) not in perms_new:
                             perms_new.append((new_member, new_perm, new_type))
                 elif k.startswith('u_perm_') or k.startswith('g_perm_'):
                     member = k[7:]
                     t = {'u': 'user',
                          'g': 'users_group'
                     }[k[0]]
                     if member == 'default':
                         if value.get('private'):
                             # set none for default when updating to private repo
                             v = EMPTY_PERM
                     perms_update.append((member, v, t))
 
             value['perms_updates'] = perms_update
             value['perms_new'] = perms_new
 
             # update permissions
             for k, v, t in perms_new:
                 try:
                     if t is 'user':
                         self.user_db = User.query()\
                             .filter(User.active == True)\
                             .filter(User.username == k).one()
                     if t is 'users_group':
                         self.user_db = UsersGroup.query()\
                             .filter(UsersGroup.users_group_active == True)\
                             .filter(UsersGroup.users_group_name == k).one()
 
                 except Exception:
                     msg = self.message('perm_new_member_name',
                                          state=State_obj)
                     raise formencode.Invalid(
                         msg, value, state, error_dict={'perm_new_member_name': msg}
                     )
             return value
     return _ValidPerms
 
 
 class ValidSettings(formencode.validators.FancyValidator):
 
     def to_python(self, value, state):
         # settings  form can't edit user
         if 'user' in value:
             del['value']['user']
         return value
 
 
 class ValidPath(formencode.validators.FancyValidator):
     def to_python(self, value, state):
 
         if not os.path.isdir(value):
             msg = _('This is not a valid path')
             raise formencode.Invalid(msg, value, state,
                                      error_dict={'paths_root_path': msg})
         return value
 
 
 def UniqSystemEmail(old_data):
     class _UniqSystemEmail(formencode.validators.FancyValidator):
         def to_python(self, value, state):
             value = value.lower()
             if old_data.get('email', '').lower() != value:
                 user = User.get_by_email(value, case_insensitive=True)
                 if user:
                     raise formencode.Invalid(
                         _("This e-mail address is already taken"), value, state
                     )
             return value
 
     return _UniqSystemEmail
 
 
 class ValidSystemEmail(formencode.validators.FancyValidator):
     def to_python(self, value, state):
         value = value.lower()
         user = User.get_by_email(value, case_insensitive=True)
         if  user is None:
             raise formencode.Invalid(
                 _("This e-mail address doesn't exist."), value, state
             )
 
         return value
 
 
 class LdapLibValidator(formencode.validators.FancyValidator):
 
     def to_python(self, value, state):
 
         try:
             import ldap
         except ImportError:
             raise LdapImportError
         return value
 
 
 class AttrLoginValidator(formencode.validators.FancyValidator):
 
     def to_python(self, value, state):
 
         if not value or not isinstance(value, (str, unicode)):
             raise formencode.Invalid(
                 _("The LDAP Login attribute of the CN must be specified - "
                   "this is the name of the attribute that is equivalent "
                   "to 'username'"), value, state
             )
 
         return value
 
 
 #==============================================================================
 # FORMS
 #==============================================================================
 class LoginForm(formencode.Schema):
     allow_extra_fields = True
     filter_extra_fields = True
     username = UnicodeString(
         strip=True,
         min=1,
         not_empty=True,
         messages={
            'empty': _('Please enter a login'),
            'tooShort': _('Enter a value %(min)i characters long or more')}
     )
 
     password = UnicodeString(
         strip=True,
         min=3,
         not_empty=True,
         messages={
             'empty': _('Please enter a password'),
             'tooShort': _('Enter %(min)i characters or more')}
     )
 
     remember = StringBoolean(if_missing=False)
 
     chained_validators = [ValidAuth]
 
 
 def UserForm(edit=False, old_data={}):
     class _UserForm(formencode.Schema):
         allow_extra_fields = True
         filter_extra_fields = True
         username = All(UnicodeString(strip=True, min=1, not_empty=True),
                        ValidUsername(edit, old_data))
         if edit:
             new_password = All(UnicodeString(strip=True, min=6, not_empty=False))
             password_confirmation = All(UnicodeString(strip=True, min=6,
                                                       not_empty=False))
             admin = StringBoolean(if_missing=False)
         else:
             password = All(UnicodeString(strip=True, min=6, not_empty=True))
             password_confirmation = All(UnicodeString(strip=True, min=6,
                                                       not_empty=False))
 
         active = StringBoolean(if_missing=False)
         name = UnicodeString(strip=True, min=1, not_empty=False)
         lastname = UnicodeString(strip=True, min=1, not_empty=False)
         email = All(Email(not_empty=True), UniqSystemEmail(old_data))
 
         chained_validators = [ValidPasswordsMatch, ValidPassword]
 
     return _UserForm
 
 
 def UsersGroupForm(edit=False, old_data={}, available_members=[]):
     class _UsersGroupForm(formencode.Schema):
         allow_extra_fields = True
         filter_extra_fields = True
 
         users_group_name = All(UnicodeString(strip=True, min=1, not_empty=True),
                        ValidUsersGroup(edit, old_data))
 
         users_group_active = StringBoolean(if_missing=False)
 
         if edit:
             users_group_members = OneOf(available_members, hideList=False,
                                         testValueList=True,
                                         if_missing=None, not_empty=False)
 
     return _UsersGroupForm
 
 
 def ReposGroupForm(edit=False, old_data={}, available_groups=[]):
     class _ReposGroupForm(formencode.Schema):
         allow_extra_fields = True
         filter_extra_fields = False
 
         group_name = All(UnicodeString(strip=True, min=1, not_empty=True),
                                SlugifyName())
         group_description = UnicodeString(strip=True, min=1,
                                                 not_empty=True)
         group_parent_id = OneOf(available_groups, hideList=False,
                                         testValueList=True,
                                         if_missing=None, not_empty=False)
 
         chained_validators = [ValidReposGroup(edit, old_data), ValidPerms('group')]
 
     return _ReposGroupForm
 
 
 def RegisterForm(edit=False, old_data={}):
     class _RegisterForm(formencode.Schema):
         allow_extra_fields = True
         filter_extra_fields = True
         username = All(ValidUsername(edit, old_data),
                        UnicodeString(strip=True, min=1, not_empty=True))
         password = All(UnicodeString(strip=True, min=6, not_empty=True))
         password_confirmation = All(UnicodeString(strip=True, min=6, not_empty=True))
         active = StringBoolean(if_missing=False)
         name = UnicodeString(strip=True, min=1, not_empty=False)
         lastname = UnicodeString(strip=True, min=1, not_empty=False)
         email = All(Email(not_empty=True), UniqSystemEmail(old_data))
 
         chained_validators = [ValidPasswordsMatch, ValidPassword]
 
     return _RegisterForm
 
 
 def PasswordResetForm():
     class _PasswordResetForm(formencode.Schema):
         allow_extra_fields = True
         filter_extra_fields = True
         email = All(ValidSystemEmail(), Email(not_empty=True))
     return _PasswordResetForm
 
 
 def RepoForm(edit=False, old_data={}, supported_backends=BACKENDS.keys(),
              repo_groups=[]):
     class _RepoForm(formencode.Schema):
         allow_extra_fields = True
         filter_extra_fields = False
         repo_name = All(UnicodeString(strip=True, min=1, not_empty=True),
                         SlugifyName())
-        clone_uri = All(UnicodeString(strip=True, min=1, not_empty=False),
-                        ValidCloneUri()())
+        clone_uri = All(UnicodeString(strip=True, min=1, not_empty=False))
         repo_group = OneOf(repo_groups, hideList=True)
         repo_type = OneOf(supported_backends)
         description = UnicodeString(strip=True, min=1, not_empty=True)
         private = StringBoolean(if_missing=False)
         enable_statistics = StringBoolean(if_missing=False)
         enable_downloads = StringBoolean(if_missing=False)
 
         if edit:
             #this is repo owner
             user = All(UnicodeString(not_empty=True), ValidRepoUser)
 
-        chained_validators = [ValidRepoName(edit, old_data), ValidPerms()]
+        chained_validators = [ValidCloneUri()(),
+                              ValidRepoName(edit, old_data),
+                              ValidPerms()]
     return _RepoForm
 
 
 def RepoForkForm(edit=False, old_data={}, supported_backends=BACKENDS.keys(),
                  repo_groups=[]):
     class _RepoForkForm(formencode.Schema):
         allow_extra_fields = True
         filter_extra_fields = False
         repo_name = All(UnicodeString(strip=True, min=1, not_empty=True),
                         SlugifyName())
         repo_group = OneOf(repo_groups, hideList=True)
         repo_type = All(ValidForkType(old_data), OneOf(supported_backends))
         description = UnicodeString(strip=True, min=1, not_empty=True)
         private = StringBoolean(if_missing=False)
         copy_permissions = StringBoolean(if_missing=False)
         update_after_clone = StringBoolean(if_missing=False)
         fork_parent_id = UnicodeString()
         chained_validators = [ValidForkName(edit, old_data)]
 
     return _RepoForkForm
 
 
 def RepoSettingsForm(edit=False, old_data={}, supported_backends=BACKENDS.keys(),
                      repo_groups=[]):
     class _RepoForm(formencode.Schema):
         allow_extra_fields = True
         filter_extra_fields = False
         repo_name = All(UnicodeString(strip=True, min=1, not_empty=True),
                         SlugifyName())
         description = UnicodeString(strip=True, min=1, not_empty=True)
         repo_group = OneOf(repo_groups, hideList=True)
         private = StringBoolean(if_missing=False)
 
         chained_validators = [ValidRepoName(edit, old_data), ValidPerms(),
                               ValidSettings]
     return _RepoForm
 
 
 def ApplicationSettingsForm():
     class _ApplicationSettingsForm(formencode.Schema):
         allow_extra_fields = True
         filter_extra_fields = False
         rhodecode_title = UnicodeString(strip=True, min=1, not_empty=True)
         rhodecode_realm = UnicodeString(strip=True, min=1, not_empty=True)
         rhodecode_ga_code = UnicodeString(strip=True, min=1, not_empty=False)
 
     return _ApplicationSettingsForm
 
 
 def ApplicationUiSettingsForm():
     class _ApplicationUiSettingsForm(formencode.Schema):
         allow_extra_fields = True
         filter_extra_fields = False
         web_push_ssl = OneOf(['true', 'false'], if_missing='false')
         paths_root_path = All(ValidPath(), UnicodeString(strip=True, min=1, not_empty=True))
         hooks_changegroup_update = OneOf(['True', 'False'], if_missing=False)
         hooks_changegroup_repo_size = OneOf(['True', 'False'], if_missing=False)
         hooks_pretxnchangegroup_push_logger = OneOf(['True', 'False'], if_missing=False)
         hooks_preoutgoing_pull_logger = OneOf(['True', 'False'], if_missing=False)
 
     return _ApplicationUiSettingsForm
 
 
 def DefaultPermissionsForm(perms_choices, register_choices, create_choices):
     class _DefaultPermissionsForm(formencode.Schema):
         allow_extra_fields = True
         filter_extra_fields = True
         overwrite_default = StringBoolean(if_missing=False)
         anonymous = OneOf(['True', 'False'], if_missing=False)
         default_perm = OneOf(perms_choices)
         default_register = OneOf(register_choices)
         default_create = OneOf(create_choices)
 
     return _DefaultPermissionsForm
 
 
 def LdapSettingsForm(tls_reqcert_choices, search_scope_choices, tls_kind_choices):
     class _LdapSettingsForm(formencode.Schema):
         allow_extra_fields = True
         filter_extra_fields = True
         pre_validators = [LdapLibValidator]
         ldap_active = StringBoolean(if_missing=False)
         ldap_host = UnicodeString(strip=True,)
         ldap_port = Number(strip=True,)
         ldap_tls_kind = OneOf(tls_kind_choices)
         ldap_tls_reqcert = OneOf(tls_reqcert_choices)
         ldap_dn_user = UnicodeString(strip=True,)
         ldap_dn_pass = UnicodeString(strip=True,)
         ldap_base_dn = UnicodeString(strip=True,)
         ldap_filter = UnicodeString(strip=True,)
         ldap_search_scope = OneOf(search_scope_choices)
         ldap_attr_login = All(AttrLoginValidator, UnicodeString(strip=True,))
         ldap_attr_firstname = UnicodeString(strip=True,)
         ldap_attr_lastname = UnicodeString(strip=True,)
         ldap_attr_email = UnicodeString(strip=True,)
 
     return _LdapSettingsForm

rhodecode/model/repos_group.py

@@ -1,310 +1,310 b''
 # -*- coding: utf-8 -*-
 """
     rhodecode.model.user_group
     ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
     users groups model for RhodeCode
 
     :created_on: Jan 25, 2011
     :author: marcink
     :copyright: (C) 2011-2012 Marcin Kuzminski <[e-mail]>
     :license: GPLv3, see COPYING for more details.
 """
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 3 of the License, or
 # (at your option) any later version.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import os
 import logging
 import traceback
 import shutil
 
 from rhodecode.lib import LazyProperty
 
 from rhodecode.model import BaseModel
 from rhodecode.model.db import RepoGroup, RhodeCodeUi, UserRepoGroupToPerm, \
     User, Permission, UsersGroupRepoGroupToPerm, UsersGroup
 
 log = logging.getLogger(__name__)
 
 
 class ReposGroupModel(BaseModel):
 
     def __get_user(self, user):
         return self._get_instance(User, user, callback=User.get_by_username)
 
     def __get_users_group(self, users_group):
         return self._get_instance(UsersGroup, users_group,
                                   callback=UsersGroup.get_by_group_name)
 
     def __get_repos_group(self, repos_group):
         return self._get_instance(RepoGroup, repos_group,
                                   callback=RepoGroup.get_by_group_name)
 
     def __get_perm(self, permission):
         return self._get_instance(Permission, permission,
                                   callback=Permission.get_by_key)
 
     @LazyProperty
     def repos_path(self):
         """
         Get's the repositories root path from database
         """
 
         q = RhodeCodeUi.get_by_key('/').one()
         return q.ui_value
 
     def _create_default_perms(self, new_group):
         # create default permission
         repo_group_to_perm = UserRepoGroupToPerm()
         default_perm = 'group.read'
         for p in User.get_by_username('default').user_perms:
             if p.permission.permission_name.startswith('group.'):
                 default_perm = p.permission.permission_name
                 break
 
         repo_group_to_perm.permission_id = self.sa.query(Permission)\
                 .filter(Permission.permission_name == default_perm)\
                 .one().permission_id
 
         repo_group_to_perm.group = new_group
         repo_group_to_perm.user_id = User.get_by_username('default').user_id
 
         self.sa.add(repo_group_to_perm)
 
     def __create_group(self, group_name):
         """
         makes repositories group on filesystem
 
         :param repo_name:
         :param parent_id:
         """
 
         create_path = os.path.join(self.repos_path, group_name)
         log.debug('creating new group in %s' % create_path)
 
         if os.path.isdir(create_path):
             raise Exception('That directory already exists !')
 
         os.makedirs(create_path)
 
     def __rename_group(self, old, new):
         """
         Renames a group on filesystem
 
         :param group_name:
         """
 
         if old == new:
             log.debug('skipping group rename')
             return
 
         log.debug('renaming repos group from %s to %s' % (old, new))
 
         old_path = os.path.join(self.repos_path, old)
         new_path = os.path.join(self.repos_path, new)
 
         log.debug('renaming repos paths from %s to %s' % (old_path, new_path))
 
         if os.path.isdir(new_path):
             raise Exception('Was trying to rename to already '
                             'existing dir %s' % new_path)
         shutil.move(old_path, new_path)
 
     def __delete_group(self, group):
         """
         Deletes a group from a filesystem
 
         :param group: instance of group from database
         """
         paths = group.full_path.split(RepoGroup.url_sep())
         paths = os.sep.join(paths)
 
         rm_path = os.path.join(self.repos_path, paths)
         if os.path.isdir(rm_path):
             # delete only if that path really exists
             os.rmdir(rm_path)
 
     def create(self, group_name, group_description, parent, just_db=False):
         try:
             new_repos_group = RepoGroup()
             new_repos_group.group_description = group_description
             new_repos_group.parent_group = self.__get_repos_group(parent)
             new_repos_group.group_name = new_repos_group.get_new_name(group_name)
 
             self.sa.add(new_repos_group)
             self._create_default_perms(new_repos_group)
 
             if not just_db:
                 # we need to flush here, in order to check if database won't
                 # throw any exceptions, create filesystem dirs at the very end
                 self.sa.flush()
                 self.__create_group(new_repos_group.group_name)
 
             return new_repos_group
         except:
             log.error(traceback.format_exc())
             raise
 
     def update(self, repos_group_id, form_data):
 
         try:
             repos_group = RepoGroup.get(repos_group_id)
 
             # update permissions
             for member, perm, member_type in form_data['perms_updates']:
                 if member_type == 'user':
                     # this updates also current one if found
                     ReposGroupModel().grant_user_permission(
                         repos_group=repos_group, user=member, perm=perm
                     )
                 else:
                     ReposGroupModel().grant_users_group_permission(
                         repos_group=repos_group, group_name=member, perm=perm
                     )
             # set new permissions
             for member, perm, member_type in form_data['perms_new']:
                 if member_type == 'user':
                     ReposGroupModel().grant_user_permission(
                         repos_group=repos_group, user=member, perm=perm
                     )
                 else:
                     ReposGroupModel().grant_users_group_permission(
                         repos_group=repos_group, group_name=member, perm=perm
                     )
 
             old_path = repos_group.full_path
 
             # change properties
             repos_group.group_description = form_data['group_description']
             repos_group.parent_group = RepoGroup.get(form_data['group_parent_id'])
+            repos_group.group_parent_id = form_data['group_parent_id']
             repos_group.group_name = repos_group.get_new_name(form_data['group_name'])
-
             new_path = repos_group.full_path
 
             self.sa.add(repos_group)
 
-            self.__rename_group(old_path, new_path)
-
             # we need to get all repositories from this new group and
             # rename them accordingly to new group path
             for r in repos_group.repositories:
                 r.repo_name = r.get_new_name(r.just_name)
                 self.sa.add(r)
 
+            self.__rename_group(old_path, new_path)
+
             return repos_group
         except:
             log.error(traceback.format_exc())
             raise
 
     def delete(self, users_group_id):
         try:
             users_group = RepoGroup.get(users_group_id)
             self.sa.delete(users_group)
             self.__delete_group(users_group)
         except:
             log.error(traceback.format_exc())
             raise
 
     def grant_user_permission(self, repos_group, user, perm):
         """
         Grant permission for user on given repositories group, or update
         existing one if found
 
         :param repos_group: Instance of ReposGroup, repositories_group_id,
             or repositories_group name
         :param user: Instance of User, user_id or username
         :param perm: Instance of Permission, or permission_name
         """
 
         repos_group = self.__get_repos_group(repos_group)
         user = self.__get_user(user)
         permission = self.__get_perm(perm)
 
         # check if we have that permission already
         obj = self.sa.query(UserRepoGroupToPerm)\
             .filter(UserRepoGroupToPerm.user == user)\
             .filter(UserRepoGroupToPerm.group == repos_group)\
             .scalar()
         if obj is None:
             # create new !
             obj = UserRepoGroupToPerm()
         obj.group = repos_group
         obj.user = user
         obj.permission = permission
         self.sa.add(obj)
 
     def revoke_user_permission(self, repos_group, user):
         """
         Revoke permission for user on given repositories group
 
         :param repos_group: Instance of ReposGroup, repositories_group_id,
             or repositories_group name
         :param user: Instance of User, user_id or username
         """
 
         repos_group = self.__get_repos_group(repos_group)
         user = self.__get_user(user)
 
         obj = self.sa.query(UserRepoGroupToPerm)\
             .filter(UserRepoGroupToPerm.user == user)\
             .filter(UserRepoGroupToPerm.group == repos_group)\
             .one()
         self.sa.delete(obj)
 
     def grant_users_group_permission(self, repos_group, group_name, perm):
         """
         Grant permission for users group on given repositories group, or update
         existing one if found
 
         :param repos_group: Instance of ReposGroup, repositories_group_id,
             or repositories_group name
         :param group_name: Instance of UserGroup, users_group_id,
             or users group name
         :param perm: Instance of Permission, or permission_name
         """
         repos_group = self.__get_repos_group(repos_group)
         group_name = self.__get_users_group(group_name)
         permission = self.__get_perm(perm)
 
         # check if we have that permission already
         obj = self.sa.query(UsersGroupRepoGroupToPerm)\
             .filter(UsersGroupRepoGroupToPerm.group == repos_group)\
             .filter(UsersGroupRepoGroupToPerm.users_group == group_name)\
             .scalar()
 
         if obj is None:
             # create new
             obj = UsersGroupRepoGroupToPerm()
 
         obj.group = repos_group
         obj.users_group = group_name
         obj.permission = permission
         self.sa.add(obj)
 
     def revoke_users_group_permission(self, repos_group, group_name):
         """
         Revoke permission for users group on given repositories group
 
         :param repos_group: Instance of ReposGroup, repositories_group_id,
             or repositories_group name
         :param group_name: Instance of UserGroup, users_group_id,
             or users group name
         """
         repos_group = self.__get_repos_group(repos_group)
         group_name = self.__get_users_group(group_name)
 
         obj = self.sa.query(UsersGroupRepoGroupToPerm)\
             .filter(UsersGroupRepoGroupToPerm.group == repos_group)\
             .filter(UsersGroupRepoGroupToPerm.users_group == group_name)\
             .one()
         self.sa.delete(obj)

rhodecode/tests/functional/test_admin_users_groups.py

@@ -1,87 +1,134 b''
 from rhodecode.tests import *
-from rhodecode.model.db import UsersGroup
+from rhodecode.model.db import UsersGroup, UsersGroupToPerm, Permission
 
 TEST_USERS_GROUP = 'admins_test'
 
+
 class TestAdminUsersGroupsController(TestController):
 
     def test_index(self):
         response = self.app.get(url('users_groups'))
         # Test response...
 
     def test_index_as_xml(self):
         response = self.app.get(url('formatted_users_groups', format='xml'))
 
     def test_create(self):
         self.log_user()
         users_group_name = TEST_USERS_GROUP
         response = self.app.post(url('users_groups'),
-                                 {'users_group_name':users_group_name,
+                                 {'users_group_name': users_group_name,
                                   'active':True})
         response.follow()
 
         self.checkSessionFlash(response,
                                'created users group %s' % TEST_USERS_GROUP)
 
     def test_new(self):
         response = self.app.get(url('new_users_group'))
 
     def test_new_as_xml(self):
         response = self.app.get(url('formatted_new_users_group', format='xml'))
 
     def test_update(self):
         response = self.app.put(url('users_group', id=1))
 
     def test_update_browser_fakeout(self):
         response = self.app.post(url('users_group', id=1),
                                  params=dict(_method='put'))
 
     def test_delete(self):
         self.log_user()
         users_group_name = TEST_USERS_GROUP + 'another'
         response = self.app.post(url('users_groups'),
                                  {'users_group_name':users_group_name,
                                   'active':True})
         response.follow()
 
         self.checkSessionFlash(response,
                                'created users group %s' % users_group_name)
 
-
         gr = self.Session.query(UsersGroup)\
                            .filter(UsersGroup.users_group_name ==
                                    users_group_name).one()
 
         response = self.app.delete(url('users_group', id=gr.users_group_id))
 
         gr = self.Session.query(UsersGroup)\
                            .filter(UsersGroup.users_group_name ==
                                    users_group_name).scalar()
 
         self.assertEqual(gr, None)
 
+    def test_enable_repository_read_on_group(self):
+        self.log_user()
+        users_group_name = TEST_USERS_GROUP + 'another2'
+        response = self.app.post(url('users_groups'),
+                                 {'users_group_name': users_group_name,
+                                  'active':True})
+        response.follow()
+
+        ug = UsersGroup.get_by_group_name(users_group_name)
+        self.checkSessionFlash(response,
+                               'created users group %s' % users_group_name)
+
+        response = self.app.put(url('users_group_perm', id=ug.users_group_id),
+                                 {'create_repo_perm': True})
+
+        response.follow()
+        ug = UsersGroup.get_by_group_name(users_group_name)
+        p = Permission.get_by_key('hg.create.repository')
+        # check if user has this perm
+        perms = UsersGroupToPerm.query()\
+            .filter(UsersGroupToPerm.users_group == ug).all()
+        perms = [[x.__dict__['users_group_id'],
+                  x.__dict__['permission_id'],] for x in perms]
+        self.assertEqual(
+            perms,
+            [[ug.users_group_id, p.permission_id]]
+        )
+
+        # DELETE !
+        ug = UsersGroup.get_by_group_name(users_group_name)
+        ugid = ug.users_group_id
+        response = self.app.delete(url('users_group', id=ug.users_group_id))
+        response = response.follow()
+        gr = self.Session.query(UsersGroup)\
+                           .filter(UsersGroup.users_group_name ==
+                                   users_group_name).scalar()
+
+        self.assertEqual(gr, None)
+        p = Permission.get_by_key('hg.create.repository')
+        perms = UsersGroupToPerm.query()\
+            .filter(UsersGroupToPerm.users_group_id == ugid).all()
+        perms = [[x.__dict__['users_group_id'],
+                  x.__dict__['permission_id'],] for x in perms]
+        self.assertEqual(
+            perms,
+            []
+        )
 
     def test_delete_browser_fakeout(self):
         response = self.app.post(url('users_group', id=1),
                                  params=dict(_method='delete'))
 
     def test_show(self):
         response = self.app.get(url('users_group', id=1))
 
     def test_show_as_xml(self):
         response = self.app.get(url('formatted_users_group', id=1, format='xml'))
 
     def test_edit(self):
         response = self.app.get(url('edit_users_group', id=1))
 
     def test_edit_as_xml(self):
         response = self.app.get(url('formatted_edit_users_group', id=1, format='xml'))
 
     def test_assign_members(self):
         pass
 
     def test_add_create_permission(self):
         pass
 
     def test_revoke_members(self):
         pass

rhodecode/tests/test_models.py

@@ -1,555 +1,580 b''
 import os
 import unittest
 from rhodecode.tests import *
 
 from rhodecode.model.repos_group import ReposGroupModel
 from rhodecode.model.repo import RepoModel
 from rhodecode.model.db import RepoGroup, User, Notification, UserNotification, \
     UsersGroup, UsersGroupMember, Permission
 from sqlalchemy.exc import IntegrityError
 from rhodecode.model.user import UserModel
 
 from rhodecode.model.meta import Session
 from rhodecode.model.notification import NotificationModel
 from rhodecode.model.users_group import UsersGroupModel
 from rhodecode.lib.auth import AuthUser
 
 
 def _make_group(path, desc='desc', parent_id=None,
                  skip_if_exists=False):
 
     gr = RepoGroup.get_by_group_name(path)
     if gr and skip_if_exists:
         return gr
 
     gr = ReposGroupModel().create(path, desc, parent_id)
-    Session.commit()
     return gr
 
 
 class TestReposGroups(unittest.TestCase):
 
     def setUp(self):
         self.g1 = _make_group('test1', skip_if_exists=True)
+        Session.commit()
         self.g2 = _make_group('test2', skip_if_exists=True)
+        Session.commit()
         self.g3 = _make_group('test3', skip_if_exists=True)
+        Session.commit()
 
     def tearDown(self):
         print 'out'
 
     def __check_path(self, *path):
+        """
+        Checks the path for existance !
+        """
         path = [TESTS_TMP_PATH] + list(path)
         path = os.path.join(*path)
         return os.path.isdir(path)
 
     def _check_folders(self):
         print os.listdir(TESTS_TMP_PATH)
 
     def __delete_group(self, id_):
         ReposGroupModel().delete(id_)
 
     def __update_group(self, id_, path, desc='desc', parent_id=None):
-        form_data = dict(group_name=path,
-                         group_description=desc,
-                         group_parent_id=parent_id,
-                         perms_updates=[],
-                         perms_new=[])
-
+        form_data = dict(
+            group_name=path,
+            group_description=desc,
+            group_parent_id=parent_id,
+            perms_updates=[],
+            perms_new=[]
+        )
         gr = ReposGroupModel().update(id_, form_data)
         return gr
 
     def test_create_group(self):
         g = _make_group('newGroup')
         self.assertEqual(g.full_path, 'newGroup')
 
         self.assertTrue(self.__check_path('newGroup'))
 
     def test_create_same_name_group(self):
         self.assertRaises(IntegrityError, lambda:_make_group('newGroup'))
         Session.rollback()
 
     def test_same_subgroup(self):
         sg1 = _make_group('sub1', parent_id=self.g1.group_id)
         self.assertEqual(sg1.parent_group, self.g1)
         self.assertEqual(sg1.full_path, 'test1/sub1')
         self.assertTrue(self.__check_path('test1', 'sub1'))
 
         ssg1 = _make_group('subsub1', parent_id=sg1.group_id)
         self.assertEqual(ssg1.parent_group, sg1)
         self.assertEqual(ssg1.full_path, 'test1/sub1/subsub1')
         self.assertTrue(self.__check_path('test1', 'sub1', 'subsub1'))
 
     def test_remove_group(self):
         sg1 = _make_group('deleteme')
         self.__delete_group(sg1.group_id)
 
         self.assertEqual(RepoGroup.get(sg1.group_id), None)
         self.assertFalse(self.__check_path('deteteme'))
 
         sg1 = _make_group('deleteme', parent_id=self.g1.group_id)
         self.__delete_group(sg1.group_id)
 
         self.assertEqual(RepoGroup.get(sg1.group_id), None)
         self.assertFalse(self.__check_path('test1', 'deteteme'))
 
     def test_rename_single_group(self):
         sg1 = _make_group('initial')
 
         new_sg1 = self.__update_group(sg1.group_id, 'after')
         self.assertTrue(self.__check_path('after'))
         self.assertEqual(RepoGroup.get_by_group_name('initial'), None)
 
     def test_update_group_parent(self):
 
         sg1 = _make_group('initial', parent_id=self.g1.group_id)
 
         new_sg1 = self.__update_group(sg1.group_id, 'after', parent_id=self.g1.group_id)
         self.assertTrue(self.__check_path('test1', 'after'))
         self.assertEqual(RepoGroup.get_by_group_name('test1/initial'), None)
 
         new_sg1 = self.__update_group(sg1.group_id, 'after', parent_id=self.g3.group_id)
         self.assertTrue(self.__check_path('test3', 'after'))
         self.assertEqual(RepoGroup.get_by_group_name('test3/initial'), None)
 
         new_sg1 = self.__update_group(sg1.group_id, 'hello')
         self.assertTrue(self.__check_path('hello'))
 
         self.assertEqual(RepoGroup.get_by_group_name('hello'), new_sg1)
 
     def test_subgrouping_with_repo(self):
 
         g1 = _make_group('g1')
         g2 = _make_group('g2')
 
         # create new repo
         form_data = dict(repo_name='john',
                          repo_name_full='john',
                          fork_name=None,
                          description=None,
                          repo_group=None,
                          private=False,
                          repo_type='hg',
                          clone_uri=None)
         cur_user = User.get_by_username(TEST_USER_ADMIN_LOGIN)
         r = RepoModel().create(form_data, cur_user)
 
         self.assertEqual(r.repo_name, 'john')
 
         # put repo into group
         form_data = form_data
         form_data['repo_group'] = g1.group_id
         form_data['perms_new'] = []
         form_data['perms_updates'] = []
         RepoModel().update(r.repo_name, form_data)
         self.assertEqual(r.repo_name, 'g1/john')
 
         self.__update_group(g1.group_id, 'g1', parent_id=g2.group_id)
         self.assertTrue(self.__check_path('g2', 'g1'))
 
         # test repo
         self.assertEqual(r.repo_name, os.path.join('g2', 'g1', r.just_name))
 
 
+    def test_move_to_root(self):
+        g1 = _make_group('t11')
+        Session.commit()
+        g2 = _make_group('t22',parent_id=g1.group_id)
+        Session.commit()
+
+        self.assertEqual(g2.full_path,'t11/t22')
+        self.assertTrue(self.__check_path('t11', 't22'))
+
+        g2 = self.__update_group(g2.group_id, 'g22', parent_id=None)
+        Session.commit()
+
+        self.assertEqual(g2.group_name,'g22')
+        # we moved out group from t1 to '' so it's full path should be 'g2'
+        self.assertEqual(g2.full_path,'g22')
+        self.assertFalse(self.__check_path('t11', 't22'))
+        self.assertTrue(self.__check_path('g22'))
+
+
 class TestUser(unittest.TestCase):
     def __init__(self, methodName='runTest'):
         Session.remove()
         super(TestUser, self).__init__(methodName=methodName)
 
     def test_create_and_remove(self):
         usr = UserModel().create_or_update(username=u'test_user', password=u'qweqwe',
                                      email=u'[e-mail]',
                                      name=u'u1', lastname=u'u1')
         Session.commit()
         self.assertEqual(User.get_by_username(u'test_user'), usr)
 
         # make users group
         users_group = UsersGroupModel().create('some_example_group')
         Session.commit()
 
         UsersGroupModel().add_user_to_group(users_group, usr)
         Session.commit()
 
         self.assertEqual(UsersGroup.get(users_group.users_group_id), users_group)
         self.assertEqual(UsersGroupMember.query().count(), 1)
         UserModel().delete(usr.user_id)
         Session.commit()
 
         self.assertEqual(UsersGroupMember.query().all(), [])
 
 
 class TestNotifications(unittest.TestCase):
 
     def __init__(self, methodName='runTest'):
         Session.remove()
         self.u1 = UserModel().create_or_update(username=u'u1',
                                         password=u'qweqwe',
                                         email=u'[e-mail]',
                                         name=u'u1', lastname=u'u1')
         Session.commit()
         self.u1 = self.u1.user_id
 
         self.u2 = UserModel().create_or_update(username=u'u2',
                                         password=u'qweqwe',
                                         email=u'[e-mail]',
                                         name=u'u2', lastname=u'u3')
         Session.commit()
         self.u2 = self.u2.user_id
 
         self.u3 = UserModel().create_or_update(username=u'u3',
                                         password=u'qweqwe',
                                         email=u'[e-mail]',
                                         name=u'u3', lastname=u'u3')
         Session.commit()
         self.u3 = self.u3.user_id
 
         super(TestNotifications, self).__init__(methodName=methodName)
 
     def _clean_notifications(self):
         for n in Notification.query().all():
             Session.delete(n)
 
         Session.commit()
         self.assertEqual(Notification.query().all(), [])
 
     def tearDown(self):
         self._clean_notifications()
 
     def test_create_notification(self):
         self.assertEqual([], Notification.query().all())
         self.assertEqual([], UserNotification.query().all())
 
         usrs = [self.u1, self.u2]
         notification = NotificationModel().create(created_by=self.u1,
                                            subject=u'subj', body=u'hi there',
                                            recipients=usrs)
         Session.commit()
         u1 = User.get(self.u1)
         u2 = User.get(self.u2)
         u3 = User.get(self.u3)
         notifications = Notification.query().all()
         self.assertEqual(len(notifications), 1)
 
         unotification = UserNotification.query()\
             .filter(UserNotification.notification == notification).all()
 
         self.assertEqual(notifications[0].recipients, [u1, u2])
         self.assertEqual(notification.notification_id,
                          notifications[0].notification_id)
         self.assertEqual(len(unotification), len(usrs))
         self.assertEqual([x.user.user_id for x in unotification], usrs)
 
     def test_user_notifications(self):
         self.assertEqual([], Notification.query().all())
         self.assertEqual([], UserNotification.query().all())
 
         notification1 = NotificationModel().create(created_by=self.u1,
                                             subject=u'subj', body=u'hi there1',
                                             recipients=[self.u3])
         Session.commit()
         notification2 = NotificationModel().create(created_by=self.u1,
                                             subject=u'subj', body=u'hi there2',
                                             recipients=[self.u3])
         Session.commit()
         u3 = Session.query(User).get(self.u3)
 
         self.assertEqual(sorted([x.notification for x in u3.notifications]),
                          sorted([notification2, notification1]))
 
     def test_delete_notifications(self):
         self.assertEqual([], Notification.query().all())
         self.assertEqual([], UserNotification.query().all())
 
         notification = NotificationModel().create(created_by=self.u1,
                                            subject=u'title', body=u'hi there3',
                                     recipients=[self.u3, self.u1, self.u2])
         Session.commit()
         notifications = Notification.query().all()
         self.assertTrue(notification in notifications)
 
         Notification.delete(notification.notification_id)
         Session.commit()
 
         notifications = Notification.query().all()
         self.assertFalse(notification in notifications)
 
         un = UserNotification.query().filter(UserNotification.notification
                                              == notification).all()
         self.assertEqual(un, [])
 
     def test_delete_association(self):
 
         self.assertEqual([], Notification.query().all())
         self.assertEqual([], UserNotification.query().all())
 
         notification = NotificationModel().create(created_by=self.u1,
                                            subject=u'title', body=u'hi there3',
                                     recipients=[self.u3, self.u1, self.u2])
         Session.commit()
 
         unotification = UserNotification.query()\
                             .filter(UserNotification.notification ==
                                     notification)\
                             .filter(UserNotification.user_id == self.u3)\
                             .scalar()
 
         self.assertEqual(unotification.user_id, self.u3)
 
         NotificationModel().delete(self.u3,
                                    notification.notification_id)
         Session.commit()
 
         u3notification = UserNotification.query()\
                             .filter(UserNotification.notification ==
                                     notification)\
                             .filter(UserNotification.user_id == self.u3)\
                             .scalar()
 
         self.assertEqual(u3notification, None)
 
         # notification object is still there
         self.assertEqual(Notification.query().all(), [notification])
 
         #u1 and u2 still have assignments
         u1notification = UserNotification.query()\
                             .filter(UserNotification.notification ==
                                     notification)\
                             .filter(UserNotification.user_id == self.u1)\
                             .scalar()
         self.assertNotEqual(u1notification, None)
         u2notification = UserNotification.query()\
                             .filter(UserNotification.notification ==
                                     notification)\
                             .filter(UserNotification.user_id == self.u2)\
                             .scalar()
         self.assertNotEqual(u2notification, None)
 
     def test_notification_counter(self):
         self._clean_notifications()
         self.assertEqual([], Notification.query().all())
         self.assertEqual([], UserNotification.query().all())
 
         NotificationModel().create(created_by=self.u1,
                             subject=u'title', body=u'hi there_delete',
                             recipients=[self.u3, self.u1])
         Session.commit()
 
         self.assertEqual(NotificationModel()
                          .get_unread_cnt_for_user(self.u1), 1)
         self.assertEqual(NotificationModel()
                          .get_unread_cnt_for_user(self.u2), 0)
         self.assertEqual(NotificationModel()
                          .get_unread_cnt_for_user(self.u3), 1)
 
         notification = NotificationModel().create(created_by=self.u1,
                                            subject=u'title', body=u'hi there3',
                                     recipients=[self.u3, self.u1, self.u2])
         Session.commit()
 
         self.assertEqual(NotificationModel()
                          .get_unread_cnt_for_user(self.u1), 2)
         self.assertEqual(NotificationModel()
                          .get_unread_cnt_for_user(self.u2), 1)
         self.assertEqual(NotificationModel()
                          .get_unread_cnt_for_user(self.u3), 2)
 
 
 class TestUsers(unittest.TestCase):
 
     def __init__(self, methodName='runTest'):
         super(TestUsers, self).__init__(methodName=methodName)
 
     def setUp(self):
         self.u1 = UserModel().create_or_update(username=u'u1',
                                         password=u'qweqwe',
                                         email=u'[e-mail]',
                                         name=u'u1', lastname=u'u1')
 
     def tearDown(self):
         perm = Permission.query().all()
         for p in perm:
             UserModel().revoke_perm(self.u1, p)
 
         UserModel().delete(self.u1)
         Session.commit()
 
     def test_add_perm(self):
         perm = Permission.query().all()[0]
         UserModel().grant_perm(self.u1, perm)
         Session.commit()
         self.assertEqual(UserModel().has_perm(self.u1, perm), True)
 
     def test_has_perm(self):
         perm = Permission.query().all()
         for p in perm:
             has_p = UserModel().has_perm(self.u1, p)
             self.assertEqual(False, has_p)
 
     def test_revoke_perm(self):
         perm = Permission.query().all()[0]
         UserModel().grant_perm(self.u1, perm)
         Session.commit()
         self.assertEqual(UserModel().has_perm(self.u1, perm), True)
 
         #revoke
         UserModel().revoke_perm(self.u1, perm)
         Session.commit()
         self.assertEqual(UserModel().has_perm(self.u1, perm), False)
 
 
 class TestPermissions(unittest.TestCase):
     def __init__(self, methodName='runTest'):
         super(TestPermissions, self).__init__(methodName=methodName)
 
     def setUp(self):
         self.u1 = UserModel().create_or_update(
             username=u'u1', password=u'qweqwe',
             email=u'[e-mail]', name=u'u1', lastname=u'u1'
         )
         self.a1 = UserModel().create_or_update(
             username=u'a1', password=u'qweqwe',
             email=u'[e-mail]', name=u'a1', lastname=u'a1', admin=True
         )
         Session.commit()
 
     def tearDown(self):
         UserModel().delete(self.u1)
         UserModel().delete(self.a1)
         if hasattr(self, 'g1'):
             ReposGroupModel().delete(self.g1.group_id)
         if hasattr(self, 'g2'):
             ReposGroupModel().delete(self.g2.group_id)
 
         if hasattr(self, 'ug1'):
             UsersGroupModel().delete(self.ug1, force=True)
 
         Session.commit()
 
     def test_default_perms_set(self):
         u1_auth = AuthUser(user_id=self.u1.user_id)
         perms = {
             'repositories_groups': {},
             'global': set([u'hg.create.repository', u'repository.read',
                            u'hg.register.manual_activate']),
             'repositories': {u'vcs_test_hg': u'repository.read'}
         }
         self.assertEqual(u1_auth.permissions['repositories'][HG_REPO],
                          perms['repositories'][HG_REPO])
         new_perm = 'repository.write'
         RepoModel().grant_user_permission(repo=HG_REPO, user=self.u1, perm=new_perm)
         Session.commit()
 
         u1_auth = AuthUser(user_id=self.u1.user_id)
         self.assertEqual(u1_auth.permissions['repositories'][HG_REPO], new_perm)
 
     def test_default_admin_perms_set(self):
         a1_auth = AuthUser(user_id=self.a1.user_id)
         perms = {
             'repositories_groups': {},
             'global': set([u'hg.admin']),
             'repositories': {u'vcs_test_hg': u'repository.admin'}
         }
         self.assertEqual(a1_auth.permissions['repositories'][HG_REPO],
                          perms['repositories'][HG_REPO])
         new_perm = 'repository.write'
         RepoModel().grant_user_permission(repo=HG_REPO, user=self.a1, perm=new_perm)
         Session.commit()
         # cannot really downgrade admins permissions !? they still get's set as
         # admin !
         u1_auth = AuthUser(user_id=self.a1.user_id)
         self.assertEqual(u1_auth.permissions['repositories'][HG_REPO],
                          perms['repositories'][HG_REPO])
 
     def test_default_group_perms(self):
         self.g1 = _make_group('test1', skip_if_exists=True)
         self.g2 = _make_group('test2', skip_if_exists=True)
         u1_auth = AuthUser(user_id=self.u1.user_id)
         perms = {
             'repositories_groups': {u'test1': 'group.read', u'test2': 'group.read'},
             'global': set([u'hg.create.repository', u'repository.read', u'hg.register.manual_activate']),
             'repositories': {u'vcs_test_hg': u'repository.read'}
         }
         self.assertEqual(u1_auth.permissions['repositories'][HG_REPO],
                          perms['repositories'][HG_REPO])
         self.assertEqual(u1_auth.permissions['repositories_groups'],
                          perms['repositories_groups'])
 
     def test_default_admin_group_perms(self):
         self.g1 = _make_group('test1', skip_if_exists=True)
         self.g2 = _make_group('test2', skip_if_exists=True)
         a1_auth = AuthUser(user_id=self.a1.user_id)
         perms = {
             'repositories_groups': {u'test1': 'group.admin', u'test2': 'group.admin'},
             'global': set(['hg.admin']),
             'repositories': {u'vcs_test_hg': 'repository.admin'}
         }
 
         self.assertEqual(a1_auth.permissions['repositories'][HG_REPO],
                          perms['repositories'][HG_REPO])
         self.assertEqual(a1_auth.permissions['repositories_groups'],
                          perms['repositories_groups'])
 
     def test_propagated_permission_from_users_group(self):
         # make group
         self.ug1 = UsersGroupModel().create('G1')
         # add user to group
         UsersGroupModel().add_user_to_group(self.ug1, self.u1)
 
         # set permission to lower
         new_perm = 'repository.none'
         RepoModel().grant_user_permission(repo=HG_REPO, user=self.u1, perm=new_perm)
         Session.commit()
         u1_auth = AuthUser(user_id=self.u1.user_id)
         self.assertEqual(u1_auth.permissions['repositories'][HG_REPO],
                          new_perm)
 
         # grant perm for group this should override permission from user
         new_perm = 'repository.write'
         RepoModel().grant_users_group_permission(repo=HG_REPO,
                                                  group_name=self.ug1,
                                                  perm=new_perm)
         # check perms
         u1_auth = AuthUser(user_id=self.u1.user_id)
         perms = {
             'repositories_groups': {},
             'global': set([u'hg.create.repository', u'repository.read',
                            u'hg.register.manual_activate']),
             'repositories': {u'vcs_test_hg': u'repository.read'}
         }
         self.assertEqual(u1_auth.permissions['repositories'][HG_REPO],
                          new_perm)
         self.assertEqual(u1_auth.permissions['repositories_groups'],
                          perms['repositories_groups'])
 
     def test_propagated_permission_from_users_group_lower_weight(self):
         # make group
         self.ug1 = UsersGroupModel().create('G1')
         # add user to group
         UsersGroupModel().add_user_to_group(self.ug1, self.u1)
 
         # set permission to lower
         new_perm_h = 'repository.write'
         RepoModel().grant_user_permission(repo=HG_REPO, user=self.u1,
                                           perm=new_perm_h)
         Session.commit()
         u1_auth = AuthUser(user_id=self.u1.user_id)
         self.assertEqual(u1_auth.permissions['repositories'][HG_REPO],
                          new_perm_h)
 
         # grant perm for group this should NOT override permission from user
         # since it's lower than granted
         new_perm_l = 'repository.read'
         RepoModel().grant_users_group_permission(repo=HG_REPO,
                                                  group_name=self.ug1,
                                                  perm=new_perm_l)
         # check perms
         u1_auth = AuthUser(user_id=self.u1.user_id)
         perms = {
             'repositories_groups': {},
             'global': set([u'hg.create.repository', u'repository.read',
                            u'hg.register.manual_activate']),
             'repositories': {u'vcs_test_hg': u'repository.write'}
         }
         self.assertEqual(u1_auth.permissions['repositories'][HG_REPO],
                          new_perm_h)
         self.assertEqual(u1_auth.permissions['repositories_groups'],
                          perms['repositories_groups'])