api: add a max_file_bytes parameter to get_nodes so that large

files can be skipped

docs/api/api.rst

@@ -1,2824 +1,2825 b''
 .. _api:
 
 API Documentation
 =================
 
 The |RCE| API uses a single scheme for calling all API methods. The API is
 implemented with JSON protocol in both directions. To send API requests to
 your instance of |RCE|, use the following URL format
 ``<your_server>/_admin``
 
 .. note::
 
    To use the API, you should configure the :file:`~/.rhoderc` file with
    access details per instance. For more information, see
    :ref:`config-rhoderc`.
 
 
 API ACCESS FOR WEB VIEWS
 ------------------------
 
 API access can also be turned on for each web view in |RCE| that is
 decorated with a `@LoginRequired` decorator. To enable API access, change
 the standard login decorator to `@LoginRequired(api_access=True)`.
 
 From |RCM| version 1.7.0 you can configure a white list
 of views that have API access enabled by default. To enable these,
 edit the |RCM| configuration ``.ini`` file. The default location is:
 
 * |RCM| Pre-2.2.7 :file:`root/rhodecode/data/production.ini`
 * |RCM| 3.0 :file:`/home/{user}/.rccontrol/{instance-id}/rhodecode.ini`
 
 To configure the white list, edit this section of the file. In this
 configuration example, API access is granted to the patch/diff raw file and
 archive.
 
 .. code-block:: ini
 
     ## List of controllers (using glob syntax) that AUTH TOKENS could be used for access.
     ## Adding ?auth_token = <token> to the url authenticates this request as if it
     ## came from the the logged in user who own this authentication token.
     ##
     ## Syntax is <ControllerClass>:<function_pattern>.
     ## The list should be "," separated and on a single line.
     ##
     api_access_controllers_whitelist = ChangesetController:changeset_patch,ChangesetController:changeset_raw,ilesController:raw,FilesController:archivefile,
 
 After this change, a |RCE| view can be accessed without login by adding a
 GET parameter ``?auth_token=<auth_token>`` to a url. For example to
 access the raw diff.
 
 .. code-block:: html
 
    http://<server>/<repo>/changeset-diff/<sha>?auth_token=<auth_token>
 
 By default this is only enabled on RSS/ATOM feed views. Exposing raw diffs is a
 good way to integrate with 3rd party services like code review, or build farms
 that could download archives.
 
 API ACCESS
 ----------
 
 All clients are required to send JSON-RPC spec JSON data.
 
 .. code-block:: bash
 
     {
         "id:"<id>",
         "auth_token":"<auth_token>",
         "method":"<method_name>",
         "args":{"<arg_key>":"<arg_val>"}
     }
 
 Example call for auto pulling from remote repositories using curl:
 
 .. code-block:: bash
 
     curl https://server.com/_admin/api -X POST -H 'content-type:text/plain' --data-binary '{"id":1,
     "auth_token":"xe7cdb2v278e4evbdf5vs04v832v0efvcbcve4a3","method":"pull", "args":{"repo":"CPython"}}'
 
 Provide those parameters:
  - **id** A value of any type, which is used to match the response with the
    request that it is replying to.
  - **auth_token** for access and permission validation.
  - **method** is name of method to call
  - **args** is an ``key:value`` list of arguments to pass to method
 
 .. note::
 
     To get your |authtoken|, from the |RCE| interface,
     go to:
     :menuselection:`username --> My account --> Auth tokens`
 
     For security reasons you should always create a dedicated |authtoken| for
     API use only.
 
 
 The |RCE| API will always return a JSON-RPC response:
 
 .. code-block:: bash
 
     {
         "id": <id>, # matching id sent by request
         "result": "<result>"|null, # JSON formatted result, null if any errors
         "error": "null"|<error_message> # JSON formatted error (if any)
     }
 
 All responses from API will be with `HTTP/1.0 200 OK` status code.
 If there is an error when calling the API, the *error* key will contain a
 failure description and the *result* will be `null`.
 
 API CLIENT
 ----------
 
 To install the |RCE| API, see :ref:`install-tools`. To configure the API per
 instance, see the :ref:`rc-tools` section as you need to configure a
 :file:`~/.rhoderc` file with your |authtokens|.
 
 Once you have set up your instance API access, use the following examples to
 get started.
 
 .. code-block:: bash
 
     # Getting the 'rhodecode' repository
     # from a RhodeCode Enterprise instance
     rhodecode-api --instance-name=enterprise-1 get_repo repoid:rhodecode
 
     Calling method get_repo => http://127.0.0.1:5000
     Server response
     {
         <json data>
     }
 
     # Creating a new mercurial repository called 'brand-new'
     # with a description 'Repo-description'
     rhodecode-api --instance-name=enterprise-1 create_repo repo_name:brand-new repo_type:hg description:Repo-description
     {
       "error": null,
       "id": 1110,
       "result": {
         "msg": "Created new repository `brand-new`",
         "success": true,
         "task": null
       }
     }
 
 A broken example, what not to do.
 
 .. code-block:: bash
 
     # A call missing the required arguments
     # and not specifying the instance
     rhodecode-api get_repo
 
     Calling method get_repo => http://127.0.0.1:5000
     Server response
     "Missing non optional `repoid` arg in JSON DATA"
 
 You can specify pure JSON using the ``--format`` parameter.
 
 .. code-block:: bash
 
     rhodecode-api --format=json get_repo repoid:rhodecode
 
 In such case only output that this function shows is pure JSON, we can use that
 and pipe output to some json formatter.
 
 If output is in pure JSON format, you can pipe output to a JSON formatter.
 
 .. code-block:: bash
 
     rhodecode-api --instance-name=enterprise-1 --format=json get_repo repoid:rhodecode | python -m json.tool
 
 API METHODS
 -----------
 
 Each method by default required following arguments.
 
 .. code-block:: bash
 
     id :      "<id_for_response>"
     auth_token : "<auth_token>"
     method :  "<method name>"
     args :    {}
 
 Use each **param** from docs and put it in args, Optional parameters
 are not required in args.
 
 .. code-block:: bash
 
     args: {"repoid": "rhodecode"}
 
 .. Note: From this point on things are generated by the script in
    `scripts/fabfile.py`. To change things below, update the docstrings in the
    ApiController.
 
 .. --- API DEFS MARKER ---
 
 pull
 ----
 
 .. py:function:: pull(apiuser, repoid)
 
    Triggers a pull on the given repository from a remote location. You
    can use this to keep remote repositories up-to-date.
 
    This command can only be run using an |authtoken| with admin
    rights to the specified repository. For more information,
    see :ref:`config-token-ref`.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "Pulled from `<repository name>`"
        "repository": "<repository name>"
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "Unable to pull changes from `<reponame>`"
      }
 
 
 strip
 -----
 
 .. py:function:: strip(apiuser, repoid, revision, branch)
 
    Strips the given revision from the specified repository.
 
    * This will remove the revision and all of its decendants.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param revision: The revision you wish to strip.
    :type revision: str
    :param branch: The branch from which to strip the revision.
    :type branch: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "'Stripped commit <commit_hash> from repo `<repository name>`'"
        "repository": "<repository name>"
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "Unable to strip commit <commit_hash> from repo `<repository name>`"
      }
 
 
 rescan_repos
 ------------
 
 .. py:function:: rescan_repos(apiuser, remove_obsolete=<Optional:False>)
 
    Triggers a rescan of the specified repositories.
 
    * If the ``remove_obsolete`` option is set, it also deletes repositories
      that are found in the database but not on the file system, so called
      "clean zombies".
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param remove_obsolete: Deletes repositories from the database that
        are not found on the filesystem.
    :type remove_obsolete: Optional(``True`` | ``False``)
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'added': [<added repository name>,...]
        'removed': [<removed repository name>,...]
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        'Error occurred during rescan repositories action'
      }
 
 
 invalidate_cache
 ----------------
 
 .. py:function:: invalidate_cache(apiuser, repoid, delete_keys=<Optional:False>)
 
    Invalidates the cache for the specified repository.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Sets the repository name or repository ID.
    :type repoid: str or int
    :param delete_keys: This deletes the invalidated keys instead of
        just flagging them.
    :type delete_keys: Optional(``True`` | ``False``)
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'msg': Cache for repository `<repository name>` was invalidated,
        'repository': <repository name>
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error : {
         'Error occurred during cache invalidation action'
      }
 
 
 lock
 ----
 
 .. py:function:: lock(apiuser, repoid, locked=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Sets the lock state of the specified |repo| by the given user.
    From more information, see :ref:`repo-locking`.
 
    * If the ``userid`` option is not set, the repository is locked to the
      user who called the method.
    * If the ``locked`` parameter is not set, the current lock state of the
      repository is displayed.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Sets the repository name or repository ID.
    :type repoid: str or int
    :param locked: Sets the lock state.
    :type locked: Optional(``True`` | ``False``)
    :param userid: Set the repository lock to this user.
    :type userid: Optional(str or int)
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'repo': '<reponame>',
        'locked': <bool: lock state>,
        'locked_since': <int: lock timestamp>,
        'locked_by': <username of person who made the lock>,
        'lock_reason': <str: reason for locking>,
        'lock_state_changed': <bool: True if lock state has been changed in this request>,
        'msg': 'Repo `<reponame>` locked by `<username>` on <timestamp>.'
        or
        'msg': 'Repo `<repository name>` not locked.'
        or
        'msg': 'User `<user name>` set lock state for repo `<repository name>` to `<new lock state>`'
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        'Error occurred locking repository `<reponame>`
      }
 
 
 get_locks
 ---------
 
 .. py:function:: get_locks(apiuser, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Displays all repositories locked by the specified user.
 
    * If this command is run by a non-admin user, it returns
      a list of |repos| locked by that user.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param userid: Sets the userid whose list of locked |repos| will be
        displayed.
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result : {
            [repo_object, repo_object,...]
        }
        error :  null
 
 
 get_ip
 ------
 
 .. py:function:: get_ip(apiuser, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Displays the IP Address as seen from the |RCE| server.
 
    * This command displays the IP Address, as well as all the defined IP
      addresses for the specified user. If the ``userid`` is not set, the
      data returned is for the user calling the method.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from |authtoken|.
    :type apiuser: AuthUser
    :param userid: Sets the userid for which associated IP Address data
        is returned.
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result : {
                     "server_ip_addr": "<ip_from_clien>",
                     "user_ips": [
                                    {
                                       "ip_addr": "<ip_with_mask>",
                                       "ip_range": ["<start_ip>", "<end_ip>"],
                                    },
                                    ...
                                 ]
        }
 
 
 show_ip
 -------
 
 .. py:function:: show_ip(apiuser, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Displays the IP Address as seen from the |RCE| server.
 
    * This command displays the IP Address, as well as all the defined IP
      addresses for the specified user. If the ``userid`` is not set, the
      data returned is for the user calling the method.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from |authtoken|.
    :type apiuser: AuthUser
    :param userid: Sets the userid for which associated IP Address data
        is returned.
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result : {
                     "server_ip_addr": "<ip_from_clien>",
                     "user_ips": [
                                    {
                                       "ip_addr": "<ip_with_mask>",
                                       "ip_range": ["<start_ip>", "<end_ip>"],
                                    },
                                    ...
                                 ]
        }
 
 
 get_license_info
 ----------------
 
 .. py:function:: get_license_info(apiuser)
 
    Returns the |RCE| license information.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'rhodecode_version': <rhodecode version>,
        'token': <license token>,
        'issued_to': <license owner>,
        'issued_on': <license issue date>,
        'expires_on': <license expiration date>,
        'type': <license type>,
        'users_limit': <license users limit>,
        'key': <license key>
      }
      error :  null
 
 
 set_license_key
 ---------------
 
 .. py:function:: set_license_key(apiuser, key)
 
    Sets the |RCE| license key.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param key: This is the license key to be set.
    :type key: str
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "updated license information",
                  "key": <key>
                }
        error:  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "license key is not valid"
        or
        "trial licenses cannot be uploaded"
        or
        "error occurred while updating license"
      }
 
 
 get_server_info
 ---------------
 
 .. py:function:: get_server_info(apiuser)
 
    Returns the |RCE| server information.
 
    This includes the running version of |RCE| and all installed
    packages. This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'modules': [<module name>,...]
        'py_version': <python version>,
        'platform': <platform type>,
        'rhodecode_version': <rhodecode version>
      }
      error :  null
 
 
 get_user
 --------
 
 .. py:function:: get_user(apiuser, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Returns the information associated with a username or userid.
 
    * If the ``userid`` is not set, this command returns the information
      for the ``userid`` calling the method.
 
    .. note::
 
       Normal users may only run this command against their ``userid``. For
       full privileges you must run this command using an |authtoken| with
       admin rights.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param userid: Sets the userid for which data will be returned.
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
        {
          "error": null,
          "id": <id>,
          "result": {
            "active": true,
            "admin": false,
            "api_key": "api-key",
            "api_keys": [ list of keys ],
            "email": "[e-mail]",
            "emails": [
              "[e-mail]"
            ],
            "extern_name": "rhodecode",
            "extern_type": "rhodecode",
            "firstname": "username",
            "ip_addresses": [],
            "language": null,
            "last_login": "Timestamp",
            "lastname": "surnae",
            "permissions": {
              "global": [
                "hg.inherit_default_perms.true",
                "usergroup.read",
                "hg.repogroup.create.false",
                "hg.create.none",
                "hg.extern_activate.manual",
                "hg.create.write_on_repogroup.false",
                "hg.usergroup.create.false",
                "group.none",
                "repository.none",
                "hg.register.none",
                "hg.fork.repository"
              ],
              "repositories": { "username/example": "repository.write"},
              "repositories_groups": { "user-group/repo": "group.none" },
              "user_groups": { "user_group_name": "usergroup.read" }
            },
            "user_id": 32,
            "username": "username"
          }
        }
 
 
 get_users
 ---------
 
 .. py:function:: get_users(apiuser)
 
    Lists all users in the |RCE| user database.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
            result: [<user_object>, ...]
        error:  null
 
 
 create_user
 -----------
 
 .. py:function:: create_user(apiuser, username, email, password=<Optional:''>, firstname=<Optional:''>, lastname=<Optional:''>, active=<Optional:True>, admin=<Optional:False>, extern_name=<Optional:'rhodecode'>, extern_type=<Optional:'rhodecode'>, force_password_change=<Optional:False>)
 
    Creates a new user and returns the new user object.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param username: Set the new username.
    :type username: str or int
    :param email: Set the user email address.
    :type email: str
    :param password: Set the new user password.
    :type password: Optional(str)
    :param firstname: Set the new user firstname.
    :type firstname: Optional(str)
    :param lastname: Set the new user surname.
    :type lastname: Optional(str)
    :param active: Set the user as active.
    :type active: Optional(``True`` | ``False``)
    :param admin: Give the new user admin rights.
    :type admin: Optional(``True`` | ``False``)
    :param extern_name: Set the authentication plugin name.
        Using LDAP this is filled with LDAP UID.
    :type extern_name: Optional(str)
    :param extern_type: Set the new user authentication plugin.
    :type extern_type: Optional(str)
    :param force_password_change: Force the new user to change password
        on next login.
    :type force_password_change: Optional(``True`` | ``False``)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "created new user `<username>`",
                  "user": <user_obj>
                }
        error:  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "user `<username>` already exist"
        or
        "email `<email>` already exist"
        or
        "failed to create user `<username>`"
      }
 
 
 update_user
 -----------
 
 .. py:function:: update_user(apiuser, userid, username=<Optional:None>, email=<Optional:None>, password=<Optional:None>, firstname=<Optional:None>, lastname=<Optional:None>, active=<Optional:None>, admin=<Optional:None>, extern_type=<Optional:None>, extern_name=<Optional:None>)
 
    Updates the details for the specified user, if that user exists.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from |authtoken|.
    :type apiuser: AuthUser
    :param userid: Set the ``userid`` to update.
    :type userid: str or int
    :param username: Set the new username.
    :type username: str or int
    :param email: Set the new email.
    :type email: str
    :param password: Set the new password.
    :type password: Optional(str)
    :param firstname: Set the new first name.
    :type firstname: Optional(str)
    :param lastname: Set the new surname.
    :type lastname: Optional(str)
    :param active: Set the new user as active.
    :type active: Optional(``True`` | ``False``)
    :param admin: Give the user admin rights.
    :type admin: Optional(``True`` | ``False``)
    :param extern_name: Set the authentication plugin user name.
        Using LDAP this is filled with LDAP UID.
    :type extern_name: Optional(str)
    :param extern_type: Set the authentication plugin type.
    :type extern_type: Optional(str)
 
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "updated user ID:<userid> <username>",
                  "user": <user_object>,
                }
        error:  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to update user `<username>`"
      }
 
 
 delete_user
 -----------
 
 .. py:function:: delete_user(apiuser, userid)
 
    Deletes the specified user from the |RCE| user database.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    .. important::
 
       Ensure all open pull requests and open code review
       requests to this user are close.
 
       Also ensure all repositories, or repository groups owned by this
       user are reassigned before deletion.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param userid: Set the user to delete.
    :type userid: str or int
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "deleted user ID:<userid> <username>",
                  "user": null
                }
        error:  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to delete user ID:<userid> <username>"
      }
 
 
 get_user_group
 --------------
 
 .. py:function:: get_user_group(apiuser, usergroupid)
 
    Returns the data of an existing user group.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Set the user group from which to return data.
    :type usergroupid: str or int
 
    Example error output:
 
    .. code-block:: bash
 
        {
          "error": null,
          "id": <id>,
          "result": {
            "active": true,
            "group_description": "group description",
            "group_name": "group name",
            "members": [
              {
                "name": "owner-name",
                "origin": "owner",
                "permission": "usergroup.admin",
                "type": "user"
              },
              {
              {
                "name": "user name",
                "origin": "permission",
                "permission": "usergroup.admin",
                "type": "user"
              },
              {
                "name": "user group name",
                "origin": "permission",
                "permission": "usergroup.write",
                "type": "user_group"
              }
            ],
            "owner": "owner name",
            "users": [],
            "users_group_id": 2
          }
        }
 
 
 get_user_groups
 ---------------
 
 .. py:function:: get_user_groups(apiuser)
 
    Lists all the existing user groups within RhodeCode.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
 
    Example error output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result : [<user_group_obj>,...]
        error : null
 
 
 create_user_group
 -----------------
 
 .. py:function:: create_user_group(apiuser, group_name, description=<Optional:''>, owner=<Optional:<OptionalAttr:apiuser>>, active=<Optional:True>)
 
    Creates a new user group.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param group_name: Set the name of the new user group.
    :type group_name: str
    :param description: Give a description of the new user group.
    :type description: str
    :param owner: Set the owner of the new user group.
        If not set, the owner is the |authtoken| user.
    :type owner: Optional(str or int)
    :param active: Set this group as active.
    :type active: Optional(``True`` | ``False``)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg": "created new user group `<groupname>`",
                  "user_group": <user_group_object>
                }
        error:  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "user group `<group name>` already exist"
        or
        "failed to create group `<group name>`"
      }
 
 
 update_user_group
 -----------------
 
 .. py:function:: update_user_group(apiuser, usergroupid, group_name=<Optional:''>, description=<Optional:''>, owner=<Optional:None>, active=<Optional:True>)
 
    Updates the specified `user group` with the details provided.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Set the id of the `user group` to update.
    :type usergroupid: str or int
    :param group_name: Set the new name the `user group`
    :type group_name: str
    :param description: Give a description for the `user group`
    :type description: str
    :param owner: Set the owner of the `user group`.
    :type owner: Optional(str or int)
    :param active: Set the group as active.
    :type active: Optional(``True`` | ``False``)
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": 'updated user group ID:<user group id> <user group name>',
        "user_group": <user_group_object>
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to update user group `<user group name>`"
      }
 
 
 delete_user_group
 -----------------
 
 .. py:function:: delete_user_group(apiuser, usergroupid)
 
    Deletes the specified `user group`.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified repository.
 
    This command takes the following options:
 
    :param apiuser: filled automatically from apikey
    :type apiuser: AuthUser
    :param usergroupid:
    :type usergroupid: int
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "deleted user group ID:<user_group_id> <user_group_name>"
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to delete user group ID:<user_group_id> <user_group_name>"
        or
        "RepoGroup assigned to <repo_groups_list>"
      }
 
 
 add_user_to_user_group
 ----------------------
 
 .. py:function:: add_user_to_user_group(apiuser, usergroupid, userid)
 
    Adds a user to a `user group`. If the user already exists in the group
    this command will return false.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified user group.
 
    This command takes the following options:
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Set the name of the `user group` to which a
        user will be added.
    :type usergroupid: int
    :param userid: Set the `user_id` of the user to add to the group.
    :type userid: int
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
          "success": True|False # depends on if member is in group
          "msg": "added member `<username>` to user group `<groupname>` |
                  User is already in that group"
 
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to add member to user group `<user_group_name>`"
      }
 
 
 remove_user_from_user_group
 ---------------------------
 
 .. py:function:: remove_user_from_user_group(apiuser, usergroupid, userid)
 
    Removes a user from a user group.
 
    * If the specified user is not in the group, this command will return
      `false`.
 
    This command can only be run using an |authtoken| with admin rights to
    the specified user group.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Sets the user group name.
    :type usergroupid: str or int
    :param userid: The user you wish to remove from |RCE|.
    :type userid: str or int
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "success":  True|False,  # depends on if member is in group
                  "msg": "removed member <username> from user group <groupname> |
                          User wasn't in group"
                }
        error:  null
 
 
 grant_user_permission_to_user_group
 -----------------------------------
 
 .. py:function:: grant_user_permission_to_user_group(apiuser, usergroupid, userid, perm)
 
    Set permissions for a user in a user group.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Set the user group to edit permissions on.
    :type usergroupid: str or int
    :param userid: Set the user from whom you wish to set permissions.
    :type userid: str
    :param perm: (usergroup.(none|read|write|admin))
    :type perm: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "Granted perm: `<perm_name>` for user: `<username>` in user group: `<user_group_name>`",
        "success": true
      }
      error :  null
 
 
 revoke_user_permission_from_user_group
 --------------------------------------
 
 .. py:function:: revoke_user_permission_from_user_group(apiuser, usergroupid, userid)
 
    Revoke a users permissions in a user group.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Set the user group from which to revoke the user
        permissions.
    :type: usergroupid: str or int
    :param userid: Set the userid of the user whose permissions will be
        revoked.
    :type userid: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "Revoked perm for user: `<username>` in user group: `<user_group_name>`",
        "success": true
      }
      error :  null
 
 
 grant_user_group_permission_to_user_group
 -----------------------------------------
 
 .. py:function:: grant_user_group_permission_to_user_group(apiuser, usergroupid, sourceusergroupid, perm)
 
    Give one user group permissions to another user group.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Set the user group on which to edit permissions.
    :type usergroupid: str or int
    :param sourceusergroupid: Set the source user group to which
        access/permissions will be granted.
    :type sourceusergroupid: str or int
    :param perm: (usergroup.(none|read|write|admin))
    :type perm: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "Granted perm: `<perm_name>` for user group: `<source_user_group_name>` in user group: `<user_group_name>`",
        "success": true
      }
      error :  null
 
 
 revoke_user_group_permission_from_user_group
 --------------------------------------------
 
 .. py:function:: revoke_user_group_permission_from_user_group(apiuser, usergroupid, sourceusergroupid)
 
    Revoke the permissions that one user group has to another.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param usergroupid: Set the user group on which to edit permissions.
    :type usergroupid: str or int
    :param sourceusergroupid: Set the user group from which permissions
        are revoked.
    :type sourceusergroupid: str or int
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "Revoked perm for user group: `<user_group_name>` in user group: `<target_user_group_name>`",
        "success": true
      }
      error :  null
 
 
 get_pull_request
 ----------------
 
 .. py:function:: get_pull_request(apiuser, repoid, pullrequestid)
 
    Get a pull request based on the given ID.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Repository name or repository ID from where the pull
        request was opened.
    :type repoid: str or int
    :param pullrequestid: ID of the requested pull request.
    :type pullrequestid: int
 
    Example output:
 
    .. code-block:: bash
 
      "id": <id_given_in_input>,
      "result":
        {
            "pull_request_id":   "<pull_request_id>",
            "url":               "<url>",
            "title":             "<title>",
            "description":       "<description>",
            "status" :           "<status>",
            "created_on":        "<date_time_created>",
            "updated_on":        "<date_time_updated>",
            "commit_ids":        [
                                     ...
                                     "<commit_id>",
                                     "<commit_id>",
                                     ...
                                 ],
            "review_status":    "<review_status>",
            "mergeable":         {
                                     "status":  "<bool>",
                                     "message": "<message>",
                                 },
            "source":            {
                                     "clone_url":     "<clone_url>",
                                     "repository":    "<repository_name>",
                                     "reference":
                                     {
                                         "name":      "<name>",
                                         "type":      "<type>",
                                         "commit_id": "<commit_id>",
                                     }
                                 },
            "target":            {
                                     "clone_url":   "<clone_url>",
                                     "repository":    "<repository_name>",
                                     "reference":
                                     {
                                         "name":      "<name>",
                                         "type":      "<type>",
                                         "commit_id": "<commit_id>",
                                     }
                                 },
           "author":             <user_obj>,
           "reviewers":          [
                                     ...
                                     {
                                        "user":          "<user_obj>",
                                        "review_status": "<review_status>",
                                     }
                                     ...
                                 ]
        },
       "error": null
 
 
 get_pull_requests
 -----------------
 
 .. py:function:: get_pull_requests(apiuser, repoid, status=<Optional:'new'>)
 
    Get all pull requests from the repository specified in `repoid`.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Repository name or repository ID.
    :type repoid: str or int
    :param status: Only return pull requests with the specified status.
        Valid options are.
        * ``new`` (default)
        * ``open``
        * ``closed``
    :type status: str
 
    Example output:
 
    .. code-block:: bash
 
      "id": <id_given_in_input>,
      "result":
        [
            ...
            {
                "pull_request_id":   "<pull_request_id>",
                "url":               "<url>",
                "title" :            "<title>",
                "description":       "<description>",
                "status":            "<status>",
                "created_on":        "<date_time_created>",
                "updated_on":        "<date_time_updated>",
                "commit_ids":        [
                                         ...
                                         "<commit_id>",
                                         "<commit_id>",
                                         ...
                                     ],
                "review_status":    "<review_status>",
                "mergeable":         {
                                        "status":      "<bool>",
                                        "message:      "<message>",
                                     },
                "source":            {
                                         "clone_url":     "<clone_url>",
                                         "reference":
                                         {
                                             "name":      "<name>",
                                             "type":      "<type>",
                                             "commit_id": "<commit_id>",
                                         }
                                     },
                "target":            {
                                         "clone_url":   "<clone_url>",
                                         "reference":
                                         {
                                             "name":      "<name>",
                                             "type":      "<type>",
                                             "commit_id": "<commit_id>",
                                         }
                                     },
               "author":             <user_obj>,
               "reviewers":          [
                                         ...
                                         {
                                            "user":          "<user_obj>",
                                            "review_status": "<review_status>",
                                         }
                                         ...
                                     ]
            }
            ...
        ],
      "error": null
 
 
 merge_pull_request
 ------------------
 
 .. py:function:: merge_pull_request(apiuser, repoid, pullrequestid, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Merge the pull request specified by `pullrequestid` into its target
    repository.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The Repository name or repository ID of the
        target repository to which the |pr| is to be merged.
    :type repoid: str or int
    :param pullrequestid: ID of the pull request which shall be merged.
    :type pullrequestid: int
    :param userid: Merge the pull request as this user.
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
      "id": <id_given_in_input>,
      "result":
        {
            "executed":         "<bool>",
            "failure_reason":   "<int>",
            "merge_commit_id":  "<merge_commit_id>",
            "possible":         "<bool>"
        },
      "error": null
 
 
 close_pull_request
 ------------------
 
 .. py:function:: close_pull_request(apiuser, repoid, pullrequestid, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Close the pull request specified by `pullrequestid`.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Repository name or repository ID to which the pull
        request belongs.
    :type repoid: str or int
    :param pullrequestid: ID of the pull request to be closed.
    :type pullrequestid: int
    :param userid: Close the pull request as this user.
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
      "id": <id_given_in_input>,
      "result":
        {
            "pull_request_id":  "<int>",
            "closed":           "<bool>"
        },
      "error": null
 
 
 comment_pull_request
 --------------------
 
 .. py:function:: comment_pull_request(apiuser, repoid, pullrequestid, message=<Optional:None>, status=<Optional:None>, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Comment on the pull request specified with the `pullrequestid`,
    in the |repo| specified by the `repoid`, and optionally change the
    review status.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param pullrequestid: The pull request ID.
    :type pullrequestid: int
    :param message: The text content of the comment.
    :type message: str
    :param status: (**Optional**) Set the approval status of the pull
        request. Valid options are:
        * not_reviewed
        * approved
        * rejected
        * under_review
    :type status: str
    :param userid: Comment on the pull request as this user
    :type userid: Optional(str or int)
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result :
        {
            "pull_request_id":  "<Integer>",
            "comment_id":       "<Integer>"
        }
      error :  null
 
 
 create_pull_request
 -------------------
 
 .. py:function:: create_pull_request(apiuser, source_repo, target_repo, source_ref, target_ref, title, description=<Optional:''>, reviewers=<Optional:None>)
 
    Creates a new pull request.
 
    Accepts refs in the following formats:
 
        * branch:<branch_name>:<sha>
        * branch:<branch_name>
        * bookmark:<bookmark_name>:<sha> (Mercurial only)
        * bookmark:<bookmark_name> (Mercurial only)
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param source_repo: Set the source repository name.
    :type source_repo: str
    :param target_repo: Set the target repository name.
    :type target_repo: str
    :param source_ref: Set the source ref name.
    :type source_ref: str
    :param target_ref: Set the target ref name.
    :type target_ref: str
    :param title: Set the pull request title.
    :type title: str
    :param description: Set the pull request description.
    :type description: Optional(str)
    :param reviewers: Set the new pull request reviewers list.
    :type reviewers: Optional(list)
 
 
 update_pull_request
 -------------------
 
 .. py:function:: update_pull_request(apiuser, repoid, pullrequestid, title=<Optional:''>, description=<Optional:''>, reviewers=<Optional:None>, update_commits=<Optional:None>, close_pull_request=<Optional:None>)
 
    Updates a pull request.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param pullrequestid: The pull request ID.
    :type pullrequestid: int
    :param title: Set the pull request title.
    :type title: str
    :param description: Update pull request description.
    :type description: Optional(str)
    :param reviewers: Update pull request reviewers list with new value.
    :type reviewers: Optional(list)
    :param update_commits: Trigger update of commits for this pull request
    :type: update_commits: Optional(bool)
    :param close_pull_request: Close this pull request with rejected state
    :type: close_pull_request: Optional(bool)
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result :
        {
            "msg": "Updated pull request `63`",
            "pull_request": <pull_request_object>,
            "updated_reviewers": {
              "added": [
                "username"
              ],
              "removed": []
            },
            "updated_commits": {
              "added": [
                "<sha1_hash>"
              ],
              "common": [
                "<sha1_hash>",
                "<sha1_hash>",
              ],
              "removed": []
            }
        }
      error :  null
 
 
 get_repo
 --------
 
 .. py:function:: get_repo(apiuser, repoid, cache=<Optional:True>)
 
    Gets an existing repository by its name or repository_id.
 
    The members section so the output returns users groups or users
    associated with that repository.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository id.
    :type repoid: str or int
    :param cache: use the cached value for last changeset
    :type: cache: Optional(bool)
 
    Example output:
 
    .. code-block:: bash
 
        {
          "error": null,
          "id": <repo_id>,
          "result": {
            "clone_uri": null,
            "created_on": "timestamp",
            "description": "repo description",
            "enable_downloads": false,
            "enable_locking": false,
            "enable_statistics": false,
            "followers": [
              {
                "active": true,
                "admin": false,
                "api_key": "****************************************",
                "api_keys": [
                  "****************************************"
                ],
                "email": "[e-mail]",
                "emails": [
                  "[e-mail]"
                ],
                "extern_name": "rhodecode",
                "extern_type": "rhodecode",
                "firstname": "username",
                "ip_addresses": [],
                "language": null,
                "last_login": "2015-09-16T17:16:35.854",
                "lastname": "surname",
                "user_id": <user_id>,
                "username": "name"
              }
            ],
            "fork_of": "parent-repo",
            "landing_rev": [
              "rev",
              "tip"
            ],
            "last_changeset": {
              "author": "User <[e-mail]>",
              "branch": "default",
              "date": "timestamp",
              "message": "last commit message",
              "parents": [
                {
                  "raw_id": "commit-id"
                }
              ],
              "raw_id": "commit-id",
              "revision": <revision number>,
              "short_id": "short id"
            },
            "lock_reason": null,
            "locked_by": null,
            "locked_date": null,
            "members": [
              {
                "name": "super-admin-name",
                "origin": "super-admin",
                "permission": "repository.admin",
                "type": "user"
              },
              {
                "name": "owner-name",
                "origin": "owner",
                "permission": "repository.admin",
                "type": "user"
              },
              {
                "name": "user-group-name",
                "origin": "permission",
                "permission": "repository.write",
                "type": "user_group"
              }
            ],
            "owner": "owner-name",
            "permissions": [
              {
                "name": "super-admin-name",
                "origin": "super-admin",
                "permission": "repository.admin",
                "type": "user"
              },
              {
                "name": "owner-name",
                "origin": "owner",
                "permission": "repository.admin",
                "type": "user"
              },
              {
                "name": "user-group-name",
                "origin": "permission",
                "permission": "repository.write",
                "type": "user_group"
              }
            ],
            "private": true,
            "repo_id": 676,
            "repo_name": "user-group/repo-name",
            "repo_type": "hg"
          }
        }
 
 
 get_repos
 ---------
 
 .. py:function:: get_repos(apiuser)
 
    Lists all existing repositories.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: [
                  {
                    "repo_id" :          "<repo_id>",
                    "repo_name" :        "<reponame>"
                    "repo_type" :        "<repo_type>",
                    "clone_uri" :        "<clone_uri>",
                    "private": :         "<bool>",
                    "created_on" :       "<datetimecreated>",
                    "description" :      "<description>",
                    "landing_rev":       "<landing_rev>",
                    "owner":             "<repo_owner>",
                    "fork_of":           "<name_of_fork_parent>",
                    "enable_downloads":  "<bool>",
                    "enable_locking":    "<bool>",
                    "enable_statistics": "<bool>",
                  },
                  ...
                ]
        error:  null
 
 
 get_repo_changeset
 ------------------
 
 .. py:function:: get_repo_changeset(apiuser, repoid, revision, details=<Optional:'basic'>)
 
    Returns information about a changeset.
 
    Additionally parameters define the amount of details returned by
    this function.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository id
    :type repoid: str or int
    :param revision: revision for which listing should be done
    :type revision: str
    :param details: details can be 'basic|extended|full' full gives diff
        info details like the diff itself, and number of changed files etc.
    :type details: Optional(str)
 
 
 get_repo_changesets
 -------------------
 
 .. py:function:: get_repo_changesets(apiuser, repoid, start_rev, limit, details=<Optional:'basic'>)
 
    Returns a set of changesets limited by the number of commits starting
    from the `start_rev` option.
 
    Additional parameters define the amount of details returned by this
    function.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param start_rev: The starting revision from where to get changesets.
    :type start_rev: str
    :param limit: Limit the number of changesets to this amount
    :type limit: str or int
    :param details: Set the level of detail returned. Valid option are:
        ``basic``, ``extended`` and ``full``.
    :type details: Optional(str)
 
    .. note::
 
       Setting the parameter `details` to the value ``full`` is extensive
       and returns details like the diff itself, and the number
       of changed files.
 
 
 get_repo_nodes
 --------------
 
 .. py:function:: get_repo_nodes(apiuser, repoid, revision, root_path, ret_type=<Optional:'all'>, details=<Optional:'basic'>)
 
    Returns a list of nodes and children in a flat list for a given
    path at given revision.
 
    It's possible to specify ret_type to show only `files` or `dirs`.
 
    This command can only be run using an |authtoken| with admin rights,
    or users with at least read rights to |repos|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: The repository name or repository ID.
    :type repoid: str or int
    :param revision: The revision for which listing should be done.
    :type revision: str
    :param root_path: The path from which to start displaying.
    :type root_path: str
    :param ret_type: Set the return type. Valid options are
        ``all`` (default), ``files`` and ``dirs``.
    :type ret_type: Optional(str)
    :param details: Returns extended information about nodes, such as
        md5, binary, and or content.  The valid options are ``basic`` and
        ``full``.
    :type details: Optional(str)
+   :param max_file_bytes: Only return file content under this file size bytes
+   :type details: Optional(int)
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: [
                  {
                    "name" : "<name>"
                    "type" : "<type>",
                    "binary": "<true|false>" (only in extended mode)
                    "md5"  : "<md5 of file content>" (only in extended mode)
                  },
                  ...
                ]
        error:  null
 
 
 create_repo
 -----------
 
 .. py:function:: create_repo(apiuser, repo_name, repo_type, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, landing_rev=<Optional:'rev:tip'>, enable_statistics=<Optional:False>, enable_locking=<Optional:False>, enable_downloads=<Optional:False>, copy_permissions=<Optional:False>)
 
    Creates a repository.
 
    * If the repository name contains "/", all the required repository
      groups will be created.
 
      For example "foo/bar/baz" will create |repo| groups "foo" and "bar"
      (with "foo" as parent). It will also create the "baz" repository
      with "bar" as |repo| group.
 
    This command can only be run using an |authtoken| with at least
    write permissions to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repo_name: Set the repository name.
    :type repo_name: str
    :param repo_type: Set the repository type; 'hg','git', or 'svn'.
    :type repo_type: str
    :param owner: user_id or username
    :type owner: Optional(str)
    :param description: Set the repository description.
    :type description: Optional(str)
    :param private:
    :type private: bool
    :param clone_uri:
    :type clone_uri: str
    :param landing_rev: <rev_type>:<rev>
    :type landing_rev: str
    :param enable_locking:
    :type enable_locking: bool
    :param enable_downloads:
    :type enable_downloads: bool
    :param enable_statistics:
    :type enable_statistics: bool
    :param copy_permissions: Copy permission from group in which the
        repository is being created.
    :type copy_permissions: bool
 
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg": "Created new repository `<reponame>`",
                  "success": true,
                  "task": "<celery task id or None if done sync>"
                }
        error:  null
 
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
         'failed to create repository `<repo_name>`
      }
 
 
 add_field_to_repo
 -----------------
 
 .. py:function:: add_field_to_repo(apiuser, repoid, key, label=<Optional:''>, description=<Optional:''>)
 
    Adds an extra field to a repository.
 
    This command can only be run using an |authtoken| with at least
    write permissions to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository id.
    :type repoid: str or int
    :param key: Create a unique field key for this repository.
    :type key: str
    :param label:
    :type label: Optional(str)
    :param description:
    :type description: Optional(str)
 
 
 remove_field_from_repo
 ----------------------
 
 .. py:function:: remove_field_from_repo(apiuser, repoid, key)
 
    Removes an extra field from a repository.
 
    This command can only be run using an |authtoken| with at least
    write permissions to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param key: Set the unique field key for this repository.
    :type key: str
 
 
 update_repo
 -----------
 
 .. py:function:: update_repo(apiuser, repoid, name=<Optional:None>, owner=<Optional:<OptionalAttr:apiuser>>, group=<Optional:None>, fork_of=<Optional:None>, description=<Optional:''>, private=<Optional:False>, clone_uri=<Optional:None>, landing_rev=<Optional:'rev:tip'>, enable_statistics=<Optional:False>, enable_locking=<Optional:False>, enable_downloads=<Optional:False>, fields=<Optional:''>)
 
    Updates a repository with the given information.
 
    This command can only be run using an |authtoken| with at least
    write permissions to the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: repository name or repository ID.
    :type repoid: str or int
    :param name: Update the |repo| name.
    :type name: str
    :param owner: Set the |repo| owner.
    :type owner: str
    :param group: Set the |repo| group the |repo| belongs to.
    :type group: str
    :param fork_of: Set the master |repo| name.
    :type fork_of: str
    :param description: Update the |repo| description.
    :type description: str
    :param private: Set the |repo| as private. (True | False)
    :type private: bool
    :param clone_uri: Update the |repo| clone URI.
    :type clone_uri: str
    :param landing_rev: Set the |repo| landing revision. Default is
        ``tip``.
    :type landing_rev: str
    :param enable_statistics: Enable statistics on the |repo|,
        (True | False).
    :type enable_statistics: bool
    :param enable_locking: Enable |repo| locking.
    :type enable_locking: bool
    :param enable_downloads: Enable downloads from the |repo|,
        (True | False).
    :type enable_downloads: bool
    :param fields: Add extra fields to the |repo|. Use the following
        example format: ``field_key=field_val,field_key2=fieldval2``.
        Escape ', ' with \,
    :type fields: str
 
 
 fork_repo
 ---------
 
 .. py:function:: fork_repo(apiuser, repoid, fork_name, owner=<Optional:<OptionalAttr:apiuser>>, description=<Optional:''>, copy_permissions=<Optional:False>, private=<Optional:False>, landing_rev=<Optional:'rev:tip'>)
 
    Creates a fork of the specified |repo|.
 
    * If using |RCE| with Celery this will immediately return a success
      message, even though the fork will be created asynchronously.
 
    This command can only be run using an |authtoken| with fork
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set repository name or repository ID.
    :type repoid: str or int
    :param fork_name: Set the fork name.
    :type fork_name: str
    :param owner: Set the fork owner.
    :type owner: str
    :param description: Set the fork descripton.
    :type description: str
    :param copy_permissions: Copy permissions from parent |repo|. The
        default is False.
    :type copy_permissions: bool
    :param private: Make the fork private. The default is False.
    :type private: bool
    :param landing_rev: Set the landing revision. The default is tip.
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_for_response>
        api_key : "<api_key>"
        args:     {
                    "repoid" :          "<reponame or repo_id>",
                    "fork_name":        "<forkname>",
                    "owner":            "<username or user_id = Optional(=apiuser)>",
                    "description":      "<description>",
                    "copy_permissions": "<bool>",
                    "private":          "<bool>",
                    "landing_rev":      "<landing_rev>"
                  }
 
    Example error output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg": "Created fork of `<reponame>` as `<forkname>`",
                  "success": true,
                  "task": "<celery task id or None if done sync>"
                }
        error:  null
 
 
 delete_repo
 -----------
 
 .. py:function:: delete_repo(apiuser, repoid, forks=<Optional:''>)
 
    Deletes a repository.
 
    * When the `forks` parameter is set it's possible to detach or delete
      forks of deleted repository.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param forks: Set to `detach` or `delete` forks from the |repo|.
    :type forks: Optional(str)
 
    Example error output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg": "Deleted repository `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 comment_commit
 --------------
 
 .. py:function:: comment_commit(apiuser, repoid, commit_id, message, userid=<Optional:<OptionalAttr:apiuser>>, status=<Optional:None>)
 
    Set a commit comment, and optionally change the status of the commit.
    This command can be executed only using api_key belonging to user
    with admin rights, or repository administrator.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param commit_id: Specify the commit_id for which to set a comment.
    :type commit_id: str
    :param message: The comment text.
    :type message: str
    :param userid: Set the user name of the comment creator.
    :type userid: Optional(str or int)
    :param status: status, one of 'not_reviewed', 'approved', 'rejected',
       'under_review'
    :type status: str
 
    Example error output:
 
    .. code-block:: json
 
        {
            "id" : <id_given_in_input>,
            "result" : {
                "msg": "Commented on commit `<commit_id>` for repository `<repoid>`",
                "status_change": null or <status>,
                "success": true
            },
            "error" :  null
        }
 
 
 changeset_comment
 -----------------
 
 .. py:function:: changeset_comment(apiuser, repoid, revision, message, userid=<Optional:<OptionalAttr:apiuser>>, status=<Optional:None>)
 
    .. deprecated:: 3.4.0
 
       Please use method `comment_commit` instead.
 
 
    Set a changeset comment, and optionally change the status of the
    changeset.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param revision: Specify the revision for which to set a comment.
    :type revision: str
    :param message: The comment text.
    :type message: str
    :param userid: Set the user name of the comment creator.
    :type userid: Optional(str or int)
    :param status: Set the comment status. The following are valid options:
        * not_reviewed
        * approved
        * rejected
        * under_review
    :type status: str
 
    Example error output:
 
    .. code-block:: json
 
        {
            "id" : <id_given_in_input>,
            "result" : {
                "msg": "Commented on commit `<revision>` for repository `<repoid>`",
                "status_change": null or <status>,
                "success": true
            },
            "error" : null
        }
 
 
 grant_user_permission
 ---------------------
 
 .. py:function:: grant_user_permission(apiuser, repoid, userid, perm)
 
    Grant permissions for the specified user on the given repository,
    or update existing permissions if found.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param userid: Set the user name.
    :type userid: str
    :param perm: Set the user permissions, using the following format
        ``(repository.(none|read|write|admin))``
    :type perm: str
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Granted perm: `<perm>` for user: `<username>` in repo: `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 revoke_user_permission
 ----------------------
 
 .. py:function:: revoke_user_permission(apiuser, repoid, userid)
 
    Revoke permission for a user on the specified repository.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param userid: Set the user name of revoked user.
    :type userid: str or int
 
    Example error output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Revoked perm for user: `<username>` in repo: `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 grant_user_group_permission
 ---------------------------
 
 .. py:function:: grant_user_group_permission(apiuser, repoid, usergroupid, perm)
 
    Grant permission for a user group on the specified repository,
    or update existing permissions.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param usergroupid: Specify the ID of the user group.
    :type usergroupid: str or int
    :param perm: Set the user group permissions using the following
        format: (repository.(none|read|write|admin))
    :type perm: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg" : "Granted perm: `<perm>` for group: `<usersgroupname>` in repo: `<reponame>`",
        "success": true
 
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to edit permission for user group: `<usergroup>` in repo `<repo>`'
      }
 
 
 revoke_user_group_permission
 ----------------------------
 
 .. py:function:: revoke_user_group_permission(apiuser, repoid, usergroupid)
 
    Revoke the permissions of a user group on a given repository.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo|.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repoid: Set the repository name or repository ID.
    :type repoid: str or int
    :param usergroupid: Specify the user group ID.
    :type usergroupid: str or int
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Revoked perm for group: `<usersgroupname>` in repo: `<reponame>`",
                  "success": true
                }
        error:  null
 
 
 get_repo_group
 --------------
 
 .. py:function:: get_repo_group(apiuser, repogroupid)
 
    Return the specified |repo| group, along with permissions,
    and repositories inside the group
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repogroupid: Specify the name of ID of the repository group.
    :type repogroupid: str or int
 
 
    Example output:
 
    .. code-block:: bash
 
        {
          "error": null,
          "id": repo-group-id,
          "result": {
            "group_description": "repo group description",
            "group_id": 14,
            "group_name": "group name",
            "members": [
              {
                "name": "super-admin-username",
                "origin": "super-admin",
                "permission": "group.admin",
                "type": "user"
              },
              {
                "name": "owner-name",
                "origin": "owner",
                "permission": "group.admin",
                "type": "user"
              },
              {
                "name": "user-group-name",
                "origin": "permission",
                "permission": "group.write",
                "type": "user_group"
              }
            ],
            "owner": "owner-name",
            "parent_group": null,
            "repositories": [ repo-list ]
          }
        }
 
 
 get_repo_groups
 ---------------
 
 .. py:function:: get_repo_groups(apiuser)
 
    Returns all repository groups.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
 
 
 create_repo_group
 -----------------
 
 .. py:function:: create_repo_group(apiuser, group_name, description=<Optional:''>, owner=<Optional:<OptionalAttr:apiuser>>, copy_permissions=<Optional:False>)
 
    Creates a repository group.
 
    * If the repository group name contains "/", all the required repository
      groups will be created.
 
      For example "foo/bar/baz" will create |repo| groups "foo" and "bar"
      (with "foo" as parent). It will also create the "baz" repository
      with "bar" as |repo| group.
 
    This command can only be run using an |authtoken| with admin
    permissions.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param group_name: Set the repository group name.
    :type group_name: str
    :param description: Set the |repo| group description.
    :type description: str
    :param owner: Set the |repo| group owner.
    :type owner: str
    :param copy_permissions:
    :type copy_permissions:
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
          "msg": "Created new repo group `<repo_group_name>`"
          "repo_group": <repogroup_object>
      }
      error :  null
 
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        failed to create repo group `<repogroupid>`
      }
 
 
 update_repo_group
 -----------------
 
 .. py:function:: update_repo_group(apiuser, repogroupid, group_name=<Optional:''>, description=<Optional:''>, owner=<Optional:<OptionalAttr:apiuser>>, parent=<Optional:None>, enable_locking=<Optional:False>)
 
    Updates repository group with the details given.
 
    This command can only be run using an |authtoken| with admin
    permissions.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repogroupid: Set the ID of repository group.
    :type repogroupid: str or int
    :param group_name: Set the name of the |repo| group.
    :type group_name: str
    :param description: Set a description for the group.
    :type description: str
    :param owner: Set the |repo| group owner.
    :type owner: str
    :param parent: Set the |repo| group parent.
    :type parent: str or int
    :param enable_locking: Enable |repo| locking. The default is false.
    :type enable_locking: bool
 
 
 delete_repo_group
 -----------------
 
 .. py:function:: delete_repo_group(apiuser, repogroupid)
 
    Deletes a |repo| group.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repogroupid: Set the name or ID of repository group to be
        deleted.
    :type repogroupid: str or int
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        'msg': 'deleted repo group ID:<repogroupid> <repogroupname>
        'repo_group': null
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to delete repo group ID:<repogroupid> <repogroupname>"
      }
 
 
 grant_user_permission_to_repo_group
 -----------------------------------
 
 .. py:function:: grant_user_permission_to_repo_group(apiuser, repogroupid, userid, perm, apply_to_children=<Optional:'none'>)
 
    Grant permission for a user on the given repository group, or update
    existing permissions if found.
 
    This command can only be run using an |authtoken| with admin
    permissions.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repogroupid: Set the name or ID of repository group.
    :type repogroupid: str or int
    :param userid: Set the user name.
    :type userid: str
    :param perm: (group.(none|read|write|admin))
    :type perm: str
    :param apply_to_children: 'none', 'repos', 'groups', 'all'
    :type apply_to_children: str
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Granted perm: `<perm>` (recursive:<apply_to_children>) for user: `<username>` in repo group: `<repo_group_name>`",
                  "success": true
                }
        error:  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to edit permission for user: `<userid>` in repo group: `<repo_group_name>`"
      }
 
 
 revoke_user_permission_from_repo_group
 --------------------------------------
 
 .. py:function:: revoke_user_permission_from_repo_group(apiuser, repogroupid, userid, apply_to_children=<Optional:'none'>)
 
    Revoke permission for a user in a given repository group.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo| group.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repogroupid: Set the name or ID of the repository group.
    :type repogroupid: str or int
    :param userid: Set the user name to revoke.
    :type userid: str
    :param apply_to_children: 'none', 'repos', 'groups', 'all'
    :type apply_to_children: str
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Revoked perm (recursive:<apply_to_children>) for user: `<username>` in repo group: `<repo_group_name>`",
                  "success": true
                }
        error:  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to edit permission for user: `<userid>` in repo group: `<repo_group_name>`"
      }
 
 
 grant_user_group_permission_to_repo_group
 -----------------------------------------
 
 .. py:function:: grant_user_group_permission_to_repo_group(apiuser, repogroupid, usergroupid, perm, apply_to_children=<Optional:'none'>)
 
    Grant permission for a user group on given repository group, or update
    existing permissions if found.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo| group.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repogroupid: Set the name or id of repository group
    :type repogroupid: str or int
    :param usergroupid: id of usergroup
    :type usergroupid: str or int
    :param perm: (group.(none|read|write|admin))
    :type perm: str
    :param apply_to_children: 'none', 'repos', 'groups', 'all'
    :type apply_to_children: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg" : "Granted perm: `<perm>` (recursive:<apply_to_children>) for user group: `<usersgroupname>` in repo group: `<repo_group_name>`",
        "success": true
 
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to edit permission for user group: `<usergroup>` in repo group: `<repo_group_name>`"
      }
 
 
 revoke_user_group_permission_from_repo_group
 --------------------------------------------
 
 .. py:function:: revoke_user_group_permission_from_repo_group(apiuser, repogroupid, usergroupid, apply_to_children=<Optional:'none'>)
 
    Revoke permission for user group on given repository.
 
    This command can only be run using an |authtoken| with admin
    permissions on the |repo| group.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param repogroupid: name or id of repository group
    :type repogroupid: str or int
    :param usergroupid:
    :param apply_to_children: 'none', 'repos', 'groups', 'all'
    :type apply_to_children: str
 
    Example output:
 
    .. code-block:: bash
 
        id : <id_given_in_input>
        result: {
                  "msg" : "Revoked perm (recursive:<apply_to_children>) for user group: `<usersgroupname>` in repo group: `<repo_group_name>`",
                  "success": true
                }
        error:  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to edit permission for user group: `<usergroup>` in repo group: `<repo_group_name>`"
      }
 
 
 get_gist
 --------
 
 .. py:function:: get_gist(apiuser, gistid, content=<Optional:False>)
 
    Get the specified gist, based on the gist ID.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param gistid: Set the id of the private or public gist
    :type gistid: str
    :param content: Return the gist content. Default is false.
    :type content: Optional(bool)
 
 
 get_gists
 ---------
 
 .. py:function:: get_gists(apiuser, userid=<Optional:<OptionalAttr:apiuser>>)
 
    Get all gists for given user. If userid is empty returned gists
    are for user who called the api
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param userid: user to get gists for
    :type userid: Optional(str or int)
 
 
 create_gist
 -----------
 
 .. py:function:: create_gist(apiuser, files, owner=<Optional:<OptionalAttr:apiuser>>, gist_type=<Optional:u'public'>, lifetime=<Optional:-1>, acl_level=<Optional:u'acl_public'>, description=<Optional:''>)
 
    Creates a new Gist.
 
    :param apiuser: This is filled automatically from the |authtoken|.
    :type apiuser: AuthUser
    :param files: files to be added to the gist. The data structure has
        to match the following example::
 
          {'filename': {'content':'...', 'lexer': null},
           'filename2': {'content':'...', 'lexer': null}}
 
    :type files: dict
    :param owner: Set the gist owner, defaults to api method caller
    :type owner: Optional(str or int)
    :param gist_type: type of gist ``public`` or ``private``
    :type gist_type: Optional(str)
    :param lifetime: time in minutes of gist lifetime
    :type lifetime: Optional(int)
    :param acl_level: acl level for this gist, can be
        ``acl_public`` or ``acl_private`` If the value is set to
        ``acl_private`` only logged in users are able to access this gist.
        If not set it defaults to ``acl_public``.
    :type acl_level: Optional(str)
    :param description: gist description
    :type description: Optional(str)
 
    Example  output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "msg": "created new gist",
        "gist": {}
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to create gist"
      }
 
 
 delete_gist
 -----------
 
 .. py:function:: delete_gist(apiuser, gistid)
 
    Deletes existing gist
 
    :param apiuser: filled automatically from apikey
    :type apiuser: AuthUser
    :param gistid: id of gist to delete
    :type gistid: str
 
    Example output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : {
        "deleted gist ID: <gist_id>",
        "gist": null
      }
      error :  null
 
    Example error output:
 
    .. code-block:: bash
 
      id : <id_given_in_input>
      result : null
      error :  {
        "failed to delete gist ID:<gist_id>"
      }
-

rhodecode/api/tests/test_get_repo_nodes.py

@@ -1,119 +1,142 b''
 # -*- coding: utf-8 -*-
 
 # Copyright (C) 2010-2016  RhodeCode GmbH
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License, version 3
 # (only), as published by the Free Software Foundation.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 # This program is dual-licensed. If you wish to learn more about the
 # RhodeCode Enterprise Edition, including its added features, Support services,
 # and proprietary license terms, please see https://rhodecode.com/licenses/
 
 
 import pytest
 
 from rhodecode.model.meta import Session
 from rhodecode.model.repo import RepoModel
 from rhodecode.api.tests.utils import (
     build_data, api_call, assert_error, assert_ok)
 
 
 @pytest.mark.usefixtures("testuser_api", "app")
 class TestGetRepoNodes(object):
     @pytest.mark.parametrize("name, ret_type", [
         ('all', 'all'),
         ('dirs', 'dirs'),
         ('files', 'files'),
     ])
     def test_api_get_repo_nodes(self, name, ret_type, backend):
         commit_id = 'tip'
         path = '/'
         id_, params = build_data(
             self.apikey, 'get_repo_nodes',
             repoid=backend.repo_name, revision=commit_id,
             root_path=path,
             ret_type=ret_type)
         response = api_call(self.app, params)
 
         # we don't the actual return types here since it's tested somewhere
         # else
         expected = response.json['result']
         assert_ok(id_, expected, given=response.body)
 
     def test_api_get_repo_nodes_bad_commits(self, backend):
         commit_id = 'i-dont-exist'
         path = '/'
         id_, params = build_data(
             self.apikey, 'get_repo_nodes',
             repoid=backend.repo_name, revision=commit_id,
             root_path=path, )
         response = api_call(self.app, params)
 
         expected = 'failed to get repo: `%s` nodes' % (backend.repo_name,)
         assert_error(id_, expected, given=response.body)
 
     def test_api_get_repo_nodes_bad_path(self, backend):
         commit_id = 'tip'
         path = '/idontexits'
         id_, params = build_data(
             self.apikey, 'get_repo_nodes',
             repoid=backend.repo_name, revision=commit_id,
             root_path=path, )
         response = api_call(self.app, params)
 
         expected = 'failed to get repo: `%s` nodes' % (backend.repo_name,)
         assert_error(id_, expected, given=response.body)
 
+    def test_api_get_repo_nodes_max_file_bytes(self, backend):
+        commit_id = 'tip'
+        path = '/'
+        max_file_bytes = 500
+
+        id_, params = build_data(
+            self.apikey, 'get_repo_nodes',
+            repoid=backend.repo_name, revision=commit_id, details='full',
+            root_path=path)
+        response = api_call(self.app, params)
+        assert any(file['content'] and len(file['content']) > max_file_bytes
+                   for file in response.json['result'])
+
+        id_, params = build_data(
+            self.apikey, 'get_repo_nodes',
+            repoid=backend.repo_name, revision=commit_id,
+            root_path=path, details='full',
+            max_file_bytes=max_file_bytes)
+        response = api_call(self.app, params)
+        assert all(
+            file['content'] is None if file['size'] > max_file_bytes else True
+            for file in response.json['result'])
+
     def test_api_get_repo_nodes_bad_ret_type(self, backend):
         commit_id = 'tip'
         path = '/'
         ret_type = 'error'
         id_, params = build_data(
             self.apikey, 'get_repo_nodes',
             repoid=backend.repo_name, revision=commit_id,
             root_path=path,
             ret_type=ret_type)
         response = api_call(self.app, params)
 
         expected = ('ret_type must be one of %s'
                     % (','.join(['all', 'dirs', 'files'])))
         assert_error(id_, expected, given=response.body)
 
     @pytest.mark.parametrize("name, ret_type, grant_perm", [
         ('all', 'all', 'repository.write'),
         ('dirs', 'dirs', 'repository.admin'),
         ('files', 'files', 'repository.read'),
     ])
     def test_api_get_repo_nodes_by_regular_user(
             self, name, ret_type, grant_perm, backend):
         RepoModel().grant_user_permission(repo=backend.repo_name,
                                           user=self.TEST_USER_LOGIN,
                                           perm=grant_perm)
         Session().commit()
 
         commit_id = 'tip'
         path = '/'
         id_, params = build_data(
             self.apikey_regular, 'get_repo_nodes',
             repoid=backend.repo_name, revision=commit_id,
             root_path=path,
             ret_type=ret_type)
         response = api_call(self.app, params)
 
         # we don't the actual return types here since it's tested somewhere
         # else
         expected = response.json['result']
         try:
             assert_ok(id_, expected, given=response.body)
         finally:
             RepoModel().revoke_user_permission(
                 backend.repo_name, self.TEST_USER_LOGIN)

rhodecode/api/views/repo_api.py

@@ -1,1882 +1,1886 b''
 # -*- coding: utf-8 -*-
 
 # Copyright (C) 2011-2016  RhodeCode GmbH
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License, version 3
 # (only), as published by the Free Software Foundation.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 # This program is dual-licensed. If you wish to learn more about the
 # RhodeCode Enterprise Edition, including its added features, Support services,
 # and proprietary license terms, please see https://rhodecode.com/licenses/
 
 import logging
 import time
 
 import colander
 
 from rhodecode import BACKENDS
 from rhodecode.api import jsonrpc_method, JSONRPCError, JSONRPCForbidden, json
 from rhodecode.api.utils import (
     has_superadmin_permission, Optional, OAttr, get_repo_or_error,
     get_user_group_or_error, get_user_or_error, has_repo_permissions,
     get_perm_or_error, store_update, get_repo_group_or_error, parse_args,
     get_origin, build_commit_data)
 from rhodecode.lib.auth import (
     HasPermissionAnyApi, HasRepoGroupPermissionAnyApi,
     HasUserGroupPermissionAnyApi)
 from rhodecode.lib.exceptions import StatusChangeOnClosedPullRequestError
 from rhodecode.lib.utils import map_groups
 from rhodecode.lib.utils2 import str2bool, time_to_datetime
 from rhodecode.model.changeset_status import ChangesetStatusModel
 from rhodecode.model.comment import ChangesetCommentsModel
 from rhodecode.model.db import (
     Session, ChangesetStatus, RepositoryField, Repository)
 from rhodecode.model.repo import RepoModel
 from rhodecode.model.repo_group import RepoGroupModel
 from rhodecode.model.scm import ScmModel, RepoList
 from rhodecode.model.settings import SettingsModel, VcsSettingsModel
 from rhodecode.model.validation_schema import RepoSchema
 
 log = logging.getLogger(__name__)
 
 
 @jsonrpc_method()
 def get_repo(request, apiuser, repoid, cache=Optional(True)):
     """
     Gets an existing repository by its name or repository_id.
 
     The members section so the output returns users groups or users
     associated with that repository.
 
     This command can only be run using an |authtoken| with admin rights,
     or users with at least read rights to the |repo|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: The repository name or repository id.
     :type repoid: str or int
     :param cache: use the cached value for last changeset
     :type: cache: Optional(bool)
 
     Example output:
 
     .. code-block:: bash
 
         {
           "error": null,
           "id": <repo_id>,
           "result": {
             "clone_uri": null,
             "created_on": "timestamp",
             "description": "repo description",
             "enable_downloads": false,
             "enable_locking": false,
             "enable_statistics": false,
             "followers": [
               {
                 "active": true,
                 "admin": false,
                 "api_key": "****************************************",
                 "api_keys": [
                   "****************************************"
                 ],
                 "email": "[e-mail]",
                 "emails": [
                   "[e-mail]"
                 ],
                 "extern_name": "rhodecode",
                 "extern_type": "rhodecode",
                 "firstname": "username",
                 "ip_addresses": [],
                 "language": null,
                 "last_login": "2015-09-16T17:16:35.854",
                 "lastname": "surname",
                 "user_id": <user_id>,
                 "username": "name"
               }
             ],
             "fork_of": "parent-repo",
             "landing_rev": [
               "rev",
               "tip"
             ],
             "last_changeset": {
               "author": "User <[e-mail]>",
               "branch": "default",
               "date": "timestamp",
               "message": "last commit message",
               "parents": [
                 {
                   "raw_id": "commit-id"
                 }
               ],
               "raw_id": "commit-id",
               "revision": <revision number>,
               "short_id": "short id"
             },
             "lock_reason": null,
             "locked_by": null,
             "locked_date": null,
             "members": [
               {
                 "name": "super-admin-name",
                 "origin": "super-admin",
                 "permission": "repository.admin",
                 "type": "user"
               },
               {
                 "name": "owner-name",
                 "origin": "owner",
                 "permission": "repository.admin",
                 "type": "user"
               },
               {
                 "name": "user-group-name",
                 "origin": "permission",
                 "permission": "repository.write",
                 "type": "user_group"
               }
             ],
             "owner": "owner-name",
             "permissions": [
               {
                 "name": "super-admin-name",
                 "origin": "super-admin",
                 "permission": "repository.admin",
                 "type": "user"
               },
               {
                 "name": "owner-name",
                 "origin": "owner",
                 "permission": "repository.admin",
                 "type": "user"
               },
               {
                 "name": "user-group-name",
                 "origin": "permission",
                 "permission": "repository.write",
                 "type": "user_group"
               }
             ],
             "private": true,
             "repo_id": 676,
             "repo_name": "user-group/repo-name",
             "repo_type": "hg"
           }
         }
     """
 
     repo = get_repo_or_error(repoid)
     cache = Optional.extract(cache)
     include_secrets = False
     if has_superadmin_permission(apiuser):
         include_secrets = True
     else:
         # check if we have at least read permission for this repo !
         _perms = (
             'repository.admin', 'repository.write', 'repository.read',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     permissions = []
     for _user in repo.permissions():
         user_data = {
             'name': _user.username,
             'permission': _user.permission,
             'origin': get_origin(_user),
             'type': "user",
         }
         permissions.append(user_data)
 
     for _user_group in repo.permission_user_groups():
         user_group_data = {
             'name': _user_group.users_group_name,
             'permission': _user_group.permission,
             'origin': get_origin(_user_group),
             'type': "user_group",
         }
         permissions.append(user_group_data)
 
     following_users = [
         user.user.get_api_data(include_secrets=include_secrets)
         for user in repo.followers]
 
     if not cache:
         repo.update_commit_cache()
     data = repo.get_api_data(include_secrets=include_secrets)
     data['members'] = permissions  # TODO: this should be deprecated soon
     data['permissions'] = permissions
     data['followers'] = following_users
     return data
 
 
 @jsonrpc_method()
 def get_repos(request, apiuser):
     """
     Lists all existing repositories.
 
     This command can only be run using an |authtoken| with admin rights,
     or users with at least read rights to |repos|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
 
     Example output:
 
     .. code-block:: bash
 
         id : <id_given_in_input>
         result: [
                   {
                     "repo_id" :          "<repo_id>",
                     "repo_name" :        "<reponame>"
                     "repo_type" :        "<repo_type>",
                     "clone_uri" :        "<clone_uri>",
                     "private": :         "<bool>",
                     "created_on" :       "<datetimecreated>",
                     "description" :      "<description>",
                     "landing_rev":       "<landing_rev>",
                     "owner":             "<repo_owner>",
                     "fork_of":           "<name_of_fork_parent>",
                     "enable_downloads":  "<bool>",
                     "enable_locking":    "<bool>",
                     "enable_statistics": "<bool>",
                   },
                   ...
                 ]
         error:  null
     """
 
     include_secrets = has_superadmin_permission(apiuser)
     _perms = ('repository.read', 'repository.write', 'repository.admin',)
     extras = {'user': apiuser}
 
     repo_list = RepoList(
         RepoModel().get_all(), perm_set=_perms, extra_kwargs=extras)
     return [repo.get_api_data(include_secrets=include_secrets)
             for repo in repo_list]
 
 
 @jsonrpc_method()
 def get_repo_changeset(request, apiuser, repoid, revision,
                        details=Optional('basic')):
     """
     Returns information about a changeset.
 
     Additionally parameters define the amount of details returned by
     this function.
 
     This command can only be run using an |authtoken| with admin rights,
     or users with at least read rights to the |repo|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: The repository name or repository id
     :type repoid: str or int
     :param revision: revision for which listing should be done
     :type revision: str
     :param details: details can be 'basic|extended|full' full gives diff
         info details like the diff itself, and number of changed files etc.
     :type details: Optional(str)
 
     """
     repo = get_repo_or_error(repoid)
     if not has_superadmin_permission(apiuser):
         _perms = (
             'repository.admin', 'repository.write', 'repository.read',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     changes_details = Optional.extract(details)
     _changes_details_types = ['basic', 'extended', 'full']
     if changes_details not in _changes_details_types:
         raise JSONRPCError(
             'ret_type must be one of %s' % (
                 ','.join(_changes_details_types)))
 
     pre_load = ['author', 'branch', 'date', 'message', 'parents',
                 'status', '_commit', '_file_paths']
 
     try:
         cs = repo.get_commit(commit_id=revision, pre_load=pre_load)
     except TypeError as e:
         raise JSONRPCError(e.message)
     _cs_json = cs.__json__()
     _cs_json['diff'] = build_commit_data(cs, changes_details)
     if changes_details == 'full':
         _cs_json['refs'] = {
             'branches': [cs.branch],
             'bookmarks': getattr(cs, 'bookmarks', []),
             'tags': cs.tags
         }
     return _cs_json
 
 
 @jsonrpc_method()
 def get_repo_changesets(request, apiuser, repoid, start_rev, limit,
                         details=Optional('basic')):
     """
     Returns a set of commits limited by the number starting
     from the `start_rev` option.
 
     Additional parameters define the amount of details returned by this
     function.
 
     This command can only be run using an |authtoken| with admin rights,
     or users with at least read rights to |repos|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: The repository name or repository ID.
     :type repoid: str or int
     :param start_rev: The starting revision from where to get changesets.
     :type start_rev: str
     :param limit: Limit the number of commits to this amount
     :type limit: str or int
     :param details: Set the level of detail returned. Valid option are:
         ``basic``, ``extended`` and ``full``.
     :type details: Optional(str)
 
     .. note::
 
        Setting the parameter `details` to the value ``full`` is extensive
        and returns details like the diff itself, and the number
        of changed files.
 
     """
     repo = get_repo_or_error(repoid)
     if not has_superadmin_permission(apiuser):
         _perms = (
             'repository.admin', 'repository.write', 'repository.read',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     changes_details = Optional.extract(details)
     _changes_details_types = ['basic', 'extended', 'full']
     if changes_details not in _changes_details_types:
         raise JSONRPCError(
             'ret_type must be one of %s' % (
                 ','.join(_changes_details_types)))
 
     limit = int(limit)
     pre_load = ['author', 'branch', 'date', 'message', 'parents',
                 'status', '_commit', '_file_paths']
 
     vcs_repo = repo.scm_instance()
     # SVN needs a special case to distinguish its index and commit id
     if vcs_repo and vcs_repo.alias == 'svn' and (start_rev == '0'):
         start_rev = vcs_repo.commit_ids[0]
 
     try:
         commits = vcs_repo.get_commits(
             start_id=start_rev, pre_load=pre_load)
     except TypeError as e:
         raise JSONRPCError(e.message)
     except Exception:
         log.exception('Fetching of commits failed')
         raise JSONRPCError('Error occurred during commit fetching')
 
     ret = []
     for cnt, commit in enumerate(commits):
         if cnt >= limit != -1:
             break
         _cs_json = commit.__json__()
         _cs_json['diff'] = build_commit_data(commit, changes_details)
         if changes_details == 'full':
             _cs_json['refs'] = {
                 'branches': [commit.branch],
                 'bookmarks': getattr(commit, 'bookmarks', []),
                 'tags': commit.tags
             }
         ret.append(_cs_json)
     return ret
 
 
 @jsonrpc_method()
 def get_repo_nodes(request, apiuser, repoid, revision, root_path,
-                   ret_type=Optional('all'), details=Optional('basic')):
+                   ret_type=Optional('all'), details=Optional('basic'),
+                   max_file_bytes=Optional(None)):
     """
     Returns a list of nodes and children in a flat list for a given
     path at given revision.
 
     It's possible to specify ret_type to show only `files` or `dirs`.
 
     This command can only be run using an |authtoken| with admin rights,
     or users with at least read rights to |repos|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: The repository name or repository ID.
     :type repoid: str or int
     :param revision: The revision for which listing should be done.
     :type revision: str
     :param root_path: The path from which to start displaying.
     :type root_path: str
     :param ret_type: Set the return type. Valid options are
         ``all`` (default), ``files`` and ``dirs``.
     :type ret_type: Optional(str)
     :param details: Returns extended information about nodes, such as
         md5, binary, and or content.  The valid options are ``basic`` and
         ``full``.
     :type details: Optional(str)
+    :param max_file_bytes: Only return file content under this file size bytes
+    :type details: Optional(int)
 
     Example output:
 
     .. code-block:: bash
 
         id : <id_given_in_input>
         result: [
                   {
                     "name" : "<name>"
                     "type" : "<type>",
                     "binary": "<true|false>" (only in extended mode)
                     "md5"  : "<md5 of file content>" (only in extended mode)
                   },
                   ...
                 ]
         error:  null
     """
 
     repo = get_repo_or_error(repoid)
     if not has_superadmin_permission(apiuser):
         _perms = (
             'repository.admin', 'repository.write', 'repository.read',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     ret_type = Optional.extract(ret_type)
     details = Optional.extract(details)
     _extended_types = ['basic', 'full']
     if details not in _extended_types:
         raise JSONRPCError(
             'ret_type must be one of %s' % (','.join(_extended_types)))
     extended_info = False
     content = False
     if details == 'basic':
         extended_info = True
 
     if details == 'full':
         extended_info = content = True
 
     _map = {}
     try:
         # check if repo is not empty by any chance, skip quicker if it is.
         _scm = repo.scm_instance()
         if _scm.is_empty():
             return []
 
         _d, _f = ScmModel().get_nodes(
             repo, revision, root_path, flat=False,
-            extended_info=extended_info, content=content)
+            extended_info=extended_info, content=content,
+            max_file_bytes=max_file_bytes)
         _map = {
             'all': _d + _f,
             'files': _f,
             'dirs': _d,
         }
         return _map[ret_type]
     except KeyError:
         raise JSONRPCError(
             'ret_type must be one of %s' % (','.join(sorted(_map.keys()))))
     except Exception:
         log.exception("Exception occurred while trying to get repo nodes")
         raise JSONRPCError(
             'failed to get repo: `%s` nodes' % repo.repo_name
         )
 
 
 @jsonrpc_method()
 def get_repo_refs(request, apiuser, repoid):
     """
     Returns a dictionary of current references. It returns
     bookmarks, branches, closed_branches, and tags for given repository
 
     It's possible to specify ret_type to show only `files` or `dirs`.
 
     This command can only be run using an |authtoken| with admin rights,
     or users with at least read rights to |repos|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: The repository name or repository ID.
     :type repoid: str or int
 
     Example output:
 
     .. code-block:: bash
 
         id : <id_given_in_input>
         result: [
                   TODO...
                 ]
         error:  null
     """
 
     repo = get_repo_or_error(repoid)
     if not has_superadmin_permission(apiuser):
         _perms = ('repository.admin', 'repository.write', 'repository.read',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     try:
         # check if repo is not empty by any chance, skip quicker if it is.
         vcs_instance = repo.scm_instance()
         refs = vcs_instance.refs()
         return refs
     except Exception:
         log.exception("Exception occurred while trying to get repo refs")
         raise JSONRPCError(
             'failed to get repo: `%s` references' % repo.repo_name
         )
 
 
 @jsonrpc_method()
 def create_repo(request, apiuser, repo_name, repo_type,
                 owner=Optional(OAttr('apiuser')), description=Optional(''),
                 private=Optional(False), clone_uri=Optional(None),
                 landing_rev=Optional('rev:tip'),
                 enable_statistics=Optional(False),
                 enable_locking=Optional(False),
                 enable_downloads=Optional(False),
                 copy_permissions=Optional(False)):
     """
     Creates a repository.
 
     * If the repository name contains "/", all the required repository
       groups will be created.
 
       For example "foo/bar/baz" will create |repo| groups "foo" and "bar"
       (with "foo" as parent). It will also create the "baz" repository
       with "bar" as |repo| group.
 
     This command can only be run using an |authtoken| with at least
     write permissions to the |repo|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repo_name: Set the repository name.
     :type repo_name: str
     :param repo_type: Set the repository type; 'hg','git', or 'svn'.
     :type repo_type: str
     :param owner: user_id or username
     :type owner: Optional(str)
     :param description: Set the repository description.
     :type description: Optional(str)
     :param private:
     :type private: bool
     :param clone_uri:
     :type clone_uri: str
     :param landing_rev: <rev_type>:<rev>
     :type landing_rev: str
     :param enable_locking:
     :type enable_locking: bool
     :param enable_downloads:
     :type enable_downloads: bool
     :param enable_statistics:
     :type enable_statistics: bool
     :param copy_permissions: Copy permission from group in which the
         repository is being created.
     :type copy_permissions: bool
 
 
     Example output:
 
     .. code-block:: bash
 
         id : <id_given_in_input>
         result: {
                   "msg": "Created new repository `<reponame>`",
                   "success": true,
                   "task": "<celery task id or None if done sync>"
                 }
         error:  null
 
 
     Example error output:
 
     .. code-block:: bash
 
       id : <id_given_in_input>
       result : null
       error :  {
          'failed to create repository `<repo_name>`
       }
 
     """
     schema = RepoSchema()
     try:
         data = schema.deserialize({
             'repo_name': repo_name
         })
     except colander.Invalid as e:
         raise JSONRPCError("Validation failed: %s" % (e.asdict(),))
     repo_name = data['repo_name']
 
     (repo_name_cleaned,
      parent_group_name) = RepoGroupModel()._get_group_name_and_parent(
         repo_name)
 
     if not HasPermissionAnyApi(
             'hg.admin', 'hg.create.repository')(user=apiuser):
         # check if we have admin permission for this repo group if given !
 
         if parent_group_name:
             repogroupid = parent_group_name
             repo_group = get_repo_group_or_error(parent_group_name)
 
             _perms = ('group.admin',)
             if not HasRepoGroupPermissionAnyApi(*_perms)(
                     user=apiuser, group_name=repo_group.group_name):
                 raise JSONRPCError(
                     'repository group `%s` does not exist' % (
                         repogroupid,))
         else:
             raise JSONRPCForbidden()
 
     if not has_superadmin_permission(apiuser):
         if not isinstance(owner, Optional):
             # forbid setting owner for non-admins
             raise JSONRPCError(
                 'Only RhodeCode admin can specify `owner` param')
 
     if isinstance(owner, Optional):
         owner = apiuser.user_id
 
     owner = get_user_or_error(owner)
 
     if RepoModel().get_by_repo_name(repo_name):
         raise JSONRPCError("repo `%s` already exist" % repo_name)
 
     defs = SettingsModel().get_default_repo_settings(strip_prefix=True)
     if isinstance(private, Optional):
         private = defs.get('repo_private') or Optional.extract(private)
     if isinstance(repo_type, Optional):
         repo_type = defs.get('repo_type')
     if isinstance(enable_statistics, Optional):
         enable_statistics = defs.get('repo_enable_statistics')
     if isinstance(enable_locking, Optional):
         enable_locking = defs.get('repo_enable_locking')
     if isinstance(enable_downloads, Optional):
         enable_downloads = defs.get('repo_enable_downloads')
 
     clone_uri = Optional.extract(clone_uri)
     description = Optional.extract(description)
     landing_rev = Optional.extract(landing_rev)
     copy_permissions = Optional.extract(copy_permissions)
 
     try:
         # create structure of groups and return the last group
         repo_group = map_groups(repo_name)
         data = {
             'repo_name': repo_name_cleaned,
             'repo_name_full': repo_name,
             'repo_type': repo_type,
             'repo_description': description,
             'owner': owner,
             'repo_private': private,
             'clone_uri': clone_uri,
             'repo_group': repo_group.group_id if repo_group else None,
             'repo_landing_rev': landing_rev,
             'enable_statistics': enable_statistics,
             'enable_locking': enable_locking,
             'enable_downloads': enable_downloads,
             'repo_copy_permissions': copy_permissions,
         }
 
         if repo_type not in BACKENDS.keys():
             raise Exception("Invalid backend type %s" % repo_type)
         task = RepoModel().create(form_data=data, cur_user=owner)
         from celery.result import BaseAsyncResult
         task_id = None
         if isinstance(task, BaseAsyncResult):
             task_id = task.task_id
         # no commit, it's done in RepoModel, or async via celery
         return {
             'msg': "Created new repository `%s`" % (repo_name,),
             'success': True,  # cannot return the repo data here since fork
             # cann be done async
             'task': task_id
         }
     except Exception:
         log.exception(
             u"Exception while trying to create the repository %s",
             repo_name)
         raise JSONRPCError(
             'failed to create repository `%s`' % (repo_name,))
 
 
 @jsonrpc_method()
 def add_field_to_repo(request, apiuser, repoid, key, label=Optional(''),
                       description=Optional('')):
     """
     Adds an extra field to a repository.
 
     This command can only be run using an |authtoken| with at least
     write permissions to the |repo|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: Set the repository name or repository id.
     :type repoid: str or int
     :param key: Create a unique field key for this repository.
     :type key: str
     :param label:
     :type label: Optional(str)
     :param description:
     :type description: Optional(str)
     """
     repo = get_repo_or_error(repoid)
     if not has_superadmin_permission(apiuser):
         _perms = ('repository.admin',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     label = Optional.extract(label) or key
     description = Optional.extract(description)
 
     field = RepositoryField.get_by_key_name(key, repo)
     if field:
         raise JSONRPCError('Field with key '
                            '`%s` exists for repo `%s`' % (key, repoid))
 
     try:
         RepoModel().add_repo_field(repo, key, field_label=label,
                                    field_desc=description)
         Session().commit()
         return {
             'msg': "Added new repository field `%s`" % (key,),
             'success': True,
         }
     except Exception:
         log.exception("Exception occurred while trying to add field to repo")
         raise JSONRPCError(
             'failed to create new field for repository `%s`' % (repoid,))
 
 
 @jsonrpc_method()
 def remove_field_from_repo(request, apiuser, repoid, key):
     """
     Removes an extra field from a repository.
 
     This command can only be run using an |authtoken| with at least
     write permissions to the |repo|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: Set the repository name or repository ID.
     :type repoid: str or int
     :param key: Set the unique field key for this repository.
     :type key: str
     """
 
     repo = get_repo_or_error(repoid)
     if not has_superadmin_permission(apiuser):
         _perms = ('repository.admin',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     field = RepositoryField.get_by_key_name(key, repo)
     if not field:
         raise JSONRPCError('Field with key `%s` does not '
                            'exists for repo `%s`' % (key, repoid))
 
     try:
         RepoModel().delete_repo_field(repo, field_key=key)
         Session().commit()
         return {
             'msg': "Deleted repository field `%s`" % (key,),
             'success': True,
         }
     except Exception:
         log.exception(
             "Exception occurred while trying to delete field from repo")
         raise JSONRPCError(
             'failed to delete field for repository `%s`' % (repoid,))
 
 
 @jsonrpc_method()
 def update_repo(request, apiuser, repoid, name=Optional(None),
                 owner=Optional(OAttr('apiuser')),
                 group=Optional(None),
                 fork_of=Optional(None),
                 description=Optional(''), private=Optional(False),
                 clone_uri=Optional(None), landing_rev=Optional('rev:tip'),
                 enable_statistics=Optional(False),
                 enable_locking=Optional(False),
                 enable_downloads=Optional(False),
                 fields=Optional('')):
     """
     Updates a repository with the given information.
 
     This command can only be run using an |authtoken| with at least
     write permissions to the |repo|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: repository name or repository ID.
     :type repoid: str or int
     :param name: Update the |repo| name.
     :type name: str
     :param owner: Set the |repo| owner.
     :type owner: str
     :param group: Set the |repo| group the |repo| belongs to.
     :type group: str
     :param fork_of: Set the master |repo| name.
     :type fork_of: str
     :param description: Update the |repo| description.
     :type description: str
     :param private: Set the |repo| as private. (True | False)
     :type private: bool
     :param clone_uri: Update the |repo| clone URI.
     :type clone_uri: str
     :param landing_rev: Set the |repo| landing revision. Default is
         ``tip``.
     :type landing_rev: str
     :param enable_statistics: Enable statistics on the |repo|,
         (True | False).
     :type enable_statistics: bool
     :param enable_locking: Enable |repo| locking.
     :type enable_locking: bool
     :param enable_downloads: Enable downloads from the |repo|,
         (True | False).
     :type enable_downloads: bool
     :param fields: Add extra fields to the |repo|. Use the following
         example format: ``field_key=field_val,field_key2=fieldval2``.
         Escape ', ' with \,
     :type fields: str
     """
     repo = get_repo_or_error(repoid)
     include_secrets = False
     if has_superadmin_permission(apiuser):
         include_secrets = True
     else:
         _perms = ('repository.admin',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     updates = {
         # update function requires this.
         'repo_name': repo.just_name
     }
     repo_group = group
     if not isinstance(repo_group, Optional):
         repo_group = get_repo_group_or_error(repo_group)
         repo_group = repo_group.group_id
 
     repo_fork_of = fork_of
     if not isinstance(repo_fork_of, Optional):
         repo_fork_of = get_repo_or_error(repo_fork_of)
         repo_fork_of = repo_fork_of.repo_id
 
     try:
         store_update(updates, name, 'repo_name')
         store_update(updates, repo_group, 'repo_group')
         store_update(updates, repo_fork_of, 'fork_id')
         store_update(updates, owner, 'user')
         store_update(updates, description, 'repo_description')
         store_update(updates, private, 'repo_private')
         store_update(updates, clone_uri, 'clone_uri')
         store_update(updates, landing_rev, 'repo_landing_rev')
         store_update(updates, enable_statistics, 'repo_enable_statistics')
         store_update(updates, enable_locking, 'repo_enable_locking')
         store_update(updates, enable_downloads, 'repo_enable_downloads')
 
         # extra fields
         fields = parse_args(Optional.extract(fields), key_prefix='ex_')
         if fields:
             updates.update(fields)
 
         RepoModel().update(repo, **updates)
         Session().commit()
         return {
             'msg': 'updated repo ID:%s %s' % (
                 repo.repo_id, repo.repo_name),
             'repository': repo.get_api_data(
                 include_secrets=include_secrets)
         }
     except Exception:
         log.exception(
             u"Exception while trying to update the repository %s",
             repoid)
         raise JSONRPCError('failed to update repo `%s`' % repoid)
 
 
 @jsonrpc_method()
 def fork_repo(request, apiuser, repoid, fork_name,
               owner=Optional(OAttr('apiuser')),
               description=Optional(''), copy_permissions=Optional(False),
               private=Optional(False), landing_rev=Optional('rev:tip')):
     """
     Creates a fork of the specified |repo|.
 
     * If using |RCE| with Celery this will immediately return a success
       message, even though the fork will be created asynchronously.
 
     This command can only be run using an |authtoken| with fork
     permissions on the |repo|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: Set repository name or repository ID.
     :type repoid: str or int
     :param fork_name: Set the fork name.
     :type fork_name: str
     :param owner: Set the fork owner.
     :type owner: str
     :param description: Set the fork descripton.
     :type description: str
     :param copy_permissions: Copy permissions from parent |repo|. The
         default is False.
     :type copy_permissions: bool
     :param private: Make the fork private. The default is False.
     :type private: bool
     :param landing_rev: Set the landing revision. The default is tip.
 
     Example output:
 
     .. code-block:: bash
 
         id : <id_for_response>
         api_key : "<api_key>"
         args:     {
                     "repoid" :          "<reponame or repo_id>",
                     "fork_name":        "<forkname>",
                     "owner":            "<username or user_id = Optional(=apiuser)>",
                     "description":      "<description>",
                     "copy_permissions": "<bool>",
                     "private":          "<bool>",
                     "landing_rev":      "<landing_rev>"
                   }
 
     Example error output:
 
     .. code-block:: bash
 
         id : <id_given_in_input>
         result: {
                   "msg": "Created fork of `<reponame>` as `<forkname>`",
                   "success": true,
                   "task": "<celery task id or None if done sync>"
                 }
         error:  null
 
     """
     if not has_superadmin_permission(apiuser):
         if not HasPermissionAnyApi('hg.fork.repository')(user=apiuser):
             raise JSONRPCForbidden()
 
     repo = get_repo_or_error(repoid)
     repo_name = repo.repo_name
 
     (fork_name_cleaned,
      parent_group_name) = RepoGroupModel()._get_group_name_and_parent(
         fork_name)
 
     if not has_superadmin_permission(apiuser):
         # check if we have at least read permission for
         # this repo that we fork !
         _perms = (
             'repository.admin', 'repository.write', 'repository.read')
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
         if not isinstance(owner, Optional):
             # forbid setting owner for non super admins
             raise JSONRPCError(
                 'Only RhodeCode admin can specify `owner` param'
             )
         # check if we have a create.repo permission if not maybe the parent
         # group permission
         if not HasPermissionAnyApi('hg.create.repository')(user=apiuser):
             if parent_group_name:
                 repogroupid = parent_group_name
                 repo_group = get_repo_group_or_error(parent_group_name)
 
                 _perms = ('group.admin',)
                 if not HasRepoGroupPermissionAnyApi(*_perms)(
                         user=apiuser, group_name=repo_group.group_name):
                     raise JSONRPCError(
                         'repository group `%s` does not exist' % (
                             repogroupid,))
             else:
                 raise JSONRPCForbidden()
 
     _repo = RepoModel().get_by_repo_name(fork_name)
     if _repo:
         type_ = 'fork' if _repo.fork else 'repo'
         raise JSONRPCError("%s `%s` already exist" % (type_, fork_name))
 
     if isinstance(owner, Optional):
         owner = apiuser.user_id
 
     owner = get_user_or_error(owner)
 
     try:
         # create structure of groups and return the last group
         repo_group = map_groups(fork_name)
         form_data = {
             'repo_name': fork_name_cleaned,
             'repo_name_full': fork_name,
             'repo_group': repo_group.group_id if repo_group else None,
             'repo_type': repo.repo_type,
             'description': Optional.extract(description),
             'private': Optional.extract(private),
             'copy_permissions': Optional.extract(copy_permissions),
             'landing_rev': Optional.extract(landing_rev),
             'fork_parent_id': repo.repo_id,
         }
 
         task = RepoModel().create_fork(form_data, cur_user=owner)
         # no commit, it's done in RepoModel, or async via celery
         from celery.result import BaseAsyncResult
         task_id = None
         if isinstance(task, BaseAsyncResult):
             task_id = task.task_id
         return {
             'msg': 'Created fork of `%s` as `%s`' % (
                 repo.repo_name, fork_name),
             'success': True,  # cannot return the repo data here since fork
             # can be done async
             'task': task_id
         }
     except Exception:
         log.exception("Exception occurred while trying to fork a repo")
         raise JSONRPCError(
             'failed to fork repository `%s` as `%s`' % (
                 repo_name, fork_name))
 
 
 @jsonrpc_method()
 def delete_repo(request, apiuser, repoid, forks=Optional('')):
     """
     Deletes a repository.
 
     * When the `forks` parameter is set it's possible to detach or delete
       forks of deleted repository.
 
     This command can only be run using an |authtoken| with admin
     permissions on the |repo|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: Set the repository name or repository ID.
     :type repoid: str or int
     :param forks: Set to `detach` or `delete` forks from the |repo|.
     :type forks: Optional(str)
 
     Example error output:
 
     .. code-block:: bash
 
         id : <id_given_in_input>
         result: {
                   "msg": "Deleted repository `<reponame>`",
                   "success": true
                 }
         error:  null
     """
 
     repo = get_repo_or_error(repoid)
     if not has_superadmin_permission(apiuser):
         _perms = ('repository.admin',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     try:
         handle_forks = Optional.extract(forks)
         _forks_msg = ''
         _forks = [f for f in repo.forks]
         if handle_forks == 'detach':
             _forks_msg = ' ' + 'Detached %s forks' % len(_forks)
         elif handle_forks == 'delete':
             _forks_msg = ' ' + 'Deleted %s forks' % len(_forks)
         elif _forks:
             raise JSONRPCError(
                 'Cannot delete `%s` it still contains attached forks' %
                 (repo.repo_name,)
             )
 
         RepoModel().delete(repo, forks=forks)
         Session().commit()
         return {
             'msg': 'Deleted repository `%s`%s' % (
                 repo.repo_name, _forks_msg),
             'success': True
         }
     except Exception:
         log.exception("Exception occurred while trying to delete repo")
         raise JSONRPCError(
             'failed to delete repository `%s`' % (repo.repo_name,)
         )
 
 
 #TODO: marcink, change name ?
 @jsonrpc_method()
 def invalidate_cache(request, apiuser, repoid, delete_keys=Optional(False)):
     """
     Invalidates the cache for the specified repository.
 
     This command can only be run using an |authtoken| with admin rights to
     the specified repository.
 
     This command takes the following options:
 
     :param apiuser: This is filled automatically from |authtoken|.
     :type apiuser: AuthUser
     :param repoid: Sets the repository name or repository ID.
     :type repoid: str or int
     :param delete_keys: This deletes the invalidated keys instead of
         just flagging them.
     :type delete_keys: Optional(``True`` | ``False``)
 
     Example output:
 
     .. code-block:: bash
 
       id : <id_given_in_input>
       result : {
         'msg': Cache for repository `<repository name>` was invalidated,
         'repository': <repository name>
       }
       error :  null
 
     Example error output:
 
     .. code-block:: bash
 
       id : <id_given_in_input>
       result : null
       error : {
          'Error occurred during cache invalidation action'
       }
 
     """
 
     repo = get_repo_or_error(repoid)
     if not has_superadmin_permission(apiuser):
         _perms = ('repository.admin', 'repository.write',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     delete = Optional.extract(delete_keys)
     try:
         ScmModel().mark_for_invalidation(repo.repo_name, delete=delete)
         return {
             'msg': 'Cache for repository `%s` was invalidated' % (repoid,),
             'repository': repo.repo_name
         }
     except Exception:
         log.exception(
             "Exception occurred while trying to invalidate repo cache")
         raise JSONRPCError(
             'Error occurred during cache invalidation action'
         )
 
 
 #TODO: marcink, change name ?
 @jsonrpc_method()
 def lock(request, apiuser, repoid, locked=Optional(None),
          userid=Optional(OAttr('apiuser'))):
     """
     Sets the lock state of the specified |repo| by the given user.
     From more information, see :ref:`repo-locking`.
 
     * If the ``userid`` option is not set, the repository is locked to the
       user who called the method.
     * If the ``locked`` parameter is not set, the current lock state of the
       repository is displayed.
 
     This command can only be run using an |authtoken| with admin rights to
     the specified repository.
 
     This command takes the following options:
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: Sets the repository name or repository ID.
     :type repoid: str or int
     :param locked: Sets the lock state.
     :type locked: Optional(``True`` | ``False``)
     :param userid: Set the repository lock to this user.
     :type userid: Optional(str or int)
 
     Example error output:
 
     .. code-block:: bash
 
       id : <id_given_in_input>
       result : {
         'repo': '<reponame>',
         'locked': <bool: lock state>,
         'locked_since': <int: lock timestamp>,
         'locked_by': <username of person who made the lock>,
         'lock_reason': <str: reason for locking>,
         'lock_state_changed': <bool: True if lock state has been changed in this request>,
         'msg': 'Repo `<reponame>` locked by `<username>` on <timestamp>.'
         or
         'msg': 'Repo `<repository name>` not locked.'
         or
         'msg': 'User `<user name>` set lock state for repo `<repository name>` to `<new lock state>`'
       }
       error :  null
 
     Example error output:
 
     .. code-block:: bash
 
       id : <id_given_in_input>
       result : null
       error :  {
         'Error occurred locking repository `<reponame>`
       }
     """
 
     repo = get_repo_or_error(repoid)
     if not has_superadmin_permission(apiuser):
         # check if we have at least write permission for this repo !
         _perms = ('repository.admin', 'repository.write',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
         # make sure normal user does not pass someone else userid,
         # he is not allowed to do that
         if not isinstance(userid, Optional) and userid != apiuser.user_id:
             raise JSONRPCError('userid is not the same as your user')
 
     if isinstance(userid, Optional):
         userid = apiuser.user_id
 
     user = get_user_or_error(userid)
 
     if isinstance(locked, Optional):
         lockobj = repo.locked
 
         if lockobj[0] is None:
             _d = {
                 'repo': repo.repo_name,
                 'locked': False,
                 'locked_since': None,
                 'locked_by': None,
                 'lock_reason': None,
                 'lock_state_changed': False,
                 'msg': 'Repo `%s` not locked.' % repo.repo_name
             }
             return _d
         else:
             _user_id, _time, _reason = lockobj
             lock_user = get_user_or_error(userid)
             _d = {
                 'repo': repo.repo_name,
                 'locked': True,
                 'locked_since': _time,
                 'locked_by': lock_user.username,
                 'lock_reason': _reason,
                 'lock_state_changed': False,
                 'msg': ('Repo `%s` locked by `%s` on `%s`.'
                         % (repo.repo_name, lock_user.username,
                            json.dumps(time_to_datetime(_time))))
             }
             return _d
 
     # force locked state through a flag
     else:
         locked = str2bool(locked)
         lock_reason = Repository.LOCK_API
         try:
             if locked:
                 lock_time = time.time()
                 Repository.lock(repo, user.user_id, lock_time, lock_reason)
             else:
                 lock_time = None
                 Repository.unlock(repo)
             _d = {
                 'repo': repo.repo_name,
                 'locked': locked,
                 'locked_since': lock_time,
                 'locked_by': user.username,
                 'lock_reason': lock_reason,
                 'lock_state_changed': True,
                 'msg': ('User `%s` set lock state for repo `%s` to `%s`'
                         % (user.username, repo.repo_name, locked))
             }
             return _d
         except Exception:
             log.exception(
                 "Exception occurred while trying to lock repository")
             raise JSONRPCError(
                 'Error occurred locking repository `%s`' % repo.repo_name
             )
 
 
 @jsonrpc_method()
 def comment_commit(
         request, apiuser, repoid, commit_id, message,
         userid=Optional(OAttr('apiuser')), status=Optional(None)):
     """
     Set a commit comment, and optionally change the status of the commit.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: Set the repository name or repository ID.
     :type repoid: str or int
     :param commit_id: Specify the commit_id for which to set a comment.
     :type commit_id: str
     :param message: The comment text.
     :type message: str
     :param userid: Set the user name of the comment creator.
     :type userid: Optional(str or int)
     :param status: status, one of 'not_reviewed', 'approved', 'rejected',
        'under_review'
     :type status: str
 
     Example error output:
 
     .. code-block:: json
 
         {
             "id" : <id_given_in_input>,
             "result" : {
                 "msg": "Commented on commit `<commit_id>` for repository `<repoid>`",
                 "status_change": null or <status>,
                 "success": true
             },
             "error" :  null
         }
 
     """
     repo = get_repo_or_error(repoid)
     if not has_superadmin_permission(apiuser):
         _perms = ('repository.read', 'repository.write', 'repository.admin')
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     if isinstance(userid, Optional):
         userid = apiuser.user_id
 
     user = get_user_or_error(userid)
     status = Optional.extract(status)
 
     allowed_statuses = [x[0] for x in ChangesetStatus.STATUSES]
     if status and status not in allowed_statuses:
         raise JSONRPCError('Bad status, must be on '
                            'of %s got %s' % (allowed_statuses, status,))
 
     try:
         rc_config = SettingsModel().get_all_settings()
         renderer = rc_config.get('rhodecode_markup_renderer', 'rst')
 
         comm = ChangesetCommentsModel().create(
             message, repo, user, revision=commit_id, status_change=status,
             renderer=renderer)
         if status:
             # also do a status change
             try:
                 ChangesetStatusModel().set_status(
                     repo, status, user, comm, revision=commit_id,
                     dont_allow_on_closed_pull_request=True
                 )
             except StatusChangeOnClosedPullRequestError:
                 log.exception(
                     "Exception occurred while trying to change repo commit status")
                 msg = ('Changing status on a changeset associated with '
                        'a closed pull request is not allowed')
                 raise JSONRPCError(msg)
 
         Session().commit()
         return {
             'msg': (
                 'Commented on commit `%s` for repository `%s`' % (
                     comm.revision, repo.repo_name)),
             'status_change': status,
             'success': True,
         }
     except JSONRPCError:
         # catch any inside errors, and re-raise them to prevent from
         # below global catch to silence them
         raise
     except Exception:
         log.exception("Exception occurred while trying to comment on commit")
         raise JSONRPCError(
             'failed to set comment on repository `%s`' % (repo.repo_name,)
         )
 
 
 @jsonrpc_method()
 def grant_user_permission(request, apiuser, repoid, userid, perm):
     """
     Grant permissions for the specified user on the given repository,
     or update existing permissions if found.
 
     This command can only be run using an |authtoken| with admin
     permissions on the |repo|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: Set the repository name or repository ID.
     :type repoid: str or int
     :param userid: Set the user name.
     :type userid: str
     :param perm: Set the user permissions, using the following format
         ``(repository.(none|read|write|admin))``
     :type perm: str
 
     Example output:
 
     .. code-block:: bash
 
         id : <id_given_in_input>
         result: {
                   "msg" : "Granted perm: `<perm>` for user: `<username>` in repo: `<reponame>`",
                   "success": true
                 }
         error:  null
     """
 
     repo = get_repo_or_error(repoid)
     user = get_user_or_error(userid)
     perm = get_perm_or_error(perm)
     if not has_superadmin_permission(apiuser):
         _perms = ('repository.admin',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     try:
 
         RepoModel().grant_user_permission(repo=repo, user=user, perm=perm)
 
         Session().commit()
         return {
             'msg': 'Granted perm: `%s` for user: `%s` in repo: `%s`' % (
                 perm.permission_name, user.username, repo.repo_name
             ),
             'success': True
         }
     except Exception:
         log.exception(
             "Exception occurred while trying edit permissions for repo")
         raise JSONRPCError(
             'failed to edit permission for user: `%s` in repo: `%s`' % (
                 userid, repoid
             )
         )
 
 
 @jsonrpc_method()
 def revoke_user_permission(request, apiuser, repoid, userid):
     """
     Revoke permission for a user on the specified repository.
 
     This command can only be run using an |authtoken| with admin
     permissions on the |repo|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: Set the repository name or repository ID.
     :type repoid: str or int
     :param userid: Set the user name of revoked user.
     :type userid: str or int
 
     Example error output:
 
     .. code-block:: bash
 
         id : <id_given_in_input>
         result: {
                   "msg" : "Revoked perm for user: `<username>` in repo: `<reponame>`",
                   "success": true
                 }
         error:  null
     """
 
     repo = get_repo_or_error(repoid)
     user = get_user_or_error(userid)
     if not has_superadmin_permission(apiuser):
         _perms = ('repository.admin',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     try:
         RepoModel().revoke_user_permission(repo=repo, user=user)
         Session().commit()
         return {
             'msg': 'Revoked perm for user: `%s` in repo: `%s`' % (
                 user.username, repo.repo_name
             ),
             'success': True
         }
     except Exception:
         log.exception(
             "Exception occurred while trying revoke permissions to repo")
         raise JSONRPCError(
             'failed to edit permission for user: `%s` in repo: `%s`' % (
                 userid, repoid
             )
         )
 
 
 @jsonrpc_method()
 def grant_user_group_permission(request, apiuser, repoid, usergroupid, perm):
     """
     Grant permission for a user group on the specified repository,
     or update existing permissions.
 
     This command can only be run using an |authtoken| with admin
     permissions on the |repo|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: Set the repository name or repository ID.
     :type repoid: str or int
     :param usergroupid: Specify the ID of the user group.
     :type usergroupid: str or int
     :param perm: Set the user group permissions using the following
         format: (repository.(none|read|write|admin))
     :type perm: str
 
     Example output:
 
     .. code-block:: bash
 
       id : <id_given_in_input>
       result : {
         "msg" : "Granted perm: `<perm>` for group: `<usersgroupname>` in repo: `<reponame>`",
         "success": true
 
       }
       error :  null
 
     Example error output:
 
     .. code-block:: bash
 
       id : <id_given_in_input>
       result : null
       error :  {
         "failed to edit permission for user group: `<usergroup>` in repo `<repo>`'
       }
 
     """
 
     repo = get_repo_or_error(repoid)
     perm = get_perm_or_error(perm)
     if not has_superadmin_permission(apiuser):
         _perms = ('repository.admin',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     user_group = get_user_group_or_error(usergroupid)
     if not has_superadmin_permission(apiuser):
         # check if we have at least read permission for this user group !
         _perms = ('usergroup.read', 'usergroup.write', 'usergroup.admin',)
         if not HasUserGroupPermissionAnyApi(*_perms)(
                 user=apiuser, user_group_name=user_group.users_group_name):
             raise JSONRPCError(
                 'user group `%s` does not exist' % (usergroupid,))
 
     try:
         RepoModel().grant_user_group_permission(
             repo=repo, group_name=user_group, perm=perm)
 
         Session().commit()
         return {
             'msg': 'Granted perm: `%s` for user group: `%s` in '
                    'repo: `%s`' % (
                        perm.permission_name, user_group.users_group_name,
                        repo.repo_name
                    ),
             'success': True
         }
     except Exception:
         log.exception(
             "Exception occurred while trying change permission on repo")
         raise JSONRPCError(
             'failed to edit permission for user group: `%s` in '
             'repo: `%s`' % (
                 usergroupid, repo.repo_name
             )
         )
 
 
 @jsonrpc_method()
 def revoke_user_group_permission(request, apiuser, repoid, usergroupid):
     """
     Revoke the permissions of a user group on a given repository.
 
     This command can only be run using an |authtoken| with admin
     permissions on the |repo|.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: Set the repository name or repository ID.
     :type repoid: str or int
     :param usergroupid: Specify the user group ID.
     :type usergroupid: str or int
 
     Example output:
 
     .. code-block:: bash
 
         id : <id_given_in_input>
         result: {
                   "msg" : "Revoked perm for group: `<usersgroupname>` in repo: `<reponame>`",
                   "success": true
                 }
         error:  null
     """
 
     repo = get_repo_or_error(repoid)
     if not has_superadmin_permission(apiuser):
         _perms = ('repository.admin',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     user_group = get_user_group_or_error(usergroupid)
     if not has_superadmin_permission(apiuser):
         # check if we have at least read permission for this user group !
         _perms = ('usergroup.read', 'usergroup.write', 'usergroup.admin',)
         if not HasUserGroupPermissionAnyApi(*_perms)(
                 user=apiuser, user_group_name=user_group.users_group_name):
             raise JSONRPCError(
                 'user group `%s` does not exist' % (usergroupid,))
 
     try:
         RepoModel().revoke_user_group_permission(
             repo=repo, group_name=user_group)
 
         Session().commit()
         return {
             'msg': 'Revoked perm for user group: `%s` in repo: `%s`' % (
                 user_group.users_group_name, repo.repo_name
             ),
             'success': True
         }
     except Exception:
         log.exception("Exception occurred while trying revoke "
                       "user group permission on repo")
         raise JSONRPCError(
             'failed to edit permission for user group: `%s` in '
             'repo: `%s`' % (
                 user_group.users_group_name, repo.repo_name
             )
         )
 
 
 @jsonrpc_method()
 def pull(request, apiuser, repoid):
     """
     Triggers a pull on the given repository from a remote location. You
     can use this to keep remote repositories up-to-date.
 
     This command can only be run using an |authtoken| with admin
     rights to the specified repository. For more information,
     see :ref:`config-token-ref`.
 
     This command takes the following options:
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: The repository name or repository ID.
     :type repoid: str or int
 
     Example output:
 
     .. code-block:: bash
 
       id : <id_given_in_input>
       result : {
         "msg": "Pulled from `<repository name>`"
         "repository": "<repository name>"
       }
       error :  null
 
     Example error output:
 
     .. code-block:: bash
 
       id : <id_given_in_input>
       result : null
       error :  {
         "Unable to pull changes from `<reponame>`"
       }
 
     """
 
     repo = get_repo_or_error(repoid)
     if not has_superadmin_permission(apiuser):
         _perms = ('repository.admin',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     try:
         ScmModel().pull_changes(repo.repo_name, apiuser.username)
         return {
             'msg': 'Pulled from `%s`' % repo.repo_name,
             'repository': repo.repo_name
         }
     except Exception:
         log.exception("Exception occurred while trying to "
                       "pull changes from remote location")
         raise JSONRPCError(
             'Unable to pull changes from `%s`' % repo.repo_name
         )
 
 
 @jsonrpc_method()
 def strip(request, apiuser, repoid, revision, branch):
     """
     Strips the given revision from the specified repository.
 
     * This will remove the revision and all of its decendants.
 
     This command can only be run using an |authtoken| with admin rights to
     the specified repository.
 
     This command takes the following options:
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: The repository name or repository ID.
     :type repoid: str or int
     :param revision: The revision you wish to strip.
     :type revision: str
     :param branch: The branch from which to strip the revision.
     :type branch: str
 
     Example output:
 
     .. code-block:: bash
 
       id : <id_given_in_input>
       result : {
         "msg": "'Stripped commit <commit_hash> from repo `<repository name>`'"
         "repository": "<repository name>"
       }
       error :  null
 
     Example error output:
 
     .. code-block:: bash
 
       id : <id_given_in_input>
       result : null
       error :  {
         "Unable to strip commit <commit_hash> from repo `<repository name>`"
       }
 
     """
 
     repo = get_repo_or_error(repoid)
     if not has_superadmin_permission(apiuser):
         _perms = ('repository.admin',)
         has_repo_permissions(apiuser, repoid, repo, _perms)
 
     try:
         ScmModel().strip(repo, revision, branch)
         return {
             'msg': 'Stripped commit %s from repo `%s`' % (
                 revision, repo.repo_name),
             'repository': repo.repo_name
         }
     except Exception:
         log.exception("Exception while trying to strip")
         raise JSONRPCError(
             'Unable to strip commit %s from repo `%s`' % (
                 revision, repo.repo_name)
         )
 
 
 @jsonrpc_method()
 def get_repo_settings(request, apiuser, repoid, key=Optional(None)):
     """
     Returns all settings for a repository. If key is given it only returns the
     setting identified by the key or null.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: The repository name or repository id.
     :type repoid: str or int
     :param key: Key of the setting to return.
     :type: key: Optional(str)
 
     Example output:
 
     .. code-block:: bash
 
         {
             "error": null,
             "id": 237,
             "result": {
                 "extensions_largefiles": true,
                 "hooks_changegroup_push_logger": true,
                 "hooks_changegroup_repo_size": false,
                 "hooks_outgoing_pull_logger": true,
                 "phases_publish": "True",
                 "rhodecode_hg_use_rebase_for_merging": true,
                 "rhodecode_pr_merge_enabled": true,
                 "rhodecode_use_outdated_comments": true
             }
         }
     """
 
     # Restrict access to this api method to admins only.
     if not has_superadmin_permission(apiuser):
         raise JSONRPCForbidden()
 
     try:
         repo = get_repo_or_error(repoid)
         settings_model = VcsSettingsModel(repo=repo)
         settings = settings_model.get_global_settings()
         settings.update(settings_model.get_repo_settings())
 
         # If only a single setting is requested fetch it from all settings.
         key = Optional.extract(key)
         if key is not None:
             settings = settings.get(key, None)
     except Exception:
         msg = 'Failed to fetch settings for repository `{}`'.format(repoid)
         log.exception(msg)
         raise JSONRPCError(msg)
 
     return settings
 
 
 @jsonrpc_method()
 def set_repo_settings(request, apiuser, repoid, settings):
     """
     Update repository settings. Returns true on success.
 
     :param apiuser: This is filled automatically from the |authtoken|.
     :type apiuser: AuthUser
     :param repoid: The repository name or repository id.
     :type repoid: str or int
     :param settings: The new settings for the repository.
     :type: settings: dict
 
     Example output:
 
     .. code-block:: bash
 
         {
             "error": null,
             "id": 237,
             "result": true
         }
     """
     # Restrict access to this api method to admins only.
     if not has_superadmin_permission(apiuser):
         raise JSONRPCForbidden()
 
     if type(settings) is not dict:
         raise JSONRPCError('Settings have to be a JSON Object.')
 
     try:
         settings_model = VcsSettingsModel(repo=repoid)
 
         # Merge global, repo and incoming settings.
         new_settings = settings_model.get_global_settings()
         new_settings.update(settings_model.get_repo_settings())
         new_settings.update(settings)
 
         # Update the settings.
         inherit_global_settings = new_settings.get(
             'inherit_global_settings', False)
         settings_model.create_or_update_repo_settings(
             new_settings, inherit_global_settings=inherit_global_settings)
         Session().commit()
     except Exception:
         msg = 'Failed to update settings for repository `{}`'.format(repoid)
         log.exception(msg)
         raise JSONRPCError(msg)
 
     # Indicate success.
     return True

rhodecode/model/scm.py

@@ -1,1099 +1,1102 b''
 # -*- coding: utf-8 -*-
 
 # Copyright (C) 2010-2016  RhodeCode GmbH
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License, version 3
 # (only), as published by the Free Software Foundation.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 # This program is dual-licensed. If you wish to learn more about the
 # RhodeCode Enterprise Edition, including its added features, Support services,
 # and proprietary license terms, please see https://rhodecode.com/licenses/
 
 """
 Scm model for RhodeCode
 """
 
 import os.path
 import re
 import sys
 import time
 import traceback
 import logging
 import cStringIO
 import pkg_resources
 
 import pylons
 from pylons.i18n.translation import _
 from sqlalchemy import func
 from zope.cachedescriptors.property import Lazy as LazyProperty
 
 import rhodecode
 from rhodecode.lib.vcs import get_backend
 from rhodecode.lib.vcs.exceptions import RepositoryError, NodeNotChangedError
 from rhodecode.lib.vcs.nodes import FileNode
 from rhodecode.lib.vcs.backends.base import EmptyCommit
 from rhodecode.lib import helpers as h
 
 from rhodecode.lib.auth import (
     HasRepoPermissionAny, HasRepoGroupPermissionAny,
     HasUserGroupPermissionAny)
 from rhodecode.lib.exceptions import NonRelativePathError, IMCCommitError
 from rhodecode.lib import hooks_utils, caches
 from rhodecode.lib.utils import (
     get_filesystem_repos, action_logger, make_db_config)
 from rhodecode.lib.utils2 import (
     safe_str, safe_unicode, get_server_url, md5)
 from rhodecode.model import BaseModel
 from rhodecode.model.db import (
     Repository, CacheKey, UserFollowing, UserLog, User, RepoGroup,
     PullRequest, DbMigrateVersion)
 from rhodecode.model.settings import VcsSettingsModel
 
 log = logging.getLogger(__name__)
 
 
 class UserTemp(object):
     def __init__(self, user_id):
         self.user_id = user_id
 
     def __repr__(self):
         return "<%s('id:%s')>" % (self.__class__.__name__, self.user_id)
 
 
 class RepoTemp(object):
     def __init__(self, repo_id):
         self.repo_id = repo_id
 
     def __repr__(self):
         return "<%s('id:%s')>" % (self.__class__.__name__, self.repo_id)
 
 
 class SimpleCachedRepoList(object):
     """
     Lighter version of of iteration of repos without the scm initialisation,
     and with cache usage
     """
     def __init__(self, db_repo_list, repos_path, order_by=None, perm_set=None):
         self.db_repo_list = db_repo_list
         self.repos_path = repos_path
         self.order_by = order_by
         self.reversed = (order_by or '').startswith('-')
         if not perm_set:
             perm_set = ['repository.read', 'repository.write',
                         'repository.admin']
         self.perm_set = perm_set
 
     def __len__(self):
         return len(self.db_repo_list)
 
     def __repr__(self):
         return '<%s (%s)>' % (self.__class__.__name__, self.__len__())
 
     def __iter__(self):
         for dbr in self.db_repo_list:
             # check permission at this level
             has_perm = HasRepoPermissionAny(*self.perm_set)(
                 dbr.repo_name, 'SimpleCachedRepoList check')
             if not has_perm:
                 continue
 
             tmp_d = {
                 'name': dbr.repo_name,
                 'dbrepo': dbr.get_dict(),
                 'dbrepo_fork': dbr.fork.get_dict() if dbr.fork else {}
             }
             yield tmp_d
 
 
 class _PermCheckIterator(object):
 
     def __init__(
             self, obj_list, obj_attr, perm_set, perm_checker,
             extra_kwargs=None):
         """
         Creates iterator from given list of objects, additionally
         checking permission for them from perm_set var
 
         :param obj_list: list of db objects
         :param obj_attr: attribute of object to pass into perm_checker
         :param perm_set: list of permissions to check
         :param perm_checker: callable to check permissions against
         """
         self.obj_list = obj_list
         self.obj_attr = obj_attr
         self.perm_set = perm_set
         self.perm_checker = perm_checker
         self.extra_kwargs = extra_kwargs or {}
 
     def __len__(self):
         return len(self.obj_list)
 
     def __repr__(self):
         return '<%s (%s)>' % (self.__class__.__name__, self.__len__())
 
     def __iter__(self):
         checker = self.perm_checker(*self.perm_set)
         for db_obj in self.obj_list:
             # check permission at this level
             name = getattr(db_obj, self.obj_attr, None)
             if not checker(name, self.__class__.__name__, **self.extra_kwargs):
                 continue
 
             yield db_obj
 
 
 class RepoList(_PermCheckIterator):
 
     def __init__(self, db_repo_list, perm_set=None, extra_kwargs=None):
         if not perm_set:
             perm_set = [
                 'repository.read', 'repository.write', 'repository.admin']
 
         super(RepoList, self).__init__(
             obj_list=db_repo_list,
             obj_attr='repo_name', perm_set=perm_set,
             perm_checker=HasRepoPermissionAny,
             extra_kwargs=extra_kwargs)
 
 
 class RepoGroupList(_PermCheckIterator):
 
     def __init__(self, db_repo_group_list, perm_set=None, extra_kwargs=None):
         if not perm_set:
             perm_set = ['group.read', 'group.write', 'group.admin']
 
         super(RepoGroupList, self).__init__(
             obj_list=db_repo_group_list,
             obj_attr='group_name', perm_set=perm_set,
             perm_checker=HasRepoGroupPermissionAny,
             extra_kwargs=extra_kwargs)
 
 
 class UserGroupList(_PermCheckIterator):
 
     def __init__(self, db_user_group_list, perm_set=None, extra_kwargs=None):
         if not perm_set:
             perm_set = ['usergroup.read', 'usergroup.write', 'usergroup.admin']
 
         super(UserGroupList, self).__init__(
             obj_list=db_user_group_list,
             obj_attr='users_group_name', perm_set=perm_set,
             perm_checker=HasUserGroupPermissionAny,
             extra_kwargs=extra_kwargs)
 
 
 class ScmModel(BaseModel):
     """
     Generic Scm Model
     """
 
     @LazyProperty
     def repos_path(self):
         """
         Gets the repositories root path from database
         """
 
         settings_model = VcsSettingsModel(sa=self.sa)
         return settings_model.get_repos_location()
 
     def repo_scan(self, repos_path=None):
         """
         Listing of repositories in given path. This path should not be a
         repository itself. Return a dictionary of repository objects
 
         :param repos_path: path to directory containing repositories
         """
 
         if repos_path is None:
             repos_path = self.repos_path
 
         log.info('scanning for repositories in %s', repos_path)
 
         config = make_db_config()
         config.set('extensions', 'largefiles', '')
         repos = {}
 
         for name, path in get_filesystem_repos(repos_path, recursive=True):
             # name need to be decomposed and put back together using the /
             # since this is internal storage separator for rhodecode
             name = Repository.normalize_repo_name(name)
 
             try:
                 if name in repos:
                     raise RepositoryError('Duplicate repository name %s '
                                           'found in %s' % (name, path))
                 elif path[0] in rhodecode.BACKENDS:
                     klass = get_backend(path[0])
                     repos[name] = klass(path[1], config=config)
             except OSError:
                 continue
         log.debug('found %s paths with repositories', len(repos))
         return repos
 
     def get_repos(self, all_repos=None, sort_key=None):
         """
         Get all repositories from db and for each repo create it's
         backend instance and fill that backed with information from database
 
         :param all_repos: list of repository names as strings
             give specific repositories list, good for filtering
 
         :param sort_key: initial sorting of repositories
         """
         if all_repos is None:
             all_repos = self.sa.query(Repository)\
                 .filter(Repository.group_id == None)\
                 .order_by(func.lower(Repository.repo_name)).all()
         repo_iter = SimpleCachedRepoList(
             all_repos, repos_path=self.repos_path, order_by=sort_key)
         return repo_iter
 
     def get_repo_groups(self, all_groups=None):
         if all_groups is None:
             all_groups = RepoGroup.query()\
                 .filter(RepoGroup.group_parent_id == None).all()
         return [x for x in RepoGroupList(all_groups)]
 
     def mark_for_invalidation(self, repo_name, delete=False):
         """
         Mark caches of this repo invalid in the database. `delete` flag
         removes the cache entries
 
         :param repo_name: the repo_name for which caches should be marked
             invalid, or deleted
         :param delete: delete the entry keys instead of setting bool
             flag on them
         """
         CacheKey.set_invalidate(repo_name, delete=delete)
         repo = Repository.get_by_repo_name(repo_name)
 
         if repo:
             config = repo._config
             config.set('extensions', 'largefiles', '')
             repo.update_commit_cache(config=config, cs_cache=None)
             caches.clear_repo_caches(repo_name)
 
     def toggle_following_repo(self, follow_repo_id, user_id):
 
         f = self.sa.query(UserFollowing)\
             .filter(UserFollowing.follows_repo_id == follow_repo_id)\
             .filter(UserFollowing.user_id == user_id).scalar()
 
         if f is not None:
             try:
                 self.sa.delete(f)
                 action_logger(UserTemp(user_id),
                               'stopped_following_repo',
                               RepoTemp(follow_repo_id))
                 return
             except Exception:
                 log.error(traceback.format_exc())
                 raise
 
         try:
             f = UserFollowing()
             f.user_id = user_id
             f.follows_repo_id = follow_repo_id
             self.sa.add(f)
 
             action_logger(UserTemp(user_id),
                           'started_following_repo',
                           RepoTemp(follow_repo_id))
         except Exception:
             log.error(traceback.format_exc())
             raise
 
     def toggle_following_user(self, follow_user_id, user_id):
         f = self.sa.query(UserFollowing)\
             .filter(UserFollowing.follows_user_id == follow_user_id)\
             .filter(UserFollowing.user_id == user_id).scalar()
 
         if f is not None:
             try:
                 self.sa.delete(f)
                 return
             except Exception:
                 log.error(traceback.format_exc())
                 raise
 
         try:
             f = UserFollowing()
             f.user_id = user_id
             f.follows_user_id = follow_user_id
             self.sa.add(f)
         except Exception:
             log.error(traceback.format_exc())
             raise
 
     def is_following_repo(self, repo_name, user_id, cache=False):
         r = self.sa.query(Repository)\
             .filter(Repository.repo_name == repo_name).scalar()
 
         f = self.sa.query(UserFollowing)\
             .filter(UserFollowing.follows_repository == r)\
             .filter(UserFollowing.user_id == user_id).scalar()
 
         return f is not None
 
     def is_following_user(self, username, user_id, cache=False):
         u = User.get_by_username(username)
 
         f = self.sa.query(UserFollowing)\
             .filter(UserFollowing.follows_user == u)\
             .filter(UserFollowing.user_id == user_id).scalar()
 
         return f is not None
 
     def get_followers(self, repo):
         repo = self._get_repo(repo)
 
         return self.sa.query(UserFollowing)\
             .filter(UserFollowing.follows_repository == repo).count()
 
     def get_forks(self, repo):
         repo = self._get_repo(repo)
         return self.sa.query(Repository)\
             .filter(Repository.fork == repo).count()
 
     def get_pull_requests(self, repo):
         repo = self._get_repo(repo)
         return self.sa.query(PullRequest)\
             .filter(PullRequest.target_repo == repo)\
             .filter(PullRequest.status != PullRequest.STATUS_CLOSED).count()
 
     def mark_as_fork(self, repo, fork, user):
         repo = self._get_repo(repo)
         fork = self._get_repo(fork)
         if fork and repo.repo_id == fork.repo_id:
             raise Exception("Cannot set repository as fork of itself")
 
         if fork and repo.repo_type != fork.repo_type:
             raise RepositoryError(
                 "Cannot set repository as fork of repository with other type")
 
         repo.fork = fork
         self.sa.add(repo)
         return repo
 
     def pull_changes(self, repo, username):
         dbrepo = self._get_repo(repo)
         clone_uri = dbrepo.clone_uri
         if not clone_uri:
             raise Exception("This repository doesn't have a clone uri")
 
         repo = dbrepo.scm_instance(cache=False)
         # TODO: marcink fix this an re-enable since we need common logic
         # for hg/git remove hooks so we don't trigger them on fetching
         # commits from remote
         repo.config.clear_section('hooks')
 
         repo_name = dbrepo.repo_name
         try:
             # TODO: we need to make sure those operations call proper hooks !
             repo.pull(clone_uri)
 
             self.mark_for_invalidation(repo_name)
         except Exception:
             log.error(traceback.format_exc())
             raise
 
     def commit_change(self, repo, repo_name, commit, user, author, message,
                       content, f_path):
         """
         Commits changes
 
         :param repo: SCM instance
 
         """
         user = self._get_user(user)
 
         # decoding here will force that we have proper encoded values
         # in any other case this will throw exceptions and deny commit
         content = safe_str(content)
         path = safe_str(f_path)
         # message and author needs to be unicode
         # proper backend should then translate that into required type
         message = safe_unicode(message)
         author = safe_unicode(author)
         imc = repo.in_memory_commit
         imc.change(FileNode(path, content, mode=commit.get_file_mode(f_path)))
         try:
             # TODO: handle pre-push action !
             tip = imc.commit(
                 message=message, author=author, parents=[commit],
                 branch=commit.branch)
         except Exception as e:
             log.error(traceback.format_exc())
             raise IMCCommitError(str(e))
         finally:
             # always clear caches, if commit fails we want fresh object also
             self.mark_for_invalidation(repo_name)
 
         # We trigger the post-push action
         hooks_utils.trigger_post_push_hook(
             username=user.username, action='push_local', repo_name=repo_name,
             repo_alias=repo.alias, commit_ids=[tip.raw_id])
         return tip
 
     def _sanitize_path(self, f_path):
         if f_path.startswith('/') or f_path.startswith('./') or '../' in f_path:
             raise NonRelativePathError('%s is not an relative path' % f_path)
         if f_path:
             f_path = os.path.normpath(f_path)
         return f_path
 
     def get_dirnode_metadata(self, commit, dir_node):
         if not dir_node.is_dir():
             return []
 
         data = []
         for node in dir_node:
             if not node.is_file():
                 # we skip file-nodes
                 continue
 
             last_commit = node.last_commit
             last_commit_date = last_commit.date
             data.append({
                 'name': node.name,
                 'size': h.format_byte_size_binary(node.size),
                 'modified_at': h.format_date(last_commit_date),
                 'modified_ts': last_commit_date.isoformat(),
                 'revision': last_commit.revision,
                 'short_id': last_commit.short_id,
                 'message': h.escape(last_commit.message),
                 'author': h.escape(last_commit.author),
                 'user_profile': h.gravatar_with_user(last_commit.author),
             })
 
         return data
 
     def get_nodes(self, repo_name, commit_id, root_path='/', flat=True,
-                  extended_info=False, content=False):
+                  extended_info=False, content=False, max_file_bytes=None):
         """
         recursive walk in root dir and return a set of all path in that dir
         based on repository walk function
 
         :param repo_name: name of repository
         :param commit_id: commit id for which to list nodes
         :param root_path: root path to list
         :param flat: return as a list, if False returns a dict with description
+        :param max_file_bytes: will not return file contents over this limit
 
         """
         _files = list()
         _dirs = list()
         try:
             _repo = self._get_repo(repo_name)
             commit = _repo.scm_instance().get_commit(commit_id=commit_id)
             root_path = root_path.lstrip('/')
             for __, dirs, files in commit.walk(root_path):
                 for f in files:
                     _content = None
                     _data = f.unicode_path
+                    over_size_limit = (max_file_bytes is not None
+                                       and f.size > max_file_bytes)
 
                     if not flat:
                         _data = {
                             "name": f.unicode_path,
                             "type": "file",
                             }
                         if extended_info:
                             _data.update({
                                 "md5": f.md5,
                                 "binary": f.is_binary,
                                 "size": f.size,
                                 "extension": f.extension,
                                 "mimetype": f.mimetype,
                                 "lines": f.lines()[0]
                             })
 
                         if content:
                             full_content = None
-                            if not f.is_binary:
+                            if not f.is_binary and not over_size_limit:
                                 full_content = safe_str(f.content)
 
                             _data.update({
                                 "content": full_content,
                             })
                     _files.append(_data)
                 for d in dirs:
                     _data = d.unicode_path
                     if not flat:
                         _data = {
                             "name": d.unicode_path,
                             "type": "dir",
                             }
                     if extended_info:
                         _data.update({
                             "md5": None,
                             "binary": None,
                             "size": None,
                             "extension": None,
                         })
                     if content:
                         _data.update({
                             "content": None
                         })
                     _dirs.append(_data)
         except RepositoryError:
             log.debug("Exception in get_nodes", exc_info=True)
             raise
 
         return _dirs, _files
 
     def create_nodes(self, user, repo, message, nodes, parent_commit=None,
                      author=None, trigger_push_hook=True):
         """
         Commits given multiple nodes into repo
 
         :param user: RhodeCode User object or user_id, the commiter
         :param repo: RhodeCode Repository object
         :param message: commit message
         :param nodes: mapping {filename:{'content':content},...}
         :param parent_commit: parent commit, can be empty than it's
            initial commit
         :param author: author of commit, cna be different that commiter
            only for git
         :param trigger_push_hook: trigger push hooks
 
         :returns: new commited commit
         """
 
         user = self._get_user(user)
         scm_instance = repo.scm_instance(cache=False)
 
         processed_nodes = []
         for f_path in nodes:
             f_path = self._sanitize_path(f_path)
             content = nodes[f_path]['content']
             f_path = safe_str(f_path)
             # decoding here will force that we have proper encoded values
             # in any other case this will throw exceptions and deny commit
             if isinstance(content, (basestring,)):
                 content = safe_str(content)
             elif isinstance(content, (file, cStringIO.OutputType,)):
                 content = content.read()
             else:
                 raise Exception('Content is of unrecognized type %s' % (
                     type(content)
                 ))
             processed_nodes.append((f_path, content))
 
         message = safe_unicode(message)
         commiter = user.full_contact
         author = safe_unicode(author) if author else commiter
 
         imc = scm_instance.in_memory_commit
 
         if not parent_commit:
             parent_commit = EmptyCommit(alias=scm_instance.alias)
 
         if isinstance(parent_commit, EmptyCommit):
             # EmptyCommit means we we're editing empty repository
             parents = None
         else:
             parents = [parent_commit]
         # add multiple nodes
         for path, content in processed_nodes:
             imc.add(FileNode(path, content=content))
         # TODO: handle pre push scenario
         tip = imc.commit(message=message,
                          author=author,
                          parents=parents,
                          branch=parent_commit.branch)
 
         self.mark_for_invalidation(repo.repo_name)
         if trigger_push_hook:
             hooks_utils.trigger_post_push_hook(
                 username=user.username, action='push_local',
                 repo_name=repo.repo_name, repo_alias=scm_instance.alias,
                 commit_ids=[tip.raw_id])
         return tip
 
     def update_nodes(self, user, repo, message, nodes, parent_commit=None,
                      author=None, trigger_push_hook=True):
         user = self._get_user(user)
         scm_instance = repo.scm_instance(cache=False)
 
         message = safe_unicode(message)
         commiter = user.full_contact
         author = safe_unicode(author) if author else commiter
 
         imc = scm_instance.in_memory_commit
 
         if not parent_commit:
             parent_commit = EmptyCommit(alias=scm_instance.alias)
 
         if isinstance(parent_commit, EmptyCommit):
             # EmptyCommit means we we're editing empty repository
             parents = None
         else:
             parents = [parent_commit]
 
         # add multiple nodes
         for _filename, data in nodes.items():
             # new filename, can be renamed from the old one, also sanitaze
             # the path for any hack around relative paths like ../../ etc.
             filename = self._sanitize_path(data['filename'])
             old_filename = self._sanitize_path(_filename)
             content = data['content']
 
             filenode = FileNode(old_filename, content=content)
             op = data['op']
             if op == 'add':
                 imc.add(filenode)
             elif op == 'del':
                 imc.remove(filenode)
             elif op == 'mod':
                 if filename != old_filename:
                     # TODO: handle renames more efficient, needs vcs lib
                     # changes
                     imc.remove(filenode)
                     imc.add(FileNode(filename, content=content))
                 else:
                     imc.change(filenode)
 
         try:
             # TODO: handle pre push scenario
             # commit changes
             tip = imc.commit(message=message,
                              author=author,
                              parents=parents,
                              branch=parent_commit.branch)
         except NodeNotChangedError:
             raise
         except Exception as e:
             log.exception("Unexpected exception during call to imc.commit")
             raise IMCCommitError(str(e))
         finally:
             # always clear caches, if commit fails we want fresh object also
             self.mark_for_invalidation(repo.repo_name)
 
         if trigger_push_hook:
             hooks_utils.trigger_post_push_hook(
                 username=user.username, action='push_local',
                 repo_name=repo.repo_name, repo_alias=scm_instance.alias,
                 commit_ids=[tip.raw_id])
 
     def delete_nodes(self, user, repo, message, nodes, parent_commit=None,
                      author=None, trigger_push_hook=True):
         """
         Deletes given multiple nodes into `repo`
 
         :param user: RhodeCode User object or user_id, the committer
         :param repo: RhodeCode Repository object
         :param message: commit message
         :param nodes: mapping {filename:{'content':content},...}
         :param parent_commit: parent commit, can be empty than it's initial
            commit
         :param author: author of commit, cna be different that commiter only
            for git
         :param trigger_push_hook: trigger push hooks
 
         :returns: new commit after deletion
         """
 
         user = self._get_user(user)
         scm_instance = repo.scm_instance(cache=False)
 
         processed_nodes = []
         for f_path in nodes:
             f_path = self._sanitize_path(f_path)
             # content can be empty but for compatabilty it allows same dicts
             # structure as add_nodes
             content = nodes[f_path].get('content')
             processed_nodes.append((f_path, content))
 
         message = safe_unicode(message)
         commiter = user.full_contact
         author = safe_unicode(author) if author else commiter
 
         imc = scm_instance.in_memory_commit
 
         if not parent_commit:
             parent_commit = EmptyCommit(alias=scm_instance.alias)
 
         if isinstance(parent_commit, EmptyCommit):
             # EmptyCommit means we we're editing empty repository
             parents = None
         else:
             parents = [parent_commit]
         # add multiple nodes
         for path, content in processed_nodes:
             imc.remove(FileNode(path, content=content))
 
         # TODO: handle pre push scenario
         tip = imc.commit(message=message,
                          author=author,
                          parents=parents,
                          branch=parent_commit.branch)
 
         self.mark_for_invalidation(repo.repo_name)
         if trigger_push_hook:
             hooks_utils.trigger_post_push_hook(
                 username=user.username, action='push_local',
                 repo_name=repo.repo_name, repo_alias=scm_instance.alias,
                 commit_ids=[tip.raw_id])
         return tip
 
     def strip(self, repo, commit_id, branch):
         scm_instance = repo.scm_instance(cache=False)
         scm_instance.config.clear_section('hooks')
         scm_instance.strip(commit_id, branch)
         self.mark_for_invalidation(repo.repo_name)
 
     def get_unread_journal(self):
         return self.sa.query(UserLog).count()
 
     def get_repo_landing_revs(self, repo=None):
         """
         Generates select option with tags branches and bookmarks (for hg only)
         grouped by type
 
         :param repo:
         """
 
         hist_l = []
         choices = []
         repo = self._get_repo(repo)
         hist_l.append(['rev:tip', _('latest tip')])
         choices.append('rev:tip')
         if not repo:
             return choices, hist_l
 
         repo = repo.scm_instance()
 
         branches_group = (
             [(u'branch:%s' % safe_unicode(b), safe_unicode(b))
                 for b in repo.branches],
             _("Branches"))
         hist_l.append(branches_group)
         choices.extend([x[0] for x in branches_group[0]])
 
         if repo.alias == 'hg':
             bookmarks_group = (
                 [(u'book:%s' % safe_unicode(b), safe_unicode(b))
                     for b in repo.bookmarks],
                 _("Bookmarks"))
             hist_l.append(bookmarks_group)
             choices.extend([x[0] for x in bookmarks_group[0]])
 
         tags_group = (
             [(u'tag:%s' % safe_unicode(t), safe_unicode(t))
                 for t in repo.tags],
             _("Tags"))
         hist_l.append(tags_group)
         choices.extend([x[0] for x in tags_group[0]])
 
         return choices, hist_l
 
     def install_git_hook(self, repo, force_create=False):
         """
         Creates a rhodecode hook inside a git repository
 
         :param repo: Instance of VCS repo
         :param force_create: Create even if same name hook exists
         """
 
         loc = os.path.join(repo.path, 'hooks')
         if not repo.bare:
             loc = os.path.join(repo.path, '.git', 'hooks')
         if not os.path.isdir(loc):
             os.makedirs(loc, mode=0777)
 
         tmpl_post = pkg_resources.resource_string(
             'rhodecode', '/'.join(
                 ('config', 'hook_templates', 'git_post_receive.py.tmpl')))
         tmpl_pre = pkg_resources.resource_string(
             'rhodecode', '/'.join(
                 ('config', 'hook_templates', 'git_pre_receive.py.tmpl')))
 
         for h_type, tmpl in [('pre', tmpl_pre), ('post', tmpl_post)]:
             _hook_file = os.path.join(loc, '%s-receive' % h_type)
             log.debug('Installing git hook in repo %s', repo)
             _rhodecode_hook = _check_rhodecode_hook(_hook_file)
 
             if _rhodecode_hook or force_create:
                 log.debug('writing %s hook file !', h_type)
                 try:
                     with open(_hook_file, 'wb') as f:
                         tmpl = tmpl.replace('_TMPL_', rhodecode.__version__)
                         tmpl = tmpl.replace('_ENV_', sys.executable)
                         f.write(tmpl)
                     os.chmod(_hook_file, 0755)
                 except IOError:
                     log.exception('error writing hook file %s', _hook_file)
             else:
                 log.debug('skipping writing hook file')
 
     def install_svn_hooks(self, repo, force_create=False):
         """
         Creates rhodecode hooks inside a svn repository
 
         :param repo: Instance of VCS repo
         :param force_create: Create even if same name hook exists
         """
         hooks_path = os.path.join(repo.path, 'hooks')
         if not os.path.isdir(hooks_path):
             os.makedirs(hooks_path)
         post_commit_tmpl = pkg_resources.resource_string(
             'rhodecode', '/'.join(
                 ('config', 'hook_templates', 'svn_post_commit_hook.py.tmpl')))
         pre_commit_template = pkg_resources.resource_string(
             'rhodecode', '/'.join(
                 ('config', 'hook_templates', 'svn_pre_commit_hook.py.tmpl')))
         templates = {
             'post-commit': post_commit_tmpl,
             'pre-commit': pre_commit_template
         }
         for filename in templates:
             _hook_file = os.path.join(hooks_path, filename)
             _rhodecode_hook = _check_rhodecode_hook(_hook_file)
             if _rhodecode_hook or force_create:
                 log.debug('writing %s hook file !', filename)
                 template = templates[filename]
                 try:
                     with open(_hook_file, 'wb') as f:
                         template = template.replace(
                             '_TMPL_', rhodecode.__version__)
                         template = template.replace('_ENV_', sys.executable)
                         f.write(template)
                     os.chmod(_hook_file, 0755)
                 except IOError:
                     log.exception('error writing hook file %s', filename)
             else:
                 log.debug('skipping writing hook file')
 
     def install_hooks(self, repo, repo_type):
         if repo_type == 'git':
             self.install_git_hook(repo)
         elif repo_type == 'svn':
             self.install_svn_hooks(repo)
 
     def get_server_info(self, environ=None):
         import platform
         import rhodecode
         import pkg_resources
         from rhodecode.model.meta import Base as sql_base, Session
         from sqlalchemy.engine import url
         from rhodecode.lib.base import get_server_ip_addr, get_server_port
         from rhodecode.lib.vcs.backends.git import discover_git_version
         from rhodecode.model.gist import GIST_STORE_LOC
 
         try:
             # cygwin cannot have yet psutil support.
             import psutil
         except ImportError:
             psutil = None
 
         environ = environ or {}
         _NA = 'NOT AVAILABLE'
         _memory = _NA
         _uptime = _NA
         _boot_time = _NA
         _cpu = _NA
         _disk = dict(percent=0, used=0, total=0, error='')
         _load = {'1_min': _NA, '5_min': _NA, '15_min': _NA}
 
         model = VcsSettingsModel()
         storage_path = model.get_repos_location()
         gist_storage_path = os.path.join(storage_path, GIST_STORE_LOC)
         archive_storage_path = rhodecode.CONFIG.get('archive_cache_dir', '')
         search_index_storage_path = rhodecode.CONFIG.get('search.location', '')
 
         if psutil:
             # disk storage
             try:
                 _disk = dict(psutil.disk_usage(storage_path)._asdict())
             except Exception as e:
                 log.exception('Failed to fetch disk info')
                 _disk = {'percent': 0, 'used': 0, 'total': 0, 'error': str(e)}
 
             # memory
             _memory = dict(psutil.virtual_memory()._asdict())
             _memory['percent2'] = psutil._common.usage_percent(
                 (_memory['total'] - _memory['free']),
                 _memory['total'], 1)
 
             # load averages
             if hasattr(psutil.os, 'getloadavg'):
                 _load = dict(zip(
                     ['1_min', '5_min', '15_min'], psutil.os.getloadavg()))
             _uptime = time.time() - psutil.boot_time()
             _boot_time = psutil.boot_time()
             _cpu = psutil.cpu_percent(0.5)
 
         mods = dict([(p.project_name, p.version)
                      for p in pkg_resources.working_set])
 
         def get_storage_size(storage_path):
             sizes = []
             for file_ in os.listdir(storage_path):
                 storage_file = os.path.join(storage_path, file_)
                 if os.path.isfile(storage_file):
                     try:
                         sizes.append(os.path.getsize(storage_file))
                     except OSError:
                         log.exception('Failed to get size of storage file %s',
                                       storage_file)
                         pass
 
             return sum(sizes)
 
         # archive cache storage
         _disk_archive = {'percent': 0, 'used': 0, 'total': 0}
         try:
             archive_storage_path_exists = os.path.isdir(
                 archive_storage_path)
             if archive_storage_path and archive_storage_path_exists:
                 used = get_storage_size(archive_storage_path)
                 _disk_archive.update({
                     'used': used,
                     'total': used,
                 })
         except Exception as e:
             log.exception('failed to fetch archive cache storage')
             _disk_archive['error'] = str(e)
 
         # search index storage
         _disk_index = {'percent': 0, 'used': 0, 'total': 0}
         try:
             search_index_storage_path_exists = os.path.isdir(
                 search_index_storage_path)
             if search_index_storage_path_exists:
                 used = get_storage_size(search_index_storage_path)
                 _disk_index.update({
                     'percent': 100,
                     'used': used,
                     'total': used,
                 })
         except Exception as e:
             log.exception('failed to fetch search index storage')
             _disk_index['error'] = str(e)
 
         # gist storage
         _disk_gist = {'percent': 0, 'used': 0, 'total': 0, 'items': 0}
         try:
             items_count = 0
             used = 0
             for root, dirs, files in os.walk(safe_str(gist_storage_path)):
                 if root == gist_storage_path:
                     items_count = len(dirs)
 
                 for f in files:
                     try:
                         used += os.path.getsize(os.path.join(root, f))
                     except OSError:
                         pass
             _disk_gist.update({
                 'percent': 100,
                 'used': used,
                 'total': used,
                 'items': items_count
             })
         except Exception as e:
             log.exception('failed to fetch gist storage items')
             _disk_gist['error'] = str(e)
 
         # GIT info
         git_ver = discover_git_version()
 
         # SVN info
         # TODO: johbo: Add discover_svn_version to replace this code.
         try:
             import svn.core
             svn_ver = svn.core.SVN_VERSION
         except ImportError:
             svn_ver = None
 
         # DB stuff
         db_info = url.make_url(rhodecode.CONFIG['sqlalchemy.db1.url'])
         db_type = db_info.__to_string__()
         try:
             engine = sql_base.metadata.bind
             db_server_info = engine.dialect._get_server_version_info(
                 Session.connection(bind=engine))
             db_version = '%s %s' % (db_info.drivername,
                                     '.'.join(map(str, db_server_info)))
         except Exception:
             log.exception('failed to fetch db version')
             db_version = '%s %s' % (db_info.drivername, '?')
 
         db_migrate = DbMigrateVersion.query().filter(
             DbMigrateVersion.repository_id == 'rhodecode_db_migrations').one()
         db_migrate_version = db_migrate.version
 
         info = {
             'py_version': ' '.join(platform._sys_version()),
             'py_path': sys.executable,
             'py_modules': sorted(mods.items(), key=lambda k: k[0].lower()),
 
             'platform': safe_unicode(platform.platform()),
             'storage': storage_path,
             'archive_storage': archive_storage_path,
             'index_storage': search_index_storage_path,
             'gist_storage': gist_storage_path,
 
 
             'db_type': db_type,
             'db_version': db_version,
             'db_migrate_version': db_migrate_version,
 
             'rhodecode_version': rhodecode.__version__,
             'rhodecode_config_ini': rhodecode.CONFIG.get('__file__'),
             'server_ip': '%s:%s' % (
                 get_server_ip_addr(environ, log_errors=False),
                 get_server_port(environ)
             ),
             'server_id': rhodecode.CONFIG.get('instance_id'),
 
             'git_version': safe_unicode(git_ver),
             'hg_version': mods.get('mercurial'),
             'svn_version': svn_ver,
 
             'uptime': _uptime,
             'boot_time': _boot_time,
             'load': _load,
             'cpu': _cpu,
             'memory': _memory,
             'disk': _disk,
             'disk_archive': _disk_archive,
             'disk_gist': _disk_gist,
             'disk_index': _disk_index,
         }
         return info
 
 
 def _check_rhodecode_hook(hook_path):
     """
     Check if the hook was created by RhodeCode
     """
     if not os.path.exists(hook_path):
         return True
 
     log.debug('hook exists, checking if it is from rhodecode')
     hook_content = _read_hook(hook_path)
     matches = re.search(r'(?:RC_HOOK_VER)\s*=\s*(.*)', hook_content)
     if matches:
         try:
             version = matches.groups()[0]
             log.debug('got %s, it is rhodecode', version)
             return True
         except Exception:
             log.exception("Exception while reading the hook version.")
 
     return False
 
 
 def _read_hook(hook_path):
     with open(hook_path, 'rb') as f:
         content = f.read()
-    return content
  No newline at end of file
+    return content

rhodecode/tests/models/test_scm.py

@@ -1,318 +1,335 b''
 # -*- coding: utf-8 -*-
 
 # Copyright (C) 2010-2016  RhodeCode GmbH
 #
 # This program is free software: you can redistribute it and/or modify
 # it under the terms of the GNU Affero General Public License, version 3
 # (only), as published by the Free Software Foundation.
 #
 # This program is distributed in the hope that it will be useful,
 # but WITHOUT ANY WARRANTY; without even the implied warranty of
 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
 #
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #
 # This program is dual-licensed. If you wish to learn more about the
 # RhodeCode Enterprise Edition, including its added features, Support services,
 # and proprietary license terms, please see https://rhodecode.com/licenses/
 
 import os
 import stat
 import sys
 
 import pytest
 from mock import Mock, patch, DEFAULT
 
 import rhodecode
 from rhodecode.model import db, scm
 
 
 def test_scm_instance_config(backend):
     repo = backend.create_repo()
     with patch.multiple('rhodecode.model.db.Repository',
                         _get_instance=DEFAULT,
                         _get_instance_cached=DEFAULT) as mocks:
         repo.scm_instance()
         mocks['_get_instance'].assert_called_with(
             config=None, cache=False)
 
         config = {'some': 'value'}
         repo.scm_instance(config=config)
         mocks['_get_instance'].assert_called_with(
             config=config, cache=False)
 
         with patch.dict(rhodecode.CONFIG, {'vcs_full_cache': 'true'}):
             repo.scm_instance(config=config)
             mocks['_get_instance_cached'].assert_called()
 
 
 def test__get_instance_config(backend):
     repo = backend.create_repo()
     vcs_class = Mock()
     with patch.multiple('rhodecode.lib.vcs.backends',
                         get_scm=DEFAULT,
                         get_backend=DEFAULT) as mocks:
         mocks['get_scm'].return_value = backend.alias
         mocks['get_backend'].return_value = vcs_class
         with patch('rhodecode.model.db.Repository._config') as config_mock:
             repo._get_instance()
             vcs_class.assert_called_with(
                 repo_path=repo.repo_full_path, config=config_mock,
                 create=False, with_wire={'cache': True})
 
         new_config = {'override': 'old_config'}
         repo._get_instance(config=new_config)
         vcs_class.assert_called_with(
             repo_path=repo.repo_full_path, config=new_config, create=False,
             with_wire={'cache': True})
 
 
 def test_mark_for_invalidation_config(backend):
     repo = backend.create_repo()
     with patch('rhodecode.model.db.Repository.update_commit_cache') as _mock:
         scm.ScmModel().mark_for_invalidation(repo.repo_name)
         _, kwargs = _mock.call_args
         assert kwargs['config'].__dict__ == repo._config.__dict__
 
 
 def test_mark_for_invalidation_with_delete_updates_last_commit(backend):
     commits = [{'message': 'A'}, {'message': 'B'}]
     repo = backend.create_repo(commits=commits)
     scm.ScmModel().mark_for_invalidation(repo.repo_name, delete=True)
     assert repo.changeset_cache['revision'] == 1
 
 
 def test_mark_for_invalidation_with_delete_updates_last_commit_empty(backend):
     repo = backend.create_repo()
     scm.ScmModel().mark_for_invalidation(repo.repo_name, delete=True)
     assert repo.changeset_cache['revision'] == -1
 
 
 def test_strip_with_multiple_heads(backend_hg):
     commits = [
         {'message': 'A'},
         {'message': 'a'},
         {'message': 'b'},
         {'message': 'B', 'parents': ['A']},
         {'message': 'a1'},
     ]
     repo = backend_hg.create_repo(commits=commits)
     commit_ids = backend_hg.commit_ids
 
     model = scm.ScmModel()
     model.strip(repo, commit_ids['b'], branch=None)
 
     vcs_repo = repo.scm_instance()
     rest_commit_ids = [c.raw_id for c in vcs_repo.get_changesets()]
     assert len(rest_commit_ids) == 4
     assert commit_ids['b'] not in rest_commit_ids
 
 
 def test_strip_with_single_heads(backend_hg):
     commits = [
         {'message': 'A'},
         {'message': 'a'},
         {'message': 'b'},
     ]
     repo = backend_hg.create_repo(commits=commits)
     commit_ids = backend_hg.commit_ids
 
     model = scm.ScmModel()
     model.strip(repo, commit_ids['b'], branch=None)
 
     vcs_repo = repo.scm_instance()
     rest_commit_ids = [c.raw_id for c in vcs_repo.get_changesets()]
     assert len(rest_commit_ids) == 2
     assert commit_ids['b'] not in rest_commit_ids
 
 
 def test_get_nodes_returns_unicode_flat(backend_random):
     repo = backend_random.repo
     directories, files = scm.ScmModel().get_nodes(
         repo.repo_name, repo.get_commit(commit_idx=0).raw_id,
         flat=True)
     assert_contains_only_unicode(directories)
     assert_contains_only_unicode(files)
 
 
 def test_get_nodes_returns_unicode_non_flat(backend_random):
     repo = backend_random.repo
     directories, files = scm.ScmModel().get_nodes(
         repo.repo_name, repo.get_commit(commit_idx=0).raw_id,
         flat=False)
     # johbo: Checking only the names for now, since that is the critical
     # part.
     assert_contains_only_unicode([d['name'] for d in directories])
     assert_contains_only_unicode([f['name'] for f in files])
 
 
+def test_get_nodes_max_file_bytes(backend_random):
+    repo = backend_random.repo
+    max_file_bytes = 10
+    directories, files = scm.ScmModel().get_nodes(
+        repo.repo_name, repo.get_commit(commit_idx=0).raw_id, content=True,
+        extended_info=True, flat=False)
+    assert any(file['content'] and len(file['content']) > max_file_bytes
+               for file in files)
+
+    directories, files = scm.ScmModel().get_nodes(
+        repo.repo_name, repo.get_commit(commit_idx=0).raw_id, content=True,
+        extended_info=True, flat=False, max_file_bytes=max_file_bytes)
+    assert all(
+        file['content'] is None if file['size'] > max_file_bytes else True
+        for file in files)
+
+
 def assert_contains_only_unicode(structure):
     assert structure
     for value in structure:
         assert isinstance(value, unicode)
 
 
 @pytest.mark.backends("hg", "git")
 def test_get_non_unicode_reference(backend):
     model = scm.ScmModel()
     non_unicode_list = ["Adını".decode("cp1254")]
 
     def scm_instance():
         return Mock(
             branches=non_unicode_list, bookmarks=non_unicode_list,
             tags=non_unicode_list, alias=backend.alias)
 
     repo = Mock(__class__=db.Repository, scm_instance=scm_instance)
     choices, __ = model.get_repo_landing_revs(repo=repo)
     if backend.alias == 'hg':
         valid_choices = [
             'rev:tip', u'branch:Ad\xc4\xb1n\xc4\xb1',
             u'book:Ad\xc4\xb1n\xc4\xb1', u'tag:Ad\xc4\xb1n\xc4\xb1']
     else:
         valid_choices = [
             'rev:tip', u'branch:Ad\xc4\xb1n\xc4\xb1',
             u'tag:Ad\xc4\xb1n\xc4\xb1']
 
     assert choices == valid_choices
 
 
 class TestInstallSvnHooks(object):
     HOOK_FILES = ('pre-commit', 'post-commit')
 
     def test_new_hooks_are_created(self, backend_svn):
         model = scm.ScmModel()
         repo = backend_svn.create_repo()
         vcs_repo = repo.scm_instance()
         model.install_svn_hooks(vcs_repo)
 
         hooks_path = os.path.join(vcs_repo.path, 'hooks')
         assert os.path.isdir(hooks_path)
         for file_name in self.HOOK_FILES:
             file_path = os.path.join(hooks_path, file_name)
             self._check_hook_file_mode(file_path)
             self._check_hook_file_content(file_path)
 
     def test_rc_hooks_are_replaced(self, backend_svn):
         model = scm.ScmModel()
         repo = backend_svn.create_repo()
         vcs_repo = repo.scm_instance()
         hooks_path = os.path.join(vcs_repo.path, 'hooks')
         file_paths = [os.path.join(hooks_path, f) for f in self.HOOK_FILES]
 
         for file_path in file_paths:
             self._create_fake_hook(
                 file_path, content="RC_HOOK_VER = 'abcde'\n")
 
         model.install_svn_hooks(vcs_repo)
 
         for file_path in file_paths:
             self._check_hook_file_content(file_path)
 
     def test_non_rc_hooks_are_not_replaced_without_force_create(
             self, backend_svn):
         model = scm.ScmModel()
         repo = backend_svn.create_repo()
         vcs_repo = repo.scm_instance()
         hooks_path = os.path.join(vcs_repo.path, 'hooks')
         file_paths = [os.path.join(hooks_path, f) for f in self.HOOK_FILES]
         non_rc_content = "exit 0\n"
 
         for file_path in file_paths:
             self._create_fake_hook(file_path, content=non_rc_content)
 
         model.install_svn_hooks(vcs_repo)
 
         for file_path in file_paths:
             with open(file_path, 'rt') as hook_file:
                 content = hook_file.read()
             assert content == non_rc_content
 
     def test_non_rc_hooks_are_replaced_with_force_create(self, backend_svn):
         model = scm.ScmModel()
         repo = backend_svn.create_repo()
         vcs_repo = repo.scm_instance()
         hooks_path = os.path.join(vcs_repo.path, 'hooks')
         file_paths = [os.path.join(hooks_path, f) for f in self.HOOK_FILES]
         non_rc_content = "exit 0\n"
 
         for file_path in file_paths:
             self._create_fake_hook(file_path, content=non_rc_content)
 
         model.install_svn_hooks(vcs_repo, force_create=True)
 
         for file_path in file_paths:
             self._check_hook_file_content(file_path)
 
     def _check_hook_file_mode(self, file_path):
         assert os.path.exists(file_path)
         stat_info = os.stat(file_path)
 
         file_mode = stat.S_IMODE(stat_info.st_mode)
         expected_mode = int('755', 8)
         assert expected_mode == file_mode
 
     def _check_hook_file_content(self, file_path):
         with open(file_path, 'rt') as hook_file:
             content = hook_file.read()
 
         expected_env = '#!{}'.format(sys.executable)
         expected_rc_version = "\nRC_HOOK_VER = '{}'\n".format(
             rhodecode.__version__)
         assert content.strip().startswith(expected_env)
         assert expected_rc_version in content
 
     def _create_fake_hook(self, file_path, content):
         with open(file_path, 'w') as hook_file:
             hook_file.write(content)
 
 
 class TestCheckRhodecodeHook(object):
 
     @patch('os.path.exists', Mock(return_value=False))
     def test_returns_true_when_no_hook_found(self):
         result = scm._check_rhodecode_hook('/tmp/fake_hook_file.py')
         assert result
 
     @pytest.mark.parametrize("file_content, expected_result", [
         ("RC_HOOK_VER = '3.3.3'\n", True),
         ("RC_HOOK = '3.3.3'\n", False),
     ])
     @patch('os.path.exists', Mock(return_value=True))
     def test_signatures(self, file_content, expected_result):
         hook_content_patcher = patch.object(
             scm, '_read_hook', return_value=file_content)
         with hook_content_patcher:
             result = scm._check_rhodecode_hook('/tmp/fake_hook_file.py')
 
         assert result is expected_result
 
 
 class TestInstallHooks(object):
     def test_hooks_are_installed_for_git_repo(self, backend_git):
         repo = backend_git.create_repo()
         model = scm.ScmModel()
         scm_repo = repo.scm_instance()
         with patch.object(model, 'install_git_hook') as hooks_mock:
             model.install_hooks(scm_repo, repo_type='git')
         hooks_mock.assert_called_once_with(scm_repo)
 
     def test_hooks_are_installed_for_svn_repo(self, backend_svn):
         repo = backend_svn.create_repo()
         scm_repo = repo.scm_instance()
         model = scm.ScmModel()
         with patch.object(scm.ScmModel, 'install_svn_hooks') as hooks_mock:
             model.install_hooks(scm_repo, repo_type='svn')
         hooks_mock.assert_called_once_with(scm_repo)
 
     @pytest.mark.parametrize('hook_method', [
         'install_svn_hooks',
         'install_git_hook'])
     def test_mercurial_doesnt_trigger_hooks(self, backend_hg, hook_method):
         repo = backend_hg.create_repo()
         scm_repo = repo.scm_instance()
         model = scm.ScmModel()
         with patch.object(scm.ScmModel, hook_method) as hooks_mock:
             model.install_hooks(scm_repo, repo_type='hg')
         assert hooks_mock.call_count == 0