Extra notes about keyring problems. Version bump to 4.8

HISTORY.txt

@@ -1,335 +1,338 b''
+
+1.2.1
+~~~~~~~~~~~~
+
+Tested against hg 4.8 (no changes needed).
 
 1.2.0
 ~~~~~~~~~~~~
 
 #62 Compatible with Mercurial 4.7 (some fixes were needed to avoid
 crashes while working in this version). Most important fixes were made
 in meu (so here we just depend on proper version), but there are also
 individual fixes here (related to smtp handling).
 
 #61 In case keyring library raises exceptions (like recent versions do
 when no backend is available), we don't crash mercurial anymore.
 mercurial_keyring emits the warning and falls back to normal password
 handling (propts).
 
 Refreshing smtp monkeypatch to be a bit more like modern mercurials
 (in particular, obligatory cert falidaton).
 
-
-
 1.1.9
 ~~~~~~~~~~~~
 
 4.6-compatibility, 4.5-testing.
 
 1.1.8
 ~~~~~~~~~~~~~
 
 Updated links after bitbucket changes.
 
 1.1.7
 ~~~~~~~~~~~~~~~~~~
 
 #52 hg keyring_check and hg keyring_clear did not work since
 Mercurial 3.9 (incompatibility caused by commit 2c019aac6b99,
 introducing passwdb).
 
 1.1.6
 ~~~~~~~~~~~~~~~~~~
 
 Fixed NameError showing up in some password saving scenarios, in
 particular in case of password save failures (thanks to Andrew
 Taumoefolau for reporting and fixing).
 
 1.1.5
 ~~~~~~~~~~~~~~~~~~
 
 Mercurial 3.9 compatibility.
 
 1.1.4
 ~~~~~~~~~~~~~~~~~~
 
 Gracefully handle failures to save passwords - they are reported
 as warnings, but don't break the operation being executed.
 
 Compatibility fixes for upcoming 3.9 release (which changes SSL API
 noticeably, what impact SMTP passwords handling in mercurial_keyring).
 
 1.1.3
 ~~~~~~~~~~~~~~~~~~
 
 Mercurial 3.8 compatibility for email over SSL/TLS (SMTPS/STARTTLS
 constructors changed). Should not spoil older versions.
 
 1.1.2
 ~~~~~~~~~~~~~~~~~~
 
 The keyring_check and keyring_clear commands can be run outside
 repository (if given some path as parameter).
 
 Fixed some messages.
 
 README updates (a few language fixes, added note about GUI tools).
 
 1.1.1
 ~~~~~~~~~~~~~~~~~~
 
 #49 Fixed the bug due to url-stored usernames did not work (introduced
 in 1.0.0 and not completely fixed in 1.0.1).
 
 #50 Bad doc url in error message
 
 
 1.1.0
 ~~~~~~~~~~~~~~~~~~
 
 Forward compatibility for Mercurial 3.8 (should not break old mercurials)
 
 1.0.1
 ~~~~~~~~~~~~~~~~~~
 
 URLs containing usernames (https://[e-mail]/somewhat) were
 not working unless username was also configured separately (username
 presence in url was not detected properly).
 
 Liberated prefix matching, path https://[e-mail]/somewhat can
 be matched both against prefix https://some.service and against
 https://[e-mail]. That mostly matches what mercurial itself
 does.
 
 1.0.0
 ~~~~~~~~~~~~~~~~~~
 
 Added
     hg keyring_check
 and
     hg keyring_clear PATH-OR-ALIAS
 commands
 
 Removed obsolete workarounds (compatibility for very old Mercurials -
 some for pre-1.0, some for 1.4, some for 1.8/1.9). 
 Mercurial 2.0 is now required.
 
 Improved information about path prefix. In particular it is shown
 whenever user is asked for password, for example:
      hg pull bitbucket
      http authorization required
      realm: BitBucket
      url: https://bitbucket.org/Mekk
      user: Mekk (fixed in hgrc or url)
      password: 
 
 Improved README.
 
 Improved debug information.
 
 0.8.0
 ~~~~~~~~~~~~~~~~~~
 
 Module is simplified a bit, but requires mercurial_extension_utils.
 Debug messages are prefixed with keyring: not [HgKeyring]
 
 0.7.1
 ~~~~~~~~~~~~~~~~~~
 
 #48 NullHandler import failure no longer breaks the extension.
 May help python 2.6 compatibility.
 
 0.7.0
 ~~~~~~~~~~~~~~~~~~~
 
 Delaying keyring module import until passwords are really needed. It
 can noticeably improve Mercurial (non pull/push) performance in some
 cases (no longer slow hg status because D-Bus is busy an keyring tries
 to activate KDE Wallet through it…).
 
 0.6.7
 ~~~~~~~~~~~~~~~~~
 
 #46 Fixed syntax of smtp.tls configuration setting (current Mercurials
 doesn't handle "true" anymore, TortoiseHG crashed with mercurial
 keyring enabled while currently recommended starttls/smtps/none values
 were in use).
 
 0.6.6
 ~~~~~~~~~~~~~~~~~ 
 
 #44 Handling some more mercurial versions in demandimport-detection
 logic.
 
 0.6.5
 ~~~~~~~~~~~~~~~~~
 
 #36 Shutting up warning about no logging handlers.
 
 0.6.4
 ~~~~~~~~~~~~~~~~~
 
 #44 Pre-2.9.1 Mercurials compatibility (probing for active
 demandimport differently).
 
 0.6.3
 ~~~~~~~~~~~~~~~~~
 
 #41 Fix for incorrect demandimport activity check logic, which could
 cause various problems with imports after mercurial_keyring is
 imported.
 
 0.6.2
 ~~~~~~~~~~~~~~~~~
 
 #33 Fix for UnicodeDecodeErrors happening on some backends (especially
 Vault) when passwords with non-ascii characters are in use and native
 locale is not utf-8. Passwords are no longer saved to keyring backends
 as-entered, they are now decoded from local encoding (whichever is
 detected by Mercurial), then encoded to unicode.
 
 0.6.1
 ~~~~~~~~~~~~~~~~~
 
 #30 Yet another demandimport conflict fixed.
 
 0.6.0
 ~~~~~~~~~~~~~~~~~
 
 #28 Disable demandimport completely during keyring import. Mayhaps it
 will resolve (most) demandimport conflict errors.
 
 0.5.7
 ~~~~~~~~~~~~~~~~~
 
 #27 Some more demandimport ignores.
 
 0.5.6
 ~~~~~~~~~~~~~~~~~
 
 #24, #25 Demandimport fixes (import failures in specific cases).
 
 Better way of demandimport-ignoring modules. In particular, we append
 more of them if gobject happens to be on the list.
 
 0.5.5
 ~~~~~~~~~~~~~~~~~
 
 Fix for gnome keyring import problems.
 
 0.5.4
 ~~~~~~~~~~~~~~~~~
 
 #22 Some more demandimport ignores (fix import failures).
 
 SMTP password was not cleared properly (after detecting that it is
 invalid).
 
 Clarified license to be modified BSD style license.
 
 0.5.3
 ~~~~~~~~~~~~~~~~~
 
 Remove useless import which caused problems on Mercurial 2.3 when
 demandimport was not enabled
 
 0.5.1
 ~~~~~~~~~~~~~~~~~
 
 Add help text to output for hg help.
 
 0.5.0
 ~~~~~~~~~~~~~~~~~
 
 Improved bad password detection. Internally: extension is now able to
 properly differentiate between an authentication failure and a new
 request to the same url.
 
 Fixes in debug message
 
 Further debug messages patching
 
 Improving debug messages handling.
 
 Mercurial Keyring debug messages are now prefixed with
 [HgKeyring] to make distinguishing them easier
 
 0.4.6
 ~~~~~~~~~~~~~~~~~
 
 More compatibility (changed signature of httpconnection.readauthforuri
 , introduced post Mercurial 1.9 - since hg.0593e8f81c71)
 
 Fix compatibility code which did not work due to demandimport issues
 (attempts to catch ImportErrors on "from mercurial.url import
 readauthforuri" were not working properly).
 
 0.4.5
 ~~~~~~~~~~~~~~~~~
 
 Mercurial 1.9 compatibility (readauthforuri has been moved into new
 httpconnection module).
 
 0.4.4
 ~~~~~~~~~~~~~~~~~
 
 Mercurial 1.8 compatibility (passwordmgr.readauthtoken() has been
 moved into mercurial.url.readauthforuri).
 
 0.4.3
 ~~~~~~~~~~~~~~~~~
 
 Keyring fork no longer is needed as keyring releases are available
 again.
 
 Workaround for gnomekeyring mercurial.demandimport incompatibility:
 mercurial.demandimport, which is enabled while in a mercurial
 extensions, prevents the correct import of gobject._gobject and
 consequently doesn't allow the loading of the gnomekeyring module,
 which can be used by keyring. This just adds the proper module to
 demandimport ignore list.
 
 0.4.2
 ~~~~~~~~~~~~~~~~~
 
 No longer raising an error when username is specified both in ~/.hgrc
 and <repo>/.hg/hgrc if it is the same in both places.
 
 Docs recommend sborho keyring fork.
 
 0.4.1
 ~~~~~~~~~~~~~~~~~
 
 Some tweaks and docs related to prefix handling.
 
 Explicit information that keyring is not used due to lack of username.
 
 0.4.0
 ~~~~~~~~~~~~~~~~~
 
 Store and lookup prefix from [auth] so that password is shared amongst
 shared auth entries
 
 0.3.3
 ~~~~~~~~~~~~~~~~~
 
 Better error message
 
 0.3.2
 ~~~~~~~~~~~~~~~~~
 
 Doc tweaks
 
 0.3.1
 ~~~~~~~~~~~~~~~~~
 
 Introduced and documented PyPi package, added setup.py
 
 0.2.0
 ~~~~~~~~~~~~~~~~~
 
 Added handling of SMTP passwords (tested on patchbomb extension but
 should work on anything what utilizes mercurial.mail)
 
 Docstrings mention Debian keyring packages.
 
 0.1.1
 ~~~~~~~~~~~~~~~~~
 
 Initial public release

README.txt

@@ -1,408 +1,418 b''
 .. -*- mode: rst; compile-command: "rst2html README.txt README.html" -*-
 
 =======================================================
 Mercurial Keyring
 =======================================================
 
 Mercurial Keyring is a Mercurial_ extension used to securely save HTTP
 and SMTP authentication passwords in password databases (Gnome
 Keyring, KDE KWallet, OSXKeyChain, Windows Vault etc).
 
 With ``mercurial_keyring`` active, Mercurial remembers your passwords
 and reuses them without prompting (as if you stored them in ``.hgrc``),
 but password storage is reasonably secure.
 
 Actual password storage is implemented by the keyring_ library, this 
 extension glues it to Mercurial.
 
 .. contents::
    :local:
    :depth: 2
 
 .. sectnum::
 
 .. _keyring: http://pypi.python.org/pypi/keyring
 .. _Mercurial: http://mercurial.selenic.com
 
 How does it work
 =======================================================
 
 On your first pull or push to HTTP url (or first email sent via given
 SMTP server), you are prompted for the password, just like bare
 Mercurial does. But the password you entered is saved to appropriate
 password database. On successive runs, whenever the password is
 needed, ``mercurial_keyring`` checks for password in password
 database, and uses it without troubling you.
 
 In case password turns out to be incorrect (for example, because you
 changed it, or entered it incorrectly), ``mercurial_keyring`` prompts
 you again, and overwrites the password.
 
 You can use many passwords (for various remote urls).  Saved passwords
 are identified by pair of username and url prefix. See below for
 information how to configure those properly.
 
 Installation
 =======================================================
 
 Prerequisites
 -------------
 
 This extension requires keyring_ and `mercurial_extension_utils`_ to
 work. In many cases both will be installed automatically while you
 install ``mercurial_keyring``, but you may need to control the process.
 
 The keyring_ library can usually be installed by::
 
     pip install --user keyring
 
 (or ``easy_install keyring``), but on some systems it is preferable to
 use official distribution archive. For example, on Debian and Ubuntu,
 you may install ``python-keyring`` and either ``python-keyring-gnome``
 or ``python-keyring-kwallet`` packages::
 
     sudo apt-get install python-keyring python-keyring-gnome
 
 (this will save you the need to provide working compiler and various
 development libraries).
 
 The `mercurial_extension_utils`_ module is tiny Python-only module,
 which can be installed by::
 
     pip install --user mercurial_extension_utils
 
 but in some cases (Windows…) requires more care. See
 `mercurial_extension_utils`_ documentation.
     
 
 Extension installation
 ----------------------
 
 There are two possible ways of installing the extension: using PyPi package,
 or using source clone.
 
 To install as a package::
 
     pip install --user mercurial_keyring
 
 (or ``sudo pip install mercurial_keyring`` for system-wide
 installation) and then enable it in ``~/.hgrc`` (or
 ``/etc/mercurial/hgrc`` or ``Mercurial.ini``) using::
 
     [extensions]
     mercurial_keyring = 
 
 To install using source clone, install keyring_ according to the
 instructions above, then clone::
     
     hg clone https://bitbucket.org/Mekk/mercurial_keyring/
     hg clone https://bitbucket.org/Mekk/mercurial-extension_utils/
 
 and configure Mercurial using  full path to the extension module::
 
     [extensions]
     mercurial_keyring = /path/to/mercurial_keyring/mercurial_keyring.py
 
 .. _the code: 
 .. _mercurial_keyring.py: http://bitbucket.org/Mekk/mercurial_keyring/src/tip/mercurial_keyring.py
 
 Password backend configuration
 =======================================================
 
 The most appropriate password backend should usually be picked without
 configuration (considering installed libraries, operating system,
 active desktop session). Still, if necessary, it can be configured
 using ``keyringrc.cfg`` file.  Refer to keyring_ docs for more
 details.
 
 .. note::
 
    With current (as I write) keyring (5.6), this file is (on Linux)
    located at ``~/.local/share/python_keyring/keyringrc.cfg`` and
    it's example content looks like::
 
         [backend]
         default-keyring=keyring.backends.Gnome.Keyring
         # default-keyring=keyring.backends.kwallet.Keyring
 
    For list of known backends run ``pydoc keyring.backends`` or
    ``keyring --list-backends`` (which of those commands work,
    depends on the keyring_ version).
 
 
 ``hgrc`` configuration (HTTP)
 =======================================================
 
 Mercurial Keyring uses standard Mercurial ``[auth]`` configuration to
 detect your username (on given remote) and url prefix. You are
 strongly advised to configure both.
 
 Without the username ``mercurial_keyring`` can't save or restore
 passwords, so it disables itself.
 
 Without url prefix ``mercurial_keyring`` works, but binds passwords to
 repository urls. That means you will have to (re)enter password for
 every repository cloned from given remote (and that there will be many
 copies of this password in secure storage).
 
 Repository level configuration
 ------------------------------------
 
 Edit repository-local ``.hg/hgrc`` and save there the remote
 repository path and the username, but do not save the password. For
 example:
 
 ::
 
     [paths]
     myremote = https://my.server.com/hgrepo/someproject
 
     [auth]
     myremote.prefix = https://my.server.com/hgrepo
     myremote.username = John
 
 Simpler form with url-embedded name can also be used:
 
 ::
 
     [paths]
     bitbucket = https://[e-mail]/hgrepo/someproject/
 
 but is not recommended.
 
 Note that all repositories sharing the same ``prefix`` share the same
 password.
 
 Mercurial allows also for password in ``.hg/hgrc`` (either given by
 ``«prefix».password``, or embedded in url). If such password is found,
 Mercurial Keyring disables itself.
 
 
 Account-level configuration
 ---------------------------
 
 If you are consistent about remote repository nicknames, you can
 configure the username in your `~/.hgrc` (`.hgrc` in your home
 directory). For example, write there::
 
     [auth]
     acme.prefix = hg.acme.com/repositories
     acme.username = johnny
     acme.schemes = http https
     bitbucket.prefix = https://bitbucket.org
     bitbucket.username = Mekk
     mydep.prefix = https://dev.acmeorg.com
     mydep.username = drmartin
 
 and as long as you use  ``acme`` alias for repositories like
 ``https://hg.acme.com/repositories/my_beautiful_app``, username
 ``johnny`` will be used, and the same password reused. Similarly
 any ``hg push bitbucket`` will share the same password.
 
 With such config repository-level ``.hg/hgrc`` need only contain
 ``[paths]``.
 
 Additional advantage of this method is that it works also during
 `clone`.
 
 
 .. note::
 
    Mercurial Keyring works well with `Path Pattern`_. On my setup I use
    prefix as above, and::
 
        [path_pattern]
        bitbucket.local = ~/devel/{below}
        bitbucket.remote = https://bitbucket.org/Mekk/{below:/=-}
  
    so all my repositories understand ``hg push bitbucket`` without
    any repository-level configuration.
 
 
 ``hgrc`` configuration (SMTP)
 =======================================================
 
 Edit either repository-local ``.hg/hgrc``, or ``~/.hgrc`` and set
 there all standard email and smtp properties, including SMTP
 username, but without SMTP password. For example:
 
 ::
 
     [email]
     method = smtp
     from = Joe Doe <[e-mail]>
 
     [smtp]
     host = smtp.gmail.com
     port = 587
     username = [e-mail]
     tls = true
 
 Just as in case of HTTP, you *must* set username, but *must not* set
 password here to use the extension, in other cases it will revert to
 the default behavior.
 
 Usage
 ======================================================
 
 Saving and restoring passwords
 -------------------------------------------------------
 
 Configure the repository as above, then just ``hg pull``, ``hg push``,
 etc.  You should be asked for the password only once (per every
 username and remote repository prefix or url combination).
 
 Similarly, for email, configure as above and just ``hg email``.
 Again, you will be asked for the password once (per every username and
 email server address combination).
 
 Checking password status (``hg keyring_check``)
 -------------------------------------------------------
 
 The ``keyring_check`` command can be used to check whether/which
 password(s) are saved. It can be used in three ways:
 
 - without parameters, it prints info related to all HTTP paths
   defined for current repository (everything from ``hg paths``
   that resolves to HTTP url)::
 
     hg keyring_check
 
 - given alias as param, it prints info about this alias::
 
     hg keyring_check work
 
 - finally, any path can be checked::
 
     hg keyring_check https://bitbucket.org/Mekk/mercurial_keyring
 
 Deleting saved password (``hg keyring_clear``)
 -------------------------------------------------------
 
 The ``keyring_clear`` command removes saved password related to given
 path. It can be used in two ways:
 
 - given alias as param, it drops password used by this alias::
 
     hg keyring_clear work
 
 - given full path, it drops password related to this path::
 
     hg keyring_clear https://bitbucket.org/Mekk/mercurial_keyring
 
 Managing passwords using GUI tools
 ------------------------------------------------------
 
 Many password backends provide GUI tools for password management,
 for example Gnome Keyring passwords can be managed using ``seahorse``,
 and KDE Wallet using ``kwalletmanager``. Those GUI tools can be used
 to review, edit, or delete saved passwords.
 
 Unfortunately, as I write, keyring_ library does not allow one to
 configure how/where exactly saved passwords are put in the hierarchy,
 and the place is not always intuitive. For example, in KDE Wallet, all
 passwords saved using ``mercurial_keyring`` show up in the folder
 named ``Python``.
 
 .. note::
 
    This is slightly problematic in case ``mercurial_keyring`` is not
    the only program using keyring_ library. Passwords saved by another
    Python application or script (which also uses keyring_) will be put
    into the same place, and it may be unclear which password belongs
    to which program. To remedy this, ``mercurial_keyring`` applies
    slightly unusual labels of the form
    ``«username»@@«urlprefix»@Mercurial`` - for example my bitbucket
    password is labelled ``Mekk@@https://bitbucket.org@Mercurial``.
 
 Implementation details
 =======================================================
 
 The extension is monkey-patching the mercurial ``passwordmgr`` class
 to replace the ``find_user_password`` method. Detailed order of operations
 is described in the comments inside `the code`_.
 
 Frequent problems
 =======================================================
 
 Most problems people face while using ``mercurial_keyring`` are in
 fact problems with ``keyring`` library and it's backends. In
 particular, those can manifest by:
 
 - technical errors mentioning sentences like ``No recommended backend
   was available. Install the keyrings.alt package…`` (or similar),
 
 - warnings like ``keyring: keyring backend doesn't seem to work…``
 
 - password prompts on every action (= passwords not being saved).
 
 Those almost always mean that *natural* keyring backend for given
 desktop type doesn't work, or is not present at all.  For example,
 some necessary runtime component can be down (say, you use Linux, but
 have neither Gnome Keyring, nor KDE Wallet, running). Or appropriate
 backend is not installed because it could not be build during
 keyring_ library installation (maybe because some required library
 was not present at the moment of keyring installation, or maybe because compiler
 as such is not present on the system).
 
 To diagnose such problems, try using ``keyring`` utility, as described
 on keyring_ documentation page, for example by::
 
     keyring --list-backends
     keyring -b keyrings.alt.Gnome.Keyring set testsvc testuser
     keyring -b keyrings.alt.Gnome.Keyring get testsvc testuser
 
 (of course using appropriate backend). If you miss the ``keyring`` command
 as such, try ``python -m keyring`` instead::
 
     python -m keyring --list-backends
     python -m keyring -b keyrings.alt.Gnome.Keyring set testsvc testuser
     python -m keyring -b keyrings.alt.Gnome.Keyring get testsvc testuser
 
 If appropriate backend is missing (not listed), or doesn't work
 (second or third command fails), your keyring is broken. Try looking
 for further pointers in keyring_ documentation, that project mailing
 list, or issue tracker. Typically it will turn out, that you need to
 install some missing tool, or library, and reinstall keyring.
 
+.. note::
+
+   Depending on keyring_ version, installation of some dependency may
+   resolve problem. For example (as of late 2018), I got KDE Wallet
+   backend working with pip-installed keyring after::
+
+       pip install dbus-python
+
+
 If ``keyring`` command works, but mercurial with mercurial_keyring does not,
 try enforcing  proper backend (by means of ``keyringrc.cfg``, see above).
 Only if this doesn't help, there may be a bug in mercurial_keyring.
 
 .. note::
 
    By far easiest way to have properly working keyring is to use
    packaged binary version (like ``python-keyring`` Ubuntu package, or
    keyring bundled with TortoiseHG on some systems). If you pip-installed
    keyring and it doesn't work, you may consider ``pip uninstall keyring``
    and look for binary package instead.
 
 
 
+
 History
 =======================================================
 
 See `HISTORY.txt`_.
 
 Development
 =======================================================
 
 Development is tracked on BitBucket, see 
 http://bitbucket.org/Mekk/mercurial_keyring/
 
 
 Additional notes
 =======================================================
 
 Information about this extension is also available
 on Mercurial Wiki: http://mercurial.selenic.com/wiki/KeyringExtension
 
 Check also `other Mercurial extensions I wrote`_.
 
 .. _other Mercurial extensions I wrote: http://mekk.bitbucket.io/mercurial.html
 
 .. _HISTORY.txt: http://bitbucket.org/Mekk/mercurial_keyring/src/tip/HISTORY.txt
 .. _TortoiseHg: http://tortoisehg.bitbucket.org/
 .. _Mercurial: http://mercurial.selenic.com
 .. _mercurial_extension_utils: https://bitbucket.org/Mekk/mercurial-extension_utils/
 .. _Path Pattern: https://bitbucket.org/Mekk/mercurial-path_pattern/