updated vim-ipython

Various small updates to vim integration.

Closes  #1120, rebased to avoid unnecessary recursive merge.

docs/examples/vim/README.rst

@@ -4,11 +4,10 b' vim-ipython'
 
 A two-way integration between Vim and IPython 0.11+
 
-author: Paul Ivanov (http://pirsquared.org)
-
-github: http://github.com/ivanov/vim-ipython
-
-demo: http://pirsquared.org/vim-ipython/
+* author: Paul Ivanov (http://pirsquared.org)
+* github: http://github.com/ivanov/vim-ipython
+* demos: http://pirsquared.org/vim-ipython/
+* blogpost: http://pirsquared.org/blog/2011/07/28/vim-ipython/
 
 Using this plugin, you can send lines or whole files for IPython to
 execute, and also get back object introspection and word completions in
@@ -16,30 +15,46 b' Vim, like what you get with: ``object?<enter>`` and ``object.<tab>`` in'
 IPython.
 
 The big change from previous versions of ``ipy.vim`` is that it no longer
-requires the brittle ipy_vimserver.py instantiation, and since it uses
-just vim and python, it is platform independent (i.e. should work even
-on windows, unlike the previous \*nix only solution)
+requires the old brittle ``ipy_vimserver.py`` instantiation, and since
+it uses just vim and python, it is platform independent (i.e. should work
+even on windows, unlike the previous \*nix only solution). The requirements
+are IPython 0.11+ with zeromq capabilities, vim compiled with +python.
 
+If you can launch ``ipython qtconsole`` or ``ipython kernel``, and
+``:echo has('python')`` returns 1 in vim, you should be good to go.
 
 -----------------
 Quickstart Guide:
 -----------------
-Start ``ipython qtconsole`` and copy the connection string.
+Start ``ipython qtconsole`` [*]_ and copy the connection string.
 Source ``ipy.vim`` file, which provides new IPython command::
 
-  :source ipy.vim  
+  :source ipy.vim
   (or copy it to ~/.vim/ftplugin/python to load automatically)
 
-  :IPythonClipboard   
+  :IPythonClipboard
   (or :IPythonXSelection if you're using X11 without having to copy)
 
-The :IPython command allows you to put the full string, e.g.::
+The :IPython command allows you to put the full connection string. For IPython
+0.11, it would look like this::
 
   :IPython --existing --shell=41882 --iopub=43286 --stdin=34987 --hb=36697
 
+and for IPython 0.12, like this::
+
+  :IPython --existing kernel-85997.json
+
 The ``:IPythonClipboard`` command just uses the ``+`` register to get the
 connection string, whereas ``:IPythonXSelection`` uses the ``*`` register
 
+.. [*] Though the demos above use ``qtconsole``, it is not required
+    for this workflow, it's just that it was the easiest way to show how to
+    make use of the new functionality in 0.11 release. In the current git
+    trunk of IPython, you can use ``ipython kernel`` to create a kernel and
+    get the connection string to use for any frontend (including vim-ipython).
+    If you are still using 0.11, you can launch a regular kernel using
+    ``python -c "from IPython.zmq.ipkernel import main; main()"``
+
 ------------------------
 Sending lines to IPython
 ------------------------
@@ -50,10 +65,12 b' Now type out a line and send it to IPython using ``<Ctrl-S>`` from Command mode:'
 You should see a notification message confirming the line was sent, along
 with the input number for the line, like so ``In[1]: import os``.
 
-``<Ctrl-S>`` also works from insert mode, but doesn't show notification
+``<Ctrl-S>`` also works from insert mode, but doesn't show notification,
+unless ``monitor_subchannel`` is set to ``True`` (see `vim-ipython 'shell'`_,
+below)
 
-It also works blockwise in Visual Mode. Strip the leading double quotes and
-send these lines using ``<Ctrl-S>``::
+It also works blockwise in Visual Mode. Select and send these lines using
+``<Ctrl-S>``::
 
   import this,math # secret decoder ring
   a,b,c,d,e,f,g,h,i = range(1,10)
@@ -65,7 +82,7 b' send these lines using ``<Ctrl-S>``::'
   '%d'*len(code)%code == str(int(math.pi*1e5))
 
 Then, go to the qtconsole and run this line::
-  
+
   print secret_decoder(_i,_)
 
 You can also send whole files to IPython's ``%run`` magic using ``<F5>``.
@@ -86,22 +103,87 b" IPython's tab-completion Functionality"
 vim-ipython activates a 'completefunc' that queries IPython.
 A completefunc is activated using ``Ctrl-X Ctrl-U`` in Insert Mode (vim
 default). You can combine this functionality with SuperTab to get tab
-completion 
+completion.
 
----------------
-Current issues:
----------------
-For now, vim-ipython only connects to an ipython session in progress.
+-------------------
+vim-ipython 'shell'
+-------------------
+**NEW since IPython 0.11**!
+
+By monitoring km.sub_channel, we can recreate what messages were sent to
+IPython, and what IPython sends back in response.
+
+``monitor_subchannel`` is a parameter that sets whether this 'shell' should
+updated on every sent command (default: True).
+
+If at any later time you wish to bring this shell up, including if you've set
+``monitor_subchannel=False``, hit ``<leader>s``.
 
-ipy.vim takes a while to load, I'll eventually move the python code to its
-own file and do a lazy import (only when the IPython command is called)
+-------
+Options
+-------
+You can change these at the top of the ipy.vim::
 
-The ipdb integration is not yet re-implemented.
+  reselect = False            # reselect lines after sending from Visual mode
+  show_execution_count = True # wait to get numbers for In[43]: feedback?
+  monitor_subchannel = True   # update vim-ipython 'shell' on every send?
+  run_flags= "-i"             # flags to for IPython's run magic when using <F5>
 
-Need to add more message handling for sub_channel messages from IPython
-(i.e. notification of changes which were not sent from vim).
+**Disabling default mappings**
+In your own ``.vimrc``, if you don't like the mappings provided by default,
+you can define a variable ``let g:ipy_perform_mappings=0`` which will prevent
+vim-ipython from defining any of the default mappings.
 
-------
-Thanks
-------
-@MinRK for guiding me through the IPython kernel manager protocol.
+---------------
+Current issues:
+---------------
+- For now, vim-ipython only connects to an ipython session in progress.
+- The ipdb integration is not yet re-implemented.
+- If you're running inside ``screen``, read about the ``<CTRL-S>`` issue `here
+  <http://munkymorgy.blogspot.com/2008/07/screen-ctrl-s-bug.html>`_, and add
+  this line to your ``.bashrc`` to fix it::
+
+    stty stop undef # to unmap ctrl-s
+
+- In vim, if you're getting ``ImportError: No module named
+  IPython.zmq.blockingkernelmanager`` but are able to import it in regular
+  python, **either**
+
+  1. your ``sys.path`` in vim differs from the ``sys.path`` in regular python.
+     Try running these two lines, and comparing their output files::
+
+      $ vim -c 'py import vim, sys; vim.current.buffer.append(sys.path)' -c ':wq vim_syspath'
+      $ python -c "import sys; f=file('python_syspath','w'); f.write('\n'.join(sys.path)); f.close()"
+
+  **or**
+
+  2. your vim is compiled against a different python than you are launching. See
+     if there's a difference between ::
+
+      $ vim -c ':py import os; print os.__file__' -c ':q'
+      $ python -c ':py import os; print os.__file__'
+
+- For vim inside a terminal, using the arrow keys won't work inside a
+  documentation buffer, because the mapping for ``<Esc>`` overlaps with
+  ``^[OA`` and so on, and we use ``<Esc>`` as a quick way of closing the
+  documentation preview window. If you want go without this quick close
+  functionality and want to use the arrow keys instead, look for instructions
+  starting with "Known issue: to enable the use of arrow keys..." in the
+  ``get_doc_buffer`` function.
+
+- @fholgado's update to ``minibufexpl.vim`` that is up on GitHub will always
+  put the cursor in the minibuf after sending a command when
+  ``monitor_subchannel`` is set. This is a bug in minibufexpl.vim and the workaround
+  is described in vim-ipython issue #7.
+
+----------------------------
+Thanks and Bug Participation
+----------------------------
+* @MinRK for guiding me through the IPython kernel manager protocol.
+* @nakamuray and @tcheneau for reporting and providing a fix for when vim is compiled without a gui (#1)
+* @unpingco for reporting Windows bugs (#3,#4)
+* @simon-b for terminal vim arrow key issue (#5)
+* @jorgesca and @kwgoodman for shell (#6)
+* @zeekay for easily allowing custom mappings (#9)
+* @minrk for support of connection_file-based IPython connection (#13)
+* @jorgesca for reporting the lack of profile handling capability (#14)

docs/examples/vim/ipy.vim

@@ -17,10 +17,17 b''
 "
 " written by Paul Ivanov (http://pirsquared.org)
 python << EOF
-import time
+reselect = False            # reselect lines after sending from Visual mode
+show_execution_count = True # wait to get numbers for In[43]: feedback?
+monitor_subchannel = True   # update vim-ipython 'shell' on every send?
+run_flags= "-i"             # flags to for IPython's run magic when using <F5>
+
 import vim
 import sys
 
+# get around unicode problems when interfacing with vim
+vim_encoding=vim.eval('&encoding') or 'utf-8'
+
 try:
     sys.stdout.flush
 except AttributeError:
@@ -34,10 +41,6 b' except AttributeError:'
     sys.stdout = WithFlush(sys.stdout)
     sys.stderr = WithFlush(sys.stderr)
 
-from IPython.zmq.blockingkernelmanager import BlockingKernelManager
-
-from IPython.config.loader import KeyValueConfigLoader
-from IPython.zmq.kernelapp import kernel_aliases
 
 
 ip = '127.0.0.1'
@@ -46,34 +49,62 b' try:'
 except NameError:
     km = None
 
-def km_from_string(s):
+def km_from_string(s=''):
     """create kernel manager from IPKernelApp string
-    such as '--shell=47378 --iopub=39859 --stdin=36778 --hb=52668'
+    such as '--shell=47378 --iopub=39859 --stdin=36778 --hb=52668' for IPython 0.11
+    or just 'kernel-12345.json' for IPython 0.12
     """
-    global km,send
-    # vim interface currently only deals with existing kernels
-    s = s.replace('--existing','')
-    loader = KeyValueConfigLoader(s.split(), aliases=kernel_aliases)
-    cfg = loader.load_config()['KernelApp']
-    try:
-        km = BlockingKernelManager(
-            shell_address=(ip, cfg['shell_port']),
-            sub_address=(ip, cfg['iopub_port']),
-            stdin_address=(ip, cfg['stdin_port']),
-            hb_address=(ip, cfg['hb_port']))
-    except KeyError,e:
-        echo(":IPython " +s + " failed", "Info")
-        echo("^-- failed --"+e.message.replace('_port','')+" not specified", "Error")
-        return
+    from os.path import join as pjoin
+    from IPython.zmq.blockingkernelmanager import BlockingKernelManager, Empty
+    from IPython.config.loader import KeyValueConfigLoader
+    from IPython.zmq.kernelapp import kernel_aliases
+    global km,send,Empty
+
+    s = s.replace('--existing', '')
+    if 'connection_file' in BlockingKernelManager.class_trait_names():
+        from IPython.lib.kernel import find_connection_file
+        # 0.12 uses files instead of a collection of ports
+        # include default IPython search path
+        # filefind also allows for absolute paths, in which case the search
+        # is ignored
+        try:
+            # XXX: the following approach will be brittle, depending on what
+            # connection strings will end up looking like in the future, and
+            # whether or not they are allowed to have spaces. I'll have to sync
+            # up with the IPython team to address these issues -pi
+            if '--profile' in s:
+                k,p = s.split('--profile')
+                k = k.lstrip().rstrip() # kernel part of the string
+                p = p.lstrip().rstrip() # profile part of the string
+                fullpath = find_connection_file(k,p)
+            else:
+                fullpath = find_connection_file(s.lstrip().rstrip())
+        except IOError,e:
+            echo(":IPython " + s + " failed", "Info")
+            echo("^-- failed '" + s + "' not found", "Error")
+            return
+        km = BlockingKernelManager(connection_file = fullpath)
+        km.load_connection_file()
+    else:
+        if s == '':
+            echo(":IPython 0.11 requires the full connection string")
+            return
+        loader = KeyValueConfigLoader(s.split(), aliases=kernel_aliases)
+        cfg = loader.load_config()['KernelApp']
+        try:
+            km = BlockingKernelManager(
+                shell_address=(ip, cfg['shell_port']),
+                sub_address=(ip, cfg['iopub_port']),
+                stdin_address=(ip, cfg['stdin_port']),
+                hb_address=(ip, cfg['hb_port']))
+        except KeyError,e:
+            echo(":IPython " +s + " failed", "Info")
+            echo("^-- failed --"+e.message.replace('_port','')+" not specified", "Error")
+            return
     km.start_channels()
     send = km.shell_channel.execute
     return km
 
-
-reselect = False
-show_id= True
-run_flags= "-i"
-
 def echo(arg,style="Question"):
     try:
         vim.command("echohl %s" % style)
@@ -88,12 +119,12 b' def disconnect():'
     pass
 
 def get_doc(word):
+    if km is None:
+        return ["Not connected to IPython, cannot query \"%s\"" %word]
     msg_id = km.shell_channel.object_info(word)
-    time.sleep(.1)
     doc = get_doc_msg(msg_id)
-    #if len(doc):
-    #    echo(word, 'Special')
-    return doc
+    # get around unicode problems when interfacing with vim
+    return [d.encode(vim_encoding) for d in doc]
 
 import re
 # from http://serverfault.com/questions/71285/in-centos-4-4-how-can-i-strip-escape-sequences-from-a-text-file
@@ -102,25 +133,19 b' def strip_color_escapes(s):'
     return strip.sub('',s)
     
 def get_doc_msg(msg_id):
-    content = get_child_msgs(msg_id)[0]['content']
-    n = 13 # longest field name
-    ##XXX: debug (print the whole message)
+    n = 13 # longest field name (empirically)
     b=[]
+    try:
+        content = get_child_msg(msg_id)['content']
+    except Empty:
+        # timeout occurred
+        return ["no reply from IPython kernel"]
 
     if not content['found']:
         return b
 
-    ## debugging the whole message
-    #for k in content:
-    #    if isinstance(content[k],str) and content[k].find('\n')!=-1:
-    #        b.append(k.ljust(n)+":")
-    #        b.append(content[k].splitlines())
-    #    else:
-    #        b.append(k.ljust(n)+":"+str(content[k]))
-
     for field in ['type_name','base_class','string_form','namespace',
             'file','length','definition','source','docstring']:
-        # XXX: strip the 'definition' rich formatting
         c = content.get(field,None)
         if c:
             if field in ['definition']:
@@ -135,21 +160,27 b' def get_doc_msg(msg_id):'
     return b
 
 def get_doc_buffer(level=0):
-    word = vim.eval('expand("<cfile>")')
+    # empty string in case vim.eval return None
+    word = vim.eval('expand("<cfile>")') or ''
     doc = get_doc(word)
     if len(doc) ==0:
         echo(word+" not found","Error")
         return
+    # close any currently open preview windows
     vim.command('pcl')
+    # documentation buffer name is same as the query made to ipython
     vim.command('new '+word)
     vim.command('setlocal pvw modifiable noro')
+    # doc window quick quit keys: 'q' and 'escape'
     vim.command('map <buffer> q :q<CR>')
-    vim.command('map <buffer>  :q<CR>')
-    #vim.command('pedit '+docbuf.name)
+    # Known issue: to enable the use of arrow keys inside the terminal when
+    # viewing the documentation, comment out the next line
+    vim.command('map <buffer> <Esc> :q<CR>')
+    # and uncomment this line (which will work if you have a timoutlen set)
+    #vim.command('map <buffer> <Esc><Esc> :q<CR>')
     b = vim.current.buffer
-    #b.append(doc)
+    b[:] = None
     b[:] = doc
-    #b.append(doc)
     vim.command('setlocal nomodified bufhidden=wipe')
     #vim.command('setlocal previewwindow nomodifiable nomodified ro')
     #vim.command('set previewheight=%d'%len(b))# go to previous window
@@ -158,35 +189,153 b' def get_doc_buffer(level=0):'
     #vim.command('pedit doc')
     #vim.command('normal ') # go to previous window
 
-def get_child_msgs(msg_id):
+def update_subchannel_msgs(debug=False):
+    msgs = km.sub_channel.get_msgs()
+    if debug:
+        #try:
+        #    vim.command("b debug_msgs")
+        #except vim.error:
+        #    vim.command("new debug_msgs")
+        #finally:
+        db = vim.current.buffer
+    else:
+        db = []
+    b = vim.current.buffer
+    startedin_vimipython = vim.eval('@%')=='vim-ipython'
+    if not startedin_vimipython:
+        # switch to preview window
+        vim.command(
+            "try"
+            "|silent! wincmd P"
+            "|catch /^Vim\%((\a\+)\)\=:E441/"
+            "|silent pedit +set\ ma vim-ipython"
+            "|silent! wincmd P"
+            "|endtry")
+        # if the current window is called 'vim-ipython'
+        if vim.eval('@%')=='vim-ipython':
+            # set the preview window height to the current height
+            vim.command("set pvh=" + vim.eval('winheight(0)'))
+        else:
+            # close preview window, it was something other than 'vim-ipython'
+            vim.command("pcl")
+            vim.command("silent pedit +set\ ma vim-ipython")
+            vim.command("wincmd P") #switch to preview window
+            # subchannel window quick quit key 'q'
+            vim.command('map <buffer> q :q<CR>')
+            vim.command("set bufhidden=hide buftype=nofile ft=python")
+
+    #syntax highlighting for python prompt
+    # QtConsole In[] is blue, but I prefer the oldschool green
+    # since it makes the vim-ipython 'shell' look like the holidays!
+    #vim.command("hi Blue ctermfg=Blue guifg=Blue")
+    vim.command("hi Green ctermfg=Green guifg=Green")
+    vim.command("hi Red ctermfg=Red guifg=Red")
+    vim.command("syn keyword Green 'In\ []:'")
+    vim.command("syn match Green /^In \[[0-9]*\]\:/")
+    vim.command("syn match Red /^Out\[[0-9]*\]\:/")
+    b = vim.current.buffer
+    for m in msgs:
+        #db.append(str(m).splitlines())
+        s = ''
+        if 'msg_type' not in m['header']:
+            # debug information
+            #echo('skipping a message on sub_channel','WarningMsg')
+            #echo(str(m))
+            continue
+        elif m['header']['msg_type'] == 'status':
+            continue
+        elif m['header']['msg_type'] == 'stream':
+            s = strip_color_escapes(m['content']['data'])
+        elif m['header']['msg_type'] == 'pyout':
+            s = "Out[%d]: " % m['content']['execution_count']
+            s += m['content']['data']['text/plain']
+        elif m['header']['msg_type'] == 'pyin':
+            # TODO: the next line allows us to resend a line to ipython if
+            # %doctest_mode is on. In the future, IPython will send the
+            # execution_count on subchannel, so this will need to be updated
+            # once that happens
+            if 'execution_count' in m['content']:
+                s = "\nIn [%d]: "% m['content']['execution_count']
+            else:
+                s = "\nIn [00]: "
+            s += m['content']['code'].strip()
+        elif m['header']['msg_type'] == 'pyerr':
+            c = m['content']
+            s = "\n".join(map(strip_color_escapes,c['traceback']))
+            s += c['ename'] + ":" + c['evalue']
+        if s.find('\n') == -1:
+            # somewhat ugly unicode workaround from
+            # http://vim.1045645.n5.nabble.com/Limitations-of-vim-python-interface-with-respect-to-character-encodings-td1223881.html
+            if isinstance(s,unicode):
+                s=s.encode(vim_encoding)
+            b.append(s)
+        else:
+            try:
+                b.append(s.splitlines())
+            except:
+                b.append([l.encode(vim_encoding) for l in s.splitlines()])
+    # make a newline so we can just start typing there
+    if b[-1] != '':
+        b.append([''])
+    vim.command('normal G') # go to the end of the file
+    if not startedin_vimipython:
+        vim.command('normal p') # go back to where you were
+
+def get_child_msg(msg_id):
     # XXX: message handling should be split into its own process in the future
-    msgs= km.shell_channel.get_msgs()
-    children = [m for m in msgs if m['parent_header']['msg_id'] == msg_id]
-    return children
+    while True:
+        # get_msg will raise with Empty exception if no messages arrive in 1 second
+        m= km.shell_channel.get_msg(timeout=1)
+        if m['parent_header']['msg_id'] == msg_id:
+            break
+        else:
+            #got a message, but not the one we were looking for
+            echo('skipping a message on shell_channel','WarningMsg')
+    return m
             
-
-
-def run_this_file():
-    send('run %s %s' % (run_flags, vim.current.buffer.name,))
-    echo("In[]: run %s %s" % (run_flags, vim.current.buffer.name))
-
 def print_prompt(prompt,msg_id=None):
-    global show_id
-    if show_id and msg_id:
-        time.sleep(.1) # wait to get message back from kernel
-        children = get_child_msgs(msg_id)
-        if len(children):
-            count = children[0]['content']['execution_count']
+    """Print In[] or In[42] style messages"""
+    global show_execution_count
+    if show_execution_count and msg_id:
+        # wait to get message back from kernel
+        try:
+            child = get_child_msg(msg_id)
+            count = child['content']['execution_count']
             echo("In[%d]: %s" %(count,prompt))
-        else:
-            echo("In[]: %s (no reply from kernel)" % prompt)
+        except Empty:
+            echo("In[]: %s (no reply from IPython kernel)" % prompt)
     else:
         echo("In[]: %s" % prompt)
 
+def with_subchannel(f,*args):
+    "conditionally monitor subchannel"
+    def f_with_update(*args):
+        try:
+            f(*args)
+            if monitor_subchannel:
+                update_subchannel_msgs()
+        except AttributeError: #if km is None
+            echo("not connected to IPython", 'Error')
+    return f_with_update
+
+@with_subchannel
+def run_this_file():
+    msg_id = send('run %s %s' % (run_flags, repr(vim.current.buffer.name),))
+    print_prompt("In[]: run %s %s" % (run_flags, repr(vim.current.buffer.name)),msg_id)
+
+@with_subchannel
 def run_this_line():
     msg_id = send(vim.current.line)
     print_prompt(vim.current.line, msg_id)
 
+@with_subchannel
+def run_command(cmd):
+    msg_id = send(cmd)
+    print_prompt(cmd, msg_id)
+    if monitor_subchannel:
+        update_subchannel_msgs()
+
+@with_subchannel
 def run_these_lines():
     r = vim.current.range
     lines = "\n".join(vim.current.buffer[r.start:r.end+1])
@@ -208,16 +357,19 b' def run_these_lines():'
 def dedent_run_this_line():
     vim.command("left")
     run_this_line()
-    vim.command("undo")
+    vim.command("silent undo")
 
 def dedent_run_these_lines():
-    vim.command("'<,'>left")
+    r = vim.current.range
+    shiftwidth = vim.eval('&shiftwidth')
+    count = int(vim.eval('indent(%d+1)/%s' % (r.start,shiftwidth)))
+    vim.command("'<,'>" + "<"*count)
     run_these_lines()
-    vim.command("undo")
+    vim.command("silent undo")
     
 #def set_this_line():
 #    # not sure if there's a way to do this, since we have multiple clients
-#    send("_ip.IP.rl_next_input= \'%s\'" % vim.current.line.replace("\'","\\\'"))
+#    send("get_ipython().shell.set_next_input(\'%s\')" % vim.current.line.replace("\'","\\\'"))
 #    #print "line \'%s\' set at ipython prompt"% vim.current.line
 #    echo("line \'%s\' set at ipython prompt"% vim.current.line,'Statement')
 
@@ -262,34 +414,45 b' fun! <SID>toggle_send_on_save()'
     endif
 endfun
 
-map <silent> <F5> :python run_this_file()<CR>
-map <silent> <S-F5> :python run_this_line()<CR>
-map <silent> <F9> :python run_these_lines()<CR>
-map <leader>d :py get_doc_buffer()<CR>
-map <silent> <S-F9> :python toggle_reselect()<CR>
-"map <silent> <C-F6> :python send('%pdb')<CR>
-"map <silent> <F6> :python set_breakpoint()<CR>
-"map <silent> <s-F6> :python clear_breakpoint()<CR>
-"map <silent> <F7> :python run_this_file_pdb()<CR>
-"map <silent> <s-F7> :python clear_all_breaks()<CR>
-imap <C-F5> <C-O><F5>
-imap <S-F5> <C-O><S-F5>
-imap <silent> <F5> <C-O><F5>
-map <C-F5> :call <SID>toggle_send_on_save()<CR>
-
-"pi custom
-map <silent> <C-Return> :python run_this_file()<CR>
-map <silent> <C-s> :python run_this_line()<CR>
-map <silent> <M-s> :python dedent_run_this_line()<CR>
-vmap <silent> <C-S> :python run_these_lines()<CR>
-vmap <silent> <M-s> :python dedent_run_these_lines()<CR>
-"map <silent> <C-p> :python set_this_line()<CR>
-map <silent> <M-c> I#<ESC>
-vmap <silent> <M-c> I#<ESC>
-map <silent> <M-C> :s/^\([ \t]*\)#/\1/<CR>
-vmap <silent> <M-C> :s/^\([ \t]*\)#/\1/<CR>
-
-command! -nargs=+ IPython :py km_from_string("<args>")
+" Allow custom mappings
+if !exists('g:ipy_perform_mappings')
+    let g:ipy_perform_mappings = 1
+endif
+if g:ipy_perform_mappings != 0
+    map <silent> <F5> :python run_this_file()<CR>
+    map <silent> <S-F5> :python run_this_line()<CR>
+    map <silent> <F9> :python run_these_lines()<CR>
+    map <silent> <leader>d :py get_doc_buffer()<CR>
+    map <silent> <leader>s :py update_subchannel_msgs(); echo("vim-ipython shell updated",'Operator')<CR>
+    map <silent> <S-F9> :python toggle_reselect()<CR>
+    "map <silent> <C-F6> :python send('%pdb')<CR>
+    "map <silent> <F6> :python set_breakpoint()<CR>
+    "map <silent> <s-F6> :python clear_breakpoint()<CR>
+    "map <silent> <F7> :python run_this_file_pdb()<CR>
+    "map <silent> <s-F7> :python clear_all_breaks()<CR>
+    imap <C-F5> <C-O><F5>
+    imap <S-F5> <C-O><S-F5>
+    imap <silent> <F5> <C-O><F5>
+    map <C-F5> :call <SID>toggle_send_on_save()<CR>
+    "" Example of how to quickly clear the current plot with a keystroke
+    map <silent> <F12> :python run_command("plt.clf()")<cr>
+    "" Example of how to quickly close all figures with a keystroke
+    map <silent> <F11> :python run_command("plt.close('all')")<cr>
+
+    "pi custom
+    map <silent> <C-Return> :python run_this_file()<CR>
+    map <silent> <C-s> :python run_this_line()<CR>
+    imap <silent> <C-s> <C-O>:python run_this_line()<CR>
+    map <silent> <M-s> :python dedent_run_this_line()<CR>
+    vmap <silent> <C-S> :python run_these_lines()<CR>
+    vmap <silent> <M-s> :python dedent_run_these_lines()<CR>
+    map <silent> <M-c> I#<ESC>
+    vmap <silent> <M-c> I#<ESC>
+    map <silent> <M-C> :s/^\([ \t]*\)#/\1/<CR>
+    vmap <silent> <M-C> :s/^\([ \t]*\)#/\1/<CR>
+endif
+
+command! -nargs=* IPython :py km_from_string("<args>")
 command! -nargs=0 IPythonClipboard :py km_from_string(vim.eval('@+'))
 command! -nargs=0 IPythonXSelection :py km_from_string(vim.eval('@*'))
 
@@ -297,13 +460,14 b' function! IPythonBalloonExpr()'
 python << endpython
 word = vim.eval('v:beval_text')
 reply = get_doc(word)
-#reply = reply[:40]
 vim.command("let l:doc = %s"% reply)
 endpython
 return l:doc
 endfunction
-set bexpr=IPythonBalloonExpr()
-set ballooneval
+if has('balloon_eval')
+    set bexpr=IPythonBalloonExpr()
+    set ballooneval
+endif
 
 fun! CompleteIPython(findstart, base)
 	  if a:findstart
@@ -322,21 +486,35 b' fun! CompleteIPython(findstart, base)'
 base = vim.eval("a:base")
 findstart = vim.eval("a:findstart")
 msg_id = km.shell_channel.complete(base, vim.current.line, vim.eval("col('.')"))
-time.sleep(.1)
-m = get_child_msgs(msg_id)[0]
-
-matches = m['content']['matches']
-#end = len(base)
-#completions = [m[end:]+findstart+base for m in matches]
-matches.insert(0,base) # the "no completion" version
-completions = matches
-vim.command("let l:completions = %s"% completions)
+try:
+    m = get_child_msg(msg_id)
+    matches = m['content']['matches']
+    matches.insert(0,base) # the "no completion" version
+    # we need to be careful with unicode, because we can have unicode
+    # completions for filenames (for the %run magic, for example). So the next
+    # line will fail on those:
+    #completions= [str(u) for u in matches]
+    # because str() won't work for non-ascii characters
+    # and we also have problems with unicode in vim, hence the following:
+    completions = [s.encode(vim_encoding) for s in matches]
+except Empty:
+    echo("no reply from IPython kernel")
+    completions=['']
+## Additionally, we have no good way of communicating lists to vim, so we have
+## to turn in into one long string, which can be problematic if e.g. the
+## completions contain quotes. The next line will not work if some filenames
+## contain quotes - but if that's the case, the user's just asking for
+## it, right?
+#completions = '["'+ '", "'.join(completions)+'"]'
+#vim.command("let completions = %s" % completions)
+## An alternative for the above, which will insert matches one at a time, so
+## if there's a problem with turning a match into a string, it'll just not
+## include the problematic match, instead of not including anything. There's a
+## bit more indirection here, but I think it's worth it
+for c in completions:
+    vim.command('call add(res,"'+c+'")')
 endpython
-	    for m in l:completions
-	      "if m =~ '^' . a:base
-            call add(res, m)
-	      "endif
-	    endfor
+        "call extend(res,completions)
 	    return res
 	  endif
 	endfun