docs/source/interactive/shell.txt

.. _ipython_as_shell:

=========================
IPython as a system shell
=========================

.. warning::

    As of the 0.11 version of IPython, some of the features and APIs
    described in this section have been deprecated or are broken.  Our plan
    is to continue to support these features, but they need to be updated
    to take advantage of recent API changes.  Furthermore, this section
    of the documentation need to be updated to reflect all of these changes.

Overview
========

The 'sh' profile optimizes IPython for system shell usage. Apart from
certain job control functionality that is present in unix (ctrl+z does
"suspend"), the sh profile should provide you with most of the
functionality you use daily in system shell, and more. Invoke IPython
in 'sh' profile by doing 'ipython -p sh', or (in win32) by launching
the "pysh" shortcut in start menu.

If you want to use the features of sh profile as your defaults (which
might be a good idea if you use other profiles a lot of the time but
still want the convenience of sh profile), add ``import ipy_profile_sh``
to your ~/.ipython/ipy_user_conf.py.

The 'sh' profile is different from the default profile in that:

 * Prompt shows the current directory
 * Spacing between prompts and input is more compact (no padding with
   empty lines). The startup banner is more compact as well.
 * System commands are directly available (in alias table) without
   requesting %rehashx - however, if you install new programs along
   your PATH, you might want to run %rehashx to update the persistent
   alias table
 * Macros are stored in raw format by default. That is, instead of
   '_ip.system("cat foo"), the macro will contain text 'cat foo')
 * Autocall is in full mode
 * Calling "up" does "cd .."

The 'sh' profile is different from the now-obsolete (and unavailable)
'pysh' profile in that:

 * '$$var = command' and '$var = command' syntax is not supported
 * anymore. Use 'var = !command' instead (incidentally, this is
 * available in all IPython profiles). Note that !!command *will*
 * work.

Aliases
=======

All of your $PATH has been loaded as IPython aliases, so you should be
able to type any normal system command and have it executed. See
%alias?  and %unalias? for details on the alias facilities. See also
%rehashx? for details on the mechanism used to load $PATH.


Directory management
====================

Since each command passed by ipython to the underlying system is executed
in a subshell which exits immediately, you can NOT use !cd to navigate
the filesystem.

IPython provides its own builtin '%cd' magic command to move in the
filesystem (the % is not required with automagic on). It also maintains
a list of visited directories (use %dhist to see it) and allows direct
switching to any of them. Type 'cd?' for more details.

%pushd, %popd and %dirs are provided for directory stack handling.


Enabled extensions
==================

Some extensions, listed below, are enabled as default in this profile.

envpersist
----------

%env can be used to "remember" environment variable manipulations. Examples::

	%env - Show all environment variables
	%env VISUAL=jed  - set VISUAL to jed
	%env PATH+=;/foo - append ;foo to PATH
	%env PATH+=;/bar - also append ;bar to PATH
	%env PATH-=/wbin; - prepend /wbin; to PATH
	%env -d VISUAL   - forget VISUAL persistent val
	%env -p          - print all persistent env modifications

ipy_which
---------

%which magic command. Like 'which' in unix, but knows about ipython aliases.

Example::

 [C:/ipython]|14> %which st
 st -> start .
 [C:/ipython]|15> %which d
 d -> dir /w /og /on
 [C:/ipython]|16> %which cp
 cp -> cp
   == c:\bin\cp.exe
 c:\bin\cp.exe

ipy_app_completers
------------------

Custom tab completers for some apps like svn, hg, bzr, apt-get. Try 'apt-get install <TAB>' in debian/ubuntu.

ipy_rehashdir
-------------

Allows you to add system command aliases for commands that are not along your path. Let's say that you just installed Putty and want to be able to invoke it without adding it to path, you can create the alias for it with rehashdir::

 [~]|22> cd c:/opt/PuTTY/
 [c:opt/PuTTY]|23> rehashdir .
              <23> ['pageant', 'plink', 'pscp', 'psftp', 'putty', 'puttygen', 'unins000']

Now, you can execute any of those commams directly::

 [c:opt/PuTTY]|24> cd
 [~]|25> putty

(the putty window opens).

If you want to store the alias so that it will always be available, do '%store putty'. If you want to %store all these aliases persistently, just do it in a for loop::

 [~]|27> for a in _23:
    |..>     %store $a
    |..>
    |..>
 Alias stored: pageant (0, 'c:\\opt\\PuTTY\\pageant.exe')
 Alias stored: plink (0, 'c:\\opt\\PuTTY\\plink.exe')
 Alias stored: pscp (0, 'c:\\opt\\PuTTY\\pscp.exe')
 Alias stored: psftp (0, 'c:\\opt\\PuTTY\\psftp.exe')
 ...

mglob
-----

Provide the magic function %mglob, which makes it easier (than the 'find' command) to collect (possibly recursive) file lists. Examples::

 [c:/ipython]|9> mglob *.py
 [c:/ipython]|10> mglob *.py rec:*.txt
 [c:/ipython]|19> workfiles = %mglob !.svn/ !.hg/ !*_Data/ !*.bak rec:.

Note that the first 2 calls will put the file list in result history (_, _9, _10), and the last one will assign it to 'workfiles'.


Prompt customization
====================

The sh profile uses the following prompt configurations::

    o.prompt_in1= r'\C_LightBlue[\C_LightCyan\Y2\C_LightBlue]\C_Green|\#>'
    o.prompt_in2= r'\C_Green|\C_LightGreen\D\C_Green>'

You can change the prompt configuration to your liking by editing
ipy_user_conf.py.  

String lists
============

String lists (IPython.utils.text.SList) are handy way to process output
from system commands. They are produced by ``var = !cmd`` syntax.

First, we acquire the output of 'ls -l'::

    [Q:doc/examples]|2> lines = !ls -l
     ==
    ['total 23',
     '-rw-rw-rw- 1 ville None 1163 Sep 30  2006 example-demo.py',
     '-rw-rw-rw- 1 ville None 1927 Sep 30  2006 example-embed-short.py',
     '-rwxrwxrwx 1 ville None 4606 Sep  1 17:15 example-embed.py',
     '-rwxrwxrwx 1 ville None 1017 Sep 30  2006 example-gnuplot.py',
     '-rwxrwxrwx 1 ville None  339 Jun 11 18:01 extension.py',
     '-rwxrwxrwx 1 ville None  113 Dec 20  2006 seteditor.py',
     '-rwxrwxrwx 1 ville None  245 Dec 12  2006 seteditor.pyc']

Now, let's take a look at the contents of 'lines' (the first number is
the list element number)::

    [Q:doc/examples]|3> lines
                    <3> SList (.p, .n, .l, .s, .grep(), .fields() available). Value:

    0: total 23
    1: -rw-rw-rw- 1 ville None 1163 Sep 30  2006 example-demo.py
    2: -rw-rw-rw- 1 ville None 1927 Sep 30  2006 example-embed-short.py
    3: -rwxrwxrwx 1 ville None 4606 Sep  1 17:15 example-embed.py
    4: -rwxrwxrwx 1 ville None 1017 Sep 30  2006 example-gnuplot.py
    5: -rwxrwxrwx 1 ville None  339 Jun 11 18:01 extension.py
    6: -rwxrwxrwx 1 ville None  113 Dec 20  2006 seteditor.py
    7: -rwxrwxrwx 1 ville None  245 Dec 12  2006 seteditor.pyc

Now, let's filter out the 'embed' lines::

    [Q:doc/examples]|4> l2 = lines.grep('embed',prune=1)
    [Q:doc/examples]|5> l2
                    <5> SList (.p, .n, .l, .s, .grep(), .fields() available). Value:

    0: total 23
    1: -rw-rw-rw- 1 ville None 1163 Sep 30  2006 example-demo.py
    2: -rwxrwxrwx 1 ville None 1017 Sep 30  2006 example-gnuplot.py
    3: -rwxrwxrwx 1 ville None  339 Jun 11 18:01 extension.py
    4: -rwxrwxrwx 1 ville None  113 Dec 20  2006 seteditor.py
    5: -rwxrwxrwx 1 ville None  245 Dec 12  2006 seteditor.pyc

Now, we want strings having just file names and permissions::

    [Q:doc/examples]|6> l2.fields(8,0)
                    <6> SList (.p, .n, .l, .s, .grep(), .fields() available). Value:

    0: total
    1: example-demo.py -rw-rw-rw-
    2: example-gnuplot.py -rwxrwxrwx
    3: extension.py -rwxrwxrwx
    4: seteditor.py -rwxrwxrwx
    5: seteditor.pyc -rwxrwxrwx

Note how the line with 'total' does not raise IndexError.

If you want to split these (yielding lists), call fields() without
arguments::

    [Q:doc/examples]|7> _.fields()
                    <7>
    [['total'],
     ['example-demo.py', '-rw-rw-rw-'],
     ['example-gnuplot.py', '-rwxrwxrwx'],
     ['extension.py', '-rwxrwxrwx'],
     ['seteditor.py', '-rwxrwxrwx'],
     ['seteditor.pyc', '-rwxrwxrwx']]

If you want to pass these separated with spaces to a command (typical
for lists if files), use the .s property::


    [Q:doc/examples]|13> files = l2.fields(8).s
    [Q:doc/examples]|14> files
                    <14> 'example-demo.py example-gnuplot.py extension.py seteditor.py seteditor.pyc'
    [Q:doc/examples]|15> ls $files
    example-demo.py  example-gnuplot.py  extension.py  seteditor.py  seteditor.pyc

SLists are inherited from normal python lists, so every list method is
available::

    [Q:doc/examples]|21> lines.append('hey')


Real world example: remove all files outside version control
============================================================

First, capture output of "hg status"::

    [Q:/ipython]|28> out = !hg status
     ==
    ['M IPython\\extensions\\ipy_kitcfg.py',
     'M IPython\\extensions\\ipy_rehashdir.py',
    ...
     '? build\\lib\\IPython\\Debugger.py',
     '? build\\lib\\IPython\\extensions\\InterpreterExec.py',
     '? build\\lib\\IPython\\extensions\\InterpreterPasteInput.py',
    ...

(lines starting with ? are not under version control).

::

    [Q:/ipython]|35> junk = out.grep(r'^\?').fields(1)
    [Q:/ipython]|36> junk
                <36> SList (.p, .n, .l, .s, .grep(), .fields() availab
    ...
    10: build\bdist.win32\winexe\temp\_ctypes.py
    11: build\bdist.win32\winexe\temp\_hashlib.py
    12: build\bdist.win32\winexe\temp\_socket.py

Now we can just remove these files by doing 'rm $junk.s'. 

The .s, .n, .p properties
=========================

The '.s' property returns one string where lines are separated by
single space (for convenient passing to system commands). The '.n'
property return one string where the lines are separated by '\n'
(i.e. the original output of the function). If the items in string
list are file names, '.p' can be used to get a list of "path" objects
for convenient file manipulation.