[hooke.git] / doc / tutorial.txt

********
Tutorial
********

`A short video showing Hooke in action`_! (courtesy of Fabrizio
Benedetti, EPFL, Lausanne)

.. _A short video showing Hooke in action:
  https://documents.epfl.ch/users/f/fb/fbenedet/www/hooke_short_demostration.ogv

.. toctree::
   :maxdepth: 2

   gui

Introduction
============

This tutorial will focus on the command-line interface as the most
powerful, and leave the :doc:`gui` interface to another document.

.. _command-line: `Command-line interface`_

Installation
============

See :doc:`install` for details on downloading and installing Hooke.


Command-line interface
======================

Running the hooke shell
-----------------------

Hooke has a set of commands that depend on the loaded
:class:`hooke.plugin.Plugin`\s.  To access these commands, you'll need
to run the Hooke shell.::

    $ hooke

If you are running hooke from the source directory (see
:doc:`install`), the equivalent command is::

    $ python bin/hk.py

You may need to give the full path for Python on Windows systems, and
also check that the current working directory (`.`) is in your
`PYTHONPATH`.  See :manpage:`python(1)` for details.

As Hooke launches, you should see something like the following in your
terminal::

    Hooke version 0.9.0.devel (Kenzo)

    Copyright (C) 2006-2010 A. Seeholzer, Alberto Gomez-Casado, Allen
    Chen, Fabrizio Benedetti, Francesco Musiani, Marco Brucale, Massimo
    Sandal, Pancaldi Paolo, Richard Naud, Rolf Schmidt, W. Trevor King

    Hooke comes with ABSOLUTELY NO WARRANTY and is licensed under the GNU
    Lesser General Public License.  For details, run `license`.
    ----
    hooke>

The final line, ``hooke>``, is the Hooke prompt.  It allows you to
enter commands to interact with the interpreter.

Help
----

All commands have help text explaining their purpose and usage.  The
text is stored in the code itself, and therefore more likely to be up
to date than this tutorial.  You can get a list of commands and topics
with::

    hooke> help

Or see specific help on ``TOPIC`` with::

    hooke> help TOPIC

for example::

    hooke> help load_playlist

will give help on the ``load_playlist`` command.

Leaving Hooke
-------------

When you're done with an interactive Hooke session, you can close the
session with ``exit`` or its aliases ``quit`` and ``EOF`` (``EOF`` is
the end of the stdin stream, which is Ctrl-d in many shells).

Creating a playlist
-------------------

To start analyzing your curves, you first have to build a playlist.
The playlist is just an index of the force curve files you want to
analyze.  Imagine it as a music playlist (that’s why it is called a
playlist), but with data files instead of audio files.

Suppose you have 100 PicoForce curve files in your curves directory,
starting from :file:`mycurve.000` and ending in :file:`mycurve.100`
and you want to analyze them all.

You then can ``cd`` (change directory) to the directory::

    hooke> cd --path c:\curves

Type ``pwd`` (print working directory) to check the directory is correct.::

    hooke> pwd
    c:\curves

You can list the files in the directory using ``ls`` or ``dir``
(they’re synonyms).::

    hooke> ls
    mycurve.000
    mycurve.001
    ...

Now you are ready to generate the playlist.  First, create a blank playlist::

    hooke> new_playlist --output_playlist mylist

Ensure that the new playlist is active::

    hooke> jump_to_playlist -- -1
    hooke> get_playlist
    <FilePlaylist mylist>

The ``--`` in the ``jump_to_playlist`` command lets
``jump_to_playlist`` know that ``-1`` is an argument and not an
option.  Using the bare ``--`` is a POSIX specification [#POSIX]_
supported by the `optparse module`_.  You don't need to jump if
the new playlist is your only loaded playlist.

.. _optparse module:
  http://docs.python.org/library/optparse.html#callback-example-6-variable-arguments

.. [#POSIX] `Guideline 10 of POSIX:2008's section 12.2 <http://www.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html#tag_12_02>`_ states:

    "The first ``--`` argument that is not an option-argument should be
    accepted as a delimiter indicating the end of options. Any
    following arguments should be treated as operands, even if they
    begin with the ``-`` character."

Then glob your curves onto the new list::

    hooke> glob_curves_to_playlist mycurve.*

You can also be more specific with wildcards.  For example::

    hooke> glob_curve_to_playlist mycurve.05*

will take only curves from :file:`mycurve.050` to :file:`mycurve.059`.

Note that by using ``glob_curves_to_playlist`` you just generate the
playlist in the local session. To save your playlist to a file for
future reuse, type::

    hooke> save_playlist --output mylist

In this example, the list will be saved in the file
:file:`mylist.hkp`.  Hooke will add the extension ``.hkp`` (Hooke
playlist) to the playlist if you forget to.  The ``.hkp`` file is an
XML file you can read and edit with any text editor (i.e. Wordpad), if
needed.  If you want to load it, simply issue ``load_playlist
mylist.hkp`` or ``load_playlist mylist``, Hooke will add ``.hkp`` if
necessary.

If, generating the playlist, you are including by chance a non-force
curve file that Hooke cannot open, Hooke will log a warning and
continue on.

Navigating the playlist
-----------------------

Now you can navigate through your playlist using the commands
``next_curve`` and ``previous_curve``. You don’t need to type
``next_curve`` every time to run along a list of curves.  You can
navigate through the command history by using the up and down arrows,
or auto-complete partial commands with TAB.  From the last curve of
your playlist, ``next_curve`` will wrap around to the first curve.
Analogously, issuing ``previous_curve`` at the first curve will jump
to the last.

You can also jump to a given curve::

    hooke> jump_to_curve 14

will jump to the 14th curve in the zero-indexed playlist.

.. todo:: ``jump_to_curve <PATH>``, where the path can be either an
  absolute path or a path relative to the directory holding the
  playlist file.

Replace ``curve`` with ``playlist`` in the above commands to navigate
around through the list of loaded playlists.

Because the playlist name is usually saved in the playlist file
itself, there is a ``name_playlist`` command that allows you to rename
playlists on the fly.

    hooke> name_playlist 'my old playlist'

Taking notes
------------

You can take notes about the curves you are looking at.  Just type
``set_note`` followed by the text you want to attach to that curve.
Hooke will save the text in your current playlist and in an external
log file.

.. todo:: No external file yet.  Is this important?

The output will look like this::

    Notes taken at Sun Sep 17 20:42:07 2006
    /home/cyclopia/work/tris/20060620a.041 | This is a note
    /home/cyclopia/work/tris/20060620a.207 | This is another note
    /home/cyclopia/work/tris/20060620a.286 | This is a third one

The log file name can be configured (:doc:`config`), but it defaults
to :file:`hooke.log`.

Usually curves you annotated are useful later.  You can create a
playlist for only annotated curves with

    hooke> note_filter_playlist --output_playlist nice_list

will create sub-playlist `nice_list`.  Remember to save the new list
if you like it.

If you change your mind about a note, you can remove it by setting a
blank note string with ``set_note ''``.

Exporting curves
----------------

You can export Hooke curves as images and as text columns.  To export
as images or text, use the ``export_block`` command.  Supported
formats are PNG (Portable Network Graphic, raster) and EPS
(Encapsulated Postscript, vector).  The export format is determined by
the filename extension, so ``export_block --output foo.png``,
``export_block --output foo.eps``, and ``export_block --output
foo.txt`` will save PNG, EPS, and TAB-delimited text files
respectively.

.. todo:: Currently no PNG or EPS output, use the GUI and the plot
  panel's toolbar for non-text exports.

.. todo:: Multiple cycles in exported data?  Solution: blank lines for
  "breaks", add option to extract specific sections using Python's
  slice notation.

If you don't want the entire block, try the ``cut`` command.

Analysis
--------

The commands we have covered so far allow basic bookkeeping.  The
point of Hooke, though, is to allow you to easily analyze force
spectroscopy data.  We cover those analysis commands in this section.

Measuring distances and forces
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To measure the distance between points, use the ``delta`` command.
For example,::

    hooke> delta 300 500

will measure the distance between the 300th point and the 500th point.
One difficulty with the command line interface is that is difficult
to know which points you're interested without seeing the plot.  The
two ways around this are:

1) Export the block (with ``export_block``), and graph the exported
   file with a program of your choice (e.g. Gnuplot_).  Use the
   resulting graph to determine the indices of the points you are
   interested in.
2) Run Hooke's GUI instead of the command line when you need to make
   manual measurements.  See :doc:`gui` for details.

.. _Gnuplot: http://gnuplot.sourceforge.net/

Worm like chain and freely jointed chain fitting
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Polymer model fitting is a complicated beast.  To correctly fit your
chosen model (WLC, FJC, etc.), you need to execute a multi-step
analysis.  Hooke provides a flexible chain of curve analyisis commands
that create new data columns (e.g. `deflection (N)`) or store
information in a curve's `info` dictionary (e.g. `flat filter peaks`).
You can, if necessary, adjust the names of input and output columns
and `info` values to combine the available commands in new and useful
ways.::

    hooke> zero_surface_contact_point --block retract
    hooke> flat_filter_peaks --block retract --min_points 1
    hooke> zero_surface_contact_point --block retract
    ...        --ignore_after_last_peak_info_name 'flat filter peaks'
    hooke> convert_distance_to_force --block retract
    ...        --deflection_column 'surface deflection (m)'
    hooke> remove_cantilever_from_extension --block retract
    hooke> flat_peaks_to_polymer_peaks --block retract
    hooke> polymer_fit_peaks --block retract

This stores the fit parameters in the block's
:attr:`~hooke.curve.Data.info` dictionary (see each command's `Help`_
for details).  To access the parameters, save the playlist,::

    hooke> save_playlist --output mylist.hkp

load it with PyYAML_::

    $ python
    >>> import yaml
    >>> import hooke.util.yaml
    >>> mylist = yaml.load(open('mylist.hkp', 'r'))

navigate to your curve,::

    >>> mylist.jump(1234)

and extract your parameter.::

    >>> mylist.current().data[1].info['polymer peak 0']['contour length (m)']
    2.124...e-08

You can also get a list of all available parameters::

    >>> mylist.current().data[1].info['polymer peak 0'].keys()
    ['model', 'contour length (m)', 'temperature (K)', 'fit',
     'persistence length (m)']

or just pprint_ the whole thing::

    >>> from pprint import pprint
    >>> pprint(mylist.current().data[1].info['polymer peak 0'])
    {'contour length (m)': 2.1248220207858103e-08,
     'fit': {'active fitted parameters': 3.7024307200788127,
             'active parameters': 3.7024307200788127,
             'convergence flag': 1,
             'covariance matrix': None,
             'data scale factor': 5.2716380732198972e-10,
             'fitted parameters': -2.5663294148411571,
             'info': {'fjac': None,
                      'fvec': None,
                      'ipvt': None,
                      'nfev': 16,
                      'qtf': None},
             'initial parameters': [-0.69314718055994529],
             'message': 'Both actual and predicted relative reductions in the sum of squares\n  are at most 0.000000',
             'rescaled': True,
             'scale': None},
     'model': u'WLC',
     'persistence length (m)': 4.0000000000000001e-10,
     'temperature (K)': 301.0}

.. _PyYAML: http://pyyaml.org/
.. _pprint: http://docs.python.org/library/pprint.html

.. todo:: UI access to block (and curve?) info dicts.  Someone should
   make a tree-based object browser ;).

Command stacks
~~~~~~~~~~~~~~

Since you are likely to apply similar analysis to several curves,
Hooke provides :mod:`command stacks <hooke.command_stack>` for
bundling groups of commands.::

    hooke> start_command_capture
    hooke> zero_surface_contact_point --block retract
    hooke> flat_filter_peaks --block retract --min_points 1
    ...
    hooke> stop_command_capture

You can check the state of the command stack with
``get_command_stack`` and the state of capture with
``get_command_capture_state``.  If you make mistakes, you can pop
commands from the stack with ``pop_command_from_stack``.  If you stop
capturing a command stack (e.g. to test a complicated command before
continuing), you can continue adding to the same stack with
``restart_command_capture.``

To execute a command stack, run::

    hooke> execute_command_stack

To execute a command stack on every curve in a playlist, run::

    hooke> apply_command_stack_to_playlist

If you decide that there are commands in a curve's stack that you
don't want, you can clear the stack with::

    hooke> clear_curve_command_stack

You can also save command stacks to disk (and reload them later,
potentially in a different Hooke session).::

    hooke> save_command_stack --output my_stack
    hooke> load_command_stack --input my_stack

Multiple curve analysis
~~~~~~~~~~~~~~~~~~~~~~~

You can analyze multiple curves by combining `Worm like chain and
freely jointed chain fitting`_ and `Command stacks`_.::

    hooke> start_command_capture
    hooke> zero_surface_contact_point --block retract
    hooke> flat_filter_peaks --block retract --min_points 1
    hooke> zero_surface_contact_point --block retract
    ...        --ignore_after_last_peak_info_name 'flat filter peaks'
    hooke> convert_distance_to_force --block retract
    ...        --deflection_column 'surface deflection (m)'
    hooke> remove_cantilever_from_extension --block retract
    hooke> flat_peaks_to_polymer_peaks --block retract
    hooke> polymer_fit_peaks --block retract
    hooke> export_block --block retract --output myblock.dat
    hooke> stop_command_capture
    hooke> apply_command_stack_to_playlist

Configuring Hooke
-----------------

You can set environment variables to influence the behaviour of Hooke.
The command to use is ``set_config``.  Use ``get_config`` to read a
particular option and ``print_config`` to display the entire
configuration file.  To save changes, either run ``save_config`` or
start Hooke with the ``--save-config`` option.  See :doc:`config` for
details.