mailpipe: use `messages.sort(...)`, not `m = sorted(m, ...)`.

[pygrader.git] / README

``pygrader`` is a directory-based grade database for grading course
assignments.  Besides tracking grades locally, you can also use it to
automatically mail grades to students and professors associated with
the course.  For secure communication, PGP_ can be used to sign and/or
encrypt any of these emails.

Installation
============

Packages
--------

Gentoo
~~~~~~

I've packaged ``pygrader`` for Gentoo_.  You need layman_ and
my `wtk overlay`_.  Install with::

  # emerge -av app-portage/layman
  # layman --add wtk
  # emerge -av dev-python/pygrader

Dependencies
------------

``pygrader`` is a simple package.  The only external dependency
outside the Python 3 standard library is my `pgp-mime`_ package.

If you are developing ``pygrader``, you can use `update-copyright`_ to
keep the copyright blurbs up to date.

Installing by hand
------------------

``pygrader`` is available as a Git_ repository::

  $ git clone git://tremily.us/pygrader.git

See the homepage_ for details.  To install the checkout, run the
standard::

  $ python setup.py install

Usage
=====

Pygrader will help keep you organized in a course where the students
submit homework via email, or the homework submissions are otherwise
digital (i.e. scanned in after submission).  There is currently no
support for multiple graders, although I will likely add this in the
future.  In the following sections, I'll walk you through
administering the homework for the ``test`` course.

All of the processing involves using the ``pg.py`` command.  Run::

  $ pg.py --help

for details.

Sending email
-------------

Pygrader receives submissions and assigns grades via email.  In order
to send email, it needs to connect to an SMTP_ server.  See the
pgp-mime documentation for details on configuring you SMTP connection.
You can test your SMTP configuration by sending yourself a test
message::

  $ pg.py -VVV smtp -a rincewind@uu.edu -t rincewind@uu.edu

Defining the course
-------------------

Once you've got email submission working, you need to configure the
course you'll be grading.  Each course lives in its own directory, and
the basic setup looks like the ``test`` example distributed with
pygrader.  The file that you need to get started is the config file in
the course directory::

  $ cat test/course.conf
  [course]
  name: Physics 101
  assignments: Attendance 1, Attendance 2, Attendance 3, Attendance 4,
    Attendance 5, Attendance 6, Attendance 7, Attendance 8, Attendance 9,
    Assignment 1, Assignment 2, Exam 1, Exam 2
  robot: Robot101
  professors: Gandalf
  assistants: Sauron
  students: Bilbo Baggins, Frodo Baggins, Aragorn

  [Attendance 1]
  points: 1
  weight: 0.1/9
  due: 2011-10-03

  [Attendance 2]
  points: 1
  weight: 0.1/9
  due: 2011-10-04

  …

  [Assignment 1]
  points: 10
  weight: 0.4/2
  due: 2011-10-10
  submittable: yes

  …

  [Exam 2]
  points: 10
  weight: 0.4/2
  due: 2011-10-17

  [Robot101]
  nickname: phys-101 robot
  emails: phys101@tower.edu
  pgp-key: 4332B6E3

  [Gandalf]
  nickname: G-Man
  emails: g@grey.edu
  pgp-key: 4332B6E3

  [Sauron]
  emails: eye@tower.edu

  [Bilbo Baggins]
  nickname: Bill
  emails: bb@shire.org, bb@greyhavens.net

  …

The format is a bit wordy, but it is also explicit and easily
extensible.  The time it takes to construct this configuration file
should be a small portion of the time you will spend grading
submissions.

If a person has the ``pgp-key`` option set, that key will be used to
encrypt messages to that person and sign messages from that person
with PGP_.  It will also be used to authenticate ownership of incoming
emails.  You'll need to have GnuPG_ on your local host for this to
work, and the user running pygrader should have the associated keys in
their keychain.

The ``course.robot`` option defines a dummy person used to sign
automatically generated emails (e.g. responses to mailpipe-processed
submissions).

The ``submittable`` option marks assignments that accept direct
submission from students (e.g. homeworks).  You probably don't want to
set this option for attendance, since it would allow students to mark
themselves as having attended a class.  ``submittable`` default to
``False``.

Processing submissions
----------------------

As the due date approaches, student submissions will start arriving in
your inbox.  Use ``pg.py``'s ``mailpipe`` command to sort them into
directories (using the ``pygrader.handler.submission`` handler).  This
will also extract any files that were attached to the emails and place
them in that persons assignment directory::

  $ pg.py -d test mailpipe -m maildir -i ~/.maildir -o ./mail-old

Use ``pg.py``'s ``todo`` command to check for ungraded submissions::

  $ pg.py -d test todo mail grade

Then create ``grade`` files using your favorite editor.  The first
line of the grade file should be the student's grade for that
assigment, expressed in a syntax that Python's ``float()`` understands
(``1``, ``95``, ``2.5``, ``6.022e23``, etc.).  If you wish, you may
add additional comment lines after the grade line, offering
suggestions for improvement, etc.  This comment (if present) will be
mailed to the student along with the grade itself.  There are a number
of example grade files in the ``test`` directory in ``pygrader``'s Git
source.

To see how everyone's doing, you can print a table of grades with
``pg.py``'s ``tabulate`` command::

  $ pg.py -d test tabulate -s

When you want to notify students of their grades, you can send them
all out with ``pg.py``'s ``email`` command::

  $ pg.py -d test email assignment 'Exam 1'

Mailpipe details
~~~~~~~~~~~~~~~~

Mailpipe is the most complicated part of ``pygrader``, and the place
where things are most likely to get sticky.  Since there are several

If you get tired of filtering your inbox by hand using ``pg.py
mailpipe``, you can (depending on how your mail delivery is setup) use
procmail_ to automatically run ``mailpipe`` automatically on incoming
email.  There is an example ``.procmailrc`` in the
``pygrader.mailpipe.mailpipe`` docstring that runs ``mailpipe``
whenever incoming emails have ``[phys160:submit]`` in their subject
somewhere.

The use of ``[TARGET]`` tags in the email subject allows users to
unambiguously specify the purpose of their email.  Currently supported
targets include (see the ``handlers`` argument to
``pygrader.mailpipe``):

``submit``
  student assignment submission.  The remainder of the email subject
  should include the case insensitive name of the assignment being
  submitted (see ``pygrader.handler.submission._match_assignment``).
  An example subject would be::

    [submit] assignment 1

``get``
  request information from the grade database.  For students, the
  remainder of the email subject is irrelevant.  Grades and comments
  for all graded assignments are returned in a single email.  An
  example subject would be::

    [get] my grades

  Professors and TAs may request either a table of all grades for the
  course (à la ``tabulate``), the full grades for a particular
  student, or a particular student's submission for a particular
  assignment.  Example subjects are (respectively):

    [get] don't match any student names
    [get] Bilbo Baggins
    [get] Bilbo Baggins Assignment 1

To allow you to easily sort the email, you can also prefix the target
with additional information (see
``pygrader.mailpipe._get_message_target``).  For example, if you were
running several courses from the same email account, you'd want a way
for users to specify which course they were interacting with so you
could filter appropriately in your procmail rules.  Everything in the
subject tag before an optional semicolon is ignored by ``mailpipe``,
so the following subjects will be handled identically::

  [submit] assignment 1
  [phys101:submit] assignment 1
  [phys101:section2:submit] assignment 1

Testing
=======

Run the internal unit tests using nose_::

  $ nosetests --with-doctest --doctest-tests pygrader

If a Python-3-version of ``nosetests`` is not the default on your
system, you may need to try something like::

  $ nosetests-3.2 --with-doctest --doctest-tests pygrader

Licence
=======

This project is distributed under the `GNU General Public License
Version 3`_ or greater.

Author
======

W. Trevor King
wking@tremily.us

Related work
============

For a similar project, see `Alex Heitzmann's pygrade`_, which keeps
the grade history in a single log file and provides more support for
using graphical interfaces.


.. _PGP: http://en.wikipedia.org/wiki/Pretty_Good_Privacy
.. _Gentoo: http://www.gentoo.org/
.. _layman: http://layman.sourceforge.net/
.. _wtk overlay: http://blog.tremily.us/posts/Gentoo_overlay/
.. _pgp-mime: http://blog.tremily.us/posts/pgp-mime/
.. _update-copyright: http://blog.tremily.us/posts/update-copyright/
.. _Git: http://git-scm.com/
.. _homepage: http://blog.tremily.us/posts/pygrader/
.. _SMTP: http://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol
.. _GnuPG: http://www.gnupg.org/
.. _procmail: http://www.procmail.org/
.. _nose: http://readthedocs.org/docs/nose/en/latest/
.. _GNU General Public License Version 3: http://www.gnu.org/licenses/gpl.html
.. _Alex Heitzmann's pygrade: http://code.google.com/p/pygrade/