From: W. Trevor King <wking@drexel.edu>
Date: Wed, 27 Oct 2010 19:59:04 +0000 (-0400)
Subject: Add pysawsim-manager post.
X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=23701851abc088f7d6df6d7413cdc638cbffda39;p=mw2txt.git

Add pysawsim-manager post.
---

diff --git a/posts/pysawsim-manager.mdwn b/posts/pysawsim-manager.mdwn
new file mode 100644
index 0000000..e8ce2cd
--- /dev/null
+++ b/posts/pysawsim-manager.mdwn
@@ -0,0 +1,40 @@
+[[!meta  title="pysawsim.manager: parallelization-framework-agnostic testing"]]
+[[!template id=gitrepo repo=sawsim]]
+
+Over the past week I've been working up a [[Python]] framework for my
+[[sawsim]] simulator, to allow parallel execution on a variety of
+systems.  I think I've got the kinks worked out now, and `sawsim` runs
+(or anything else that can subclass `pysawsim.manager.Job` can now be
+transparently executed in parallel using any of
+
+* Python's [threading][] module (the `thread` manager).
+* Python's [multiprocessing][] module (the `subproc` manager).
+* [[MPI]] (via [mpi4py][], the `mpi` manager).
+* [[PBS]] (via [pbs_python][], the `pbs` manager).
+
+Most of the difficulty is in handling systems where not all of these
+approaches are viable, (e.g. Python 2.5, so no `multiprocessing`
+module, or an SMP host that wants to use `subproc` without installing
+mpi4py or pbs_python).  I also ran up against Python's [issue5155][]
+when I tried to test the `subproc` manager from within a:
+
+    $ nosetests --with-doctest --doctest-tests --processes $N pysawsim
+
+on a Python 2.6.2 system (it was fixed by 2.6.3).
+
+Note that Python's [GIL restricts threads to a single core][GIL].  This
+means that the `thread` manager is basically decorative, but I wrote
+it anyway because it forms the basis of several other managers, and
+it's the only manager that will definitely work on Python 2.5.
+
+[threading]: http://docs.python.org/library/threading.html
+[multiprocessing]: http://docs.python.org/library/multiprocessing.html
+[mpi4py]: http://mpi4py.scipy.org/
+[pbs_python]: https://subtrac.sara.nl/oss/pbs_python
+[issue5155]: http://bugs.python.org/issue5155
+[GIL]: http://docs.python.org/faq/library#can-t-we-get-rid-of-the-global-interpreter-lock
+
+
+[[!tag tags/programming]]
+[[!tag tags/python]]
+[[!tag tags/sawsim]]