Add Abax post.

[blog.git] / posts / Abax / pbs_queues / utilities.tex

diff --git a/posts/Abax/pbs_queues/utilities.tex b/posts/Abax/pbs_queues/utilities.tex
new file mode 100644 (file)
index 0000000..b18433d
--- /dev/null
+++ b/posts/Abax/pbs_queues/utilities.tex
@@ -0,0 +1,50 @@
+\section{Utilities}
+\label{sec.utilities}
+
+After playing around on the cluster for a while, I've developed some utilities to integrate job submission with the rest of my workflow.
+So far, I have written
+ |dup\_env| for setting up an environment like the one you called qsub from,
+ |qcmd| for submitting a single command as a job,
+ and
+ |qcmds| for submitting lots of one-line jobs and waiting until they complete.
+
+\subsection{dup\_env}
+
+When |qsub| starts your job on a compute node, it sets up your environmental variables for you.
+Unfortunately, it doesn't do this by cloning your old environment, for example your new |PATH| will come from |/etc/bash.bashrc| (I think?).
+Your old |PATH| get's stored in |PBS\_O\_PATH| (see |man qsub|).
+|dup\_env| just loops through your current environment and overrides |qsub|'s decisions and reinstalls your original enviroment.
+There may be, of course, some exceptions, since you don't want to set your |HOST| to the host you called |qsub| from.
+Remember to place |dup\_env| somewhere where your restricted |qsub| path will find it.
+\inputfile{file/dup_env}{\home/bin/dup\_env}
+
+\subsection{qcmd}
+
+I often have scripts with a few big, slow jobs at the beginning, generating some data that I process in the remainder of the script.
+Rather than chew up the processors on the master node, I'd rather bump them out to the compute nodes.
+To make this alteration as painless as possible, I've written |qcmd| which submits the command in your current environment, waits for the job to finish, and outputs the job's |stdout|/|stderr|/exit-status as it's own.
+You can carry on exactly as if the job executed on your local machine.
+The script works by listening at your mailbox for the job completion email from the queue manager, but I talk about all that in the script itself.
+\inputfile{file/qcmd}{\home/script/qcmd}
+
+\subsection{qcmds}
+
+With three processes open for each waiting job, |qcmd| scales pretty poorly.
+For scripts where you need a bit more muscle to handle several tens to thousands of jobs, I wrote |qcmds|.
+|qcmds| is almost identical to |qcmd|, except that where you had one |qcmd| instance per job, now you have one |qcmds| instance per \emph{set} of jobs.
+With a whole bunch of jobs and only one |stdout|, |qcmds| needed a seperate interface.
+You just pipe your endline (`|\bs n|') delimited jobs into |qcmds| |stdin|, and out the |stdout| comes a list of completing jobs, in a tab delimited list for easy processing.
+Run the examples suggested in the script comments to get a feel for how these work.
+\inputfile{file/qcmds}{\home/script/qcmds}
+
+\subsection{qcleanmail}
+
+Both |qcmd| and |qcmds| listen for job-completion emails from the queue manager, and these can fill up your inbox pretty quickly.
+I wrote this little script to run through my mailboxes and delete all PBS-related messages.
+\inputfile{file/qcleanmail}{\home/script/qcleanmail}
+
+The referenced |procmail| filter is:
+\inputfile{file/qcleanmail.proc}{\home/.mailspool/mailfilter/qcleanmail.proc}
+
+In order to avoid filling up my system mailox with PBS junk between calls to |qcleanmail|, I use the following |\$HOME/.procmailrc| filter to deflect PBS messages to |\$HOME/.mailspool/completed|:
+\inputfile{file/_procmailrc}{\home/.procmailrc}