Submitting Jobs to the Queue Manager

In order to make it easier to interact with the queue manager on Karst, we are creating a series of graphical user interfaces (GUI's) that collect information from a user and submit specific types of image processing jobs to the IU clusters. These GUI's are created using a toolkit called rappture that is being developed as part of the NanoHub project (based at Purdue University).

The rappture (Rapid APPlication infrastrucTURE) toolkit can create complex GUI's using a flexible description encoded in rather simple extensible markup language (XML). The intent of rappture as originally developed by the Nanohub group was to have the GUI spawn a computer simulation of various nano-science related problems and data (e.g., carbon nanotube structures, band gaps, various nano-electronics, etc.) and to return results to the GUI very quickly. The overall design plan for rappture was that the GUI should be able to launch any sort of computer program that could be run from a unix/linux command line, and part of the internal design of rappture is that it can easily interact with both programming languages such as Fortran, C and C++ and with scripting languages such as Perl, Python, Matlab, Octave and Tcl.

The "use model" for rappture is that the user creates a specific application GUI by running the rappture program in a directory that contains a particular XML-based GUI descriptor. The user supplies information to the GUI using a variety of inputs (text boxes, drop-down selections, check boxes, etc.) and starts a simulation by pressing a large button labeled "Simulate." When the job finishes, output of many types is returned to the GUI and displayed in various ways (log files, 2d and 3d graphs, images, structure viewers, etc.). Demonstrations of what running a rappture GUI looks like can be found on the NanoHub website in their Animations section.

We have co-opted the overall design of rappture so that the "Simulate" button in our rappture GUI's starts a Python script that both writes out a job script (containing information for the queue manager on Karst and also the actual image processing commands) and then submits the job script to the queue. The information returned to the GUI is simply the success (or failure) of submitting a job to the queue manager.

At present, the overall design of our GUI's is that the GUI supplies the user with some information about the GUI itself (including contact information from the author) and about the job that will be sent to the queue, and then the actual user input is divided into at least two tabbed "pages" where the first page collects information for the queue manager and the second page collects information about the actual image processing job. For example, the the GUI that launches an EMAN1 refine command to the cluster has authorship information on the right-hand side, some general information about the refine command itself at the top of the left-hand side and two tabbed pages below this general information. The first page (shown to the left) collects information specific to the queue manager: an e-mail address the queue manager users to contact the user when the job starts and finishes, computational resources requested (number of nodes, number of cores/node and "wall clock" time) and the specific queue to use. There is also a place to include any other instructions for the queue manager ("Other PBS directives") and a place to give the script a user-friendly name.

The second page (shown to the right) collects information for the refine command itself. For example, because of the way that EMAN1 implements parallelization, not only must we tell Karst's queue manager the number of nodes and cores/node we want, but we must also include in the refine command itself the total number of cores that will be used ("Number of processors"). There are then fields for each of the obligatory and commonly used command line arguments for the refine command and a place to supply any other command line arguments for the refine command ("Other arguments").

At the very bottom of second page is a check box labeled "Help." The authors of EMAN1 have made it easy to obtain help for the many programs EMAN1 refine - help GUI(simply type the program name followed by "-help" on the command line) and we have incorporated this into our GUI: if the user checks this box and clicks the "Simulate" button, the usual EMAN1 help GUI appears (as seen at the left) and can be consulted while filling in the information needed for the refine command.

When the help box is not checked and the "Simulate" button is clicked, the GUI launches a Python script that writes out a file containing all the input information (in proper format for both the queue manager and for the refine command itself), checks that the proper input files for the refine command actually exist and sends the command file to the queue manager. If there are missing input files, the Python script returns an error (and an explanation of the missing files) and allows the user either to exit the GUI or to supply the missing files and try to submit the job again.

When the job has been successfully submitted to the queue manager, the GUI replaces the information about authorship on its right-hand side with a display of a "log file" containing information about the job. At the top of the log file is the actual command that was used to submit the job to the queue:

qsub ref_1274139913.2h

This is followed by a listing of all the user's jobs that are currently in the queue (including this newest one).

At this point, the user simply needs to click the "File" button in the upper left corner of the window and exit the GUI.

The major advantage of using the rappture toolkit to build our GUI's is that the toolkit does all the hard work: the GUI shown here was generated in much less than an hour using only very simple XML. The toolkit has built-in functions that supply the default values shown here, that magically create the text boxes and check boxes, that allow the author to specify that certain inputs must be (for example) integers between 1 and 16, etc.

There is also a way to supply "hints" to the user about each of the input fields:

Additional resources on Karst and Rappture can be found below:

• Using the Rappture Graphical User Interfaces
• Information on Using Specific Rappture Graphical User Interfaces on Karst