Using Specific Rappture Graphical User Interfaces on Karst
This page describes a number of scripts on Karst that can be used to submit image processing jobs to the queue. Most of these operate through rappture-based graphical user interfaces, but in at least one case, a script is produced that must be sent to the queue using the normal "qsub" command:
$ qsub script_name
For the most part, use of these scripts requires no detailed knowledge of how to deal with the queue manager on Karst or of the exact command syntax necessary to control these jobs. Limited help is available either while running these scripts or from the rappture GUI created after running one of the scripts, but it is also assumed that users have familiarity with jobs that are created and sent to the queue.
Note: All of the rappture GUI's are fully functional but they are not finished products. For example, the HTML help file associated with each of these scripts (not the mouse-over hints, but the file used to display information in the upper left corner of each GUI) may simply be a "place-holder" file instead of something useful. Also, we have left in place in all the GUI's the mechanism for obtaining help through the GUI even in cases where the program the GUI runs does not support such an option. All these issues will be addressed in the future (especially when users bring these issues to the attention of the EMC staff!).
In the following accordion, the folds are various commands. Click on the command in the accordion for a brief description of what can be accomplished after running these commands.
In cases where a job can simply be sent to the queue with no regard for things like how parallelization is implemented, the rappture GUI created after this command has been run can be used to allocate computer resources and submit a job to the queue.
Because any command can be sent to the queue from the rappture GUI that this command creates, the GUI does not perform any sort of error checking for the presence of any required input files. It will be up to the user to make certain that the desired job runs (and that no damage to existing data occurs should the requested job fail to start properly).
Jobs sent to the queue after using this command are ones that can be executed using a single command line command. This does not include the EMAN1 commands that request more than a single processor because of the way EMAN1 has implemented parallelization.
This script starts auto3dem's automagic option to create a starting icosahedral model using a subset of randomly chosen images. Running this command generates a command file called "RMC_run" that must be submitted to the queue using qsub:
$ qsub RMC_run
The setup_rmc script on Karst is a modification of the one released by Tim Baker's lab at UCSD, where we have modifed the original file in order to make it interact properly with the queue manager on Karst. Typing "setup_rmc" on the command line prints some help-style information to the terminal window (including default values for parameters the user might desire to change) and stops at a point where the user can exit without doing anything. If the user continues from here, the RMC_run script is created and can be submitted to the queueing system.
Use this script to set up a rappture GUI that can send any auto3dem command sequence (other than RMC option described above) to the queue. The GUI controls the requested computational resources and also checks that the chosen auto3dem parameter file is present. It is the user's responsibility to control what auto3dem does by changing the contents of such parameter files.
Keep in mind that while auto3dem is running, it constantly generates what the package calls "restart" files that can be used to resume a job should the cluster stop for some reason (or should the job time-out before its completion). Such restart files are simply parameter files describing the "remaining job" after various steps have been completed, and as such, they can be used as the parameter files that the rappture GUI expects.
This script sets up a rappture GUI that in turn creates a script to run Giovanne Cardone's et-fsceo commands (both the setup and the actual resolution calculation). The GUI checks for the existence of an installation of IMOD and for the imod tilt.com script that is used to generate the proper input volumes. The et-fsceo_setup program uses IMOD and this tilt.com script to create volumes made from "even" and "odd" numbered tilt angles and then et-fsceo itself to calculate a Fourier Shell Correlation curve based on the two volumes. The setup step is the most computationally intensive since it makes two full tomographic reconstructions. The et-fsceo script that is submitted to Karst ends by generating some plot files using gnuplot.
This script sets up a rappture GUI that creates a script to run Giovanne Cardone's et-nloo commands (both the setup and the actual resolution calculation). The GUI checks for the existence of an installation of IMOD and the imod tilt.com script that is used to generate the proper input data. The et-nloo_setup program uses IMOD and this tilt.com script to make all the reprojection images required by the "noise-compensated leave-one-out" (nloo) approach. This step can be quite slow since it generates a full reconstruction after leaving out each of the individual tilted images. The et-nloo script that is submitted to quarry ends by generating some plot files using gnuplot.
This script can be used to set up a rappture GUI that will send any EMAN1 command to Karst's queue. This script differs from the setup_genericCommand.sh script in that this script explicitly handles EMAN1's implementation of parallelization.
Similarly to that script, because any EMAN1 command can be submitted to the queue from the rappture GUI that this command creates, it is not possible for the GUI to perform any sort of error checking for the presence of appropriate input files. It will again be up to the user to ensure that the requested job actually runs (and that no damage to existing data can occur should the requested job fail to start properly).
The EMAN1 command that calculates resolution using the FSC approach is called eotest (for even/odd test - the input images are split into even- and odd-numbered groups, two reconstructions are generated and then compared using the FSC). This script sets up a rappture GUI to submit eotest to Karst's queue. The GUI is designed to check for the presence of the necessary input files to perform the FSC calculation. Note that this is not the so-called gold standard FSC but rather that the set of images is split into two parts after an alignment to a single reference volume and the two resulting (non-independent) reconstructions are compared.
This script sets up a rappture GUI that submits the make3d command to Karst's queue. The make3d command generates a 3d reconstruction from a set of existing class averages, and the GUI will check for the presence of the required input files.
Note: This command has not been parallelized and the GUI cannot allocate multiple processors to jobs it sends to the queue.
This is the least useful of the scripts that set up rappture-based GUI's (but was useful for benchmarking various computational aspects of how EMAN1 behaves on Karst). This scripts sets up a GUI that will submit the project3d command to Karst's queue and checks that the required 3d volume exists.
The heart of EMAN1 (and the most computationally intensive command) is refine, which iteratively calculates reference projections, aligns all the input images to them, calculates class averages from the aligned images and generates 3d reconstructions from the class averages. This script sets up the rappture GUI that will submit a refine job to Karst's queue and checks that the required input files exist before submitting a job to the queue. This GUI prompts the user for only a very small subset of the 50+ options available to the refine command (but does provide a way for all possible options to be included with the command, provided that the user knows the exact syntax to use for any further options).
Note: If additional options to refine are commonly used and should be included in this GUI, please contact the EMC staff.
