Changes between Initial Version and Version 1 of oldwiki/queue

Timestamp:

Oct 28, 2014, 11:14:56 AM (10 years ago)

Author:

les

Comment:

--

oldwiki/queue

@@ +1 @@
+Queue System
+
+How to submit jobs
+Before you submit a job, you need to decide on a couple of things:
+How long time do you expect your job to be running? Start off with a guess. Once you have run your program a few times, you will learn how long time you need to assign.
+How much memory do you need? Usually you only need modest amounts of memory, less than 4G.
+You should be aware that the more resources you request, the longer it will take before your job will be run. 
+Once you know these, you must create a job batch file, this you will run to get a place in the queue for your specific job. 
+
+
+You can create this batch file using either pico or nano when you are logged into the grid using Terminal or Putty.
+To start either text editing program just simply type the program name at the prompt - $ nano or pico
+
+*** - You can not use any other text editing programs to write your scripts. Use only nano or pico from Terminal or Putty.
+
+The program will start and then you can begin typing the commands for the batch file.
+Note: the control panel at the bottom of the editor, use ctrl+Shift to execute the commands. For example ctrl+Shift+X will exit the program.
+
+Once the file is ready and saved in your directory, submit it by using the command - qsub [name of this file] at the prompt - $ qsub batchfile.job
+
+Here is a link to useful qsub commands - qsub
+
+Below is an example of a batch file for job submission to the Grid: 
+
+Lines beginning with #$ contains directives to the queue system
+#!/bin/bash
+
+#First inform the system of your email addres, and that you want mails sent at the beginning, end and if you job is aborted.
+
+#$ -M make_this_your_em@ail_address
+#$ -m bae
+
+# Second, tell the system that it should execute the job in your current directory
+
+#$ -cwd
+
+# Now two the things you decided earlier
+# Ask for a runtime of 2 hours, 0 minutes and 0 seconds
+
+#$ -l h_rt=2:0:0
+
+# And 1GB of memory
+
+#$ -l h_vmem=1G
+
+# NEW - YOU MUST PUT THIS LINE IN BEFORE YOU CAN RUN A PROGRAM 
+
+module load [program name] - for example: module load blast
+
+# Finally put the commands needed to run the program:
+
+my_program [-option1] [-option2]
+
+For example, a local blast would look like this:
+
+blastall -p blastn -i infile.blast -o outfile.blas -db my_db -e 1e-40
+Put all this into a file, let's call it my_job.job with a text editor and place the file in the directory where you want to run your job. Then submit it to the queueing system with the qsub command:
+
+mguser@my-mgrid:~$ qsub my_job.job
+Take a note of the job ID given to you by qsub (it's a numeric ID). To see what jobs you have in the queue use the command qstat:
+
+mguser@my-mgrid:~$ qstat
+The mails sent to you from the queue system will tell you when your jobs i done, but also if it was terminated by the system if it exceeded its runtime or memory allocation. If you didn't get any mails, and your job does not show up in qstat, use qacct with the job id you got from qsub (you kept a note of it, didn't you?) to check the status:
+
+mguser@my-mgrid:~$ qacct -j <JOBID>
+(replace <JOBID> with the numeric ID). Check the exit_status line, it should be 0, if it is not, the job failed. You should also check contents the files <JOBNAME>.o<JOBID> and <JOBNAME>.e<JOBID> for hints on what has happened.