{"id":77,"date":"2019-03-06T11:21:53","date_gmt":"2019-03-06T16:21:53","guid":{"rendered":"http:\/\/psycomp.utsc.utoronto.ca\/support\/?page_id=77"},"modified":"2019-06-21T22:02:13","modified_gmt":"2019-06-22T02:02:13","slug":"sge-to-slurm-conversion","status":"publish","type":"page","link":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/resources\/sge-to-slurm-conversion\/","title":{"rendered":"SGE to SLURM conversion"},"content":{"rendered":"

Sun Grid Engine<\/strong> (SGE<\/strong>) and SLURM<\/strong> job scheduler concepts are quite similar. Below is a table of some common SGE commands and their SLURM equivalents.<\/p>\r\n\r\n

Some common commands and flags in SGE and SLURM with their respective equivalents:<\/p>\r\n

<\/p>\r\n
User Commands<\/strong><\/td>\r\n\t\t\tSGE<\/strong><\/td>\r\n\t\t\tSLURM<\/strong><\/td>\r\n\t\t<\/tr>
Interactive login<\/strong><\/td>\r\n\t\t\tqlogin<\/td>\r\n\t\t\t\r\n\t\t\t\t
\r\nsrun --pty bash or srun -p \"partition name\" --time=4:0:0 --pty bash. For a quick dev node, just run \"sdev\"<\/pre>\r\n\t\t\t<\/td>\r\n\t\t<\/tr>
Job submission<\/strong><\/td>\r\n\t\t\tqsub [script_file]<\/td>\r\n\t\t\tsbatch [script_file]<\/td>\r\n\t\t<\/tr>
Job deletion<\/strong><\/td>\r\n\t\t\tqdel [job_id]<\/td>\r\n\t\t\tscancel [job_id]<\/td>\r\n\t\t<\/tr>
Job status by job<\/strong><\/td>\r\n\t\t\tqstat -u \\* [-j job_id]<\/td>\r\n\t\t\tsqueue [job_id]<\/td>\r\n\t\t<\/tr>
Job status by user<\/strong><\/td>\r\n\t\t\tqstat [-u user_name]<\/td>\r\n\t\t\t\r\n\t\t\t\t

squeue -u [user_name]<\/p>\r\n\t\t\t<\/td>\r\n\t\t<\/tr>

Job hold<\/strong><\/td>\r\n\t\t\tqhold [job_id]<\/td>\r\n\t\t\tscontrol hold [job_id]<\/td>\r\n\t\t<\/tr>
Job release<\/strong><\/td>\r\n\t\t\tqrls [job_id]<\/td>\r\n\t\t\tscontrol release [job_id]<\/td>\r\n\t\t<\/tr>
Queue list<\/strong><\/td>\r\n\t\t\tqconf -sql<\/td>\r\n\t\t\tsqueue<\/td>\r\n\t\t<\/tr>
List nodes<\/strong><\/td>\r\n\t\t\tqhost<\/td>\r\n\t\t\tsinfo -N OR scontrol show nodes<\/td>\r\n\t\t<\/tr>
Cluster status<\/strong><\/td>\r\n\t\t\tqhost -q<\/td>\r\n\t\t\tsinfo<\/td>\r\n\t\t<\/tr>
GUI<\/strong><\/td>\r\n\t\t\tqmon<\/td>\r\n\t\t\tsview<\/td>\r\n\t\t<\/tr>
<\/td>\r\n\t\t\t <\/td>\r\n\t\t\t <\/td>\r\n\t\t<\/tr>
Environmental<\/strong><\/td>\r\n\t\t\t <\/td>\r\n\t\t\t <\/td>\r\n\t\t<\/tr>
Job ID<\/strong><\/td>\r\n\t\t\t$JOB_ID<\/td>\r\n\t\t\t$SLURM_JOBID<\/td>\r\n\t\t<\/tr>
Submit directory<\/strong><\/td>\r\n\t\t\t$SGE_O_WORKDIR<\/td>\r\n\t\t\t$SLURM_SUBMIT_DIR<\/td>\r\n\t\t<\/tr>
Submit host<\/strong><\/td>\r\n\t\t\t$SGE_O_HOST<\/td>\r\n\t\t\t$SLURM_SUBMIT_HOST<\/td>\r\n\t\t<\/tr>
Node list<\/strong><\/td>\r\n\t\t\t$PE_HOSTFILE<\/td>\r\n\t\t\t$SLURM_JOB_NODELIST<\/td>\r\n\t\t<\/tr>
Job Array Index<\/strong><\/td>\r\n\t\t\t$SGE_TASK_ID<\/td>\r\n\t\t\t$SLURM_ARRAY_TASK_ID<\/td>\r\n\t\t<\/tr>
<\/td>\r\n\t\t\t <\/td>\r\n\t\t\t <\/td>\r\n\t\t<\/tr>
Job Specification<\/strong><\/td>\r\n\t\t\t <\/td>\r\n\t\t\t <\/td>\r\n\t\t<\/tr>
Script directive<\/strong><\/td>\r\n\t\t\t#$<\/td>\r\n\t\t\t#SBATCH<\/td>\r\n\t\t<\/tr>
queue<\/strong><\/td>\r\n\t\t\t-q [queue]<\/td>\r\n\t\t\t-p [queue]<\/td>\r\n\t\t<\/tr>
count of nodes<\/strong><\/td>\r\n\t\t\tN\/A<\/td>\r\n\t\t\t-N [min[-max]]<\/td>\r\n\t\t<\/tr>
CPU count<\/strong><\/td>\r\n\t\t\t-pe [PE] [count]<\/td>\r\n\t\t\t-n [count]<\/td>\r\n\t\t<\/tr>
Wall clock limit<\/strong><\/td>\r\n\t\t\t-l h_rt=[seconds]<\/td>\r\n\t\t\t-t [min] OR -t [days-hh:mm:ss]<\/td>\r\n\t\t<\/tr>
Standard out file<\/strong><\/td>\r\n\t\t\t-o [file_name]<\/td>\r\n\t\t\t-o [file_name]<\/td>\r\n\t\t<\/tr>
Standard error file<\/strong><\/td>\r\n\t\t\t-e [file_name]<\/td>\r\n\t\t\te [file_name]<\/td>\r\n\t\t<\/tr>
Combine STDOUT & STDERR files <\/strong><\/td>\r\n\t\t\t-j yes<\/td>\r\n\t\t\t(use -o without -e)<\/td>\r\n\t\t<\/tr>
Copy environment<\/strong><\/td>\r\n\t\t\t-V<\/td>\r\n\t\t\t–export=[ALL | NONE | variables]<\/td>\r\n\t\t<\/tr>
Event notification<\/strong><\/td>\r\n\t\t\t-m abe<\/td>\r\n\t\t\t–mail-type=[events]<\/td>\r\n\t\t<\/tr>
send notification email<\/strong><\/td>\r\n\t\t\t-M [address]<\/td>\r\n\t\t\t–mail-user=[address]<\/td>\r\n\t\t<\/tr>
Job name<\/strong><\/td>\r\n\t\t\t-N [name]<\/td>\r\n\t\t\t–job-name=[name]<\/td>\r\n\t\t<\/tr>
Restart job<\/strong><\/td>\r\n\t\t\t-r [yes|no]<\/td>\r\n\t\t\t–requeue OR –no-requeue (NOTE:
\r\n\t\t\t\tconfigurable default)<\/td>\r\n\t\t<\/tr>
Set working directory<\/strong><\/td>\r\n\t\t\t-wd [directory]<\/td>\r\n\t\t\t–workdir=[dir_name]<\/td>\r\n\t\t<\/tr>
Resource sharing<\/strong><\/td>\r\n\t\t\t-l exclusive<\/td>\r\n\t\t\t–exclusive OR–shared<\/td>\r\n\t\t<\/tr>
Memory size<\/strong><\/td>\r\n\t\t\t-l mem_free=[memory][K|M|G]<\/td>\r\n\t\t\t–mem=[mem][M|G|T] OR –mem-per-cpu=
\r\n\t\t\t\t[mem][M|G|T]<\/td>\r\n\t\t<\/tr>
Charge to an account<\/strong><\/td>\r\n\t\t\t-A [account]<\/td>\r\n\t\t\t–account=[account]<\/td>\r\n\t\t<\/tr>
Tasks per node<\/strong><\/td>\r\n\t\t\t(Fixed allocation_rule in PE)<\/td>\r\n\t\t\t–tasks-per-node=[count]<\/td>\r\n\t\t<\/tr>
<\/td>\r\n\t\t\t <\/td>\r\n\t\t\t–cpus-per-task=[count]<\/td>\r\n\t\t<\/tr>
Job dependancy<\/strong><\/td>\r\n\t\t\t-hold_jid [job_id | job_name]<\/td>\r\n\t\t\t–depend=[state:job_id]<\/td>\r\n\t\t<\/tr>
Job project<\/strong><\/td>\r\n\t\t\t-P [name]<\/td>\r\n\t\t\t–wckey=[name]<\/td>\r\n\t\t<\/tr>
Job host preference<\/strong><\/td>\r\n\t\t\t-q [queue]@[node] OR -q
\r\n\t\t\t\t[queue]@@[hostgroup]<\/td>\r\n\t\t\t		–nodelist=[nodes] AND\/OR –exclude=
\r\n\t\t\t\t[nodes]<\/td>\r\n\t\t<\/tr>
Quality of service<\/strong><\/td>\r\n\t\t\t <\/td>\r\n\t\t\t–qos=[name]<\/td>\r\n\t\t<\/tr>
Job arrays<\/strong><\/td>\r\n\t\t\t-t [array_spec]<\/td>\r\n\t\t\t–array=[array_spec] (Slurm version 2.6+)<\/td>\r\n\t\t<\/tr>
Generic Resources<\/strong><\/td>\r\n\t\t\t-l [resource]=[value]<\/td>\r\n\t\t\t–gres=[resource_spec]<\/td>\r\n\t\t<\/tr>
Lincenses<\/strong><\/td>\r\n\t\t\t-l [license]=[count]<\/td>\r\n\t\t\t–licenses=[license_spec]<\/td>\r\n\t\t<\/tr>
Begin Time<\/strong><\/td>\r\n\t\t\t-a [YYMMDDhhmm]<\/td>\r\n\t\t\t–begin=YYYY-MM-DD[THH:MM[:SS]]<\/td>\r\n\t\t<\/tr><\/tbody><\/table>

<\/p>\r\n
SGE<\/strong><\/th>\r\n\t\t\tSLURM<\/strong><\/th>\r\n\t\t<\/tr><\/tbody>
qstat
qstat -u username
\r\n\t\t\t\t\tqstat -f<\/blockquote>\r\n\t\t\t<\/td>\r\n\t\t\t
squeue
squeue -u username
\r\n\t\t\t\t\tsqueue -al<\/blockquote>\r\n\t\t\t<\/td>\r\n\t\t<\/tr>
qsub
qsub -N jobname
\r\n\t\t\t\t\tqsub -l h_rt=24:00:00
\r\n\t\t\t\t\tqsub -pe dmp4 16
\r\n\t\t\t\t\tqsub -l mem=4G
\r\n\t\t\t\t\tqsub -o filename
\r\n\t\t\t\t\tqsub -e filename
\r\n\t\t\t\t\tqsub -l scratch_free=20G<\/blockquote>\r\n\t\t\t<\/td>\r\n\t\t\t
sbatch
sbatch -J jobname
\r\n\t\t\t\t\tsbatch -t 24:00:00
\r\n\t\t\t\t\tsbatch -p node -n 16<\/blockquote>\r\n\t\t\t\t
sbatch –mem=4000
\r\n\t\t\t\t\tsbatch -o filename
\r\n\t\t\t\t\tsbatch -e filename<\/blockquote>\r\n\t\t\t<\/td>\r\n\t\t<\/tr>
# Interactive run, one core<\/td>\r\n\t\t\t# Interactive run, one core<\/td>\r\n\t\t<\/tr>
qrsh -l h_rt=8:00:00<\/td>\r\n\t\t\tsalloc -t 8:00:00
\r\n\t\t\t\tinteractive -p core -n 1 -t 8:00:00<\/td>\r\n\t\t<\/tr>

\r\n\t\t\t\tqdel<\/td>\r\n\t\t\t
\r\n\t\t\t\tscancel<\/td>\r\n\t\t<\/tr><\/tbody><\/table>

<\/p>\r\n

<\/p>\r\n
SGE for a single-core application<\/strong><\/th>\r\n\t\t\tSLURM for a single-core application<\/strong><\/th>\r\n\t\t<\/tr><\/tbody>
\r\n\t\t\t\t
\r\n\t\t\t\t\t
\r\n#!\/bin\/bash<\/span>\r\n#<\/span>\r\n#<\/span>\r\n#$ -N test<\/span>\r\n#$ -j y<\/span>\r\n#$ -o test.output<\/span>\r\n#$ -cwd<\/span>\r\n#$ -M $USER@stanford.edu<\/span>\r\n#$ -m bea<\/span>\r\n# Request 5 hours run time<\/span>\r\n#$ -l h_rt=5:0:0<\/span>\r\n#$ -P your_project_id_here<\/span>\r\n#<\/span>\r\n#$ -l mem=4G<\/span>\r\n# <\/span>\r\n \r\n<<\/span>call your app here><\/span><\/pre>\r\n\t\t\t\t<\/div>\r\n\t\t\t<\/td>\r\n\t\t\t
\r\n\t\t\t\t
\r\n\t\t\t\t\t
\r\n#!\/bin\/bash -l<\/span>\r\n# NOTE the -l flag!<\/span>\r\n#<\/span>\r\n#SBATCH -J tes<\/span>t\r\n#SBATCH -o test.\"%j\".out<\/span>\r\n#SBATCH -e test.\"%j\".err<\/span>\r\n# Default in slurm<\/span>\r\n#SBATCH --mail-user $USER@stanford.edu<\/span>\r\n#SBATCH --mail-type=ALL<\/span>\r\n# Request 5 hours run time<\/span>\r\n#SBATCH -t 5:0:0<\/span>\r\n#SBATCH --mem=4000<\/span>\r\n#SBATCH -p normal<\/span>\r\n\r\n \r\n<<\/span>load modules, call your app here><\/span><\/pre>\r\n\t\t\t\t<\/div>\r\n\t\t\t<\/td>\r\n\t\t<\/tr>
<\/td>\r\n\t\t\t <\/td>\r\n\t\t<\/tr><\/tbody><\/table>

Comparison of some parallel environments set by sge and slurm<\/p>\r\n
SGE<\/strong><\/th>\r\n\t\t\tSLURM<\/strong><\/th>\r\n\t\t<\/tr><\/tbody>
$JOB_ID<\/td>\r\n\t\t\t$SLURM_JOB_ID<\/td>\r\n\t\t<\/tr>
$NSLOTS<\/td>\r\n\t\t\t$SLURM_NPROCS<\/td>\r\n\t\t<\/tr><\/tbody><\/table>

<\/p>\r\n

<\/p>","protected":false},"excerpt":{"rendered":"

Sun Grid Engine (SGE) and SLURM job scheduler concepts are quite similar. Below is a table of some common SGE commands and their SLURM equivalents. Some common commands and flags in SGE and SLURM with their respective equivalents: User Commands …<\/p>\n

SGE to SLURM conversion<\/span> Read More »<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":50,"menu_order":100,"comment_status":"closed","ping_status":"closed","template":"full-width-page.php","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"class_list":["post-77","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/pages\/77","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/comments?post=77"}],"version-history":[{"count":11,"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/pages\/77\/revisions"}],"predecessor-version":[{"id":861,"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/pages\/77\/revisions\/861"}],"up":[{"embeddable":true,"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/pages\/50"}],"wp:attachment":[{"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/media?parent=77"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}