{"id":77,"date":"2019-03-06T11:21:53","date_gmt":"2019-03-06T16:21:53","guid":{"rendered":"http:\/\/psycomp.utsc.utoronto.ca\/support\/?page_id=77"},"modified":"2019-06-21T22:02:13","modified_gmt":"2019-06-22T02:02:13","slug":"sge-to-slurm-conversion","status":"publish","type":"page","link":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/resources\/sge-to-slurm-conversion\/","title":{"rendered":"SGE to SLURM conversion"},"content":{"rendered":"

Sun Grid Engine<\/strong> (SGE<\/strong>) and SLURM<\/strong> job scheduler concepts are quite similar. Below is a table of some common SGE commands and their SLURM equivalents.<\/p>\n

Some common commands and flags in SGE and SLURM with their respective equivalents:<\/p>\n<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
User Commands<\/strong><\/td>\nSGE<\/strong><\/td>\nSLURM<\/strong><\/td>\n<\/tr>\n
Interactive login<\/strong><\/td>\nqlogin<\/td>\n\n
\r\nsrun --pty bash or srun -p \"partition name\" --time=4:0:0 --pty bash. For a quick dev node, just run \"sdev\"<\/pre>\n<\/td>\n<\/tr>\n
Job submission<\/strong><\/td>\nqsub [script_file]<\/td>\nsbatch [script_file]<\/td>\n<\/tr>\n
Job deletion<\/strong><\/td>\nqdel [job_id]<\/td>\nscancel [job_id]<\/td>\n<\/tr>\n
Job status by job<\/strong><\/td>\nqstat -u \\* [-j job_id]<\/td>\nsqueue [job_id]<\/td>\n<\/tr>\n
Job status by user<\/strong><\/td>\nqstat [-u user_name]<\/td>\n\n

squeue -u [user_name]<\/p>\n<\/td>\n<\/tr>\n

Job hold<\/strong><\/td>\nqhold [job_id]<\/td>\nscontrol hold [job_id]<\/td>\n<\/tr>\n
Job release<\/strong><\/td>\nqrls [job_id]<\/td>\nscontrol release [job_id]<\/td>\n<\/tr>\n
Queue list<\/strong><\/td>\nqconf -sql<\/td>\nsqueue<\/td>\n<\/tr>\n
List nodes<\/strong><\/td>\nqhost<\/td>\nsinfo -N OR scontrol show nodes<\/td>\n<\/tr>\n
Cluster status<\/strong><\/td>\nqhost -q<\/td>\nsinfo<\/td>\n<\/tr>\n
GUI<\/strong><\/td>\nqmon<\/td>\nsview<\/td>\n<\/tr>\n
<\/td>\n <\/td>\n <\/td>\n<\/tr>\n
Environmental<\/strong><\/td>\n <\/td>\n <\/td>\n<\/tr>\n
Job ID<\/strong><\/td>\n$JOB_ID<\/td>\n$SLURM_JOBID<\/td>\n<\/tr>\n
Submit directory<\/strong><\/td>\n$SGE_O_WORKDIR<\/td>\n$SLURM_SUBMIT_DIR<\/td>\n<\/tr>\n
Submit host<\/strong><\/td>\n$SGE_O_HOST<\/td>\n$SLURM_SUBMIT_HOST<\/td>\n<\/tr>\n
Node list<\/strong><\/td>\n$PE_HOSTFILE<\/td>\n$SLURM_JOB_NODELIST<\/td>\n<\/tr>\n
Job Array Index<\/strong><\/td>\n$SGE_TASK_ID<\/td>\n$SLURM_ARRAY_TASK_ID<\/td>\n<\/tr>\n
<\/td>\n <\/td>\n <\/td>\n<\/tr>\n
Job Specification<\/strong><\/td>\n <\/td>\n <\/td>\n<\/tr>\n
Script directive<\/strong><\/td>\n#$<\/td>\n#SBATCH<\/td>\n<\/tr>\n
queue<\/strong><\/td>\n-q [queue]<\/td>\n-p [queue]<\/td>\n<\/tr>\n
count of nodes<\/strong><\/td>\nN\/A<\/td>\n-N [min[-max]]<\/td>\n<\/tr>\n
CPU count<\/strong><\/td>\n-pe [PE] [count]<\/td>\n-n [count]<\/td>\n<\/tr>\n
Wall clock limit<\/strong><\/td>\n-l h_rt=[seconds]<\/td>\n-t [min] OR -t [days-hh:mm:ss]<\/td>\n<\/tr>\n
Standard out file<\/strong><\/td>\n-o [file_name]<\/td>\n-o [file_name]<\/td>\n<\/tr>\n
Standard error file<\/strong><\/td>\n-e [file_name]<\/td>\ne [file_name]<\/td>\n<\/tr>\n
Combine STDOUT & STDERR files <\/strong><\/td>\n-j yes<\/td>\n(use -o without -e)<\/td>\n<\/tr>\n
Copy environment<\/strong><\/td>\n-V<\/td>\n–export=[ALL | NONE | variables]<\/td>\n<\/tr>\n
Event notification<\/strong><\/td>\n-m abe<\/td>\n–mail-type=[events]<\/td>\n<\/tr>\n
send notification email<\/strong><\/td>\n-M [address]<\/td>\n–mail-user=[address]<\/td>\n<\/tr>\n
Job name<\/strong><\/td>\n-N [name]<\/td>\n–job-name=[name]<\/td>\n<\/tr>\n
Restart job<\/strong><\/td>\n-r [yes|no]<\/td>\n–requeue OR –no-requeue (NOTE:
\n\t\t\t\tconfigurable default)<\/td>\n<\/tr>\n
Set working directory<\/strong><\/td>\n-wd [directory]<\/td>\n–workdir=[dir_name]<\/td>\n<\/tr>\n
Resource sharing<\/strong><\/td>\n-l exclusive<\/td>\n–exclusive OR–shared<\/td>\n<\/tr>\n
Memory size<\/strong><\/td>\n-l mem_free=[memory][K|M|G]<\/td>\n–mem=[mem][M|G|T] OR –mem-per-cpu=
\n\t\t\t\t[mem][M|G|T]<\/td>\n<\/tr>\n
Charge to an account<\/strong><\/td>\n-A [account]<\/td>\n–account=[account]<\/td>\n<\/tr>\n
Tasks per node<\/strong><\/td>\n(Fixed allocation_rule in PE)<\/td>\n–tasks-per-node=[count]<\/td>\n<\/tr>\n
<\/td>\n <\/td>\n–cpus-per-task=[count]<\/td>\n<\/tr>\n
Job dependancy<\/strong><\/td>\n-hold_jid [job_id | job_name]<\/td>\n–depend=[state:job_id]<\/td>\n<\/tr>\n
Job project<\/strong><\/td>\n-P [name]<\/td>\n–wckey=[name]<\/td>\n<\/tr>\n
Job host preference<\/strong><\/td>\n-q [queue]@[node] OR -q
\n\t\t\t\t[queue]@@[hostgroup]<\/td>\n		–nodelist=[nodes] AND\/OR –exclude=
\n\t\t\t\t[nodes]<\/td>\n<\/tr>\n
Quality of service<\/strong><\/td>\n <\/td>\n–qos=[name]<\/td>\n<\/tr>\n
Job arrays<\/strong><\/td>\n-t [array_spec]<\/td>\n–array=[array_spec] (Slurm version 2.6+)<\/td>\n<\/tr>\n
Generic Resources<\/strong><\/td>\n-l [resource]=[value]<\/td>\n–gres=[resource_spec]<\/td>\n<\/tr>\n
Lincenses<\/strong><\/td>\n-l [license]=[count]<\/td>\n–licenses=[license_spec]<\/td>\n<\/tr>\n
Begin Time<\/strong><\/td>\n-a [YYMMDDhhmm]<\/td>\n–begin=YYYY-MM-DD[THH:MM[:SS]]<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n\n\n\n\n\n\n\n\n
SGE<\/strong><\/th>\nSLURM<\/strong><\/th>\n<\/tr>\n<\/tbody>\n
qstat<\/p>\n

qstat -u username
\n\t\t\t\t\tqstat -f<\/p><\/blockquote>\n<\/td>\n

squeue<\/p>\n

squeue -u username
\n\t\t\t\t\tsqueue -al<\/p><\/blockquote>\n<\/td>\n<\/tr>\n

qsub<\/p>\n

qsub -N jobname
\n\t\t\t\t\tqsub -l h_rt=24:00:00
\n\t\t\t\t\tqsub -pe dmp4 16
\n\t\t\t\t\tqsub -l mem=4G
\n\t\t\t\t\tqsub -o filename
\n\t\t\t\t\tqsub -e filename
\n\t\t\t\t\tqsub -l scratch_free=20G<\/p><\/blockquote>\n<\/td>\n

sbatch<\/p>\n

sbatch -J jobname
\n\t\t\t\t\tsbatch -t 24:00:00
\n\t\t\t\t\tsbatch -p node -n 16<\/p><\/blockquote>\n

sbatch –mem=4000
\n\t\t\t\t\tsbatch -o filename
\n\t\t\t\t\tsbatch -e filename<\/p><\/blockquote>\n<\/td>\n<\/tr>\n

# Interactive run, one core<\/td>\n# Interactive run, one core<\/td>\n<\/tr>\n
qrsh -l h_rt=8:00:00<\/td>\nsalloc -t 8:00:00
\n\t\t\t\tinteractive -p core -n 1 -t 8:00:00<\/td>\n<\/tr>\n
\n\t\t\t\tqdel<\/td>\n\n\t\t\t\tscancel<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/p>\n\n\n\n\n\n\n
SGE for a single-core application<\/strong><\/th>\nSLURM for a single-core application<\/strong><\/th>\n<\/tr>\n<\/tbody>\n
\n
\n
\r\n#!\/bin\/bash<\/span>\r\n#<\/span>\r\n#<\/span>\r\n#$ -N test<\/span>\r\n#$ -j y<\/span>\r\n#$ -o test.output<\/span>\r\n#$ -cwd<\/span>\r\n#$ -M $USER@stanford.edu<\/span>\r\n#$ -m bea<\/span>\r\n# Request 5 hours run time<\/span>\r\n#$ -l h_rt=5:0:0<\/span>\r\n#$ -P your_project_id_here<\/span>\r\n#<\/span>\r\n#$ -l mem=4G<\/span>\r\n# <\/span>\r\n \r\n<<\/span>call your app here><\/span><\/pre>\n<\/p><\/div>\n<\/td>\n
\n
\n
\r\n#!\/bin\/bash -l<\/span>\r\n# NOTE the -l flag!<\/span>\r\n#<\/span>\r\n#SBATCH -J tes<\/span>t\r\n#SBATCH -o test.\"%j\".out<\/span>\r\n#SBATCH -e test.\"%j\".err<\/span>\r\n# Default in slurm<\/span>\r\n#SBATCH --mail-user $USER@stanford.edu<\/span>\r\n#SBATCH --mail-type=ALL<\/span>\r\n# Request 5 hours run time<\/span>\r\n#SBATCH -t 5:0:0<\/span>\r\n#SBATCH --mem=4000<\/span>\r\n#SBATCH -p normal<\/span>\r\n\r\n \r\n<<\/span>load modules, call your app here><\/span><\/pre>\n<\/p><\/div>\n<\/td>\n<\/tr>\n
<\/td>\n <\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Comparison of some parallel environments set by sge and slurm<\/p>\n\n\n\n\n\n\n
SGE<\/strong><\/th>\nSLURM<\/strong><\/th>\n<\/tr>\n<\/tbody>\n
$JOB_ID<\/td>\n$SLURM_JOB_ID<\/td>\n<\/tr>\n
$NSLOTS<\/td>\n$SLURM_NPROCS<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/p>\n","protected":false},"excerpt":{"rendered":"

Sun Grid Engine (SGE) and SLURM job scheduler concepts are quite similar. Below is a table of some common SGE commands and their SLURM equivalents. Some common commands and flags in SGE and SLURM with their respective equivalents: User Commands SGE SLURM Interactive login qlogin srun –pty bash or srun -p “partition name” –time=4:0:0 –pty … Continue reading “SGE to SLURM conversion”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":50,"menu_order":100,"comment_status":"closed","ping_status":"closed","template":"full-width-page.php","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"class_list":["post-77","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/pages\/77","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/comments?post=77"}],"version-history":[{"count":11,"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/pages\/77\/revisions"}],"predecessor-version":[{"id":861,"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/pages\/77\/revisions\/861"}],"up":[{"embeddable":true,"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/pages\/50"}],"wp:attachment":[{"href":"https:\/\/psycomp.utsc.utoronto.ca\/support\/index.php\/wp-json\/wp\/v2\/media?parent=77"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}