mpiexec + taskset: code snippet to generate config file

We usually use Pete Wyckoff’s mpiexec to start MPI processes from within batch jobs. The following code snippet may be used to pin individual MPI processes to their CPU. A similar snippet could be used to allow round-robin distribution of the MPI processes …

[shell]
#!/bin/bash
# usage: run.sh 8 “0 2 4 6” ./a.out “-b -c”
# start a.out with arguments -b -c
# use 4 MPI processes
# pin processes to CPUs 0,2,4,6

NUMPROC=$1
TASKCPUS=”$2″
EXE=$3
ARGS=$4

TASKS=`echo $TASKCPUS | wc -w`
echo “running $NUMPROC MPI processes with $TASKS per node”

cat /dev/null > cfg.$PBS_JOBID
for node in `uniq $PBS_NODEFILE`; do
for cpu in $TASKCPUS; do
echo “$node : taskset -c $cpu $EXE $ARGS” >> cfg.$PBS_JOBID
NUMPROC=$((NUMPROC – 1))
if [ $NUMPROC -eq 0 ]; then
break 2
fi
done
done
mpiexec -comm pmi -kill -config cfg.$PBS_JOBID
[/shell]