Thomas Zeiser

Some comments by Thomas Zeiser about HPC@RRZE and other things

Content

STAR-CD 3.24 Update 025 on SGI Altix

For some reasons, the update 025 of STAR-CD 3.24 on our SGI Altix system tried to use Intel MPI by default instead of SGI MPT. After setting the environment variable STARFLAGS=-mpi=sgi, SGI MPT is used again also for this version.

MpCCI auf SGI Altix

MpCCI is a library for coupling different codes (e.g. fluid mechanics and aeroaccustics) and excahnging mesh-based data between them. It is developed and sold by Fraunhofer-Institute SCAI (http://www.scai.fraunhofer.de/mpcci.html).

MpCCI is available for several platforms and relies on MPI for communication.

The good news: MpCCI SDK is available for IA64.
The bad news: it relies on mpich

On our SGI Altix system we use PBS Professional as batch queuing system and each running job gets its own CPU-set.

When now starting a MpCCI job, a procgroup file is generated and the processes are started via ssh. And that’s exactly the problem: the sshd daemon (started by root at boot-time) runs outside the CPU-set. Consequently, all processes started via ssh are also outside the allocated cpuset … 🙂

Solutions?
* shared-memory mpich does not work as the shm device of mpich does not work with MPMD, i.e. a procgroup file is not suppoerted
* using SGI MPT (SGI’s own MPI) does not work as the binary-only MpCCI library relies on some mpich symbols
* starting the code with mpiexec does not work as there are some problems with accessing stdin from within the application
* …