tmux-mpi connects MPI processes to tmux windows. MPI launches n instances of dtach, these n instances are then connected to in tmux windows. To launch a command use:
tmux-mpi <npro 6C45 c> <command>
For example,
tmux-mpi 2 hostname
will start a tmux session called tmux-mpi that can be connected to in a different terminal with
tmux attach -t tmux-mpi
or use
tmux attach
if there are no other tmux sessions. You may need to scroll up in the tmux window to see the output of this example. This dtach approach does mean that if the program crashes manual cleanup may need to happen. For example:
pkill -9 dtach
Copyright 2020, WR Saunders
Requires dtach and tmux to be installed. Pip installable with:
pip install --upgrade --no-cache-dir git+https://github.com/wrs20/tmux-mpi@master
Some environment variables can be set to configure behaviour. These are environment variables such that they can be set once somewhere in an environment.
TMUX_MPI_MPIRUN- Configure how
mpiexecis called, e.g.TMUX_MPI_MPIRUN="mpiexec -s all"for recent Intel MPI. TMUX_MPI_POST_LAUNCH- Configure a command to launch after the processes have started, e.g.
TMUX_MPI_POST_LAUNCH="xterm -e 'tmux attach -t TMUX_MPI_SESSION_NAME'"would launch a newxtermand connect to the created server. The stringTMUX_MPI_SESSION_NAMEwill be replaced with the actual session name. TMUX_MPI_MODE- Choose if each rank should be in a new tmux window using
window(default) orpane, e.g.TMUX_MPI_MODE=pane. TMUX_MPI_SYNC_PANES- Synchronises the input to the tmux panes when using
TMUX_MPI_MODE=pane, e.g.TMUX_MPI_SYNC_PANES=1. Default off. Note that this uses the pane synchronisation built into tmux.
Try the following to start debugging a Python program using a C extension in GDB and 2 MPI ranks. The environment variable will automatically launch a xterm window to connect to the session. Note that tmux window numbers are not MPI ranks (this functionality would require this tool to access the MPI implementation somehow).
# Start a xterm that connects to the session export TMUX_MPI_POST_LAUNCH="xterm -e 'tmux attach -t TMUX_MPI_SESSION_NAME'" # launch tmux-mpi tmux-mpi 2 gdb --ex run --args python <script-name>
Same as above but using tmux panes instead of windows and synchronised input:
# Start a xterm that connects to the session export TMUX_MPI_POST_LAUNCH="xterm -e 'tmux attach -t TMUX_MPI_SESSION_NAME'" # use panes not windows export TMUX_MPI_MODE=pane #synchronised inputs export TMUX_MPI_SYNC_PANES=1 # launch tmux-mpi tmux-mpi 2 gdb --ex run --args python <script-name>
Some (recent?) versions of Intel MPI try to do stdin redirection by default. If you have issues with ranks not starting try the following (as demonstrated using python):
TMUX_MPI_MPIRUN="mpiexec -s all" tmux-mpi 2 python