A workstation-based parallel direct-execution simulator

It is important to understand and efficiently predict the performance of large codes executing on massively parallel machines. However, these very large machines are scarce, expensive, and generally unavailable to large segments of the research community. It is therefore important to implement performance analysis tools for such machines on platforms that are readily available to the research community at large. To meet this need, we have ported LAPSE, a parallel direct-execution simulator, from the Intel Paragon to an ordinary cluster of workstations. The goal of this research is to provide researchers the opportunity to study codes designed for execution on a massively parallel machine while physically executing on a workstation cluster. However, we encountered significant performance problems when moving to a workstation cluster, due primarily to high communication and context switching costs. To reduce these costs, we implemented the virtual processors of the simulated system using light-weight threads rather than heavy-weight Unix processes. In this paper, we discuss the issues involved in moving from a process-based to a thread-based simulator, and demonstrate up to a four fold increase in performance by doing so.