List view
First, an attempt to profile a real, moderately-complex, mixed-language application using the SALT-FM instru- mentor prototype developed in Task 2 and the updated TAU capabilities developed in Task 3 will be undertaken, with the aim of finding and documenting requirements for implementing a hardened, robust and easy to use SALT-FM product. This evaluation will drive the development of a Phase II work plan. It will also further develop documentation on the usage of the prototype, user workflows and any limitations. The results of this test, the feasibility of the approach and requirements for Phase II will be included in a final report. The report will also document the architecture and usage of the instrumentor prototype, the libTooling for Flang library developed, and any other nascent software libraries or components developed in support of this effort that appear viable to continue to build upon in a follow-on effort. The report will document any pain points encountered when attempting to use the prototype, lessons learned and ideas to explore to ameliorate any pain points and improve the ease of use and capabilities of SALT-FM. This final report, along with the prototype instrumentor, the libTooling for Flang library, and an updated TAU software package will serve as the four main deliverables at the end of the Phase I effort
No due dateUsing the libTooling for Flang implementation from Task 1, and the instrumentor tool implemented in Task 2, we will extend the instrumentor for compatibility with TAU’s selective instrumentation files, and wrapper scripts. This will allow applications to be easily instrumented without needing to extensively modify their build systems. This SALT-FM command-line instrumentor will be updated to run in place of an actual compiler using a limited subset of the same arguments. The tool will call LLVM-Flang to process the command line after translating and/or sanitizing flags and options that are intended for a different compiler. For backward compatibility with the existing PDT implementation, the instrumentor will be updated support at least a subset of TAU’s existing Selective Instrumentation File (SIF) format. This allows the user to specify which routines, loops, and other constructs should be instrumented or skipped and what type of instrumentation should be used. It consists of a series of directives specifying patterns for function declarations which are to be in- cluded or excluded from instrumentation, and directives which specify other source locations where timers will be inserted. Furthermore, TAU will be updated, in particular, the Fortran compiler wrapper scripts, to be compatible with SALT- FM and use it to parse, instrument and compile select Fortran and mixed language test cases. The scripts will call the SALT-FM frontend to generate instrumented source code and seamlessly compile it by forwarding the original command-line arguments to the original compiler while substituting the original source file with the instrumented one. This is not quite as straight forwards as it seems since extra care must be taken when handling Fortran’s modules and submodules. Compilers produce module files, each with their own proprietary format, and the module files associated with dependencies of a particular source file must be present when the compiler starts to parse that file. Therefore, the order in which SALT-FM processes source files is critical, and duplicate sets of module files may be produced: one by the SALT-FM frontend (produced by LLVM-Flang) and the other produced by the underlying compiler that is actually building the instrumented code. As a consequence, the instrumentor will likely need to visit every Fortran file in a project, even those that the user does not wish to instrument. In addition, the emitted instrumented code will need to have all modules and submodules namespaced, e.g., with an additional prefix, or it will need to be built in a parallel directory tree or the search and output locations of module files will need to be carefully managed and separated between the actual compiler and the instrumentor. In Phase II and follow on efforts, producing a Compilation Database, or using a preexisting Compilation Database that can be produced by build systems such as CMake21 and Bear22 will be investigated. Compilation databases will allow end users to easily instrument or wrap complex applications built with commonly used build systems, including GNU Autotools and CMake. To more easily accommodate complex software projects, and support use cases beyond instrumentation with TAU a more generic, stand-alone compiler wrapper script will also be developed at that time.
No due date•1/1 issues closedFor this task, we will implement an instrumentor that will insert TAU instrumentation around code regions of interest. This will perform certain types of common transformation, such as starting a timer upon entry to a procedure and stopping the timer at every exit point, and utilize the libTooling for Flang library developed in Task 1. The deliverable from this task is a SALT-FM instrumentor tool that can parse and instrument ba- sic Fortran programs such as the NAS Parallel Bench- marks and other simple and mixed Fortran and C/C++ programs using more modern Fortran language fea- tures. From previous work with SALT, we have found that it is useful to separate the parsing and instrumenting tasks into their own libraries and to use an interme- diary query library. The basic design and data flow of SALT can be seen in Figure 5, and will likely be adopted for SALT-FM eventually. The instrumentation library will generate Instrumentation Requests, which a Query Library will use to identify source locations in the Fortran source. Transformed source files will be gener- ated, likely by calls into Flang’s parse tree unparser. During Phase I, these components will not necessarily be split into individual libraries nor are they expected to be fully robust and generic. However, the previous development of SALT will guide our design choices, and the prototype will be designed in a modular way so that these tasks can be abstracted into their own libraries during follow on efforts, if not during this Phase I effort. Follow on Phase II efforts will further develop and abstract these components to be more robust and generic. This includes factoring out a SALT-FM Query Library that takes a Flang syntax tree(s) as input and outputs Instrumentation Request objects that identify source locations for later transformation, in addition to the library for the simplified API onto the Flang parse tree discussed in Task 1.
No due date•1/1 issues closedThis task begins the process of developing functionality for modern Fortran similar to that provided for C and modern C++ in SALT. While LLVM Flang has been under development in its current form only since 2019, it is still relatively immature but it has been making fast progress. In comparison, Clang has been under continuous development since its first release more than 15 years ago. Clang has a component called libTooling,18 which provides support for writing Clang-based standalone tools. The library enables such tools to have an interface that is identical to that provided by the actual Clang compiler and handles command-line processing and exposes an API for access to the Clang Abstract Syntax Tree. One of the limitations we will face in developing a Fortran instrumentor is that Flang currently has no equivalent to Clang’s libTooling.19 A prototype, minimal libTooling for Flang will be created to demonstrate the feasibility of of developing a full libTooling-like system for Flang. Communications will be established with the Flang developers through the Flang mailing lists, Slack, and bi-weekly conference call to ensure that there is no duplication of efforts and that our efforts are in line with LLVM/Flang policies and design specifications. A challenge we will face in developing the Fortran instrumentor is that, in addition to lacking an equivalent to libTooling, the design of the Flang front end does not include an Abstract Syntax Tree at all.20 Instead, the Flang front-end provides direct access to the parse tree, a concrete syntax tree that represents the Fortran gram- mar precisely without any additional layer of abstraction. This is then lowered directly to a custom intermediate language, Flang Intermediate Representation (FIR), which is then lowered to MLIR and then finally to LLVM IR. Additionally, the development of the lowering stages is still under intense development. During any follow-on Phase II efforts, this “libTooling for Flang” library will be extended to more robustly support the generic instrumentation use cases, and other other program transformations such as wrapper generation and code translation. In addition, the feasibility of creating a simplified API onto the Flang parse tree would be explored during Phase II. Functionality relating to this will be abstracted from—and factored out of—the libTooling for Flang library and the instrumentor application, to be put into a dedicated library component. The ultimate goal is contributing the libTooling for Flang library component back upstream to LLVM, which will help ensure it’s continued maintenance and development.
No due date•1/1 issues closed