ReTrace: Topological Evaluation of White Matter Tractography Algorithms Using Reeb Graphs

We present ReTrace, a novel graph matching-based topological evaluation and validation method for tractography algorithms. ReTrace uses a Reeb graph whose nodes and edges capture the topology of white matter fiber bundles. We evaluate the performance of 96 algorithms from the ISMRM Tractography Challenge and the standard algorithms implemented in DSI Studio for the population-averaged Human Connectome Project (HCP) dataset. The existing evaluation metrics such as the f-score, bundle overlap, and bundle overreach fail to account for fiber continuity resulting in high scores even for broken fibers, branching artifacts, and mis-tracked fiber crossing. In contrast, we show that ReTrace effectively penalizes the incorrect tracking of fibers within bundles while concurrently pinpointing positions with significant deviation from the ground truth. Based on our analysis of ISMRM challenge data, we find that no single algorithm consistently outperforms others across all known white matter fiber bundles, highlighting the limitations of the current tractography methods. We also observe that deterministic tractography algorithms perform better in tracking the fundamental properties of fiber bundles, specifically merging and splitting, compared to probabilistic tractography. We compare different algorithmic approaches for a given bundle to highlight the specific characteristics that contribute to successful tracking, thus providing topological insights into the development of advanced tractography algorithms.

1. 1.

   https://github.com/s-shailja/ReTrace.

2. 2.

   https://zenodo.org/record/840086.

3. 3.

   https://github.com/s-shailja/ReTrace.

4. 4.

   https://brain.labsolver.org/hcp_template.html.

5. 5.

   https://dsi-studio.labsolver.org/

The authors acknowledge Vikram Bhagavatula for his preliminary implementation of the edit distance. Data were provided by The ISMRM 2015 Tractography Challenge and by HCP, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University.

Authors and Affiliations

1. University of California, Santa Barbara, CA, 93117, USA

   S. Shailja, Scott T. Grafton & B. S. Manjunath

2. Irvine Medical Center, University of California, Santa Barbara, CA, 92868, USA

   Jefferson W. Chen

Corresponding author

Correspondence to S. Shailja .

Editors and Affiliations

1. University of Illinois at Chicago, Chicago, IL, USA

   Muge Karaman

2. Florey Institute of Neuroscience and Mental Health, Heidelberg, VIC, Australia

   Remika Mito

3. University College London, London, UK

   Elizabeth Powell

4. Université de Sherbrooke, Sherbrooke, QC, Canada

   Francois Rheault

5. Microsoft Research Cambridge, Milton, UK

   Stefan Winzeck

Mapping of algorithm IDs to the original submission IDs:

HCP Dataset

ReTrace handles fiber crossings effectively, as demonstrated in Fig. 5. We use the 1 mm population-averaged FIB file in the ICBM152 space for fiber tracking in DSI Studio. We select a small region of interest (a 2 mm isotropic 3D region) from the probabilistic atlas in an area with a high probability (\(\sim 0.9\)) of double-crossing fibers. We use the deterministic streamline tracking method implemented in DSI Studio^{Footnote 5} to compute the fibers with the parameters set (angular threshold, step size, min length, max length, terminate if seeds, iterations for topological pruning) to 35, 1, 70, 200, 1000, and 16, respectively. This allowed us to observe successful tracking without broken or distorted fibers. To mimic the spurious broken fibers that tractography methods may yield, we set the parameters to 35, 1, 0, 100, 1000, and 16. For observing the angular distortion where the fiber bends and follows a different path, we set the parameters as 90, 1, 70, 100, 1000, and 16. The resulting Reeb graphs clearly highlight how their nodes capture successful and unsuccessful tracking. Nodes formed near intersections indicate broken or bent fibers, as shown in Fig. 5. Whenever fibers travel together in a group, they form an edge in the graph. Any alteration within this group prompts a critical event, resulting in a node in the Reeb graph. Consequently, if a fiber breaks or diverges, the associated group changes, generating a node. This node, present at the merging point, contributes to a larger distance value. In the topological distance computation, \(d_{\text {edit}}\), this node could be weighted more if the goal is to assess an algorithm’s tracking ability in ambiguous fiber orientations. By providing a 3D location for our algorithm’s attention, any discrepancy within that location can significantly affect the overall \(d_{\text {edit}}\) computation. The code is open source and can be tailored to specific needs.

Reprints and permissions

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

Policies and ethics