Analysis of the BraTS 2023 Intracranial Meningioma Segmentation Challenge

Dominic LaBella1Orcid, Ujjwal Baid2,3, Omaditya Khanna4, Shan McBurney-Lin5, Ryan McLean6, Pierre Nedelec5, Arif Rashid7, Nourel Hoda Tahon8, Talissa Altes4, Radhika Bhalerao5, Yaseen Dhemesh8, Devon Godfrey1, Fathi Hilal8, Scott Floyd1, Anastasia Janas6, Anahita Fathi Kazerooni9,10, John Kirkpatrick1, Collin Kent1, Florian Kofler11,12,13,14, Kevin Leu15, Nazanin Maleki6, Bjoern Menze16,17, Maxence Pajot5, Zachary J. Reitman1, Jeffrey D. Rudie18,5, Rachit Saluja19, Yury Velichko20, Chunhao Wang1, Pranav Warman21, Maruf Adewole22, Jake Albrecht23, Udunna Anazodo24, Syed Muhammad Anwar25,26, Timothy Bergquist23, Sully Francis Chen21, Verena Chung23, Rong Chai23, Gian-Marco Conte27, Farouk Dako28, James Eddy23, Ivan Ezhov12,13, Nastaran Khalili9, Juan Eugenio Iglesias29,30,31, Zhifan Jiang25,26, Elaine Johanson32, Koen Van Leemput33, Hongwei Bran Li34,17,35, Marius George Linguraru25,26, Xinyang Liu25,26, Aria Mahtabfar4, Zeke Meier36, Ahmed W Moawad37, John Mongan5, Marie Piraud11, Russell Takeshi Shinohara38,10, Walter F. Wiggins39,40,41, Aly H. Abayazeed42, Rachel Akinola43, András Jakab44, Michel Bilello10,45, Maria Correia de Verdier46, Priscila Crivellaro47, Christos Davatzikos10,45, Keyvan Farahani48, John Freymann49,48, Christopher Hess5, Raymond Huang50, Philipp Lohmann51,52, Mana Moassefi53, Matthew W. Pease54, Phillipp Vollmuth55,56, Nico Sollmann57,58,59, David Diffley60, Khanak K. Nandolia61, Daniel I Warren62, Ali Hussain63, Pascal Fehringer64, Yulia Bronstein65, Lisa Deptula66, Evan G. Stein67, Mahsa Taherzadeh68, Eduardo Portela de Oliveira69, Aoife Haughey70, Marinos Kontzialis71, Luca Saba72, Benjamin Turner73, Melanie M. T. Brüßeler74, Shehbaz Ansari75, Athanasios Gkampenis76, David Maximilian Weiss77, Aya Mansour78, Islam H. Shawali79, Nikolay Yordanov80, Joel M. Stein45, Roula Hourani81, Mohammed Yahya Moshebah82, Ahmed Magdy Abouelatta83, Tanvir Rizvi84, Klara Willms6, Dann C. Martin85, Abdullah Okar86, Gennaro D’Anna87, Ahmed Taha88, Yasaman Sharifi89, Shahriar Faghani27, Dominic Kite90, Marco Pinho91, Muhammad Ammar Haider92, Alejandro Aristizabal93,94, Alexandros Karargyris93, Hasan Kassem93, Sarthak Pati95,2,96, Micah Sheller97,93, Michelle Alonso-Basanta7, Javier Villanueva-Meyer5, Andreas M Rauschecker5, Ayman Nada8, Mariam Aboian9, Adam E. Flanders98, Benedikt Wiestler17, Spyridon Bakas2,99,100,101Orcid, Evan Calabrese41,5
1: Duke University Medical Center, Department of Radiation Oncology, Durham, NC, USA, 2: Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, USA, 3: Center for Federated Learning in Medicine, Indiana University, Indianapolis, IN, USA, 4: Department of Neurosurgery, Thomas Jefferson University, Philadelphia, PA, USA, 5: University of California San Francisco, CA, USA, 6: Yale University, New Haven, CT, USA, 7: Department of Radiation Oncology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA, 8: University of Missouri, Columbia, MO, USA, 9: Children’s Hospital of Philadelphia, University of Pennsylvania, Philadelphia, PA, USA, 10: Center for AI and Data Science for Integrated Diagnostics (AI2D) and Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA, 11: Helmholtz AI, Helmholtz Munich, Germany, 12: Department of Informatics, Technical University Munich, Germany, 13: TranslaTUM - Central Institute for Translational Cancer Research, Technical University of Munich, Germany, 14: Department of Diagnostic and Interventional Neuroradiology, School of Medicine, Klinikum rechts der Isar, Technical University of Munich, Germany, 15: Center for Intelligent Imaging (ci2), Department of Radiology and Biomedical Imaging, University of California San Francisco (UCSF), San Francisco, CA, USA, 16: Biomedical Image Analysis and Machine Learning, Department of Quantitative Biomedicine, University of Zurich, Switzerland, 17: Department of Neuroradiology, Technical University of Munich, Munich, Germany, 18: University of San Diego, CA, USA, 19: Cornell University, Ithaca, NY, USA, 20: Department of Radiology, Northwestern University, Evanston, IL, USA, 21: Duke University Medical Center, School of Medicine, Durham, NC, USA, 22: Medical Artificial Intelligence (MAI) Lab, Crestview Radiology, Lagos, Nigeria, 23: Sage Bionetworks, USA, 24: Montreal Neurological Institute (MNI), McGill University, Montreal, QC, Canada, 25: Children’s National Hospital, Washington DC, USA, 26: George Washington University, Washington DC, USA, 27: Mayo Clinic, Rochester, MN, USA, 28: Center for Global Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA, 29: Athinoula A Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA, 30: Centre for Medical Image Computing, University College London, London, UK, 31: Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA, 32: PrecisionFDA, U.S. Food and Drug Administration, Silver Spring, MD, USA, 33: Department of Applied Mathematics and Computer Science, Technical University of Denmark, Denmark, 34: Athinoula A Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA, USA, 35: University of Zurich, Switzerland, 36: Booz Allen Hamilton, McLean, VA, USA, 37: Mercy Catholic Medical Center, Darby, PA, USA, 38: Penn Statistics in Imaging and Visualization Center, Department of Biostatistics, Epidemiology, and Informatics, 39: Greensboro Radiology, Greensboro, NC, USA, 40: Radiology Partners, El Segundo, CA, USA, 41: Duke University Medical Center, Department of Radiology, Durham, NC, USA, 42: Neosoma Inc. Stanford Medicine, Stanford, CA, USA, 43: Lagos University Teaching Hospital, Lagos Nigeria., 44: University of Zürich, Zürich,Switzerland, 45: Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA, 46: Department of Neuroradiology, Uppsala University, Sweden, 47: University of Toronto, Toronto, ON, Canada, 48: Cancer Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA, 49: Leidos Biomedical Research, Inc., Frederick National Laboratory for Cancer Research, Frederick, MD, USA, 50: Department of Radiology, Brigham and Women’s Hospital, Boston, MA, USA, 51: Institute of Neuroscience and Medicine (INM-4), Research Center Juelich, Juelich, Germany, 52: Department of Nuclear Medicine, University Hospital RWTH Aachen, Aachen, Germany, 53: Artificial Intelligence Lab, Department of Radiology, Mayo Clinic, Rochester, MN, USA, 54: Department of Neurosurgery, Indiana University, Indianapolis, IN, USA, 55: Department of Medical Image Computing, German Cancer Research Center (DKFZ), Heidelberg, Germany, 56: Department of Neuroradiology, University Hospital Bonn, Bonn Germany, 57: Department of Diagnostic and Interventional Radiology, University Hospital Ulm, Ulm, Germany, 58: Department of Diagnostic and Interventional Neuroradiology, School of Medicine, Klinikum rechts der Isar, Technical University of Munich, Munich, Germany, 59: TUM-Neuroimaging Center, Klinikum rechts der Isar, Technical University of Munich, Munich, Germany, 60: Fort Worth, TX, USA, 61: Department of Diagnostic and Interventional Radiology, All India Institute of Medical Sciences, Rishikesh, India, 62: Department of Neuroradiology, Washington University, St. Louis, MO, USA, 63: University of Rochester Medical Center, Rochester, NY, USA, 64: Faculty of Medicine, Jena University Hospital, Friedrich Schiller University Jena, Jena, Germany, 65: vRad (Radiology Partners), Minneapolis, MN, USA, 66: Ross University School of Medicine, Bridgetown, Barbados, 67: Department of Radiology, New York University Grossman School of Medicine, New York, NY, USA, 68: Department of Radiology, Arad Hospital, Tehran, Iran, 69: Department of Radiology, Faculty of Medicine, University of Ottawa, Cananda, 70: Department of Neuroradiology, JDMI, University of Toronto, TO, Canada, 71: Department of Radiology Northwestern University, Chicago, IL, 72: Department of Radiology, Azienda Ospedaliero Universitaria of Cagliari-Polo di Monserrato, Cagliari, Italy, 73: Department of Radiology, Leeds General Infirmary, Leeds, United Kingdom, 74: Ludwig Maximilians University, Munich, Bavaria, Germany, 75: Rush University Medical Center, Chicago, IL, USA, 76: Department of Neurosurgery, University Hospital of Ioannina, Ioannina, Greece, 77: Department of Neuroradiology, University Hospital Essen, Essen, North Rhine-Westphalia, Germany, 78: Egyptian Ministry of Health, Cairo, Egypt, 79: Department of Radiology, Kasr Alainy, Cairo University, Cairo, Egypt, 80: Faculty of Medicine, Medical University of Sofia, Sofia, Bulgaria, 81: Department of Radiology, American University of Beirut Medical center, Beirut, Lebanon, 82: Radiology Department, King Faisal Medical City, Abha, Saudi Arabia, 83: Department of Diagnostic and Interventional Radiology, Cairo University, Cairo, Egypt, 84: Department of Radiology and Medical Imaging, University of Virginia Health, Charlottesville, VA, USA, 85: Department of Radiology and Radiologic Sciences, Vanderbilt University Medical Center, TN, USA, 86: Faculty of Medicine, Hamburg University, Hamburg, Germany, 87: Neuroimaging Unit, ASST Ovest Milanese, Legnano, Milan, Italy, 88: University of Manitoba, Manitoba, Canada, 89: Department of Radiology, School of Medicine, Iran University of Medical Sciences, Iran, Tehran, 90: Department of Radiology, University Hospitals Bristol and Weston NHS Foundation Trust, Bristol, United Kingdom, 91: Department of Radiology, University of Texas Southwestern Medical Center, TX, USA, 92: CMH Lahore Medical College, Lahore, Pakistan, 93: MLCommons, 94: Factored AI, 95: Center For Federated Learning in Medicine, Indiana University, Indianapolis, IN, USA, 96: Medical Working Group, MLCommons, San Fransisco, CA, USA, 97: Intel, 98: Department of Radiology, Thomas Jefferson University, Philadelphia, PA, USA, 99: Department of Radiology and Imaging Sciences, Indiana University School of Medicine, Indianapolis, IN, USA, 100: Department of Neurological Surgery, Indiana University School of Medicine, Indianapolis, IN, USA, 101: Department of Biostatistics and Health Data Science, Indiana University School of Medicine, Indianapolis, IN, USA
Publication date: 2025/03/07
https://doi.org/10.59275/j.melba.2025-bea1
PDF · arXiv

Abstract

We describe the design and results from the BraTS 2023 Intracranial Meningioma Segmentation Challenge. The BraTS Meningioma Challenge differed from prior BraTS Glioma challenges in that it focused on meningiomas, which are typically benign extra-axial tumors with diverse radiologic and anatomical presentation and a propensity for multiplicity. Nine participating teams each developed deep-learning automated segmentation models using image data from the largest multi-institutional systematically expert annotated multilabel multi-sequence meningioma MRI dataset to date, which included 1000 training set cases, 141 validation set cases, and 283 hidden test set cases. Each case included T2, FLAIR, T1, and T1Gd brain MRI sequences with associated tumor compartment labels delineating enhancing tumor, non-enhancing tumor, and surrounding non-enhancing FLAIR hyperintensity. Participant automated segmentation models were evaluated and ranked based on a scoring system evaluating lesion-wise metrics including dice similarity coefficient (DSC) and 95% Hausdorff Distance. The top ranked team had a lesion-wise median dice similarity coefficient (DSC) of 0.976, 0.976, and 0.964 for enhancing tumor, tumor core, and whole tumor, respectively and a corresponding average DSC of 0.899, 0.904, and 0.871, respectively. These results serve as state-of-the-art benchmarks for future pre-operative meningioma automated segmentation algorithms. Additionally, we found that 1286 of 1424 cases (90.3%) had at least 1 compartment voxel abutting the edge of the skull-stripped image edge, which requires further investigation into optimal pre-processing face anonymization steps.

Keywords

Meningioma · BraTS · Machine Learning · Segmentation · BraTS-Meningioma · Image Analysis Challenge · artificial intelligence · AI

Bibtex @article{melba:2025:003:labella, title = "Analysis of the BraTS 2023 Intracranial Meningioma Segmentation Challenge", author = "LaBella, Dominic and Baid, Ujjwal and Khanna, Omaditya and McBurney-Lin, Shan and McLean, Ryan and Nedelec, Pierre and Rashid, Arif and Tahon, Nourel Hoda and Altes, Talissa and Bhalerao, Radhika and Dhemesh, Yaseen and Godfrey, Devon and Hilal, Fathi and Floyd, Scott and Janas, Anastasia and Kazerooni, Anahita Fathi and Kirkpatrick, John and Kent, Collin and Kofler, Florian and Leu, Kevin and Maleki, Nazanin and Menze, Bjoern and Pajot, Maxence and Reitman, Zachary J. and Rudie, Jeffrey D. and Saluja, Rachit and Velichko, Yury and Wang, Chunhao and Warman, Pranav and Adewole, Maruf and Albrecht, Jake and Anazodo, Udunna and Anwar, Syed Muhammad and Bergquist, Timothy and Chen, Sully Francis and Chung, Verena and Chai, Rong and Conte, Gian-Marco and Dako, Farouk and Eddy, James and Ezhov, Ivan and Khalili, Nastaran and Iglesias, Juan Eugenio and Jiang, Zhifan and Johanson, Elaine and Van Leemput, Koen and Li, Hongwei Bran and Linguraru, Marius George and Liu, Xinyang and Mahtabfar, Aria and Meier, Zeke and Moawad, Ahmed W and Mongan, John and Piraud, Marie and Shinohara, Russell Takeshi and Wiggins, Walter F. and Abayazeed, Aly H. and Akinola, Rachel and Jakab, András and Bilello, Michel and Correia de Verdier, Maria and Crivellaro, Priscila and Davatzikos, Christos and Farahani, Keyvan and Freymann, John and Hess, Christopher and Huang, Raymond and Lohmann, Philipp and Moassefi, Mana and Pease, Matthew W. and Vollmuth, Phillipp and Sollmann, Nico and Diffley, David and Nandolia, Khanak K. and Warren, Daniel I and Hussain, Ali and Fehringer, Pascal and Bronstein, Yulia and Deptula, Lisa and Stein, Evan G. and Taherzadeh, Mahsa and Portela de Oliveira, Eduardo and Haughey, Aoife and Kontzialis, Marinos and Saba, Luca and Turner, Benjamin and Brüßeler, Melanie M. T. and Ansari, Shehbaz and Gkampenis, Athanasios and Weiss, David Maximilian and Mansour, Aya and Shawali, Islam H. and Yordanov, Nikolay and Stein, Joel M. and Hourani, Roula and Moshebah, Mohammed Yahya and Abouelatta, Ahmed Magdy and Rizvi, Tanvir and Willms, Klara and Martin, Dann C. and Okar, Abdullah and D’Anna, Gennaro and Taha, Ahmed and Sharifi, Yasaman and Faghani, Shahriar and Kite, Dominic and Pinho, Marco and Haider, Muhammad Ammar and Aristizabal, Alejandro and Karargyris, Alexandros and Kassem, Hasan and Pati, Sarthak and Sheller, Micah and Alonso-Basanta, Michelle and Villanueva-Meyer, Javier and Rauschecker, Andreas M and Nada, Ayman and Aboian, Mariam and Flanders, Adam E. and Wiestler, Benedikt and Bakas, Spyridon and Calabrese, Evan", journal = "Machine Learning for Biomedical Imaging", volume = "3", issue = "March 2025 issue", year = "2025", pages = "38--58", issn = "2766-905X", doi = "https://doi.org/10.59275/j.melba.2025-bea1", url = "https://melba-journal.org/2025:003" }
RISTY - JOUR AU - LaBella, Dominic AU - Baid, Ujjwal AU - Khanna, Omaditya AU - McBurney-Lin, Shan AU - McLean, Ryan AU - Nedelec, Pierre AU - Rashid, Arif AU - Tahon, Nourel Hoda AU - Altes, Talissa AU - Bhalerao, Radhika AU - Dhemesh, Yaseen AU - Godfrey, Devon AU - Hilal, Fathi AU - Floyd, Scott AU - Janas, Anastasia AU - Kazerooni, Anahita Fathi AU - Kirkpatrick, John AU - Kent, Collin AU - Kofler, Florian AU - Leu, Kevin AU - Maleki, Nazanin AU - Menze, Bjoern AU - Pajot, Maxence AU - Reitman, Zachary J. AU - Rudie, Jeffrey D. AU - Saluja, Rachit AU - Velichko, Yury AU - Wang, Chunhao AU - Warman, Pranav AU - Adewole, Maruf AU - Albrecht, Jake AU - Anazodo, Udunna AU - Anwar, Syed Muhammad AU - Bergquist, Timothy AU - Chen, Sully Francis AU - Chung, Verena AU - Chai, Rong AU - Conte, Gian-Marco AU - Dako, Farouk AU - Eddy, James AU - Ezhov, Ivan AU - Khalili, Nastaran AU - Iglesias, Juan Eugenio AU - Jiang, Zhifan AU - Johanson, Elaine AU - Van Leemput, Koen AU - Li, Hongwei Bran AU - Linguraru, Marius George AU - Liu, Xinyang AU - Mahtabfar, Aria AU - Meier, Zeke AU - Moawad, Ahmed W AU - Mongan, John AU - Piraud, Marie AU - Shinohara, Russell Takeshi AU - Wiggins, Walter F. AU - Abayazeed, Aly H. AU - Akinola, Rachel AU - Jakab, András AU - Bilello, Michel AU - Correia de Verdier, Maria AU - Crivellaro, Priscila AU - Davatzikos, Christos AU - Farahani, Keyvan AU - Freymann, John AU - Hess, Christopher AU - Huang, Raymond AU - Lohmann, Philipp AU - Moassefi, Mana AU - Pease, Matthew W. AU - Vollmuth, Phillipp AU - Sollmann, Nico AU - Diffley, David AU - Nandolia, Khanak K. AU - Warren, Daniel I AU - Hussain, Ali AU - Fehringer, Pascal AU - Bronstein, Yulia AU - Deptula, Lisa AU - Stein, Evan G. AU - Taherzadeh, Mahsa AU - Portela de Oliveira, Eduardo AU - Haughey, Aoife AU - Kontzialis, Marinos AU - Saba, Luca AU - Turner, Benjamin AU - Brüßeler, Melanie M. T. AU - Ansari, Shehbaz AU - Gkampenis, Athanasios AU - Weiss, David Maximilian AU - Mansour, Aya AU - Shawali, Islam H. AU - Yordanov, Nikolay AU - Stein, Joel M. AU - Hourani, Roula AU - Moshebah, Mohammed Yahya AU - Abouelatta, Ahmed Magdy AU - Rizvi, Tanvir AU - Willms, Klara AU - Martin, Dann C. AU - Okar, Abdullah AU - D’Anna, Gennaro AU - Taha, Ahmed AU - Sharifi, Yasaman AU - Faghani, Shahriar AU - Kite, Dominic AU - Pinho, Marco AU - Haider, Muhammad Ammar AU - Aristizabal, Alejandro AU - Karargyris, Alexandros AU - Kassem, Hasan AU - Pati, Sarthak AU - Sheller, Micah AU - Alonso-Basanta, Michelle AU - Villanueva-Meyer, Javier AU - Rauschecker, Andreas M AU - Nada, Ayman AU - Aboian, Mariam AU - Flanders, Adam E. AU - Wiestler, Benedikt AU - Bakas, Spyridon AU - Calabrese, Evan PY - 2025 TI - Analysis of the BraTS 2023 Intracranial Meningioma Segmentation Challenge T2 - Machine Learning for Biomedical Imaging VL - 3 IS - March 2025 issue SP - 38 EP - 58 SN - 2766-905X DO - https://doi.org/10.59275/j.melba.2025-bea1 UR - https://melba-journal.org/2025:003 ER -
2025:003 cover