[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Analysis
  • Published:

Crowdsourced analysis of clinical trial data to predict amyotrophic lateral sclerosis progression

Nature Biotechnology volume 33, pages 51–57 (2015)Cite this article

Subjects

Abstract

Amyotrophic lateral sclerosis (ALS) is a fatal neurodegenerative disease with substantial heterogeneity in its clinical presentation. This makes diagnosis and effective treatment difficult, so better tools for estimating disease progression are needed. Here, we report results from the DREAM-Phil Bowen ALS Prediction Prize4Life challenge. In this crowdsourcing competition, competitors developed algorithms for the prediction of disease progression of 1,822 ALS patients from standardized, anonymized phase 2/3 clinical trials. The two best algorithms outperformed a method designed by the challenge organizers as well as predictions by ALS clinicians. We estimate that using both winning algorithms in future trial designs could reduce the required number of patients by at least 20%. The DREAM-Phil Bowen ALS Prediction Prize4Life challenge also identified several potential nonstandard predictors of disease progression including uric acid, creatinine and surprisingly, blood pressure, shedding light on ALS pathobiology. This analysis reveals the potential of a crowdsourcing competition that uses clinical trial data for accelerating ALS research and development.

Access through your institution
Buy or subscribe

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Challenge outline.
Figure 2: Performance of methods.
Figure 3: Prediction and classification by algorithms and clinicians.
Figure 4: Analysis of predictive features.

Similar content being viewed by others

References

  1. Drigo, D. et al. The incidence of amyotrophic lateral sclerosis in Friuli Venezia Giulia, Italy, from 2002 to 2009: a retrospective population-based study. Neuroepidemiology 41, 54–61 (2013).

    Article  Google Scholar 

  2. Johnston, C.A. et al. Amyotrophic lateral sclerosis in an urban setting: a population based study of inner city London. J. Neurol. 253, 1642–1643 (2006).

    Article  Google Scholar 

  3. Kiernan, M.C. et al. Amyotrophic lateral sclerosis. Lancet 377, 942–955 (2011).

    Article  CAS  Google Scholar 

  4. Miller, R., Mitchell, J., Lyon, M. & Moore, D. Riluzole for amyotrophic lateral sclerosis (ALS)/motor neuron disease (MND). Cochrane Database Syst. Rev. CD001447 (2007).

  5. Cedarbaum, J.M. & Stambler, N. Performance of the amyotrophic lateral sclerosis functional rating scale (ALSFRS) in multicenter clinical trials. J. Neurol. Sci. 152, S1–S9 (1997).

    Article  Google Scholar 

  6. Cedarbaum, J.M. et al. The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respiratory function. J. Neurol. Sci. 169, 13–21 (1999).

    Article  CAS  Google Scholar 

  7. Grossman, I. et al. Alzheimer's disease: diagnostics, prognostics and the road to prevention. EPMA J. 1, 293–303 (2010).

    Article  Google Scholar 

  8. Tangri, N. et al. A predictive model for progression of chronic kidney disease to kidney failure. J. Am. Med. Assoc. 305, 1553–1559 (2011).

    Article  CAS  Google Scholar 

  9. Cutter, G.R. et al. Development of a multiple sclerosis functional composite as a clinical trial outcome measure. Brain 122, 871–882 (1999).

    Article  Google Scholar 

  10. Romero, K. et al. The coalition against major diseases: developing tools for an integrated drug development process for Alzheimer's and Parkinson's diseases. Clin. Pharmacol. Ther. 86, 365–367 (2009).

    Article  CAS  Google Scholar 

  11. Ontaneda, D., LaRocca, N., Coetzee, T. & Rudick, R. Revisiting the multiple sclerosis functional composite: proceedings from the national multiple sclerosis society (NMSS) task force on clinical disability measures. Mult. Scler. 18, 1074–1080 (2012).

    Article  CAS  Google Scholar 

  12. Rogers, J.A. et al. Combining patient-level and summary-level data for Alzheimer's disease modeling and simulation: a beta regression meta-analysis. J. Pharmacokinet. Pharmacodyn. 39, 479–498 (2012).

    Article  Google Scholar 

  13. Marbach, D. et al. Wisdom of crowds for robust gene network inference. Nat. Methods 9, 796–804 (2012).

    Article  CAS  Google Scholar 

  14. R Core Team. R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, Vienna, Austria, 2013).

  15. Hothorn, T. & Jung, H.H. RandomForest4Life: a random forest for predicting ALS disease progression. Amyotroph. Lateral Scler. Frontotemporal Degener. 15, 444–452 (2014).

    Article  Google Scholar 

  16. Costello, J. & Stolovitzky, G. Seeking the wisdom of crowds through challenge-based competitions in biomedical research. Clin. Pharmacol. Ther. 93, 396–398 (2013).

    Article  CAS  Google Scholar 

  17. Magnus, T. et al. Disease progression in amyotrophic lateral sclerosis: predictors of survival. Muscle Nerve 25, 709–714 (2002).

    Article  CAS  Google Scholar 

  18. del Aguila, M., Longstreth, W., McGuire, V., Koepsell, T. & Van Belle, G. Prognosis in amyotrophic lateral sclerosis: a population-based study. Neurology 60, 813–819 (2003).

    Article  CAS  Google Scholar 

  19. Paganoni, S., Deng, J., Jaffa, M., Cudkowicz, M.E. & Wills, A.M. Body mass index, not dyslipidemia, is an independent predictor of survival in amyotrophic lateral sclerosis. Muscle Nerve 44, 20–24 (2011).

    Article  Google Scholar 

  20. Paganoni, S. et al. Uric acid levels predict survival in men with amyotrophic lateral sclerosis. J. Neurol. 259, 1923–1928 (2012).

    Article  CAS  Google Scholar 

  21. Gomeni, R. & Fava, M. The pooled resource open-access, ALSCTC amyotrophic lateral sclerosis disease progression model. Amyotroph. Lateral Scler. Frontotemporal Degener. 15, 119–129 (2014).

    Article  Google Scholar 

  22. Lakhani, K.R. et al. Prize-based contests can provide solutions to computational biology problems. Nat. Biotechnol. 31, 108–111 (2013).

    Article  CAS  Google Scholar 

  23. Chiò, A. et al. ALS clinical trials: do enrolled patients accurately represent the ALS population? Neurology 77, 1432–1437 (2011).

    Article  Google Scholar 

  24. Czaplinski, A., Yen, A.A., Simpson, E.P. & Appel, S.H. Predictability of disease progression in amyotrophic lateral sclerosis. Muscle Nerve 34, 702–708 (2006).

    Article  Google Scholar 

  25. Pastula, D.M. et al. Factors associated with survival in the national registry of veterans with ALS. Amyotroph. Lateral Scler. 10, 332–338 (2009).

    Article  Google Scholar 

  26. Qureshi, M.M. et al. Analysis of factors that modify susceptibility and rate of progression in amyotrophic lateral sclerosis (ALS). Amyotroph. Lateral Scler. 7, 173–182 (2006).

    Article  Google Scholar 

  27. Fujimura-Kiyono, C. et al. Onset and spreading patterns of lower motor neuron involvements predict survival in sporadic amyotrophic lateral sclerosis. J. Neurol. Neurosurg. Psychiatry 82, 1244–1249 (2011).

    Article  Google Scholar 

  28. Qureshi, M. et al. Medications and laboratory parameters as prognostic factors in amyotrophic lateral sclerosis. Amyotroph. Lateral Scler. 9, 369–374 (2008).

    Article  CAS  Google Scholar 

  29. Zoccolella, S. et al. Analysis of survival and prognostic factors in amyotrophic lateral sclerosis: a population based study. J. Neurol. Neurosurg. Psychiatry 79, 33–37 (2008).

    Article  CAS  Google Scholar 

  30. Turner, M.R. et al. Pattern of spread and prognosis in lower limb-onset ALS. Amyotroph. Lateral Scler. 11, 369–373 (2010).

    Article  Google Scholar 

  31. Kollewe, K. et al. ALSFRS-R score and its ratio: a useful predictor for ALS-progression. J. Neurol. Sci. 275, 69–73 (2008).

    Article  Google Scholar 

  32. Traynor, B., Zhang, H., Shefner, J., Schoenfeld, D. & Cudkowicz, M. Functional outcome measures as clinical trial endpoints in ALS. Neurology 63, 1933–1935 (2004).

    Article  CAS  Google Scholar 

  33. Vender, R.L., Mauger, D., Walsh, S., Alam, S. & Simmons, Z. Respiratory systems abnormalities and clinical milestones for patients with amyotrophic lateral sclerosis with emphasis upon survival. Amyotroph. Lateral Scler. 8, 36–41 (2007).

    Article  Google Scholar 

  34. Paganoni, S., Deng, J., Jaffa, M., Cudkowicz, M.E. & Wills, A.M. What does body mass index measure in amyotrophic lateral sclerosis and why should we care? Muscle Nerve 45, 612 (2012).

    Article  Google Scholar 

  35. Reich-Slotky, R. et al. Body mass index (BMI) as predictor of ALSFRS-R score decline in ALS patients. Amyotroph. Lateral Scler. Frontotemporal Degener. 14, 212–216 (2013).

    Article  Google Scholar 

  36. Ascherio, A. et al. Urate as a predictor of the rate of clinical decline in Parkinson disease. Arch. Neurol. 66, 1460–1468 (2009).

    Article  Google Scholar 

  37. Auinger, P., Kieburtz, K. & Mcdermott, M.P. The relationship between uric acid levels and Huntington's disease progression. Mov. Disord. 25, 224–228 (2010).

    Article  Google Scholar 

  38. de Lau, L.M., Koudstaal, P.J., Hofman, A. & Breteler, M. Serum uric acid levels and the risk of Parkinson disease. Ann. Neurol. 58, 797–800 (2005).

    Article  CAS  Google Scholar 

  39. Euser, S., Hofman, A., Westendorp, R. & Breteler, M.M. Serum uric acid and cognitive function and dementia. Brain 132, 377–382 (2009).

    Article  CAS  Google Scholar 

  40. Ikeda, K., Hirayama, T., Takazawa, T., Kawabe, K. & Iwasaki, Y. Relationships between disease progression and serum levels of lipid, urate, creatinine and ferritin in Japanese patients with amyotrophic lateral sclerosis: a cross-sectional study. Intern. Med. 51, 1501–1508 (2012).

    Article  CAS  Google Scholar 

  41. Chen, X. et al. An exploratory study of serum creatinine levels in patients with amyotrophic lateral sclerosis. Neurol. Sci. 35, 1591–1597 (2014).

    Article  CAS  Google Scholar 

  42. Chahin, N. & Sorenson, E.J. Serum creatine kinase levels in spinobulbar muscular atrophy and amyotrophic lateral sclerosis. Muscle Nerve 40, 126–129 (2009).

    Article  CAS  Google Scholar 

  43. Rafiq, M., Lee, E., Bradburn, M., McDermott, C. & Shaw, P. Elevated creatine kinase suggests better prognosis in patients with amyotrophic lateral sclerosis. J. Neurol. Neurosurg. Psychiatry 84, e2 (2013).

    Article  Google Scholar 

  44. Murata, Y., Harada, T., Ishizaki, F., Izumi, Y. & Nakamura, S. An abnormal relationship between blood pressure and pulse rate in amyotrophic lateral sclerosis. Acta Neurol. Scand. 96, 118–122 (1997).

    Article  CAS  Google Scholar 

  45. Baltadzhieva, R., Gurevich, T. & Korczyn, A.D. Autonomic impairment in amyotrophic lateral sclerosis. Curr. Opin. Neurol. 18, 487–493 (2005).

    Article  Google Scholar 

  46. Kandinov, B., Drory, V.E., Tordjman, K. & Korczyn, A.D. Blood pressure measurements in a transgenic SOD1–G93A mouse model of amyotrophic lateral sclerosis. Amyotroph. Lateral Scler. 13, 509–513 (2012).

    Article  CAS  Google Scholar 

  47. Pavlovic, S. et al. Impairment of cardiac autonomic control in patients with amyotrophic lateral sclerosis. Amyotroph. Lateral Scler. 11, 272–276 (2010).

    Article  Google Scholar 

  48. Miller, R.G. et al. Phase II screening trial of lithium carbonate in amyotrophic lateral sclerosis: examining a more efficient trial design. Neurology 77, 973–979 (2011).

    Article  CAS  Google Scholar 

  49. Atassi, N. et al. The PRO-ACT Database: Design, initial analyses, and predictive features. Neurology 10.1212/WNL.0000000000000951 (8 October 2014).

Download references

Acknowledgements

We are grateful to the following people for their important assistance with this manuscript: the pharmaceutical companies which provided data to the PRO-ACT data set that enabled this entire endeavor, the Bowen family and D. Lautman for their generous support for this project, R. Betensky from the Harvard School of Public Health for her statistical advice and contributions to the earliest stage of developing the challenge, L. Reinhold from InnoCentive for her support and management of the challenge, our challenge sponsors, Nature, The Economist and Popular Science, the clinicians who participated in the clinicians' assessment and follow-on discussions about the results, L.A. White and D. Kerr from Biogen Idec for their assistance with estimating the financial impact of the algorithms and, of course, the solvers who participated in the challenge and the patients who inspired this effort.

Author information

Author notes

  1. Robert Küffner and Neta Zach: These authors contributed equally to this work.

Authors and Affiliations

  1. Institute of Bioinformatics and Systems Biology, German Research Center for Environmental Health, Munich, Germany

    Robert Küffner

  2. Department of Informatics, Ludwig-Maximilians-University, Munich, Germany

    Robert Küffner & Johann Hawe

  3. Prize4Life, Tel Aviv, Israel and Cambridge, Massachusetts, USA.,

    Neta Zach & Melanie L Leitner

  4. IBM T.J. Watson Research Center, Yorktown Heights, New York, USA

    Raquel Norel & Gustavo Stolovitzky

  5. MGH Biostatistics Center, Massachusetts General Hospital, Boston, Massachusetts, USA

    David Schoenfeld

  6. Harvard Medical School, Charlestown, Massachusetts, USA

    David Schoenfeld

  7. Sentrana Inc., Washington, DC, USA

    Liuxia Wang & Guang Li

  8. Latham&Watkins LLP, Silicon Valley, California, USA

    Lilly Fang

  9. Department of Statistics, Stanford University, Stanford, California, USA

    Lester Mackey

  10. Department of Neuroscience, Beaumont Hospital and Trinity College Dublin, Dublin, Ireland

    Orla Hardiman

  11. Neurological Clinical Research Institute, Massachusetts General Hospital, Charlestown, Massachusetts, USA

    Merit Cudkowicz & Alexander Sherman

  12. Luxembourg Centre for Systems Biomedicine, University of Luxembourg, Esch/Alzette, Luxembourg

    Gokhan Ertaylan

  13. Max Planck Institute for Intelligent Systems, Tübingen, Germany

    Moritz Grosse-Wentrup, Timm Meyer & Bernhard Schölkopf

  14. Institute of Social- and Preventive Medicine, University of Zürich, Zürich, Switzerland

    Torsten Hothorn

  15. Orca XL Problem Solvers, Amsterdam, the Netherlands

    Jules van Ligtenberg & Rubio Vaughan

  16. Max Planck Institute for Biological Cybernetics and Bernstein Center for Computational Neuroscience, Tübingen, Germany

    Jakob H Macke

  17. Berkeley School of Public Health, University of California, Berkeley, California, USA

    Linh Tran

  18. ALS Innovation Hub, Biogen Idec, Cambridge, Massachusetts, USA

    Melanie L Leitner

Authors
  1. Robert Küffner
    View author publications

    Search author on:PubMed Google Scholar

  2. Neta Zach
    View author publications

    Search author on:PubMed Google Scholar

  3. Raquel Norel
    View author publications

    Search author on:PubMed Google Scholar

  4. Johann Hawe
    View author publications

    Search author on:PubMed Google Scholar

  5. David Schoenfeld
    View author publications

    Search author on:PubMed Google Scholar

  6. Liuxia Wang
    View author publications

    Search author on:PubMed Google Scholar

  7. Guang Li
    View author publications

    Search author on:PubMed Google Scholar

  8. Lilly Fang
    View author publications

    Search author on:PubMed Google Scholar

  9. Lester Mackey
    View author publications

    Search author on:PubMed Google Scholar

  10. Orla Hardiman
    View author publications

    Search author on:PubMed Google Scholar

  11. Merit Cudkowicz
    View author publications

    Search author on:PubMed Google Scholar

  12. Alexander Sherman
    View author publications

    Search author on:PubMed Google Scholar

  13. Gokhan Ertaylan
    View author publications

    Search author on:PubMed Google Scholar

  14. Moritz Grosse-Wentrup
    View author publications

    Search author on:PubMed Google Scholar

  15. Torsten Hothorn
    View author publications

    Search author on:PubMed Google Scholar

  16. Jules van Ligtenberg
    View author publications

    Search author on:PubMed Google Scholar

  17. Jakob H Macke
    View author publications

    Search author on:PubMed Google Scholar

  18. Timm Meyer
    View author publications

    Search author on:PubMed Google Scholar

  19. Bernhard Schölkopf
    View author publications

    Search author on:PubMed Google Scholar

  20. Linh Tran
    View author publications

    Search author on:PubMed Google Scholar

  21. Rubio Vaughan
    View author publications

    Search author on:PubMed Google Scholar

  22. Gustavo Stolovitzky
    View author publications

    Search author on:PubMed Google Scholar

  23. Melanie L Leitner
    View author publications

    Search author on:PubMed Google Scholar

Contributions

R.K., N.Z., R.N., J.H., D.S., O.H., M.C., G.S. and M.L.L. designed the challenge. R.K. and J.H. prepared the data and baseline algorithm, A.S. helped with data preparation. N.Z. managed the challenge. L.W., G.L., L.F., L.M., G.E., M.G.-W., T.H., J.v.L., J.H.M., T.M., B.S., L.T. and R.V. submitted algorithms. D.S., L.W., G.L., L.F. and L.M. contributed further analysis on challenge performance. R.K. and N.Z. analyzed the results and wrote the paper.

Corresponding authors

Correspondence to Robert Küffner or Neta Zach.

Ethics declarations

Competing interests

R.N. and G.S. are employees of IBM; L.F. is an employee of Latham&Watkins; R.V. is an employee of Orca XL Problem Solvers; M.L.L. is an employee of Biogen Idec; L.W. and G.L. are employees of Sentrana.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–13, Supplementary Tables 1–10, Supplementary Notes 1–3, Supplementary Results 1–5 and Supplementary Data 1 (PDF 4479 kb)

Supplementary Software

Algorithms participating (ZIP 153886 kb)

Supplementary Predictions

Predictions made by algorithms participating (ZIP 75 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Küffner, R., Zach, N., Norel, R. et al. Crowdsourced analysis of clinical trial data to predict amyotrophic lateral sclerosis progression. Nat Biotechnol 33, 51–57 (2015). https://doi.org/10.1038/nbt.3051

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nbt.3051

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing