Abstract
Use of patient-reported measures (PRMs) in healthcare is increasing internationally. In New South Wales (NSW) Australia the implementation of a system-wide PRMs program is underway. This program is an enabler for value based healthcare. We report an evaluation (2015/16) of this program’s first year. We evaluated the program using mixed methods. People living with chronic conditions from 18 sites across NSW completed PROMIS10 at all sites; DASS21 and CAT administered at selected sites depending on patient cohorts. PRM completion rates and mean scores were calculated. Stakeholder interviews were analysed using thematic analysis. PRM completion rates were high at baseline (69%) but suboptimal at follow-up time-points. Mean scores indicated people with back pain had worse physical health and stress compared to those with other conditions. People with alcohol or drug problems had worse mental health, disease symptoms, depression and anxiety compared to other conditions. Stakeholders reported collection of PRMs could improve health outcomes for patients, enhance service delivery, and reduce avoidable hospital admissions. Routine collection and use of PRMs can be implemented across NSW care settings. Several challenges need to be overcome to realise the full benefits of the PRM program and to improve follow-up completion rates.
Similar content being viewed by others
Notes
Names of authors and funders replaced with XXX for blind manuscript
References
Aaronson, N.K., Elliott, T.E., Greenhalgh, J., Halyard, M.Y., Hess, R., Miller, D., Reeve, B.B., Santana, M., Snyder, C. (2015). User’s Guide to Implementing Patient-Reported Outcomes Assessment in Clinical Practice. http://www.isoqol.org/UserFiles/2015UsersGuide-Version2.pdf. Accessed 11 March 2019.
Agency for Clinical Innovation (2019): Patient Reported Measures: Outcomes that Matter to Patients. https://www.aci.health.nsw.gov.au/make-it-happen/prms. Accessed 11 March 2019.
Australian Bureau of Statistics (2019): 2017–18 National Health Survey (NHS). https://www.abs.gov.au/ausstats/abs@.nsf/mf/4364.0.55.001. Accessed 16 Oct 2019.
Australian Institute of Health and Welfare (2018): Mental health services—in brief 2018. https://www.aihw.gov.au/getmedia/0e102c2f-694b-4949-84fb-e5db1c941a58/aihw-hse-211.pdf.aspx?inline=true. Accessed 16 Oct 2019.
Barbera, L., Sutradhar, R., Howell, D., Sussman, J., Seow, H., Dudgeon, D., Atzema, C., Earle, C., Husain, A., Liu, Y., & Krzyzanowska, M. K. (2015). Does routine symptom screening with ESAS decrease ED visits in breast cancer patients undergoing adjuvant chemotherapy? Supportive Care in Cancer, 23(10), 3025–3032. https://doi.org/10.1007/s00520-015-2671-3.
Basch, E., Deal, A. M., Dueck, A. C., Scher, H. I., Kris, M. G., Hudis, C., & Schrag, D. (2017). Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine Cancer treatment. JAMA, 318(2), 197–198. https://doi.org/10.1001/jama.2017.7156.
Cella, D., Hahn, E.A., Jensen, S.E., et al. (2015): Patient-Reported Outcomes in Performance Measurement. In: Method and Mode of Administration, Data Collection, and Analysis. RTI Press, Research Triangle Park (NC).
Chen, J., Ou, L., & Hollis, S. J. (2013). A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Services Research, 13, 211. https://doi.org/10.1186/1472-6963-13-211.
Cheung, N.W., Crampton, M., Nesire, V., Hng, T.-M., Chow, C.K., on behalf of the Western Sydney Integrated Care Program Investigators (2019): Model for integrated care for chronic disease in the Australian context: Western Sydney Integrated Care Program. Australian Health Review. https://doi.org/10.1071/AH18152
Cleeland, C. S., Wang, X. S., Shi, Q., Mendoza, T. R., Wright, S. L., Berry, M. D., Malveaux, D., Shah, P. K., Gning, I., Hofstetter, W. L., Putnam Jr., J. B., & Vaporciyan, A. A. (2011). Automated symptom alerts reduce postoperative symptom severity after cancer surgery: A randomized controlled clinical trial. Journal of Clinical Oncology : Official Journal of the American Society of Clinical Oncology, 29(8), 994–1000. https://doi.org/10.1200/jco.2010.29.8315.
Denis, F., Lethrosne, C., Pourel, N., Molinier, O., Pointreau, Y., Domont, J., Bourgeois, H., Senellart, H., Trémolières, P., Lizée, T., Bennouna, J., Urban, T., El Khouri, C., Charron, A., Septans, A.-L., Balavoine, M., Landry, S., Solal-Céligny, P., Letellier, C. (2017). Randomized Trial Comparing a Web-Mediated Follow-up With Routine Surveillance in Lung Cancer Patients. Journal of the National Cancer Institute 109(9). https://doi.org/10.1093/jnci/djx029
Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., & Sutton, A. (2005). Synthesising qualitative and quantitative evidence: A review of possible methods. Journal of Health Services Research & Policy, 10(1), 45–53. https://doi.org/10.1177/135581960501000110.
Fairclough, D. L., Peterson, H. F., Cella, D., & Bonomi, P. (1998a). Comparison of several model-based methods for analysing incomplete quality of life data in cancer clinical trials. Statistics in Medicine, 17(5–7), 781–796.
Fairclough, D. L., Peterson, H. F., & Chang, V. (1998b). Why are missing quality of life data a problem in clinical trials of cancer therapy? Statistics in Medicine, 17(5–7), 667–677.
Foster, A., Croot, L., Brazier, J., Harris, J., & O'Cathain, A. (2018). The facilitators and barriers to implementing patient reported outcome measures in organisations delivering health related services: A systematic review of reviews. Journal of Patient-Reported Outcomes, 2, 46. https://doi.org/10.1186/s41687-018-0072-3.
Glickman, S. W., Ndubuizu, A., Weinfurt, K. P., Hamilton, C. D., Glickman, L. T., Schulman, K. A., & Cairns, C. B. (2011). Perspective: The case for research justice: Inclusion of patients with limited English proficiency in clinical research. Academic Medicine : Journal of the Association of American Medical Colleges, 86(3), 389–393. https://doi.org/10.1097/ACM.0b013e318208289a.
Goldstein, D., Bell, M. L., Butow, P., Sze, M., Vaccaro, L., Dong, S., Liauw, W., Hui, R., Tattersall, M., Ng, W., Asghari, R., Steer, C., Vardy, J., Parente, P., Harris, M., Karanth, N. V., King, M., Girgis, A., Eisenbruch, M., & Jefford, M. (2014). Immigrants' perceptions of the quality of their cancer care: an Australian comparative study, identifying potentially modifiable factors. Annals of Oncology: Official Journal of the European Society for Medical Oncology, 25(8), 1643–1649. https://doi.org/10.1093/annonc/mdu182.
Greenhalgh, J., & Meadows, K. (1999). The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: A literature review. Journal of Evaluation in Clinical Practice, 5(4), 401–416.
Greenhalgh, T., Jackson, C., Shaw, S., & Janamian, T. (2016). Achieving research impact through co-creation in community-based health services: Literature review and case study. The Milbank Quarterly, 94(2), 392–429. https://doi.org/10.1111/1468-0009.12197.
Greenhalgh, J., Dalkin, S., Gooding, K., Gibbons, E., Wright, J., Meads, D., Black, N., Valderas, J.M., Pawson, R. (2017): Functionality and feedback: a realist synthesis of the collation, interpretation and utilisation of patient-reported outcome measures data to improve patient care. Health Services and Delivery Research, 5(2). doi:https://doi.org/10.3310/hsdr05020
Hays, R. D., Bjorner, J. B., Revicki, D. A., Spritzer, K. L., & Cella, D. (2009). Development of physical and mental health summary scores from the patient-reported outcomes measurement information system (PROMIS) global items. Quality of Life Research: an International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 18(7), 873–880. https://doi.org/10.1007/s11136-009-9496-9.
Health and Social Policy (2018): NSW Health Strategic Framework for Integrating Care. https://www.health.nsw.gov.au/integratedcare/Publications/strategic-framework-for-integrating-care.PDF. Accessed 11 March 2019.
Ishaque, S., Karnon, J., Chen, G., Nair, R., & Salter, A. B. (2019). A systematic review of randomised controlled trials evaluating the use of patient-reported outcome measures (PROMs). Quality of Life Research: an International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 28(3), 567–592. https://doi.org/10.1007/s11136-018-2016-z.
Jang, S.M., Kim, K.U., Na, H.J., Song, S.E., Lee, S.H., Lee, H., Kim, Y.S., Lee, M.K., Park, H.-K. (2019). Depression is a major determinant of both disease-specific and generic health-related quality of life in people with severe COPD. Chronic Respiratory Disease 16. https://doi.org/10.1177/1479972318775422
Jones, P. W., Harding, G., Berry, P., Wiklund, I., Chen, W. H., & Kline Leidy, N. (2009). Development and first validation of the COPD assessment test. The European Respiratory Journal, 34(3), 648–654. https://doi.org/10.1183/09031936.00102509.
Lovibond, S. H., & Lovibond, P. F. (1995). Manual for the depression anxiety stress scales (2nd ed.). Sydney: Psychology Foundation.
Mejdahl, C. T., Schougaard, L. M. V., Hjollund, N. H., Riiskjaer, E., & Lomborg, K. (2018). Exploring organisational mechanisms in PRO-based follow-up in routine outpatient care - an interpretive description of the clinician perspective. BMC Health Services Research, 18(1), 546. https://doi.org/10.1186/s12913-018-3352-y.
Mercieca-Bebber, R., Palmer, M. J., Brundage, M., Calvert, M., Stockler, M. R., & King, M. T. (2016). Design, implementation and reporting strategies to reduce the instance and impact of missing patient-reported outcome (PRO) data: A systematic review. BMJ Open, 6(6), e010938. https://doi.org/10.1136/bmjopen-2015-010938.
NSW Ministry of Health (2018): NSW integrated care journey. https://www.health.nsw.gov.au/integratedcare/Pages/Our-Plan.aspx. Accessed 11 March 2019.
OECD (2010): Improving value in health care. https://doi.org/10.1787/9789264094819-en.
Porter, I., Goncalves-Bradley, D., Ricci-Cabello, I., Gibbons, C., Gangannagaripalli, J., Fitzpatrick, R., Black, N., Greenhalgh, J., & Valderas, J. M. (2016). Framework and guidance for implementing patient-reported outcomes in clinical practice: Evidence, challenges and opportunities. Journal of Comparative Effectiveness Research, 5(5), 507–519. https://doi.org/10.2217/cer-2015-0014.
Snyder, C. F., Aaronson, N. K., Choucair, A. K., Elliott, T. E., Greenhalgh, J., Halyard, M. Y., Hess, R., Miller, D. M., Reeve, B. B., & Santana, M. (2012). Implementing patient-reported outcomes assessment in clinical practice: A review of the options and considerations. Quality of Life Research: an International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 21(8), 1305–1314. https://doi.org/10.1007/s11136-011-0054-x.
Solano, J. P., Gomes, B., & Higginson, I. J. (2006). A comparison of symptom prevalence in far advanced cancer, AIDS, heart disease, chronic obstructive pulmonary disease and renal disease. Journal of Pain and Symptom Management, 31(1), 58–69.
Stanton, R., Rosenbaum, S., Rebar, A., & Happell, B. (2019). Prevalence of chronic health conditions in Australian adults with depression and/or anxiety. Issues in Mental Health Nursing, 40(10), 902–907. https://doi.org/10.1080/01612840.2019.1613701.
Sze, M., Butow, P., Bell, M., Vaccaro, L., Dong, S., Eisenbruch, M., Jefford, M., Girgis, A., King, M., McGrane, J., Ng, W., Asghari, R., Parente, P., Liauw, W., Goldstein, D., & Psycho-oncology co-operative research group, C., linguistically diverse, T. (2015). Migrant health in cancer: Outcome disparities and the determinant role of migrant-specific variables. Oncologist, 20(5), 523–531. https://doi.org/10.1634/theoncologist.2014-0274.
Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8, 45. https://doi.org/10.1186/1471-2288-8-45.
Velikova, G., Booth, L., Smith, A. B., Brown, P. M., Lynch, P., Brown, J. M., & Selby, P. J. (2004). Measuring quality of life in routine oncology practice improves communication and patient well-being: A randomized controlled trial. Journal of Clinical Oncology : Official Journal of the American Society of Clinical Oncology, 22(4), 714–724. https://doi.org/10.1200/jco.2004.06.078.
Funding
This study was sponsored by the Agency of Clinical Innovation (ACI) with funding from the NSW Department of Health.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
ESM 1
(DOCX 13 kb)
Rights and permissions
About this article
Cite this article
Rutherford, C., Campbell, R., Tinsley, M. et al. Implementing Patient-Reported Outcome Measures into Clinical Practice Across NSW: Mixed Methods Evaluation of the First Year. Applied Research Quality Life 16, 1265–1284 (2021). https://doi.org/10.1007/s11482-020-09817-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11482-020-09817-2