[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
IDEAS home Printed from https://ideas.repec.org/p/boe/boeewp/0984.html
   My bibliography  Save this paper

An interpretable machine learning workflow with an application to economic forecasting

Author

Listed:
  • Buckmann, Marcus

    (Bank of England)

  • Joseph, Andreas

    (Bank of England)

Abstract
We propose a generic workflow for the use of machine learning models to inform decision making and to communicate modelling results with stakeholders. It involves three steps: (1) a comparative model evaluation, (2) a feature importance analysis and (3) statistical inference based on Shapley value decompositions. We discuss the different steps of the workflow in detail and demonstrate each by forecasting changes in US unemployment one year ahead using the well-established FRED-MD dataset. We find that universal function approximators from the machine learning literature, including gradient boosting and artificial neural networks, outperform more conventional linear models. This better performance is associated with greater flexibility, allowing the machine learning models to account for time-varying and nonlinear relationships in the data generating process. The Shapley value decomposition identifies economically meaningful nonlinearities learned by the models. Shapley regressions for statistical inference on machine learning models enable us to assess and communicate variable importance akin to conventional econometric approaches. While we also explore high-dimensional models, our findings suggest that the best trade-off between interpretability and performance of the models is achieved when a small set of variables is selected by domain experts.

Suggested Citation

  • Buckmann, Marcus & Joseph, Andreas, 2022. "An interpretable machine learning workflow with an application to economic forecasting," Bank of England working papers 984, Bank of England.
    • Handle: RePEc:boe:boeewp:0984
    as

    Download full text from publisher

    File URL: https://www.bankofengland.co.uk/-/media/boe/files/working-paper/2022/an-interpretable-machine-learning-workflow-with-an-application-to-economic-forecasting.pdf
    File Function: Full text
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. David E. Runkle, 1998. "Revisionist history: how data revisions distort economic policy research," Quarterly Review, Federal Reserve Bank of Minneapolis, vol. 22(Fall), pages 3-12.
    2. Mukund Sundararajan & Amir Najmi, 2019. "The many Shapley values for model explanation," Papers 1908.08474, arXiv.org, revised Feb 2020.
    3. Sydney C. Ludvigson & Serena Ng, 2009. "A Factor Analysis of Bond Risk Premia," NBER Working Papers 15188, National Bureau of Economic Research, Inc.
    4. Marcelo C. Medeiros & Gabriel F. R. Vasconcelos & Álvaro Veiga & Eduardo Zilberman, 2021. "Forecasting Inflation in a Data-Rich Environment: The Benefits of Machine Learning Methods," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(1), pages 98-119, January.
    5. Serena Ng & Jonathan H. Wright, 2013. "Facts and Challenges from the Great Recession for Forecasting and Macroeconomic Modeling," Journal of Economic Literature, American Economic Association, vol. 51(4), pages 1120-1154, December.
    6. Stan Lipovetsky & Michael Conklin, 2001. "Analysis of regression in game theory approach," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 17(4), pages 319-330, October.
    7. Racine, Jeff, 2000. "Consistent cross-validatory model-selection for dependent data: hv-block cross-validation," Journal of Econometrics, Elsevier, vol. 99(1), pages 39-61, November.
    8. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    9. Pagan, Adrian, 1984. "Econometric Issues in the Analysis of Regressions with Generated Regressors," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 25(1), pages 221-247, February.
    10. Kim, Hyun Hak & Swanson, Norman R., 2018. "Mining big data using parsimonious factor, machine learning, variable selection and shrinkage methods," International Journal of Forecasting, Elsevier, vol. 34(2), pages 339-354.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lenza, Michele & Moutachaker, Inès & Paredes, Joan, 2023. "Density forecasts of inflation: a quantile regression forest approach," CEPR Discussion Papers 18298, C.E.P.R. Discussion Papers.
    2. Sambandh Bhusan Dhal & Debashish Kar, 2024. "Transforming Agricultural Productivity with AI-Driven Forecasting: Innovations in Food Security and Supply Chain Optimization," Forecasting, MDPI, vol. 6(4), pages 1-27, October.
    3. Kozyrev, Boris, 2024. "Forecast combination and interpretability using random subspace," IWH Discussion Papers 21/2024, Halle Institute for Economic Research (IWH).
    4. Alexandre Aspremont & Simon Ben Arous & Jean-Charles Bricongne & Benjamin Lietti & Baptiste Meunier, 2023. "Satellites Turn “Concrete”: Tracking Cement with Satellite Data and Neural Networks," Working papers 916, Banque de France.
    5. Kumar, Rishabh & Koshiyama, Adriano & da Costa, Kleyton & Kingsman, Nigel & Tewarrie, Marvin & Kazim, Emre & Roy, Arunita & Treleaven, Philip & Lovell, Zac, 2023. "Deep learning model fragility and implications for financial stability and regulation," Bank of England working papers 1038, Bank of England.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Joseph, Andreas & Potjagailo, Galina & Chakraborty, Chiranjit & Kapetanios, George, 2024. "Forecasting UK inflation bottom up," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1521-1538.
    2. Joseph, Andreas & Potjagailo, Galina & Chakraborty, Chiranjit & Kapetanios, George, 2024. "Forecasting UK inflation bottom up," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1521-1538.
    3. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    4. Jonas Krampe & Luca Margaritella, 2021. "Factor Models with Sparse VAR Idiosyncratic Components," Papers 2112.07149, arXiv.org, revised May 2022.
    5. Pierre‐Daniel Sarte, 2014. "When Is Sticky Information More Information?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 46(7), pages 1345-1379, October.
    6. Varlam Kutateladze, 2021. "The Kernel Trick for Nonlinear Factor Modeling," Papers 2103.01266, arXiv.org.
    7. Goulet Coulombe, Philippe & Leroux, Maxime & Stevanovic, Dalibor & Surprenant, Stéphane, 2021. "Macroeconomic data transformations matter," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1338-1354.
    8. Kutateladze, Varlam, 2022. "The kernel trick for nonlinear factor modeling," International Journal of Forecasting, Elsevier, vol. 38(1), pages 165-177.
    9. Daniel Wochner, 2020. "Dynamic Factor Trees and Forests – A Theory-led Machine Learning Framework for Non-Linear and State-Dependent Short-Term U.S. GDP Growth Predictions," KOF Working papers 20-472, KOF Swiss Economic Institute, ETH Zurich.
    10. Chen, Qitong & Hong, Yongmiao & Li, Haiqi, 2024. "Time-varying forecast combination for factor-augmented regressions with smooth structural changes," Journal of Econometrics, Elsevier, vol. 240(1).
    11. Simplice A. Asongu & Nicholas M. Odhiambo, 2019. "Governance, capital flight and industrialisation in Africa," Journal of Economic Structures, Springer;Pan-Pacific Association of Input-Output Studies (PAPAIOS), vol. 8(1), pages 1-22, December.
    12. Massacci, Daniele, 2017. "Least squares estimation of large dimensional threshold factor models," Journal of Econometrics, Elsevier, vol. 197(1), pages 101-129.
    13. Thomas Despois & Catherine Doz, 2023. "Identifying and interpreting the factors in factor models via sparsity: Different approaches," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(4), pages 533-555, June.
    14. Georges Bresson & Jean-Michel Etienne & Pierre Mohnen, 2011. "How important is innovation? A Bayesian factor-augmented productivity model on panel data," TEPP Working Paper 2011-06, TEPP.
    15. Simplice Asongu & Jacinta C Nwachukwu, 2015. "The incremental effect of education on corruption: evidence of synergy from lifelong learning," Economics Bulletin, AccessEcon, vol. 35(4), pages 2288-2308.
    16. Simplice A. Asongu & Joseph Nnanna, 2020. "Governance and the Capital Flight Trap in Africa," Working Papers of the African Governance and Development Institute. 20/024, African Governance and Development Institute..
    17. Rama K. Malladi, 2024. "Benchmark Analysis of Machine Learning Methods to Forecast the U.S. Annual Inflation Rate During a High-Decile Inflation Period," Computational Economics, Springer;Society for Computational Economics, vol. 64(1), pages 335-375, July.
    18. Asongu, Simplice & Nwachukwu, Jacinta, 2015. "Drivers of FDI in Fast Growing Developing Countries: Evidence from Bundling and Unbundling Governance," MPRA Paper 67294, University Library of Munich, Germany.
    19. Borup, Daniel & Christensen, Bent Jesper & Mühlbach, Nicolaj Søndergaard & Nielsen, Mikkel Slot, 2023. "Targeting predictors in random forest regression," International Journal of Forecasting, Elsevier, vol. 39(2), pages 841-868.
    20. Gonçalves, Sílvia & McCracken, Michael W. & Perron, Benoit, 2017. "Tests of equal accuracy for nested models with estimated factors," Journal of Econometrics, Elsevier, vol. 198(2), pages 231-252.

    More about this item

    Keywords

    machine learning; model interpretability; forecasting; unemployment; Shapley values;
    All these keywords.

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C38 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Classification Methdos; Cluster Analysis; Principal Components; Factor Analysis
    • C45 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Neural Networks and Related Topics
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C71 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Cooperative Games
    • E24 - Macroeconomics and Monetary Economics - - Consumption, Saving, Production, Employment, and Investment - - - Employment; Unemployment; Wages; Intergenerational Income Distribution; Aggregate Human Capital; Aggregate Labor Productivity

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:boe:boeewp:0984. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Digital Media Team (email available below). General contact details of provider: https://edirc.repec.org/data/boegvuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.