[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1803.10883.html
   My bibliography  Save this paper

Tests for Forecast Instability and Forecast Failure under a Continuous Record Asymptotic Framework

Author

Listed:
  • Alessandro Casini
Abstract
We develop a novel continuous-time asymptotic framework for inference on whether the predictive ability of a given forecast model remains stable over time. We formally define forecast instability from the economic forecaster's perspective and highlight that the time duration of the instability bears no relationship with stable period. Our approach is applicable in forecasting environment involving low-frequency as well as high-frequency macroeconomic and financial variables. As the sampling interval between observations shrinks to zero the sequence of forecast losses is approximated by a continuous-time stochastic process (i.e., an Ito semimartingale) possessing certain pathwise properties. We build an hypotheses testing problem based on the local properties of the continuous-time limit counterpart of the sequence of losses. The null distribution follows an extreme value distribution. While controlling the statistical size well, our class of test statistics feature uniform power over the location of the forecast failure in the sample. The test statistics are designed to have power against general form of insatiability and are robust to common forms of non-stationarity such as heteroskedasticty and serial correlation. The gains in power are substantial relative to extant methods, especially when the instability is short-lasting and when occurs toward the tail of the sample.

Suggested Citation

  • Alessandro Casini, 2018. "Tests for Forecast Instability and Forecast Failure under a Continuous Record Asymptotic Framework," Papers 1803.10883, arXiv.org, revised Dec 2018.
    • Handle: RePEc:arx:papers:1803.10883
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1803.10883
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Wei Biao Wu & Zhibiao Zhao, 2007. "Inference of trends in time series," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(3), pages 391-410, June.
    2. Jushan Bai & Pierre Perron, 1998. "Estimating and Testing Linear Models with Multiple Structural Changes," Econometrica, Econometric Society, vol. 66(1), pages 47-78, January.
    3. Clements, Michael P. & Hendry, David F., 2006. "Forecasting with Breaks," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 12, pages 605-657, Elsevier.
    4. Graham Elliott & Allan Timmermann, 2016. "Forecasting in Economics and Finance," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 81-110, October.
    5. Cheung, Yin-Wong & Chinn, Menzie David & Garcia Pascual, Antonio, 2003. "Empirical Exchange Rate Models of the Nineties: Are Any Fit to Survive?," Santa Cruz Department of Economics, Working Paper Series qt12z9x4c5, Department of Economics, UC Santa Cruz.
    6. Yacine Aït-Sahalia & Jean Jacod, 2012. "Analyzing the Spectrum of Asset Returns: Jump and Volatility Components in High Frequency Data," Journal of Economic Literature, American Economic Association, vol. 50(4), pages 1007-1050, December.
    7. Raffaella Giacomini & Barbara Rossi, 2009. "Detecting and Predicting Forecast Breakdowns," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(2), pages 669-705.
    8. Hansen, Bruce E., 2000. "Testing for structural change in conditional models," Journal of Econometrics, Elsevier, vol. 97(1), pages 93-115, July.
    9. Deng, Ai & Perron, Pierre, 2008. "A non-local perspective on the power properties of the CUSUM and CUSUM of squares tests for structural change," Journal of Econometrics, Elsevier, vol. 142(1), pages 212-240, January.
    10. Fossati, Sebastian, 2017. "Testing for State-Dependent Predictive Ability," Working Papers 2017-9, University of Alberta, Department of Economics.
    11. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    12. Graham Elliott & Ulrich K. Muller, 2006. "Efficient Tests for General Persistent Time Variation in Regression Coefficients," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 73(4), pages 907-940.
    13. Andrews, Donald W K & Ploberger, Werner, 1994. "Optimal Tests When a Nuisance Parameter Is Present Only under the Alternative," Econometrica, Econometric Society, vol. 62(6), pages 1383-1414, November.
    14. Ole E. Barndorff-Nielsen & Neil Shephard, 2004. "Econometric Analysis of Realized Covariation: High Frequency Based Covariance, Regression, and Correlation in Financial Economics," Econometrica, Econometric Society, vol. 72(3), pages 885-925, May.
    15. Raffaella Giacomini & Barbara Rossi, 2010. "Forecast comparisons in unstable environments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 595-620.
    16. Simon Gilchrist & Egon Zakrajsek, 2012. "Credit Spreads and Business Cycle Fluctuations," American Economic Review, American Economic Association, vol. 102(4), pages 1692-1720, June.
    17. Li, Jia & Todorov, Viktor & Tauchen, George, 2017. "Adaptive estimation of continuous-time regression models using high-frequency data," Journal of Econometrics, Elsevier, vol. 200(1), pages 36-47.
    18. Li, Jia & Todorov, Viktor & Tauchen, George, 2016. "Inference theory for volatility functional dependencies," Journal of Econometrics, Elsevier, vol. 193(1), pages 17-34.
    19. M. Hashem Pesaran & Davide Pettenuzzo & Allan Timmermann, 2006. "Forecasting Time Series Subject to Multiple Structural Breaks," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 73(4), pages 1057-1084.
    20. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    21. Cheung, Yin-Wong & Chinn, Menzie D. & Pascual, Antonio Garcia, 2005. "Empirical exchange rate models of the nineties: Are any fit to survive?," Journal of International Money and Finance, Elsevier, vol. 24(7), pages 1150-1175, November.
    22. Andrews, Donald W K, 1991. "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, Econometric Society, vol. 59(3), pages 817-858, May.
    23. Foster, Dean P & Nelson, Daniel B, 1996. "Continuous Record Asymptotics for Rolling Sample Variance Estimators," Econometrica, Econometric Society, vol. 64(1), pages 139-174, January.
    24. Kim, Dukpa & Perron, Pierre, 2009. "Assessing the relative power of structural break tests using a framework based on the approximate Bahadur slope," Journal of Econometrics, Elsevier, vol. 149(1), pages 26-51, April.
    25. Luis Filipe Martins & Pierre Perron, 2016. "Improved Tests for Forecast Comparisons in the Presence of Instabilities," Journal of Time Series Analysis, Wiley Blackwell, vol. 37(5), pages 650-659, September.
    26. Pierre Perron & Yohei Yamamoto, 2016. "On the Usefulness or Lack Thereof of Optimality Criteria for Structural Change Tests," Econometric Reviews, Taylor & Francis Journals, vol. 35(5), pages 782-844, May.
    27. Atsushi Inoue & Barbara Rossi, 2011. "Identifying the Sources of Instabilities in Macroeconomic Fluctuations," The Review of Economics and Statistics, MIT Press, vol. 93(4), pages 1186-1204, November.
    28. Zhongjun Qu & Pierre Perron, 2007. "Estimating and Testing Structural Changes in Multivariate Regressions," Econometrica, Econometric Society, vol. 75(2), pages 459-502, March.
    29. Anindya Banerjee & Massimiliano Marcellino & Igor Masten, 2008. "Forecasting Macroeconomic Variables Using Diffusion Indexes in Short Samples with Structural Change," Working Papers 334, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    30. Serena Ng & Jonathan H. Wright, 2013. "Facts and Challenges from the Great Recession for Forecasting and Macroeconomic Modeling," Journal of Economic Literature, American Economic Association, vol. 51(4), pages 1120-1154, December.
    31. Alessandro Casini & Pierre Perron, 2017. "Continuous Record Laplace-based Inference about the Break Date in Structural Change Models," Boston University - Department of Economics - Working Papers Series WP2018-011, Boston University - Department of Economics.
    32. Perron, Pierre, 1989. "The Great Crash, the Oil Price Shock, and the Unit Root Hypothesis," Econometrica, Econometric Society, vol. 57(6), pages 1361-1401, November.
    33. Douglas Staiger & James H. Stock & Mark W. Watson, 1997. "The NAIRU, Unemployment and Monetary Policy," Journal of Economic Perspectives, American Economic Association, vol. 11(1), pages 33-49, Winter.
    34. Per A. Mykland & Lan Zhang, 2009. "Inference for Continuous Semimartingales Observed at High Frequency," Econometrica, Econometric Society, vol. 77(5), pages 1403-1445, September.
    35. Clark, Todd E. & McCracken, Michael W., 2005. "The power of tests of predictive ability in the presence of structural breaks," Journal of Econometrics, Elsevier, vol. 124(1), pages 1-31, January.
    36. Raffaella Giacomini, 2015. "Economic theory and forecasting: lessons from the literature," Econometrics Journal, Royal Economic Society, vol. 18(2), pages 22-41, June.
    37. Alessandro Casini & Pierre Perron, 2018. "Structural Breaks in Time Series," Boston University - Department of Economics - Working Papers Series WP2019-02, Boston University - Department of Economics.
    38. Bandi, F.M. & Renò, R., 2016. "Price and volatility co-jumps," Journal of Financial Economics, Elsevier, vol. 119(1), pages 107-146.
    39. Alessandro Casini & Pierre Perron, 2015. "Continuous Record Asymptotics for Structural Change Models," Boston University - Department of Economics - Working Papers Series WP2018-010, Boston University - Department of Economics, revised Nov 2017.
    40. Meese, Richard A. & Rogoff, Kenneth, 1983. "Empirical exchange rate models of the seventies : Do they fit out of sample?," Journal of International Economics, Elsevier, vol. 14(1-2), pages 3-24, February.
    41. Klein, Lawrence R, 1969. "Estimation on Interdependent Systems in Macroeconometrics," Econometrica, Econometric Society, vol. 37(2), pages 171-192, April.
    42. Yacine Aït-Sahalia & Jean Jacod, 2014. "High-Frequency Financial Econometrics," Economics Books, Princeton University Press, edition 1, number 10261.
    43. Andrew Atkeson & Lee E. Ohanian, 2001. "Are Phillips curves useful for forecasting inflation?," Quarterly Review, Federal Reserve Bank of Minneapolis, vol. 25(Win), pages 2-11.
    44. Clements, Michael P. & Hendry, David F., 1998. "Forecasting economic processes," International Journal of Forecasting, Elsevier, vol. 14(1), pages 111-131, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mwasi Paza Mboya & Philipp Sibbertsen, 2023. "Optimal forecasts in the presence of discrete structural breaks under long memory," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1889-1908, November.
    2. Casini, Alessandro & Perron, Pierre, 2022. "Generalized Laplace Inference In Multiple Change-Points Models," Econometric Theory, Cambridge University Press, vol. 38(1), pages 35-65, February.
    3. Casini, Alessandro & Perron, Pierre, 2021. "Continuous record Laplace-based inference about the break date in structural change models," Journal of Econometrics, Elsevier, vol. 224(1), pages 3-21.
    4. Federico Belotti & Alessandro Casini & Leopoldo Catania & Stefano Grassi & Pierre Perron, 2023. "Simultaneous bandwidths determination for DK-HAC estimators and long-run variance estimation in nonparametric settings," Econometric Reviews, Taylor & Francis Journals, vol. 42(3), pages 281-306, February.
    5. Alessandro Casini, 2021. "Theory of Evolutionary Spectra for Heteroskedasticity and Autocorrelation Robust Inference in Possibly Misspecified and Nonstationary Models," Papers 2103.02981, arXiv.org, revised Aug 2024.
    6. Casini, Alessandro, 2023. "Theory of evolutionary spectra for heteroskedasticity and autocorrelation robust inference in possibly misspecified and nonstationary models," Journal of Econometrics, Elsevier, vol. 235(2), pages 372-392.
    7. Alessandro Casini & Pierre Perron, 2018. "Structural Breaks in Time Series," Papers 1805.03807, arXiv.org.
    8. Pierre Perron & Yohei Yamamoto, 2021. "Testing for Changes in Forecasting Performance," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(1), pages 148-165, January.
    9. Casini, Alessandro & Perron, Pierre, 2024. "Change-point analysis of time series with evolutionary spectra," Journal of Econometrics, Elsevier, vol. 242(2).
    10. Alessandro Casini & Taosong Deng & Pierre Perron, 2021. "Theory of Low Frequency Contamination from Nonstationarity and Misspecification: Consequences for HAR Inference," Papers 2103.01604, arXiv.org, revised Sep 2024.
    11. Casini, Alessandro & Perron, Pierre, 2024. "Prewhitened long-run variance estimation robust to nonstationarity," Journal of Econometrics, Elsevier, vol. 242(1).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alessandro Casini & Pierre Perron, 2018. "Structural Breaks in Time Series," Boston University - Department of Economics - Working Papers Series WP2019-02, Boston University - Department of Economics.
    2. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    3. Alessandro Casini & Taosong Deng & Pierre Perron, 2021. "Theory of Low Frequency Contamination from Nonstationarity and Misspecification: Consequences for HAR Inference," Papers 2103.01604, arXiv.org, revised Sep 2024.
    4. Casini, Alessandro & Perron, Pierre, 2021. "Continuous record Laplace-based inference about the break date in structural change models," Journal of Econometrics, Elsevier, vol. 224(1), pages 3-21.
    5. Raffaella Giacomini & Barbara Rossi, 2015. "Forecasting in Nonstationary Environments: What Works and What Doesn't in Reduced-Form and Structural Models," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 207-229, August.
    6. Rapach, David & Zhou, Guofu, 2013. "Forecasting Stock Returns," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 328-383, Elsevier.
    7. Pierre Perron & Yohei Yamamoto, 2022. "Structural change tests under heteroskedasticity: Joint estimation versus two‐steps methods," Journal of Time Series Analysis, Wiley Blackwell, vol. 43(3), pages 389-411, May.
    8. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.
    9. Pierre Perron & Yohei Yamamoto, 2021. "Testing for Changes in Forecasting Performance," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(1), pages 148-165, January.
    10. Alessandro Casini, 2021. "Theory of Evolutionary Spectra for Heteroskedasticity and Autocorrelation Robust Inference in Possibly Misspecified and Nonstationary Models," Papers 2103.02981, arXiv.org, revised Aug 2024.
    11. Casini, Alessandro & Perron, Pierre, 2024. "Prewhitened long-run variance estimation robust to nonstationarity," Journal of Econometrics, Elsevier, vol. 242(1).
    12. Pierre Perron & Yohei Yamamoto, 2016. "On the Usefulness or Lack Thereof of Optimality Criteria for Structural Change Tests," Econometric Reviews, Taylor & Francis Journals, vol. 35(5), pages 782-844, May.
    13. Casini, Alessandro, 2023. "Theory of evolutionary spectra for heteroskedasticity and autocorrelation robust inference in possibly misspecified and nonstationary models," Journal of Econometrics, Elsevier, vol. 235(2), pages 372-392.
    14. Seong Yeon Chang & Pierre Perron, 2018. "A comparison of alternative methods to construct confidence intervals for the estimate of a break date in linear regression models," Econometric Reviews, Taylor & Francis Journals, vol. 37(6), pages 577-601, July.
    15. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    16. Alessandro Casini & Pierre Perron, 2018. "Continuous Record Asymptotics for Change-Points Models," Papers 1803.10881, arXiv.org, revised Nov 2021.
    17. Matei Demetrescu & Christoph Hanck & Robinson Kruse‐Becher, 2022. "Robust inference under time‐varying volatility: A real‐time evaluation of professional forecasters," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 1010-1030, August.
    18. Rossi, Barbara & Sekhposyan, Tatevik, 2011. "Understanding models' forecasting performance," Journal of Econometrics, Elsevier, vol. 164(1), pages 158-172, September.
    19. Kenneth S. Rogoff & Vania Stavrakeva, 2008. "The Continuing Puzzle of Short Horizon Exchange Rate Forecasting," NBER Working Papers 14071, National Bureau of Economic Research, Inc.
    20. West, Kenneth D., 2006. "Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 3, pages 99-134, Elsevier.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1803.10883. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.