Correlation based test of predictability and Mean Squared Prediction Error Comparisons: an empirical evaluation

Abstract

In the literature forecasts are mostly evaluated using the traditional Mean Squared Prediction Error (MSPE) loss function and with this approach the authors usually find evidence of predictability at the population level, while the evidence of predictability at the sample is usually scarce and weak. However, in recent research (See Pincheira and Hardy ;2021, “The Mean Squared Prediction Error Paradox”) the authors find that there are some not empty spaces where correlations between forecasts and the predictand grows when MSPE grows too. This means that in some cases MSPE can suggest that a given forecast is not a good predictor when this is not necessarily the case. In this paper we propose to evaluate predictions measuring the correlation between forecasts and actual values of the predictand to see if using correlations as an alternative approach, we can also find evidence of predictability at sample level. We want to evaluate if results with correlation-based tests of predictability are equivalent to the results coming from the Diebold-Mariano test comparing MSPE at sample level. We will evaluate this hypothesis in the context of the commodity-currencies literature.