Demonstration of an open source platform for reproducible comparison of predictive models

This paper discusses about a tool PredictTestbench, which is an R package which provides a testbench to do comparison of prediction methods. This package compares a proposed time series prediction method with other default methods like Autoregressive integrated moving average (ARIMA) and Pattern Sequence based Forecasting (PSF). The testbench is not limited to these methods. It allows user to add or remove multiple numbers of methods in the existing methods in the study. By default, testbench compares different imputation methods considering different error metrics RMSE, MAE or MAPE. Along with this, it facilitates user to add new error metrics as per requirements. The simplicity of the package usage and significant reduction in efforts and time consumption in state of art procedure, adds valuable advantage to it. The aim of the testbench is reduce the efforts for coding, experiments on output visualization and time for different steps involved in such study. This paper explains the use of all functions in PredictTestbench package with the demonstration of examples.