Abstract
When the amount of data is reasonably small, we can usually fit this data to a simple model and use the traditional statistical methods both to estimate the parameters of this model and to gauge this model’s accuracy. For big data, it is often no longer possible to fit them by a simple model. Thus, we need to use generic machine learning techniques to find the corresponding model. The current machine learning techniques estimate the values of the corresponding parameters, but they usually do not gauge the accuracy of the corresponding general non-linear model. In this paper, we show how to modify the existing machine learning methodology so that it will not only estimate the parameters, but also estimate the accuracy of the resulting model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)
Mayer-Schönberger, V., Cukier, K.: Big Data: The Essential Guide to Work, Life, and Learning in the Age of Insight. John Murray Publishers, London (2017)
Sheskin, D.J.: Handbook of Parametric and Nonparametric Statistical Procedures. Chapman & Hall/CRC, Boca Raton, Florida (2011)
Turchetti, C.: Stochastic Models of Neural Networks. IOS Press, Amsterdam (2004)
Werbos, P.: A brain-like design to learn optimal decision strategies in complex environments. In: Karny, M., Warwick, K., Kurkova, V. (eds.) Dealing with Complexity: A Neural Networks Approach. Springer, London (1998)
Werbos, P.: Intelligence in the brain: a theory of how it works and how to build it. Neural Netw. 22(3), 200–212 (2009)
Acknowledgments
This work was supported by Chiang Mai University, Thailand. This work was also supported in part by the National Science Foundation grants HRD-0734825 and HRD-1242122 (Cyber-ShARE Center of Excellence) and DUE-0926721, and by an award “UTEP and Prudential Actuarial Science Academy and Pipeline Initiative” from Prudential Foundation.
One of the authors (VK) is thankful to Paul Werbos for inspiring talks and discussions.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Kreinovich, V., Dumrongpokaphan, T., Nguyen, H.T., Kosheleva, O. (2018). How to Gauge Accuracy of Processing Big Data: Teaching Machine Learning Techniques to Gauge Their Own Accuracy. In: Kreinovich, V., Sriboonchitta, S., Chakpitak, N. (eds) Predictive Econometrics and Big Data. TES 2018. Studies in Computational Intelligence, vol 753. Springer, Cham. https://doi.org/10.1007/978-3-319-70942-0_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-70942-0_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70941-3
Online ISBN: 978-3-319-70942-0
eBook Packages: EngineeringEngineering (R0)