More Web Proxy on the site http://driver.im/

tutorial

Parallel Computing of Support Vector Machines: A Survey

Author:

Shirin TavaraAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 51, Issue 6

Article No.: 123, Pages 1 - 38

https://doi.org/10.1145/3280989

Published: 28 January 2019 Publication History

Abstract

The immense amount of data created by digitalization requires parallel computing for machine-learning methods. While there are many parallel implementations for support vector machines (SVMs), there is no clear suggestion for every application scenario. Many factor—including optimization algorithm, problem size and dimension, kernel function, parallel programming stack, and hardware architecture—impact the efficiency of implementations. It is up to the user to balance trade-offs, particularly between computation time and classification accuracy. In this survey, we review the state-of-the-art implementations of SVMs, their pros and cons, and suggest possible avenues for future research.

References

[1]

L. T. Yang and M. Guo. 2005. High-Performance Computing: Paradigm and Infrastructure. Wiley. https://books.google.se/books?id=qA4DbnFB2XcC

[2]

Vladimir Vapnik. 2013. The Nature of Statistical Learning Theory. Springer Science 8 Business Media. 314 pages.

[3]

Hyeran Byun and Seong-Whan Lee. 2002. Applications of support vector machines for pattern recognition: A survey. In Pattern Recognition with Support Vector Machines. Springer, 213--236.

Digital Library

[4]

Zhao Bin, Liu Yong, and Xia Shao-Wei. 2000. Support vector machine and its application in handwritten numeral recognition. In Proceedings of the 15th International Conference on Pattern Recognition. Vol. 2. 720--723.

[5]

Peter Bartlett and John Shawe-Taylor. 1999. Generalization performance of support vector machines and other pattern classifiers. Advances in Kernel Methods-Support Vector Learning, C. Burges and B. Scholkopf (Eds.). MIT Press, 43--54.

Digital Library

[6]

Guo-Xun Yuan, Chia-Hua Ho, and Chih-Jen Lin. 2012. Recent advances of large-scale linear classification. Proceedings of IEEE 100, 9 (2012), 2584--2603.

[7]

Janmenjoy Nayak, Bighnaraj Naik, and H. S. Behera. 2015. A comprehensive survey on support vector machine in data mining tasks: Applications 8 challenges. International Journal of Database Theory and Application 8, 1 (2015), 169--186.

[8]

R. Ravinder Reddy, B. Kavya, and Y. Ramadevi. 2014. A survey on SVM classifiers for intrusion detection. In International Journal of Computer Applications 98, 19 (2014), 34--44.

[9]

G. Wang. 2008. A survey on training algorithms for support vector machine classifiers. In 4th International Conference on Networked Computing and Advanced Information Management (NCM’08). Vol. 1. 123--128.

Digital Library

[10]

Thanh-Nghi Do, Van-Hoa Nguyen, and François Poulet. 2008. Speed up SVM algorithm for massive classification tasks. In Advanced Data Mining and Applications. Springer, 147--157.

Digital Library

[11]

Kristian Woodsend and Jacek Gondzio. 2009. High-performance parallel support vector machine training. In Parallel Scientific Computing and Optimization. Springer, 83--92.

[12]

Nanbo Peng, Yanxia Zhang, and Yongheng Zhao. 2011. CUDA-accelerated SVM for celestial object classification. In Astronomical Data Analysis Software and Systems Xx, Vol. 442. 119--122. Astronomical Society of the Pacific.

[13]

H. X. Zhao and Frédéric Magoules. 2011. Parallel support vector machines on multi-core and multiprocessor systems. In 11th International Conference on Artificial Intelligence and Applications (AIA’11), R. Fox (Ed.). IASTED, 75--81.

[14]

Dominik Brugger. 2006. Parallel Support Vector Machines. Arbeitsbereich Technische Informatik, Universität Tübingen. 27 pages. https://publikationen.uni-tuebingen.de/xmlui/bitstream/handle/10900/49015/pdf/tech_21.pdf?sequence=18isAllowed=y

[15]

Emmanuel Didiot and Fabien Lauer. 2015. Efficient optimization of multi-class support vector machines with MSVMpack. In Modelling, Computation and Optimization in Information Systems and Management Sciences. Springer, 23--34.

[16]

Ana Gainaru, Emil Slusanschi, and Stefan Trausan-Matu. 2011. Mapping data mining algorithms on a GPU architecture: A study. In Foundations of Intelligent Systems. Springer, 102--112.

Digital Library

[17]

Hans P. Graf, Eric Cosatto, Leon Bottou, Igor Dourdanovic, and Vladimir Vapnik. 2004. Parallel support vector machines: The cascade SVM. In Proceedings of the 17th International Conference on Neural Information Processing Systems. MIT Press, 521--528.

Digital Library

[18]

Stephen Tyree, Jacob R. Gardner, Kilian Q. Weinberger, Kunal Agrawal, and John Tran. 2014. Parallel support vector machines in practice. arXiv preprint arXiv:1404.1066 (2014).

[19]

Evans Data Software Developer surveys 2011-2013. Performance: Ready to Use. Retrieved October 11, 2015 from https://software.intel.com/en-us/intel-mkl.

[20]

NVIDIA Corporation. 2015. cuBLAS. Retrieved October 11, 2015 from https://developer.nvidia.com/cublas.

[21]

Yunmei Lu, Yun Zhu, Meng Han, Jing (Selena) He, and Yanqing Zhang. 2014. A survey of GPU accelerated SVM. In Proceedings of the ACM Southeast Regional Conference (ACM SE’14). ACM, Article 15, 7 pages.

Digital Library

[22]

John Platt. 1998. Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines. Technical Report MSR-TR-98-14. Microsoft Research. 21 pages. Retrieved December 12, 2018 from http://research.microsoft.com/apps/pubs/default.aspx?id=69644

[23]

Bernhard E. Boser, Isabelle M. Guyon, and Vladimir N. Vapnik. 1992. A training algorithm for optimal margin classifiers. In Proceedings of the 5th Annual Workshop on Computational Learning Theory. ACM, 144--152.

Digital Library

[24]

Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Machine Learning 20, 3 (1995), 273--297.

[25]

Ovidiu Ivanciuc. 2007. Applications of support vector machines in chemistry. Reviews in Computational Chemistry 23 (2007), 291--400.

[26]

Gaetano Zanghirati and Luca Zanni. 2003. A parallel solver for large quadratic programs in training support vector machines. Parallel Computing 29, 4 (2003), 535--551.

Digital Library

[27]

AUSTIN Carpenter. 2009. cuSVM: A CUDA implementation of support vector classification and regression. Retrieved December 12, 2018 from http://www.patternsonascreen.net/cuSVMDesc.pdf.

[28]

Tao Li, Hua Li, Xuechen Liu, Shuai Zhang, Kai Wang, and Yulu Yang. 2013. GPU acceleration of interior point methods in large scale SVM training. In 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom’13). IEEE, 863--870.

Digital Library

[29]

Thanh-Nghi Doan, Thanh-Nghi Do, and François Poulet. 2013. Large scale visual classification with parallel, imbalanced bagging of incremental LIBLINEAR SVM. In Proceedings of the International Conference on Data Mining (DMIN’13). The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp), 1.

[30]

W. W. Hwu. 2011. GPU Computing Gems Emerald Edition. Elsevier Science. https://books.google.se/books?id=lGMzmbUhpiAC

Digital Library

[31]

Volkan Vural and Jennifer G. Dy. 2004. A hierarchical method for multi-class support vector machines. In Proceedings of the 21st International Conference on Machine Learning. ACM, 105--112.

Digital Library

[32]

L. J. Cao, S. S. Keerthi, Chong-Jin Ong, J. Q. Zhang, and H. P. Lee. 2006. Parallel sequential minimal optimization for the training of support vector machines. IEEE Transactions on Neural Networks, 17, 4 (July 2006), 1039--1049.

Digital Library

[33]

Kristian Woodsend and Jacek Gondzio. 2009. Hybrid MPI/OpenMP parallel linear support vector machine training. Journal of Machine Learning Research 10 (Dec. 2009), 1937--1953. http://dl.acm.org/citation.cfm?id=1577069.1755850.

Digital Library

[34]

John C. Platt. 1999. Using analytic QP and sparseness to speed training of support vector machines. In Proceedings of the 1998 Conference on Advances in Neural Information Processing Systems II. MIT Press, 557--563.

Digital Library

[35]

Valeriu Codreanu, Bob Dröge, David Williams, Burhan Yasar, Po Yang, Baoquan Liu, Feng Dong, Olarik Surinta, Lambert R. B. Schomaker, Jos B. T. M. Roerdink, et al. 2016. Evaluating automatically parallelized versions of the support vector machine. In Concurrency and Computation: Practice and Experience 28, 7 (2016), 2274--2294.

Digital Library

[36]

Tatjana Eitrich and Bruno Lang. 2005. Efficient Implementation of Serial and Parallel Support Vector Machine Training with a Multi-parameter Kernel for Large-scale Data Mining. Citeseer.

[37]

Fernando Pérez-Cruz, Anıbal R. Figueiras-Vidal, and Antonio Artés-Rodríguez. 2004. Double chunking for solving SVMs for very large datasets. In Proceedings of Learning (2004).

[38]

Luca Zanni, Thomas Serafini, and Gaetano Zanghirati. 2006. Parallel software for training large scale support vector machines on multiprocessor systems. Journal of Machine Learning Research 7, 7 (2006), 1467--1492.

Digital Library

[39]

Thomas Serafini, Gaetano Zanghirati, and Luca Zanni. 2005. Gradient projection methods for quadratic programs and applications in training support vector machines. Optimization Methods and Software 20, 2-3 (2005), 353--378.

[40]

Andrew Cotter, Nathan Srebro, and Joseph Keshet. 2011. A GPU-tailored approach for training kernelized SVMs. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 805--813.

Digital Library

[41]

Edward Y. Chang. 2011. PSVM: Parallelizing Support Vector Machines on Distributed Computers. Springer, Berlin, 213--230.

[42]

Roberto Díaz-Morales, Harold Y. Molina-Bulla, and Angel Navia-Vázquez. 2011. Parallel semiparametric support vector machines. In International Joint Conference on Neural Networks (IJCNN’11). IEEE, 475--481.

[43]

Godwin Caruana, Maozhen Li, and Man Qi. 2011. A MapReduce based parallel SVM for large scale spam filtering. In 8th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD’11), Vol. 4. IEEE, 2659--2662.

[44]

Andreas Athanasopoulos, Anastasios Dimou, Vasileios Mezaris, and Ioannis Kompatsiaris. 2011. GPU acceleration for support vector machines. In 12th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS’11), Delft, the Netherlands, April 13-15, 2011. TU Delft; EWI; MM; PRB.

[45]

W. C. C. Chu, H. C. Chao, and S. J. H. Yang. 2015. Intelligent Systems and Applications: Proceedings of the International Computer Symposium (ICS) Held at Taichung, Taiwan, December 12--14, 2014. IOS Press. https://books.google.se/books?id=dOqbCAAAQBAJ

Digital Library

[46]

Jian-xiong Dong, Adam Krzyżak, and Ching_Y. Suen. 2003. A fast parallel optimization for training support vector machine. In Machine Learning and Data Mining in Pattern Recognition. Springer, 96--105.

Digital Library

[47]

Bryan Catanzaro, Narayanan Sundaram, and Kurt Keutzer. 2008. Fast support vector machine training and classification on graphics processors. In Proceedings of the 25th International Conference on Machine Learning. ACM, 104--111.

Digital Library

[48]

Léon Bottou and Chih-Jen Lin. 2007. Support vector machine solvers. In Large Scale Kernel Machines. MIT Press, 301--320.

[49]

Xueqin Zhang, Yifeng Zhang, and Chunhua Gu. 2014. GPU implementation of parallel support vector machine algorithm with applications to detection intruder. Journal of Computers 9, 5 (2014), 1117--1124.

[50]

Léon Bottou. 2007. Large-scale Kernel Machines. MIT Press.

[51]

Qi Li, Raied Salman, Erik Test, Robert Strack, and Vojislav Kecman. 2011. GPUSVM: A comprehensive CUDA based support vector machine package. Open Computer Science 1, 4 (2011), 387--405.

[52]

Lingfeng Niu and Ya-xiang Yuan. 2011. A parallel decomposition algorithm for training multiclass kernel-based vector machines. Optimization Methods and Software 26, 3 (2011), 431--454.

Digital Library

[53]

Qi Li, Raied Salman, Erik Test, Robert Strack, and Vojislav Kecman. 2013. Parallel multitask cross validation for support vector machine using GPU. Journal of Parallel and Distributed Computing 73, 3 (2013), 293--302.

Digital Library

[54]

Jian-xiong Dong, Adam Krzyżak, and Ching Y Suen. 2005. Fast SVM training algorithm with decomposition on very large data sets. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 4 (2005), 603--618.

Digital Library

[55]

Yang You, S. L. Song, Haohuan Fu, A. Marquez, M. M. Dehnavi, K. Barker, K. W. Cameron, A. P. Randles, and Guangwen Yang. 2014. MIC-SVM: Designing a highly efficient support vector machine for advanced modern multi-core and many-core architectures. In IEEE 28th International Parallel and Distributed Processing Symposium. IEEE, 809--818.

Digital Library

[56]

Sriram Venkateshan, Alap Patel, and Kuruvilla Varghese. 2015. Hybrid working set algorithm for SVM learning with a kernel coprocessor on FPGA. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 23, 10 (2015), 2221--2232.

[57]

T. Joachims. 1998. Making Large-Scale SVM Learning Practical. LS8-Report 24. Universität Dortmund, LS VIII-Report.

[58]

Yunmei Lu, Yun Zhu, Meng Han, Jing Selena He, and Yanqing Zhang. 2014. A survey of GPU accelerated SVM. In Proceedings of the ACM Southeast Regional Conference. ACM, 15.

Digital Library

[59]

Thorsten Joachims. 1999. Making Large Scale SVM Learning Practical. Technical Report. Universität Dortmund.

[60]

Moreno Marzolla. 2011. Fast training of support vector machines on the cell processor. Neurocomputing 74, 17 (2011), 3700--3707.

Digital Library

[61]

Jing Jin, Xianggao Cai, and Xiaola Lin. 2013. Efficient SVM training using parallel primal-dual interior point method on GPU. In International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT’13). IEEE, 12--17.

Digital Library

[62]

Jeyanthi Narasimhan, Abhinav Vishnu, Lawrence Holder, and Adolfy Hoisie. 2014. Fast support vector machines using parallel adaptive shrinking on distributed systems. arXiv preprint arXiv:1406.5161 (2014).

[63]

João Gonçalves, Noel Lopes, and Bernardete Ribeiro. 2012. Multi-threaded support vector machines for pattern recognition. In Neural Information Processing. Springer, 616--623.

Digital Library

[64]

Pengfei Chang, Zhuo Bi, and Yiyong Feng. 2014. Parallel SMO algorithm implementation based on OpenMP. In IEEE International Conference on System Science and Engineering (ICSS’14). 236--240.

[65]

Tatjana Eitrich and Bruno Lang. 2006. Data mining with parallel support vector machines for classification. In Advances in Information Systems. Springer, 197--206.

Digital Library

[66]

Hwancheol Jeong, Sunghoon Kim, Weonjong Lee, and Seok-Ho Myung. 2012. Performance of SSE and AVX instruction sets. arXiv preprint arXiv:1211.0820 (2012).

[67]

Abhinav Vishnu, Jeyanthi Narasimhan, Lawrence Holder, Darren Kerbyson, and Adolfy Hoisie. 2015. Fast and accurate support vector machines on large scale systems. In IEEE International Conference on Cluster Computing (CLUSTER’15). IEEE, 110--119.

Digital Library

[68]

L. V. Ferreira, E. Kaszkurewicz, and Amit Bhaya. 2006. Parallel implementation of gradient-based neural networks for SVM training. In International Joint Conference on Neural Networks (IJCNN’06). IEEE, 339--346.

[69]

S. K. Shevade, S. S. Keerthi, C. Bhattacharyya, and K. R. K. Murthy. 2000. Improvements to the SMO algorithm for SVM regression. IEEE Transactions on Neural Networks 11, 5 (Sep 2000), 1188--1193.

Digital Library

[70]

F. Aiolli and A. Sperduti. 2002. An efficient SMO-like algorithm for multiclass SVM. In Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing. IEEE, 297--306.

[71]

Peng Peng, Qian-Li Ma, and Lei-Ming Hong. 2009. The research of the parallel SMO algorithm for solving SVM. In International Conference on Machine Learning and Cybernetics, Vol. 3. 1271--1274.

[72]

Elad Yom-Tov. 2005. A parallel training algorithm for large scale support vector machines. In Neural Information Processing Systems (NIPS’05), Workshop on Large Scale Kernel Machines. Whistler, Canada.

[73]

T. Hazan, A. Man, and A. Shashua. 2008. A parallel decomposition solver for SVM: Distributed dual ascend using fenchel duality. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’08).IEEE, 1--8.

[74]

Nasullah Khalid Alham, Maozhen Li, and Yang Liu. 2014. Parallelizing multiclass support vector machines for scalable image annotation. Neural Computing and Applications 24, 2 (2014), 367--381.

[75]

MPI Forum. 2015. Message Passing Interface Forum. Retrieved October 18, 2015 from http://www.mpi-forum.org/.

[76]

Sergio Herrero-Lopez, John R. Williams, and Abel Sanchez. 2010. Parallel multiclass classification using SVMs on GPUs. In Proceedings of the 3rd Workshop on General-Purpose Computation on Graphics Processing Units (GPGPU-3). ACM, New York, NY, 2--11.

Digital Library

[77]

Quan Liao, Jibo Wang, Yue Webster, and Ian A. Watson. 2009. GPU accelerated support vector machines for mining high-throughput screening data. Journal of Chemical Information and Modeling 49, 12 (2009), 2718--2725.

[78]

Krzysztof Sopyła, Paweł Drozda, and Przemysław Górecki. 2012. SVM with CUDA accelerated kernels for big sparse problems. In Artificial Intelligence and Soft Computing. Springer, 439--447.

Digital Library

[79]

Qi Li. 2011. Fast Parallel Machine Learning Algorithms for Large Datasets Using Graphic Processing Unit. Ph.D. thesis. AAI3489482.

Digital Library

[80]

Tsung-Kai Lin and Shao-Yi Chien. 2010. Support vector machines on GPU with sparse matrix format. In 9th International Conference on Machine Learning and Applications. IEEE, 313--318.

Digital Library

[81]

J. Vaněk, J. Michálek, and J. Psutka. 2017. A GPU-architecture optimized hierarchical decomposition algorithm for support vector machine training. IEEE Transactions on Parallel and Distributed Systems 28, 12 (Dec 2017), 3330--3343.

[82]

Yang You and James Demmel. 2017. Runtime data layout scheduling for machine learning dataset. In 46th International Conference on Parallel Processing (ICPP’17). IEEE, 452--461.

[83]

Noel Lopes and Bernardete Ribeiro. 2011. GPUMLib: An efficient open-source GPU machine learning library. International Journal of Computer Information Systems and Industrial Management Applications 3 (2011), 355--362.

[84]

Qing He, Changying Du, Qun Wang, Fuzhen Zhuang, and Zhongzhi Shi. 2011. A parallel incremental extreme SVM classifier. Neurocomputing 74, 16 (2011), 2532--2540.

[85]

Amund Tveit and Havard Engum. 2003. Parallelization of the incremental proximal support vector machine classifier using a heap-based tree topology. In Parallel and Distributed Computing for Machine Learning: Proceedings of the 14th European Conference on Machine Learning and the 7th European Conference on Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD’03), Cavtat-Dubrovnik, Croatia. Citeseer.

[86]

Gang Wu, Edward Chang, Yen Kuang Chen, and Christopher Hughes. 2006. Incremental approximate matrix factorization for speeding up support vector machines. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 760--766.

Digital Library

[87]

Shin Matsushima, S. V. N. Vishwanathan, and Alexander J. Smola. 2012. Linear support vector machines via dual cached loops. In Proceedings of the 18th ACM SIGKDD international Conference on Knowledge Discovery and Data Mining. ACM, 177--185.

Digital Library

[88]

Thanh-Nghi Do and François Poulet. 2006. Classifying one billion data with a new distributed SVM algorithm. In International Conference on Research, Innovation and Vision for the Future (RIVF'06). IEEE, 59--66.

[89]

Pedro A. Forero, Alfonso Cano, and Georgios B. Giannakis. 2010. Consensus-based distributed support vector machines. Journal of Machine Learning Research 11 (2010), 1663--1707.

Digital Library

[90]

Yumao Lu, Vwani Roychowdhury, and Lieven Vandenberghe. 2008. Distributed parallel support vector machines in strongly connected networks. IEEE Transactions on Neural Networks 19, 7 (2008), 1167--1178.

Digital Library

[91]

Thanh-Nghi Do, Van-Hoa Nguyen, and François Poulet. 2008. A fast parallel SVM algorithm for massive classification tasks. In Modelling, Computation and Optimization in Information Systems and Management Sciences. Springer, 419--428.

[92]

Nadeem Ahmed Syed, Syed Huan, Liu Kah, and Kay Sung. 1999. Incremental learning with support vector machines. In Proceedings of the Workshop on Support Vector Machines at the International Joint Conference on Artificial Intelligence. CiteSeer.

[93]

Cornelia Caragea, Doina Caragea, and Vasant Honavar. 2005. Learning support vector machines from distributed data sources. In Proceedings of the 20th National Conference on Artificial Intelligence (AAAI'05), Vol. 4. AAAI Press, 1602--1603.

Digital Library

[94]

FAL Labs. 2011. Bukkoji Cabinet: a straightforward implementation of DBM. Retrieved October 2, 2016 from http://fallabs.com/kyotocabinet/.

[95]

R. Bekkerman, M. Bilenko, and J. Langford. 2012. Scaling Up Machine Learning: Parallel and Distributed Approaches. Cambridge University Press. https://books.google.se/books?id=c5v5USMvcMYC

Digital Library

[96]

Xue-Qiang Zeng and Guo-Zheng Li. 2014. Incremental partial least squares analysis of big streaming data. Pattern Recognition 47, 11 (2014), 3726--3735.

Digital Library

[97]

Jian-Pei Zhang, Zhong-Wei Li, and Jing Yang. 2005. A parallel SVM training algorithm on large-scale classification problems. In Proceedings of International Conference on Machine Learning and Cybernetics. Vol. 3. IEEE, 1637--1641.

[98]

Oliver Meyer, Bernd Bischl, and Claus Weihs. 2014. Support vector machines on large data sets: Simple parallel approaches. In Data Analysis, Machine Learning and Knowledge Discovery. Springer, 87--95.

[99]

Mingsheng Hu and Weixu Hao. 2010. A parallel approach for SVM with multi-core CPU. In International Conference on Computer Application and System Modeling (ICCASM’10), Vol. 15. IEEE, V15--373--V15--377.

[100]

Jing Yang. 2006. An improved cascade SVM training algorithm with crossed feedbacks. In 1st International Multi-Symposiums on Computer and Computational Sciences (IMSCCS’06). Vol. 2. IEEE, 735--738.

Digital Library

[101]

Hock Hee Ang, Vivekanand Gopalkrishnan, Steven CH Hoi, and Wee Keong Ng. 2008. Cascade RSVM in peer-to-peer networks. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 55--70.

[102]

Yang You, J. Demmel, K. Czechowski, Le Song, and R. Vuduc. 2015. CA-SVM: Communication-avoiding support vector machines on distributed systems. In IEEE International Parallel and Distributed Processing Symposium (IPDPS’15). IEEE, 847--859.

Digital Library

[103]

Hongle Du, Shaohua Teng, Xiufen Fu, Wei Zhang, and Yuanfang Pu. 2009. A cooperative intrusion detection system based on improved parallel SVM. In Joint Conferences on Pervasive Computing (JCPC’09). IEEE, 515--518.

[104]

Daniel Weimer, Sebastian Köhler, Christian Hellert, Konrad Doll, Ulrich Brunsmann, and Roland Krzikalla. 2011. GPU architecture for stationary multisensor pedestrian detection at smart intersections. In IEEE Intelligent Vehicles Symposium (IV’11),. IEEE, 89--94.

[105]

Qi Li, Raied Salman, and Vojislav Kecman. 2010. An intelligent system for accelerating parallel SVM classification problems on large datasets using GPU. In 10th International Conference on Intelligent Systems Design and Applications (ISDA’10). IEEE, 1131--1135.

[106]

Nasullah Khalid Alham, Maozhen Li, Yang Liu, Suhel Hammoud, and Mahesh Ponraj. 2011. A distributed SVM for scalable image annotation. In 8th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD’11), Vol. 4. IEEE, 2655--2658.

[107]

Zhanquan Sun and Geoffrey Fox. 2012. Study on parallel SVM based on MapReduce. In Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA'12). 16--19.

[108]

Ke Xu, Cui Wen, Qiong Yuan, Xiangzhu He, and Jun Tie. 2014. A MapReduce based parallel SVM for email classification. Journal of Networks 9, 6 (2014), 1640--1647.

[109]

N. Z. Tarapore, D. B. Kulkarni, and V. K. Prasad. 2016. Implementation of parallel algorithm for support vector machine applied to intrusion detection systems. In International Conference on Computing, Analytics and Security Trends (CAST’16). IEEE, 179--184.

[110]

Igor Durdanovic, Eric Cosatto, and Hans-Peter Graf. 2007. Large Scale Parallel SVM Implementation. MIT Press, 105--138.

[111]

Zhen-Yu Chen and Zhi-Ping Fan. 2014. Parallel multiple kernel learning: A hybrid alternating direction method of multipliers. Knowledge and Information Systems 40, 3 (2014), 673--696.

Digital Library

[112]

Yimin Wen and Baoliang Lu. 2005. A hierarchical and parallel method for training support vector machines. In Proceedings of the 2nd International Conference on Advances in Neural Networks - Volume Part I (ISNN’05). Springer, Berlin, 881--886.

Digital Library

[113]

Yuh-Jye Lee and Olvi L. Mangasarian. 2001. RSVM: Reduced support vector machines. In Proceedings of the 2001 Society for Industrial and Applied Mathematics (SIAM) International Conference on Data Mining. 1--17.

[114]

Wenming Guo, Nasullah Khalid Alham, Yang Liu, Maozhen Li, and Man Qi. 2015. A resource aware MapReduce based parallel SVM for large scale image classifications. Neural Processing Letters 44, 1 (2015), 161--184.

Digital Library

[115]

Parisa Pouladzadeh, Shervin Shirmohammadi, Aslan Bakirov, Ahmet Bulut, and Abdulsalam Yassine. 2014. Cloud-based SVM for food categorization. Multimedia Tools and Applications 74, 14 (2014), 5243--5260.

Digital Library

[116]

Guang-Bin Huang, KZ Mao, Chee-Kheong Siew, and De-Shuang Huang. 2005. Fast modular network implementation for support vector machines. IEEE Transactions on Neural Networks, 16, 6 (2005), 1651--1663.

Digital Library

[117]

Marc Claesen, Frank De Smet, Johan AK Suykens, and Bart De Moor. 2014. EnsembleSVM: A library for ensemble learning using support vector machines.Journal of Machine Learning Research 15, 1 (2014), 141--145.

Digital Library

[118]

Anirban Chatterjee, Kelly Fermoyle, and Padma Raghavan. 2010. Characterizing sparse preconditioner performance for the support vector machine kernel. Procedia Computer Science 1, 1 (2010), 367--375.

[119]

E. Michael Gertz and Joshua D. Griffin. 2010. Using an iterative linear solver in an interior-point method for generating support vector machines. Computational Optimization and Applications 47, 3 (2010), 431--453.

Digital Library

[120]

Kaihua Zhu, Hang Cui, Hongjie Bai, Jian Li, Zhihuan Qiu, Hao Wang, Hui Xu, and Edward Y. Chang. 2007. Parallel approximate matrix factorization for kernel methods. In IEEE International Conference on Multimedia and Expo. IEEE, 1275--1278.

[121]

Edward Y. Chang, Hongjie Bai, and Kaihua Zhu. 2009. Parallel algorithms for mining large-scale rich-media data. In Proceedings of the 17th ACM International Conference on Multimedia. ACM, 917--918.

Digital Library

[122]

Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research 9, Aug (2008), 1871--1874.

Digital Library

[123]

S. Tavara, H. Sundell, and A. Dahlbom. 2015. Empirical study of time efficiency and accuracy of support vector machines using an improved version of PSVM. In Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA’15). The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp), 177.

[124]

T. Li, X. Liu, Q. Dong, W. Ma, and K. Wang. 2016. HPSVM: Heterogeneous parallel SVM with factorization based IPM algorithm on CPU-GPU cluster. In 24th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP’16). IEEE, 74--81.

[125]

Aliaksei Severyn and Alessandro Moschitti. 2011. Fast support vector machines for structural kernels. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 175--190.

Digital Library

[126]

D. Steinkraus, I. Buck, and P. Y. Simard. 2005. Using GPUs for machine learning algorithms. In Proceedings of the 8th International Conference on Document Analysis and Recognition, Vol. 2. IEEE Computer Society, 1115--1120.

Digital Library

[127]

R . A. Reyna-Rojas, D. Dragomirescu, D. Houzet, and D. Esteve. 2003. Implementation of the SVM generalization function on FPGA. In Proceedings of the International Signal Processing Conference (ISPC’03), Dallas, TX. 147--153.

[128]

Ian Biasi, Andrea Boni, and Alessandro Zorat. 2005. A reconfigurable parallel architecture for SVM classification. In Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN’05). Vol. 5. IEEE, 2867--2872.

[129]

Hans P. Graf, Srihari Cadambi, Venkata Jakkula, Murugan Sankaradass, Eric Cosatto, Srimat Chakradhar, and Igor Dourdanovic. 2009. A massively parallel digital learning processor. In Proceedings of the 21st International Conference on Neural Information Processing Systems (NIPS'08). Curran Associates Inc, 529--536.

Digital Library

[130]

Markos Papadonikolakis, Christos-Savvas Bouganis, and George Constantinides. 2009. Performance comparison of GPU and FPGA architectures for the SVM training problem. In International Conference on Field-Programmable Technology (FPT’09). IEEE, 388--391.

[131]

Clive Maxfield. 2011. FPGAs: Instant Access. Newnes.

Digital Library

[132]

Ronan Collobert, Samy Bengio, and Yoshua Bengio. 2002. A parallel mixture of SVMs for very large scale problems. Neural Computation 14, 5 (2002), 1105--1114.

Digital Library

[133]

Danny Bickson, Elad Yom-Tov, and Danny Dolev. 2008. A Gaussian belief propagation solver for large scale support vector machines. arXiv preprint arXiv:0810.1648 (2008).

[134]

Liu Fei, Zhang Wen-Ju, Yu Shui, Ma Fan-Yuan, and Li Ming-Lu. 2004. A peer-to-peer hypertext categorization using directed acyclic graph support vector machines. In Parallel and Distributed Computing: Applications and Technologies. Springer, 54--57.

Digital Library

[135]

Zeyuan Allen Zhu, Weizhu Chen, Gang Wang, Chenguang Zhu, and Zheng Chen. 2009. P-packSVM: Parallel primal gradient descent kernel SVM. In IEEE International Conference on Data Mining (ICDM’09). IEEE, 677--686.

Digital Library

[136]

Angel Navia-Vázquez and Emilio Parrado-Hernandez. 2006. Distributed support vector machines. IEEE Transactions on Neural Networks 17, 4 (2006), 1091--1097.

Digital Library

[137]

Thanh-Nghi Do. 2015. Non-linear classification of massive datasets with a parallel algorithm of local support vector machines. In Advanced Computational Methods for Knowledge Engineering. Springer, 231--241.

[138]

Nasullah Khalid Alham, Maozhen Li, Yang Liu, and Man Qi. 2013. A MapReduce-based distributed SVM ensemble for scalable image classification and annotation. Computers 8 Mathematics with Applications 66, 10 (2013), 1920--1934.

Digital Library

[139]

Naveeen Kumar Shrivastava, Praneet Saurabh, and Bhupendra Verma. 2011. An efficient approach parallel support vector machine for classification of diabetes dataset. International Journal of Computer Applications 36, 6 (2011), 19--24.

[140]

Shibin Qiu and Terran Lane. 2005. Parallel computation of RBF kernels for support vector classifiers. In SDM. SIAM, 334--345.

[141]

Caoxie Zhang, Honglak Lee, and Kang G Shin. 2012. Efficient distributed linear classification algorithms via the alternating direction method of multipliers. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, Neil D. Lawrence and Mark Girolami (Eds.), and In Proceedings of Machine Learning Research (PMLR'12), Vol. 22. 1398--1406.

[142]

João F. C. Mota, João M. F. Xavier, Pedro M. Q. Aguiar, and Markus Puschel. 2013. D-ADMM: A communication-efficient distributed algorithm for separable optimization. IEEE Transactions on Signal Processing 61, 10 (2013), 2718--2723.

Digital Library

[143]

Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine Learning 3, 1 (2011), 1--122.

Digital Library

[144]

Timo M. Deist, A. Jochems, Johan van Soest, Georgi Nalbantov, Cary Oberije, SeÃ¡n Walsh, Michael Eble, Paul Bulens, Philippe Coucke, Wim Dries, Andre Dekker, and Philippe Lambin. 2017. Infrastructure and distributed learning methodology for privacy-preserving multi-centric rapid learning health care: euroCAT. Clinical and Translational Radiation Oncology 4, Supplement C (2017), 24--31.

[145]

H. Wang, Y. Gao, Y. Shi, and R. Wang. 2016. Group-based alternating direction method of multipliers for distributed linear classification. IEEE Transactions on Cybernetics PP, 99 (2016), 1--15.

[146]

Hsiang-Fu Yu, Cho-Jui Hsieh, Kai-Wei Chang, and Chih-Jen Lin. 2012. Large linear classification when data cannot fit in memory. ACM Transactions on Knowledge Discovery from Data 5, 4 (2012), 23:1--23:23.

Digital Library

[147]

Martin Zinkevich, Markus Weimer, Lihong Li, and Alex J. Smola. 2010. Parallelized stochastic gradient descent. In Advances in Neural Information Processing Systems 23, J. D. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, and A. Culotta (Eds.). Curran Associates, 2595--2603.

Digital Library

[148]

H. Wang, Y. Gao, Y. Shi, and R. Wang. 2017. Group-based alternating direction method of multipliers for distributed linear classification. IEEE Transactions on Cybernetics 47, 11 (Nov 2017), 3568--3582.

[149]

Dmitry Pechyony, Libin Shen, and Rosie Jones. 2011. Solving large scale linear SVM with distributed block minimization. In NIPS 2011 Workshop on Big Learning: Algorithms, Systems and Tools for Learning at Scale. Citeseer.

[150]

Roberto Díaz Morales and Ángel Navia Vázquez. 2016. Improving the efficiency of IRWLS SVMs using parallel Cholesky factorization. Pattern Recognition Letters 84, Supplement C (2016), 91--98.

Digital Library

[151]

Roberto Díaz Morales and Ángel Navia Vázquez. 2017. LIBIRWLS: A parallel IRWLS library for full and budgeted SVMs. Knowledge-Based Systems 136 (2017), 183--186.

Digital Library

[152]

Roberto Díaz Morales, H. Y. Molina-Bulla, and Ángel Navia Vázquez. 2011. Parallel semiparametric support vector machines. In International Joint Conference on Neural Networks. IEEE, 475--481.

[153]

Doan Thanh Nghi. 2015. Parallel, imbalanced bagging power mean SVM for large scale visual classification with million images and thousand classes. Journal of Science 4, 4 (2015), 26--40.

[154]

Stephen Winters-Hilt and Kenneth Armond Jr. 2008. Distributed SVM learning and support vector reduction. International Journal of Computing and Optimization 4, 1 (2017), 91--114.

[155]

Yuanqing Lin, Fengjun Lv, Shenghuo Zhu, Ming Yang, Timothee Cour, Kai Yu, Liangliang Cao, and Thomas Huang. 2011. Large-scale image classification: Fast feature extraction and SVM training. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 1689--1696.

Digital Library

[156]

Jeffrey Dean and Sanjay Ghemawat. 2008. MapReduce: Simplified data processing on large clusters. Communications of the ACM 51, 1 (Jan. 2008), 107--113.

Digital Library

[157]

Cheng Chu, Sang Kyun Kim, Yi-An Lin, YuanYuan Yu, Gary Bradski, Andrew Y. Ng, and Kunle Olukotun. 2007. Map-reduce for machine learning on multicore. Advances in Neural Information Processing Systems 19 (2007), 281.

[158]

H. Tao, B. Wu, and X. Lin. 2014. Budgeted mini-batch parallel gradient descent for support vector machines on spark. In 20th IEEE International Conference on Parallel and Distributed Systems (ICPADS’14). IEEE, 945--950.

[159]

Thanh-Nghi Do and Jean-Daniel Fekete. 2007. Large scale classification with support vector machine algorithms. In 6th International Conference on Machine Learning and Applications (ICMLA’07). IEEE, 7--12.

Digital Library

[160]

Katarina Grolinger, Michael Hayes, Wilson A. Higashino, Alexandra L’Heureux, David S. Allison, and Miriam A. M. Capretz. 2014. Challenges for MapReduce in big data. In IEEE World Congress on Services. IEEE, 182--189.

Digital Library

[161]

Yang You, Haohuan Fu, Shuaiwen Leon Song, Amanda Randles, Darren Kerbyson, Andres Marquez, Guangwen Yang, and Adolfy Hoisie. 2015. Scaling support vector machines on modern HPC platforms. Journal of Parallel and Distributed Computing 76 (2015), 16--31.

Digital Library

Cited By

Zhang XMa SWang XHe ZChang YJiang X(2025)Machine Learning Prediction of Particle Self-rotation Velocity Distribution in Cyclone SeparatorsEnergy10.1016/j.energy.2025.134452(134452)Online publication date: Jan-2025
https://doi.org/10.1016/j.energy.2025.134452
Hao JYan GWang LPei HXiao XZhang B(2024)A Lightweight Transmission Line Foreign Object Detection Algorithm Incorporating Adaptive Weight PoolingElectronics10.3390/electronics1323464513:23(4645)Online publication date: 25-Nov-2024
https://doi.org/10.3390/electronics13234645
Nguyen TQuoc Viet hung NNguyen THuynh TNguyen TWeidlich MYin H(2024)Manipulating Recommender Systems: A Survey of Poisoning Attacks and CountermeasuresACM Computing Surveys10.1145/367732857:1(1-39)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677328
Show More Cited By

Index Terms

Parallel Computing of Support Vector Machines: A Survey
1. Computing methodologies
  1. Machine learning
  2. Parallel computing methodologies
    1. Parallel algorithms

Recommendations

Wavelet twin support vector machines based on glowworm swarm optimization

Twin support vector machine is a machine learning algorithm developing from standard support vector machine. The performance of twin support vector machine is always better than support vector machine on datasets that have cross regions. Recently ...
A parallel vector quantization algorithm for SIMD multiprocessor systems
DCC '95: Proceedings of the Conference on Data Compression

Summary form only given , as follows. This article proposes a parallel vector quantization (VQ) algorithm for an exhaustive search of codebooks on a single-instruction-multiple-data (SIMD) multiprocessor. The proposed parallel VQ algorithm can be ...
A secure and efficient file protecting system based on SHA3 and parallel AES

We propose a secure and efficient file protecting system (SEFPS) that uses currently excellent SHA3 and parallel AES.We prove the security of SEFPS in terms of confidentiality and integrity.We design and implement the three variants of SEFPS adopting ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 51, Issue 6

November 2019

786 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/3303862

Editor:
Sartaj Sahni
Department of Computer and Information Science and Engineering

Issue’s Table of Contents

Copyright © 2019 Owner/Author.

This work is licensed under a Creative Commons Attribution-NoDerivs International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 January 2019

Accepted: 01 September 2018

Revised: 01 July 2018

Received: 01 June 2017

Published in CSUR Volume 51, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Tutorial
Research
Refereed

Funding Sources

Universities of Borås, Skövde and Gothenburg in Sweden
Schliep’s lab

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
1,273
Total Downloads

Downloads (Last 12 months)97
Downloads (Last 6 weeks)9

Reflects downloads up to 07 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang XMa SWang XHe ZChang YJiang X(2025)Machine Learning Prediction of Particle Self-rotation Velocity Distribution in Cyclone SeparatorsEnergy10.1016/j.energy.2025.134452(134452)Online publication date: Jan-2025
https://doi.org/10.1016/j.energy.2025.134452
Hao JYan GWang LPei HXiao XZhang B(2024)A Lightweight Transmission Line Foreign Object Detection Algorithm Incorporating Adaptive Weight PoolingElectronics10.3390/electronics1323464513:23(4645)Online publication date: 25-Nov-2024
https://doi.org/10.3390/electronics13234645
Nguyen TQuoc Viet hung NNguyen THuynh TNguyen TWeidlich MYin H(2024)Manipulating Recommender Systems: A Survey of Poisoning Attacks and CountermeasuresACM Computing Surveys10.1145/367732857:1(1-39)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677328
Liang GLai ZKong H(2024)Support vector machine with discriminative low‐rank embeddingCAAI Transactions on Intelligence Technology10.1049/cit2.123299:5(1249-1262)Online publication date: 2-Apr-2024
https://dl.acm.org/doi/10.1049/cit2.12329
Zhou LYin HZhao HWei JHu DLeung V(2024)A Comprehensive Survey of Artificial Intelligence Applications in UAV-Enabled Wireless NetworksDigital Communications and Networks10.1016/j.dcan.2024.11.005Online publication date: Nov-2024
https://doi.org/10.1016/j.dcan.2024.11.005
Li HQian LHong MHuang HZhang YYan Q(2024)Effective anti-submarine decision support system based on heuristic rank-based Dijkstra and adaptive threshold partitioning mechanismApplied Soft Computing10.1016/j.asoc.2024.111718161(111718)Online publication date: Aug-2024
https://doi.org/10.1016/j.asoc.2024.111718
Chhetri ASaini DYadav MPal N(2024)Performance analysis of machine learning algorithms for estimation of EV penetrationMicrosystem Technologies10.1007/s00542-024-05804-xOnline publication date: 29-Oct-2024
https://doi.org/10.1007/s00542-024-05804-x
Ranjith Kumar MSudeesh Kumar VTharun Kaarthik GChatiyode VSrinivasan Anusha JRevathi P(2024)Snowfall Forecasting with Enhanced Accuracy: Leveraging Multi-Class SVM for Meteorological PredictionsComputational Intelligence in Data Science10.1007/978-3-031-69986-3_24(312-328)Online publication date: 30-Aug-2024
https://doi.org/10.1007/978-3-031-69986-3_24
van den Berg CEybers S(2024)Investigating Machine Learning Techniques Used for the Detection of Class Noise in Data: A Systematic Literature ReviewIntelligent Computing10.1007/978-3-031-62277-9_9(128-147)Online publication date: 13-Jun-2024
https://doi.org/10.1007/978-3-031-62277-9_9
Wan LLi XXu JSun LWang XLiu K(2023)Application of Graph Learning With Multivariate Relational Representation Matrix in Vehicular Social NetworksIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2022.322432624:3(2789-2799)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1109/TITS.2022.3224326
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents