Abstract
This paper designs a parallel video retrieval based on Spark and deep hash. The method comprises deep feature extraction using a convolution neural network based on partial semantic weighted aggregation; filtering features of image information in deep networks; the extraction and distributed storage of video summary keys; the establishment of distributed product quantitative hash coding model of image, realizing the distributed coding compression of high-dimensional features. The video parallel retrieval method proposed in this design has the advantages of high retrieval accuracy and good retrieval efficiency.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Xiong, S., Lei, M., Liu, C.: Batch Images parallel retrieval based on deep hashing. J. Chongqing Univ. Technol. (Nat. Sci.) (2018)
Jiang, Y., Zhuo, J., Zhang, J., et al.: The optimization of parallel convolutional RBM based on spark. Int. J. Wavelets Multiresolut. Inf. Process. (2018)
Mou, F.U., Yang, H., Tangmei, W.U., et al.: Fast video transcoding method based on spark streaming. J. Comput. Appl. 38, 3500 (2018)
Qin, J., Chen, J., Xiang, X., et al.: A privacy-preserving image retrieval method based on deep learning and adaptive weighted fusion. J. Real-Time Image Process. 17(1), 161–173 (2020)
Dong, Y., Li, J.: Video retrieval based on deep convolutional neural network (2017)
Wan, L., Zhang, G., Li, H., et al.: A novel bearing fault diagnosis method using spark-based parallel ACO-K-means clustering algorithm. IEEE Access PP(99), 1 (2021)
Shi, L., Liu, S., Shi, Y., et al.: Sea ice concentration products over polar regions with Chinese FY3C/MWRI Data (2021)
Liu, P., Ye, S., Wang, C., et al.: Spark-based parallel genetic algorithm for simulating a solution of optimal deployment of an underwater sensor network. Sensors 19(12), 2717–2727 (2019)
Langer, M., Hall, A., He, Z., et al.: MPCA SGD—a method for distributed training of deep learning models on spark. IEEE Trans. Parallel Distrib. Syst. 29, 2540–2556 (2018)
Hou, J., Sheng, W.U., Yingna, L.I.: Research on parallel K-means clustering model based on spark. Comput. Digit. Eng. (2018)
Qin, J., Li, H., Xiang, X., et al.: An encrypted image retrieval method based on Harris corner optimization and LSH in cloud computing. IEEE Access 7, 24626–24633 (2019)
Cao, Y., Wang, N., Xu, Z., et al.: Network big data classification processing method based on Spark and distributed KNN classifier. Appl. Res. Comput. (2019)
Deng, Q., Yang, N.: Research of improved parallel K-means algorithm based on spark framework. Intell. Comput. Appl. 8, 76–78 (2018)
Jing, W., Zhang, D., Song, H.: An application of ternary hash retrieval method for remote sensing images in panoramic video. IEEE Access 8, 140822–140830 (2020)
Wang, Y., Nie, X., Shi, Y., et al.: Attention-based video hashing for large-scale video retrieval. IEEE Trans. Cogn. Dev. Syst. PP(99), 1 (2019)
Hajkacem, M., N’Cir, C., Essoussi, N.: A parallel text clustering method using Spark and hashing. Computing (6) (2021)
Gupta, A., Thakur, H., Shrivastava, R., et al.: A big data analysis framework using apache spark and deep learning. In: IEEE International Conference on Data Mining Workshops (2017)
Lunga, D., Gerrand, J., Yang, L., et al.: Apache spark accelerated deep learning inference for large scale satellite image analytics. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 13, 271–283 (2020)
Khan, S., Liu, X., Alam, M.: A spark ML driven preprocessing approach for deep learning based scholarly data applications (2019)
Uddin, M.A., Joolee, J.B., Sohn, K.A.: Dynamic facial expression understanding using deep spatiotemporal LDSP on spark. IEEE Access PP(99), 1 (2021)
Yang, F., Wang, H., Fu, J.: Improvement of recommendation algorithm based on collaborative deep learning and its parallelization on spark. J. Parallel Distrib. Comput. 148(2), 58–68 (2021)
Zaouk, K., Song, F., Lyu, C., et al.: Neural-based modeling for performance tuning of spark data analytics (2021)
Takam, C.A., Samba, O., Kouanou, A.T., et al.: Spark architecture for deep learning-based dose optimization in medical imaging. Inform. Med. Unlocked 19, 100335 (2020)
Sundareswaran, A., Lavanya, K.: Real-time vehicle traffic prediction in apache spark using ensemble learning for deep neural networks. Int. J. Intell. Inf. Technol. (IJIIT) 16, 19–36 (2020)
Khan, S., Khan, M., Iqbal, N., et al.: Spark based parallel deep neural network model for classification of large scale RNAs into piRNAs and non-piRNAs. IEEE Access PP(99), 1 (2020)
Singh, P., Singh, S., Mishra, P.K., et al.: A data structure perspective to the RDD-based Apriori algorithm on spark. SSRN Electron. J. (2019)
Wang, L., Chen, C., Yun, S.U.: Design and implementation of massive ship density distribution calculation system based on spark. Ship Electron. Eng. (2019)
Lunga, D., Gerrand, J., Yang, H.L., et al.: Apache spark accelerated deep learning inference for large scale satellite image analytics (2019)
Shmeis, Z., Jaber, M.: A rewrite-based optimizer for spark. Future Gener. Comput. Syst. 98, 586–599 (2019)
Acknowledgement
This work was supported in part by Xuzhou Science and Technology Plan Project (Grant No. KC19003). Science and technology project of Jiangsu Provincial Department of housing and construction (Grant No. 2018ZD265). Major projects of natural science research in Colleges and universities of Jiangsu Province (Grant No. 19KJA470002) and Xuzhou Science and Technology Plan Project (KC21303).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Li, J., Bei, L., Li, D., Cui, P., Huang, K. (2022). A Video Parallel Retrieval Method Based on Deep Hash. In: Jiang, D., Song, H. (eds) Simulation Tools and Techniques. SIMUtools 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 424. Springer, Cham. https://doi.org/10.1007/978-3-030-97124-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-97124-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-97123-6
Online ISBN: 978-3-030-97124-3
eBook Packages: Computer ScienceComputer Science (R0)