[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article

HCFNN: : High-order coverage function neural network for image classification

Published: 01 November 2022 Publication History

Highlights

A more flexible HCF neuron model for DNNs is introduced; it constructs geometries in an n-dimensional space by changing weights and hyper-parameters and thus, possesses higher variability and plasticity. Furthermore, the approximation theorem and proof for arbitrary continuous infinite functions are presented, and the fitting ability of the HCF neuron 95 model is demonstrated.
HCFNN architecture based on the HCF neuron is proposed; it is used to mine specific feature representations and achieve adaptive parameter learning. Next, a novel adaptive optimization method for weights and hyper-parameters is proposed to achieve effective network learning. The 100 learned network model has better expression and learning ability with fewer neurons.
We conduct experiments on nine datasets in several domains, including the two-spirals problem, natural object recognition, face recognition, and person re-ID. Experimental results show that the proposed method has better 105 learning performance and generalization ability than the commonly used M-P and RBF neural networks. In addition, our method can improve the performance of various image recognition tasks and acquire good generalization.

Abstract

Recent advances in deep neural networks (DNNs) have mainly focused on innovations in network architecture and loss function. In this paper, we introduce a flexible high-order coverage function (HCF) neuron model to replace the fully-connected (FC) layers. The approximation theorem and proof for the HCF are also presented to demonstrate its fitting ability. Unlike the FC layers, which cannot handle high-dimensional data well, the HCF utilizes weight coefficients and hyper-parameters to mine underlying geometries with arbitrary shapes in an n-dimensional space. To explore the power and potential of our HCF neuron model, a high-order coverage function neural network (HCFNN) is proposed, which incorporates the HCF neuron as the building block. Moreover, a novel adaptive optimization method for weights and hyper-parameters is designed to achieve effective network learning. Comprehensive experiments on nine datasets in several domains validate the effectiveness and generalizability of the HCF and HCFNN. The proposed method provides a new perspective for further developments in DNNs and ensures wide application in the field of image classification. The source code is available at https://github.com/Tough2011/HCFNet.git

References

[1]
S. He, L. Schomaker, GR-RNN: global-context residual recurrent neural networks for writer identification, Pattern Recognit 117 (2021) 107975.
[2]
Z. Lei, S. Yang, A.U. Haq, D. Zhang, A.O. Francis, An entity-weights-based convolutional neural network for large-sale complex knowledge embedding, Pattern Recognit (2022) 108841.
[3]
C. Yan, G. Pang, X. Bai, C. Liu, X. Ning, L. Gu, J. Zhou, Beyond triplet loss: person re-identification with fine-grained difference-aware pairwise loss, IEEE Trans Multimedia 24 (2021) 1665–1677.
[4]
H. Shi, L. Wang, N. Zheng, G. Hua, W. Tang, Loss functions for pose guided person image generation, Pattern Recognit 122 (2022) 108351.
[5]
C. Wang, X. Ning, L. Sun, L. Zhang, W. Li, X. Bai, Learning discriminative features by covering local geometric space for point cloud analysis, IEEE Trans. Geosci. Remote Sens. 60 (2022) 1–15.
[6]
C. Wang, X. Wang, J. Zhang, L. Zhang, X. Bai, X. Ning, J. Zhou, E. Hancock, Uncertainty estimation for stereo matching based on evidential deep learning, Pattern Recognit 124 (2022) 108498.
[7]
G. Wang, W. Li, L. Zhang, L. Sun, P. Chen, L. Yu, X. Ning, Encoder-x: solving unknown coefficients automatically in polynomial fitting by using an autoencoder, IEEE Trans Neural Netw Learn Syst (2021).
[8]
Geldmacher, S. David, The neurobiology of learning and memory, Journal of Neuro-Ophthalmology, volume 29, 2010, p. 257.
[9]
W.S. McCulloch, W. Pitts, A logical calculus of the ideas immanent in nervous activity, Bull Math Biophys 5 (4) (1943) 115–133.
[10]
S. Gao, M. Zhou, Y. Wang, J. Cheng, H. Yachi, J. Wang, Dendritic neuron model with effective learning algorithms for classification, approximation, and prediction, IEEE Transactions on Neural Networks and Learning Systems, volume 30, 2019, pp. 601–614.
[11]
C. Koch, I. Segev, The role of single neurons in information processing, Nature Neuroscience, volume 3, 2000, pp. 1171–1177.
[12]
Costa, Rui, One cell to rule them all, and in the dendrites bind them, Frontiers in Synaptic Neuroscience, volume 3, 2011, p. 5.
[13]
D.S. Broomhead, D. Lowe, Radial basis functions, multi-variable functional interpolation and adaptive networks, Technical Report, Royal Signals and Radar Establishment Malvern (United Kingdom), 1988.
[14]
S.-Q. Zhang, Z.-H. Zhou, Flexible transmitter network, Neural Comput 33 (11) (2021) 2951–2970.
[15]
S. Kiranyaz, J. Malik, H.B. Abdallah, T. Ince, A. Iosifidis, M. Gabbouj, Self-organized operational neural networks with generative neurons, Neural Networks 140 (2021) 294–308.
[16]
J. Xie, Z. Ma, D. Chang, G. Zhang, J. Guo, Gpca: a probabilistic framework for gaussian process embedded channel attention, IEEE Trans Pattern Anal Mach Intell (2021).
[17]
R. Du, J. Xie, Z. Ma, D. Chang, Y.-Z. Song, J. Guo, Progressive learning of category-consistent multi-granularity features for fine-grained visual classification, IEEE Trans Pattern Anal Mach Intell (2021).
[18]
L. Zhang, B. Zhang, A geometrical representation of mcculloch-pitts neural model and its applications, IEEE Trans. Neural Networks 10 (4) (1999) 925–929.
[19]
W. Shoujue, L. Jiangliang, Geometrical learning, descriptive geometry, and biomimetic pattern recognition, Neurocomputing 67 (2005) 9–28,.
[20]
S.-j. Wang, Z.-z. Li, Discussion on the basic mathematical models of neurons in general purpose neurocomputer, Acta Electronica Sinica 29 (5) (2001) 577–580.
[21]
G. Cybenko, Approximation by superpositions of a sigmoidal function, Mathematics of control, signals and systems 2 (4) (1989) 303–314.
[22]
D.E. Rumelhart, G.E. Hinton, R.J. Williams, Learning internal representations by error propagation, Technical Report, California Univ San Diego La Jolla Inst for Cognitive Science, 1985.
[23]
M. Mitchell, An introduction to genetic algorithms, MIT press, 1998.
[24]
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al., Imagenet large scale visual recognition challenge, Int J Comput Vis 115 (3) (2015) 211–252.
[25]
I. Kemelmacher-Shlizerman, S.M. Seitz, D. Miller, E. Brossard, The megaface benchmark: 1 million faces for recognition at scale, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 4873–4882.
[26]
E. Ristani, F. Solera, R. Zou, R. Cucchiara, C. Tomasi, Performance measures and a data set for multi-target, multi-camera tracking, European Conference on Computer Vision, Springer, 2016, pp. 17–35.
[27]
J.A. Suykens, J. Vandewalle, Least squares support vector machine classifiers, Neural processing letters 9 (3) (1999) 293–300.
[28]
A. Krizhevsky, Learning multiple layers of features from tiny images, Technical Report, 2009.
[29]
G.B. Huang, M. Mattar, T. Berg, E. Learned-Miller, Labeled faces in the wild: A database forstudying face recognition in unconstrained environments, Workshop on faces in’Real-Life’Images: detection, alignment, and recognition, 2008.
[30]
L. Zheng, L. Shen, L. Tian, S. Wang, J. Wang, Q. Tian, Scalable person re-identification: A benchmark, Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1116–1124.
[31]
W. Li, R. Zhao, T. Xiao, X. Wang, Deepreid: Deep filter pairing neural network for person re-identification, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 152–159.
[32]
K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.
[33]
K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, Computer Science (2014).
[34]
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1–9.
[35]
M. Wang, W. Deng, Deep face recognition: a survey, Neurocomputing 429 (2021) 215–244.
[36]
Y. Taigman, M. Yang, M. Ranzato, L. Wolf, Deepface: Closing the gap to human-level performance in face verification, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 1701–1708.
[37]
F. Schroff, D. Kalenichenko, J. Philbin, Facenet: A unified embedding for face recognition and clustering, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 815–823.
[38]
Y. Wen, K. Zhang, Z. Li, Y. Qiao, A discriminative feature learning approach for deep face recognition, European Conference on Computer Vision, Springer, 2016, pp. 499–515.
[39]
W. Liu, Y. Wen, B. Raj, R. Singh, A. Weller, Sphereface revived: unifying hyperspherical face recognition, IEEE Trans Pattern Anal Mach Intell (2022).
[40]
J. Deng, J. Guo, N. Xue, S. Zafeiriou, Arcface: Additive angular margin loss for deep face recognition, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 4690–4699.
[41]
H. Wang, Y. Wang, Z. Zhou, X. Ji, D. Gong, J. Zhou, Z. Li, W. Liu, Cosface: Large margin cosine loss for deep face recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 5265–5274.
[42]
S. Li, J. Xu, X. Xu, P. Shen, S. Li, B. Hooi, Spherical confidence learning for face recognition, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 15629–15637.
[43]
Y. Shi, X. Yu, K. Sohn, M. Chandraker, A.K. Jain, Towards universal representation learning for deep face recognition, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 6817–6826.
[44]
S. Liao, Z. Lei, D. Yi, S.Z. Li, A benchmark study of large-scale unconstrained face recognition, IEEE International Joint Conference on Biometrics, IEEE, 2014, pp. 1–8.
[45]
J. Si, H. Zhang, C.-G. Li, J. Kuen, X. Kong, A.C. Kot, G. Wang, Dual attention matching network for context-aware feature sequence based person re-identification, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 5363–5372.
[46]
Y. Sun, L. Zheng, Y. Yang, Q. Tian, S. Wang, Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline), Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 480–496.
[47]
W. Li, X. Zhu, S. Gong, Harmonious attention network for person re-identification, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 2285–2294.
[48]
C.-P. Tay, S. Roy, K.-H. Yap, Aanet: Attribute attention network for person re-identifications, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 7134–7143.
[49]
Z. Dai, M. Chen, X. Gu, S. Zhu, P. Tan, Batch dropblock network for person re-identification and beyond, Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 3691–3701.
[50]
H. Luo, W. Jiang, Y. Gu, F. Liu, X. Liao, S. Lai, J. Gu, A strong baseline and batch normalization neck for deep person re-identification, IEEE Trans Multimedia 22 (10) (2019) 2597–2609.
[51]
W.-D. Jin, J. Xu, Q. Han, Y. Zhang, M.-M. Cheng, Cdnet: complementary depth network for rgb-d salient object detection, IEEE Trans. Image Process. 30 (2021) 3376–3390.
[52]
B.N. Xia, Y. Gong, Y. Zhang, C. Poellabauer, Second-order non-local attention networks for person re-identification, Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 3760–3769.
[53]
Y. Ge, F. Zhu, D. Chen, R. Zhao, et al., Self-paced contrastive learning with hybrid memory for domain adaptive object re-id, Adv Neural Inf Process Syst 33 (2020) 11309–11321.
[54]
R. Quan, X. Dong, Y. Wu, L. Zhu, Y. Yang, Auto-reid: Searching for a part-aware convnet for person re-identification, Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 3750–3759.

Cited By

View all

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Pattern Recognition
Pattern Recognition  Volume 131, Issue C
Nov 2022
837 pages

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 November 2022

Author Tags

  1. DNNs
  2. Neuron modeling
  3. Heuristic algorithm
  4. Back propagation
  5. Computer vision

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 21 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)The Analysis of Smarter FutureJournal of Organizational and End User Computing10.4018/JOEUC.35724936:1(1-20)Online publication date: 7-Nov-2024
  • (2024)Heading Towards Robust SolutionsJournal of Organizational and End User Computing10.4018/JOEUC.35441336:1(1-23)Online publication date: 24-Sep-2024
  • (2024)Real-Time Classification Model of Public Emergencies Using Fusion Expansion NetworkJournal of Organizational and End User Computing10.4018/JOEUC.34524536:1(1-25)Online publication date: 21-Jun-2024
  • (2024)Improving Robot-Assisted Virtual Teaching Using Transformers, GANs, and Computer VisionJournal of Organizational and End User Computing10.4018/JOEUC.33648136:1(1-32)Online publication date: 30-Jan-2024
  • (2024)TAE-RWPInternational Journal of Intelligent Systems10.1155/2024/60541722024Online publication date: 1-Jan-2024
  • (2024)CIM-KF: Efficient Computing-in-memory Circuits for Full-Process Execution of Kalman Filter AlgorithmProceedings of the 53rd International Conference on Parallel Processing10.1145/3673038.3673079(1032-1041)Online publication date: 12-Aug-2024
  • (2024)FGRC-NetExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.121167235:COnline publication date: 10-Jan-2024
  • (2024)Remote sensing image location based on improved Yolov7 target detectionPattern Analysis & Applications10.1007/s10044-024-01276-x27:2Online publication date: 29-Apr-2024
  • (2024)Enhancing machine vision: the impact of a novel innovative technology on video question-answeringSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-023-09536-428:11-12(6969-6982)Online publication date: 1-Jun-2024
  • (2024)Fast continuous patch-based artistic style transfer for videosThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-03157-640:9(6123-6136)Online publication date: 1-Sep-2024
  • Show More Cited By

View Options

View options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media