[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to main content

Non-uniform Initialization of Inputs Groupings in Contextual Neural Networks

  • Conference paper
  • First Online:
Intelligent Information and Database Systems (ACIIDS 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11432))

Included in the following conference series:

Abstract

Contextual neural networks which are using neurons with conditional aggregation functions were found to be efficient and useful generalizations of classical multilayer perceptrons. They allow to generate neural classification models with good generalization and low activity of connections between neurons in hidden layers. The key factor to build such solutions is achieving self-consistency between continuous values of weights of neurons’ connections and their mutually related non-continuous aggregation priorities. This allows to optimize neuron inputs aggregation priorities by simultaneous gradient-based optimization of connections’ weights with generalized BP algorithm. But such method additionally needs initial setting of connections groupings (scan-paths) to define priorities of signals during first ω epochs of training. In earlier studies all connections were initially assigned to a single group to give neurons access to all input signals at the beginning of training. We found out that such uniform solution not always is the best one. Thus within this text we compare efficiency of training of contextual neural networks with uniform and non-uniform, random initialization of connections groupings. On this basis we also discuss the properties of analyzed training algorithm which are related to characteristics of used scan-paths initialization methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
GBP 19.95
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
GBP 35.99
Price includes VAT (United Kingdom)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
GBP 44.99
Price includes VAT (United Kingdom)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Huk, M.: Learning distributed selective attention strategies with the Sigma-if neural network. In: Akbar, M., Hussain, D. (eds.) Advances in Computer Science and IT, pp. 209–232. InTech, Vukovar (2009)

    Google Scholar 

  2. Huk, M.: Backpropagation generalized delta rule for the selective attention Sigma-if artificial neural network. Int. J. Appl. Math. Comput. Sci. 22, 449–459 (2012)

    Article  MathSciNet  Google Scholar 

  3. Huk, M.: Notes on the generalized backpropagation algorithm for contextual neural networks with conditional aggregation functions. J. Intell. Fuzzy Syst. 32, 1365–1376 (2017)

    Article  Google Scholar 

  4. Szczepanik, M., Jóźwiak, I.: Data management for fingerprint recognition algorithm based on characteristic points’ groups. In: Pechenizkiy, M., Wojciechowski, M. (eds.) New Trends in Databases and Information Systems. AISC, vol. 185, pp. 425–432. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-32518-2_40

    Google Scholar 

  5. Huk, M.: Measuring the effectiveness of hidden context usage by machine learning methods under conditions of increased entropy of noise. In: 2017 3rd IEEE International Conference on Cybernetics (CYBCONF), pp. 1–6. IEEE (2017)

    Google Scholar 

  6. Privitera, C.M., Azzariti, M., Stark, L.W.: Locating regions-of-interest for the Mars Rover expedition. Int. J. Remote Sens. 21, 3327–3347 (2000)

    Article  Google Scholar 

  7. Raczkowski, D., Canning, A.: Thomas-Fermi charge mixing for obtaining self-consistency in density functional calculations. Phys. Rev. B 64, 121101–121105 (2001)

    Article  Google Scholar 

  8. UCI Machine Learning Repository. http://archive.ics.uci.edu/ml

  9. Golub, T.R., et al.: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286, 531–537 (1999)

    Article  Google Scholar 

  10. Armstrong, S.A.: MLL translocations specify a distinct gene expression profile that distinguishes a unique leukemia. Nat. Genet. 30, 41–47 (2002)

    Article  Google Scholar 

  11. Khan, J., et al.: Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks. Nat. Med. 7(6), 673–679 (2001)

    Article  Google Scholar 

  12. Miller, G.A.: The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol. Rev. 63(2), 81–97 (1956)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Maciej Huk .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Huk, M. (2019). Non-uniform Initialization of Inputs Groupings in Contextual Neural Networks. In: Nguyen, N., Gaol, F., Hong, TP., Trawiński, B. (eds) Intelligent Information and Database Systems. ACIIDS 2019. Lecture Notes in Computer Science(), vol 11432. Springer, Cham. https://doi.org/10.1007/978-3-030-14802-7_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-14802-7_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-14801-0

  • Online ISBN: 978-3-030-14802-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics