research-article

An improved algorithm for learning long-term dependency problems in adaptive processing of data structures

Authors:

Siu-Yeung Cho,

Zheru Chi,

Wan-Chi Siu,

Ah Chung TsoiAuthors Info & Claims

IEEE Transactions on Neural Networks, Volume 14, Issue 4

Pages 781 - 793

https://doi.org/10.1109/TNN.2003.813831

Published: 01 July 2003 Publication History

Abstract

Many researchers have explored the use of neural-network representations for the adaptive processing of data structures. One of the most popular learning formulations of data structure processing is backpropagation through structure (BPTS). The BPTS algorithm has been successful applied to a number of learning tasks that involve structural patterns such as logo and natural scene classification. The main limitations of the BPTS algorithm are attributed to slow convergence speed and the long-term dependency problem for the adaptive processing of data structures. In this paper, an improved algorithm is proposed to solve these problems. The idea of this algorithm is to optimize the free learning parameters of the neural network in the node representation by using least-squares-based optimization methods in a layer-by-layer fashion. Not only can fast convergence speed be achieved, but the long-term dependency problem can also be overcome since the vanishing of gradient information is avoided when our approach is applied to very deep tree structures.

Cited By

View all

Zhang JWang XZhang HSun HWang KLiu XAtlee JBultan TWhittle J(2019)A novel neural source code representation based on abstract syntax treeProceedings of the 41st International Conference on Software Engineering10.1109/ICSE.2019.00086(783-794)Online publication date: 25-May-2019
https://dl.acm.org/doi/10.1109/ICSE.2019.00086
Shaila SVadivel A(2016)Indexing and encoding based image feature representation with bin overlapped similarity measure for CBIR applicationsJournal of Visual Communication and Image Representation10.1016/j.jvcir.2016.01.00336:C(40-55)Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.1016/j.jvcir.2016.01.003
Rahman MChow T(2010)Content-based hierarchical document organization using multi-layer hybrid network and tree-structured featuresExpert Systems with Applications: An International Journal10.1016/j.eswa.2009.09.00237:4(2874-2881)Online publication date: 1-Apr-2010
https://dl.acm.org/doi/10.1016/j.eswa.2009.09.002
Show More Cited By

Recommendations

Efficient Learning in Adaptive Processing of Data Structures

Many researchers have explored the use of neural network models for the adaptive processing of data structures. The learning formulation for one of the models is known as the Backpropagation Through Structure (BPTS) algorithm. The main limitations of ...
An adaptive backpropagation algorithm for long-term electricity load forecasting
Abstract
Artificial Neural Networks (ANNs) have been widely used to determine future demand for power in the short, medium, and long terms. However, research has identified that ANNs could cause inaccurate predictions of load when used for long-term ...
Adaptive Processing of Data Structures
ICCIMA '99: Proceedings of the 3rd International Conference on Computational Intelligence and Multimedia Applications

Many artificial and natural systems are often more adequately modeled using data structures, e.g., graphs, trees. For example, it is often more convenient to represent an image using data structures than by representing it using pixels. The data ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Neural Networks

IEEE Transactions on Neural Networks Volume 14, Issue 4

July 2003

246 pages

ISSN:1045-9227

Issue’s Table of Contents

Publisher

IEEE Press

Publication History

Published: 01 July 2003

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zhang JWang XZhang HSun HWang KLiu XAtlee JBultan TWhittle J(2019)A novel neural source code representation based on abstract syntax treeProceedings of the 41st International Conference on Software Engineering10.1109/ICSE.2019.00086(783-794)Online publication date: 25-May-2019
https://dl.acm.org/doi/10.1109/ICSE.2019.00086
Shaila SVadivel A(2016)Indexing and encoding based image feature representation with bin overlapped similarity measure for CBIR applicationsJournal of Visual Communication and Image Representation10.1016/j.jvcir.2016.01.00336:C(40-55)Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.1016/j.jvcir.2016.01.003
Rahman MChow T(2010)Content-based hierarchical document organization using multi-layer hybrid network and tree-structured featuresExpert Systems with Applications: An International Journal10.1016/j.eswa.2009.09.00237:4(2874-2881)Online publication date: 1-Apr-2010
https://dl.acm.org/doi/10.1016/j.eswa.2009.09.002
Fu HZhang SChi ZFeng DZhao X(2009)Tree structures with attentive objects for image classification using a neural networkProceedings of the 2009 international joint conference on Neural Networks10.5555/1704555.1704714(3057-3061)Online publication date: 14-Jun-2009
https://dl.acm.org/doi/10.5555/1704555.1704714
Chen JChaudhari N(2009)Segmented-memory recurrent neural networksIEEE Transactions on Neural Networks10.1109/TNN.2009.202298020:8(1267-1280)Online publication date: 1-Aug-2009
https://dl.acm.org/doi/10.1109/TNN.2009.2022980
Wong JCho S(2009)A local experts organization model with application to face emotion recognitionExpert Systems with Applications: An International Journal10.1016/j.eswa.2007.10.03036:1(804-819)Online publication date: 1-Jan-2009
https://dl.acm.org/doi/10.1016/j.eswa.2007.10.030
Cho S(2008)Probabilistic based recursive model for adaptive processing of data structuresExpert Systems with Applications: An International Journal10.1016/j.eswa.2007.01.02134:2(1403-1422)Online publication date: 1-Feb-2008
https://dl.acm.org/doi/10.1016/j.eswa.2007.01.021
Rahman MPi Yang WChow TWu S(2007)A flexible multi-layer self-organizing map for generic processing of tree-structured dataPattern Recognition10.1016/j.patcog.2006.10.01040:5(1406-1424)Online publication date: 1-May-2007
https://dl.acm.org/doi/10.1016/j.patcog.2006.10.010
Wong JCho SHaddad H(2006)Facial emotion recognition by adaptive processing of tree structuresProceedings of the 2006 ACM symposium on Applied computing10.1145/1141277.1141282(23-30)Online publication date: 23-Apr-2006
https://dl.acm.org/doi/10.1145/1141277.1141282
Cho SWong J(2005)Probabilistic based recursive model for face recognitionProceedings of the Second international conference on Fuzzy Systems and Knowledge Discovery - Volume Part II10.1007/11540007_164(1245-1254)Online publication date: 27-Aug-2005
https://dl.acm.org/doi/10.1007/11540007_164

Abstract

Cited By

Recommendations

Efficient Learning in Adaptive Processing of Data Structures

An adaptive backpropagation algorithm for long-term electricity load forecasting

Adaptive Processing of Data Structures

Comments

Information

Published In

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Share

Share this Publication link

Share on social media

Affiliations