Abstract
The research project AgNeT develops Agents for Neural Text routing in the internet. Unrestricted potentially faulty text messages arrive at a certain delivery point (e.g. email address or world wide web address). These text messages are scanned and then distributed to one of several expert agents according to a certain task criterium. Possible specific scenarios within this framework include the learning of the routing of publication titles or news titles. In this paper we describe extensive experiments for semantic text routing based on classified library titles and newswire titles. This task is challenging since incoming messages may contain constructions which have not been anticipated. Therefore, the contributions of this research are in learning and generalizing neural architectures for the robust interpretation of potentially noisy unrestricted messages. Neural networks were developed and examined for this topic since they support robustness and learning in noisy unrestricted real-world texts. We describe and compare different sets of experiments. The first set of experiments tests a recurrent neural network for the task of library title classification. Then we describe a larger more difficult newswire classification task from information retrieval. The comparison of the examined models demonstrates that techniques from information retrieval integrated into recurrent plausibility networks performed well even under noise and for different corpora.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Belew RK (1989) Adaptive information retrieval. In: Proceedings of the 12th Annual International Conference on Research and Development in Information Retrieval-SIGIR 89, pp. 11-20.
Bordogna G and Pasi G (1996) A user adaptive neural network supporting rule based relevance feedback. Fuzzy Sets and Systems, 82:201-211.
Briscoe T (1997) Co-evolution of language and of the language acquisition device. In: Proceedings of the Meeting of the Association for Computational Linguistics.
Chan SC, Choo CL and Wu JK (1994) Retrieval of images using fuzzy interactive activation neural networks. In: Werbos P, Szu H and Widrow B, Ed., Proceedings of theWorld Congress on Neural Networks, San Diego, CA. Lawrence Erlbaum Associates, Hillsdale, NJ. INNS, Vol. 1, pp. 723-731.
Chen H (1995) Machine learning for information retrieval: Neural networks, symbolic learning and genetic algorithms. Journal of the American Society of Information Sciences, 46(3):124-216.
Cherkassky V and Vassilas N (1988) Performance of backpropagation networks for associative database retrieval. In: Proceedings of the International Conference on Neural Networks.
Crestani F (1993) An adaptive information retrieval system based on neural networks. Lecture Notes in Computer Science, Springer, Vol. 686.
Cunningham H, Wilks Y and Gaizauskas R (1996) New methods, current trends and software infrastructure for NLP. In: Proceedings of the NEMLAP-2, Ankara.
Elman JL (1990) Finding structure in time. Cognitive Science, 14:179-211.
Elman JL, Bates EA, Johnson MH, Karmiloff-Smith A, Parisi D and Plunkett K (1996) Rethinking Innateness. MIT Press, Cambridge, MA.
GershoMand Reiter R (1990) Information retrieval using self-organizing and heteroassociative supervised neural networks. In: Proceedings of the International Neural Network Conference, pp. 361-364.
Joachims T (1998) Text categorization with support vector machines: Learning with many relevant features. In Proceedings of the European Conference on Machine Learning, Chemnitz, Germany.
Jordan MI (1986) Attractor dynamics and parallelism in a connectionist sequential machine. In: Proceedings of the Eighth Conference of the Cognitive Science Society, Amherst, MA, pp. 531-546.
Kwok KL (1990) Application of neural networks to information retrieval. In: Caudill M, Ed., Proceedings of the International Joint Conference on Neural Networks,Washington, D.C. Lawrence Erlbaum Associates, Inc., Hilsdale, NJ., Vol. II, pp. 623-626.
Lange T and Wharton C (1992) REMIND: Retrieval from episodic memory by inferencing and disambiguation. In: Barnden J and Holyoak K, Eds., Advances in Connectionist and Neural Computation Theory, Vol. 2. Ablex, Norwood, New Jersey.
Layaida R, BoughanemMand Caron A (1994) Constructing an information retrieval system with neural networks. Lecture Notes in Computer Science, Springer, Vol. 856.
Lelu A and Francois C (1992) Hypertext paradigm in the field of information retrieval: A neural approach. In: Proceedings of the Fourth ACM Conference on Hypertext, Information Retrieval, pp. 112-121.
Lewis DD (1991) Representation and learning in information retrieval. Technical Report UM-CS-1991-093, University of Massachusetts, Amherst, Computer Science.
Lewis DD (1997) Reuters-21578 text categorization test collection. http://www.research.att.com/»lewis.
Medsker LR (1995) Hybrid Intelligent Systems. Kluwer Academic Publishers, Boston.
Merkl D (1995) A connectionist view on document classification. In: Proceedings of the 6th Australian Database Conference.
Niki K (1997) Self-organizing information retrieval system on the web: SirWeb. In: Kasabov N, Kozma R, Ko K, O'Shea R, Coghill G and Gedeon T, Eds., Progress in Connectionsist-Based Information Systems. Proceedings of the 1997 International Conference on Neural Information Processing and Intelligent Information Systems, Springer, Singapore, Vol. 2, pp. 881-884.
Nishimori H, Nakamura T and Shiino M (1990) Retrieval of spatio-temporal sequence in asynchronous neural network. Physical Review A, 41:3346-3354.
Papka R, Callan JP and Barto AG (1997) Text-based information retrieval using exponentiated gradient descent. In: Mozer MC, Jordan MI and Petsche T, Eds., Advances in Neural Information Processing Systems, The MIT Press, Vol. 9, p. 3.
Reilly RG and Sharkey NE (1992) Connectionist Approaches to Natural Language Processing. Lawrence Erlbaum Associates, Hillsdale, NJ.
Rijsbergen CJV (1979) Information Retrieval. Butterworths, London.
Salton G (1989) Automatic Text Processing. Addison-Wesley, New York.
Scholtes JC (1993). Neural networks in natural language processing and information retrieval. Ph.D. Thesis, Universiteit van Amsterdam, Amsterdam, Netherlands.
Sparck-Jones K (1986) Synonymy and Semantic Classification. Edinburgh University Press, Edinburgh.
TREC (1996) In: Proceedings of the text retrieval conference 5, Gaithersburg, Maryland.
TREC (1997) In: Proceedings of the text retrieval conference 6, Gaithersburg, Maryland.
Wermter S (1995) Hybrid Connectionist Natural Language Processing. Chapman and Hall, Thomson International, London, UK.
Wermter S (1999) Preference Moore machines for neural fuzzy integration. In: Proceedings of the International Joint Conference on Artificial Intelligence, Stockholm, pp. 840-845.
Wermter S, Arevian G and Panchev C (1999a) Recurrent neural network learning for text routing. In: Proceedings of the International Conference on Artificial Neural Networks, Edinburgh, UK, pp. 898-903.
Wermter S, Panchev C and Arevian G(1999b) Hybrid neural plausibility networks for news agents. In: Proceedings of the National Conference on Artificial Intelligence, Orlando, USA, pp. 93-98.
Wermter S, Riloff E and Scheler G (1996) Connectionist, Statistical and Symbolic Approaches to Learning for Natural Language Processing. Springer, Berlin.
Wermter S and Sun R (2000) Hybrid neural systems. Springer, Heidelberg.
Wermter S and Weber V (1997) SCREEN: Learning a flat syntactic and semantic spoken language analysis using artificial neural networks. Journal of Artificial Intelligence Research, 6(1):35-85.
Wettler M and Ratt R (1989) A connectionist system to simulate lexical decisions in information retrieval. In: Pfeifer R, Schreter Z, Fogelman F, and Steels L, Eds., Connectionism in Perspective, North-Holland, Amsterdam, Netherlands, pp. 463-469.
Wilkinson R and Hingston P (1991) Using the cosine measure in a neural network for document retrieval. In: Proceedings of the Fourteenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Modeling Information Retrieval Systems II, pp. 202-210.
Zavrel J (1995) Neural information retrieval-an experimental study of clustering and browsing of document collections with neural networks. Master's Thesis, University of Amsterdam, Amsterdam, Netherlands.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Wermter, S. Neural Network Agents for Learning Semantic Text Classification. Information Retrieval 3, 87–103 (2000). https://doi.org/10.1023/A:1009942513170
Issue Date:
DOI: https://doi.org/10.1023/A:1009942513170