Article

Evaluating machine learning for information extraction

Authors:

Alberto LavelliAuthors Info & Claims

ICML '05: Proceedings of the 22nd international conference on Machine learning

Pages 345 - 352

https://doi.org/10.1145/1102351.1102395

Published: 07 August 2005 Publication History

Get Access

Abstract

Comparative evaluation of Machine Learning (ML) systems used for Information Extraction (IE) has suffered from various inconsistencies in experimental procedures. This paper reports on the results of the Pascal Challenge on Evaluating Machine Learning for Information Extraction, which provides a standardised corpus, set of tasks, and evaluation methodology. The challenge is described and the systems submitted by the ten participants are briefly introduced and their performance is analysed.

References

[1]

Califf, M. E., (1998). Relational Learning Techniques for Natural Language Information Extraction. Doctoral dissertation, University of Texas at Austin.

Digital Library

Google Scholar

[2]

Daelemans, W. & Hoste, V., (2002). Evaluation of machine learning methods for natural language processing tasks. In Proceedings of the Third International Conference on Language Resources and Evaluation (LREC 2002). Las Palmas, Spain.

Google Scholar

[3]

Douthat, A., (1998). The message understanding conference scoring software user's manual. In Proceedings of the 7th Message Understanding Conference (MUC-7).

Google Scholar

[4]

Freitag, D., 1998. Machine Learning for Information Extraction in Informal Domains. Doctoral dissertation, Carnegie Mellon University.

Digital Library

Google Scholar

[5]

Hirschman, L., (1998). The evolution of evaluation: Lessons from the Message Understanding Conferences. Computer Speech and Language, 12, 281--305.

Crossref

Google Scholar

[6]

Lavelli, A. Califf, M. E., Ciravegna, F., Freitag, D., Giuliano, C., Kushmerick, N., Romano (2004) IE Evaluation: Criticisms and recommendations. In Proceedings of the AAAI-04 workshop on Adaptive Text Extraction and Mining (ATEM 2004), San Jose, California

Google Scholar

Cited By

View all

Jiménez PCorchuelo R(2022)On validating web information extraction proposalsExpert Systems with Applications10.1016/j.eswa.2022.116700199(116700)Online publication date: Aug-2022
https://doi.org/10.1016/j.eswa.2022.116700
Malo SThiombiano JTraore Y(2021)Extraction of events and qualitative data from texts for the monitoring of infectious diseases: case of meningitis and COVID-192021 16th Iberian Conference on Information Systems and Technologies (CISTI)10.23919/CISTI52073.2021.9476498(1-4)Online publication date: 23-Jun-2021
https://doi.org/10.23919/CISTI52073.2021.9476498
Issertial LTsuji H(2020)Information Extraction for Call for PaperNatural Language Processing10.4018/978-1-7998-0951-7.ch020(394-409)Online publication date: 2020
https://doi.org/10.4018/978-1-7998-0951-7.ch020
Show More Cited By

Evaluating machine learning for information extraction
1. Computing methodologies

Recommendations

Lifelong Machine Learning
Machine learning for information retrieval
SIGIR '11: Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval

In recent years, we have witnessed successful application of machine learning techniques to a wide range of information retrieval problems, including Web search engines, recommendation systems, online advertising, etc. It is thus critical for ...
Machine Learning: The State of the Art

The two fundamental problems in machine learning (ML) are statistical analysis and algorithm design. The former tells us the principles of the mathematical models that we establish from the observation data. The latter defines the conditions on which ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ICML '05: Proceedings of the 22nd international conference on Machine learning

August 2005

1113 pages

ISBN:1595931805

DOI:10.1145/1102351

General Chair:
Saso Dzeroski
Jozef Stefan Institute, Slovenia
,
Program Chairs:
Luc De Raedt,
Stefan Wrobel

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 August 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

32
Total Citations
View Citations
504
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)1

Reflects downloads up to 04 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Jiménez PCorchuelo R(2022)On validating web information extraction proposalsExpert Systems with Applications10.1016/j.eswa.2022.116700199(116700)Online publication date: Aug-2022
https://doi.org/10.1016/j.eswa.2022.116700
Malo SThiombiano JTraore Y(2021)Extraction of events and qualitative data from texts for the monitoring of infectious diseases: case of meningitis and COVID-192021 16th Iberian Conference on Information Systems and Technologies (CISTI)10.23919/CISTI52073.2021.9476498(1-4)Online publication date: 23-Jun-2021
https://doi.org/10.23919/CISTI52073.2021.9476498
Issertial LTsuji H(2020)Information Extraction for Call for PaperNatural Language Processing10.4018/978-1-7998-0951-7.ch020(394-409)Online publication date: 2020
https://doi.org/10.4018/978-1-7998-0951-7.ch020
Qiu QXie ZWu LTao L(2020)Automatic spatiotemporal and semantic information extraction from unstructured geoscience reports using text mining techniquesEarth Science Informatics10.1007/s12145-020-00527-9Online publication date: 19-Sep-2020
https://doi.org/10.1007/s12145-020-00527-9
Liu SWang XCollins CDou WOuyang FEl-Assady MJiang LKeim D(2019)Bridging Text Visualization and Mining: A Task-Driven SurveyIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2018.283434125:7(2482-2504)Online publication date: 1-Jul-2019
https://doi.org/10.1109/TVCG.2018.2834341
Issertial LTsuji H(2018)Information Extraction for Call for PaperInternational Journal of Knowledge and Systems Science10.4018/IJKSS.20151001036:4(35-49)Online publication date: 16-Dec-2018
https://dl.acm.org/doi/10.4018/IJKSS.2015100103
Sharma AThung FKochhar PSulistya ALo DMendes ECounsell SPetersen K(2017)Cataloging GitHub RepositoriesProceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering10.1145/3084226.3084287(314-319)Online publication date: 15-Jun-2017
https://dl.acm.org/doi/10.1145/3084226.3084287
Jayaram KSangeeta K(2017)A review: Information extraction techniques from research papers2017 International Conference on Innovative Mechanisms for Industry Applications (ICIMIA)10.1109/ICIMIA.2017.7975532(56-59)Online publication date: Feb-2017
https://doi.org/10.1109/ICIMIA.2017.7975532
Pelham JHockley C(2017)Analysis of Short form Maintenance Records for NFF Using NLP, Phrase Matching, and Bayesian LearningProcedia CIRP10.1016/j.procir.2016.10.12359(257-262)Online publication date: 2017
https://doi.org/10.1016/j.procir.2016.10.123
Meng FMorioka C(2015)Automating the generation of lexical patterns for processing free text in clinical documentsJournal of the American Medical Informatics Association10.1093/jamia/ocv01222:5(980-986)Online publication date: 14-May-2015
https://doi.org/10.1093/jamia/ocv012
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Lifelong Machine Learning

Machine learning for information retrieval

Machine Learning: The State of the Art