research-article

Mining Web Pages for Data Records

Authors:

Bing Liu,

Robert Grossman,

Yanhong ZhaiAuthors Info & Claims

IEEE Intelligent Systems, Volume 19, Issue 6

Pages 49 - 55

https://doi.org/10.1109/MIS.2004.68

Published: 01 November 2004 Publication History

Publisher Site

Abstract

Much information on the Web is contained in regularly structured objects, or data records. Data records often present their host pages' essential information, such as lists of products and services. Mining data records to extract this information can help you provide value-added services. Existing approaches to data extraction on the Web include supervised learning and automatic techniques. Supervised learning requires substantial human effort, and current automatic techniques provide poor results. To solve this problem, the MDR (mining data records) system exploits two key observations about the layout of data records in Web pages and employs a string-matching algorithm. Experiments show that this new automatic technique significantly outperforms existing methods. In addition, it mines both contiguous and noncontiguous data records.

References

[1]

R. Baeza-Yates, "Algorithms for String Matching: A Survey," ACM SIGIR Forum, vol. 23, nos. 3–4, 1989, pp. 34–58.

Digital Library

Google Scholar

[2]

D. Embley, Y. Jiang, and Y. Ng, "Record-Boundary Discovery in Web Documents," Proc. ACM Int'l Conf. Management of Data (SIGMOD 99), ACM Press, 1999, pp. 467–478.

Digital Library

Google Scholar

[3]

D. Buttler, L. Liu, and C. Pu, "A Fully Automated Extraction System for the World Wide Web," Proc. 21st Int'l Conf. Distributed Computing Systems (ICDCS 01), IEEE CS Press, 2001, pp. 361–370.

Digital Library

Google Scholar

[4]

C.-H. Chang and S.-L. Lui, "IEPAD: Information Extraction Based on Pattern Discovery," Proc. 10th Int'l Conf. World Wide Web (WWW 01), ACM Press, 2001, pp. 681–688.

Digital Library

Google Scholar

[5]

K. Lerman, C. Knoblock, and S. Minton, "Automatic Data Extraction from Lists and Tables in Web Sources," Proc. IJCAI 2001 Workshop Adaptive Text Extraction and Mining, 2001;

Google Scholar

[6]

V. Crescenzi, G. Mecca, and P. Merialdo, "RoadRunner: Towards Automatic Data Extraction from Large Web Sites," Proc. 27th Int'l Conf. Very Large Data Bases (VLDB 01), Morgan Kaufmann, 2001, pp. 109–118.

Digital Library

Google Scholar

[7]

C. Knoblock and A. Levy, eds.Proc. 1998 Workshop AI and Information Integration, AAAI Press, 1998;

Google Scholar

[8]

N. Kushmerick, D. Weld, and R. Doorenbos, "Wrapper Induction for Information Extraction," Proc. 15th Int'l Joint Conf. Artificial Intelligence (IJCAI 97), Morgan Kaufmann, 1997, pp. 729–735.

Google Scholar

[9]

C.-N. Hsu and M.-T. Dung, "Generating Finite-State Transducers for Semi-structured Data Extraction from the Web," Information Systems, vol. 23, no. 8, 1998, pp. 521–538.

Digital Library

Google Scholar

[10]

I. Muslea, S. Minton, and C. Knoblock, "A Hierarchical Approach to Wrapper Induction," Proc. 3rd Int'l Conf. Autonomous Agents (Agents 99), ACM Press, 1999, pp. 190–197.

Digital Library

Google Scholar

[11]

W. Cohen, M. Hurst, and L. Jensen, "A Flexible Learning System for Wrapping Tables and Lists in HTML Documents," Proc. 11th Int'l Conf. World Wide Web (WWW 02), ACM Press, 2002, pp. 232–241.

Digital Library

Google Scholar

Cited By

View all

Manna MOro ERuffolo MAlviano MLeone N(2012)The HiLeX system for semantic information extractionTransactions on Large-Scale Data- and Knowledge-Centered Systems V10.5555/2184170.2184175(91-125)Online publication date: 1-Jan-2012
https://dl.acm.org/doi/10.5555/2184170.2184175
Yan XJeschke SDubey AWilke MSchütze H(2009)RENS --- Enabling a Robot to Identify a PersonProceedings of the 2nd International Conference on Intelligent Robotics and Applications10.1007/978-3-642-10817-4_5(43-54)Online publication date: 16-Dec-2009
https://dl.acm.org/doi/10.1007/978-3-642-10817-4_5
Choi CKang JChoi J(2007)Extraction of user-defined data blocks using the regularity of dynamic web pagesProceedings of the intelligent computing 3rd international conference on Advanced intelligent computing theories and applications10.5555/1777454.1777470(123-133)Online publication date: 21-Aug-2007
https://dl.acm.org/doi/10.5555/1777454.1777470
Show More Cited By

Mining Web Pages for Data Records
1. Information systems
  1. Information systems applications

Recommendations

Mining data records in Web pages
KDD '03: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining

A large amount of information on the Web is contained in regularly structured objects, which we call data records. Such data records are important because they often present the essential information of their host pages, e.g., lists of products or ...
Web Data Mining System Based on Web Services
HIS '09: Proceedings of the 2009 Ninth International Conference on Hybrid Intelligent Systems - Volume 03

Web data mining is a key technology to find meaningful patterns and rules from the WWW online mass data. Because of the complexity of Web data, currently a lot of manufacturers are committing to develop specialized software for Web Mining and ...
Web data mining: exploring hyperlinks, contents, and usage data

This paper presents a review of the book "Web Data Mining - Exploring Hyperlinks, Contents, and Usage Data" by Bing Liu. The review concludes that the breadth and depth of this book makes it a required staple for every Web mining researcher, student, or ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

IEEE Intelligent Systems Volume 19, Issue 6

November 2004

93 pages

ISSN:1541-1672

Issue’s Table of Contents

Publisher

IEEE Educational Activities Department

United States

Publication History

Published: 01 November 2004

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Manna MOro ERuffolo MAlviano MLeone N(2012)The HiLeX system for semantic information extractionTransactions on Large-Scale Data- and Knowledge-Centered Systems V10.5555/2184170.2184175(91-125)Online publication date: 1-Jan-2012
https://dl.acm.org/doi/10.5555/2184170.2184175
Yan XJeschke SDubey AWilke MSchütze H(2009)RENS --- Enabling a Robot to Identify a PersonProceedings of the 2nd International Conference on Intelligent Robotics and Applications10.1007/978-3-642-10817-4_5(43-54)Online publication date: 16-Dec-2009
https://dl.acm.org/doi/10.1007/978-3-642-10817-4_5
Choi CKang JChoi J(2007)Extraction of user-defined data blocks using the regularity of dynamic web pagesProceedings of the intelligent computing 3rd international conference on Advanced intelligent computing theories and applications10.5555/1777454.1777470(123-133)Online publication date: 21-Aug-2007
https://dl.acm.org/doi/10.5555/1777454.1777470
Ling YMeng XMeng W(2006)Automated extraction of hit numbers from search result pagesProceedings of the 7th international conference on Advances in Web-Age Information Management10.1007/11775300_7(73-84)Online publication date: 17-Jun-2006
https://dl.acm.org/doi/10.1007/11775300_7

Abstract

References

Cited By

Recommendations

Mining data records in Web pages

Web Data Mining System Based on Web Services

Web data mining: exploring hyperlinks, contents, and usage data

Comments

Information

Published In

Publisher

Publication History

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations