[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3292500.3330900acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

QuesNet: A Unified Representation for Heterogeneous Test Questions

Published: 25 July 2019 Publication History

Abstract

Understanding learning materials (e.g. test questions) is a crucial issue in online learning systems, which can promote many applications in education domain. Unfortunately, many supervised approaches suffer from the problem of scarce human labeled data, whereas abundant unlabeled resources are highly underutilized. To alleviate this problem, an effective solution is to use pre-trained representations for question understanding. However, existing pre-training methods in NLP area are infeasible to learn test question representations due to several domain-specific characteristics in education. First, questions usually comprise of heterogeneous data including content text, images and side information. Second, there exists both basic linguistic information as well as domain logic and knowledge. To this end, in this paper, we propose a novel pre-training method, namely QuesNet, for comprehensively learning question representations. Specifically, we first design a unified framework to aggregate question information with its heterogeneous inputs into a comprehensive vector. Then we propose a two-level hierarchical pre-training algorithm to learn better understanding of test questions in an unsupervised way. Here, a novel holed language model objective is developed to extract low-level linguistic features, and a domain-oriented objective is proposed to learn high-level logic and knowledge. Moreover, we show that QuesNet has good capability of being fine-tuned in many question-based tasks. We conduct extensive experiments on large-scale real-world question data, where the experimental results clearly demonstrate the effectiveness of QuesNet for question understanding as well as its superior applicability.

References

[1]
Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2014. Engaging with massive online courses. In Proceedings of the 23rd international conference on World wide web. ACM, 687--698.
[2]
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450 (2016).
[3]
C Boopathiraj and K Chellamani. 2013. Analysis of test items on difficulty level and discrimination index in the test for research in education. International journal of social science & interdisciplinary research, Vol. 2, 2 (2013), 189--193.
[4]
Michel C Desmarais, Behzad Beheshti, and Rhouma Naceur. 2012. Item to skills mapping: deriving a conjunctive q-matrix from data. In International Conference on Intelligent Tutoring Systems. Springer, 454--463.
[5]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
[6]
David E Douglas and Glen Van Der Vyver. 2004. Effectiveness of e-learning course materials for learning database management systems: An experimental investigation. Journal of Computer Information Systems, Vol. 44, 4 (2004), 41--48.
[7]
Huizhong Duan, Yunbo Cao, Chin-Yew Lin, and Yong Yu. 2008. Searching questions by identifying question topic and question focus. Proceedings of ACL-08: HLT (2008), 156--164.
[8]
Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics. 249--256.
[9]
Arthur C Graesser, Zhiqiang Cai, Max M Louwerse, and Frances Daniel. 2006. Question Understanding Aid (QUAID) a web facility that tests question comprehensibility. Public Opinion Quarterly, Vol. 70, 1 (2006), 3--22.
[10]
Ulf Hermjakob. 2001. Parsing and question classification for question answering. In Proceedings of the workshop on Open-domain question answering-Volume 12. Association for Computational Linguistics, 1--6.
[11]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.
[12]
Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 328--339.
[13]
Zhenya Huang, Qi Liu, Enhong Chen, Hongke Zhao, Mingyong Gao, Si Wei, Yu Su, and Guoping Hu. 2017. Question Difficulty Prediction for READING Problems in Standard Tests. In AAAI . 1352--1359.
[14]
Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015).
[15]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[16]
George D Kuh, Jillian Kinzie, Jennifer A Buckley, Brian K Bridges, and John C Hayek. 2011. Piecing together the student success puzzle: research, propositions, and recommendations: ASHE Higher Education Report . Vol. 116. John Wiley & Sons.
[17]
Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via bi-directional lstm-cnns-crf. arXiv preprint arXiv:1603.01354 (2016).
[18]
Md Anwar Hossain Masud and Xiaodi Huang. 2012. An e-learning system architecture based on cloud computing. system, Vol. 10, 11 (2012), 255--259.
[19]
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).
[20]
Michael G Moore and Greg Kearsley. 2011. Distance education: A systems view of online learning .Cengage Learning.
[21]
Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew Y Ng. 2011. Multimodal deep learning. In Proceedings of the 28th international conference on machine learning (ICML-11). 689--696.
[22]
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) . 1532--1543.
[23]
Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018).
[24]
Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J Guibas, and Jascha Sohl-Dickstein. 2015. Deep knowledge tracing. In Advances in Neural Information Processing Systems . 505--513.
[25]
Norbert Ed Schwarz and Seymour Ed Sudman. 1996. Answering questions: Methodology for determining cognitive and communicative processes in survey research. Jossey-Bass.
[26]
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, Vol. 15, 1 (2014), 1929--1958.
[27]
Yu Su, Qingwen Liu, Qi Liu, Zhenya Huang, Yu Yin, Enhong Chen, Chris Ding, Si Wei, and Guoping Hu. 2018. Exercise-Enhanced Sequential Modeling for Student Performance Prediction. (2018).
[28]
Martin Sundermeyer, Ralf Schlüter, and Hermann Ney. 2012. LSTM neural networks for language modeling. In Thirteenth annual conference of the international speech communication association .
[29]
Ming Tan, Cicero dos Santos, Bing Xiang, and Bowen Zhou. 2015. LSTM-based deep learning models for non-factoid answer selection. arXiv preprint arXiv:1511.04108 (2015).
[30]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems . 5998--6008.
[31]
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 1480--1489.
[32]
Yu Yin, Zhenya Huang, Enhong Chen, Qi Liu, Fuzheng Zhang, Xing Xie, and Guoping Hu. 2018. Transcribing Content from Structural Images with Spotlight Mechanism. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2643--2652.
[33]
Dell Zhang and Wee Sun Lee. 2003. Question classification using support vector machines. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval. ACM, 26--32.
[34]
Jiani Zhang, Xingjian Shi, Irwin King, and Dit-Yan Yeung. 2017. Dynamic key-value memory networks for knowledge tracing. In Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 765--774.
[35]
Liang Zhang, Keli Xiao, Hengshu Zhu, Chuanren Liu, Jingyuan Yang, and Bo Jin. 2018. CADEN: A Context-Aware Deep Embedding Network for Financial Opinions Mining. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 757--766.

Cited By

View all
  • (2024)Learning Relation-Enhanced Hierarchical Solver for Math Word ProblemsIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.327211435:10(13830-13844)Online publication date: Oct-2024
  • (2024)A Survey of Knowledge Tracing: Models, Variants, and ApplicationsIEEE Transactions on Learning Technologies10.1109/TLT.2024.338332517(1898-1919)Online publication date: 1-Jan-2024
  • (2024)EBERT: A lightweight expression-enhanced large-scale pre-trained language model for mathematics educationKnowledge-Based Systems10.1016/j.knosys.2024.112118300(112118)Online publication date: Sep-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
July 2019
3305 pages
ISBN:9781450362016
DOI:10.1145/3292500
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. heterogeneous data
  2. pre-training
  3. question representation

Qualifiers

  • Research-article

Funding Sources

  • National Key Research and Development Program of China
  • Young Elite Scientist Sponsorship Program of CAST and the Youth Innovation Promotion Association of CAS
  • National Natural Science Foundation of China

Conference

KDD '19
Sponsor:

Acceptance Rates

KDD '19 Paper Acceptance Rate 110 of 1,200 submissions, 9%;
Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)41
  • Downloads (Last 6 weeks)4
Reflects downloads up to 11 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Learning Relation-Enhanced Hierarchical Solver for Math Word ProblemsIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.327211435:10(13830-13844)Online publication date: Oct-2024
  • (2024)A Survey of Knowledge Tracing: Models, Variants, and ApplicationsIEEE Transactions on Learning Technologies10.1109/TLT.2024.338332517(1898-1919)Online publication date: 1-Jan-2024
  • (2024)EBERT: A lightweight expression-enhanced large-scale pre-trained language model for mathematics educationKnowledge-Based Systems10.1016/j.knosys.2024.112118300(112118)Online publication date: Sep-2024
  • (2024)Multi-task Information Enhancement Recommendation model for educational Self-Directed Learning SystemExpert Systems with Applications10.1016/j.eswa.2024.124073(124073)Online publication date: May-2024
  • (2024)Heterogeneous graph-based knowledge tracing with spatiotemporal evolutionExpert Systems with Applications10.1016/j.eswa.2023.122249238(122249)Online publication date: Mar-2024
  • (2024)Dynamic heterogeneous graph contrastive networks for knowledge tracingApplied Soft Computing10.1016/j.asoc.2024.112194166(112194)Online publication date: Nov-2024
  • (2024)Text mining applied to distance higher education: A systematic literature reviewEducation and Information Technologies10.1007/s10639-023-12235-029:9(10851-10878)Online publication date: 1-Jun-2024
  • (2024)DP-MFRNN: Difficulty Prediction for Examination Questions Based on Neural Network FrameworkKnowledge Science, Engineering and Management10.1007/978-981-97-5489-2_14(155-164)Online publication date: 27-Jul-2024
  • (2023)Item Difficulty Prediction Using Item Text Features: Comparison of Predictive Performance across Machine-Learning AlgorithmsMathematics10.3390/math1119410411:19(4104)Online publication date: 28-Sep-2023
  • (2023)An Efficient and Robust Semantic Hashing Framework for Similar Text SearchACM Transactions on Information Systems10.1145/357072541:4(1-31)Online publication date: 30-Jan-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media