More Web Proxy on the site http://driver.im/

research-article

Differentiable learning of rules with constants in knowledge graph

Authors:

Wen ZhangAuthors Info & Claims

Volume 275, Issue C

https://doi.org/10.1016/j.knosys.2023.110686

Published: 05 September 2023 Publication History

Abstract

Knowledge reasoning helps overcome the incompleteness of knowledge graphs (KGs) and has significantly contributed to the development of large KGs. Rule mining, one of the key tasks of knowledge reasoning, studies the problem of learning interpretable inference patterns over KGs. Existing rule mining methods mainly focus on learning rules that consist of different relations and variables, restricting the form of rules to be closed path. While rules could be diverse and in order to enrich the forms of rules, we argue that constants should also be considered in the rule mining process. We propose an Elegant Differentiable rUle learning with Constant mEthod (EduCe). Source code of EduCeis available at https://github.com/yep96/Educe. We propose a constant operator and dynamic weight mechanism, which choose the constants that should be added and decrease the number of parameters, respectively. The model could mine diverse and accurate rules in an efficient way with these modules. The experimental results on several knowledge graph completion benchmarks show that EduCe achieves state-of-the-art link prediction results among differentiable rule mining methods and successfully learns diverse and high-quality rules.

Highlights

•

Draw attention to expanding the diversity of target rules for rule mining and emphasize the importance of constants to rules.

•

Propose an end-to-end differentiable rule mining method that can mine rules in an efficient way.

•

Experimentally demonstrate the importance of rule with constants.

•

Design an algorithm that outputs high-quality symbolic rules with constants.

References

[1]

Miller G.A., WordNet: A lexical database for English, Commun. ACM 38 (11) (1995) 39–41,.

Digital Library

[2]

Wang M., Wang H., Qi G., Zheng Q., Richpedia: A large-scale, comprehensive multi-modal knowledge graph, 22, 2020, p. 100159,.

[3]

Bollacker K.D., Evans C., Paritosh P.K., Sturge T., Taylor J., Freebase: A collaboratively created graph database for structuring human knowledge, in: Wang J.T. (Ed.), Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2008, Vancouver, BC, Canada, June 10-12, 2008, ACM, 2008, pp. 1247–1250,.

Digital Library

[4]

Wang Q., Mao Z., Wang B., Guo L., Knowledge graph embedding: A survey of approaches and applications, IEEE Trans. Knowl. Data Eng. 29 (12) (2017) 2724–2743,.

[5]

Bordes A., Usunier N., García-Durán A., Weston J., Yakhnenko O., Translating embeddings for modeling multi-relational data, in: Burges C.J.C., Bottou L., Ghahramani Z., Weinberger K.Q. (Eds.), Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a Meeting Held December 5-8, 2013, Lake Tahoe, Nevada, United States, 2013, pp. 2787–2795.

[6]

Lin Y., Liu Z., Sun M., Liu Y., Zhu X., Learning entity and relation embeddings for knowledge graph completion, in: Bonet B., Koenig S. (Eds.), Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25-30, 2015, Austin, Texas, USA, AAAI Press, 2015, pp. 2181–2187.

[7]

Yang B., Yih W., He X., Gao J., Deng L., Embedding entities and relations for learning and inference in knowledge bases, in: Bengio Y., LeCun Y. (Eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.

[8]

Trouillon T., Welbl J., Riedel S., Gaussier É., Bouchard G., Complex embeddings for simple link prediction, in: Balcan M., Weinberger K.Q. (Eds.), Proceedings of the 33nd International Conference on Machine Learning, in: JMLR Workshop and Conference Proceedings, vol. 48, ICML 2016, New York City, NY, USA, June 19-24, 2016, JMLR.org, 2016, pp. 2071–2080.

[9]

Dettmers T., Minervini P., Stenetorp P., Riedel S., Convolutional 2D knowledge graph embeddings, in: McIlraith S.A., Weinberger K.Q. (Eds.), Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th Innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI-18, New Orleans, Louisiana, USA, February 2-7, 2018, AAAI Press, 2018, pp. 1811–1818.

[10]

Sun Z., Deng Z., Nie J., Tang J., RotatE: Knowledge graph embedding by relational rotation in complex space, in: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, la, USA, May 6-9, 2019, OpenReview.net, 2019.

[11]

Li Z., Liu X., Wang X., Liu P., Shen Y., TransO: a knowledge-driven representation learning method with ontology information constraints, 26, 2023, pp. 297–319,.

Digital Library

[12]

Schlichtkrull M.S., Kipf T.N., Bloem P., van den Berg R., Titov I., Welling M., Modeling relational data with graph convolutional networks, in: Gangemi A., Navigli R., Vidal M., Hitzler P., Troncy R., Hollink L., Tordai A., Alam M. (Eds.), The Semantic Web - 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3-7, 2018, Proceedings, in: Lecture Notes in Computer Science, vol. 10843, Springer, 2018, pp. 593–607,.

Digital Library

[13]

Vashishth S., Sanyal S., Nitin V., Talukdar P.P., Composition-based multi-relational graph convolutional networks, in: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, OpenReview.net, 2020.

[14]

Nathani D., Chauhan J., Sharma C., Kaul M., Learning attention-based embeddings for relation prediction in knowledge graphs, in: Korhonen A., Traum D.R., Màrquez L. (Eds.), Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, Association for Computational Linguistics, 2019, pp. 4710–4723,.

[15]

Bansal T., Juan D., Ravi S., McCallum A., A2N: attending to neighbors for knowledge graph inference, in: Korhonen A., Traum D.R., Màrquez L. (Eds.), Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, Association for Computational Linguistics, 2019, pp. 4387–4392,.

[16]

Meilicke C., Chekol M.W., Ruffinelli D., Stuckenschmidt H., Anytime bottom-up rule learning for knowledge graph completion, in: Kraus S. (Ed.), Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, ijcai.org, 2019, pp. 3137–3143,.

[17]

Ortona S., Meduri V.V., Papotti P., Robust discovery of positive and negative rules in knowledge bases, in: 34th IEEE International Conference on Data Engineering, ICDE 2018, Paris, France, April 16-19, 2018, IEEE Computer Society, 2018, pp. 1168–1179,.

[18]

Chen Y., Goldberg S., Wang D.Z., Johri S.S., Ontological pathfinding, in: Özcan F., Koutrika G., Madden S. (Eds.), Proceedings of the 2016 International Conference on Management of Data, SIGMOD Conference 2016, San Francisco, CA, USA, June 26 - July 01, 2016, ACM, 2016, pp. 835–846,.

Digital Library

[19]

Galárraga L.A., Teflioudi C., Hose K., Suchanek F.M., AMIE: Association rule mining under incomplete evidence in ontological knowledge bases, in: Schwabe D., Almeida V.A.F., Glaser H., Baeza-Yates R., Moon S.B. (Eds.), 22nd International World Wide Web Conference, WWW ’13, Rio de Janeiro, Brazil, May 13-17, 2013, International World Wide Web Conferences Steering Committee / ACM, 2013, pp. 413–422,.

Digital Library

[20]

Cohen W.W., TensorLog: A differentiable deductive database, 2016, CoRR abs/1605.06523, arXiv:1605.06523.

[21]

Yang F., Yang Z., Cohen W.W., Differentiable learning of logical rules for knowledge base reasoning, in: Guyon I., von Luxburg U., Bengio S., Wallach H.M., Fergus R., Vishwanathan S.V.N., Garnett R. (Eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, 2017, pp. 2319–2328.

[22]

Sadeghian A., Armandpour M., Ding P., Wang D.Z., DRUM: end-to-end differentiable rule mining on knowledge graphs, in: Wallach H.M., Larochelle H., Beygelzimer A., d’Alché-Buc F., Fox E.B., Garnett R. (Eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, 2019, pp. 15321–15331.

[23]

Koller D., Friedman N., Džeroski S., Sutton C., McCallum A., Pfeffer A., Abbeel P., Wong M.-F., Meek C., Neville J., et al., Introduction to Statistical Relational Learning, MIT Press, 2007.

[24]

Muggleton S.H., Inductive logic programming: Inverse resolution and beyond, in: Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, IJCAI 95, Montréal Québec, Canada, August 20-25 1995, 2 Volumes, Morgan Kaufmann, 1995, p. 997.

[25]

Galárraga L., Teflioudi C., Hose K., Suchanek F.M., Fast rule mining in ontological knowledge bases with AMIE+, VLDB J. 24 (6) (2015) 707–730,.

Digital Library

[26]

Liu M., Yang Z., Guo Y., Jiang J., Yang K., MICAR: Nonlinear association rule mining based on maximal information coefficient, Knowl. Inf. Syst. 64 (11) (2022) 3017–3042.

[27]

Sultana T., Lee Y.-K., Efficient rule mining and compression for RDF style KB based on horn rules, J. Supercomput. 78 (14) (2022) 16553–16580.

[28]

Wang P., Stepanova D., Domokos C., Kolter J.Z., Differentiable learning of numerical rules in knowledge graphs, in: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, OpenReview.net, 2020.

[29]

Xu Z., Ye P., Chen H., Zhao M., Chen H., Zhang W., Ruleformer: Context-aware rule mining over knowledge graph, in: Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, October 12-17, 2022, International Committee on Computational Linguistics, 2022, pp. 2551–2560.

[30]

Yang Y., Song L., Learn to explain efficiently via neural logic inductive learning, in: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, OpenReview.net, 2020.

[31]

Ji G., He S., Xu L., Liu K., Zhao J., Knowledge graph embedding via dynamic mapping matrix, in: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers, The Association for Computer Linguistics, 2015, pp. 687–696,.

[32]

Wang Z., Zhang J., Feng J., Chen Z., Knowledge graph embedding by translating on hyperplanes, in: Brodley C.E., Stone P. (Eds.), Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, July 27 -31, 2014, QuÉBec City, QuÉBec, Canada, AAAI Press, 2014, pp. 1112–1119.

[33]

Guo S., Wang Q., Wang L., Wang B., Guo L., Jointly embedding knowledge graphs and logical rules, in: Su J., Carreras X., Duh K. (Eds.), Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, The Association for Computational Linguistics, 2016, pp. 192–202,.

[34]

Guo S., Wang Q., Wang L., Wang B., Guo L., Knowledge graph embedding with iterative guidance from soft rules, in: McIlraith S.A., Weinberger K.Q. (Eds.), Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th Innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI-18, New Orleans, Louisiana, USA, February 2-7, 2018, AAAI Press, 2018, pp. 4816–4823.

[35]

Zhang W., Paudel B., Wang L., Chen J., Zhu H., Zhang W., Bernstein A., Chen H., Iteratively learning embeddings and rules for knowledge graph reasoning, in: Liu L., White R.W., Mantrach A., Silvestri F., McAuley J.J., Baeza-Yates R., Zia L. (Eds.), The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019, ACM, 2019, pp. 2366–2377,.

Digital Library

[36]

Kok S., Domingos P.M., Statistical predicate invention, in: Ghahramani Z. (Ed.), Machine Learning, Proceedings of the Twenty-Fourth International Conference, in: ACM International Conference Proceeding Series, vol. 227, ICML 2007, Corvallis, Oregon, USA, June 20-24, 2007, ACM, 2007, pp. 433–440,.

Digital Library

[37]

Toutanova K., Chen D., Pantel P., Poon H., Choudhury P., Gamon M., Representing text for joint embedding of text and knowledge bases, in: Màrquez L., Callison-Burch C., Su J., Pighin D., Marton Y. (Eds.), Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015, The Association for Computational Linguistics, 2015, pp. 1499–1509,.

[38]

Balazevic I., Allen C., Hospedales T.M., TuckER: Tensor factorization for knowledge graph completion, in: Inui K., Jiang J., Ng V., Wan X. (Eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, Association for Computational Linguistics, 2019, pp. 5184–5193,.

[39]

Lao N., Cohen W.W., Relational retrieval using a combination of path-constrained random walks, Mach. Learn. 81 (1) (2010) 53–67,.

Digital Library

[40]

Shen Y., Chen J., Huang P., Guo Y., Gao J., M-walk: Learning to walk over graphs using Monte Carlo tree search, in: Bengio S., Wallach H.M., Larochelle H., Grauman K., Cesa-Bianchi N., Garnett R. (Eds.), Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, 2018, pp. 6787–6798.

[41]

Han X., Cao S., Lv X., Lin Y., Liu Z., Sun M., Li J., OpenKE: An open toolkit for knowledge embedding, in: Blanco E., Lu W. (Eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018: System Demonstrations, Brussels, Belgium, October 31 - November 4, 2018, Association for Computational Linguistics, 2018, pp. 139–144,.

Cited By

Wang LGeng ZWang XDong YLi J(2024)Open-world knowledge embedding in a low-text resource environmentApplied Intelligence10.1007/s10489-024-05744-z54:22(11564-11576)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10489-024-05744-z
Cai KWang XLuo X(2024)Optimize Rule Mining Based on Constraint Learning in Knowledge GraphKnowledge Science, Engineering and Management10.1007/978-981-97-5498-4_7(82-98)Online publication date: 16-Aug-2024
https://dl.acm.org/doi/10.1007/978-981-97-5498-4_7

Index Terms

Differentiable learning of rules with constants in knowledge graph

Index terms have been assigned to the content through auto-classification.

Recommendations

Efficient and interpretable fuzzy classifiers from data with support vector learning

The construction of fuzzy rule-based classification systems with both good generalization ability and interpretability is a chalenging issue. The paper aims to present a novel framework for the realization of these important (and many times conflicting) ...
Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning
WWW '19: The World Wide Web Conference

Reasoning is essential for the development of large knowledge graphs, especially for completion, which aims to infer new triples based on existing ones. Both rules and embeddings can be used for knowledge graph reasoning and they have their own ...
Uncertain Knowledge Graph Completion with Rule Mining
Web Information Systems and Applications
Abstract
To model the uncertainty within knowledge graphs (KGs), existing studies define uncertain knowledge graphs (UKGs), which assign a confidence score to each triple to measure its likelihood of being true and make more precisely downstream tasks such ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Knowledge-Based Systems

Knowledge-Based Systems Volume 275, Issue C

Sep 2023

723 pages

ISSN:0950-7051

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 05 September 2023

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang LGeng ZWang XDong YLi J(2024)Open-world knowledge embedding in a low-text resource environmentApplied Intelligence10.1007/s10489-024-05744-z54:22(11564-11576)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10489-024-05744-z
Cai KWang XLuo X(2024)Optimize Rule Mining Based on Constraint Learning in Knowledge GraphKnowledge Science, Engineering and Management10.1007/978-981-97-5498-4_7(82-98)Online publication date: 16-Aug-2024
https://dl.acm.org/doi/10.1007/978-981-97-5498-4_7

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents