Abstract
Statistical Relational Learning (SRL) relies on statistical and probabilistic modeling to represent, learn, and reason about domains with complex relational and rich probability structures. Although SRL techniques have succeeded in many real-world applications, they follow the same assumption as most ML techniques by assuming training and testing data have the same distribution and are sampled from the same feature space. Changes between these distributions might require training a new model using new data. Transfer Learning adapts knowledge already learned to other tasks and domains to help create new models, particularly in a low-data regime setting. Many recent works have succeeded in applying Transfer Learning to relational domains. However, most focus on what and how to transfer. When to transfer is still an open research problem as a pre-trained model is not guaranteed to help or improve performance for learning a new model. Besides, testing every possible pair of source and target domains to perform transference is costly. In this work, we focus on when by proposing a method that relies on probabilistic representations of relational databases and distributions learned by models to indicate the most suitable source domain for transferring. To evaluate our approach, we analyze the performances of two transfer learning-based algorithms given the most similar target domain to a source domain according to our proposal. In the experimental results, our method has succeeded as both algorithms reach their best performance when transferring between the most similar pair of source and target domains.
Supported by CAPES, FAPERJ, and CNPq.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahmadvand, M., Tahmoresnezhad, J.: Metric transfer learning via geometric knowledge embedding. Appl. Intell. 51, 921–934 (2021)
Azevedo Santos, R., Paes, A., Zaverucha, G.: Transfer learning by mapping and revising boosted relational dependency networks. Mach. Learn. 109, 1435–1463 (2020)
Bilenko, M., Mooney, R.J.: Adaptive duplicate detection using learnable string similarity measures. In: Proceeding of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2003, pp. 39–48. Association for Computing Machinery, New York (2003)
Davis, J., Goadrich, M.: The relationship between precision-recall and roc curves. In: Proceedings of the 23rd International Conference on Machine Learning, ICML 2006, pp. 233–240. Association for Computing Machinery, New York (2006)
De Raedt, L.: Logical and relational learning. In: Zaverucha, G., da Costa, A.L. (eds.) Advances in Artificial Intelligence - SBIA 2008, pp. 1–1. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-68856-3
de Figueiredo, L.F., Paes, A., Zaverucha, G.: Transfer learning for boosted relational dependency networks through genetic algorithm. In: Katzouris, N., Artikis, A. (eds.) ILP 2021. LNCS, vol. 13191, pp. 125–139. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-97454-1_9
Flach, P., Lachiche, N.: 1BC: a first-order bayesian classifier. In: Džeroski, S., Flach, P. (eds.) ILP 1999. LNCS (LNAI), vol. 1634, pp. 92–103. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-48751-4_10
França, M.V., Zaverucha, G., D’avila Garcez, A.S.: Fast relational learning using bottom clause propositionalization with artificial neural networks. Mach. Learn. 94(1), 81–104 (2014)
Getoor, L., Taskar, B.: Introduction to Statistical Relational Learning. MIT press, Cambridge (2007)
Han, X., Huang, Z., An, B., Bai, J.: Adaptive transfer learning on graph neural networks, pp. 565–574. Association for Computing Machinery, New York (2021)
Khosravi, H., Schulte, O., Hu, J., Gao, T.: Learning compact markov logic networks with decision trees. Mach. Learn. 89(3), 257–277 (2012)
Kramer, S., Lavrač, N., Flach, P.: Propositionalization Approaches to Relational Data Mining, pp. 262–291. Springer, Heidelberg (2001)
Kumaraswamy, R., Odom, P., Kersting, K., Leake, D., Natarajan, S.: Transfer learning via relational type matching. In: 2015 IEEE International Conference on Data Mining, pp. 811–816. IEEE (2015)
Lachiche, N., Flach, P.A.: 1BC2: a true first-order bayesian classifier. In: Matwin, S., Sammut, C. (eds.) ILP 2002. LNCS (LNAI), vol. 2583, pp. 133–148. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-36468-4_9
Lee, C.K.: Transfer learning with graph neural networks for optoelectronic properties of conjugated oligomers. J. Chem. Phys. 154(2), 024906 (2021)
Luca, T., Paes, A., Zaverucha, G.: Mapping across relational domains for transfer learning with word embeddings-based similarity. In: Katzouris, N., Artikis, A. (eds.) International Conference on Inductive Logic Programming, pp. 167–182. Springer, Heidelberg (2021). https://doi.org/10.1007/978-3-030-97454-1_12
Luca, T., Paes, A., Zaverucha, G.: Combining word embeddings-based similarity measures for transfer learning across relational domains. In: International Conference on Inductive Logic Programming. Springer, Heidelberg (2023)
Luca, T., Paes, A., Zaverucha, G.: Word embeddings-based transfer learning for boosted relational dependency networks. Mach. Learn. 1–34 (2023)
Luo, Y., Wen, Y., Duan, L.Y., Tao, D.: Transfer metric learning: algorithms, applications and outlooks. arXiv preprint arXiv:1810.03944 (2018)
MacKay, D.J.: Information Theory, Inference and Learning Algorithms. Cambridge University Press, Cambridge (2003)
Menéndez, M., Pardo, J., Pardo, L., Pardo, M.: The Jensen-Shannon divergence. J. Franklin Inst. 334(2), 307–318 (1997)
Mihalkova, L., Huynh, T., Mooney, R.J.: Mapping and revising markov logic networks for transfer learning. In: AAAI, vol. 7, pp. 608–614 (2007)
Mihalkova, L., Mooney, R.J.: Bottom-up learning of markov logic network structure. In: Proceedings of the 24th International Conference on Machine Learning, ICML 2007, pp. 625–632. Association for Computing Machinery, New York (2007)
Muggleton, S.: Inductive logic programming. New Gener. Comput. 8, 295–318 (1991)
Muggleton, S.H.: Inverse entailment and progol. New Gener. Comput. 13(3 &4), 245–286 (1995)
Natarajan, S., Khot, T., Kersting, K., Gutmann, B., Shavlik, J.: Gradient-based boosting for statistical relational learning: the relational dependency network case. Mach. Learn. 86(1), 25–56 (2012)
Neville, J., Jensen, D.: Relational dependency networks. J. Mach. Learn. Res. 8(3) (2007)
Pan, J.: Review of metric learning with transfer learning. In: AIP Conference Proceedings, vol. 1864. AIP Publishing (2017)
Pan, S.J., Kwok, J.T., Yang, Q., et al.: Transfer learning via dimensionality reduction. In: AAAI, vol. 8, pp. 677–682 (2008)
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)
Rish, I., et al.: An empirical study of the naive bayes classifier. In: IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence, vol. 3, pp. 41–46 (2001)
Tamaddoni-Nezhad, A., Muggleton, S.: The lattice structure and refinement operators for the hypothesis space bounded by a bottom clause. Mach. Learn. 76, 37–72 (2009)
Tang, X., Li, Y., Sun, Y., Yao, H., Mitra, P., Wang, S.: Transferring robustness for graph neural network against poisoning attacks, pp. 600–608. Association for Computing Machinery, New York (2020)
Torrey, L., Shavlik, J.: Transfer learning. In: Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques, pp. 242–264. IGI global (2010)
Van Haaren, J., Kolobov, A., Davis, J.: Todtler: two-order-deep transfer learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 29 (2015)
Wan, C., Pan, R., Li, J.: Bi-weighting domain adaptation for cross-language text classification. In: Twenty-Second International Joint Conference on Artificial Intelligence. Citeseer (2011)
Wrobel, S.: First order theory refinement. Adv. Inductive Logic Program. 32, 14–33 (1996)
Yang, Q., Zhang, Y., Dai, W., Pan, S.J.: Transfer Learning. Cambridge University Press, Cambridge (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Luca, T., Paes, A., Zaverucha, G. (2023). Select First, Transfer Later: Choosing Proper Datasets for Statistical Relational Transfer Learning. In: Bellodi, E., Lisi, F.A., Zese, R. (eds) Inductive Logic Programming. ILP 2023. Lecture Notes in Computer Science(), vol 14363. Springer, Cham. https://doi.org/10.1007/978-3-031-49299-0_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-49299-0_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-49298-3
Online ISBN: 978-3-031-49299-0
eBook Packages: Computer ScienceComputer Science (R0)