More Web Proxy on the site http://driver.im/

Article

TON-ViT: A Neuro-Symbolic AI Based on Task Oriented Network with a Vision Transformer

Authors:

Andrew W. Kirkpatrick,

Ross Candelore,

Chad Gorbatkin,

Christopher Colombo,

Bradley Duerstock,

Juan WachsAuthors Info & Claims

Medical Image Understanding and Analysis: 27th Annual Conference, MIUA 2023, Aberdeen, UK, July 19–21, 2023, Proceedings

Pages 157 - 170

https://doi.org/10.1007/978-3-031-48593-0_12

Published: 02 December 2023 Publication History

Abstract

The objective of this paper is to present a neuro-symbolic AI based technique to represent field-medicine knowledge, referred as to TON-ViT. TON-ViT integrates a Deep Learning Model with an explicit symbolic manipulation, a task graph. This task graph describes the steps of each trauma resuscitation as denoted by a verb and noun pair. Through this representation, symbolic processing and manipulation on task graphs, we can find stereotypical procedures, regardless of style of the performer. Furthermore, we can use this technique to find differences in styles, errors, shortcuts and generate procedures never seen before. When used in combination with a transformer, it can help recognize actions in egocentric vision datasets. Last, through symbolic manipulations on the graph, it is possible to generate medical knowledge which the model has not seen before. We present preliminary results after testing the TON-ViT with the Trauma Thompson Dataset.

References

[1]

Fensel, D., et al.: Introduction: what is a knowledge graph? In: Knowledge Graphs, pp. 1–10. Springer, Cham (2020).

[2]

Zou, X.: A survey on application of knowledge graph. J. Phys. Conf. Ser. 1487(1), 012016 (2020).

[3]

Yang, S., Zou, L., Wang, Z., Yan, J., Wen, J.-R.: Efficiently answering technical questions—a knowledge graph approach. Proc. AAAI Conf. Artif. Intell. 31(1),(2017).

[4]

Liew, C.Y., Labadin, J., Kok, W.C., Eze, M.O.: A methodology framework for bipartite network modeling. Appl. Netw. Sci. 8(1), 6 (2023).

[5]

Li, Z., et al.: Temporal knowledge graph reasoning based on evolutional representation learning. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 408–417. ACM (2021). ISBN 978-1-4503-8037-9.

[6]

Guo, Q., Zhuang, F., Qin, C., Zhu, H., Xie, X., Xiong, H., He, Q.: A survey on knowledge graph-based recommender systems. IEEE Trans. Knowl. Data Eng. 34(8), 3549–3568 (2022).

[7]

Wang, X., et al.: Learning intents behind interactions with knowledge graph for recommendation. In: Proceedings of the Web Conference 2021, pp. 878–887. ACM (2021). ISBN 978-1-4503-8312-7.

[8]

Chen, H., Luo, X.: An automatic literature knowledge graph and reasoning network modeling framework based on ontology and natural language processing. Adv. Eng. Informatics 42, 100959 (2019).

[9]

Peña, J., Rochat, Y.: Bipartite graphs as models of population structures in evolutionary multiplayer games. PLoS ONE 7(9), e44514 (2012).

[10]

Paulius, D., Huang, Y., Milton, R., Buchanan, W.D., Sam, J., Sun, Y.: Functional object-oriented network for manipulation learning. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2655–2662.

[11]

Tiddi, I., Schlobach, S.: Knowledge graphs as tools for explainable machine learning: a survey. Artif. Intell. 302, 103627 (2022).

[12]

Marino, K., Salakhutdinov, R., Gupta, A.: The more you know: using knowledge graphs for image classification. arXiv preprint arXiv:1612.04844 (2017)

[13]

Paulheim, H.: Knowledge graph refinement: a survey of approaches and evaluation methods. Semantic Web 8(3), 489–508 (2016).

[14]

Chaudhri, V.K., et al.: Knowledge graphs: introduction, history, and perspectives. AI Magazine 43(1), 17–29 (2022).

[15]

Lecue, F.: On the role of knowledge graphs in explainable AI. Semantic Web 11(1), 41–51 (2020).

[16]

Zhang, W., Paudel, B., Zhang, W., Bernstein, A., Chen, H.: Interaction embeddings for prediction and explanation in knowledge graphs. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pp. 96–104. ACM (2019). ISBN 978-1-4503-5940-5.

[17]

Manghani, K.: Quality assurance: importance of systems and standard operating procedures. Perspect. Clin. Res. 2(1), 34 (2011).

[18]

Hitzler, P., Eberhart, A., Ebrahimi, M., Sarker, M.K., Zhou, L.: Neuro-symbolic approaches in artificial intelligence. Natl. Sci. Rev. 9(6), nwac035 (2022).

[19]

Hitzler, P.: Some advances regarding ontologies and neuro-symbolic artificial intelligence. In: Brazdil, P., van Rijn, J.N., Gouk, H., Mohr, F. (eds.) ECMLPKDD Workshop on Meta-Knowledge Transfer, volume 191 of Proceedings of Machine Learning Research, pp. 8–10. PMLR (2022). www.proceedings.mlr.press/v191/hitzler22a.html

[20]

Xie, X., Kersting, K., Neider, D.: Neuro-symbolic verification of deep neural networks. arXiv preprint arXiv:2203.00938 (2022)

[21]

Hamilton, K., Nayak, A., Božić, B., Longo, L.: Is Neuro-symbolic AI Meeting Its Promises in Natural Language Processing? A Structured Review, pp. 1–42 (2022). www.medra.org/servlet/aliasResolver?alias=iospress &doi=10.3233/SW-223228

[22]

Oltramari, A., Francis, J., Henson, C., Ma, K., Wickramarachchi, R.: Neuro-symbolic architectures for context understanding. arXiv preprint arxiv.org/abs/2003.04707 (2020). Publisher: arXiv Version Number: 1

[23]

Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2021)

[24]

Pan, X., Ye, T., Han, D., Song, S., Huang, G.: Contrastive language-image pre-training with knowledge graphs. arXiv preprint arXiv:2210.08901 (2022)

[25]

Zhao, H., Torralba, A., Torresani, L., Yan, Z.: HACS: human action clips and segments dataset for recognition and temporal localization. arXiv preprint arXiv:1712.09374 (2019)

[26]

Rao, T.S., Radhakrishnan, R., Andrade, C.: Standard operating procedures for clinical practice. Ind. J. Psychiatry 53(1), 1–3 (2011).

Cited By

Zhuo YKirkpatrick ACouperus KTran OBeck JDeVane DCandelore RMcKee JColombo CGorbatkin CBirch EDuerstock BWachs J(2023)Overview of the Trauma THOMPSON Challenge at MICCAI 2023AI for Brain Lesion Detection and Trauma Video Action Recognition10.1007/978-3-031-71626-3_7(47-60)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1007/978-3-031-71626-3_7

Index Terms

TON-ViT: A Neuro-Symbolic AI Based on Task Oriented Network with a Vision Transformer
1. Computing methodologies
  1. Artificial intelligence
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

ImplantFormer: vision transformer-based implant position regression using dental CBCT data
Abstract
Implant prosthesis is the most appropriate treatment for dentition defect or dentition loss, which usually involves a surgical guide design process to decide the implant position. However, such design heavily relies on the subjective experiences ...
‘Precision Health’: Balancing Reactive Care and Proactive Care Through the Evidence Based Knowledge Graph Constructed from Real-World Electronic Health Records, Disease Trajectories, Diseasome, and Patholome
Big Data Analytics
Abstract
Health care can be either ‘Reactive Care’ or ‘Proactive Care’. Reactive care is self-referral where a medical help is solicited by the person or family members on suspecting illness. In Proactive care an individual seeks medical help before the ...
A prefetch-aware scheduling for FPGA-based multi-task graph systems
Abstract
In partially run-time reconfigurable (PRR) FPGAs, hardware tasks should be configured before their execution. The configuration delay imposed by the reconfiguration process increases the total execution time of the hardware tasks and task graphs. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

Medical Image Understanding and Analysis: 27th Annual Conference, MIUA 2023, Aberdeen, UK, July 19–21, 2023, Proceedings

Jul 2023

345 pages

ISBN:978-3-031-48592-3

DOI:10.1007/978-3-031-48593-0

Editors:
Gordon Waiter
https://ror.org/016476m91University of Aberdeen, Aberdeen, UK
,
Tryphon Lambrou
https://ror.org/016476m91University of Aberdeen, Aberdeen, UK
,
Georgios Leontidis
https://ror.org/016476m91University of Aberdeen, Aberdeen, UK
,
Nir Oren
https://ror.org/016476m91University of Aberdeen, Aberdeen, UK
,
Teresa Morris
https://ror.org/016476m91University of Aberdeen, Aberdeen, UK
,
Sharon Gordon
https://ror.org/016476m91University of Aberdeen, Aberdeen, UK

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 02 December 2023

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhuo YKirkpatrick ACouperus KTran OBeck JDeVane DCandelore RMcKee JColombo CGorbatkin CBirch EDuerstock BWachs J(2023)Overview of the Trauma THOMPSON Challenge at MICCAI 2023AI for Brain Lesion Detection and Trauma Video Action Recognition10.1007/978-3-031-71626-3_7(47-60)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1007/978-3-031-71626-3_7

View Options

View options

Media

Figures

Other

Tables

View Table of Contents