More Web Proxy on the site http://driver.im/

survey

Pre-trained Language Models in Biomedical Domain: A Systematic Survey

Authors:

Jie FuAuthors Info & Claims

ACM Computing Surveys, Volume 56, Issue 3

Article No.: 55, Pages 1 - 52

https://doi.org/10.1145/3611651

Published: 05 October 2023 Publication History

Abstract

Pre-trained language models (PLMs) have been the de facto paradigm for most natural language processing tasks. This also benefits the biomedical domain: researchers from informatics, medicine, and computer science communities propose various PLMs trained on biomedical datasets, e.g., biomedical text, electronic health records, protein, and DNA sequences for various biomedical tasks. However, the cross-discipline characteristics of biomedical PLMs hinder their spreading among communities; some existing works are isolated from each other without comprehensive comparison and discussions. It is nontrivial to make a survey that not only systematically reviews recent advances in biomedical PLMs and their applications but also standardizes terminology and benchmarks. This article summarizes the recent progress of pre-trained language models in the biomedical domain and their applications in downstream biomedical tasks. Particularly, we discuss the motivations of PLMs in the biomedical domain and introduce the key concepts of pre-trained language models. We then propose a taxonomy of existing biomedical PLMs that categorizes them from various perspectives systematically. Plus, their applications in biomedical downstream tasks are exhaustively discussed, respectively. Last, we illustrate various limitations and future trends, which aims to provide inspiration for the future research.

References

[1]

Asma Ben Abacha and Dina Demner-Fushman. 2016. Recognizing question entailment for medical question answering. In AMIA Annual Symposium Proceedings, Vol. 2016. American Medical Informatics Association, 310.

[2]

Asma Ben Abacha, Chaitanya Shivade, and Dina Demner-Fushman. 2019. Overview of the mediqa 2019 shared task on textual inference, question entailment and question answering. In BioNLP Workshop and Shared Task. 370–379.

[3]

Arda Akdemir and Tetsuo Shibuya. 2020. Transfer learning for biomedical question answering. In CLEF (Working Notes).

[4]

Liliya Akhtyamova. 2020. Named entity recognition in spanish biomedical literature: Short review and bert model. In FRUCT. IEEE, 1–7.

[5]

Liliya Akhtyamova, Paloma Martínez, Karin Verspoor, and John Cardiff. 2020. Testing contextualized word embeddings to improve ner in spanish clinical case narratives. IEEE Access 8 (2020), 164717–164726.

[6]

Israa Alghanmi, Luis Espinosa-Anke, and Steven Schockaert. 2021. Probing pre-trained language models for disease knowledge. arXiv:2106.07285. Retrieved from https://arxiv.org/abs/2106.07285

[7]

Ethan C. Alley, Grigory Khimulya, Surojit Biswas, Mohammed AlQuraishi, and George M. Church. 2019. Unified rational protein engineering with sequence-only deep representation learning (unpublished).

[8]

Emily Alsentzer, John R. Murphy, Willie Boag, Wei-Hung Weng, Di Jin, Tristan Naumann, and Matthew B. A. McDermott. 2019. Publicly available clinical BERT embeddings. arxiv:1904.03323. Retrieved from http://arxiv.org/abs/1904.03323

[9]

Sophia Ananiadou, Sampo Pyysalo, Jun’ichi Tsujii, and Douglas B. Kell. 2010. Event extraction for systems biology by text mining the literature. Trends Biotechnol. 28, 7 (2010), 381–390.

[10]

Christian B. Anfinsen. 1973. Principles that govern the folding of protein chains. Science 181, 4096 (1973), 223–230.

[11]

Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023. Palm 2 technical report. arXiv:2305.10403. Retrieved from https://arxiv.org/abs/2305.10403

[12]

Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Arabert: Transformer-based model for Arabic language understanding. arXiv:2003.00104. Retrieved from https://arxiv.org/abs/2003.00104

[13]

Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jing Jin, Xin Jiang, Qun Liu, Michael Lyu, and Irwin King. 2020. Binarybert: Pushing the limit of bert quantization. arxiv:cs.CL/2012.15701. Retrieved from https://arxiv.org/abs/2012.15701

[14]

Pratyay Banerjee, Kuntal Kumar Pal, Murthy Devarakonda, and Chitta Baral. 2021. Biomedical named entity recognition via knowledge guidance and question answering. ACM Trans. Comput. Healthc. 2, 4 (2021), 1–24.

Digital Library

[15]

Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization. 65–72.

Digital Library

[16]

Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. 2021. BEiT: BERT pre-training of image transformers. In ICLR.

[17]

Marco Basaldella, Fangyu Liu, Ehsan Shareghi, and Nigel Collier. 2020. Cometa: A corpus for medical entity linking in the social media. arXiv:2010.03295. Retrieved from https://arxiv.org/abs/2010.03295

[18]

Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. SciBERT: A pretrained language model for scientific text. In EMNLP-IJCNLP, Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, 3613–3618. DOI:

[19]

Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv: 2004.05150. Retrieved from https://arxiv.org/abs/2004.05150

[20]

Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. J. Mach. Learn. Res. 3 (Feb. 2003), 1137–1155.

Digital Library

[21]

Tristan Bepler and Bonnie Berger. 2018. Learning protein sequence embeddings using information from structure. In ICLR.

[22]

Tristan Bepler and Bonnie Berger. 2019. Learning protein sequence embeddings using information from structure. arXiv:1902.08661. Retrieved from https://arxiv.org/abs/1902.08661

[23]

Jennifer Bishop, Qianqian Xie, and Sophia Ananiadou. 2022. GenCompareSum: A hybrid unsupervised summarization method using salience. In Proceedings of the 21st Workshop on Biomedical Language Processing. 220–240.

[24]

Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of “bias” in nlp. arXiv:2005.14050. Retrieved from https://arxiv.org/abs/2005.14050

[25]

Olivier Bodenreider. 2004. The unified medical language system (umls): integrating biomedical terminology. Nucl. Acids Res. 32, suppl_1 (2004), D267–D270.

[26]

Benedikt Boecking, Naoto Usuyama, Shruthi Bannur, Daniel C. Castro, Anton Schwaighofer, Stephanie Hyland, Maria Wetscherek, Tristan Naumann, Aditya Nori, Javier Alvarez-Valle, et al. 2022. Making the most of text semantics to improve biomedical vision–language processing. arXiv:2204.09817. Retrieved from https://arxiv.org/abs/2204.09817

[27]

Rishi Bommasani and et al.2021. On the opportunities and risks of foundation models. arxiv:cs.LG/2108.07258. Retrieved from https://arxiv.org/abs/2103.07258

[28]

Nada Boudjellal, Huaping Zhang, Asif Khan, Arshad Ahmad, Rashid Naseem, Jianyun Shang, and Lin Dai. 2021. Abioner: A bert-based model for arabic biomedical named-entity recognition (unpublished).

[29]

Nadav Brandes, Dan Ofer, Yam Peleg, Nadav Rappoport, and Michal Linial. 2022. ProteinBERT: A universal deep-learning model of protein sequence and function. Bioinformatics 38, 8 (2022), 2102–2110.

[30]

Àlex Bravo, Janet Piñero, Núria Queralt-Rosinach, Michael Rautschka, and Laura I. Furlong. 2015. Extraction of relations between genes and diseases from text and large-scale data analysis: Implications for translational research. BMC Bioinf. 16, 1 (2015), 1–17.

[31]

Keno K. Bressem, Lisa C. Adams, Robert A. Gaudin, Daniel Tröltzsch, Bernd Hamm, Marcus R. Makowski, Chan-Yong Schüle, Janis L. Vahldiek, and Stefan M. Niehues. 2020. Highly accurate classification of chest radiographic reports using a deep learning natural language model pre-trained on 3.8 million text reports. Bioinformatics 36, 21 (2020), 5255–5261.

[32]

Eric Brochu, Vlad M. Cora, and Nando De Freitas. 2010. A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv:1012.2599. Retrieved from https://arxiv.org/abs/1012.2599

[33]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. arXiv:2005.14165. Retrieved from https://arxiv.org/abs/2005.14165

[34]

Boxi Cao, Hongyu Lin, Xianpei Han, Le Sun, Lingyong Yan, Meng Liao, Tong Xue, and Jin Xu. 2021. Knowledgeable or educated guess? revisiting language models as knowledge bases. In ACL. 1860–1874.

[35]

Rosario Catelli, Francesco Gargiulo, Valentina Casola, Giuseppe De Pietro, Hamido Fujita, and Massimo Esposito. 2020. Crosslingual named entity recognition for clinical de-identification applied to a covid-19 italian data set. Appl. Soft Comput. 97 (2020), 106779.

[36]

Asli Celikyilmaz, Elizabeth Clark, and Jianfeng Gao. 2020. Evaluation of text generation: A survey. arXiv:2006.14799. Retrieved from https://arxiv.org/abs/2006.14799

[37]

Souradip Chakraborty, Ekaba Bisong, Shweta Bhatt, Thomas Wagner, Riley Elliott, and Francesco Mosconi. 2020. Biomedbert: A pre-trained biomedical language model for qa and ir. In ICCL. 669–679.

[38]

Pierre Chambon, Christian Bluethgen, Jean-Benoit Delbrouck, Rogier Van der Sluijs, Małgorzata Połacin, Juan Manuel Zambrano Chaves, Tanishq Mathew Abraham, Shivanshu Purohit, Curtis P. Langlotz, and Akshay Chaudhari. 2022. RoentGen: Vision-language foundation model for chest x-ray generation. arXiv:2211.12737. Retrieved from https://arxiv.org/abs/2211.12737

[39]

Pierre Chambon, Christian Bluethgen, Curtis P. Langlotz, and Akshay Chaudhari. 2022. Adapting pretrained vision-language foundational models to medical imaging domains. arXiv:2210.04133. Retrieved from https://arxiv.org/abs/2210.04133

[40]

Qingyu Chen, Jingcheng Du, Sun Kim, W. John Wilbur, and Zhiyong Lu. 2019. Evaluation of five sentence similarity models on electronic medical records. In ACM-BCB. 533–533.

[41]

Qingyu Chen, Jingcheng Du, Sun Kim, W. John Wilbur, and Zhiyong Lu. 2020. Deep learning with sentence embeddings pre-trained on biomedical corpora improves the performance of finding similar sentences in electronic medical records. BMC Med. Inf. Decis. Mak. 20 (2020), 1–10.

[42]

Qingyu Chen, Yifan Peng, and Zhiyong Lu. 2019. Biosentvec: Creating sentence embeddings for biomedical texts. In ICHI. IEEE, 1–5.

[43]

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In ICML. PMLR, 1597–1607.

[44]

Tao Chen, Mingfen Wu, and Hexi Li. 2019. A general approach for improving deep learning-based medical relation extraction using a pre-trained model and fine-tuning. Database (2019), baz116.

[45]

Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020. Uniter: Universal image-text representation learning. In European Conference on Computer Vision. Springer, 104–120.

Digital Library

[46]

Zhihong Chen, Yuhao Du, Jinpeng Hu, Yang Liu, Guanbin Li, Xiang Wan, and Tsung-Hui Chang. 2022. Multi-modal masked autoencoders for medical vision-and-language pre-training. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 679–689.

Digital Library

[47]

Zhihong Chen, Guanbin Li, and Xiang Wan. 2022. Align, reason and learn: Enhancing medical vision-and-language pre-training with knowledge. In Multimedia. 5152–5161.

[48]

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv:2204.02311. Retrieved from https://arxiv.org/abs/2204.02311

[49]

Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. arxiv:cs.CL/2003.10555. Retrieved from https://arxiv.org/abs/2003.10555

[50]

Aaron M. Cohen and William R. Hersh. 2005. A survey of current work in biomedical text mining. Brief. Bioinf. 6, 1 (2005), 57–71.

[51]

Jacques Cohen. 2004. Bioinformatics—An introduction for computer scientists. ACM Comput. Surv. 36, 2 (Jun.2004), 122–158. DOI:

Digital Library

[52]

Kevin Bretonnel Cohen and Dina Demner-Fushman. 2014. Biomedical Natural Language Processing. Vol. 11. John Benjamins Publishing Company.

[53]

Pedro Colon-Hernandez, Catherine Havasi, Jason Alonso, Matthew Huggins, and Cynthia Breazeal. 2021. Combining pre-trained language models and structured knowledge. arXiv:2101.12294. Retrieved from https://arxiv.org/abs/2101.12294

[54]

Jenny Copara, Julien Knafou, Nona Naderi, Claudia Moro, Patrick Ruch, and Douglas Teodoro. 2020. Contextualized french language models for biomedical named entity recognition. In JEP/TALN/RÉCITAL. ATALA, 36–48.

[55]

Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, and Guoping Hu. 2019. Pre-training with whole word masking for chinese bert. arXiv:1906.08101. Retrieved from https://arxiv.org/abs/1906.08101

[56]

Dina Demner-Fushman, Marc D. Kohli, Marc B. Rosenman, Sonya E. Shooshan, Laritza Rodriguez, Sameer Antani, George R. Thoma, and Clement J. McDonald. 2016. Preparing a collection of radiology examinations for distribution and retrieval. J. Am. Med. Inf. Assoc. 23, 2 (2016), 304–310.

[57]

Dina Demner-Fushman, Marc D. Kohli, Marc B. Rosenman, Sonya E. Shooshan, Laritza Rodriguez, Sameer Antani, George R. Thoma, and Clement J. McDonald. 2016. Preparing a collection of radiology examinations for distribution and retrieval. J. Am. Med. Inf. Assoc. 23, 2 (2016), 304–310.

[58]

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In CVPR. IEEE, 248–255.

[59]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805. Retrieved from https://arxiv.org/abs/1810.04805

[60]

Jay DeYoung, Iz Beltagy, Madeleine van Zuylen, Bailey Kuehl, and Lucy Wang. 2021. MSˆ2: Multi-document summarization of medical studies. In EMNLP. 7494–7513.

[61]

George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In HLTR. 138–145.

[62]

Rezarta Islamaj Doğan, Robert Leaman, and Zhiyong Lu. 2014. Ncbi disease corpus: A resource for disease name recognition and concept normalization. J. Biomed. Inf. 47 (2014), 1–10.

[63]

Yihe Dong, Jean-Baptiste Cordonnier, and Andreas Loukas. 2021. Attention is not all you need: Pure attention loses rank doubly exponentially with depth. arXiv:2103.03404. Retrieved from https://arxiv.org/abs/a2103.03404

[64]

Nan Du, Mingqiu Wang, Linh Tran, Gang Li, and Izhak Shafran. 2020. Learning to infer entities, properties and their relations from clinical conversations. In EMNLP-IJCNLP. Association for Computational Linguistics, 4978–4989.

[65]

Yongping Du, Qingxiao Li, Lulin Wang, and Yanqing He. 2020. Biomedical-domain pre-trained language model for extractive summarization. Knowl.-Bas. Syst. 199 (2020), 105964.

[66]

Yongping Du, Bingbing Pei, Xiaozheng Zhao, and Junzhong Ji. 2020. Deep scaled dot-product attention based domain adaptation model for biomedical question answering. Methods 173 (2020), 69–74.

[67]

Marco Eichelberg, Thomas Aden, Jörg Riesmeier, Asuman Dogac, and Gokce B. Laleci. 2005. A survey and analysis of electronic healthcare record standards. ACM Comput. Surv. 37, 4 (Dec.2005), 277–315. DOI:

Digital Library

[68]

Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rihawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al. 2020. Prottrans: Towards cracking the language of life’s code through self-supervised deep learning and high performance computing. arXiv:2007.06225. Retrieved from https://arxiv.org/abs/2007.06225

[69]

Andre Esteva, Anuprit Kale, Romain Paulus, Kazuma Hashimoto, Wenpeng Yin, Dragomir Radev, and Richard Socher. 2020. Co-search: COVID-19 information retrieval with semantic search, question answering, and abstractive summarization. arXiv:2006.09595. Retrieved from https://arxiv.org/abs/2006.09595

[70]

Yixing Fan, Liang Pang, JianPeng Hou, Jiafeng Guo, Yanyan Lan, and Xueqi Cheng. 2017. Matchzoo: A toolkit for deep text matching. arXiv:1707.07270. Retrieved from https://arxiv.org/abs/1707.07270

[71]

Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. 2020. Codebert: A pre-trained model for programming and natural languages. arXiv:2002.08155. Retrieved from https://arxiv.org/abs/2002.08155

[72]

Shang Gao, Mohammed Alawad, Michael Todd Young, John Gounley, Noah Schaefferkoetter, Hong-Jun Yoon, Xiao-Cheng Wu, Eric B. Durbin, Jennifer Doherty, Antoinette Stroup, et al. 2021. Limitations of transformers on clinical text classification. IEEE J. Biomed. Health Inf. (2021).

[73]

Shang Gao, Olivera Kotevska, Alexandre Sorokine, and J. Blair Christian. 2021. A pre-training and self-training approach for biomedical named entity recognition. PLoS ONE 16, 2 (2021), e0246310.

[74]

Ismael Garrido-Muñoz, Arturo Montejo-Ráez, Fernando Martínez-Santiago, and L. Alfonso Ureña-López. 2021. A survey on bias in deep nlp. Appl. Sci. 11, 7 (2021), 3184.

[75]

Sajad Sotudeh Gharebagh, Nazli Goharian, and Ross Filice. 2020. Attend to medical ontologies: Content selection for clinical abstractive summarization. In ACL. 1899–1905.

[76]

John Giorgi, Xindi Wang, Nicola Sahar, Won Young Shin, Gary D. Bader, and Bo Wang. 2019. End-to-end named entity recognition and relation extraction using pre-trained language models. arXiv:1912.13415. Retrieved from https://arxiv.org/abs/1912.134.15

[77]

Graciela Gonzalez-Hernandez, Abeed Sarker, Karen O’Connor, and Guergana Savova. 2017. Capturing the patient’s perspective: A review of advances in natural language processing of health-related text. Yrbk. Med. Inf. 26, 1 (2017), 214.

[78]

Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. 2020. Bootstrap your own latent-a new approach to self-supervised learning. Adv. Neural Inf. Process. Syst. 33 (2020), 21271–21284.

[79]

Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Domain-specific language model pretraining for biomedical natural language processing. ACM Trans. Comput. Healthc. 3, 1 (2021), 1–23.

Digital Library

[80]

Hong Guan, Jianfu Li, Hua Xu, and Murthy Devarakonda. 2020. Robustly pre-trained neural model for direct temporal relation extraction. arXiv:2004.06216. Retrieved from https://arxiv.org/abs/2004.06216

[81]

Yuting Guo, Xiangjue Dong, Mohammed Ali Al-Garadi, Abeed Sarker, Cécile Paris, and Diego Mollá Aliod. 2020. Benchmarking of transformer-based pre-trained models on social media text classification datasets. In Workshop of the Australasian Language Technology Association. 86–91.

[82]

Yue Guo, Wei Qiu, Yizhong Wang, and Trevor Cohen. 2020. Automated lay language summarization of biomedical scientific reviews. arXiv:2012.12573. Retrieved from https://arxiv.org/abs/2012.12573

[83]

Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In ACL, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 8342–8360. DOI:

[84]

Bernal Jimenez Gutierrez, Jucheng Zeng, Dongdong Zhang, Ping Zhang, and Yu Su. 2020. Document classification for covid-19 literature. In EMNLP: Findings. 3715–3722.

[85]

Ridong Han, Tao Peng, Chaohao Yang, Benyou Wang, Lu Liu, and Xiang Wan. 2023. Is information extraction solved by ChatGPT? An analysis of performance, evaluation criteria, robustness and errors. arXiv:2305.14450. Retrieved from https://arxiv.org/abs/2305.14450

[86]

Xu Han, Zhengyan Zhang, Ning Ding, Yuxian Gu, Xiao Liu, Yuqi Huo, Jiezhong Qiu, Liang Zhang, Wentao Han, Minlie Huang, Qin Jin, Yanyan Lan, Yang Liu, Zhiyuan Liu, Zhiwu Lu, Xipeng Qiu, Ruihua Song, Jie Tang, Ji-Rong Wen, Jinhui Yuan, Wayne Xin Zhao, and Jun Zhu. 2021. Pre-trained models: Past, present and future. arxiv:cs.AI/2106.07139. Retrieved from https://arxiv.org/abs/2106.07139

[87]

Douglas Hanahan and Robert A. Weinberg. 2000. The hallmarks of cancer. Cell 100, 1 (2000), 57–70.

[88]

Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. 2022. Masked autoencoders are scalable vision learners. In CVPR. 16000–16009.

[89]

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In CVPR. 9729–9738.

[90]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. 770–778.

[91]

Yun He, Ziwei Zhu, Yin Zhang, Qin Chen, and James Caverlee. 2020. Infusing disease knowledge into bert for health question answering, medical inference and disease name recognition. In EMNLP. 4604–4614.

[92]

Michael Heinzinger, Ahmed Elnaggar, Yu Wang, Christian Dallago, Dmitrii Nechaev, Florian Matthes, and Burkhard Rost. 2019. Modeling the language of life–deep learning protein sequences (unpublished).

[93]

Michael Heinzinger, Ahmed Elnaggar, Yu Wang, Christian Dallago, Dmitrii Nechaev, Florian Matthes, and Burkhard Rost. 2019. Modeling aspects of the language of life through transfer-learning protein sequences. BMC Bioinf. 20, 1 (2019), 1–17.

[94]

Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (gelus). arXiv:1606.08415. Retrieved from https://arxiv.org/abs/1606.08415

[95]

Sam Henry, Kevin Buchan, Michele Filannino, Amber Stubbs, and Ozlem Uzuner. 2020. 2018 n2c2 shared task on adverse drug events and medication extraction in electronic health records. J. Am. Med. Inf. Assoc. 27, 1 (2020), 3–12.

[96]

María Herrero-Zazo, Isabel Segura-Bedmar, Paloma Martínez, and Thierry Declerck. 2013. The ddi corpus: An annotated corpus with pharmacological substances and drug–drug interactions. J. Biomed. Inf. 46, 5 (2013), 914–920.

Digital Library

[97]

Emily Herrett, Arlene M. Gallagher, Krishnan Bhaskaran, Harriet Forbes, Rohini Mathur, Tjeerd Van Staa, and Liam Smeeth. 2015. Data resource profile: Clinical practice research datalink (cprd). Int. J. Epidemiol. 44, 3 (2015), 827–836.

[98]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput. 9, 8 (1997), 1735–1780.

Digital Library

[99]

Zengyan Hong, Xiangxiang Zeng, Leyi Wei, and Xiangrong Liu. 2020. Identifying enhancer–promoter interactions with neural network based on pre-trained DNA vectors and attention mechanism. Bioinformatics 36, 4 (2020), 1037–1043.

[100]

Lu Hou, Zhiqi Huang, Lifeng Shang, Xin Jiang, Xiao Chen, and Qun Liu. 2020. Dynabert: Dynamic bert with adaptive width and depth. arxiv:cs.CL/2004.04037. Retrieved from https://arxiv.org/abs/2004.04037

[101]

Tzu-Ming Harry Hsu, Wei-Hung Weng, Willie Boag, Matthew McDermott, and Peter Szolovits. 2018. Unsupervised multimodal representation learning across medical images and reports. arXiv:1811.08615. Retrieved from https://arxiv.org/abs/1811.08615

[102]

Jinpeng Hu, Zhuo Li, Zhihong Chen, Zhen Li, Xiang Wan, and Tsung-Hui Chang. 2022. Graph enhanced contrastive learning for radiology findings summarization. In ACL (Volume 1: Long Papers). 4677–4688.

[103]

Kexin Huang, Jaan Altosaar, and Rajesh Ranganath. 2019. Clinicalbert: Modeling clinical notes and predicting hospital readmission. arxiv:1904.05342. Retrieved from http://arxiv.org/abs/1904.05342

[104]

Kexin Huang, Abhishek Singh, Sitong Chen, Edward Moseley, Chin ying Deng, Naomi George, and Charlotta Lindvall. 2019. Clinical xlnet: Modeling sequential clinical notes and predicting prolonged mechanical ventilation. arXiv:1912.11975. Retrieved from https://arxiv.org/abs/1912.11975

[105]

Shih-Cheng Huang, Liyue Shen, Matthew P. Lungren, and Serena Yeung. 2021. Gloria: A multimodal global-local representation learning framework for label-efficient medical image recognition. In IEEE/CVF ICCV. 3942–3951.

[106]

Zhicheng Huang, Zhaoyang Zeng, Bei Liu, Dongmei Fu, and Jianlong Fu. 2020. Pixel-bert: Aligning image pixels with text by deep multi-modal transformers. arXiv:2004.00849. Retrieved from https://arxiv.org/abs/2004.00849

[107]

Minbyul Jeong, Mujeen Sung, Gangwoo Kim, Donghyeon Kim, Wonjin Yoon, Jaehyo Yoo, and Jaewoo Kang. 2020. Transferability of natural language inference to biomedical question answering. arXiv:2007.00217. Retrieved from https://arxiv.org/abs/2007.00217

[108]

Kishlay Jha and Aidong Zhang. 2022. Continual knowledge infusion into pre-trained biomedical language models. Bioinformatics 38, 2 (2022), 494–502.

[109]

Yanrong Ji, Zhihan Zhou, Han Liu, and Ramana V. Davuluri. 2020. Dnabert: Pre-trained bidirectional encoder representations from transformers model for dna-language in genome (unpublished).

[110]

Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2020. Tinybert: Distilling bert for natural language understanding. arxiv:cs.CL/1909.10351. Retrieved from https://arxiv.org/abs/1909.10351

[111]

Qiao Jin, Bhuwan Dhingra, William Cohen, and Xinghua Lu. 2019. Probing biomedical embeddings from language models. In Workshop on Evaluating Vector Space Representations for NLP. 82–89.

[112]

Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. Pubmedqa: A dataset for biomedical research question answering. In EMNLP-IJCNLP. 2567–2577.

[113]

Alistair E. W. Johnson, Tom J. Pollard, Seth J. Berkowitz, Nathaniel R. Greenbaum, Matthew P. Lungren, Chih-ying Deng, Roger G. Mark, and Steven Horng. 2019. MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Sci. Data 6, 1 (2019), 317.

[114]

Alistair E. W. Johnson, Tom J. Pollard, Nathaniel R. Greenbaum, Matthew P. Lungren, Chih-ying Deng, Yifan Peng, Zhiyong Lu, Roger G. Mark, Seth J. Berkowitz, and Steven Horng. 2019. MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs. arXiv:1901.07042. Retrieved from https://arxiv.org/abs/1901.07042

[115]

Alistair E. W. Johnson, Tom J. Pollard, Lu Shen, H. Lehman Li-Wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. 2016. Mimic-iii, a freely accessible critical care database. Sci. Data 3, 1 (2016), 1–9.

[116]

John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. 2021. Highly accurate protein structure prediction with alphafold. Nature (2021), 1.

[117]

Dan Jurafsky. 2000. Speech & Language Processing. Pearson Education India.

Digital Library

[118]

Katikapalli Subramanyam Kalyan, Ajit Rajasekharan, and Sivanesan Sangeetha. 2021. Ammu—A survey of transformer-based biomedical pretrained language models. arXiv:2105.00827. Retrieved from https://arxiv.org/abs/2105.00827

[119]

Katikapalli Subramanyam Kalyan and S. Sangeetha. 2020. Secnlp: A survey of embeddings in clinical natural language processing. J. Biomed. Inf. 101 (2020), 103323.

Digital Library

[120]

Sanjay Kamath, Brigitte Grau, and Yue Ma. 2019. How to pre-train your model? comparison of different pre-training models for biomedical question answering. In ECML-PKDD. Springer, 646–660.

[121]

Yoshimasa Kawazoe, Daisaku Shibata, Emiko Shinohara, Eiji Aramaki, and Kazuhiko Ohe. 2020. A clinical specific bert developed with huge size of japanese clinical narrative (unpublished).

[122]

Muhammad Raza Khan, Morteza Ziyadi, and Mohamed AbdelHady. 2020. Mt-bioner: Multi-task learning for biomedical named entity recognition using deep bidirectional transformers. arXiv:2001.08904. Retrieved from https://arxiv.org/abs/2001.08904

[123]

Faiza Khan Khattak, Serena Jeblee, Chloé Pou-Prom, Mohamed Abdalla, Christopher Meaney, and Frank Rudzicz. 2019. A survey of word embeddings for clinical text. J. Biomed. Inf. X 4 (2019), 100057.

Digital Library

[124]

Virapat Kieuvongngam, Bowen Tan, and Yiming Niu. 2020. Automatic text summarization of covid-19 medical research articles using bert and gpt-2. arXiv:2006.01997. Retrieved from https://arxiv/org/abs/2006.01997

[125]

Jin-Dong Kim, Tomoko Ohta, Yoshimasa Tsuruoka, Yuka Tateisi, and Nigel Collier. 2004. Introduction to the bio-entity recognition task at jnlpba. In NLPBA/BioNLP. Citeseer, 70–75.

[126]

Jin-Dong Kim, Yue Wang, Toshihisa Takagi, and Akinori Yonezawa. 2011. Overview of genia event task in bionlp shared task 2011. In BioNLP Shared Task 2011 Workshop. 7–15.

[127]

Yoon Kim. 2014. Convolutional neural networks for sentence classification. In EMNLP. 1746–1751.

[128]

Young-Min Kim and Tae-Hoon Lee. 2020. Korean clinical entity recognition from diagnosis text using bert. BMC Med. Inf. Decis. Mak. 20, 7 (2020), 1–9.

[129]

David T. Kingsbury. 1996. Computational biology. ACM Comput. Surv. 28, 1 (Mar.1996), 101–103. DOI:

Digital Library

[130]

Samantha Kleinberg and George Hripcsak. 2011. A review of causal inference for biomedical informatics. J. Biomed. Inf. 44, 6 (2011), 1102–1112.

Digital Library

[131]

Vaishnavi Kommaraju, Karthick Gunasekaran, Kun Li, Trapit Bansal, Andrew McCallum, Ivana Williams, and Ana-Maria Istrate. 2020. Unsupervised pre-training for biomedical question answering. arXiv:2009.12952. Retrieved from https://arxiv.org/abs/2009.12952

[132]

Fajri Koto, Jey Han Lau, and Timothy Baldwin. 2021. Discourse probing of pretrained language models. arXiv:2104.05882. Retrieved from https://arxiv.org/abs/2104.05882

[133]

Zeljko Kraljevic, Anthony Shek, Daniel Bean, Rebecca Bendayan, James Teo, and Richard Dobson. 2021. Medgpt: Medical concept prediction from clinical narratives. arXiv:2107.03134. Retrieved from https://arxiv.org/abs/2107.03134

[134]

Martin Krallinger, Florian Leitner, Carlos Rodriguez-Penagos, and Alfonso Valencia. 2008. Overview of the protein-protein interaction annotation extraction task of BioCreative II. Genome Biol. 9, 2 (2008), 1–19.

[135]

Martin Krallinger, Obdulia Rabal, Saber A. Akhondi, Martın Pérez Pérez, Jesús Santamaría, Gael Pérez Rodríguez, et al. 2017. Overview of the biocreative vi chemical-protein interaction track. In BioCreative Challenge Evaluation Workshop, Vol. 1. 141–146.

[136]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2017. Imagenet classification with deep convolutional neural networks. Commun. ACM 60, 6 (2017), 84–90.

Digital Library

[137]

Andriy Kryshtafovych, Torsten Schwede, Maya Topf, Krzysztof Fidelis, and John Moult. 2019. Critical assessment of methods of protein structure prediction (CASP)–Round XIII. Proteins: Struct. Funct. Bioinf. 87, 12 (2019), 1011–1020.

[138]

Taku Kudo and John Richardson. 2018. SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing. In EMNLP: System Demonstrations. 66–71.

[139]

Tiffany H. Kung, Morgan Cheatham, Arielle Medinilla, ChatGPT, Czarina Sillos, Lorie De Leon, Camille Elepano, Marie Madriaga, Rimel Aggabao, Giezel Diaz-Candido, et al. 2022. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models (unpublished).

[140]

Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv:1909.11942. Retrieved from https://arxiv.org/abs/1909.11942

[141]

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278–2324.

[142]

Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. Biobert: A pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36, 4 (2020), 1234–1240. DOI:

[143]

Eric Lehman, Sarthak Jain, Karl Pichotta, Yoav Goldberg, and Byron C. Wallace. 2021. Does bert pretrained on clinical notes reveal sensitive data? arXiv:2104.07762. Retrieved from https://arxiv.org/abs/2104.07762

[144]

Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv:1910.13461. Retrieved from https://arxiv.org/abs/1910.13461

[145]

Patrick Lewis, Myle Ott, Jingfei Du, and Veselin Stoyanov. 2020. Pretrained language models for biomedical and clinical tasks: Understanding and extending the state-of-the-art. In Clinical Natural Language Processing Workshop. 146–157.

[146]

Fei Li, Yonghao Jin, Weisong Liu, Bhanu Pratap Singh Rawat, Pengshan Cai, and Hong Yu. 2019. Fine-tuning bidirectional encoder representations from transformers (bert)–based models on large-scale electronic health record notes: An empirical study. JMIR Med. Inf. 7, 3 (2019), e14830.

[147]

Jiao Li, Yueping Sun, Robin J. Johnson, Daniela Sciaky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J. Mattingly, Thomas C. Wiegers, and Zhiyong Lu. 2016. Biocreative v cdr task corpus: A resource for chemical disease relation extraction (unpublished).

[148]

Jianquan Li, Xidong Wang, Xiangbo Wu, Zhiyi Zhang, Xiaolong Xu, Jie Fu, Prayag Tiwari, Xiang Wan, and Benyou Wang. 2023. Huatuo-26M, a large-scale chinese medical QA dataset. arXiv:2305.01526. Retrieved from https://arxiv.org/abs/2305.01526

[149]

Junlong Li, Zhuosheng Zhang, Hai Zhao, Xi Zhou, and Xiang Zhou. 2020. Task-specific objectives of pre-trained language models for dialogue adaptation. arXiv:2009.04984. Retrieved from https://arxiv.org/abs/2009.04984

[150]

Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. 2019. Visualbert: A simple and performant baseline for vision and language. arXiv:1908.03557. Retrieved from https://arxiv.org/abs/1908.03557

[151]

Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. 2020. Federated learning: Challenges, methods, and future directions. IEEE Sign. Process. Mag. 37, 3 (2020), 50–60.

[152]

Yikuan Li, Shishir Rao, José Roberto Ayala Solares, Abdelaali Hassaine, Rema Ramakrishnan, Dexter Canoy, Yajie Zhu, Kazem Rahimi, and Gholamreza Salimi-Khorshidi. 2020. Behrt: Transformer for electronic health records. Sci. Rep. 10, 1 (2020), 1–12.

[153]

Yikuan Li, Hanyin Wang, and Yuan Luo. 2020. A comparison of pre-trained vision-and-language models for multimodal representation learning across medical images and reports. In BIBM. IEEE, 1999–2004.

[154]

Yikuan Li, Ramsey M. Wehbe, Faraz S. Ahmad, Hanyin Wang, and Yuan Luo. 2022. Clinical-longformer and clinical-BigBird: Transformers for long clinical sequences. arXiv:2201.11838. Retrieved from https://arxiv.org/abs/2201.11838

[155]

Yikuan Li, Ramsey M. Wehbe, Faraz S. Ahmad, Hanyin Wang, and Yuan Luo. 2023. A comparative study of pretrained language models for long clinical text. J. Am. Med. Inf. Assoc. 30, 2 (2023), 340–347.

[156]

Zihan Li, Yunxiang Li, Qingde Li, You Zhang, Puyang Wang, Dazhou Guo, Le Lu, Dakai Jin, and Qingqi Hong. 2022. LViT: Language meets vision transformer in medical image segmentation. arXiv:2206.14718. Retrieved from https://arxiv.org/abs/2206.14718

[157]

Ruizhi Liao, Daniel Moyer, Miriam Cha, Keegan Quigley, Seth Berkowitz, Steven Horng, Polina Golland, and William M. Wells. 2021. Multimodal representation learning via maximization of local mutual information. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 273–283.

Digital Library

[158]

Valentin Liévin, Christoffer Egeberg Hother, and Ole Winther. 2022. Can large language models reason about medical questions? arXiv:2207.08143. Retrieved from https://arxiv.org/abs/2207.08143

[159]

Bill Yuchen Lin, Seyeon Lee, Rahul Khanna, and Xiang Ren. 2020. Birds have four legs?! numersense: Probing numerical commonsense knowledge of pre-trained language models. arXiv:2005.00683. Retrieved from https://arxiv.org/abs/2005.00683

[160]

Chen Lin, Timothy Miller, Dmitriy Dligach, Steven Bethard, and Guergana Savova. 2019. A bert-based universal model for both within-and cross-sentence clinical temporal relation extraction. In Clinical Natural Language Processing Workshop. 65–71.

[161]

Chen Lin, Timothy Miller, Dmitriy Dligach, Farig Sadeque, Steven Bethard, and Guergana Savova. 2020. A bert-based one-pass multi-task model for clinical temporal relation extraction. In SIGBioMed Workshop on Biomedical Language Processing. 70–75.

[162]

Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out. 74–81.

[163]

Xinzhu Lin, Xiahui He, Qin Chen, Huaixiao Tou, Zhongyu Wei, and Ting Chen. 2019. Enhancing dialogue symptom diagnosis with global attention and symptom graph. In EMNLP-IJCNLP. 5033–5042.

[164]

Zachary C. Lipton. 2018. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue 16, 3 (2018), 31–57.

Digital Library

[165]

Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv:2107.13586. Retrieved from https://arxiv.org/abs/2107.13586

[166]

Qi Liu, Matt J. Kusner, and Phil Blunsom. 2020. A survey on contextual embeddings. arxiv:cs.CL/2003.07278. Retrieved from https://arxiv.org/abs/2003.07278

[167]

Wenge Liu, Jianheng Tang, Jinghui Qin, Lin Xu, Zhen Li, and Xiaodan Liang. 2020. Meddg: A large-scale medical consultation dataset for building medical dialogue system. arXiv:2010.07497. Retrieved from https://arxiv.org/abs/2010.07497

[168]

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv:1907.11692. Retrieved from https://arxiv.org/abs/1907.11692

[169]

Pilar López-Úbeda, Manuel Carlos Díaz-Galiano, L. Alfonso Ureña-López, and M Teresa Martín-Valdivia. 2021. Pre-trained language models to extract information from radiological reports. CLEF eHealth (2021).

[170]

Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. 2022. BioGPT: Generative pre-trained transformer for biomedical text generation and mining. Brief. Bioinf. 23, 6 (2022).

[171]

Zheheng Luo, Qianqian Xie, and Sophia Ananiadou. 2022. Readability controllable biomedical document summarization. arXiv:2210.04705. Retrieved from https://arxiv.org/abs/2210.04705

[172]

Zheheng Luo, Qianqian Xie, and Sophia Ananiadou. 2023. Chatgpt as a factual inconsistency evaluator for abstractive text summarization. arXiv:2303.15621. Retrieved from https://arxiv.org/abs/2303.15621

[173]

Zheheng Luo, Qianqian Xie, and Sophia Ananiadou. 2023. CitationSum: Citation-aware graph contrastive learning for scientific paper summarization. arXiv:2301.11223. Retrieved from https://arxiv.org/abs/2301.11223

[174]

Ali Madani, Bryan McCann, Nikhil Naik, Nitish Shirish Keskar, Namrata Anand, Raphael R. Eguchi, Po-Ssu Huang, and Richard Socher. 2020. Progen: Language modeling for protein generation. arXiv:2004.03497. Retrieved from https://arxiv.org/abs/2004.03497.

[175]

Tittaya Mairittha, Nattaya Mairittha, and Sozo Inoue. 2020. Improving fine-tuned question answering models for electronic health records. In UBICOMP. 688–691.

[176]

Aurelie Mascio, Zeljko Kraljevic, Daniel Bean, Richard Dobson, Robert Stewart, Rebecca Bendayan, and Angus Roberts. 2020. Comparative analysis of text classification approaches in electronic health records. In Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing. 86–94.

[177]

Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A survey on bias and fairness in machine learning. ACM Comput. Surv. 54, 6 (2021), 1–35.

Digital Library

[178]

Xing Meng, Craig H. Ganoe, Ryan T. Sieberg, Yvonne Y. Cheung, and Saeed Hassanpour. 2019. Self-supervised contextual language representation of radiology reports to improve the identification of communication urgency. arxiv:cs.LG/1912.02703. Retrieved from https://arxiv.org/abs/1912.02703

[179]

Yiwen Meng, William Farran Speier, Michael K. Ong, and Corey Arnold. 2021. Bidirectional representation learning from transformers using multimodal electronic health record data to predict depression. IEEE J. Biomed. Health Inf. (2021), 1–1. DOI:

[180]

Amil Merchant, Elahe Rahimtoroghi, Ellie Pavlick, and Ian Tenney. 2020. What happens to bert embeddings during fine-tuning? arXiv:2004.14448. Retrieved from https://arxiv.org/abs/2004.14448

[181]

George Michalopoulos, Yuanxin Wang, Hussam Kaka, Helen Chen, and Alex Wong. 2020. Umlsbert: Clinical domain knowledge augmentation of contextual embeddings using the unified medical language system metathesaurus. arXiv:2010.10391. Retrieved from https://arxiv.org/abs/2010.10391

[182]

Claudiu Mihăilă, Tomoko Ohta, Sampo Pyysalo, and Sophia Ananiadou. 2013. Biocause: Annotating and analysing causality in the biomedical domain. BMC Bioinf. 14, 1 (2013), 1–18.

[183]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv:1301.3781. Retrieved from https://arxiv.org/abs/1301.3781

[184]

T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean. 2013. Distributed representations of words and phrases and their compositionality. NeuIPS 26 (2013), 3111–3119.

[185]

Joshua R. Minot, Nicholas Cheney, Marc Maier, Danne C. Elbers, Christopher M. Danforth, and Peter Sheridan Dodds. 2021. Interpretable bias mitigation for textual data: reducing gender bias in patient notes while maintaining classification performance. arXiv:2103.05841. Retrieved from https://arxiv.org/abs/2103.05841

[186]

Giacomo Miolo, Giulio Mantoan, and Carlotta Orsenigo. 2021. Electramed: A new pre-trained language representation model for biomedical nlp. arxiv:cs.CL/2104.09585. Retrieved from https://arxiv.org/abs/2104.09585

[187]

A. Miranda-Escalada, E. Farré, and M. Krallinger. 2020. Named entity recognition, concept normalization and clinical coding: Overview of the cantemist track for cancer text mining in spanish, corpus, guidelines, methods and results. In IberLEF.

[188]

Florian Mock, Fleming Kretschmer, Anton Kriese, Sebastian Böcker, and Manja Marz. 2021. BERTax: Taxonomic classification of DNA sequences with Deep Neural Networks (unpublished).

[189]

Masoud Monajatipoor, Mozhdeh Rouhsedaghat, Liunian Harold Li, C.-C. Jay Kuo, Aichi Chien, and Kai-Wei Chang. 2022. Berthop: An effective vision-and-language model for chest x-ray disease diagnosis. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 725–734.

Digital Library

[190]

Jong Hak Moon, Hyungyung Lee, Woncheol Shin, Young-Hak Kim, and Edward Choi. 2022. Multi-modal understanding and generation for medical images and text via vision-language pre-training. IEEE J. Biomed. Health Inf. (2022).

[191]

Marius Mosbach, Maksym Andriushchenko, and Dietrich Klakow. 2020. On the stability of fine-tuning bert: misconceptions, explanations, and strong baselines. arXiv:2006.04884. Retrieved from https://arxiv.org/abs/2006.04884

[192]

John Moult, Jan T. Pedersen, Richard Judson, and Krzysztof Fidelis. 1995. A large-scale experiment to assess protein structure prediction methods.

[193]

Martin Müller, Marcel Salathé, and Per Egil Kummervold. 2020. COVID-Twitter-BERT: A natural language processing model to analyse Covid-19 content on twitter. arxiv:2005.07503. Retrieved from https://arxiv.org/abs/2005.07503

[194]

Martin M. Müller and Marcel Salathé. 2019. Crowdbreaks: Tracking health trends using public social media data and crowdsourcing. Front. Publ. Health 7 (2019), 81.

[195]

Philip Müller, Georgios Kaissis, and Daniel Rueckert. 2022. The role of local alignment and uniformity in image-text contrastive learning on medical images. arXiv:2211.07254. Retrieved from https://arxiv.org/abs/2211.07254

[196]

Philip Müller, Georgios Kaissis, Congyu Zou, and Daniel Rueckert. 2022. Joint learning of localized representations from medical images and reports. In European Conference on Computer Vision. Springer, 685–701.

Digital Library

[197]

Yuta Nakamura, Shouhei Hanaoka, Yukihiro Nomura, Naoto Hayashi, Osamu Abe, Shuntaro Yada, Shoko Wakamiya, and Eiji Aramaki. 2020. Kart: Privacy leakage framework of language models pre-trained with clinical records. arXiv:2101.00036. Retrieved from https://arxiv.org/abs/2101.00036

[198]

Usman Naseem, Matloob Khushi, Vinay Reddy, Sakthivel Rajendran, Imran Razzak, and Jinman Kim. 2020. Bioalbert: A simple and effective pre-trained language model for biomedical named entity recognition. arXiv:2009.09223. Retrieved from https://arxiv.org/abs/2009.09223

[199]

Anastasios Nentidis, Konstantinos Bougiatiotis, Anastasia Krithara, and Georgios Paliouras. 2019. Results of the seventh edition of the bioasq challenge. In ECML PKDD. Springer, 553–568.

[200]

Oded Nov, Nina Singh, and Devin Mann. 2023. Putting ChatGPT’s Medical Advice to the (Turing) Test. arXiv.2301.10035. Retrieved from https://arxiv.org/abs/2301.10035

[201]

Benjamin Nye, Junyi Jessy Li, Roma Patel, Yinfei Yang, Iain J. Marshall, Ani Nenkova, and Byron C. Wallace. 2018. A corpus with multi-level annotations of patients, interventions and outcomes to support language processing for medical literature. In ACL, Vol. 2018. NIH Public Access, 197.

[202]

Ibrahim Burak Ozyurt. 2020. On the effectiveness of small, discriminatively pre-trained language representation models for biomedical text mining. In Workshop on Scholarly Document Processing. 104–112.

[203]

Obioma Pelka, Sven Koitka, Johannes Rückert, Felix Nensa, and Christoph M. Friedrich. 2018. Radiology objects in COntext (ROCO): A multimodal image dataset. In Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis. Springer, 180–189.

[204]

Yifan Peng, Shankai Yan, and Zhiyong Lu. 2019. Transfer learning in biomedical natural language processing: An evaluation of bert and elmo on ten benchmarking datasets. In BioNLP Workshop and Shared Task. 58–65.

[205]

Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In EMNLP. 1532–1543.

[206]

Bethany Percha. 2021. Modern clinical text mining: A guide and review. Annu. Rev. Biomed. Data Sci. 4 (2021), 165–187.

[207]

Ethan Perez, Douwe Kiela, and Kyunghyun Cho. 2021. True few-shot learning with language models. arXiv:2105.11447. Retrieved from https://arxiv.org/abs/2105.11447

[208]

Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In NAACL. 2227–2237.

[209]

Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In EMNLP-IJCNLP. 2463–2473.

[210]

Long N. Phan, James T. Anibal, Hieu Tran, Shaurya Chanana, Erol Bahadroglu, Alec Peltekian, and Grégoire Altan-Bonnet. 2021. Scifive: A text-to-text transformer model for biomedical literature. arxiv:cs.CL/2106.03598. Retrieved from https://arxiv.org/abs/2106.03598

[211]

Nina Poerner, Ulli Waltinger, and Hinrich Schütze. 2020. Inexpensive domain adaptation of pretrained language models: Case studies on biomedical NER and Covid-19 QA. In EMNLP. 1482–1490.

[212]

Sampo Pyysalo, Tomoko Ohta, Rafal Rak, Andrew Rowley, Hong-Woo Chun, Sung-Jae Jung, Sung-Pil Choi, Jun’ichi Tsujii, and Sophia Ananiadou. 2015. Overview of the cancer genetics and pathway curation tasks of bionlp shared task 2013. BMC Bioinf. 16, 10 (2015), 1–19.

[213]

Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. 2020. Pre-trained models for natural language processing: A survey. Sci. Chin. Technol. Sci. (2020), 1–26.

[214]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning. PMLR, 8748–8763.

[215]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. arXiv:2103.00020. Retrieved from https://arxiv.org/abs/2103.00020

[216]

Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving Language Understanding by Generative Pre-training. OpenAI Technical Report. (2018).

[217]

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. arxiv:cs.LG/1910.10683. Retrieved from https://arxiv.org/abs/1910.10683

[218]

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv:1606.05250. Retrieved from https://arxiv.org/abs/1606.05250

[219]

Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical text-conditional image generation with clip latents. arXiv:2204.06125. Retrieved from https://arxiv.org/abs/2204.06125

[220]

Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. Zero-shot text-to-image generation. arxiv:cs.CV/2102.12092. Retrieved from https://arxiv.org/abs/2102.12092

[221]

Alan Ramponi, Rob van der Goot, Rosario Lombardo, and Barbara Plank. 2020. Biomedical event extraction as sequence labeling. In EMNLP. 5357–5367.

[222]

Roshan Rao, Nicholas Bhattacharya, Neil Thomas, Yan Duan, Xi Chen, John Canny, Pieter Abbeel, and Yun S. Song. 2019. Evaluating protein transfer learning with tape. NeuIPS 32 (2019), 9689.

[223]

Roshan Rao, Jason Liu, Robert Verkuil, Joshua Meier, John F. Canny, Pieter Abbeel, Tom Sercu, and Alexander Rives. 2021. Msa transformer (unpublished).

[224]

Laila Rasmy, Yang Xiang, Ziqian Xie, Cui Tao, and Degui Zhi. 2020. Med-bert: Pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction. arxiv:2005.12833. Retrieved from https://arxiv.org/abs/2005.12833

[225]

Bhanu Pratap Singh Rawat, Wei-Hung Weng, So Yeon Min, Preethi Raghavan, and Peter Szolovits. 2020. Entity-enriched neural models for clinical question answering. In SIGBioMed Workshop on Biomedical Language Processing. 112–122.

[226]

Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C. Lawrence Zitnick, Jerry Ma, et al. 2021. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proc. Natl Acad. Sci. U.S.A. 118, 15 (2021).

[227]

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In CVPR. 10684–10695.

[228]

Subendhu Rongali, Abhyuday Jagannatha, Bhanu Pratap Singh Rawat, and Hong Yu. 2021. Continual domain-tuning for pretrained language models. arxiv:cs.CL/2004.02288. Retrieved from https://arxiv.org/abs/2004.02288

[229]

Frank Rudzicz and Raeid Saqur. 2020. Ethics of artificial intelligence in surgery. arXiv:2007.14302. Retrieved from https://arxiv.org/abs/2007.14302

[230]

Devendra Singh Sachan, Pengtao Xie, Mrinmaya Sachan, and Eric P. Xing. 2018. Effective use of bidirectional language modeling for transfer learning in biomedical named entity recognition. In Machine Learning for Healthcare Conference. PMLR, 383–402.

[231]

Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, et al. 2022. Photorealistic text-to-image diffusion models with deep language understanding. arXiv:2205.11487. Retrieved from https://arxiv.org/abs/2205.11487

[232]

Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2020. Distilbert, a distilled version of bert: Smaller, faster, cheaper and lighter. arxiv:cs.CL/1910.01108. Retrieved from https://arxiv.org/abs/1910.01108

[233]

Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv:2211.05100. Retrieved from https://arxiv.org/abs/2211.05100

[234]

Elisa Terumi Rubel Schneider, Joao Vitor Andrioli de Souza, Yohan Bonescki Gumiel, Claudia Moro, and Emerson Cabrera Paraiso. 2021. A gpt-2 language model for biomedical texts in portuguese. In CBMS. IEEE, 474–479.

[235]

Elisa Terumi Rubel Schneider, João Vitor Andrioli de Souza, Julien Knafou, Lucas Emanuel Silva e Oliveira, Jenny Copara, Yohan Bonescki Gumiel, Lucas Ferro Antunes de Oliveira, Emerson Cabrera Paraiso, Douglas Teodoro, and Cláudia Maria Cabral Moro Barra. 2020. BioBERTpt - A Portuguese neural language model for clinical named entity recognition. In Proceedings of the 3rd Clinical Natural Language Processing Workshop. Association for Computational Linguistics, 65–72. DOI:

[236]

Constantin Seibold, Simon Reiß, M. Saquib Sarfraz, Rainer Stiefelhagen, and Jens Kleesiek. 2022. Breaking with fixed set pathology recognition through report-guided contrastive training. arXiv:2205.07139. Retrieved from https://arxiv.org/abs/2205.07139

[237]

Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In ACL. Association for Computational Linguistics, 1715–1725.

[238]

Junyuan Shang, Tengfei Ma, Cao Xiao, and Jimeng Sun. 2019. Pre-training of graph augmented transformers for medication recommendation. arXiv:1906.00346. Retrieved from https://arxiv.org/abs/1906.00346

[239]

Shreyas Sharma and Ron Daniel Jr au2. 2019. Bioflair: Pretrained pooled contextualized embeddings for biomedical sequence labeling tasks. arxiv:cs.CL/1908.05760. Retrieved from https://arxiv.org/abs/1908.05760.

[240]

Soumya Sharma, Bishal Santra, Abhik Jana, Santosh Tokala, Niloy Ganguly, and Pawan Goyal. 2019. Incorporating domain knowledge into medical NLI using knowledge graphs. In EMNLP-IJCNLP. 6092–6097.

[241]

Golnar Sheikhshab, Inanc Birol, and Anoop Sarkar. 2018. In-domain context-aware token embeddings improve biomedical named entity recognition. In Workshop on Health Text Mining and Information Analysis. 160–164.

[242]

Xiaoming Shi, Haifeng Hu, Wanxiang Che, Zhongqian Sun, Ting Liu, and Junzhou Huang. 2020. Understanding medical conversations with scattered keyword attention and weak supervision from responses. In AAAI, Vol. 34. 8838–8845.

[243]

Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi, and Raghav Mani. 2020. Biomegatron: Larger biomedical domain language model. In EMNLP. 4700–4706.

[244]

Yuqi Si, Jingqi Wang, Hua Xu, and Kirk Roberts. 2019. Enhancing clinical concept extraction with contextual embeddings. J. Am. Med. Inf. Assoc. 26, 11 (2019), 1297–1304.

[245]

Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. 2022. Large language models encode clinical knowledge. arXiv:2212.13138. Retrieved from https://arxiv.org/abs/2212.13138

[246]

Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Le Hou, Kevin Clark, Stephen Pfohl, Heather Cole-Lewis, Darlene Neal, et al. 2023. Towards expert-level medical question answering with large language models. arXiv:2305.09617. Retrieved from https://arxiv.org/abs/2305.09617

[247]

Larry Smith, Lorraine K. Tanabe, Rie Johnson nee Ando, Cheng-Ju Kuo, I-Fang Chung, Chun-Nan Hsu, Yu-Shi Lin, Roman Klinger, Christoph M. Friedrich, Kuzman Ganchev, et al. 2008. Overview of biocreative ii gene mention recognition. Genome Biol. 9, 2 (2008), 1–19.

[248]

Gizem Soğancıoğlu, Hakime Öztürk, and Arzucan Özgür. 2017. Biosses: A semantic sentence similarity estimation system for the biomedical domain. Bioinformatics 33, 14 (2017), i49–i58.

[249]

Jose Roberto Ayala Solares, Francesca Elisa Diletta Raimondi, Yajie Zhu, Fatemeh Rahimian, Dexter Canoy, Jenny Tran, Ana Catarina Pinho Gomes, Amir H. Payberah, Mariagrazia Zottoli, Milad Nazarzadeh, et al. 2020. Deep learning for electronic health records: A comparative review of multiple deep neural architectures. J. Biomed. Inf. 101 (2020), 103337.

Digital Library

[250]

Sarvesh Soni and Kirk Roberts. 2020. Evaluation of dataset selection for pre-training and fine-tuning transformer language models for clinical question answering. In LREC. 5532–5538.

[251]

Peter Spyns. 1996. Natural language processing in medicine: An overview. Methods Inf. Med. 35, 4-5 (1996), 285–301.

[252]

Pascal Sturmfels, Jesse Vig, Ali Madani, and Nazneen Fatema Rajani. 2020. Profile Prediction: An alignment-based pre-training task for protein sequence models. arXiv:2012.00195. Retrieved from https://arxiv.org/abs/2012.00195

[253]

Dan Su, Yan Xu, Tiezheng Yu, Farhad Bin Siddique, Elham Barezi, and Pascale Fung. 2020. CAiRE-COVID: A question answering and query-focused multi-document summarization system for COVID-19 scholarly information management. In Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020.

[254]

Peng Su, Yifan Peng, and K. Vijay-Shanker. 2021. Improving BERT model using contrastive learning for biomedical relation extraction. In Proceedings of the 20th Workshop on Biomedical Language Processing. 1–10.

[255]

Peng Su and K. Vijay-Shanker. 2020. Investigation of bert model on biomedical relation extraction based on revised fine-tuning mechanism. In BIBM. IEEE, 2522–2529.

[256]

Sanjay Subramanian, Lucy Lu Wang, Ben Bogin, Sachin Mehta, Madeleine van Zuylen, Sravanthi Parasa, Sameer Singh, Matt Gardner, and Hannaneh Hajishirzi. 2020. MedICaT: A dataset of medical images, captions, and textual references. In EMNLP. 2112–2120.

[257]

Dianbo Sui, Yubo Chen, Jun Zhao, Yantao Jia, Yuantao Xie, and Weijian Sun. 2020. Feded: Federated learning via ensemble distillation for medical relation extraction. In EMNLP. 2118–2128.

[258]

Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019. How to fine-tune bert for text classification? In China National Conference on Chinese Computational Linguistics. Springer, 194–206.

Digital Library

[259]

Cong Sun, Zhihao Yang, Lei Wang, Yin Zhang, Hongfei Lin, and Jian Wang. 2020. Biomedical named entity recognition using bert in the machine reading comprehension framework. arXiv:2009.01560. Retrieved from https://arxiv.org/abs/2009.01560

[260]

Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. arXiv:1906.08976. Retrieved from https://arxiv.org/abs/1906.08976

[261]

Hao Tan and Mohit Bansal. 2019. LXMERT: Learning cross-modality encoder representations from transformers. In EMNLP-IJCNLP. 5100–5111.

[262]

Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. 2020. Efficient transformers: A survey. arXiv:2009.06732. Retrieved from https://arxiv.org/abs/2009.06732

[263]

Ashok Thillaisundaram and Theodosia Togia. 2019. Biomedical relation extraction with pre-trained language representations and minimal task-specific architecture. In Workshop on BioNLP Open Shared Tasks. 84–89.

[264]

Robert Tinn, Hao Cheng, Yu Gu, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Fine-tuning large neural language models for biomedical natural language processing. arXiv:2112.07869. Retrieved from https://arxiv.org/abs/2112.07869

[265]

Ekin Tiu, Ellie Talius, Pujan Patel, Curtis P. Langlotz, Andrew Y. Ng, and Pranav Rajpurkar. 2022. Expert-level detection of pathologies from unannotated chest X-ray images via self-supervised learning. Nat. Biomed. Eng. (2022), 1–8.

[266]

Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNll-2003 shared task: language-independent named entity recognition. In NAACL-HLT. 142–147.

[267]

Yiqi Tong, Yidong Chen, and Xiaodong Shi. 2021. A multi-task approach for improving biomedical named entity recognition by incorporating multi-granularity information. In ACL-IJCNLP. 4804–4813.

[268]

Hai-Long Trieu, Thy Thy Tran, Khoa N. A. Duong, Anh Nguyen, Makoto Miwa, and Sophia Ananiadou. 2020. Deepeventmine: End-to-end neural nested event extraction from biomedical texts. Bioinformatics 36, 19 (2020), 4910–4917.

[269]

Ryan Turner, David Eriksson, Michael McCourt, Juha Kiili, Eero Laaksonen, Zhen Xu, and Isabelle Guyon. 2021. Bayesian optimization is superior to random search for machine learning hyperparameter tuning: Analysis of the black-box optimization challenge 2020. arXiv:2104.10201. Retrieved from https://arxiv.org/abs/2104.10201

[270]

Elena Tutubalina, Ilseyar Alimova, Zulfat Miftahutdinov, Andrey Sakhovskiy, Valentin Malykh, and Sergey Nikolenko. 2021. The russian drug reaction corpus and neural models for drug reactions and effectiveness detection in user reviews. Bioinformatics 37, 2 (2021), 243–249.

[271]

Özlem Uzuner, Brett R. South, Shuying Shen, and Scott L. DuVall. 2011. 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. J. Am. Med. Inf. Assoc. 18, 5 (2011), 552–556.

[272]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS. 5998–6008.

[273]

Jesse Vig, Ali Madani, Lav R. Varshney, Caiming Xiong, Richard Socher, and Nazneen Fatema Rajani. 2021. Bertology meets biology: Interpreting attention in protein language models. arxiv:cs.CL/2006.15222. Retrieved from https://arxiv.org/abs/2006.15222.

[274]

Ivan Vulić, Edoardo Maria Ponti, Robert Litschko, Goran Glavaš, and Anna Korhonen. 2020. Probing pretrained language models for lexical semantics. arXiv:2010.05731. Retrieved from https://arxiv.org/abs/2010.05731

[275]

Shoya Wada, Toshihiro Takeda, Shiro Manabe, Shozo Konishi, Jun Kamohara, and Yasushi Matsumura. 2020. Pre-training technique to localize medical bert and enhance biomedical bert. arXiv:2005.07202. Retrieved from https://arxiv.org/abs/2005.07202

[276]

David Wadden, Ulme Wennberg, Yi Luan, and Hannaneh Hajishirzi. 2019. Entity, relation, and event extraction with contextualized span representations. In EMNLP-IJCNLP. 5788–5793.

[277]

Byron C. Wallace, Sayantan Saha, Frank Soboczenski, and Iain J. Marshall. 2020. Generating (factual?) narrative summaries of rcts: Experiments with neural multi-document summarization. arXiv:2008.11293. Retrieved from https://arxiv.org/abs/2008.11293

[278]

Zhongwei Wan, Che Liu, Mi Zhang, Jie Fu, Benyou Wang, Sibo Cheng, Lei Ma, César Quilodrán-Casas, and Rossella Arcucci. 2023. Med-UniC: Unifying cross-lingual medical vision-language pre-training by diminishing bias. arXiv:2305.19894. Retrieved from https://arxiv.org/abs/2305.19894

[279]

Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2018. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv:1804.07461. Retrieved from https://arxiv.org/abs/1804.07461

[280]

Benyou Wang, Lifeng Shang, Christina Lioma, Xin Jiang, Hao Yang, Qun Liu, and Jakob Grue Simonsen. 2021. On position embeddings in bert. In ICLR, Vol. 2. 12–13.

[281]

Benyou Wang, Donghao Zhao, Christina Lioma, Qiuchi Li, Peng Zhang, and Jakob Grue Simonsen. 2019. Encoding word order in complex embeddings. In ICLR 2020 Spotlight.

[282]

Fuying Wang, Yuyin Zhou, Shujun Wang, Varut Vardhanabhuti, and Lequan Yu. 2022. Multi-granularity cross-modal alignment for generalized medical visual representation learning. In Advances in Neural Information Processing Systems.

[283]

Xiaosong Wang, Ziyue Xu, Leo Tam, Dong Yang, and Daguang Xu. 2021. Self-supervised image-text pre-training with mixed data in chest x-rays. arXiv:2103.16022. Retrieved from https://arxiv.org/abs/2103.16022

[284]

Xing David Wang, Leon Weber, and Ulf Leser. 2020. Biomedical event extraction as multi-turn question answering. In ACL Workshop on Health Text Mining and Information Analysis. 88–96.

[285]

Yanshan Wang, Liwei Wang, Majid Rastegar-Mojarad, Sungrim Moon, Feichen Shen, Naveed Afzal, Sijia Liu, Yuqun Zeng, Saeed Mehrabi, Sunghwan Sohn, et al. 2018. Clinical information extraction applications: A literature review. J. Biomed. Inf. 77 (2018), 34–49.

[286]

Yu-An Wang and Yun-Nung Chen. 2020. What do position embeddings learn? an empirical study of pre-trained language model positional encoding. In EMNLP. 6840–6849.

[287]

Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, and Jimeng Sun. 2022. MedCLIP: Contrastive learning from unpaired medical images and text. arXiv:2210.10163. Retrieved from https://arxiv.org/abs/2210.10163

[288]

Neha Warikoo, Yung-Chun Chang, and Wen-Lian Hsu. 2021. Lbert: Lexically aware transformer-based bidirectional encoder representation model for learning universal bio-entity relations. Bioinformatics 37, 3 (2021), 404–412.

[289]

Qiang Wei, Zongcheng Ji, Yuqi Si, Jingcheng Du, Jingqi Wang, Firat Tiryaki, Stephen Wu, Cui Tao, Kirk Roberts, and Hua Xu. 2019. Relation extraction from clinical narratives using pre-trained language models. In AMIA Annual Symposium Proceedings, Vol. 2019. American Medical Informatics Association, 1236.

[290]

Zhongyu Wei, Qianlong Liu, Baolin Peng, Huaixiao Tou, Ting Chen, Xuan-Jing Huang, Kam-Fai Wong, and Xiang Dai. 2018. Task-oriented dialogue system for automatic diagnosis. In ACL. 201–207.

[291]

Wei-Hung Weng and Peter Szolovits. 2019. Representation learning for electronic health records. arXiv:1909.09248. Retrieved from https://arxiv.org/abs/1909.09248

[292]

Taesun Whang, Dongyub Lee, Chanhee Lee, Kisu Yang, Dongsuk Oh, and Heuiseok Lim. 2020. An effective domain adaptive post-training method for bert in response selection. In INTERSPEECH. 1585–1589.

[293]

Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In NAACL-HLT. Association for Computational Linguistics, 1112–1122.

[294]

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Huggingface’s transformers: State-of-the-art natural language processing. arxiv:cs.CL/1910.03771. Retrieved from https://arxiv.org/abs/1910.03771

[295]

Stephen Wu, Kirk Roberts, Surabhi Datta, Jingcheng Du, Zongcheng Ji, Yuqi Si, Sarvesh Soni, Qiong Wang, Qiang Wei, Yang Xiang, et al. 2020. Deep learning in clinical natural language processing: A methodical review. J. Am. Med. Inf. Assoc. 27, 3 (2020), 457–470.

[296]

Zhiyong Wu, Yun Chen, Ben Kao, and Qun Liu. 2020. Perturbed masking: Parameter-free probing for analyzing and interpreting bert. In ACL. 4166–4176.

[297]

Zhaofeng Wu, Yan Song, Sicong Huang, Yuanhe Tian, and Fei Xia. 2019. WTMED at MEDIQA 2019: A hybrid approach to biomedical natural language inference. In Proceedings of the 18th BioNLP Workshop and Shared Task. 415–426.

[298]

Yuan Xia, Jingbo Zhou, Zhenhui Shi, Chao Lu, and Haifeng Huang. 2020. Generative adversarial regularized mutual information policy gradient framework for automatic diagnosis. In AAAI, Vol. 34. 1062–1069.

[299]

Yijia Xiao, Jiezhong Qiu, Ziang Li, Chang-Yu Hsieh, and Jie Tang. 2021. Modeling protein using large-scale pretrain language model. arXiv:2108.07435. Retrieved from https://arxiv.org/abs/2108.07435

[300]

Qianqian Xie, Jennifer Amy Bishop, Prayag Tiwari, and Sophia Ananiadou. 2022. Pre-trained language models with domain knowledge for biomedical extractive summarization. Knowl.-Bas. Syst. 252 (2022), 109460.

Digital Library

[301]

Qianqian Xie, Jinpeng Hu, Jiayu Zhou, Yifan Peng, and Fei Wang. 2023. Factreranker: Fact-guided reranker for faithful radiology report summarization. arXiv:2303.08335. Retrieved from https://arxiv.org/abs/2303.08335

[302]

Qianqian Xie, Jimin Huang, Tulika Saha, and Sophia Ananiadou. 2022. GRETEL: Graph contrastive topic enhanced language model for long document extractive summarization. In COLING. 6259–6269.

[303]

Qianqian Xie, Zheheng Luo, Benyou Wang, and Sophia Ananiadou. 2023. A survey on biomedical text summarization with pre-trained language model. arXiv:2304.08763. Retrieved from https://arxiv.org/abs/2304.08763

[304]

Qianqian Xie and Fei Wang. 2023. Faithful AI in healthcare and medicine (unpublished).

[305]

Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. 2022. Simmim: A simple framework for masked image modeling. In CVPR. 9653–9663.

[306]

Lin Xu, Qixian Zhou, Ke Gong, Xiaodan Liang, Jianheng Tang, and Liang Lin. 2019. End-to-end knowledge-routed relational dialogue system for automatic diagnosis. In AAAI, Vol. 33. 7346–7353.

[307]

Song Xu, Haoran Li, Peng Yuan, Yujia Wang, Youzheng Wu, Xiaodong He, Ying Liu, and Bowen Zhou. 2021. K-plug: Knowledge-injected pre-trained language model for natural language understanding and generation in e-commerce. arXiv:2104.06960. Retrieved from https://arxiv.org/abs/2104.06960

[308]

Kui Xue, Yangming Zhou, Zhiyuan Ma, Tong Ruan, Huanhuan Zhang, and Ping He. 2019. Fine-tuning bert for joint entity and relation extraction in chinese medical text. In BIBM. IEEE, 892–897.

[309]

Pranjul Yadav, Michael Steinbach, Vipin Kumar, and Gyorgy Simon. 2018. Mining electronic health records (ehrs) a survey. ACM Comput. Surv. 50, 6 (2018), 1–40.

Digital Library

[310]

Shweta Yadav, Vishal Pallagani, and Amit Sheth. 2020. Medical knowledge-enriched textual entailment framework. In ICCL. 1795–1801.

[311]

Keisuke Yamada and Michiaki Hamada. 2022. Prediction of RNA–protein interactions using a nucleotide language model. Bioinformatics Advances 2, 1 (2022), vbac023.

[312]

Guojun Yan, Jiahuan Pei, Pengjie Ren, Zhaochun Ren, Xin Xin, Huasheng Liang, Maarten de Rijke, and Zhumin Chen. 2022. ReMeDi: Resources for multi-domain, multi-service, medical dialogues. In ACM SIGIR. 3013–3024.

[313]

Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. 2019. Federated machine learning: Concept and applications. ACM Trans. Intell. Syst. Technol. 10, 2 (2019), 1–19.

Digital Library

[314]

Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. 2019. Federated machine learning: Concept and applications. ACM Trans. Intell. Syst. Technol. 10, 2, Article 12 (Jan2019), 19 pages. DOI:

Digital Library

[315]

Xi Yang, Jiang Bian, William R. Hogan, and Yonghui Wu. 2020. Clinical concept extraction using transformers. J. Am. Med. Inf. Assoc. (102020). DOI:

[316]

Xi Yang, Xing He, Hansi Zhang, Yinghan Ma, Jiang Bian, and Yonghui Wu. 2020. Measurement of semantic textual similarity in clinical texts: Comparison of transformer-based models. JMIR Med. Inf. 8, 11 (2020), e19735.

[317]

Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2020. Xlnet: Generalized autoregressive pretraining for language understanding. arxiv:cs.CL/1906.08237. Retrieved from https://arxiv.org/abs/1906.08237

[318]

Michihiro Yasunaga, Jure Leskovec, and Percy Liang. 2022. LinkBERT: Pretraining language models with document links. In ACL (Volume 1: Long Papers). 8003–8016.

[319]

Wonjin Yoon, Richard Jackson, Jaewoo Kang, and Aron Lagerberg. 2021. Sequence tagging for biomedical extractive question answering. arXiv:2104.07535. Retrieved from https://arxiv.org/abs/2104.07535

[320]

Wonjin Yoon, Jinhyuk Lee, Donghyeon Kim, Minbyul Jeong, and Jaewoo Kang. 2019. Pre-trained language model for biomedical question answering. In ECML PKDD. Springer, 727–740.

[321]

Fei Yu, Hongbo Zhang, and Benyou Wang. 2023. Nature language reasoning, a survey. arXiv:2303.14725 (2023).

[322]

Xin Yu, Wenshen Hu, Sha Lu, Xiaoyan Sun, and Zhenming Yuan. 2019. Biobert based named entity recognition in electronic medical record. In ITME. IEEE, 49–52.

[323]

Hongyi Yuan, Zheng Yuan, Ruyi Gan, Jiaxing Zhang, Yutao Xie, and Sheng Yu. 2022. BioBART: Pretraining and evaluation of a biomedical generative language model. BioNLP 2022@ ACL 2022, 97.

[324]

Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. Bartscore: Evaluating generated text as text generation. Adv. Neural Inf. Process. Syst. 34 (2021), 27263–27277.

[325]

Zheng Yuan, Zhengyun Zhao, and Sheng Yu. 2020. Coder: Knowledge infused cross-lingual medical term embedding for term normalization. arXiv:2011.02947. Retrieved from https://arxiv.org/abs/2011.02947

[326]

Manzil Zaheer, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al. 2020. Big bird: Transformers for longer sequences. arXiv:2007.14062. Retrieved from https://arxiv.org/abs/2007.14062

[327]

Guangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang, Sicheng Wang, Ruisi Zhang, Meng Zhou, Jiaqi Zeng, Xiangyu Dong, Ruoyu Zhang, et al. 2020. Meddialog: A large-scale medical dialogue dataset. In EMNLP. 9241–9250.

[328]

Zhiqiang Zeng, Hua Shi, Yun Wu, and Zhiling Hong. 2015. Survey of natural language processing techniques in bioinformatics. Comput. Math. Methods Med. 2015 (2015).

[329]

Hongbo Zhang, Junying Chen, Feng Jiang, Fei Yu, Zhihong Chen, Jianquan Li, Guiming Chen, Xiangbo Wu, Zhiyi Zhang, Qingying Xiao, Xiang Wan, Benyou Wang, and Haizhou Li. 2023. HuatuoGPT, towards taming language models to be a doctor. arXiv:2305.15075. Retrieved from https://arxiv.org/abs/2305.15075

[330]

Haoran Zhang, Amy X. Lu, Mohamed Abdalla, Matthew McDermott, and Marzyeh Ghassemi. 2020. Hurtful words: Quantifying biases in clinical contextual word embeddings. In CHIL. 110–120.

[331]

Hongbo Zhang, Xiang Wan, and Benyou Wang. 2023. Injecting knowledge into biomedical pre-trained models via polymorphism and synonymous substitution. arXiv:2305.15010. Retrieved from https://arxiv.org/abs/2305.15010

[332]

Ningyu Zhang, Zhen Bi, Xiaozhuan Liang, Lei Li, Xiang Chen, Shumin Deng, Luoqiu Li, Xin Xie, Hongbin Ye, Xin Shang, Kangping Yin, Chuanqi Tan, Jian Xu, Mosha Chen, Fei Huang, Luo Si, Yuan Ni, Guotong Xie, Zhifang Sui, Baobao Chang, Hui Zong, Zheng Yuan, Linfeng Li, Jun Yan, Hongying Zan, Kunli Zhang, Huajun Chen, Buzhou Tang, and Qingcai Chen. 2021. Cblue: A chinese biomedical language understanding evaluation benchmark. arxiv:cs.CL/2106.08087. Retrieved from https://arxiv.org/abs/2106.08087

[333]

Ningyu Zhang, Qianghuai Jia, Kangping Yin, Liang Dong, Feng Gao, and Nengwei Hua. 2020. Conceptualized representation learning for chinese biomedical text mining. arXiv:2008.10813. Retrieved from https://arxiv.org/abs/2008.10813

[334]

Rong Zhang, Revanth Gangi Reddy, Md Arafat Sultan, Vittorio Castelli, Anthony Ferritto, Radu Florian, Efsun Sarioglu Kayi, Salim Roukos, Avirup Sil, and Todd Ward. 2020. Multi-stage pre-training for low-resource domain adaptation. arxiv:cs.CL/2010.05904. Retrieved from https://arxiv.org/abs/2010.05904

[335]

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. BERTScore: Evaluating text generation with BERT. In ICLR.

[336]

Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, and Qun Liu. 2020. Ternarybert: Distillation-aware ultra-low bit bert. arxiv:cs.CL/2009.12812. Retrieved from https://arxiv.org/abs/2009.12812

[337]

Yuhao Zhang, Hang Jiang, Yasuhide Miura, Christopher D. Manning, and Curtis P. Langlotz. 2020. Contrastive learning of medical visual representations from paired images and text. arXiv:2010.00747. Retrieved from https://arxiv.org/abs/2010.00747

[338]

Yuanzhe Zhang, Zhongtao Jiang, Tao Zhang, Shiwan Liu, Jiarun Cao, Kang Liu, Shengping Liu, and Jun Zhao. 2020. Mie: A medical information extractor towards medical dialogues. In ACL. 6460–6469.

[339]

Yuhao Zhang, Derek Merck, Emily Tsai, Christopher D. Manning, and Curtis Langlotz. 2020. Optimizing the factual correctness of a summary: A study of summarizing radiology reports. In ACL. 5108–5120.

[340]

Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and William B. Dolan. 2020. DIALOGPT: Large-scale generative pre-training for conversational response generation. In ACL: System Demonstrations. 270–278.

[341]

Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. arXiv:1707.09457. Retrieved from https://arxiv.org/abs/1707.09457

[342]

Huiwei Zhou, Xuefei Li, Weihong Yao, Chengkun Lang, and Shixian Ning. 2019. Dut-nlp at mediqa 2019: An adversarial multi-task network to jointly model recognizing question entailment and question answering. In BioNLP Workshop and Shared Task. 437–445.

[343]

Henghui Zhu, Ioannis C. Paschalidis, and Amir M. Tahmasebi. 2018. Clinical concept extraction with contextual word embedding. In NeuIPS Workshop on Machine Learning for Health.

[344]

Wei Zhu, Yuan Ni, Xiaoling Wang, and Guotong Xie. 2021. Discovering better model architectures for medical query understanding. In NAACL-HLT. 230–237.

[345]

Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In ICCV. 19–27.

[346]

Konstantin Weißenow, Michael Heinzinger, and Burkhard Rost. 2021. Protein language model embeddings for fast, accurate, alignment-free protein structure prediction. bioRxiv (2021).

Cited By

Joshi H(2024)Natural Language Processing of Electronic Health Records for Predicting Alzheimer's DiseaseDeep Generative Models for Integrative Analysis of Alzheimer's Biomarkers10.4018/979-8-3693-6442-0.ch006(141-174)Online publication date: 25-Oct-2024
https://doi.org/10.4018/979-8-3693-6442-0.ch006
Матвеева ААнтонов Е(2024)МЕТОДИКА ФОРМИРОВАНИЯ БАЗЫ ДАННЫХ ХАРАКТЕРИСТИК СЛОЖНОГО ТЕХНОЛОГИЧЕСКОГО ОБЪЕКТА С ИСПОЛЬЗОВАНИЕМ БОЛЬШИХ ЯЗЫКОВЫХ МОДЕЛЕЙВестник НИЯУ МИФИ10.26583/vestnik.2024.5.713:5(351-357)Online publication date: 5-Nov-2024
https://doi.org/10.26583/vestnik.2024.5.7
Sprint GSchmitter-Edgecombe MCook D(2024)Building a Human Digital Twin (HDTwin) Using Large Language Models for Cognitive Diagnosis: Algorithm Development and ValidationJMIR Formative Research10.2196/638668(e63866)Online publication date: 23-Dec-2024
https://doi.org/10.2196/63866
Show More Cited By

Index Terms

Pre-trained Language Models in Biomedical Domain: A Systematic Survey
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Natural language generation
  2. Machine learning
    1. Machine learning approaches
      1. Bio-inspired approaches
      2. Neural networks

Recommendations

A Survey of Knowledge Enhanced Pre-trained Language Models
Pre-trained language models learn informative word representations on a large-scale text corpus through self-supervised learning, which has achieved promising performance in fields of natural language processing (NLP) after fine-tuning. These models, ...
Pre-Trained Language Models for Text Generation: A Survey
Text Generation aims to produce plausible and readable text in human language from input data. The resurgence of deep learning has greatly advanced this field, in particular, with the help of neural generation models based on pre-trained language models (...
Semantics-aware open information extraction in the biomedical domain
SWAT4LS '11: Proceedings of the 4th International Workshop on Semantic Web Applications and Tools for the Life Sciences

The increasing amount of biomedical scientific literature published on the Web is demanding new tools and methods to automatically process and extract relevant information. Traditional information extraction has focused on recognizing well-defined ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 56, Issue 3

March 2024

977 pages

EISSN:1557-7341

DOI:10.1145/3613568

Editor:
Albert Zomaya
University of Sydney, Australia

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 October 2023

Online AM: 01 August 2023

Accepted: 22 June 2023

Revised: 15 June 2023

Received: 28 October 2021

Published in CSUR Volume 56, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Survey

Funding Sources

Chinese Key-Area Research and Development Program of Guangdong Province
Shenzhen Science and Technology Program
Guangdong Provincial Key Laboratory of Big Data Computing, The Chinese University of Hong Kong, Shenzhen, Shenzhen Key Research Project
Shenzhen Doctoral Startup Funding

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
4,915
Total Downloads

Downloads (Last 12 months)3,518
Downloads (Last 6 weeks)240

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Joshi H(2024)Natural Language Processing of Electronic Health Records for Predicting Alzheimer's DiseaseDeep Generative Models for Integrative Analysis of Alzheimer's Biomarkers10.4018/979-8-3693-6442-0.ch006(141-174)Online publication date: 25-Oct-2024
https://doi.org/10.4018/979-8-3693-6442-0.ch006
Матвеева ААнтонов Е(2024)МЕТОДИКА ФОРМИРОВАНИЯ БАЗЫ ДАННЫХ ХАРАКТЕРИСТИК СЛОЖНОГО ТЕХНОЛОГИЧЕСКОГО ОБЪЕКТА С ИСПОЛЬЗОВАНИЕМ БОЛЬШИХ ЯЗЫКОВЫХ МОДЕЛЕЙВестник НИЯУ МИФИ10.26583/vestnik.2024.5.713:5(351-357)Online publication date: 5-Nov-2024
https://doi.org/10.26583/vestnik.2024.5.7
Sprint GSchmitter-Edgecombe MCook D(2024)Building a Human Digital Twin (HDTwin) Using Large Language Models for Cognitive Diagnosis: Algorithm Development and ValidationJMIR Formative Research10.2196/638668(e63866)Online publication date: 23-Dec-2024
https://doi.org/10.2196/63866
Chang ESung S(2024)Use of SNOMED CT in Large Language Models: Scoping ReviewJMIR Medical Informatics10.2196/6292412(e62924)Online publication date: 7-Oct-2024
https://doi.org/10.2196/62924
Nunes MBone JFerreira JElvas L(2024)Health Care Language Models and Their Fine-Tuning for Information Extraction: Scoping ReviewJMIR Medical Informatics10.2196/6016412(e60164)Online publication date: 21-Oct-2024
https://doi.org/10.2196/60164
Matsui KUtsumi TAoki YMaruki TTakeshima MTakaesu Y(2024)Human-Comparable Sensitivity of Large Language Models in Identifying Eligible Studies Through Title and Abstract Screening: 3-Layer Strategy Using GPT-3.5 and GPT-4 for Systematic ReviewsJournal of Medical Internet Research10.2196/5275826(e52758)Online publication date: 16-Aug-2024
https://doi.org/10.2196/52758
Hao AYuan HHui SSu J(2024)Effective type label-based synergistic representation learning for biomedical event trigger detectionBMC Bioinformatics10.1186/s12859-024-05851-125:1Online publication date: 31-Jul-2024
https://doi.org/10.1186/s12859-024-05851-1
Chu ZWang ZZhang W(2024)Fairness in Large Language Models: A Taxonomic SurveyACM SIGKDD Explorations Newsletter10.1145/3682112.368211726:1(34-48)Online publication date: 25-Jul-2024
https://dl.acm.org/doi/10.1145/3682112.3682117
Xie YLu JHo JNahab FHu XYang CHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)PromptLink: Leveraging Large Language Models for Cross-Source Biomedical Concept LinkingProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657904(2589-2593)Online publication date: 11-Jul-2024
https://doi.org/10.1145/3626772.3657904
August TLo KSmith NReinecke K(2024)Know Your Audience: The benefits and pitfalls of generating plain language summaries beyond the "general" audienceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642289(1-26)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642289
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents