Leveraging TFR-BERT for ICD diagnoses ranking

• Main Author: Silva, Ana
• Publication Date: 2023
• Other Authors: Chaves, Pedro, Rijo, Sara, Boné, João, Oliveira, Tiago José Martins, Novais, Paulo
• Language: eng
• Source: Repositórios Científicos de Acesso Aberto de Portugal (RCAAP)
• Download full: https://hdl.handle.net/1822/89870
• topic: Biomedical language models; Learning-to-rank; Ranking diagnoses

This work describes applying a transformer-based ranking solution to the specific problem of ordering ICD diagnoses codes. Taking advantage of the TFR-BERT framework and adapting it to the biomedical context using pre-trained and publicly available language representation models, namely BioBERT, BlueBERT and ClinicalBERT (Bio + Discharge Summary BERT Model), we demonstrate the effectiveness of such a framework and the strengths of using pre-trained models adapted to the biomedical domain. We showcase this by using a benchmark dataset in the healthcare field—MIMIC-III—showing how it was possible to learn how to sequence the main or primary diagnoses and the order in which the secondary diagnoses are presented. A window-based approach and a summary approach (using only the sentences with diagnoses) were also tested in an attempt to circumvent the maximum sequence length limitation of BERT-based models. BioBERT demonstrated superior performance in all approaches, achieving the best results in the summary approach.