Fine-Tuning Domain-Specific LLMs for Medical Named Entity Recognition (NER) and Context-Aware Summarization

Abstract

One of the biggest challenges faced by to-day’s healthcare system is the reliance on legacy systems which rely on paper-based data storage systems. Health practitioners tend to document critical details such as medication, lab reports, and discharge information manu¬ally within the current fragmented system. In the absence of an integrated digital system, there would be increased workload among clinicians to enter data manually. Manual data entry is prone to error as the process can be exhausting for clinicians and involves legible writing. To address this persistent challenge within our healthcare system, we present an intelligent system called CareTrack. In essence, the architecture of this prototype levarage the use of multimodal vision transformers along with a large language model. Most notably, it adopts the technique of retrieval-augmented generation (RAG) that supports contextual understanding of the medical record in question instead of basic text recognition. When tested on various medical documents, the results were highly encouraging. In particular, the proposed model was able to score 98.