Authors:
S. Abinaya, M. S. Antony Vigil , K. P. Keerthika, R. V. Varshasri
Addresses:
Department of Computer Science and Engineering, SRM Institute of Science and Technology, Ramapuram, Chennai, Tamil Nadu, India.
Abstract:
Rapidly expanding medical text data (clinical notes, EHRs, biological literature) requires efficient summarization. Manual interpretation of such big data is unfeasible and time-consuming in time-sensitive healthcare settings. To solve this problem, we offer a medical text summarisation system using BART (Bidirectional and Auto-Regressive Transformer), PEFT, and Low-Rank Adaptation. Lower-parameter big language model refinement is computationally efficient using this strategy. Our method produces high-quality, coherent summaries in resource-limited situations. Our trials optimized 73,728 parameters out of 406,364,160, or 0.0181%, simplifying model training. Even with poor conditioning, the algorithm produced contextually sensitive summaries with medical content. Without modifying the architecture, LoRA enables task-specific learning via low-rank matrix decomposition and efficient task adaptability. This includes healthcare diagnostic and clinical trial reports. Our method outperformed complete fine-tuning in training time, memory usage, and scalability on benchmark medical datasets. These outcomes also demonstrate that our system is promising for realistic institutional changes. This study shows how resource-efficient, scalable medical summarization systems may work. Our solution reduces the computing load of related methods, enabling AI-powered healthcare applications. This concept enhances fine-tuning and helps construct intelligent systems to process and summarise complex medical data. The proposed approach supports parameter-efficient adaptation research, especially in key application domains where accuracy and efficiency are crucial.
Keywords: BART (Bidirectional and Auto-Regressive Transformer); Parameter-Efficient Fine-Tuning (PEFT); Low-Rank Adaptation (LoRA); Transformer Models; Fine-Tuning; Text Summarization.
Received: 26/05/2024, Revised: 07/09/2024, Accepted: 18/10/2024, Published: 07/12/2024
AVE Trends in Intelligent Health Letters, 2024 Vol. 1 No. 4 , Pages: 228-242