Page 62 - Read Online

Bala et al. Art Int Surg. 2025;5:476-89 Artificial
DOI: 10.20517/ais.2025.44
Intelligence Surgery

Original Article Open Access

Clinical insight from mesh implant narratives using
zero-shot Retrieval-Augmented Generation approach

2
1
Indu Bala , Ekta Sharma , Lewis Mitchell 1
1
School of Computer and Mathematical Sciences, The University of Adelaide, Adelaide 5005, Australia.
2
School of Mathematics, Physics, and Computing, University of Southern Queensland, Springfield 4300, Australia.
Correspondence to: Dr. Indu Bala, School of Computer and Mathematical Sciences, The University of Adelaide, Adelaide 5005,
Australia. E-mail: iinduyadav@gmail.com; indu.bala@adelaide.edu.au
How to cite this article: Bala I, Sharma E, Mitchell L. Clinical insight from mesh implant narratives using zero-shot Retrieval-
Augmented Generation approach. Art Int Surg. 2025;5:476-89. https://dx.doi.org/10.20517/ais.2025.44
Received: 8 May 2025 First Decision: 14 Jul 2025 Revised: 20 Aug 2025 Accepted: 11 Sep 2025 Published: 31 Oct 2025

Academic Editor: Gaya Spolverato Copy Editor: Xing-Yue Zhang Production Editor: Xing-Yue Zhang

Abstract
Aim: Mesh implant surgeries for hernia repair are frequently associated with adverse events that can compromise
patient outcomes. Extracting structured clinical insights from large-scale, unstructured data sources such as the
U.S. Food and Drug Administration’s Manufacturer and User Facility Device Experience (FDA MAUDE) database
remains a challenge due to variability and subjectivity in patient narratives. This study aims to develop and evaluate
a zero-shot generative artificial intelligence (AI) framework enhanced with Retrieval-Augmented Generation
(RAG) to automatically extract structured clinical information and adverse event indicators from unstructured
mesh implant reports, assessing its accuracy, interpretability, and scalability against a manually annotated
benchmark.

Methods: The study employed the LLaMA 2 (13B) model for zero-shot structured summarization and adverse
event extraction from FDA MAUDE mesh implant reports (2000–2021). The framework integrated retrieval-
based context using RAG and evaluated model performance on report date, hernia type, and adverse event flag
using accuracy, Jaccard similarity, and Chi-square tests (P < 0.05). Statistical analysis validated improvements in
output reliability and clinical relevance.

Results: The model outputs were compared to a manually annotated Benchmark Baseline. With zero-shot
prompting alone, the model achieved accuracies of 67% for report date, 60% for hernia type, and 83% for adverse
event flag. After integrating the RAG approach, these accuracies improved to 81%, 82%, and 99%, respectively.
The accuracy for adverse event extraction increased from 60% to 86%, and the Jaccard similarity improved from

© The Author(s) 2025. Open Access This article is licensed under a Creative Commons Attribution 4.0
International License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, sharing,
adaptation, distribution and reproduction in any medium or format, for any purpose, even commercially, as
long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and
indicate if changes were made.

www.oaepublish.com/ais