Page 64 - Read Online

Bala et al. Art Int Surg. 2025;5:476-89 https://dx.doi.org/10.20517/ais.2025.44 Page 478

Retrieval-Augmented Generation (RAG) offers a promising remedy to these challenges. By augmenting
generative models with external retrieval mechanisms, RAG grounds model outputs in factual evidence,
thereby improving accuracy, contextual relevance, and trustworthiness [15-17] . While RAG has been
successfully applied in open-domain question answering and certain clinical NLP applications, its potential
in regulatory science - specifically for processing patient-generated device-related narratives - remains
underexplored. Applying RAG to this setting is challenging due to the linguistic diversity and noise inherent
in patient reports, but it holds the potential to transform the use of unstructured surveillance data.

Although zero-shot prompting and RAG are well-established methods in artificial intelligence (AI)
research [18,19] , their adoption in healthcare has primarily centered on structured or semi-structured data
sources. For instance, retrieval-augmented strategies have been applied to improve interpretability and
[20]
accuracy in zero-shot predictions on structured clinical tabular data , and to assess medical fitness using
large-scale language models with retrieval grounding . Similarly, another study has extracted highprecision
[21]
relations from biomedical documents by reframing tasks as question-answering and using external
biomedical knowledge to improve reasoning and reduce hallucinations . However, these studies mostly
[22]
operate in controlled settings, emphasizing structured datasets or well-defined benchmark tasks.

In contrast, patient-generated narratives in the FDA MAUDE system present a unique and underexplored
challenge: they are highly unstructured, linguistically diverse, and inherently noisy. Existing NLP methods
struggle with synonymy, negation, and implicit mentions, often leaving critical adverse event signals
undetected. By leveraging RAG-enhanced LLMs for mesh implant surveillance, this study addresses this
crucial gap. This approach introduces new challenges related to scalability, reliability, and interpretability,
while directly advancing regulatory science by systematically extracting adverse event data from previously
underutilized patient-reported sources. Importantly, our evaluation demonstrates not only accuracy
improvements over conventional approaches but also statistically validated gains in adverse event detection,
underscoring the clinical significance of this framework.

This study specifically analyzes patient-reported mesh implant narratives from the FDA MAUDE database
(2000-2021) to transform unstructured reports into structured clinical insights. Our focus is on extracting
key metadata (e.g., report date, hernia type, adverse event classification) and identifying both explicit and
implicit adverse events. By integrating zero-shot prompting with a RAG framework, we aim to create a
scalable, interpretable, and reliable pipeline for post-market surveillance of mesh implants.

This paper presents the following key contributions:
1. We develop and validate a RAG framework tailored for FDA MAUDE mesh implant narratives. This
framework is capable of converting free-text patient reports into structured fields, which are essential for
surveillance and regulatory analysis.

2. We apply zero-shot prompting to extract and classify adverse events from unstructured narratives,
demonstrating improved reliability, reduced hallucination, and superior scalability compared with
conventional NLP methods.

METHODS
Dataset and Benchmark Baseline
The dataset for this research was obtained from the FDA MAUDE database (https://www.accessdata.fda.
gov/scripts/cdrh/cfdocs/cfmaude/search.cfm), comprising voluntary reports submitted by patients who
underwent mesh implant surgery between 2000 and 2021 [3,23] . After filtering for patient-generated narratives,