Page 227 - Read Online

Yuan et al. J. Mater. Inf. 2026, 6, 17 Page 5 of 13

that are already known to yield favorable results. The EI criterion, in particular, evaluates the expected gain
over the current best-known performance, guiding the search toward configurations with the highest
probability of improving the model metric (mAP@0.95 in our case). This dynamic balance between
exploration and exploitation is the key to achieving both efficiency and accuracy in the hyperparameter
search process. Furthermore, Bayesian optimization is well-suited for non-convex, noisy, and expensive
objective functions, which are common in deep learning scenarios. The objective landscapes of neural
networks often contain many local minima and plateaus, making random or uniform search strategies highly
inefficient. By contrast, Bayesian optimization builds a smooth approximation of the objective surface,
enabling more effective navigation of these complex landscapes. In our experiments, Bayesian optimization
successfully identified optimal hyperparameter configurations within 30 iterations, while random search with
the same number of trials failed to achieve comparable performance. Consequently, we employed a Bayesian
optimization algorithm to determine the optimal combination of hyperparameters.

We constructed a search space from the selected hyperparameters and performed 30 iterations of Bayesian
optimization. The selected hyperparameters were chosen based on their relevance to model performance
[30]
and training stability. From Figure 2A, we observed that the optimal configuration was achieved at the 13th
iteration, where mAP@0.95 reached its maximum value. The heatmap in Figure 2B further illustrates the
correlations among key hyperparameters, revealing that the initial learning rate (lr0) shows a strong negative
correlation with performance, while warmup epochs exhibit a moderate positive correlation. These
observations suggest that a proper balance among learning rate, regularization, and warmup duration is
crucial for optimizing training stability and generalization. Detailed mathematical formulations and
correlation analyses have been moved to the Supplementary Section 3. In Figure 2C, the two most influential
hyperparameters were selected based on their Pearson correlation coefficients with the model performance
metric. The scatter plots illustrate the relationships between each selected hyperparameter (e.g., initial
learning rate Ir0, warmup epochs) and mAP@0.95. Each point represents the result of a training session,
where the x-axis shows the hyperparameter value and the y-axis shows the corresponding mAP@0.95 score.
In these plots, polynomial regression curves were fitted to visually illustrate the trends between each
hyperparameter and the model performance. For example, it is clearly observed that the initial learning rate
shows a significant negative correlation with the model performance until lr0 ~ 0.03, meaning that increasing
the learning rate leads to a decrease in mAP@0.95. This trend is clearly demonstrated in both the scatter plot
and the regression curve, where the shape of the curve is closely related to the distribution of the data.

In our program, we specifically incorporated the feature of incremental learning, and among the various
incremental learning methods, we selected the most stable and convenienta replay mechanism . The replay
[31]
mechanism works by saving a portion of historical data during the training process and periodically
retraining the model with both the historical data and new data in subsequent training sessions. This
approach allows the model to retain knowledge of previously seen data while learning new information,
thereby mitigating catastrophic forgetting.

From Figure 3, the replay mechanism can be seen to select a small batch of historical data and combine it
with the current training data for model training. This process not only enables the model to continuously
adapt to new data but also ensures training stability, as the model “reviews” previously learned samples. As a
result, the model avoids overfitting to the current samples while retaining knowledge acquired from earlier
data. In this way, the model can gradually enhance its generalization ability and robustness during the
incremental learning process, leading to more effective knowledge updates and applications.

The advantage of the replay mechanism is that it allows the model to be continuously updated and optimized
as new data become available, without the need for complete retraining from scratch. This greatly improves