Page 68 - Read Online

Tang et al. J. Mater. Inf. 2025, 5, 38 https://dx.doi.org/10.20517/jmi.2025.05 Page 9 of 19

provide the additive kernel. The cost of the method does not differ from conventional kernel regression
except for the summation in the kernel. The method is robust as only one-dimensional kernels are used to
represent the component functions f (y ), avoiding issues with multidimensional kernels. One consequence
n
n
of it is a less critical sensitivity to the hyperparameters.

One notes that Equation (7), while in y it is a 1st additive model obtained with additive GPR [48,49] , in the
original coordinates (feature space) x, it has the form of a single-hidden later NN with a linear output
neuron and optimal neuron activation functions individual to each neuron. W has the meaning of the
weight matrix of an NN; contrary to a conventional NN, it is not optimized but is set by rules. Because
neuron activation functions are optimized, biases are subsumed into them. As no non-linear optimization is
done, the method is stable with respect to overfitting: one need not know an optimal number of terms N;
exceeding the optimal (sufficient for a given dataset) value of N does not lead to overfitting, contrary to
growing the number of neurons of a conventional NN beyond optimal, see reference for a demonstration.
[47]

With respect to the purpose of the present work, the method is advantageous as it allows disambiguating the
contributions from nonlinearity in an additive model and that of coupling among the features. When
setting W = I, i.e., y = x, one obtains a 1st order additive model. Magnitudes of f (x ) can serve as indicators
n
n
of feature importance and their shapes reveal the type of functional dependence of the target on individual
features. f (x ) are in general nonlinear but may also come out linear when the optimal component function
n
n
shape is linear. Growing the number of terms (neurons) N with w generates coupling terms among
n
features. Different ways of setting w are possible [47,50] ; here we take w to be elements a D-dimensional
n
n
[51]
pseudorandom Sobol sequence . A MATLAB code for GPR-NN method is available in the supporting
information of Ref. , and a version modified for the present work in available (see data availability
[47]
statement).
RESULTS AND DISCUSSION
Assessing prediction accuracy with full-dimensional regression
ML modeling, utilizing the CE feature model, was employed to predict the substitution energies of dopant
elements in Nb and α-Nb Si alloys. The results indicate that the SVR method with a nonlinear kernel
3
5
function outperforms the RF method [Table 2]. In the study of α-Nb Si alloys, the dataset size for the four
5
3
non-equivalent sites significantly affects the prediction outcomes. Notably, the Si site, despite having the
II
smallest dataset, achieves the highest prediction accuracy. This finding suggests that smaller datasets can
yield more accurate predictions in certain scenarios, particularly when data quality is high with minimal
noise and the intrinsic distribution is clearly describable. Furthermore, for comprehensive datasets that
include multiple inequivalent sites, the prediction error does not simply compound the errors from each
individual site. This implies that the interactions between different sites and the complexity of the data
distribution play a crucial role in influencing prediction accuracy. That the error for the combined set is
higher corresponds to the fact that the data from subsets only partially overlap in the feature space, which
increases the volume of feature space aggravating the data scarcity challenge, rather than increasing the
density of internal sampling that would improve ML prediction.

Analysis of the role of nonlinearity and coupling
We used the GPR-NN method to fit uncoupled (additive) and coupled models. We performed 100 fits
differing by random train-test splits of the data. Hyperparameters (kernel length parameter and the noise
parameter) were chosen to minimize the average (over the 100 fits) test set error. Representative results of
ML with the GPR-NN method in the additive model regime (i.e., y = x) are shown in Figure 5 for individual
subsets as well as for the combined set, showing correlation plots between model-predicted and reference