Page 187 - Read Online

Page 12 of 22 Hei et al. J. Mater. Inf. 2026, 6, 15

Table 3. Entity extraction performance of the proposed MatSciBERT-based model
1 2 3 4 Random
Entity
F1 P R F1 P R F1 P R F1 P R F1 P R
MAT 0.96 0.95 0.97 0.97 0.94 1 0.94 0.94 0.94 0.95 0.94 0.97 0.95 1 0.91
PRO 1 1 1 0.96 0.93 1 0.93 0.89 0.97 0.91 0.93 0.89 0.95 0.95 0.95
PRO V 0.99 1 0.98 1 1 1 0.94 0.90 0.99 1 1 1 0.97 0.94 1
CON 1 1 1 1 1 1 0.86 0.75 1 1 1 1 1 1 1
CON V 0.93 1 0.88 1 1 1 0.91 0.83 1 1 1 1 1 1 1
TOTAL 0.98 0.97 0.98 0.96 0.98 0.95 0.92 0.88 0.97 0.97 0.95 0.99 0.98 0.99 0.97
Entity extraction perfomrance of the proposed MatSciBERT-based mode with the highest F1 score for each entity highlighted in bold. MAT, PRO, PRO V, CON, and CON V represent MATERIAL, PROPERTY, PROPERTY VALUE,
CONDITION, and CONDITION VALUE, respectively.

Table 4. Entity extraction performance of the proposed BERT-based model
1 2 3 4 Random
Entity
F1 P R F1 P R F1 P R F1 P R F1 P R
MAT 0.74 0.87 0.65 0.76 0.81 0.71 0.55 0.67 0.47 0.88 0.90 0.85 0.68 0.74 0.63
PRO 0.82 0.86 0.76 0.78 0.87 0.7 0.82 0.87 0.77 0.85 0.90 0.81 0.80 0.96 0.68
PRO V 0.81 0.77 0.85 0.92 0.93 0.91 0.81 0.92 0.73 0.86 0.9 0.82 0.85 0.84 0.86
CON 0.8 1 0.67 0.5 1 0.33 1 1 1 1 1 1 0 0 0
CON V 0.47 0.44 0.5 0.57 1 0.4 0.89 1 0.8 0.62 1 0.44 0 0 0
TOTAL 0.78 0.81 0.74 0.82 0.89 0.75 0.78 0.86 0.72 0.85 0.90 0.80 0.80 0.84 0.77
MAT, PRO, PRO V, CON, and CON V represent MATERIAL, PROPERTY, PROPERTY VALUE, CONDITION, and CONDITION VALUE, respectively. BERT: Bidirectional encoder representations from transformers.

list BERT-base and SciBERT results and Table 6 summarizes totals across encoders.

For MATERIAL entities, MatSciBERT attains F1 scores of 0.94-0.97, slightly ahead of SciBERT at 0.93-0.97 and substantially better than BERT-base at 0.55-0.88. For
PROPERTY entities, MatSciBERT reaches 0.91-1.00, compared with 0.88-0.96 for SciBERT and 0.78-0.85 for BERT-base. For PROPERTY VALUE entities,
MatSciBERT achieves 0.94-1.00, vs. 0.91-0.98 for SciBERT and 0.81-0.92 for BERT-base. For CONDITION entities, MatSciBERT maintains 0.86-1.00, while BERT-base