Page 15 - Read Online

Page 8 of 28 Zhang et al. J Mater Inf 2024;4:34 https://dx.doi.org/10.20517/jmi.2024.64

[72]
model size . The introduction of the inverted residual structure enhances gradient flow, mitigating
vanishing gradients and improving overall network stability during training. The architecture also facilitates
more efficient information transfer across layers, enabling deeper networks to capture data features more
effectively. Compared to traditional residual connections, the inverted residual structure achieves greater
computational efficiency with fewer parameters, optimizing both training and inference processes.

ConvNeXt architecture
ViT faces challenges of extensive parameters and substantial computational demands, which hinder the
suitability for lightweight deployment [73,74] . By addressing the limitations above, ConvNeXt leverages the
design principles and optimization advancements inspired by ViT, combining with the efficiency of
CNNs [75,76] . The innovative architecture improves upon existing frameworks by substituting the commonly
employed ReLU activation function with the Gaussian error linear unit (GELU):

(1)

(2)

Proposed by Hendrycks and Gimpel in 2016, GELU has gained attention for its smoother nonlinear
characteristics, which improve model performance . The activation function integrates the advantages of
[77]
both Sigmoid and ReLU, providing a continuous derivative that enhances gradient propagation during
[78]
training and thereby mitigates the risk of gradient vanishing . Additionally, ConvNeXt enhances the
performance using grouped convolution within the ConvNeXt Block, which partitions input feature maps
into distinct groups for independent convolutions, thereby improving representational capacity and feature
[79]
extraction efficiency .

Automatic programming of intelligent welding based on LLMs
LLMs are advanced deep learning architectures trained on vast datasets, which can not only generate
coherent natural language text but also deeply grasp the context and meaning [80-82] . LLMs excel in various
natural language processing tasks, such as text summarization, intelligent question-answering systems and
machine translation, showcasing their versatility across multiple domains . The construction of LLMs
[83]
involves several critical steps. The process begins with a requirements analysis to delineate the model’s
intended application, functionality and performance objectives. Relevant textual data is collected from
diverse sources, including web pages, books and articles, followed by noise elimination [84,85] . The model is
trained guided by hyperparameters including learning rate, batch size and iteration count, while
performance is validated based on metrics such as perplexity, F1 score, bilingual evaluation understudy
(BLEU) and recall-oriented understudy for gisting evaluation (ROUGE). Additionally, LLMs undergo
fine-tuning for specific tasks or domains, a process that generally requires fewer resources compared to the
initial training phase [86,87] . For the purpose of optimizing operational efficiency, compression is necessary
through techniques including pruning and quantization. Finally, the model is deployed on suitable
platforms equipped with a user-friendly application programming interface (API) for integration and
subjected to real-time monitoring to ensure stability.

Figure 5 showcases the application of LLMs including ChatGPT-3.5, Copilot, Claude 3, and Ernie Bot in
facilitating automated coding processes. In a specific test case focusing on the extraction of welding arc
contours, all these models successfully generated executable programs to achieve the desired objectives. The
input provided to the LLMs for this task was as follows: “We require processing welding arc images with
three channels to extract the arc contours. It is known that the brightness of the arc area is significantly