FL coordinates model optimization across distributed data holders without requiring raw data to be centralized. In a typical server-client workflow, the server initializes and distributes a global model, selected clients perform local optimization on private data, and the server aggregates the returned model-related information to update the global model^[4,5]. Federated Averaging (FedAvg) is the canonical instance, where client updates are commonly averaged according to local data volume^[4]. This repeated distribution-local update-aggregation cycle makes FL suitable for privacy-preserving collaborative learning, but also introduces resource costs from communication, synchronization, client heterogeneity, and server-side aggregation.

The workflow can be described along three dimensions that are especially relevant to federated LLM adaptation:

• Data organization: Horizontal FL considers different samples in a shared feature space; vertical FL considers overlapping entities with different feature sets; and federated transfer learning addresses limited overlap in both samples and features.

• Coordination architecture: Server-based FL relies on a central aggregator; hierarchical FL introduces intermediate edge servers; and decentralized FL uses peer-to-peer communication and local consensus.

• Synchronization mode: Synchronous FL aggregates updates from a round-specific client set, asynchronous FL incorporates updates as they arrive, and bounded-asynchronous FL limits staleness while retaining partial flexibility^[5,8].

These dimensions determine where data remain, how model-related information moves, and when updates are fused. They therefore provide the basic workflow assumptions for analyzing communication, computation, aggregation, and synchronization costs in federated LLM systems.

LLMs are foundation models trained on large-scale text or multimodal corpora to acquire transferable capabilities for language-centered tasks. Most contemporary LLMs use the Transformer architecture, whose self-attention captures long-range token dependencies and permits parallel processing of token positions within each training layer, although autoregressive generation remains sequential. Through pretraining, typically with next-token prediction, LLMs learn general linguistic, semantic, and task-relevant patterns that support generation, reasoning, coding, and domain-specific text processing^[22,23].

Post-training adapts this pretrained foundation to specific domains, users, or preferences through supervised fine-tuning, instruction tuning, domain adaptation, or preference alignment. Full-model fine-tuning updates most or all parameters, whereas PEFT trains only low-rank matrices, adapters, prompts, prefixes, or other small components while freezing most of the backbone^[2,3,24]. It aims to preserve adaptation utility while reducing trainable state, optimizer storage, and, in federated settings, update size.

However, PEFT does not remove the main execution burden of LLM adaptation. Clients may still need to host the pretrained backbone, perform forward and backward propagation through Transformer layers, store activations and intermediate states, and process long input sequences. Long-context adaptation further increases attention computation and activation memory^[22,23]. As a result, LLM adaptation is not only a model-update problem, but also an execution problem shaped by memory, computation, bandwidth, latency, and energy. This distinction is central to resource accounting in federated LLMs, where fewer trainable parameters do not necessarily imply lower end-to-end adaptation cost.

Although FL can support language-model pretraining, this review focuses on the more common and practically relevant setting of federated post-training adaptation. In this setting, clients collaboratively adapt a pretrained LLM, or an adaptation interface built around it, over distributed private data rather than training a language model from random initialization. Depending on the system design, clients may update the full model, optimize PEFT modules^[1,14,17], tune prompts, maintain personalized components, or exchange split-model representations^[9], while raw data remain local^[25,26].

This setting changes three key objects in the federated loop:

• Adaptation object: the full model, a PEFT module, a prompt representation, a split-model component, or a personalized adapter.

• Communication object: compressed parameter deltas, low-rank factors, prompts, logits, activations, gradients, or other task-related signals.

• Federated output: a global LLM, a shared adaptation module, personalized components, or a deployment configuration.

These objects may vary across clients. In server-coordinated, hierarchical, and emerging service-assisted federated LLM systems^[27], clients can differ in adaptation configuration, model access, objectives, and available resources. Such differences create two coupled constraints: local feasibility, which determines whether a client can produce an update, and aggregation compatibility, which determines whether heterogeneous updates can be fused into a useful shared or personalized model.

These choices complicate resource accounting in federated LLMs. Unlike conventional FL, model size, computation, and communication can become decoupled: A small PEFT module may still require executing a large pretrained backbone^[1,24,28], while compact updates may introduce frequent synchronization, activation transfer, reconstruction, or alignment costs^[9]. Trust mechanisms, including privacy protection, verification, robustness, provenance, and unlearning, further add computation, communication, storage, or utility costs^[10-12].

Resource efficiency should therefore be assessed across the complete adaptation workflow at a comparable utility target, covering client execution, bidirectional communication, server-side processing, orchestration, and trust-related costs. As summarized in Figure 2, trainable parameter count or per-round payload alone cannot represent the end-to-end cost of federated LLM adaptation.

Table 2 summarizes the main differences between federated LLM adaptation and related model collaboration paradigms. Federated LLM adaptation occupies a specific design point among these collaboration paradigms. It combines local data control with collaborative adaptation of large pretrained models. Its resource challenge extends beyond conventional FL communication and local computation because LLM-scale execution, structured and potentially heterogeneous adaptation objects, and their aggregation and orchestration must be considered jointly. This distinction motivates the workflow-level resource analysis developed in Sections RESOURCE CHALLENGES IN FEDERATED LLMS and RESOURCE-EFFICIENT MECHANISMS FOR FEDERATED LLMS.

The review is organized around five resource questions that follow the federated adaptation workflow: Can a client produce a useful update within its memory, computation, time, and energy budget? Can the required information be exchanged efficiently? Can heterogeneous client contributions be aggregated into a meaningful shared or personalized model? Can participation, workload, synchronization, computation placement, and communication resources be orchestrated efficiently? Finally, can privacy, security, and lifecycle trust be maintained without excessive additional resource cost? These questions cover the principal resource-bearing stages of federated post-training adaptation and explain the one-to-one organization of the challenges in Section RESOURCE CHALLENGES IN FEDERATED LLMS and the corresponding mechanisms in Section RESOURCE CHALLENGES IN FEDERATED LLMS.

The reported link rates in Table 3 are experimental configurations adopted by the corresponding studies and should not be interpreted as constant-rate assumptions for practical wireless deployments.

A basic assumption in FL is that each selected client can complete its assigned local update. This assumption becomes fragile in federated LLMs. Representative edge platforms differ in accelerator support, memory capacity, power envelope, network conditions, and measured local-training latency, as summarized in Table 3^[34-36] The reported link rates in Table 3 are experimental configurations adopted by the corresponding studies and should not be interpreted as constant-rate assumptions for practical wireless deployments. Even when adaptation starts from a pretrained model, clients may still need to host the backbone, store activations and intermediate states, propagate gradients through Transformer layers, and maintain optimizer states. These requirements can make some local adaptation configurations infeasible, rather than merely slower, on resource-constrained clients^[1,37-39]. Local feasibility is also affected by the data used for adaptation. Redundant, noisy, or low-value instructions consume expensive Transformer updates without proportional utility, whereas data-scarce settings may require additional augmentation, privacy-preserving adaptation, or sample-selection mechanisms^[40-42].

The resource evidence in existing federated language-model studies spans substantially different model scales. Results obtained with DistilBERT, BERT, and BART are useful for studying device heterogeneity and resource-control mechanisms, but they should not be interpreted as direct feasibility evidence for multi-billion-parameter generative LLMs. The latter introduce a much larger backbone-residency and execution burden, in addition to activation and optimizer-state memory during training. For example, AssyLLM^[43] reports that full fine-tuning of LLaMA-7B with batch size 16 requires more than 40 GB of memory, compared with the 4-16 GB memory available to many of the edge devices considered in its evaluation. Its block-assembly design reduces memory consumption by up to 92%, illustrating both the severity of the memory gap and the need for LLM-specific execution mechanisms.

This creates a gap between parameter efficiency and system feasibility. PEFT methods such as LoRA and adapters reduce the number of trainable parameters^{[2-3,24,44-46]}, but producing even a lightweight update may still require substantial memory, computation, and energy because the frozen backbone must be executed. Local feasibility is best assessed through the full adaptation process, where uploaded update size is considered together with model residency, memory footprint, training time, energy use, and data utility.

Communication has long been a core bottleneck in FL, but federated LLMs change both the size and the form of exchanged information. Since full-model transmission is often impractical, clients may instead communicate PEFT modules^[47,48], dense or compressed parameter deltas^[49,50], prompts, logits, activations, gradients, or other intermediate representations^[9,51]. These objects are usually much smaller than a complete LLM, but they may be exchanged repeatedly across local batches, communication rounds, or split-model interactions, creating substantial uplink, downlink, and synchronization costs.

Thus, smaller messages do not necessarily imply lower end-to-end communication cost. Compression and low-rank transmission may reduce each payload but introduce extra processing, metadata, or reconstruction costs, and lower-fidelity updates may require more rounds to reach the same utility.

In practical wireless deployments, network conditions may vary substantially over time. Channel fading, interference, link errors and retransmissions, MAC-layer contention, bandwidth fluctuations, and uplink/downlink asymmetry can reduce effective goodput and increase communication-latency variability. This variability directly affects federated synchronization: A temporarily weak client link may become a communication straggler in synchronous FL, whereas asynchronous or partial aggregation can reduce waiting at the cost of increased update staleness. The effect can be particularly pronounced for split-federated LLMs, where activations and gradients may be exchanged repeatedly during local adaptation. Communication efficiency should therefore account not only for nominal link rates, but also for link variability, reliability, tail latency, and their effects on synchronization.

FedAvg assumes that client updates share a common parameter structure and can be directly averaged^[4,5]. This assumption weakens in federated LLMs because resource constraints often push clients toward heterogeneous adaptation configurations. A resource-rich client may train a higher-rank LoRA module or more Transformer layers, whereas a constrained client may use lower ranks, fewer trainable layers^[52-54], lower-bit precision^[37], or a prompt-based interface^[55]. These choices improve local feasibility, but they also change the structure and functional meaning of the updates received by the server.

Aggregation therefore becomes a compatibility problem, not merely an averaging operation. Client updates may differ in structure, precision, or training objective. While mild structural differences can be handled through simple alignment^[52,53], stronger heterogeneity often requires reconstruction^[54], distillation^[56], or personalized fusion^[55]. Uniform configurations simplify aggregation but may exclude weaker clients, whereas heterogeneous configurations improve participation at the cost of additional fusion complexity. The key challenge is therefore to preserve the functional or semantic content of local adaptations while keeping aggregation efficient.

Aggregation compatibility concerns the structure of updates, whereas system heterogeneity concerns the clients and infrastructure that produce them. In federated LLMs, system heterogeneity arises jointly from computing and communication capacities. Computational heterogeneity includes memory and accelerator availability, training speed, numerical-precision support, and energy or thermal constraints, while communication heterogeneity includes bandwidth, uplink/downlink asymmetry, time-varying link quality, and intermittent connectivity. Existing studies adapt quantization, LoRA configuration, or local workload to heterogeneous client capabilities^[35,37], while hierarchical and split-federated designs coordinate workload, synchronization, and communication resources across heterogeneous devices and links^[29,57,58]. A uniform workload may exclude resource-constrained clients or create stragglers^[59,60], whereas edge assistance^[61] and asynchronous participation^[8] can shift resource burdens toward communication, edge servers, or coordination overhead^[57,62]. These effects are tightly coupled: offloading can reduce client computation while increasing communication demand, whereas communication-limited clients may require lighter adaptation workloads or less frequent participation.

System orchestration must therefore assign feasible and useful roles to heterogeneous participants. This involves deciding which clients participate, what adaptation configuration they use, where computation is placed, when updates are synchronized, and how communication and computing resources are allocated. These decisions are coupled: Lowering local computation through edge assistance may increase repeated transfers of activations and gradients^[61,62]; relaxing synchronization may reduce waiting time but introduce staleness^[8,57]; and favoring capable clients may improve efficiency but reduce data coverage or fairness^[55,59]. Effective orchestration should therefore balance local feasibility, communication and computation cost, system latency, and data representativeness.

Keeping raw data local provides an important basis for privacy-preserving LLM adaptation, but the exchanged information still requires protection. Privacy mainly concerns leakage from shared updates or intermediate representations, which may reveal sensitive training content or user-specific patterns^[63-65]. Security concerns the robustness and integrity of adaptation, where malicious, biased, or backdoored updates may distort the shared model or propagate harmful behavior^[66-68]. Trust further concerns whether the adaptation process is verifiable, accountable, and auditable across the model lifecycle.

These protections introduce resource overheads. Privacy mechanisms may reduce utility or increase communication and cryptographic costs^[10,18,65]; security mechanisms such as robust aggregation and attack detection require additional computation^[66]; and trust mechanisms such as verifiable aggregation, provenance, and unlearning introduce storage, auditing, rollback, or retraining costs^[11,12]. Privacy, security, and trust mechanisms should therefore be co-designed with resource optimization, because they jointly shape the attainable balance among utility, efficiency, and trustworthiness.

Local adaptation is the first step at which resource feasibility is tested. A selected client must be able to produce a useful update within its memory, computation, time, and energy budget. Existing methods improve local feasibility by reducing trainable state, lowering execution cost, or selecting the data that justify expensive Transformer updates.

Parameter-efficient adaptation. PEFT adapts a pretrained LLM by freezing most backbone parameters and updating only a small set of trainable components. Many federated LLM systems therefore replace full fine-tuning with PEFT methods such as LoRA, adapters, prefix tuning, and prompt tuning^[1,44-46]. For LLaMA-7B, FederatedScope-LLM^[1] reports per-round messages of approximately 12,852 MB for full-model exchange, 21.40 MB for LoRA, and 0.17 MB for prompt tuning, while model-only memory remains about 13.4 GB under the study’s accounting. PEFT therefore provides its most direct savings in trainable and transmitted state, while backbone hosting and execution remain important parts of local resource accounting.

Resource-aware local execution. Beyond the choice of trainable modules, resource-aware execution controls the cost of producing each local update. Existing methods adjust numerical precision^[37], LoRA rank, trainable layers, or active modules^[39] according to client capability and module sensitivity^{[35,52,69,70]}. FedARA^[35], for example, dynamically allocates and prunes low-rank modules and reports an average 2.40-fold improvement in communication efficiency, up to 48.90% shorter total training time, and 46.95% lower energy consumption on Orin Nano. It also reports a 31.67% reduction in average peak GPU memory usage per round relative to FedLoRA. These results are obtained mainly with DistilBERT, BERT, and BART and therefore provide PLM-scale evidence for adaptive execution rather than direct feasibility evidence for multi-billion-parameter LLMs.

Recent generative-LLM studies have addressed the larger memory constraint more directly. AssyLLM^[43] assembles and adapts selected pretrained blocks to reduce the memory required for federated LLaMA-7B fine-tuning, while FedBiOT^[71] enables federated adaptation of LLaMA-2 without requiring clients to hold the complete model. Together with the LLaMA-7B evidence reported above for FederatedScope-LLM^[1], these studies show that reducing trainable parameters alone is insufficient; backbone residency, activation memory, and Transformer execution remain first-order constraints for edge-side generative-LLM adaptation.

Data-efficient local training. Local cost also depends on how many examples undergo expensive Transformer updates. Data-efficient methods reduce this cost by selecting higher-value training samples or filtering low-quality instructions. FedDQC^[40] filters low-quality instructions through a scoring stage that consumes roughly 1% of training time, whereas FedHDS^[41] retains less than 1.5% of the available samples and reports 6.66- to 48.8-fold wall-clock speedups across different settings. These results extend local efficiency from choosing trainable parameters to choosing training data.

In summary, PEFT reduces what is updated, resource-aware execution reduces the cost of each update, and data-efficient training reduces the number of updates performed on low-value samples. End-to-end deployability depends on combining these mechanisms with communication, aggregation, orchestration, and trust-aware design across the full federated adaptation workflow.

Communication efficiency in federated LLMs depends on what is exchanged, how it is encoded, and how often information is exchanged. Existing methods improve communication efficiency by exchanging compact adaptation parameters, selectively transmitting structured updates, or replacing parameter exchange with knowledge or intermediate representations.

Compact structured parameter exchange. The most direct approach is to exchange trainable adaptation parameters instead of full model weights. LoRA modules, adapters, and prompts preserve the conventional server-client training loop while substantially reducing the communicated object^[1,44]. More compact parameterizations further redesign the adaptation object itself. FedTT and FedTT+^[47] represent tensor-train-based federated adaptation, with FedTT+ further freezing selected factors. These approaches are most effective when clients share a pretrained backbone and compatible adaptation structures. Their end-to-end benefit should be assessed together with model initialization, downlink broadcasts, metadata, and convergence rounds.

Selective and compressed transmission. These methods reduce the amount of structured information sent in each exchange while keeping the adaptation representation largely unchanged. Typical strategies include low-precision encoding, sparsity, component selection, and partial transmission. EcoLoRA^[48], for example, retains the LoRA structure while rotating transmitted segments, exploiting matrix-specific sparsity, and compactly encoding indices. Other methods transmit selected low-rank components or active update portions^[69,72]. These methods mainly optimize the scheduling and encoding of structured updates rather than redesigning the adaptation representation. Their gains depend on the balance among payload reduction, update fidelity, and the number of rounds required to reach a target utility.

Knowledge and intermediate-representation exchange. Instead of transmitting model parameters, these methods use task-level signals or internal representations as the communication object. Black-box language and vision-language settings may exchange discrete prompts or compact task signals^[73-75], while split systems transmit activations and gradients across model partitions^[9,51]. These designs can support heterogeneous access modes and reduce parameter transmission, but they may introduce repeated queries, proxy-data dependence, distillation overhead, or frequent bidirectional communication. TITANIC^[9] provides a useful example of this cost shift. Under its reported representation size and exchange pattern, cumulative activation and gradient traffic can exceed a compact LoRA update after only a few batches.

A simple break-even condition helps clarify when split adaptation is preferable to direct PEFT exchange. For one local adaptation round, let D_P denote the total bidirectional PEFT-update traffic, D_S the bidirectional activation/gradient traffic per split interaction, and K the number of such interactions. With effective network throughput B_eff, the first-order completion times of PEFT and split adaptation are

(1) $$ T_{\mathrm{P}}=T_{\text {local }}+\frac{D_{\mathrm{P}}}{B_{\text {eff }}}, \quad T_{\mathrm{S}}=T_{\mathrm{c}}+T_{\mathrm{e}}+\frac{K D_{\mathrm{S}}}{B_{\text {eff }}}, \\ $$

where T_local is the local PEFT execution time, and T_c and T_e are the client-side and edge-side execution times under splitting, respectively. Defining the relative split-computation ratio as

(2) $$ \rho=\frac{T_{\mathrm{c}}+T_{\mathrm{e}}}{T_{\text {local }}}, \\ $$

when ρ < 1 and KD_S > D_P, the break-even throughput is

(3) $$ B^{*}=\frac{K D_{\mathrm{S}}-D_{\mathrm{P}}}{(1-\rho) T_{\text {local }}}, \\ $$

and split adaptation reduces latency when B_eff > B^*. Thus, greater computation offloading lowers the required bandwidth, whereas larger or more frequent activation/gradient exchanges raise it. The threshold varies with the split point, sequence length, batch size, compression, hardware capability, and network conditions, motivating joint model-partitioning and communication-resource optimization^[29,58].

Overall, communication efficiency should therefore be evaluated at the level of the complete exchange process and its computation-communication trade-off. In addition to the size of each transmitted object, assessment should include total bidirectional traffic, the number and pattern of interactions, encoding and reconstruction overhead, and the time or rounds needed to reach a comparable utility target.

Aggregation becomes a resource problem when clients adopt different adaptation configurations. FedAvg is straightforward when updates share the same structure, but uniform configurations may overburden weak clients or underuse capable ones. Heterogeneous adaptation improves local feasibility by allowing clients to use different ranks, trainable layers, precisions, or adaptation modules, while introducing additional aggregation complexity. Existing methods address this problem at three levels: shape compatibility, functional equivalence, and semantic compatibility.

Shape compatibility. Shape-compatible methods handle heterogeneous updates through layer allocation^[52], zero-padding and sparsity-weighted aggregation^[53], padding- or knowledge-distillation-based alignment^[76], masking, or partial aggregation^[72]. HETLORA^[53], for example, combines local rank self-pruning with server-side zero-padding, sparsity-weighted aggregation, and rank-specific truncation. These operations make updates structurally aggregatable, while functional consistency still depends on how the adapted parameters affect the model.

Functional equivalence. For LoRA-based adaptation, the update produced by client i is not represented by a single matrix, but by the product of two low-rank factors, i.e., ΔW_i=B_iA_i. Therefore, the desired global update is the weighted average of the actual updates, $$ \Delta W_{\text {global }}=\sum_{i=1}^{K} p_{i} B_{i} A_{i} \\ $$, where p_i denotes the normalized aggregation weight. A naïve parameter-wise extension of FedAvg would instead average the factors separately, but this does not generally recover the desired global update^[54]. The product of the averaged factors introduces cross-client combinations that were not optimized by any client^[46,54]. This creates a functional mismatch: The aggregated LoRA factors may have the right shape but fail to represent the intended average model update. Methods such as FLoRA^[54], FedOTAB^[77], and FedPipe^[70] address this issue by preserving or reconstructing LoRA updates in a functionally meaningful space rather than relying on naïve factor averaging: FLoRA^[54] stacks factors to preserve the weighted update, FedOTAB^[77] alternates the optimized factor to reduce communication, and FedPipe^[70] aggregates in the ΔW space before refactorization.

Semantic compatibility and personalization. When clients differ in backbone, adapter type, task, or output space, parameter-level alignment may be insufficient. Related federated foundation-model methods use distillation, representation alignment, or common-subspace mapping to connect heterogeneous client knowledge^[76,78,79]. Structural-bias-aware partial-layer tuning addresses a related layer-coverage heterogeneity problem^[56]. Personalized designs, including dual adapters^[80], clustered aggregation^[79], and Rest-of-World LoRA^[81], further separate globally transferable knowledge from client-specific behavior^[82,83]. These methods improve flexibility but may introduce proxy data, server inference, or multi-module management costs.

Structure-aware aggregation therefore determines whether heterogeneous local adaptations can be made shape-compatible, functionally meaningful, and semantically useful. Effective aggregation should not only align update dimensions, but also preserve the model behavior contributed by different clients and support either a coherent global model or personalized local components.

System orchestration determines how federated LLM adaptation is mapped onto heterogeneous clients, networks, and edge/cloud infrastructure. It controls which clients participate, what adaptation workload they receive, where computation is executed, when updates are incorporated, and how communication and computing resources are allocated.

Participation and role assignment. Client participation can be guided by data value, resource availability, energy status, and expected completion time. Resource-aware selection should assign training roles to clients that can provide useful updates with manageable delay, energy use, or carbon emissions, while maintaining sufficient data coverage and participation fairness^[59]. FedSustain^[59] shows how energy-aware participation can affect both learning and system cost. Its renewable-energy-aware configuration achieved 73.8% accuracy with estimated 13 kWh energy use and 9 kg CO₂ emissions, compared with 71.2%, 21 kWh, and 14 kg CO₂ under random selection. This result shows that sustainability-aware participation can jointly improve learning utility and resource efficiency when client selection is aligned with energy and deployment conditions.

Workload and synchronization control. Orchestration determines how much work each selected client performs and how this workload is coordinated with available communication resources. LoRA rank, trainable layers, numerical precision, sparsity^[37,52,69], local training time, and participation^[62] can be adapted jointly with communication and computing resources^[60,84-86]. HierFedLoRA^[57] provides a representative example by combining near-IID grouping, group-specific aggregation frequency, and fine-tuning depth across 80 Jetson devices, optimizing time-to-accuracy rather than single-round efficiency. Dynamic network conditions further couple these decisions with synchronization. Channel fading, interference, and bandwidth variation can change client communication latency over time, causing communication stragglers in synchronous FL or increased update staleness in asynchronous FL. Accordingly, client participation, bandwidth allocation, adaptation workload, and synchronization frequency can be adjusted according to both device and channel states. Wang et al.^[58] consider device scheduling and bandwidth allocation under time-varying wireless conditions; Pang et al.^[84] jointly optimize client-specific pruning and bandwidth allocation for low-latency federated LLM fine-tuning, and Zhang et al.^[87] adapt the communicated knowledge according to available channel resources. More broadly, hierarchical and asynchronous coordination can reduce cloud traffic or waiting time, but their benefits depend on how network variability, staleness, grouping overhead, and controller complexity are managed.

Computation placement and end-edge-cloud collaboration. Split and offloaded training move part of the adaptation workload from devices to edge or cloud servers. By placing selected Transformer blocks or backward computation outside the client, these methods can reduce client-side memory and computation, while shifting part of the cost to activation exchange, edge/cloud load, and coordination overhead^[61,88-90].

Related wireless split-learning and cloud-edge FL studies provide broader evidence that computation placement and representation design can reshape communication and computing costs across heterogeneous networks^[30,91]. In wireless split-federated LLM systems, these trade-offs become directly coupled with LLM adaptation and communication-resource control. Existing studies jointly optimize split points, LoRA ranks, bandwidth, transmit power, subchannel allocation, or multiple-access schemes^[61,88,89], and some designs further incorporate anti-jamming, sensing assistance, or privacy constraints^[92-94].

Decentralized and multi-participant collaboration. Federated LLM adaptation can extend beyond conventional server-coordinated aggregation to direct collaboration among multiple clients or model instances. Dec-LoRA^[32] coordinates decentralized LoRA updates through topology-based peer interaction, while Balija et al.^[33] consider asynchronous peer-to-peer model exchange. Such decentralized designs can reduce dependence on a central aggregator and alleviate communication congestion at the central node, but shift resource costs toward repeated peer exchange, heterogeneous Device-to-Device (D2D) links, topology management, synchronization, and distributed coordination. Compact PEFT exchange can reduce peer-transfer payloads, while topology-aware peer selection and clustered communication can limit unnecessary transfers. Asynchronous coordination can further reduce waiting for slow or intermittently connected peers, although update staleness must be controlled.

Beyond client-level decentralized federation, broader multi-participant training can involve collaboration among multiple model agents. MAPoRL^[95], for example, studies multi-agent post-training in which multiple LLMs generate, exchange, and discuss responses during joint optimization. Such collaboration introduces additional interaction rounds, token processing, peer communication, and coordination overhead.

Resource-aware orchestration therefore goes beyond client selection by jointly designing participation, workload configuration, synchronization, computation placement, and resource allocation across the end-edge-cloud continuum.

Privacy^[96,97], security, and lifecycle trust constraints reshape the resource-efficiency problem in federated LLM adaptation. Protection mechanisms^[98,99] improve the reliability of distributed adaptation^[100-102], but they also consume the same computation, communication, storage, and utility budgets required for training and aggregation^[103].

Privacy-aware resource efficiency. Privacy-aware mechanisms reduce leakage from shared updates and intermediate representations. Differential privacy protects updates through clipping and perturbation^[10], while encrypted or homomorphic aggregation strengthens confidentiality at the cost of additional communication and cryptographic processing^[65,98]. Selective perturbation can reduce utility loss by protecting sensitive components^[96,97,99], and reconstruction risk depends on what information the protocol exposes, such as gradients, adapters, activations, or split representations^[63,64]. The resource challenge is to provide sufficient privacy protection while controlling communication, computation, and convergence overhead.

Security-aware resource efficiency. Security-aware mechanisms address malicious or biased contributions that may distort the shared model, introduce backdoors, or weaken robustness^[67,68,104]. Robust aggregation, poisoning detection, and adversarially robust prompt-tuning can improve resilience, while adding server-side computation, screening latency, or extra optimization stages^[66,105,106]. Related security-aware resource-allocation studies in broader cloud-edge-terminal systems also show that security requirements can be coupled with communication and computing-resource allocation^[107]. This provides a complementary system-level perspective on security-resource co-design for federated LLMs. In the Shakespeare next-token-prediction experiment, GPT-2 Medium was evaluated with 100 clients and 10 selected per round; aggregation required approximately 3.9 s for a perturbation-based defense, 27.03 s for Multi-Krum, and 63.20 s for FLAME. The sentence-trigger Neurotoxin variant could nevertheless evade both screening methods^[66]. Security-aware resource efficiency requires robustness improvements to be evaluated alongside aggregation latency, server-side computation, and learning utility.

Trust- and lifecycle-aware resource efficiency. Trust- and lifecycle-aware mechanisms focus on whether the federated adaptation process is verifiable, accountable, and auditable over time. Verifiable aggregation strengthens confidence that structured updates are processed correctly, but introduces additional computation and verification overhead. FLAGuard^[11], for example, improves the efficiency of LoRA verification by more than two orders of magnitude relative to prior schemes, with approximately 8% overhead over unverified FLoRA. Provenance tracking and federated unlearning extend trust beyond the training round by requiring historical states, audit trails, rollback, retraining, or deletion verification^[12]. The resource challenge is to integrate verification, provenance, and unlearning with compact federated adaptation, so that accountability can be supported without excessive storage or retraining cost.

Privacy, security, and lifecycle trust protect different aspects of federated LLM adaptation, but they draw on the same resource budgets as training and communication. Resource-efficient design should therefore optimize protection strength, learning utility, and system cost jointly.

Across these mechanism classes, local adaptation reduces the cost of producing client updates; structured exchange reduces communication payload; structure-aware aggregation preserves the meaning of heterogeneous updates; orchestration matches workloads to available resources; and privacy, security, and trust mechanisms define the protection cost required for reliable adaptation. Resource savings should therefore be assessed over the full workflow. A reduction in trainable parameters, message size, or client memory is valuable only when it also improves end-to-end efficiency at a comparable utility target. Table 4 summarizes representative methods according to their mechanism, primary efficiency contribution, and main limitation.

Resource-efficient federated LLM methods should be evaluated by the total cost required to reach a comparable utility target. A complete accounting should include backbone distribution, client-side memory and computation^[1], bidirectional traffic, server-side processing, orchestration and synchronization overhead^[57,59], and privacy/security/trust costs^[11,65]. Studies should report time, energy, and communication cost at comparable utility levels, together with task performance, personalization quality, tail-client performance, fairness, privacy, and robustness.

Transparent reporting is also needed for fair comparison. Key information includes the backbone model, adaptation object, client participation pattern, heterogeneity setting^[35,37,52], hardware platform, network assumptions, peak memory, transferred bytes, wall-clock time, and energy measurement or estimation method^[57,70]. Privacy and security studies should further specify the threat model, server-visible information, and protected object. Resource claims should be labeled as directly measured, modeled or estimated, proxy-based, or unevaluated, while formal guarantees should be reported separately from empirical efficiency results.

Wireless evaluations should additionally report the adopted channel model or network trace, uplink and downlink variability, reliability and retransmission assumptions, synchronization protocol, and mean and tail communication latency. For asynchronous or partial aggregation, update staleness and acceptance or timeout policies should also be specified. These factors are necessary to distinguish performance under nominal link configurations from resource efficiency under realistic time-varying communication conditions.

Utility-normalized Pareto benchmarking. Motivated by multi-objective resource evaluation in FL^[108], we propose a utility-normalized Pareto framework. Let U^* denote a task-specific target utility and $$ T_{i}^{*} $$ the wall-clock time required by method i to first reach this target. For each method that reaches U^* within the evaluation budget, we define its resource vector at the target utility as

(4) $$ R_{i}\left(U^{*}\right)=\left(T_{i}^{*}, E_{i, \text { client }}, E_{i, \text { server }}, C_{i, \text { up }}, C_{\mathrm{i}, \text { down }}, M_{\mathrm{i}, \text { peak }}\right), $$

where E_i,client and E_i,server are the accumulated client- and server/edge-side energy, respectively; C_i,up and C_i,down are the cumulative uplink and downlink traffic, respectively; and M_i,peak is the peak memory, all measured up to $$ T_{i}^{*} $$. At the same U^*, method A Pareto-dominates method B if it is no worse in every reported resource dimension and strictly better in at least one. Methods that do not reach the target should instead report their terminal utility and budget-end resource vector. Fair comparisons should use the same task, backbone, data partition, client-participation setting, hardware/network scenario, and resource-accounting boundary. This framework avoids labeling a method resource-efficient when savings in one dimension are mainly obtained by shifting cost to another.

Several trade-offs recur across the literature. Lower ranks, lower bit widths, or smaller selected training subsets reduce local cost but may affect utility or convergence. Heterogeneous configurations improve participation flexibility but complicate functionally correct aggregation. Privacy, security, and lifecycle-trust mechanisms improve protection while consuming bandwidth, computation, storage, or utility. Favoring fast or well-provisioned clients can improve average efficiency, but may reduce data coverage and participation fairness^[55,59]. Methods should therefore be compared through utility-normalized and Pareto-aware evaluation rather than a single favorable metric.

Adaptive cross-layer control. Federated LLM systems require joint control across model, communication, system, and trust layers. Future methods should adapt participation^[60,85], PEFT configuration, exchange strategy, aggregation rule^[35,37,69], computation placement^[60,85], and protection level^[94] according to changing memory, bandwidth, energy, data value, and risk conditions. Such cross-layer control is important because savings in one stage may otherwise reappear as additional rounds, server reconstruction, communication traffic, or protection overhead.

Realistic-scale validation and joint heterogeneity. Current evidence remains concentrated on BERT-family models, small client populations, homogeneous backbones, and simulated network settings. Future validation should extend to larger generative LLMs, heterogeneous devices^[35], dynamic client availability, variable bandwidth^[61,88,89], asynchronous participation, and explicit client- and server-side energy accounting. Aggregation also remains challenging when clients differ simultaneously in backbone, LoRA rank, layer coverage, adapter type, precision, or model-access mode^{[53-54,56,78]}. Addressing such joint heterogeneity is essential for moving from controlled experiments to deployable federated LLM systems.

Lifecycle-aware and emerging post-training settings. Future federated LLM systems will increasingly face diverse post-training workloads and lifecycle constraints. Multimodal and agentic workloads add visual representations^[109-111], persistent memory, planning, and repeated interaction. In multi-model or multi-agent deployment, additional collaboration costs arise from agent participation, repeated model interaction, token processing, dialogue-history management, and service latency. Wang et al.^[112], for example, study agent-level collaboration strategies and their accuracy-token-cost trade-offs, while large-small model collaboration^[31] illustrates cooperation between models with different capabilities across edge resources. Personalization^[81], provenance and unlearning^[12], and renewable-energy-aware operation^[59,113] further extend resource costs beyond a single adaptation round. Evaluation should therefore consider not only federated adaptation but also the downstream collaboration and operational costs over the lifetime of the adapted model.