| dc.description.abstract | Federated learning (FL) is a promising distributed learning paradigm for protecting data privacy. In FL, edge devices collaboratively train machine learning (ML) models under the orchestration of a parameter server (PS), which only requires exchanging local learning models/gradients among devices and the PS instead of local private data. However, implementing FL in real-world wireless networks faces several challenges, e.g., data heterogeneity, device heterogeneity, limited wireless resources, and unreliable wireless channels. This thesis presents four original contributions to address these challenges by jointly designing the learning mechanism and wireless networks. Firstly, a joint representativity and latency-aware device scheduling scheme is proposed to address the limited wireless resources for FL. Specifically, we theoretically revealed that the learning performance is degraded by the difference between the aggregated gradient of scheduled devices and the full participation gradient. Based on this, the proposed scheme aims to find a subset of representative devices and their corresponding pre-device stepsizes to approximate the full participation gradient while capturing the trade-off between learning performance and latency for FL. Compared to existing device scheduling algorithms, the proposed representativity-aware device scheduling algorithm improves 6.7% and 4.02% accuracies on two typical datasets under heterogeneous local data distributions, i.e., MNIST and CIFAR-10, respectively. In addition, the proposed latency- and representativity-aware scheduling algorithm saves over 16% and 12% training time for MNIST and CIFAR-10 datasets than the scheduling algorithms based on either latency or representativity individually. Secondly, a novel knowledge-aided FL (KFL) framework is proposed to address the data heterogeneity and reduce the communication costs, which aggregates light high-level data features, namely knowledge, in the per-round learning process. This framework allows devices to design their machine-learning models independently and reduces the communication overhead in the training process. We theoretically revealed that allocating more resources in the early rounds achieves better learning performance when the total available resources are fixed during the entire learning course. Based on this, a joint device scheduling, bandwidth allocation, and power control approach is developed to optimize the learning performance of FL under limited energy budgets of devices. Experimental results on two typical datasets (i.e., MNIST and CIFAR-10) under highly heterogeneous local data distributions show that the proposed KFL is capable of reducing over 99% communication overhead while achieving better learning performance than the conventional model aggregation-based algorithms. In addition, the proposed device scheduling algorithm converges faster than the benchmark scheduling schemes. Thirdly, a novel FL framework, namely FL with gradient recycling (FL-GR), is proposed to tackle the negative effects of unreliable wireless channels and constrained resources on FL. FL-GR recycles the historical gradients of unscheduled and transmission-failure devices to improve the learning performance of FL. We theoretically revealed that minimizing the average square of local gradients' staleness (AS-GS) helps improve learning performance. Based on this, a joint device scheduling, resource allocation and power control approach is proposed to minimize the AS-GS for global loss minimization. Compared to the FL algorithm without gradient recycling, FL-GR achieves over 4% accuracy improvement. In addition, the proposed device scheduling algorithm outperforms the benchmarks in convergence speed and test accuracy. Finally, a novel adaptive model pruning-based FL (AMP-FL) framework is proposed to address the device heterogeneity, where the edge server dynamically generates sub-models by pruning the global model for devices' local training to adapt their heterogeneous computation capabilities and time-varying channel conditions. We introduced an age of information (AoI) metric to characterize the staleness of local gradients and theoretically analyzed the convergence behaviour of AMP-FL. The convergence bound shows that scheduling devices with large AoI of gradients and pruning the model regions with small AoI for devices can improve learning performance. Inspired by this, a joint device scheduling, model pruning, and resource allocation scheme is developed to enhance the learning performance of FL. Experimental results show that the proposed AMP-FL is capable of achieving 1.9x and 1.6x speed up for FL on MNIST and CIFAR-10 datasets in comparison with the FL schemes with homogeneous model settings. | en_US |
