PROJECT TITLE :

Ternary Compression for Communication-Efficient Federated Learning

ABSTRACT:

In many real-world applications, it is essential to acquire knowledge over vast amounts of data that are stored in a variety of locations. Nonetheless, there are many obstacles to overcome when it comes to data sharing as a result of the rising concerns regarding privacy and security brought on by the proliferation of smart mobile devices and Internet of Things (IoT) devices. Federated learning offers a potential solution to privacy-preserving and secure Machine Learning by means of jointly training a global model without uploading data distributed on multiple devices to a central server. This type of learning enables machine learning to take place in a secure environment. However, the majority of the currently published research on federated learning uses machine learning models with full-precision weights. Almost all of these models contain a large number of redundant parameters that do not have to be sent to the server, but doing so would consume an excessive amount of Communication costs. We propose a federated trained ternary quantization (FTTQ) algorithm as a solution to this problem. This algorithm, which optimizes the quantized networks on the clients through a self-learning quantization factor, is what we call for. There are theoretical demonstrations presented here that demonstrate the convergence of quantization factors, the unbiasedness of FTTQ, and a reduced weight divergence. We propose a ternary federated averaging protocol (T-FedAvg), which is based on FTTQ, with the goal of reducing the amount of communication that occurs between upstream and downstream federated learning systems. Our results demonstrate that the proposed T-FedAvg is effective in reducing communication costs and can even achieve slightly better performance on non-IID data in comparison to the canonical federated learning algorithms. These findings were obtained by conducting empirical experiments to train widely used Deep Learning models on publicly available data sets.