Atos and Federated Learning

In 2023, it is almost impossible to see the news, read a newspaper or check social networks without being flooded with new usages of Artificial Intelligence (AI). Nowadays AI is everywhere, having an exponential growth that seems not to have limits: software developers have the support of GitHub Copilot and other AI-powered assistants to speed up and improve the development process, Stable Diffusion is helping to generate amazing high-quality images which are difficult to differentiate from real ones, ChatGPT has a plethora of applications that go from explaining complex topics to create content in multiple languages. And this is just the beginning, AI has started also to penetrate enterprise applications in domains like manufacturing, retail, healthcare or logistics.

Most of these applications are based on training large models with huge datasets, which in some cases must be even labelled if supervised techniques are going to be applied. The process requires having access to specialised computing infrastructures including hardware accelerators during long periods of time and supposing high costs, energy consumption and consequently, carbon footprint. Having access to this kind of information and resources is a barrier that cannot be overcome by many companies, which limits the democratisation of AI.

At the same time, in some cases, the creation of large datasets is not possible due to the need to comply with privacy-preserving and data protection regulations like the European General Data Protection Regulation (GDPR). This is the case for many applications in the healthcare domain, in which sensitive data from patients cannot leave the hospitals’ software systems. Just to give one example of how this subject can restrict the development and usage of AI systems, ChatGPT has been forbidden in Italy, pushing OpenAI to create a new mode in which users are able to disable the conversations’ history. Other domains in which data may be personal or sensitive are also affected by this kind of constraint, e.g., banking, insurance, and home appliances.

Nevertheless, limitations are not only related to being compliant with regulations. In the case of services addressing companies or industries, the information needed to train a Machine Learning model may include data collected from internal processes or confidential knowledge, that are part of the core elements for the sustainability and competitiveness of the company. Moreover, the need to transfer large amounts of information and maintain centralised storage systems could drive scalability issues.

In order to overcome these obstacles, Federated Learning (FL) has emerged as a new paradigm that enables Machine Learning models to be trained in a decentralised manner that preserves data privacy. Instead of creating a single and centralised dataset with all the information, the models are trained locally with partial datasets that do not leave the perimeter in which they are collected, e.g., hospitals, IoT devices, and factories. Then, the parameters of the resulting models are sent to a central server that is responsible for aggregating the individual contributions and applying different types of algorithms to create a global model. The result is then pushed back to the local instances so it can be served to provide inferences. The whole process can be repeated multiple times to adapt the model to fresh data and continuously improve the performance.

In some situations, even the parameters of the local models could be used to extract valuable information from the local data. Thus, Federated Learning can be combined with Privacy Enhancing Technologies (PET) to also guarantee the privacy of the individual models. Examples of PET techniques are Differential Privacy, Homomorphic Encryption or PATE (Private Aggregation of Teacher Ensembles).

In ALCHIMIA, we will explore how to leverage Federated Learning to implement new services that help to improve the quality and sustainability of metallurgical manufacturing processes. The project will develop a flexible and innovative Federated Learning framework that will be applied to the use cases of Celsa and Fonderia di Torbole.

Stay tuned to receive new updates about this amazing journey!!

P.S. This blog post has been written by the human team of Atos participating in the ALCHIMIA project. No AI systems have been involved! 😊