Distributed Deep Learning Systems

Machine learning systems are often conventionally designed for centralized processing in that they first collect data from distributed sources and then execute algorithms on a single server. Due to the limited scalability of processing large amount of data and the long latency delay, there is a strong demand for a paradigm shift to distributed or decentralized ML systems which execute ML algorithms on multiple and in some cases even geographically dispersed nodes.

The aim of this course is to let students learn how to design and build distributed ML systems via paper reading, presentation, and discussion; We provide a broad overview on the design of the state-of-the-art distributed ML systems, with a strong focus on the scalability, resource efficiency, data requirements, and robustness of the solutions. We will present an array of methodologies and techniques that can efficiently scale ML analysis to a large number of distributed nodes against all operation conditions, e.g., system failures and malicious attacks. The specific course topics are listed below.

The course materials will be based on a mixture of classic and recently published papers.

Details

Code 62122
Type Course
ECTS 5
Site Neuchâtel
Track(s) T6 – Data Science
Semester S2025

Teaching

Learning Outcomes
  • To understand design principles of distributed and federated learning systems
  • To analyze distributed and federated ML in terms of the scalability and accuracy-performance tradeoff
  • To understand and implement horizontal and vertical federated learning systems
  • To understand and implement federated learn systems on different models, e.g., classification and generative models
  • To understand and analyze vulnerabilities and threat to federated learning systems, e.g., data poison attacks and freerider attacks
  • To design and implement defense strategies against adversarial clients in federated systems
Lecturer(s) Lydia Chen
Language english
Course Page

The course page in ILIAS can be found at https://ilias.unibe.ch/goto_ilias3_unibe_crs_3102287.html.

Schedules and Rooms

Period Weekly
Schedule Monday, 08:15 - 12:00
Location UniNE, Unimail
Room E213

Additional information

Comment

First Lecture
The first lecture will be announced later.