1201 Larimer Street

View map

Dept. of Mathematical and Statistical Sciences Fall 2023 Seminar Series Presents

Date and time: Monday December 4th 2023 , from 12:30pm to 1:30pm

Location: Student Commons Building room 4125

Speaker: Lionel Eyrault-Dubois, Inria Bordeaux Sud-Ouest

Title: Optimal rematerialization algorithms for Memory-efficient learning

Abstract: The training phase of Deep Neural Networks is often a very memory-intensive procedure, where large amounts of intermediate data have to be kept in memory during one iteration. One possible approach to reduce memory usage is rematerialization, aka gradient checkpointing, where some intermediate data are recomputed when needed rather than kept in memory. This provides a tradeoff between memory usage and recomputation time. In this talk I will present several approaches for the optimization problem, where one wants to minimize the recomputation time given a fixed memory budget. The corresponding algorithms have been implemented in easy-to-use libraries for the PyTorch framework, which can significantly reduce memory usage with reasonable overhead.

Event Details

  • Abdelghani Azri

1 person is interested in this event

Planning an event?

Share your event with our community by using our simple event submission form.

Submit an event
CU in the City logo
CMS Login
University of Colorado Denver Powered by the Localist Community Event Platform © All rights reserved