Subscribe by RSS

In recent years, category theorists have developed a framework to study and generalise gradient descent and neural networks. There is a construction called Para which categorises the notion of parametrised function, and a construction called Optic (a generalisation of former constructions such as Learners and Lenses), which formalises a notion of backpropagation. In this framework, neural networks are seen as maps in a small category NNet, Gradient descent is a functor from Para to Para (Optic), and deep learning can then be defined as a composition of a functor from NNet to Para with Gradient Descent.

In this talk, I will present all these notions, as well as a notion of Deep Dreaming which comes from the optic construction, with nothing as a prerequisite but first year mathematics and basic category theory!

# Math/CS Seminar

**
Wed., Aug. 23, 2023 2:00 p.m.
**

**
Location: CW 307.20
**

**Speaker:** Sacha Ikonicoff, University of Ottawa

**Title: **Gradient descent and neural networks: a categorical approach (84 kB)

**Abstract: **

In recent years, category theorists have developed a framework to study and generalise gradient descent and neural networks. There is a construction called Para which categorises the notion of parametrised function, and a construction called Optic (a generalisation of former constructions such as Learners and Lenses), which formalises a notion of backpropagation. In this framework, neural networks are seen as maps in a small category NNet, Gradient descent is a functor from Para to Para (Optic), and deep learning can then be defined as a composition of a functor from NNet to Para with Gradient Descent.

In this talk, I will present all these notions, as well as a notion of Deep Dreaming which comes from the optic construction, with nothing as a prerequisite but first year mathematics and basic category theory!