Last year's Physics Nobel prize recognized the contributions of Giorgio Parisi to the description of complex phenomena not only in physics but also in other fields such as mathematics or machine learning. In this talk, I will describe some of the connections between the theory of disordered magnets called spin glasses and computational questions such as learning with artificial neural networks. I will describe both some of the classical results from statistical physics as well as some recent progress in connection to current open questions in the theory of deep learning.