PNAS publishes our paper on the barriers of deep learning

Our paper ‘The difficulty of computing stable and accurate neural networks: On the barriers of deep learning and Smale’s 18th problem’ has been published in the Proceedings of the National Academy of Sciences, USA.

In this work, we show (mathematically) that there are certain well-conditioned problems in scientific computing where one can prove the existence of neural networks (NNs) which can solve the problems, yet there will not necessarily exist any (even randomized) algorithm which can compute these NNs. Moreover, only under certain conditions will it be possible to compute NNs for solving these problems. Given these conditions, we design such an algorithm, and we introduce fast iterative restarted networks (FIRENETs) which we both prove and numerically verify are stable.