Comments on Sejnowski's "The unreasonable effectiveness of deep learning in artificial intelligence" [arXiv:2002.04806]

Abstract

Terry Sejnowski's 2020 paper [arXiv:2002.04806] is entitled "The unreasonable effectiveness of deep learning in artificial intelligence". However, the paper itself doesn't attempt to answer the implied question of why Deep Convolutional Neural Networks (DCNNs) can approximate so many of the mappings that they have been trained to model. There are detailed mathematical analyses, but this short paper attempts to look at the issue differently, considering the way that these networks are used, the subset of these functions that can be achieved by training (starting from some location in the original function space), as well as the functions that in reality will be modelled.