Super-resolution of the undersampled and subpixel shifted image sequence by a neural network

Numerous approaches to super-resolution (SR) of sequentially observed images (image sequence) of low resolution (LR) have been presented in the past two decades. However, neural network methods are almost ignored for solving SR problems. This is because the SR problem traditionally has been regarded as the optimization of an ill-posed large set of linear equations. A designed neural network based on this has a large number of neurons, thereby requiring a long learning time. Also, the deduced cost function is overly complex. These defects limit applications of a neural network to an SR problem. We think that the underlying meaning of the SR problem should refer to super-resolving an imaging system by image sequence observation, instead of merely improving the image sequence itself. SR can be regarded as a pattern mapping from LR to SR images. The parameters of the pattern mapping can be learned from the imaging process of the image sequence. This article presents a neural network for SR based on learning from the imaging process of the image sequence. In order to speed up the convergence, we employ vector mapping to train the neural network. A mapping vector is composed of some neighbor subpixels. Such a well-trained neural network has powerful generalization ability so that it can be used directly to estimate the SR image of the other image sequences without learning again. Our simulations show the effectiveness of the proposed neural network.