The recent success in Convolutional Neural Network has drawn robust research in academic and industrial. There are numorous theoretical and experimental researches on deep neural network. However, the reason to excellent generalization performance of them are still unknown. Generally, it is used as a black box. The network architecture is based on design experience. The hyperparameters for the network are based on manual tuning and experience. Therefore, the recent studies on meta-learner[1, 2], architecture search[3, 4] give the promising result for better utilization of the tool. However, these method are not readily available for application and understand its fundamental nature. The reason is lacking of theoretical guide and study on underlying fundamental nature of the deep neural network.
In this thesis, we proposed a simple method to improve the generalization robustness of the neural and it provides a better understanding the neural network during training process. We also formulate variance analysis to explain the convergence order of layer in the network. Then, we study the convergence and function of convolutional kernel through a controllable dataset. Lastly, we study the e ectiveness of high cross correlation kernel and overparameterization effect in deep neural network by N - 1 analysis.