Network Dynamics, Connectivity, and Optimization: Exploring Synchronization of Coupled Oscillators, Network Flows and Function Space of Deep Neural Networks

Dynamical processes on networks are common and intriguing phenomena; they can be used to model dynamics of interacting many-body systems, to describe the behaviors of iteration algorithms, or even to depict the evolution of compositional functions. In this thesis, we investigate problems from different topics in relation to network dynamics and optimization, and draw connections to the network connectivities. Optimization methods are developed to enhance the synchronization stability of coupled oscillators, which has potential applications in improving the stability of power networks. Message passing algorithms are derived for solving the quadratic network flow optimization problems, which is applicable in Laplacian systems. Generating functional analysis from statistical physics of disordered systems are employed to examine the macroscopic behaviors of deep neural networks, revealing interesting phenomena of the function space.