ReLU network analysis links Fisher information to spherical harmonics

Researchers have analyzed the Fisher information matrices of simple two-layer ReLU neural networks with random hidden weights. They found that the eigenvalue distribution concentrates significantly on specific eigenspaces, with the first three accounting for nearly all of the matrix's trace. The study identifies these dominant eigenspaces as corresponding to spherical harmonic functions of order two or less, linking this to Mercer decomposition of neural tangent kernels. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Provides theoretical insights into the structure of simple neural networks, potentially informing future model design and analysis.

RANK_REASON Academic paper detailing theoretical analysis of neural network properties. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• ReLU networks
• Fisher information matrices
• spherical harmonic functions
• Mercer decomposition
• neural tangent kernels
• Yoshinari Takeishi

• paper
• other