Hattie Zhou discusses her study on the importance of signs in neural networks, finding that just keeping the sign of weights was enough to achieve comparable performance. This suggests that the model is not highly sensitive to perturbations in the initial conditions, particularly changes in magnitude. The concept of a large basin of attraction is explored, raising questions about the extent to which different magnitudes of initializations can still lead to better than chance performance.
