Statistical mechanics of neural networks: what are the differences between wide and narrow basins?

We consider training noise in neural networks as a means of tuning the structure of retrieval basins, and study how learning and retrieving properties depend on it. The stability of the replica symmetric solution and the correlation in the weight space indicate that neural networks can be roughly classified into Hebbian-like and MSN-like (MSN meaning the maximally stable network). Re-entrant retrieval, noise robustness, selectivity, damage spreading and activity distribution all illustrate the differences in retrieval behaviours arising from the different basin structures.