Learning by a neural net in a noisy environment—the pseudo-inverse solution revisited

A recurrent neural net is studied that learns a set of patterns {ξμ} in the presence of noise. The learning rule is of a Hebbian type. It is well-known that, if noise is absent during the learning process, the resulting final values of the weights wij correspond to what is usually referred to as the pseudo-inverse solution of the fixed point equation associated with the learning rule. In the limit of vanishing noise, the expressions derived in this article for the expectation value of the weights do not converge to the usual pseudo-inverse solution, in contrast to what one might expect. Since biological systems in general are noisy, the usual pseudo-inverse solution is less realistic, in principle, than the solution found in this article.