This paper describes a method of supervised learning based on forward selection branching. This method improves fault tolerance by means of combining information related to generalization performance and fault tolerance. The method presented focuses on the evolutive nature of the learning algorithm of Radial Basis Function Networks and employs optimization techniques to control the balance between the approximation error with and without faults. The technique developed is empirically analyzed and provides a simple and efficient means of learning fault tolerance. This is illustrated by examples taken from different classification and function approximation problems.