Some Issues About the Generalization of Neural Networks for Time Series Prediction

Abstract. Some issues about the generalization of ANN training are investigated through experiments with several synthetic time series and real world time series. One commonly accepted view is that when the ratio of the training sample size to the number of weights is larger than 30, the overfitting will not occur. However, it is found that even with the ratio higher than 30, overfitting still exists. In cross-validated early stopping, the ratio of cross-validation data size to training data size has no significant impact on the testing error. For stationary time series, 10% may be a practical choice. Both Bayesian regularization method and the cross-validated early stopping method are helpful when the ratio of training sample size to the number of weights is less than 20. However, the performance of early stopping is highly variable. Bayesian method outperforms the early stopping method in most cases, and in some cases even outperforms no-stop training when the training data set is la...

• Type: Conference
• Year: 2005
• Where: ICANN
• Authors: Wen Wang, Pieter H. A. J. M. van Gelder, J. K. Vrijling
• Keywords: Cross-validated Early Stopping, Early Stopping, ICANN 2005, Training Sample Size