In statistical modelling, an investigator must often choose a suitable model among a collection of viable candidates. There is no consensus in the research community on how such a comparative study is performed in a methodologically sound way. The ranking of several methods is usually performed by the use of a selection criterion, which assigns a score to every model based on some underlying statistical principles. The fitted model that is favoured is the one corresponding to the minimum (or the maximum) score. Statistical significance testing can extend this method. However, when enough pairwise tests are performed the multiplicity effect appears which can be taken into account by considering multiple comparison procedures. The existing comparison procedures can roughly be categorized as analytical or resampling based. This paper describes a resampling based multiple comparison technique. This method is illustrated on the estimate of the number of hidden units for feed-forward neural...