A key feature in population based optimization algorithms is the ability to explore a search space and make a decision based on multiple solutions. In this paper, an incremental learning strategy based on a dynamic particle swarm optimization (DPSO) algorithm allows to produce heterogeneous ensembles of classifiers for video-based face recognition. This strategy is applied to an adaptive classification system (ACS) comprised of a swarm of fuzzy ARTMAP (FAM) neural network classifiers, a DPSO algorithm, and a long term memory (LTM). The performance of this ACS with an ensemble of FAM networks selected among local bests of the swarm, is compared to that of the ACS with the global best network under different incremental learning scenarios. Performance is assessed in terms of classification rate and resource requirements for incremental learning of new data blocks extracted from real-world video streams, and are given along with reference kNN and FAM classifier optimized for batch learnin...