In this paper, a new model of Probabilistic Model-Building Genetic Algorithms (PMBGAs), Distributed PMBGA (DPMBGA), is proposed. In the DPMBGA, the correlation among the design variables is considered by Principal Component Analysis (PCA) when the offsprings are generated. The island model is also applied in the DPMBGA for maintaining the population diversity. Through the standard test functions, some models of DPMBGA are examined. The DPMBGA where PCA is executed in the half of the islands can find the good solutions in the problems whether or not the problems have the correlation among the design variables. At the same time, the search capability and some characteristics of the DPMBGA are also discussed.