Clustering algorithms such as k-means, the self-organizing map (SOM), or Neural Gas (NG) constitute popular tools for automated information analysis. Since data sets are becoming larger and larger, it is vital that the algorithms perform efficient for huge data sets. Here we propose a parallelization of patch neural gas which requires only a single run over the data set and which can work with limited memory, thus it is very efficient for streaming or massive data sets. The realization is very general such that it can easily be transferred to alternative prototype-based methods and distributed settings. Approximately linear relative speed-up can be observed depending on the number of processors.