Abstract. In this paper we introduce a multi-stage decision making procedure where decision makers' opinions are weighted by their contribution to the agreement after they sort alternatives into a fixed finite scale given by linguistic categories, each one having an associated numerical score. We add scores obtained for each alternative using an aggregation operator. Based on distances among vectors of individual and collective scores, we assign an index to decision makers showing their contributions to the agreement. Opinions of negative contributors are excluded and the process is reinitiated until all decision makers contribute positively to the agreement. To obtain the final collective weak order on the set of alternatives, we weigh the scores that decision makers assign to alternatives by indices corresponding to their contribution to the agreement.