— We study large population stochastic dynamic games where each agent receives influences from multi-classes of agents according to intra- and inter-subpopulation cost coupling. The NCE principle developed in our previous works gave decentralized asymptotic Nash strategies; however, its solubility depends on a conservative fixed point analysis which does not lead to easy computation of the solution. In this paper we apply a different algebraic approach via a state space augmentation, and it is convenient for practical computation involving first a set of algebraic Riccati equations subject to consistency constraints and next a set of ordinary differential equations.