We address the problem of learning in repeated N-player (as opposed to 2-player) general-sum games. We describe an extension to existing criteria focusing explicitly on such settings. While there have been several criteria proposed recently for evaluating learning algorithms in multi-agent systems, most of this work has focused on the two-player setting. Relatively little work has addressed situations in which there are a mixture of several agents using the algorithm in consideration against opponents using other algorithms. Roughly speaking, our proposed criteria require that the agents employing the particular learning algorithm work together to achieve a joint best-response against a target class of opponents, while guaranteeing they each achieve at least their individual security-level payoff against any possible set of opponents outside this target class. We then provide algorithms that provably meet these criteria for two target classes: stationary strategies and adaptive strate...