Abstract—This paper studies a constrained optical signal-tonoise ratio (OSNR) optimization problem in optical networks from the perspective of system performance. A system optimization problem is formulated with the objective of achieving an OSNR target for each channel while satisfying the total power constraint. In order to establish existence of a unique optimal solution, the conditions are derived, which can be used as a basis for an admission control scheme. The original problem is then converted to a relaxed system problem by using a barrier function and solved by a distributed iterative algorithm. Next, the system optimization framework developed is compared to the game theoretic one in [1]. The effects of parameters in both formulations are investigated to study efficiency of Nash equilibria in the OSNR game and pricing mechanisms affecting overall system performance. The theoretical analysis is supported by numerical simulations and experiments conducted on an optical fibe...