As an important attribute directly influencing the steganographic scheme, the embedding efficiency is defined as the average number of random data bits per one embedding change. In this paper, we propose a novel approach to improve the embedding efficiency of LSB matching based on the sum and difference covering set (SDCS) of finite cyclic group. We show that the suitable choice of SDCS will lead to a new steganographic scheme which is more efficient than LSB matching. Then we illustrate that the new scheme keeps the statistical imperceptibility of LSB matching. The detailed constructions of SDCS are given and some related problems are also discussed.