Recently, Sun proposed a private-key encryption scheme based on the product codes with the capability of correcting a special type of structured errors. In this paper, we present a novel method to improve the information rate of Sun's scheme. This method uses the added error vector to carry additional information. Some information bits are mapped into an error vector with the special structure to be added to a codeword. Once the error vector can be identified, the additional information can be recovered. Key words: cryptography, data security, private-key cryptosystem, encryption, product codes.