One of the important applications of digital signature is anonymous credential or pseudonym system. In these scenarios, it is essential that the identity of the signer is kept secret from any third party, except the trusted authority. The identity in such a system is uniquely identified by the secret key (or the signing key) rather than the public key, since the public key may be repeatedly randomized. This paper formalizes this notion by investigating a new property of digital signatures, called key indistinguishability. In this notion, given a number of digital signatures generated from two known public keys, an adversary cannot determine whether the signing keys used to generate these public keys, and hence the signatures, are the same. This property ensures that the signatures do not leak any information of the signing keys. Observing that existing digital signatures without random oracles do not provide such a property, we fill the gap with the first key indistinguishable signatur...