To access automated voice services, Voice over IP (VoIP) users sometimes are required to provide their Personal Identification Numbers (PIN) for authentication. Therefore when they enter PINs, their user-agents generate packets for each key pressed and send them immediately over the networks. This paper shows that a malicious intermediary can recover the inter-keystroke time delay for each PIN input even if the standard encryption mechanism has been applied. The inter-keystroke delay can leak information of what has been typed: Our experiments show that the average search space of a brute force attack on PIN can be reduced by around 80%.