We propose a new real-time packet scheduling algorithm for streaming scalable H.264. Our algorithm makes use of a packet importance measure, which we define, that takes into consideration transmission history, channel conditions, and the unique decoding dependencies due to the temporal wavelet encoding. Our algorithm utilizes this importance measure to minimize the expected reconstruction distortion at the decoder under a certain rate constraint. In our experimental results we show gains of more than 3 dB in decoded video quality when transmissions are controlled with our algorithm as compared to existing schedulers.