Motion information scalability is an important requirement for a fully scalable video codec, especially in low bit rate or small resolution decoding scenarios, for which the fully scalable motion model (SMM) has been proposed. SMM can collaborate flawlessly with other scalabilities, such as spatial, temporal and quality, in a scalable video codec and it performs better than nonscalable motion model. To further improve the SMM, this paper further extends the algorithm to support hierarchical B frame structure and bidirectional or multidirectional motion estimation and, moreover, provides the corresponding rate distortion optimized estimation. Several simulation results are presented to verify the improvements of this extension.