SIP (Session Initiation Protocol) is a signaling protocol standardized by IETF, aiming to manage the multimedia transmission sessions among different parties. This paper illustrates an adaptive multimedia transmission system for wired and wireless networks based on SIP with protocol selection mechanism for a certain level of QoS guarantee. In our system, SIP is not only used for call setup signaling but also for carrying the information in the protocol selection. Using Agent Server, our system can select the most suitable protocol for adapting different situations intelligently during the connections and data buffering service is also provided for various media data flows between the end users with acceptable QoS level without any interruption and disconnection regardless of types of devices, platforms and protocols used.