By using joint modulation and customized constellation set, we show that Quasi-Orthogonal Space-Time Block Code (QO-STBC) can be used to form a new differential space-time modulation (DSTM) scheme to provide full transmit diversity with non-coherent detection. Our new scheme can provide higher code rate than existing DSTM schemes based on Orthogonal STBC. It also has a lower decoding complexity than the other DSTM schemes, such as those based on Group Codes, because it only requires a joint detection of two complex symbols. We derive the design criteria for the customized constellation set and use them to construct a constellation set that provides a wide range of spectral efficiency with full diversity and maximum coding gain. Keywords - differential space-time modulation; low decoding complexity; non coherent detection; quasi-orthogonal space-time block code.