Can conventional differential unitary space time modulation (DUSTM) be applied when there is an unknown carrier frequency offset (CFO)? This paper answers this question affirmatively and derives the necessary maximum likelihood (ML) detection rule. The asymptotic performance of the proposed ML rule is analyzed, leading to a code design criterion for DUSTM by using the modified diversity product. The resulting proposed decision rule is a new differential modulation scheme in both the temporal and spatial domains. Two sub-optimal multiple-symbol decision rules with improved performance are also proposed. For the efficient implementation of these, we derive a modified bound intersection detector (BID), a generalization of the previously derived optimal BID for the conventional DUSTM. The simulation results show that the proposed differential modulation scheme is more robust against CFO drifting than the existing double temporal differential modulation.