—Significant research has been done on recognizing the daily activities using acceleration data but few works have focused on classifying the movements comprising an activity due to the shorter time scales of the movements compared to that of an activity. Recognizing the individual movements within an activity can help improve the activity recognition on the whole by using the extra information from the movement granularity. Also, for many applications such as rehabilitation, sports medicine, geriatric care, and health/fitness monitoring the importance of movement recognition cannot be overlooked. Hence, in this paper a novel machine learning algorithm using body area networks is proposed that can on the fly, jointly classify the type of movements, and starting and finishing instant of each movement within an activity. A case study on the best set of features and minimum number of accelerometers needed to correctly classify movements within a smoking activity is also presented.