An acoustic-phonetics based word-independent technique which uses syllable context for classifying the lexical syllable stress of spoken English words is presented. Nucleus based clustering is remarkably successful in moving from word-dependent syllable stress classification which is intrinsically not scalable to word-independent classification. This however is not possible without an inherent drop in accuracy due to the loss of important contextual information of the syllables. An approach based on incorporating the left and the right context-ID of the syllable nucleus is proposed which results in a 10% improvement in word-level accuracy for word-independent syllable stress classification. The proposed approach exhibits performances comparable to that of the best performing word-dependent classifiers without suffering from the latter’s scalability issues. A 7% improvement in the syllable level accuracy is also reported.