Hybrid of Neural Network (NN) and Hidden Markov Model (HMM) has been popular in word recognition, taking advantage of NN discriminative property and HMM representational capability. However, NN does not guarantee good generalization due to Empirical Risk minimization (ERM) principle that it uses. In our work, we focus on online word recognition using the support vector machine (SVM) for character recognition. SVM's use of structural risk minimization (SRM) principle has allowed simultaneous optimization of representational and discriminative capability of the character recognizer. We evaluated SVM in isolated character recognition environment using IRONOFF and UNIPEN character database. We then demonstrate the practical issues in using SVM within a hybrid setting with HMM for word recognition by testing the hybrid system on the IRONOFF word database and obtained commendable results.