Behavioral Signal Processing aims at automating behavioral coding schemes such as those prevalent in psychology and mental health research. This paper describes methods to quantify approach-andavoidance (AA) behavior in human dyadic interactions, usually described manually with ordinal labels, using visual (motion capture) and audio based features. We propose a novel ordinal regression (OR) algorithm and its extension applicable to time series. The proposed algorithm transforms the OR to multiple binary classification problems, solves them by independent score-outputting classifiers and fits the cumulative logit logistic regression model with proportional odds (CLLRMP) the classifier score vectors. The time series extension treats labels as states of the hidden Markov model with likelihood based on the probabilistic CLLRMP output. We compare performances of the proposed algorithm applying the weighted binary SVMs the second step (SVM-OLR), its extension (HMMSVM-OLR) and the baseli...