We present the procedures we implemented to carry out system oriented evaluation of a syntax-based word aligner --ALIBI. We take the approach of regarding cross-corpus evaluation as part of system oriented evaluation assuming that corpus type may impact alignment performance. We test our system on three English