The comparison of manually annotated medical images can be done using the comparison of keywords in a lexical way or using the existing medical thesauri to calculate semantic similarity. In this paper, first we introduce the KWSim measure, a fully automated technique of measuring semantic similarity by mapping concepts(keywords) to different medical thesauri and examining the "is-a" relation type. A keyword vector similarity is also presented, based on the KWSim measure. Our approach is implemented using MeSH, ICD-10 and SNOMED CT thesauri and compared with two other existing approaches. We illustrate our method with a real time online annotation assistant. R