This work aims to propose an interactive method for a iconic and textual annotation of digital mammograms. The suggested annotation tool consists of a semantic network to represent all information about a set of mammograms obtained from experts in a structured manner based on the BIRADS standard, a correlated XML file system for persistence, data exchange and storage, and a graphical user interface, allowing a combination of iconic and textual annotation. This approach allows a complete annotation of all findings in a set of mammograms in a structured way, which is also machine readable and interpretable. Thus, systematically annotated image data sets can be used for structured information retrieval as well as case based reasoning systems for computer assisted diagnosis (CAD). Furthermore, such a digital annotation system can be used to replace paper reports and hence avoid unnecessary media breaks during the process of the examination and documentation.