This paper investigates the level of agreement in applying metadata and highlights results from the analysis of agreement between and within user communities. Problems in applying and sharing metadata are diagnosed by examining the reliability of metadata application by domain experts, especially when contrasted to researchers. Three main problems are discussed: issues pertaining to data, the metadata and the application process. Strategies are proposed to resolve these issues. Our research highlights the need for diagnostic tools that identify the sources of poor agreement and therefore guide steps to improve metadata application and sharing.