In this article we claim that the linguistic-centred view within hypermedia systems needs refinement through a semiotic-based approach before real interoperation between media can be achieved. We discuss the problems of visual signification for images and video in dynamic systems, in which users can access visual material in a non-linear fashion. We describe how semiotics can help overcome such problems, by allowing descriptions of the material on both denotative and connotative levels. Finally we propose an architecture for a dynamic semiotic-aware hypermedia system. Keywords Hypermedia, hypertext, multimedia, denotation, connotation, semiotics, ontology, conceptual information space.