When working with diagrams in visual environments like graphical diagram editors, diagrams have to be represented by an internal model. Graphs and hypergraphs are well-known concepts for such internal models. This paper shows how hypergraphs can be uniformly used for a wide range of different diagram types where hyperedges are used to represent diagram components as well as spatial relationships between components. This paper also proposes a procedure for translating diagrams into their hypergraph model, i.e., a graphical scanner, and a procedure to check the hypergraph against a hypergraph grammar defining the diagrams' syntax, i.e., a parsing procedure. Such procedures are necessary to make use of such a hypergraph model in visual environments that support free-hand editing where the user can modify diagrams arbitrarily.