We describe a set of necessary conditions for a given graph to be the visibility graph of a simple polygon. For every graph satisfying these conditions we show that a uniform rank 3 oriented matroid can be constructed in polynomial time, which if affinely coordinatizable yields a simple polygon whose visibility graph is isomorphic to the given graph.