Sketch recognizers are an important part of the design of natural interfaces for many domains. However, current recognition technology is generally quite crude and difficult to work with. This paper explores Blostein’s concept of “literate” systems [1] in the context of recognizing handwritten mathematics. We describe some of the difficulties arising in complex sketch recognition domains, and offer some ideas for handling these difficulties that we have found useful in practice. Our suggestions focus on problems related to managing ambiguity, evaluating system performance, and obtaining data for training and testing recognizers.