While complex systems have been studied now for more than two decades, there still is no agreement on what complexity actually is. This lack of a definition might be a problem when asking questions about the evolution of complexity. In this article criteria against which candidate measures of complexity can be assessed are discussed. The main conclusion of this article is that because of the absence of a basic consensus on what complexity is, there is no criterion that can be used to decide whether or not a proposed measure actually measures complexity. The main recommendation is to abandon complexity as a formal notion; instead, research into the evolution of complexity should use well-understood proxy notions (as is sometimes done in the literature). For the time being ``complexity'' should remain an informal notion. Research into evolutionary trends of these proxy notions might eventually lead to an emergent community consensus on what complexity is.