Currently a wide range of different adaptive and intelligent system solutions are being proposed for use in self-managing or autonomic networks. However, there are few means by which such proposals can be compared. This paper proposes that a benchmark be developed for autonomic systems so that progress in this field can be more systematically evaluated. Our approach assumes that autonomic systems make use of and thus expose a knowledge based representation of the service they offer, the context they react to and the governance to which they are subject. This position paper focuses on some of the issues that arise when formulating a benchmark for autonomic communications and is intended to form the basis for further discussion in the area.