This paper presents ACE (Autonomous Classification Engine), a framework for using and optimizing classifiers. Given a set of feature vectors, ACE experiments with a variety of classifiers, classifier parameters, classifier ensembles and dimensionality reduction techniques in order to arrive at a good configuration for the problem at hand. In addition to evaluating classification methodologies in terms of success rates, functionality is also being incorporated into ACE allowing users to specify constraints on training and classification times as well as on the amount of time that ACE has to arrive at a solution. ACE is designed to facilitate classification for those new to pattern recognition as well as provide flexibility for those with more experience. ACE is packaged with audio and MIDI feature extraction software, although it can certainly be used with existing feature extractors. This paper includes a discussion of ways in which existing general-purpose classification software can...