We consider the problem of learning a finite automaton M of n states with input alphabet X and output alphabet Y when a teacher has helpfully or randomly labeled the states of M using labels from a set L. The learner has access to label queries; a label query with input string w returns both the output and the label of the state reached by w. Because different automata may have the same output behavior, we consider the case in which the teacher may “unfold” M to an output equivalent machine M and label the states of M for the learner. We give lower and upper bounds on the number of label queries to learn the output behavior of M in these different scenarios. We also briefly consider the case of randomly labeled automata with randomly chosen transition functions.