We define the notions of Azumaya category and Brauer group in category theory enriched over some very general base category V. We prove the equivalence of various definitions, in particular in terms of separable categories or progenerating bimodules. When V is the category of modules over a commutative ring R with unit, we recapture the classical notions of Azumaya algebra and Brauer group and provide an alternative, purely categorical treatment of those theories. But our theory applies as well to the cases of topological, metric or Banach modules, to the sheaves of such structures or graded such structures, and many other examples.