In this paper, we propose RAVITAS, a framework for realistic voice chat among multiple users in a virtual space reproducing the cocktail party effect. RAVITAS utilizes context-aware voice filtering (CAVF), pub/sub-based locality management, and controlled voice streaming to achieve this effect. Our preliminary experiments show that RAVITAS achieves satisfactory perception-based subjective results for a small group of users.