The very nature of implementing and evaluating fully distributed algorithms or protocols in application-layer overlay networks involves certain programming tasks that are at best mundane and tedious — and at worst challenging — even at the application level. These include multi-threaded message switching engines at the application layer, failure detections and reactions, measurements of QoS metrics such as loss rates and per-link throughput, application deployment and terminations, debugging and monitoring facilities, virtualizing distributed nodes, as well as emulating resource bottlenecks and asymmetric network connections. Unfortunately, such a significant set of programming tasks is inevitable when implementing a diverse variety of application-layer overlay protocols and algorithms. In this paper, we present iOverlay, a lightweight and high-performance middleware infrastructure that addresses these problems in a novel way by providing clean, well-documented layers of middlewar...