Continued device scaling enables microprocessors and other systems-on-chip (SoCs) to increase their performance, functionality, and hence, complexity. Simultaneously, relentless scaling, if uncompensated, degrades the performance and signal integrity of on-chip metal interconnects. These systems have therefore become increasingly communications-limited. The communications-centric nature of future high performance computing devices demands a fundamental change in intra- and inter-chip interconnect technologies. Optical interconnect is a promising long term solution. However, while significant progress in optical signaling has been made in recent years, applying conventional packetswitching interconnect architecture to optical networks require repeated E/O and O/E conversions that significantly diminish the advantages of optical signaling. In this paper, we propose to leverage a suite of newly-developed or emerging devices, circuits, and optics technologies to build a fully distribute...