As we enter the era of petascale computing, system architects must plan for machines composed of tens or even hundreds of thousands of processors. Although fully connected networks such as fattree configurations currently dominate HPC interconnect designs, such approaches are inadequate for such ultra-scale concurriencies due to the superlinear growth of component costs. Traditional low-degree interconnect topologies, such as 3D tori, have reemerged as a competitive solution due to the linear scaling of system components relative to the node count; however, such networks are poorly suited for the requirements of many scientific applications at extreme concurrencies. To address these limitations, we propose HFAST, a hybrid switch architecture that uses circuit switches to dynamically reconfigure lower-degree interconnects to suit the topological requirements of a given scientific application. This work presents several new research contributions. We develop an optimization strategy for...