We present Binary Refactoring: a software engineering technique for improving the implementation of programs without modifying their source code. While related to regular refactoring in preserving a program's functionality, binary refactoring aims to capture modifications that are often applied to source code, although they only improve the performance of the software application and not the code structure. We motivate binary refactoring, present a binary refactoring catalogue, describe the design and implementation of BARBER--our binary refactoring browser for Java, and demonstrate the usefulness of binary refactoring through a series of benchmarks. Categories and Subject Descriptors D.2.7 [Software Engineering]: Distribution, Maintenance, and Enhancement--Restructuring, reverse engineering, and reengineering; D2.3 [Software Engineering]: Coding Tools and Techniques--Object-oriented programming; D.3.4 [Programming Languages]: Processors--Optimization General Terms Performance, L...