New Data Structures for Matrices and Specialized Inner Kernels: Low Overhead for High Performance