A ternary self-dual code can be constructed from a Hadamard matrix of order congruent to 8 modulo 12. In this paper, we show that the Paley-Hadamard matrix is the only Hadamard matrix of order 32 which gives an extremal self-dual code of length 64. This gives a coding theoretic characterization of the Paley-Hadamard matrix of order 32.