Structured linear algebra techniques enable one to deal at once with various types of matrices, with features such as Toeplitz-, Hankel-, Vandermonde- or Cauchy-likeness. Following Kailath, Kung and Morf (1979), the usual way of measuring to what extent a matrix possesses one such structure is through its displacement rank, that is, the rank of its image through a suitable displacement operator. Then, for the families of matrices given above, the results of Bitmead-Anderson, Morf, Kaltofen, Gohberg-Olshevsky, Pan (among others) provide algorithm of complexity O(2 N), up to logarithmic factors, where N is the matrix size and its displacement rank. We show that for Toeplitz-like or Vandermonde-like matrices, this cost can be reduced to O(-1 N), where is an exponent for matrix multiplication. We present consequences for Hermite-Pad