Abstract. Writing e cient iterative solvers for irregular, sparse matrices in HPF is hard. The locality in the computations is unclear, and for e ciency we use storage schemes that obscure any structure in the matrix. Moreover, the limited capabilities of HPF to distribute and align data structures make it hard to implement the desired distributions, or to indicate these such that the compiler recognizes the e cient implementation. We propose techniques to handle these problems. We combine strategies that have become popular in message-passing parallel programming, like mesh partitioning and splitting the matrix in local submatrices, with the functionality of HPF and HPF compilers, like the implicit handling of communicationand distribution. The implementationof these techniques in HPF is not trivial, and we describe in detail how we propose to solve the problems. Our results demonstrate that e cient implementations are possible. We indicate how some of the `approved extensions' o...