Clustering methods are a useful and common first step in gene expression studies, but the results may be hard to interpret. We bring in explicitly an indicator of which genes tie each cluster, changing the setup to biclustering. Furthermore, we make the indicators hierarchical, resulting in a hierarchy of progressively more specific biclusters. A non-parametric Bayesian formulation makes the model rigorous and yet flexible, and computations feasible. The formulation additionally offers a natural information retrieval relevance measure that allows relating samples in a principled manner. We show that the model outperforms other four biclustering procedures in a large miRNA data set. We also demonstrate the model’s added interpretability and information retrieval capability in a case study that highlights the potential and novel role of miR-224 in the association between melanoma and non-Hodgkin lymphoma. Software is publicly available.1