We present a novel method for interactive retrieval of 3D shapes using physical objects. Our method is based on simple physical 3D interaction with a set of tangible blocks. As the user connects blocks, the system automatically recognizes the shape of the constructed physical structure and picks similar 3D shape models from a preset model database, in real time. Our system fully supports interactive retrieval of 3D shape models in an extremely simple fashion, which is completely non-verbal and cross-cultural. These advantages make it an ideal interface for inexperienced users, previously barred from many applications that include 3D shape retrieval tasks. Categories and Subject Descriptors H.3.3 [Information Search and Retrieval]: Query formulation General Terms Design, Algorithms, Human Factors Keywords Physical Object, 3D Shape Model, Retrieval System, Voxel Data