In this paper we present the process of designing and developing a fully functional and interactive AR handheld device. Based on a human-centered approach we describe the results of our exploration; from the contextual usage to the different rapid prototyped models and the evaluation activities. We also discuss some recommendations for how to use similar methods for developing future new AR physical interfaces.