—Hand interface is well known for its intuition and convenience. User can use their real hands to interact with the system without any assistant device. In this paper, we propose a method which uses vision-based hand interface for table-top interaction system. To do this, first, we find skin region in camera image. But the result might include false-regions which are likely to skin-color, so we use region-based segmentation. To use PCA, a method of hand gesture recognition, we have to split detected region into hand region and arm region, then eliminate hand yaw angle component. We demonstrated the usefulness and possibilities of our method by developing 'omok' game and testing in table-top interaction system.