— Recent development of information technology is making electric household appliances computerized and networked. If the environments surrounding us could recognize our activities indirectly by sensors, the novel services, which respect our activities, can be possible. In this paper, we propose human activity recognition system, which infers the next human action by taking account of the past human behaviors observed so far. We also developed cooking support robot, which suggests what the human should do next by voice and gesture. Experimental results confirmed the feasibility of the inference system and the quality of support.