A real-time visual processing system using a generalpurpose vision chip, an image sensor in which photo detectors and processing elements are integrated, is described. In order to control the vision chip and process its output at high speed, a novel architecture called SPARSIS, in which control process of the vision chip is pipelined and integrated with a RISC type integer pipeline, has been developed. This architecture can guarantee real-time operation with high temporal resolution, and even makes possible software-controlled AID conversion. Sample algorithms demonstrating its fine-grained realtimeness, and experimental results with the implemented system, are also described.