We will develop a manipulator system that can handle randomly located, moving parts. Vision algorithm to detect the planar motion of a part is implemented on an FPGA-based realtime vision system. Trajectory of a manipulator is replanned in realtime according to the detected location of a part. We will present the control architecture and will show experimental results.