Online incremental evolution for a complex high-speed pattern recognition architecture has been implemented on a Xilinx Virtex-II Pro FPGA. The fitness evaluation module is entirely hardware-based in order to increase the speed of the circuit evaluation which uses a large training set (360 images/23040 bytes). The fitness evaluation time for 1000 generations consisting of 16 individuals is 623ms, twice as fast as software fitness evaluation performed on a workstation running at a 30 times higher clock frequency. The rest of the genetic algorithm (GA) runs in software on a PowerPC 405 processor core on the FPGA. The total evolution time for 1000 generations is 1313ms, equivalent to the total time used by the workstation. Resource utilization for the fitness evaluation module is 1393 slices (10%) of a XC2VP30 device.