A new graph-based evolutionary algorithm named “Genetic Network Programming, GNP” has been proposed. GNP represents its solutions as directed graph structures, which can improve the expression ability and performance. Since GA, GP and GNP already proposed are based on evolution and they cannot change their solutions until one generation ends, we propose GNP with Reinforcement Learning (GNP with RL) in this paper in order to search solutions quickly. Evolutionary algorithm of GNP makes very compact directed graph structure which contributes to reducing the size of the Q-table and saving memory. Reinforcement Learning of GNP improves search speed for solutions because it can use the information obtained during tasks. Recently, a new directed graph-based evolutionary algorithm named “Genetic Network Programming (GNP)[1]” has been proposed. Basically, GA and GP represent solutions as string and tree structures, respectively, but GNP represents its solutions as graph structures comp...