The paper presents a pattern-oriented agent-based model to simulate the dynamics of a stock market. The model generates satisfactory market macro-level trend and volatility while the agents obey simple rules but follow the behaviors of the neighbors closely. Both the market and the agents are made to evolve in an environment where Darwin’s natural selection rules apply.