Stronger CDA strategies through empirical game-theoretic analysis and reinforcement learning

We present a general methodology to automate the search for equilibrium strategies in games derived from computational experimentation. Our approach interleaves empirical game-theoretic analysis with reinforcement learning. We apply this methodology to the classic Continuous Double Auction game, conducting the most comprehensive CDA strategic study published to date. Empirical game analysis confirms prior findings about the relative performance of known strategies. Reinforcement learning derives new bidding strategies from the empirical equilibrium environment. Iterative application of this approach yields strategies stronger than any other published CDA bidding policy, culminating in a new Nash equilibrium supported exclusively by our learned strategies. Categories and Subject Descriptors I.2.6 [Artificial Intelligence]: Learning; I.2.11 [Artificial Intelligence]: Multiagent systems; J.4 [Social and Behavioral Sciences]: Economics General Terms Economics, Experimentation Keywords...

• Type: Conference
• Year: 2009
• Where: ATAL
• Authors: L. Julian Schvartzman, Michael P. Wellman
• Keywords: Artificial Intelligence, ATAL 2009, Continuous Double Auction, Empirical Game-theoretic Analysis, Reinforcement Learning