A set of experiments for learning the values of chess pieces is described for the popular chess variants Crazyhouse Chess, Suicide Chess, and Atomic Chess. We follow an established methodology that relies on reinforcement learning from self-games. We attempt to learn piece values and the piecesquare tables for three chess variants. The piece values arrived at, are quite different from those of standard chess, and in several ways surprising, but they generally outperform the values that have been previously used in the literature, and in the implementations of computer players for these games. The results also underline the practical importance of piece-square tables for tactical variants of the game.