We study how decentralized agents can develop a shared vocabulary without global coordination. Answering this question can help us understand the emergence of many communication systems, from bacterial communication to human languages, as well as helping to design algorithms for supporting self-organizing information systems such as social tagging or ad-word systems for the web. We introduce a formal communication model in which senders and receivers can adapt their communicative behaviors through a simple reinforcement learning mechanism that adjusts each agent's vocabulary: the ways it associates words with meanings. We analyze the model's dynamics in terms of collective convergence conditions and convergence speed. Our main result on the convergence conditions is that for a given number of meanings, there exists a threshold of the number of words below which the agents can't converge to a shared vocabulary. We also give the time needed for the agents to converge to a...