Discrimination discovery is the data mining problem of unveiling discriminatory practices by analyzing a dataset of historical decision records. In this paper, we focus on discovering discrimination from tweets using deep learning models. One challenge here is that it is difficult to obtain a large well-labeled dataset required by the training of deep learning models for the purpose of discrimination analysis. We develop a two-phase deep learning model to address this challenge. Our model first learns text representations based on weakly-labeled tweets (containing some specific hashtags), then trains the classifier on a small set of well-labeled training data. Experimental results show that: (1) the proposed method can be successfully used for discrimination identification; (2) pre-training text representations, which utilizes weakly-labeled tweets, can significantly improve the accuracy of discrimination detection. Keywords deep learning; discrimination analysis; two phase learn...