In this study, we describe a conversational agent designed to support collaborative learning interactions between pairs of students. We describe a study in which we independently manipulate the social capability and goal alignment of the agent in order to investigate the impact on student learning outcomes and student perceptions. Our results show a significant interaction effect between the two independent variables on student learning outcomes. While there are only a few perceived differences related to student satisfaction and tutor performance as evidenced in the questionnaire data, we observe significant differences in student conversational behavior, which offer tentative explanations for the learning outcomes we will investigate in subsequent work.