Information overload has become an important problem in the internet, and that all kinds of existing ads flood into people’s eyes causes scarcity of user’s attention. To provide relevant information under user’s control, we propose an online video advertising framework based on user’s attention relevancy computing. Users receive relevant video ads in exchange of their attention consumption. Multimodal concept detectors are trained to annotate the video databases, and a multimodal video ads categorization and related concept-to-ad relevancy and ad-to-concept relevancy ranking algorithm are proposed to compute user’s attention relevancy. Experiments and a subjective evaluation show the feasibility and effectiveness of the proposed approach.