With the proliferation of online media services, video ads are pervasive across various platforms involving internet services and interactive TV services. Existing research efforts such as Google AdSense and MSRA VideoSense/ImageSense have been devoted to the less intrusive insertion of relevant textual or video ads in streams or web pages through text/image/video content analysis whereas the inherent semantics of video ads is much less exploited. In this paper, we propose to link video ads with relevant product/service information across E-commerce websites or portals towards ad recommendation in a cross-media manner. Firstly, we carry out semantic analysis within ad videos in which Frames Marked with Product Images (FMPI) are extracted. Secondly, we link ad videos with relevant ads on the Web by utilizing FMPI to search visually similar Product Images (e.g. appearance or logo) and to collect their accompanying text (brand name, category, description, or other tags) over popular E-co...