In this paper, we present a new method of synthesizing novel views from the virtual cameras in multiview camera configurations for 3DTV system. We introduce a semi N-view & N-depth framework in order to estimate disparity maps efficiently and correctly. This framework reduces redundancy on disparity estimation by using information from neighboring views. N views can be classified as reference and target images. The disparity maps on the reference images are only estimated by using the cost aggregation method with the weighted least square. The cost functions on the target images are computed by the proposed warping technique so that significant reduction of computation loads is possible. The occlusion problem, which significantly affects the quality of virtual view rendering, is handled by using cost functions computed with multiview images. The proposed method provides a 2D/3D freeview video for 3DTV system. User can select 2D/3D modes of freeview video and control 3D depth perce...