Non-rigid structure from motion using ranklet-based tracking and non-linear optimization

In this paper, we address the problem of estimating the 3D structure and motion of a deformable object given a set of image features tracked automatically throughout a video sequence. Our contributions are twofold: firstly, we propose a new approach to improve motion and structure estimates using a non-linear optimization scheme and secondly we propose a tracking algorithm based on ranklets, a recently developed family of orientation selective rank features. It has been shown that if the 3D deformations of an object can be modeled as a linear combination of shape bases then both its motion and shape may be recovered using an extension of Tomasi and Kanade’s factorization algorithm for affine cameras. Crucially, these new factorization methods are model free and work purely from video in an unconstrained case: a single uncalibrated camera viewing an arbitrary 3D surface which is moving and articulating. The main drawback of existing methods is that they do not provide correct struc...

• Type: Journal
• Year: 2007
• Where: IVC
• Authors: Alessio Del Bue, Fabrizio Smeraldi, Lourdes de Agapito
• Keywords: Algorithm, Factorization Algorithms, IVC 2007, Non-linear Optimization