What are the primitives of visual perception? The early feature-analysis theory insists on it being a local-to-global process which has acted as the foundation of most computer vision applications for the past 30 years. The early holistic registration theory, however, considers it as a global-to-local process, of which Chen’s theory of topological perceptual organization (TPO) has been strongly supported by psychological and physiological proofs. In this paper, inspired by Chen’s theory, we propose a novel visual organization, termed computational topological perceptual organization (CTPO), which pioneers the early holistic registration in computational vision. Empirical studies on synthetic datasets prove that CTPO is invariant to global transformation such as translation, scaling, rotation and insensitive to topological deformation. We also extend it to other applications by integrating it with local features. Experiments show that our algorithm achieves competitive performance c...