Many vision problems have been formulated as en- ergy minimization problems and there have been signif- icant advances in energy minimization algorithms. The most widely-used energy minimization algorithms include Graph Cuts, Belief Propagation and Tree-Reweighted Mes- sage Passing. Although they have obtained good results, they are still unsatisfactory when it comes to more difficult MRF problems such as non-submodular energy functions, highly connected MRFs, and high-order clique potentials. There have also been other approaches, known as stochas- tic sampling-based algorithms, which include Simulated Annealing, Markov Chain Monte Carlo and Population- based Markov Chain Monte Carlo. They are applicable to any general energy models but they are usually slower than deterministic methods. In this paper, we propose new algo- rithms which elegantly combine stochastic and determinis- tic methods. Sampling-based methods are boosted by de- terministic methods so that they can rapidly move t...