Abstract. This paper presents a prototype system for pedestrian detection on-board a moving vehicle. The system uses a generic two-step approach for efficient object detection. In the first step, contour features are used in a hierarchical template matching approach to efficiently "lock" onto candidate solutions. Shape matching is based on Distance Transforms. By capturing the objects shape variability by means of a template hierarchy and using a combined coarse-to-fine approach in shape and parameter space, this method achieves very large speed-ups compared to a brute-force method. We have measured gains of several orders of magnitude. The second step utilizes the richer set of intensity features in a pattern classification approach to verify the candidate solutions (i.e. using Radial Basis Functions). We present experimental results on pedestrian detection off-line and on-board our Urban Traffic Assistant vehicle and discuss the challenges that lie ahead.