Pedestrian detection is a key problem in computer vision,
with several applications including robotics, surveillance
and automotive safety. Much of the progress of the past
few years has been driven by the availability of challenging
public datasets. To continue the rapid rate of innovation,
we introduce the Caltech Pedestrian Dataset, which
is two orders of magnitude larger than existing datasets.
The dataset contains richly annotated video, recorded from
a moving vehicle, with challenging images of low resolution
and frequently occluded people. We propose improved
evaluation metrics, demonstrating that commonly used perwindow
measures are flawed and can fail to predict performance
on full images. We also benchmark several promising
detection systems, providing an overview of state-of-theart
performance and a direct, unbiased comparison of existing
methods. Finally, by analyzing common failure cases,
we help identify future research directions for the field.