— Mobile robots have to detect and handle a variety of potential hazards to navigate autonomously. We present a real-time stereo vision based mapping algorithm for identifying and modeling various hazards in urban environments – we focus on inclines, drop-offs, and obstacles. In our algorithm, stereo range data is used to construct a 3D model consisting of a point cloud with a 3D grid overlaid on top. A novel plane fitting algorithm is then used to segment the 3D model into distinct potentially traversable ground regions and fit planes to the regions. The planes and segments are analyzed to identify safe and unsafe regions and the information is captured in an annotated 2D grid map called a local safety map. The safety map can be used by wheeled mobile robots for planning safe paths in their local surroundings. We evaluate our algorithm comprehensively by testing it in varied environments and comparing the results to ground truth data.