Forecasting Skewed Biased Stochastic Ozone Days: Analyses and Solutions

Much work on skewed, stochastic, high dimensional, and biased datasets usually implicitly solve each problem separately. Recently however, we have been approached by Texas Commission Environmental Quality (TCEQ) to help them build highly accurate ozone level alarm forecasting models for the Houston area, where these technical difficulties come together in one single problem. Key characteristics of this problem that are challenging and interesting include: 1) the dataset is rather skewed (around 72 features, and 2% or 5% positives depending on the criteria of “ozone days”), 2) evolving over time from year to year, 3) limited in collected data size (7 years or around 2500 data entries), 4) contain a large number of irrelevant features, 5) is biased in terms of “sample selection bias”, and 6) the true model is stochastic. Besides solving a difficult application problem, this dataset offers a unique opportunity to explore new and existing data mining techniques, and to provide e...

• Type: Conference
• Year: 2006
• Where: ICDM
• Authors: Kun Zhang, Wei Fan, Xiaojing Yuan, Ian Davidson, Xiangshang Li
• Keywords: Data Mining, ICDM 2006, Irrelevant Features, Ozone Days, Ozone Level Alarm