We propose a method for estimating radiometric response functions from observation of image noise variance, not profile of its distribution. The relationship between radiance intensity and noise variance is affine, but due to the non-linearity of response functions, this affinity is not maintained in the observation domain. We use the nonaffinity relationship between the observed intensity and noise variance to estimate radiometric response functions. In addition, we theoretically derive how the response function alters the intensity-variance relationship. Since our method uses noise variance as input, it is fundamentally robust against noise. Unlike prior approaches, our method does not require images taken with different and known exposures. Real-world experiments demonstrate the effectiveness of our method.