What is the signal in noise?

Visual perception is limited by the strength of the neural signals, and by noise; however, little is known about what aspects of the input noise the human visual system is sensitive to, i.e., what is the signal in noise? To investigate this question we asked observers to discriminate differences in the strength of one-dimensional white noise. We measured their response consistency and classification images and compared the results with an ideal energy detector. The double-pass technique enabled us to partition the loss of efficiency into contributions from consistent and random noise, and the results show that random noise reduces human efficiency by about a factor of 2.25 over the approximately forty-fold range of noise levels tested. Consistent noise produces an additional efficiency reduction of about 20% at high noise and by a factor of 10–15 at near threshold noise levels. Consistent noise can be further partitioned into template noise and computational noise (which produces consistent responses not attributable to a mismatched template), by measuring the observer's classification image based on the noise energy at each frequency, and computing how the ideal observer would perform with the human observer's template. We found that the human template is bandpass, independent of the frequency range, suggesting that it is not simply the convolution of the stimulus with the observer's CSF. This template is surprisingly efficient. Template efficiency varies from about 50% at low noise levels to close to 80% at high. Computational noise produces minimal loss of efficiency at high noise levels and a nearly 6–10 fold loss at near threshold noise levels. Our results show that at high noise levels the human observer is mainly limited by random noise. At low noise levels the human observer gives surprisingly consistent responses, with a well-matched template, but with large computational inefficiency.