Eli Peli

Schepens Eye Research Institute, Harvard Medical School, Boston, MA 02114

and

New England Eye Center, Tufts University School of Medicine, Boston, MA 02111

Results of testing using simulations calculated with CSFs measured from a 2 m where veridical for the images in the 30 - 100% contrast range. The 10% image was discriminated at distances larger than the simulated distances. The 300% image was discriminated at a shorter distance. A second set CSFs were obtained from a range of observation distances (0.5 m to 8 m), to overcome limitations of the display. These data showed higher sensitivity for low frequencies and lower sensitivity at the higher frequencies as predicted by the simulation testing results. Replication of the simulation experiments with the combined CSFs resulted in a much better prediction of the discrimination distances. Thus, the CSF relevant for the image discrimination task was verified over a wide range of spatial frequencies while further validating the visual model and its use for simulations.

Simulating the appearance of a scene or an image to an observer is a useful design and analysis tool. Such pictorial representations have been attempted by many investigators over the years in a variety of applications in vision science^2,3,4,5 and engineering^6,7,8. Such simulations are frequently generated within the context of a computational vision model. One such multi-scale model of spatial vision was used to calculate local band-limited contrast in complex images⁹. This contrast measure, together with observers' contrast sensitivity functions (CSFs), expressed as thresholds, was used to simulate the appearance of images to observers, taking into account many of the non-linearities inherent in the visual system. Using the CSFs of low vision patients, the simulation was applied to the design of image enhancement algorithms for the visually impaired⁸. The local band limited contrast model was also used to simulate the appearance of images presented to the peripheral retina¹⁰ using the CSF measured at various retinal eccentricities. Others have applied the same concept of local band-limited contrast with small variations^7,11,12 and found it useful in comparing image quality¹² and in other applications⁷ .

In a recent study Peli¹ tested and demonstrated the validity of the central (foveal) visual model using simulations of the appearance of complex images generated with the model. Images generated with the model represented the appearance of images from various observation distances. Observers view the images simulated using their individual CSF from a wide range of distances side-by-side with the original image and attempted to discriminate the original from the simulated image. The distance at which discrimination performance was at threshold was compared with the simulated observation distance. That study also sought to determine what CSF data should be used in this or any other vision models of this type. While methodological changes can account for the variability of CSF data in the literature¹³, we do not yet know which of these is the appropriate method for determining CSFs that represent complex image perception in the context of pyramidal multi-scale vision models. That study demonstrated that the CSF obtained with a fixed aperture of grating could be rejected when compared with CSF obtained with fixed bandwidth Gabor patches stimuli. Further, it showed that CSF obtained with orientation discrimination task could be rejected while the CSF obtained with a detection task could not be rejected. In both cases 1-octave Gabor patches were used as stimuli. The variable distance simulation and testing method were also shown to be sensitive, affected by small differences as induced by the high frequency residual. The main limitation of the previous study was the fact that with the methodology used the validity of the CSF was tested only at one spatial frequency, as explained below. This paper sought to expand this investigation over a wide range of retinal spatial frequencies.

In applying the vision model to simulations or other applications one needs to consider both the object's contrast spectrum computed in terms of cycles/object or cycles/image and an observer's CSF expressed in terms of cycles/deg. To derive the object's spectrum at the retina (Fig. 1, solid black curve) the distance of the object to the observer needs to be known. Any information in the image that falls below the observer's threshold (i.e., below the point at which the contrast threshold curve intersects the image spectrum curve) is treated by the model as not visible to the observer. To illustrate this, the simulation should remove all that information. This is illustrated by the spectrum line turning to a thin line at the values that are below threshold. If the original and simulated images are viewed from the simulated distance or farther, they should be indistinguishable because the same information from the original that would be lacking due to the visual response was removed in the simulation as well. However, if the original and simulation are viewed from a closer distance, the difference in content between the original and the simulation should be visible.

As the size of displayed object shrinks when the distance of the object from the observer increases, its retinal spatial frequencies increase. It was previously thought by us¹⁴ and others¹⁵ that this change results in a shift of the spectrum to the right along the spatial frequency axis (in Fig.1). However, as Brady and Field¹⁶ pointed out, the spectrum actually shifts both to the right (higher frequencies) and down (lower contrast) sliding along the line with a slope of -1.0. Most natural images have a spatial frequency spectrum that behaves as 1/f, which also have a slope of -1.0. Thus a change in object size causes the spectrum to "slide along itself" (Fig. 1, dashed black curve). As a result, the spectrum of the farther image intersects the CSF curve at a essentially the same retinal frequencies. Only the mapping of the relevant object frequencies to retinal frequencies changes. Therefore, the experiments in Peli¹ have probed only a very limited range of spatial frequencies in the CSF. To examine the CSF at other frequencies one needs to use images whose spectra intersect the CSF at other frequencies. This was achieved in the current study by using higher and lower contrast versions of the same images as illustrated schematically in Fig 1. In fact it was the amplitude of the images that was increased or decreased not the contrast. This operation in which the image dc value is subtracted and the remaining values are scaled up or down is frequently referred to as contrast increase or decrease. As pointed by Peli⁹ the changes in contrast are equivalent to changes in amplitude only where the local luminance is equal to the mean luminance. I will use the terms contrast changes here in conforming to previous use and in recognition that in many places the differences are small. The lower contrast images' spectra intersect the threshold CSF curve at lower spatial frequency and thus can be used to test the CSF at lower frequencies. Similarly the higher contrast image was used to test the CSF at a higher spatial frequency.

Figure 1 is useful only to illustrate the logic of the experiments described below. The analysis it represents cannot replace the information we seek from the simulations and from direct testing of the simulation. The effects of contrast threshold on apparent contrast in the images are local, not global. Thus the effective contrast is not accurately represented by the (1-D) radially averaged amplitude spectrum, because in the simulations we were working with local contrast, not amplitude⁹, and the simulation algorithm is not represented accurately by the linear filtering depicted in the schematic.

The simulations were tested by presenting the original image side-by-side with the simulation of its appearance from a certain distance. If the simulations are valid, the simulated image and the original should be indistinguishable from a distance equal to or farther than the distance assumed in the simulation¹⁴. The two images should be progressively easier to distinguish at distances shorter than the simulated distance.

Observers viewed image pairs from various distances and were asked to make a forced-choice distinction between the simulated and the original image. The simulated images used to test each observer were calculated using her/his individual CSF. Four different images at 5 different contrast versions were used in this experiment. For each image, three simulated views were generated representing views from three different distances. For the three simulated observation distances (106, 212, and 424 cm), the images spanned visual angles of 4, 2, and 1 deg respectively. The simulated distance and the corresponding span in degrees serves to establish the proper relations between the subject's CSF expressed in c/deg and the image spatial content expressed in terms of c/image. The subjects viewed the image pairs from nine distances, including shorter (53 cm) than the shortest simulated distance and longer (848cm) than the longest simulated distance. Each image at each simulation distance was presented 10 times at each viewing distance. The position of the simulated image relative to the original (right or left) was randomly selected for each presentation. From each observation distance the percent correct identification of the processed/simulated image was calculated for each simulated distance for the 4 images. The data for each simulated distance (percent correct out of 40 responses for each observation distance) was fit with a Weibull psychometric function to determine threshold at a 75% correct level. The distance at which the subject performed at 75% level was compared to the simulated distance. If the simulations and the CSF used in the simulation represent the subject perception correctly, the measured and simulated distance should be equal.

The CSF data used in the simulations were obtained for each subject individually. The CSFs data were obtained using 1-octave Gabor patches and a simple detection task. The CSF data were collected on a Vision Works system (Durham, NH) using a M21LV-65MAX monitor with DP104 phosphor operating at 117 Hz, non-interlaced. CSF data were collected with both method-of-adjustment (MOA) and staircase procedure, as indicated. The staircase included 2 practice reversals and the threshold was calculated as the average of 4 test reversals. Seven interwoven frequencies separated by 1 octave between 0.5 and 32 c/deg were presented in each block. For the MOA six responses at each frequency were averaged. The stimuli were the same Gabor patches of 1 -octave bandwidth in all cases (vertical orientation only).

The image pairs were presented on a 19 in (48 cm), non-interlaced monochrome video monitor of the Sparc 10 Workstation (Sun Microsystems, Mountain View, CA). Linearity of the display response was obtained using an 8-bit lookup table. The calibrated screen provided a linear response over 2 log units. The images were 256256 pixels each and were presented at the middle of the screen, separated by 128 pixels. The background luminance around the images was set to the mean luminance level of the display (40 cd/m2). The four images used are common images used in image processing. These images were originally recorded with standard video cameras designed to display on a non-linearized CRT. To enable a linear relationship between the displayed luminance levels and the numerical representation of the images, we presented the images using a linearizing (Gamma corrected) lookup table. To maintain the natural appearance and contrast range of the images the original images were preprocessed to include the measured display Gamma function¹⁷. The original images were also produced at varying contrasts. This was achieved by subtracting the mean luminance level from the image multiplying each pixel by the corresponding contrast (10%, 30%, 50%, and 300%) and adding the mean luminance back. The 300% contrast image was saturated wherever the dark or bright values exceeded the dynamic range of the display. The simulations for each of the 4 images 5 contrast versions and 3 simulated observation distances were generated as described in Peli¹.

Observers were seated in a dimly lit room and adapted to the mean luminance of the display for 5 min. before beginning the experiment. Location of the simulated image (right or Left) was indicated using the right and Left buttons on a mouse. A new pair of images emerged 0.1 sec. after each response and remained on until the subject responded. The order of observation distances was randomized

3. Results

Figure 4. The distances at which the simulated images were distinguished from the corresponding original images compared with the simulated observation distance, for the same subjects as in Fig 2. Here the simulations were computed using the combined CSFs obtained from different observation distances. a) For the well practiced subject the data with the combined CSF is now very close to the prediction represented with the diagonal solid line. b) For the novice subject the practiced gained in the task results in the simulated images here being distinguished from the original image at a farther distance than simulated. In addition, the different contrast versions are detected closer to each other and closer to the prediction line than in Fig 2b.

The results of these experiments re-verified that the model proposed by Peli⁹ and used to simulate the appearance of an image from different observation distances is valid¹⁹. The simulated images are distinguishable from the original at distances close to the simulated distances. The size of the effects, as occurs when observer's distance from the display is doubled, is of the magnitude of interest in image quality metrics. Since we are able to accurately simulate such effects using the vision model employed here it stands to reason that such models could be successfully employed to calculate such differences in order to estimate image quality^7,12. In addition, the current study has demonstrated that this method can be used to test the applicability of a specific empirically derived CSF to the performance of a discrimination task with complex images.

Of particular interest in this work is the possible reasons for the differences between the CSF's obtained at different frequencies. The low frequency end is simple to account for. The low frequency Gabor patches used from a distance of 2 m are quite large physically occupying substantial part of the screen. The edge of the screen (outside the active video range) is dark and creates a high contrast feature which when close to the patch may reduce its visibility. Moving the observer closer to the screen reduces the physical size of the patches on the screen for the same spatial frequencies and thus increases their distance from the edge and reduces its masking effect. Indeed for both subjects the detection threshold for the three lowest spatial frequencies was almost equal at 2 m and 0.5 m (which were the same physical stimuli) suggesting that the reduction in sensitivity for these Gabor patches at low frequency is only a masking effect. This argue for even higher sensitivity at low frequency than represented by the "combined CSF" in Fig. 3.

The explanation for the change in CSF for high spatial frequencies we found with increase in distance is not as obvious. Although the high spatial frequency targets are physically small on the screen at 2 m distance there is sufficient resolution to adequately represent the Gabor patch (about 8 pixel/cycle). Even if this resolution is too low it is not clear why it would lead to higher apparent sensitivity. In fact the opposite effect may be expected. Using the larger stimuli from 4 or 8 m get these stimuli closer to the edge where they may be masked as the case of the low frequencies. The effect that this images are of higher contrast may be presumed to cause more masking, however , such masking should reduce observer task performance not improve it. Thus, at the moment I have no hypothesis to offer for this effect.

The methods of simulation and for testing the simulation using the paradigm presented here are sensitive enough to be affected by the differences between CSFs obtained with different methods¹⁹. As was shown here they are also sensitive enough to distinguish CSFs obtained at different distances. Thus this methodology can be used to determine the type of CSF data that more closely represent the appearance of images. Using the same method it may be possible to determine the shape of the CSF directly from simulation experiments by generating the simulation from synthetic, not measured CSF curves. Such determination is independent of the specific stimuli used for the CSF measurement and may provide us with a CSF that should be used in conjunction with visual models. Discrimination of moving video segments can be used in a similar way to determine the spatio-temporal characteristics of the CSF affecting perception.

Supported in part by grants #05957 and #10285 from the NEI. I thank Angela Labianca for valuable technical help and Brian Sperry for programming support.