Research ArticleNEUROPHYSIOLOGY

Stereopsis is adaptive for the natural environment

See allHide authors and affiliations

Science Advances  29 May 2015:
Vol. 1, no. 4, e1400254
DOI: 10.1126/sciadv.1400254
  • Fig. 1 Binocular disparities used for stereopsis.

    (A) Two views of a simple 3D scene. The eyes are fixating point F1, which lies straight ahead. Point P is positioned above and to the right of the viewer’s face, and is closer in depth than F1. The upper panel shows a side view and the lower panel a view from behind the eyes. Lines of equal azimuth and elevation in Helmholtz coordinates are drawn on each eye. (B) Retinal projections of P from the viewing geometry in (A). The yellow and orange dots correspond to the projections in the left and right eyes, respectively. The difference between the left and right eye projections is binocular disparity. The difference in azimuth is horizontal disparity, and the difference in elevation is vertical disparity. In this example, point P has crossed horizontal disparity, because it is closer than point F1 and the image of P is therefore shifted leftward in the left eye and rightward in the right eye. (C) For a given point in the scene, the disparity at the retinas can change substantially depending on where the viewer is fixating. In the left panel, the same point P is observed, but with a different fixation point F2 that is now closer to the viewer than P (indicated by the arrow). The original fixation point F1 is overlaid in gray. In the right panel, the retinal disparities projected by P are shown for both fixations [disparities from (B) are semitransparent]. For this viewing geometry, point P now has uncrossed horizontal disparity: that is, the image of P is shifted rightward in the left eye and leftward in the right eye.

  • Fig. 2 System and method for determining retinal disparities when participants are engaged in everyday tasks.

    (A) The device. A head-mounted binocular eye tracker (EyeLink II) was modified to include outward-facing stereo cameras. The eye tracker measured the gaze direction of each eye using a video-based pupil and corneal reflection tracking algorithm. The stereo cameras captured uncompressed images at 30 Hz and were synchronized to one another with a hardware trigger. The cameras were also synchronized to the eye tracker using digital inputs. Data from the tracker and cameras were stored on mobile computers in a backpack worn by the participant. Depth maps were computed from the stereo images offline. (B) Calibration. The cameras were offset from the participant’s eyes by several centimeters. To reconstruct the disparities from the eyes’ viewpoints, we performed a calibration procedure for determining the translation and rotation between the 3D locations. The participant was positioned with a bite bar to place the eyes at known positions relative to a large display. A calibration pattern was then displayed for the cameras and used to determine the transformation between the cameras’ viewpoints and the eyes’ known positions. (C) Schematic of the entire data collection and processing workflow. (D) Two example images from the stereo cameras. Images were warped to remove lens distortion, resulting in bowed edges. The disparities between these two images were used to reconstruct the 3D geometry of the scene. These disparities are illustrated as a grayscale image with bright points representing near pixels and dark points representing far pixels. Yellow indicates regions in which disparity could not be computed due to occlusions and lack of texture. (E) The 3D points from the cameras were reprojected to the two eyes. The yellow circles (20° diameter) indicate the areas over which statistical analyses were performed. (F) Horizontal disparities for this scene at one time point. Different disparities are represented by different colors as indicated by the color map on the right.

  • Fig. 3 Distributions of binocular eye position for the four tasks and weighted combination of tasks.

    (A) Distributions of horizontal vergence for the tasks and the weighted combination. Probability density is plotted as a function of vergence angle. Small and large vergences correspond to far and near fixations, respectively. For reference, vergence distance (assuming an interocular distance of 6.25 cm and version of 0°) is plotted on the abscissa at the top of the figure. Different colors represent the data from different tasks; the thick black contour represents the weighted-combination data. (B) Distances of fixation points and visible scene points. The probability of different fixation distances and scene distances is plotted as a function of distance in diopters (where diopters are inverse meters). The upper abscissa shows those distances in centimeters for comparison. The black solid curve represents the distribution of weighted-combination fixation distances. The orange dashed curve represents the distribution of distances of visible scene points, whether fixated or not, falling within 10° of the fovea. All distances are measured from the midpoint on the interocular axis (the cyclopean eye). The median distances are indicated by the arrows: the orange arrow for scene points and the black one for fixation points. (C) Distributions of version for the four tasks. Horizontal version and vertical version are plotted, respectively, on the abscissa and ordinate of each panel. By convention, positive horizontal and vertical versions are leftward and downward, respectively. The data are, of course, plotted in head-centered coordinates. Colors represent probability (darker being more probable). Marginal distributions of horizontal and vertical versions are shown, respectively, at the top and right of each panel. (D) Distribution of version for the weighted combination of the data.

  • Fig. 4 Median disparities in different parts of visual field for different tasks.

    (A) Example frames from the four activities. (B) Median horizontal disparities and visual field position. The circular plots each represent data from one of the four activities averaged across participants. The fovea is at the center of each panel. Radius is 10°, so each panel shows medians for the central 20° of the visual field. Dashed gray lines represent the horizontal and vertical meridians. Uncrossed disparities are blue, crossed disparities red, and zero disparity white. (C) Median vertical disparities and visual field position. Positive disparities are purple, negative disparities orange, and zero disparity white. The panels show the medians for each activity averaged across participants. (D) Weighted combinations. The left panel shows the median horizontal disparities when the four activities are weighted according to ATUS. The right panel shows the combined median vertical disparities. The scales of the color maps differ from the scales for the other panels.

  • Fig. 5 Distributions of horizontal and vertical disparities.

    (A) Joint distributions of horizontal and vertical disparities in different regions of the visual field. The black circle in the background represents the central 20° of the visual field (10° radius) with the fovea at the center. Each subplot shows the joint disparity distribution in a different region of the visual field. Horizontal disparity is plotted on the abscissa of each subplot and vertical disparity on the ordinate. The scales for the abscissa and ordinates differ by a factor of 10, the abscissa ranging from −0.5° to +0.5° and the ordinate from −0.05° to +0.05°. The centers of the represented regions are either at the fovea or 4° from the fovea. The regions are all 3° in diameter. Frequency of occurrence is represented by color, darker corresponding to more frequent disparities. The curves at the top and right of each subplot are the marginal distributions. Horizontal and vertical disparities are positively correlated in the upper left and lower right parts of the visual field and negatively correlated in the upper right and lower left parts of the field (absolute r values greater than 0.16, P values less than 0.001). One can see in these panels the shift from uncrossed (positive) horizontal disparity in the upper visual field to crossed (negative) disparity in the lower field. We calculated the range of horizontal disparities needed to encompass two-thirds of the observations in each of the nonfoveal positions in the figure. For the upper middle, upper right, mid right, lower right, lower middle, lower left, mid left, and upper left regions, the disparity ranges were respectively 0.69°, 1.37°, 0.53°, 0.73°, 0.49°, 0.72°, 0.25°, and 0.43°; the average across regions was 0.74°. (B) Joint distribution of horizontal and vertical disparities in the central 20° (10° radius) of the visual field. The format is the same as the plots in A. Movies S1 and S2 show how the marginal distribution of horizontal disparity changes as a function of elevation and azimuth.

  • Fig. 6 Natural disparity statistics and the horopter.

    (A) Geometric and empirical horopters. Points in 3D space that project to zero horizontal and vertical disparities are shown as a set of red points. The lines of sight are represented by the black lines, and their intersection is the fixation point. The geometric horopter contains points that lie on a circle that runs through the fixation point and the two eye centers (the horizontal horopter or Vieth-Müller Circle) and a vertical line in the head’s median plane (the vertical horopter). Points that empirically correspond are not in the same positions in the two eyes. The projections of those points define the empirical horopter, which is shown in blue. The horizontal part is less concave than the Vieth-Müller Circle, and the vertical part is convex and pitched top-back relative to the vertical line. The disparities associated with the empirical horopter have been magnified relative to their true values to make the differences between the empirical and geometric horopters evident. (B) Corresponding points are fixed in the retinas, so when the eyes move, the empirical horopter changes. Here, the eyes have diverged relative to (A) to fixate a farther point. The horizontal part of the empirical horopter becomes convex, and the vertical part becomes more convex and more pitched. (C) Distributions of horizontal disparity near the vertical and horizontal meridians of the visual field. The upper row plots horizontal disparity as a function of vertical eccentricity near the vertical meridian. The lower row plots horizontal disparity as a function of horizontal eccentricity near the horizontal meridian. The columns show the data from each of the four tasks averaged across participants. The colored regions represent the data between the 25th and 75th percentiles. The dashed lines represent the median disparities. (D) Weighted-combination data and the empirical horopter. The left panel shows the weighted-combination data and the horopter near the vertical meridian; horizontal disparity is plotted as a function of vertical eccentricity. The right panel shows the data and horopter near the horizontal meridian; horizontal disparity is plotted as a function of horizontal eccentricity. The brown lines represent percentiles of the natural disparity data: dashed lines are the medians, and solid lines are the 25th and 75th percentiles. The yellow lines represent the same percentiles for the horopter. The horopter data in the left panel come from 28 observers in (31). The horopter data in the right panel come from 15 observers in (23, 35, 42, 52). We computed the probability density of natural disparities at each eccentricity, and these distributions are underlaid in brown, with the color scale normalized to the peak disparity at each eccentricity.

  • Fig. 7 Wallpaper illusion and venetian blinds effect.

    (A) Upper part shows the wallpaper illusion. Vertical sinewave gratings of the same spatial frequency and contrast are presented to the two eyes. Cross-fuse the red dots to see the stimulus stereoscopically. Notice that the stimulus appears to lie in the same plane as the fused red dot. Now cross-fuse the green dots and notice that the stimulus now appears to lie in the same plane as the fused green dot. The lower part shows the venetian blinds effect. Vertical sinewave gratings of different spatial frequencies are presented to the two eyes (fR/fL = 0.67). Cross-fuse the red dots to see the stimulus stereoscopically. Hold fixation steady on the fused red dot and notice that the stimulus appears to be a series of slanted planes. (B) Likelihood, prior, and posterior distributions as a function of azimuth; elevation is zero. The upper row shows the distributions for the wallpaper stimulus and the lower row the distributions for the venetian blinds stimulus. The grayscale represents probability, brighter values having greater probability. (C) MAP estimates for wallpaper (upper) and venetian blinds (lower). Note the change in the scale of the ordinate relative to (B). If one wanted to visually match the stimuli in A to the units in B and C, the correct viewing distance is 148 cm, so that the stimuli subtend approximately 2×2°. Note that A shows only a portion of the stimuli used to compute the results.

  • Fig. 8 Receptive-field locations and preferred-disparity distributions among 973 disparity-sensitive neurons from macaque V1.

    (A) Receptive-field location of each neuron. The left and right subplots show neurons for which preferred horizontal disparity and preferred vertical disparity were measured, respectively. The colors represent the study from which each neuron came. Most neurons had receptive fields in the lower visual field. Very few were in the upper right quadrant. (B) Distributions of preferred horizontal disparity grouped by upper and lower visual field. The histograms represent the number of cells observed with each preferred disparity. The curves represent those histograms normalized to constant area. The preferred disparities of neurons from the upper visual field (blue) are biased toward uncrossed disparities, whereas the preferences of neurons from the lower visual field (red) are biased toward crossed disparities. (C) Distributions of preferred vertical disparities grouped by quadrants with the same expected sign of vertical disparity. Again, the histograms represent the actual cell counts, and the curves represent those histograms normalized for constant area. (D) Joint distribution of preferred horizontal and vertical disparities from the sampled neurons. Darker color represents more frequent observations. The marginal distributions for horizontal and vertical disparities are shown on the top and right, respectively.

Supplementary Materials

  • Supplementary material for this article is available at http://advances.sciencemag.org/cgi/content/full/1/4/e1400254/DC1

    Fig. S1. Distributions of fixation distances and scene distances for the four tasks.

    Fig. S2. Median, SD, skewness, and kurtosis of the distributions of horizontal and vertical disparities for each subject and task.

    Fig. S3. Cumulative version error distribution across all subjects and all calibrations.

    Table S1. Experimental setups and coordinate system conventions in Cumming (1), Durand et al. (2), Gonzalez et al. (65), Prince et al. (7, 66), and Samonds et al. (67).

    Movie S1. Video showing the distribution of horizontal disparity as a function of elevation.

    Movie S2. Video showing the distribution of horizontal disparity as a function of azimuth.

  • Supplementary Materials

    This PDF file includes:

    • Fig. S1. Distributions of fixation distances and scene distances for the four tasks.
    • Fig. S2. Median, SD, skewness, and kurtosis of the distributions of horizontal and vertical disparities for each subject and task.
    • Fig. S3. Cumulative version error distribution across all subjects and all calibrations.
    • Table S1. Experimental setups and coordinate system conventions in Cumming (1), Durand et al. (2), Gonzalez et al. (65), Prince et al. (7, 66), and Samonds et al. (67).
    • Legends for movies S1 and S2

    Download PDF

    Other Supplementary Material for this manuscript includes the following:

    • Movie S1 (.mp4 format). Video showing the distribution of horizontal disparity as a function of elevation.
    • Movie S2 (.mp4 format). Video showing the distribution of horizontal disparity as a function of azimuth.

    Download Movies S1 and S2

    Files in this Data Supplement: