Research ArticleNEUROSCIENCE

A distinct population of heterogeneously color-tuned neurons in macaque visual cortex

See allHide authors and affiliations

Science Advances  19 Feb 2021:
Vol. 7, no. 8, eabc5837
DOI: 10.1126/sciadv.abc5837

Abstract

Color is a key feature of natural environments that higher mammals routinely use to detect food, avoid predators, and interpret social signals. The distribution of color signals in natural scenes is widely variable, ranging from uniform patches to highly nonuniform regions in which different colors lie in close proximity. Whether individual neurons are tuned to this high degree of variability of color signals is unknown. Here, we identified a distinct population of cells in macaque visual cortex (area V4) that have a heterogeneous receptive field (RF) structure in which individual subfields are tuned to different colors even though the full RF is only weakly tuned. This spatial heterogeneity in color tuning indicates a higher degree of complexity of color-encoding mechanisms in visual cortex than previously believed to efficiently extract chromatic information from the environment.

INTRODUCTION

Color represents an integral component of the visual input to the brain (15). Higher vertebrates process color to make important decisions regarding food gathering, avoiding predators, and engaging in social interactions (68). In natural environments, color signals exhibit a high degree of variability ranging from spatially uniform patches containing similar colors to heterogeneous patches containing different colors within close proximity of each other (Fig. 1A) (810). However, despite this highly variable spatial arrangement of color signals, it is not known whether neurons in V4 mirror this complexity in the spatial properties of their large receptive fields (RFs) (11, 12).

Fig. 1 Subfield stimulation of V4 neurons with equiluminant color stimuli.

(A) Natural image with chromatically uniform and nonuniform regions at two different spatial locations. Image taken from McGill Calibrated Colour Image Database (76) (photographer: unknown, McGill University). (B) Schematic of chronically implanted Utah array in area V4. (C) Equiluminant color stimulus set plotted in u,v space. (D) Angular plot of same color stimulus with respect to the u,v coordinates of an equiluminant neutral gray background. (E) Schematic description of full–receptive field (RF) stimulation using a “reverse correlation” movie consisting of 16 equiluminant colors. (F) Schematic description of sub-RF stimulation using the same reverse correlation approach targeting small (~1°) subfields.

We reasoned that the visual system could have evolved to endow individual neurons with the ability to determine spatial relationships defining color-based contours and encode color-heterogeneous image regions. Mid-level cortical area V4 is heavily involved in color processing (1320), and it receives dense feedforward projections from early visual cortex (2, 2125). Because the RFs of V4 cells are, on average, several times larger than those in V1, it is conceivable that each V4 neuron receives inputs from several upstream cortical cells with different RF positions and color selectivity. This raises the issue of whether converging feedforward inputs to V4 create homogeneous color tuning within the RF or they create a more complex RF structure tuned to combinations of colors. From an evolutionary perspective, it is believed that the visual system is adapted to efficiently represent natural scenes (2629), and hence, it is conceivable that RFs in downstream cortical areas could have a more intricate and nonuniform representation of color. However, whether such a representation exists in the color domain in visual cortex is unknown.

Here, we recorded populations of neurons in area V4 of awake macaque with chronically implanted Utah arrays to examine the microstructure of RF color tuning. Unexpectedly, we discovered two types of neurons: one that is homogeneous with respect to color preference and another that consists of spatially heterogeneous subregions tuned to different colors. Reverse correlation analysis indicated that “heterogeneous” neurons have a higher latency of color tuning compared with neurons with uniformly tuned RFs, implying the possible involvement of local intracortical circuits shaping color tuning. Furthermore, heterogeneous cells are better suited than homogenous cells to extract information from complex stimuli comprising multiple color patches in close proximity. These findings indicate a higher degree of complexity of color-encoding mechanisms than previously believed and suggest that they are adapted to the fine-scale spatial diversity of color signals in natural environments.

RESULTS

We simultaneously recorded the spiking activity of multiple neurons from superficial layers of V4 using chronically implanted Utah arrays in two adult behaving macaques (trichromats) T and M (n = 26 sessions, N = 270 neurons; Fig. 1B and fig. S1). Color tuning was measured using a set of 16 equiluminant colors (9.35 ± 0.01 cd/m2) uniformly spaced in Luv color space with respect to a gray neutral point (see Materials and Methods; Fig. 1, C and D, and table S1, color patches were presented on an isoluminant neutral gray background). We first confirmed previous reports (1320) that many V4 neurons had robust color tuning (see Materials and Methods) when stimuli covered their entire RF (Figs. 1E and 2, A and B, and fig. S2, A and B). We further examined the RF microstructure of V4 cells. The full RFs were divided into a 5 × 5 square grid of subfields (~1° × 1° per square; Fig. 2C), and then, we measured the color tuning of each subfield using a rapid stimulus sequence (Fig. 1F and fig. S2, C and D). As expected, we found unimodal color-tuning curves across subfields (Fig. 2D) that peaked at the same color throughout the RF, labeled as “homogeneous” tuning population. The subfields’ preferred color (PC) matched the cell’s PC when stimulated with large color patches covering the entire RF (Fig. 2E).

Fig. 2 Heterogeneous color tuning properties in V4.

(A) Example V4 RFs generated using RF mapping stimuli (see Materials and Methods), cells 1 and 3 (monkey T), and cell 2 (monkey M). Coordinates (0,0) represent the fixation point. (B) Tuning curves for the same neurons when the entire RF is stimulated with equiluminant color stimuli. Color selectivity index (CSI) are 0.5 (cell 1), 0.24 (cell 2), and 0.30 (cell 3), respectively. (C) Zoomed in version of the heatmap in (A) (cell 1) showing the finer 5 × 5 grid for allocating subfields. (D) Tuning curves at each of the 25 locations in (C) reconstructed using reverse correlation analysis. CSImean = 0.60 ± 0.01. (E) Overlay of all subfield tuning curves (red) on the full-field tuning curve (black) for the homogeneous RF shown in (C). (F) Same as in (C) except in the case of a neuron (cell 4; monkey M) with heterogeneously tuned RF. (G) Tuning of each subfield at different spatial locations within the RF shown in (F). CSImean = 0.31 ± 0.02. (H) Same as in (E) except for the heterogeneous RF shown in (F). Note no significant tuning in responses (P > 0.05; Rayleigh’s test, CSI = 0.11) obtained with full-field stimulation of the heterogeneous RF. Error bars represent SEM.

A distinct population of heterogeneously color-tuned neurons

Unexpectedly, in addition to these more traditional color-tuned cells, we discovered a distinct population of neurons that were virtually untuned when the full RF was stimulated (Rayleigh’s test, P > 0.05; see Materials and Methods) but became sharply color tuned when small stimulus patches covered particular RF subfields (Fig. 2, F to H), labeled as heterogeneous tuning population. The RF structure of these cells was complex, as it consisted of multiple subregions tuned to different colors. To quantify the diversity of colors represented in these cells’ subfields, we calculated the circular SD of the distribution of PC of all subfields for a given RF (see Materials and Methods). This measure, defined as heterogeneity index (HI), is bounded between 0° and 81° (values close to 0° indicate cells that have homogeneous RF tuning). The distribution of HI values (Fig. 3A and fig. S3) across all neurons (N = 270) was better fit by a bimodal Gaussian distribution (R2adj = 0.91) than a unimodal one (R2adj = 0.72), thus showing the diversity of color tuning within the RFs of V4 neurons ranging from strictly homogeneous to highly heterogeneous color tuning.

Fig. 3 Tuning for multiple colors within the RF of heterogeneous neurons.

(A) Distribution of heterogeneity index (HI) of 270 V4 neurons pooled from both monkeys (N = 159, monkey T; N = 111 monkey M). Red solid line represents a bimodal Gaussian fit (R2adj = 0.72) generated using the curve fitting toolbox in MATLAB. Distribution of HI for each animal is shown in fig. S3. Black arrow points to the median HI value. (B) Representative examples of subfield color tuning for four neurons (cells 1 and 3, monkey T; cells 2 and 4, monkey M) with a diverse range of HI values. Gray-colored subfields did not show significant tuning to any of the colors in the stimulus set. (C) Number of unique preferred colors that subfields of a neuron are tuned to as a function of the HI of the neuron. Each gray circle represents a single neuron. Black filled circles represent mean values within bins of size HI = 20. Solid black line represents interpolated fit for visualization purposes. (D) CSI of neurons (N = 63) recorded in a subset of sessions with significant tuning for full-RF color stimulation as a function of their HI. Note that full-RF stimulation and sub-RF stimulation were performed on the same day but in different sessions, and only neurons with both full-RF and subfield tuning were included in the analysis. Black solid line represents linear fit to the data showing significant negative correlation (Pearson’s correlation, r = −0.31, P < 0.001).

Across cells, the degree of heterogeneity varied from moderate to abrupt shifts in color preference (Fig. 3B). To examine the diversity of color representation, we calculated the number of unique colors (see Materials and Methods) that the subfields of V4 neurons were tuned to. The number of unique colors represented within individual RFs showed a strong positive correlation with the HI of the neuron (Spearman’s rank correlation r = 0.77, P < 0.001; Fig. 3C; the most heterogeneous neurons represented up to six unique colors from those present in our stimulus set). The heterogeneously tuned neurons are only weakly tuned to full-field stimuli covering the entire RF. The neurons’ strength of color tuning [color selectivity index (CSI); see Materials and Methods] measured in response to full-RF stimulation showed a significant negative correlation with the HI (Pearson’s correlation, r = −0.33, P < 0.001; Fig. 3D), indicating that neurons with heterogeneous subfield tuning were weakly tuned to uniform color patches (n = 10 sessions, N = 63 neurons).

Homogeneous and heterogeneous neurons have different temporal dynamics

What mechanism could possibly account for the spatial heterogeneity observed in the RF tuning of V4 neurons? One possibility is that feedforward inputs from upstream cells at various retinotopic locations with diverse color tuning converge onto a recipient V4 neuron (Fig. 4A, left). Alternatively, in addition to feedforward inputs, local recurrent processing in V4 could further sharpen the color tuning within each RF subfield to give rise to the observed spatial heterogeneity (Fig. 4A, right). To investigate this issue, we examined the temporal development of color tuning in individual RF subfields as a function of the HI of color-selective neurons. Stimuli consisted of a movie sequence in which each frame was a color patch (1° × 1°) synchronized with the refresh rate of the monitor and flashed at 60 Hz (Fig. 4B). The movie stimulus was presented for 1.6 s, during which each of the 16 colors was presented six times per trial in a pseudorandom order within each subfield of homogeneous and heterogeneous cells (see Materials and Methods). We subsequently used the reverse correlation analysis (3035) on the neuronal responses (Fig. 4C) to reconstruct the tuning curves for each subfield at successive time delays (τ) with respect to movie onset. That is, for each recorded action potential, we determined which color had been presented at various preceding times in the movie sequence. Spikes from completed trials were accumulated in a two-dimensional array based on stimulus color and time delay before spiking. Mean spike counts for each stimulus color were obtained by dividing each spike counter by the number of trials and by the number of stimulus repetitions within each trial. Our analysis extends to extrastriate area V4 the use of reverse correlation analysis previously used to examine the dynamics of orientation, spatial frequency, and color tuning in V1 (3035).

Fig. 4 Differential temporal dynamics of color tuning in homogeneous and heterogeneous neurons.

(A) Schematic representation of two possible mechanisms that could account for heterogeneously tuned RFs: feedforward inputs with weak recurrent processing (left) and feedforward coupled with strong recurrent processing (right). Dashed and solid lines represent weak and strong recurrent connections. (B) Color stimulus movie flashed at 60 Hz targeting a randomly chosen subfield in a given trial. (C) Responses of two example neurons (cell 1, monkey T; cell 2, monkey M) across multiple trials to the movie consisting of a pseudorandom sequence of all 16 colors constructed from the hue wheel C(θ) as described previously. Tuning curves are constructed at multiple delays τ to examine dynamics of color tuning. Black bar represents the duration of the presentation of the color movie (1.6 s). (D) Heatmap representation of the tuning curve of an example subfield of a homogeneous cell (monkey T, HI = 16) as a function of temporal delay (y axis). (E) Same as in (D) except for a neuron (monkey M, HI = 62.5) with heterogeneously tuned RF. Note the difference in latency at which peak firing rate is attained for the example neurons. The subfield PC for each example neuron is shown to the left. (F) Mean peak tuning latency averaged over subfields as a function of the HI of neurons (N = 270). Filled color circles correspond to example neurons in (D) and (E), with blue and red representing the homogeneous and heterogeneous cells, respectively. Solid line represents linear fit to the data showing a significant positive correlation (Pearson’s correlation coefficient, r = 0.43, P < 0.001).

We illustrate the temporal behavior of two representative neurons: one homogeneous (Fig. 4D) and one heterogeneous (Fig. 4E). For both cells, color tuning gradually sharpens in time and then flattens at large time delays (because cells can only respond with a finite number of spikes to a flashed stimulus). The latency at which the subfield reached the highest CSI (see Materials and Methods) and statistically significant tuning (Rayleigh test, P < 0.05) was defined as the peak tuning latency for that subfield. Across cells, mean peak latencies spanned a wide range of values from 50 to 80 ms. Unexpectedly, the mean peak latency and the degree of RF heterogeneity were significantly correlated (Pearson’s correlation, r = 0.43, P < 0.001; Fig. 4F). That is, neurons with heterogeneous tuning within their RFs had a higher latency to peak tuning compared with homogeneous cells. In addition, the variability in peak tuning latency, i.e., the time delay when each subfield reaches peak tuning (based on CSI), showed a significantly positive correlation with the HI of the neuron (Pearson’s correlation, r = 0.74, P < 0.001; fig. S5). Across cells, we found no significant correlation between the diversity in the mean peak tuning latencies and the mean peak firing rates of subfields (Pearson’s correlation, r = −0.08, P > 0.1), which suggests that firing rate differences cannot explain the observed diversity in tuning latencies. Our findings indicate that local recurrent circuits could possibly play an important role in shaping color tuning in heterogeneously tuned neurons. That is, while the short-latency tuning of homogeneous cells could be due to feedforward mechanisms (22, 23, 25), delayed recurrent processing within local V4 circuits or inputs from downstream cortical areas other than V4 (3638), in combination with intracortical inhibition (39), could shape the color tuning of heterogeneously tuned cells.

Stimulus-specific rapid adaptation alters subfield color preference

We further examined whether the diversity of color tuning identified in heterogeneous neurons represents a fixed property of this distinct population of V4 cells or can change depending on the history of visual stimulation. This was tested using rapid adaptation to a uniform color patch at the time scale of visual fixation (31, 40), i.e., a 400-ms color patch covering the entire RF of a cell followed by a 1.6-s movie sequence in which each frame was a pseudorandom color patch (~1° × 1°) targeting a particular subfield (Fig. 5A). Rapid adaptation was previously used to examine the plasticity of orientation and motion tuning in V1 and MT (41, 42), but whether it affects color selectivity is unknown. In control trials, the adapter was replaced by an equiluminant gray screen (adaptation and control trials were randomly interleaved). Adaptation-induced changes for each subfield were quantified by calculating the shift in PC due to adaptation (Δθ) and the angular distance of the PC under control conditions from the adapter (Δϕ) (Fig. 5B). Analogous to previous reports in the orientation domain in primary visual cortex (31, 41), we found stimulus-specific effects on subfield color tuning post adaptation (Fig. 5C). When the PC of a subfield was close to the adapter (Δϕ <45°), PC shifted away from the adapter (Δθ >0, Wilcoxon signed rank test, P < 0.001), whereas adapters that were highly dissimilar from the subfield’s PC (Δϕ >120°) were associated with attractive shifts (Fig. 5D and fig. S6A). Unlike PC, tuning strength and peak firing rates of subfields did not show overall significant changes after adaptation (fig. S6, B and C). In addition, neurons with homogenous subfield tuning exhibited a significant increase in HI (Wilcoxon signed rank test, P < 0.01), whereas heterogeneously tuned neurons became less heterogeneous (Wilcoxon signed rank test, P < 0.01; Fig. 5E). This was also true when subfields were grouped on the basis of the difference between their PC and that of the adapting stimulus (fig. S7). These results indicate that rapid adaptation alters the representation of color within the subfields of V4 neurons in a stimulus-specific manner.

Fig. 5 Stimulus-specific adaptation differentially alters color tuning in heterogeneous and homogeneous neurons.

(A) Schematic description of adaptation and control trials to examine the changes in tuning for each subfield. (B) Schematic depiction of Δθ (change in PC) and Δϕ (angular difference between the preadaptation PC and that of the adapter) on the color circle defined in Fig. 1D. Red line, angular position of adapter; yellow line, preadaptation PC; and green line, post adaptation PC. (C) Three representative examples of subfields showing repulsive (Δθ = 48°, monkey M), attractive (Δθ = −18°, monkey T), and almost negligible shifts (Δθ = 3°, monkey M) after adaptation for different Δϕ values. Black arrow shows the position of the adapting color in terms of color angle. (D) Mean change in PC (Δθ) of individual subfields post adaptation as a function of the distance between the adapting color and subfield’s PC before adaptation (Δϕ). Filled red circles represent mean of the binned values of (Δθ) for bin widths of Δϕ = 22.5°. Asterisks denote statistically significant difference (two-tailed Wilcoxon signed-rank test, P < 0.01) from a distribution with zero median. (E) Change in HI post adaptation as a function of HI prior to adaptation. Filled red circles represent mean of binned values for bin widths of Δϕ = 22.5°. Asterisks denote statistically significant difference from 0 (two-tailed Wilcoxon signed-rank test, P < 0.01). Error bars in (C) to (E) represent SEM.

Homogeneous and heterogeneous neurons encode different chromatic information

Could the presence of heterogeneous and homogeneous RF neurons constitute a possible strategy used by the visual system to encode different chromatic aspects of a visual scene? One possibility is that neurons with homogeneous RFs would be necessary for processing image patches with uniform chromatic properties, whereas heterogeneously tuned neurons would extract local chromatic variations. Together, these two classes of cells could efficiently process complex chromatic stimuli in the natural environment (Fig. 6A). Alternatively, both these classes of cells could be equally informative about uniform and nonuniform image patches as long as at least one of the RF subfields is optimally stimulated. To test these hypotheses, we used nonuniform stimulus patches covering the entire RF of a heterogeneous neuron, designed such that each subfield was stimulated at its PC (Fig. 6, B and D), and ensured that these stimuli optimally stimulated at least one subfield of a homogeneously tuned neuron. In addition, we used uniform color patches covering the entire RF of both homogeneous and heterogeneous cells. In each trial, we briefly flashed (200 ms) either a uniform stimulus chosen randomly from our set of 16 colors or a nonuniform stimulus. We divided the cells into homogeneous and heterogeneous classes based on the dip in the HI histogram at HI = 25 (Fig. 3A; see Materials and Methods), i.e., cells with HI <25 were classified as homogeneous, and cells with HI >25 were classified as heterogeneous.

Fig. 6 Functional role of homogeneous and heterogeneous RFs.

(A) Schematic description of functional significance of homogeneously and heterogeneously tuned neurons. (B) Example spike raster of a homogenously tuned neuron for spatially uniform (blue) and nonuniform color stimuli (red). (C) Baseline-corrected PSTHs of the example neuron on the left stimulated at its PC (inset) using a full-RF uniform stimulus (blue trace, inset; Rayleigh test, P < 0.01) and for a nonuniform stimulus (red trace). Shaded region represents stimulus duration. (D) Example spike raster of a heterogeneous neuron across trials for uniform/nonuniform stimuli. (E) Red/blue traces represent the PSTH of responses to a nonuniform/uniform stimulus; no significant tuning (Rayleigh test, P > 0.05) when full-RF is stimulated with uniform color patches (inset). Solid lines and shaded regions (blue and red): means and SEM. (F) Baseline-subtracted peak responses for homogeneous (N = 47, blue circles) and heterogeneous cells (N = 45, filled blue circles) stimulated with full-field uniform color patches. Black filled circles with bars represent means/SEM of responses for each group. (G) Same as in (F) except when both cell classes are stimulated with nonuniform color patches customized based on PC of subfields of heterogeneous cells. Filled black circles: mean value for respective cell classes. Asterisks: statistical significance (Wilcoxon rank sum test, P < 0.05) (H) Schematic description of how homogeneously and heterogeneously tuned neurons can represent different patches of a complex image through firing rate differences. Image taken from McGill Calibrated Colour Image Database (76) (photographer: unknown, McGill University).

Example neurons (Fig. 6, B to E) illustrate that cells with homogeneous RF tuning exhibit stronger responses to a uniform color patch for which the cell is tuned (Rayleigh test, P < 0.001; Fig. 6C, inset) compared with a nonuniform color patch stimulus containing different colors with at least one subfield stimulated at its PC. This indicates that homogeneous cells send stronger signals to downstream neurons when all the subfields are stimulated with their PC as opposed to only some of the subfields being optimally stimulated. In contrast, neurons with heterogeneous RF tuning exhibit a significantly stronger response to a nonuniform color patch compared with a uniform color stimulus (Wilcoxon signed rank test, P < 0.001; Fig. 6, D and E); the example cell shows no significant color tuning when a uniform color patch stimulates its entire RF (Rayleigh test, P > 0.05; Fig. 6E, inset). A population-level analysis (N = 92 units) confirmed these findings—uniform and nonuniform color images evoke significantly different firing rates (Wilcoxon rank sum test, P < 0.01) in these two classes of neurons (Fig. 6, F and G). That is, uniform color patches elicit significantly stronger responses in homogeneously tuned neurons, whereas images composed of diverse color patches elicit stronger responses from heterogeneously tuned cells. In addition, uniform and nonuniform stimuli evoked significantly different responses within the same cell class (fig. S8). Together, these results indicate that chromatically homogeneous and heterogeneous RF neurons are best suited for extracting different types of chromatic information from natural scenes (Fig. 6H).

DISCUSSION

The distribution of color information in natural scenes is widely variable, ranging from uniform patches to highly nonuniform regions in which different color patches lie in close proximity. Whether the neurons encoding color information are adapted to this high degree of variability of chromatic signals has been unknown. The diverse RF organization of V4 neurons revealed here may constitute an adaptation to the diversity of color signals present in natural environments.

Previous color studies have shown that double opponent neurons in primary visual cortex respond differentially to cone opponent color signals at different spatial locations within their RFs (4345). The cone opponent spatial organization of responses within RFs has been shown to have either a concentrically organized center-surround structure (44) or an oriented structure (43). The heterogeneously tuned population of neurons reported here is different from the typical double opponent neurons in V1 as follows: First, the different RF subfields of the same neuron are optimally tuned to different stimuli. Second, the number of unique stimuli (i.e., different equiluminant colors distributed across the color circle) that are represented within the individual subfields of V4 neurons (which increases with the HI; Fig. 3C) is relatively large. Last, heterogeneously tuned neurons do not exhibit any distinct spatial arrangement of red and green colors within their RFs as has been reported in double opponent cells in V1 (43, 44). In addition, the exact spatial properties of RFs of the double opponent cells in V4 (16, 46) have not been as extensively characterized using cone opponent stimuli as in V1. Given the increase in complexity of visual features processed by downstream areas of the visual cortex, how converging inputs from double and single opponent cells in upstream visual areas V1 and V2 shape the RF properties of color-selective neurons in V4 remains a research topic for future studies.

An important issue is the spatial organization of heterogeneously tuned neurons in V4. While the spatial distribution of homogeneous color-tuned neurons has been extensively examined using intrinsic signal optical imaging and electrophysiology (14, 15, 20, 47), whether heterogeneous V4 neurons are functionally organized in modules similar to the organization of homogeneous color-tuned cells is unknown. One scenario could be a random arrangement of heterogeneous cells or mixed arrangements in which heterogeneous cells are interspersed with homogeneous neurons. However, while the spatial resolution of our electrode system, i.e., 400 μm, is unable to answer this question, future high-resolution optical imaging studies should focus on elucidating the spatial arrangement of the different functional types of color-tuned V4 neurons. In addition, future studies will examine the columnar organization of homogeneous and heterogeneous color-tuned neurons to further our understanding of the layer specificity of color tuning in V4.

Although we used a limited set of color stimuli, we were still able to find a substantial number of neurons in V4 with heterogeneity in color tuning within their RFs. It is likely that using a broader range of color stimuli would reveal an even greater degree of color-tuning heterogeneity in V4. To further extend this work, future studies should move beyond synthetic stimuli and focus on characterizing the responses of V4 neurons to color properties of natural images as a means to further probe the role of homogeneous and heterogeneous neurons in generating an accurate representation of complex stimuli as those encountered during natural viewing. The combination of homogeneous and heterogeneous RFs could form the basic building blocks of a more complex representation of the natural environment (4850) in higher visual areas, such as Inferior Temporal cortex.

What could be the underlying mechanism of spatial heterogeneity of color tuning in V4 populations? Our results based on the temporal evolution of tuning in heterogeneous neurons indicate significantly higher latencies compared with homogeneously tuned neurons. This asymmetry suggests the possibility of a significant degree of intracortical processing of color information by recurrent networks within V4 similar to that observed in the orientation domain in V1 (30, 51). The fact that heterogeneously tuned neurons require extra recurrent processing compared with homogeneous cells is unexpected, although it could probably be explained by the higher complexity of their RF structure. On the basis of the scheme in Fig. 2C where the RF of a heterogeneous V4 neuron can be divided into 25 subfields of various color preferences and assuming a finite set of possible colors, e.g., 16, the maximum number of color combinations that can be assigned to individual RFs exceeds 2 million (it is equal to C2516). Even if we were to discount this total number of combinations by three orders of magnitude (due to a significant proportion of untuned subfields or color repetitions), we are still left with thousands of possible combinations of heterogeneous neurons, which may increase the burden on local recurrent processing to ensure sharp color tuning. However, to further understand how feedforward, recurrent, and feedback connections shape the tuning dynamics and spatial heterogeneity, detailed computational models of color processing need to be implemented. This provides an interesting avenue for future work on color processing in V4.

Our results indicate that homogeneous and heterogeneous cell populations may be well suited to extract different types of color signals from the environment. Uniform color patches elicit stronger responses from homogeneously tuned neurons, whereas nonuniform patches containing multiple colors in close proximity elicit stronger responses from heterogeneously tuned cells. This is beneficial because higher spike counts can ensure stronger depolarization of postsynaptic targets in downstream neurons and, possibly, increased neuronal coordination of spike timing between cortical areas (52). From a metabolic cost perspective, the encoding of complex stimuli by single heterogeneous neurons may be more efficient than encoding schemes relying on the joint activation of multiple homogeneously tuned neurons. This could lead to a sparser representation of natural images by neuronal populations in agreement with the efficient coding hypothesis (53).

The presence of orientation tuning on a finer spatial scale (54) and overall shape/form tuning in V4 (11, 5559) cannot explain the responses of heterogeneous neurons to nonuniform equiluminant color stimuli. One key difference is that such shape tuning is mediated by luminance-defined edges or contours. The subfield tuning reported in our study provides an additional mechanism through which complex equiluminant contours defined by multiple colors could be distinguished by neurons in V4. This capability would be absent if neurons only had spatially homogeneous RFs tuned to a single color, in which case complex contours could be differentiated only on the basis of changes in luminosity. In addition, it has been shown that the latency to peak response of V4 neurons to curved contours exceeds 95 ms (60), which is larger than the peak tuning latencies of the heterogeneous neurons reported here. Our findings suggest that neurons in V4 could use both the heterogeneity of color tuning and the sensitivity to shape/form on a finer scale to efficiently process highly complex structure present in natural scenes.

Last, although V4 neurons are sensitive to orientation, luminance changes, shape and form, heterogeneous color tuning could enable them to efficiently represent edges and boundaries in natural images from color information alone as orientation and color in natural scenes have been shown to be statistically independent (6163). Hence, luminance and orientation changes in natural images may be insufficient to signal chromatic changes. The observed heterogeneity within the RFs of this newly discovered population of V4 neurons could help create an accurate description of the rich diversity of chromatic information contained in natural environments. The information extracted by heterogeneously tuned V4 neurons reported here could be further transmitted to downstream cortical area to help encode more complex chromatic features in natural environments, such as color-defined curvature and higher-order face and object representations (4850, 6466).

MATERIALS AND METHODS

All experiments were performed in accordance with protocols approved by the U.S. National Institutes of Health Guidelines for the Care and Use for Experimental Procedures and the Institutional Animal Care and Use Committee at the University of Texas Health Science Center at Houston. Data presented in this study were collected from two adult male rhesus monkeys (Macaca mulatta; T, 14 years old, 13 kg; M, 10 years old, 10 kg). A titanium head post was surgically implanted in the medial frontal region with the help of multiple anchor screws. After a recovery period of 4 weeks, both animals were trained for a month on visual fixation tasks (to be used later for the recordings) involving at least 1000 trials per session. After the monkeys learned to complete multiple sessions in a single day, we implanted a 96-channel Utah array in monkey T and a 64-channel Utah array in monkey M in area V4. Coordinates for craniotomies were estimated on the basis of locating the superior temporal sulcus (STS) and the lunate sulcus by comparing the magnetic resonance imaging images from both animals to brain atlases. During surgery, the grooves corresponding to the STS and the lunate sulcus were used to guide the implantation of the array. Arrays were roughly implanted at the crown of the prelunate gyrus. Postsurgery, animals went through a 2- to 3-week recovery period and additional training for reacclimatization with previously learned fixation tasks before we started recording.

Visual stimuli for single-unit recordings

Visual stimuli were presented on a gamma-corrected cathode ray tube (CRT) monitor (HP p1230). To measure the color tuning, we used a set of 16 equiluminant colors (9.35 ± 0.01 cd/m2) spanning the full color gamut of the monitor and presented at the maximum saturation allowed by the monitor. These 16 colors were uniformly spaced when plotted in the CIELUV color space (67, 68) (Fig. 1D), which is designed to be perceptually uniform. L represents the luminosity, and u,v represent the chromaticity coordinates in a two-dimensional perceptually uniform color space. Luv coordinates were measured for each color and the gray background using a Tektronix photometer (J17 LumaColor) before the start of each recording session. These colors were presented on a neutral gray background designed to have the same luminosity as the color stimuli (see table S1). Visual stimuli were presented binocularly; eye tracking was performed for only one of the eyes.

Behavioral task

Two monkeys were trained to fixate on a small point (0.2°) within a small rectangular 1° window at the center of a CRT monitor while remaining head fixed. If at any point during the trial, eye position exceeded 0.25° outside the boundaries of the rectangular box, then the trial was automatically aborted. Animals were rewarded with juice at the end of each trial, in which fixation was successfully maintained for the entire duration of stimulus presentation. Eye movements were monitored throughout the recording session using an infrared eye tracking system (EyeLink II, SR Research) at a 1-kHz sampling rate. Stimulus presentation was recorded and synchronized with the neural data using a programmable Experiment Control Module device (FHC Inc.).

Electrophysiological recordings

We recorded extracellular activity as action potentials and local field potentials simultaneously from all 96 channels (monkey T) and 64 channels (monkey M) of two chronically implanted Utah arrays (Blackrock Microsystems) while animals performed passive fixation tasks. The interelectrode spacing in these arrays was 400 μm. Data were recorded at a sampling rate of 30 kHz using a Cerebus Neural Signal Processor (Blackrock Microsystems LLC). Spike waveforms above threshold (~4 SD above the amplitude of the noise signal) were saved and sorted after data acquisition using Plexon’s Offline Sorter. Spike waveforms were manually sorted with Plexon’s Offline Sorter program using waveform clustering parameters such as spike amplitude, spike width, timing of the valley, and peak. Units that formed well-separated clusters in principal components space were identified as single or multiunits. Units that had more than 2% of their postsorted spikes within the refractory period (2 ms) were classified as multiunits and were eliminated from the analysis (44). We sorted 323 stimulus-responsive channels, of which 53 were eliminated on the basis of the above criteria. For example, percent refractory period violations are 1.1, 0.7, and 0.5% for cells 1, 2, and 3 in Fig. 2A. The remaining single units (N = 270) were subsequently analyzed using custom scripts in MATLAB. We treated each recording session performed on a particular day as an independent session, which is a common approach in multiple laboratories (6972), as it is difficult to determine whether the same units were recorded on subsequent days over the course of several months.

RF mapping

To map the RFs of the single units, we divided the right visual field into a 3 × 3 grid consisting of nine squares with each square covering 8° × 8° of visual space. The entire grid covered 24° × 24° of visual space. Each of the nine squares was further subdivided into a 6 × 6 grid (fig. S4A). In each trial, one of the nine squares was randomly chosen, and the RF mapping stimuli were presented at each of the 36 locations in a random order. The RF mapping stimuli consisted of a reverse correlation movie with red, blue, green, and white patches (~1.33° each). A complete RF session is composed of 10 presentations of the RF mapping stimuli in each of the nine squares forming the 3 × 3 grid. We averaged the responses over multiple presentations to generate the RF heatmaps as shown in Fig. 2 (see fig. S4B for additional examples). RF mapping was done at the beginning of each recording session, as it was impossible to track the same neurons over the course of the recordings, which lasted several months. The position and size of the target stimulus for examining tuning were chosen each day so that it roughly covered the overlapping RFs of only a subset of neurons. Neurons whose RFs could not be covered completely by the chosen target were not included in the analysis. This ensured that the size of the target could be kept small so as to minimize surround stimulation.

Sub-RF tuning task

To examine subfield tuning, the full RF (approximately 5° to 7°) was divided into a finer 5 × 5 grid (25 subfields). A color stimulus movie composed of a pseudorandom sequence of the 16 equiluminant colors was presented at a 60-Hz monitor refresh rate in each trial (no gray frame between colors). In each trial, the stimulus appeared at a random location chosen from the 25 possible subfield locations. Each color was presented for 33 ms (two frames), three times within a trial, and these instances were randomly distributed throughout the length of the trial. The total duration of the movie in each trial was 1.6 s (96 frames at 60 Hz). We recorded roughly 20 to 30 trials per subfield (~500 to 600 trials in a session) to characterize their tuning properties. In the task, after the animal had maintained fixation for 200 ms, the reverse correlation movie was presented. The animal was rewarded with juice if it maintained fixation for the entire duration. The next trial started at least 4.6 s after the last frame of the reverse correlation movie was displayed.

Color tuning

To determine the color tuning of the full RF and each subfield of V4 neurons, we calculated the tuning curve at multiple delays using the reverse correlation color movie (full-field or sub-RF stimulation). For each such tuning curve, we calculated the CSI in the following way: CSI=rieθiiri where ri is the response to a particular color with angular position θi on the color circle (Fig. 1D). Responses ri were vectorially summed and then normalized by the sum of the responses for all colors. CSI lies between 0 and 1, where a value of 0 represents no tuning and a value of 1 indicates that the neuron is highly tuned for a specific color. To evaluate whether the neuron’s tuning was statistically significant, we used the Rayleigh test for nonuniformity in circular data (13, 19, 73). Rayleigh’s test is a statistical test used to determine whether a circular distribution (which, in our case, is the circular tuning curve with raw firing rates) has a preferred direction, i.e., a PC the neuron is tuned to (P < 0.05). The null hypothesis is that the circular tuning curve represents a uniform circular distribution and no preferred direction exists (P > 0.05). We evaluated CSI and the P values of the tuning curves at multiple delays. To evaluate the PC, we selected the tuning curve with the highest CSI that passed the Rayleigh test for nonuniformity (P < 0.05). We selected the first local maximum in the temporal evolution of CSI to eliminate the possibility of analyzing tuning of off responses. This tuning curve was then used to calculate PC in the following way: PC=tan1(Im(rieθi)Re(rieθi)), where Im and Re stands for the imaginary and real parts of the complex sum. The delay at which this tuning curve was evaluated was defined as the latency for peak tuning. Following the above method, we calculated the PC (PCi) and the peak tuning latency of each subfield. To consider the effect of noise on estimating tuning properties, we did the following: (i) We eliminated subfields with peak firing rates <5 Hz and CSI <0.2. (ii) For each subfield, we generated multiple tuning curves from the responses for each trial rt(θ), where r is the response, t is the trial number, and θ is the stimulus angle. For each of these tuning curves, we calculated the PC(θ1PC,θ2PCθNPC)of the subfield. We then calculated the circular SD or HI from these set of PCs for a particular subfield. Higher HI values imply that the tuning curves peak at different colors in different trials (lower reliability) and lower values indicate similar PCs estimated from each trial (higher reliability). We picked a threshold of HI = 25 based on the dip in the histogram in Fig. 3A, which approximately separates neurons with similar subfield tuning from ones with diverse tuning. Subfields with HI >25 were deemed as untuned/noisy and were excluded from the analysis. The implementation of the constraints mentioned above ensured that low firing rates or trial-by-trial variability did not influence our findings.

To determine the number of unique colors represented within the RFs of each neuron, we calculated the PC for each subfield from their respective tuning curves. These PCs could take on any values between 0° and 360°, since they were calculated using the vector summation method described above. To each PC, we assigned a stimulus number 1 to 16 by calculating which of the 16 colors in our stimulus set, 0° (stimulus 1), 22.5° (stimulus 2), 45° (stimulus 3)……360° (stimulus 16), the estimated PC was closest to. The maximum number of unique stimuli that could be represented within an RF would be 16, i.e., equal to the number of stimuli used in this study.

HI in degrees of the entire RF was calculated from the PC of each subfield (PCi) using the circular SD (74) as follows: HI=180π2(1R), where R = ∣∑iePCi∣. HI is a bounded measure lying between 0 and 81 and quantifies the variability circular data. HI = 0 indicates that the PC of all subfields is identical, indicating homogeneous tuning, whereas a nonzero HI indicates that the PCs of the subfields are different. Circular statistics was computed using the CircStat Toolbox (75) for MATLAB.

Adaptation task

For a subset of sessions (n = 20), an adapter preceded the presentation of the reverse correlation movie. In adaptation trials, a full-field uniform color stimulus was presented for 400 ms. After a 50-ms delay, the color movie described above was presented to measure the color tuning post adaptation. A complete session involved randomly interleaved adaptation and control trials (no adapter). We recorded close to 1000 trials in these sessions split roughly equally between control and adaptation conditions. The duration between the last frame of the reverse correlation movie and the first frame of the stimulus in the next trial was at least 4.6 s, which allowed sufficient time for the responses to return to baseline.

Nonuniform stimulus trials

We presented uniform and nonuniform color patches to stimulate the entire RF of neurons. In each trial, one of the above two classes of stimuli were flashed for 200 ms to measure the responses of homogeneous and heterogeneous RFs. This time scale of stimulus presentation was chosen so as to match fixation times encountered during natural viewing (31, 40). Trials with either kind of stimuli were randomly interleaved. Each session consisted of roughly 800 trials, split evenly between the two classes of stimuli. The duration between when the stimulus was turned off and stimulus onset in the next trial was at least 4.6 s, which allowed sufficient time for the responses to return to baseline. For the analysis presented in Fig. 6, we divided the population of neurons into homogeneous and heterogeneous groups based on an HI value of 25, which was estimated on the basis of the “dip” in the bimodal Gaussian fits from the distribution of HI in both animals (fig. S3). Post Stimulus Time Histograms (PSTHs) were generated with spike counts evaluated in 75-ms windows sliding by 10 ms.

Statistical analysis

Quantification and statistical test for tuning were performed using the Rayleigh test implemented with the CircStat Toolbox (75). Correlations were quantified using either Pearson’s correlation or Spearman’s rank correlation to account for linear and nonlinear trends. We used the nonparametric Wilcoxon’s signed-rank test (two-tailed) to quantify whether distributions had medians significantly greater than or less than zero. In case of two distributions with unequal sample size, we used the Wilcoxon rank sum test to examine the statistical significance of the difference in their medians.

SUPPLEMENTARY MATERIALS

Supplementary material for this article is available at http://advances.sciencemag.org/cgi/content/full/7/8/eabc5837/DC1

https://creativecommons.org/licenses/by-nc/4.0/

This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial license, which permits use, distribution, and reproduction in any medium, so long as the resultant use is not for commercial advantage and provided the original work is properly cited.

REFERENCES AND NOTES

Acknowledgments: Funding: This work is supported by grants from the NIH BRAIN Initiative and NIH EUREKA program. Author contributions: S.N. and V.D. designed the experiments. S.N. performed the experiments. S.P. wrote the scripts for stimulus presentation using the Psychtoolbox. S.N. analyzed the data with guidance from V.D. S.N and V.D wrote the manuscript. Competing interests: The authors declare that they have no competing interests. Data and materials availability: All data needed to evaluate the conclusions in the paper are present in the paper and/or the Supplementary Materials. Additional data can be made available from the corresponding author upon reasonable request. The custom written software supporting the results reported in this study is available from the corresponding author upon reasonable request.

Stay Connected to Science Advances

Navigate This Article