Research ArticleAPPLIED PHYSICS

Full-field fluorescence lifetime dual-comb microscopy using spectral mapping and frequency multiplexing of dual-comb optical beats

See allHide authors and affiliations

Science Advances  01 Jan 2021:
Vol. 7, no. 1, eabd2102
DOI: 10.1126/sciadv.abd2102

Abstract

Fluorescence lifetime imaging microscopy (FLIM) is a powerful tool for quantitative fluorescence imaging because fluorescence lifetime is independent of concentration of fluorescent molecules or excitation/detection efficiency and is robust to photobleaching. However, since most FLIMs are based on point-to-point measurements, mechanical scanning of a focal spot is needed for forming an image, which hampers rapid imaging. Here, we demonstrate scan-less full-field FLIM based on a one-to-one correspondence between two-dimensional (2D) image pixels and frequency-multiplexed radio frequency (RF) signals. A vast number of dual-comb optical beats between dual optical frequency combs are effectively adopted for 2D spectral mapping and high-density frequency multiplexing in the RF region. Bimodal images of fluorescence amplitude and lifetime are obtained with high quantitativeness from amplitude and phase spectra of fluorescence RF comb modes without the need for mechanical scanning. The parallelized FLIM will be useful for rapid quantitative fluorescence imaging in life science.

INTRODUCTION

Fluorescence microscopy (1) is a powerful tool for observing the localization and migration of specific molecules and proteins in cells and tissue by molecular labeling with fluorescent probes or proteins. Although fluorescence intensity is a representative imaging modality in fluorescence microscopy, it is often influenced by not only the concentration of fluorescent molecules but also photobleaching and excitation/detection efficiency. Therefore, the absolute fluorescence intensity often lacks quantitativeness. Although the fluorescence intensity ratio between two different fluorescence or excitation wavelengths (2, 3) is an alternative method that can compensate for the drawbacks associated with absolute fluorescence intensity, it is influenced by differences in focal position between the two wavelengths.

An interesting imaging modality for quantitative fluorescence microscopy is fluorescence lifetime, which is used in fluorescence lifetime imaging microscopy (FLIM) (4). The fluorescence lifetime is determined by the types of fluorescent molecules and the environment surrounding the molecules and shows little dependence on the concentration of fluorescent molecules, photobleaching, and excitation/detection efficiency. Therefore, the fluorescence lifetime is more quantitative than the fluorescence intensity. It is also sensitive to changes in the microenvironment around fluorescent molecules, such as temperature (5) or pH (6). Furthermore, although the signals from two fluorescent molecules with similar fluorescence wavelengths are spectrally overlapped, molecular selectivity can be achieved based on different fluorescence lifetimes (7).

Fluorescence lifetime can be measured by the time-correlated single-photon counting (TC-SPC) method with pulsed excitation (4, 8, 9) or the phase measurement (PM) method with sinusoidal excitation (see Fig. 1, A and B, with the description in Materials and Methods) (4, 10, 11). Since both methods are based on point measurement, mechanical scanning of the focal point is necessary for obtaining an image using fluorescence lifetime. Such mechanical scanning often limits the frame rate of FLIM. While use of a rotating polygonal scanning mirror (12) or a combined microlens array and pinhole array (13) can boost the frame rate in fluorescence intensity microscopy, such fast mechanical scanning is not compatible with the TC-SPC or PM method. FLIM without the need for mechanical scanning, that is to say, scan-less full-field FLIM, is promising for rapid fluorescence lifetime imaging and will expand the range of applications of FLIM in life science. Since the PM method can achieve a higher data acquisition rate than the TC-SPC method, scan-less wide-field FLIM was achieved by combining the PM method with a camera (14, 15); however, the low detection efficiency of the camera hinders rapid image acquisition. Another scan-less wide-field FLIM was demonstrated by using a combination of the PM method and single-pixel imaging (16). However, the need for consecutive excitation of different spatially coded light patterns in this method hampers rapid image acquisition.

Fig. 1 Principle of operation for the PM method and the parallelized PM method.

(A) Sinusoidal excitation light and the corresponding fluorescence. (B) Phase delay between excitation light and fluorescence. (C) Parallelized PM method with spatially frequency-multiplexed excitation.

One promising method to parallelize the PM method without the need for mechanical scanning is a combination of the PM method with spatially frequency-multiplexed excitation (see Fig. 1C with the description in Materials and Methods). Spatially varying modulation of light intensity with mechanical rotation (17, 18) or scanning (19) enables the mapping of spatial position to modulation frequency for scan-less full-field FLIM. More recently, fluorescence imaging by radio frequency (RF)–tagged emission (FIRE) (2022) is demonstrated on the basis of a combination of frequency multiplexing and spectral mapping in the RF region without the need for mechanical motion. In this method, a one-dimensional (1D) array of focal spots with different RF modulation frequencies is generated by digitally synthesized optical beating with an acousto-optic deflector and is then radiated onto a sample. After passing through a confocal slit, the radiated fluorescence is detected by a point photodetector. Since the detected fluorescence signal has a one-to-one correspondence between RF beat frequencies and line pixels via frequency multiplexing and spectral mapping, a line fluorescence intensity image is reconstructed from a group of frequency-multiplexed fluorescence RF beat signals without the need for mechanical scanning of the focal point. However, the limited number of frequency-multiplexed RF beats (typically a few hundred) makes it difficult to expand this approach to scan-less 2D images. Yet, 1D mechanical scanning of the focal line is still required to obtain a 2D image. If one can increase the number of frequency-multiplexed RF beats up to its square, then scan-less full-field fluorescence imaging, namely, 2D-FIRE, could be achieved by using a combination of ultra-dense frequency multiplexing and 2D spectral mapping.

One potential method for ultra-dense frequency-multiplexed RF signals is to use an optical frequency comb (OFC) (2326). An OFC is composed of a large number of optical frequency spikes (typically tens to hundreds of thousands) with a constant frequency spacing (frep) and functions as an optical carrier with a vast number of discrete channels. Furthermore, multifrequency heterodyning interferometry between dual OFCs (frequency spacing = frep1, frep2) with slightly different frequency spacing (∆frep = frep2frep1), namely, dual-comb optical beating, generates a secondary optical beat comb in the RF region, namely, an RF comb. Since the RF comb is a replica of an OFC whose frequency scale is reduced by frep1/∆frep, it has been used for broadband high-precision spectroscopy, for example, in dual-comb spectroscopy (DCS) (2729). On the other hand, because the number of RF comb modes is equal to that of OFC modes, it is sufficient for frequency multiplexing RF signals in 2D-FIRE. 2D spectral mapping of OFC modes can be achieved by a combination of a virtually imaged phased array (VIPA) (30, 31) and a diffraction grating. Such 2D spectral mapping has been effectively used for not only optical wavelength division demultiplexing (32) and broadband spectroscopy (33) but also 2D imaging (34). Recently, 2D spectral mapping has been combined with DCS for scan-less full-field confocal microscopy based on a one-to-one correspondence between OFC modes and 2D image pixels (35, 36). However, the application of this dual-comb microscopy (DCM) is limited to coherent light, such as reflected or transmitted light, due to the use of multifrequency heterodyning interferometry.

Here, we expand DCM into scan-less full-field FLIM or 2D-FIRE, that is, so-called f-DCM, by combining ultra-dense frequency-multiplexed RF signals of dual-comb optical beats with 2D spectral mapping using VIPA and a diffraction grating. On the basis of the one-to-one correspondence between dual-comb optical RF beats and 2D image pixels, a 2D image of fluorescence intensity is obtained from the amplitude spectrum of ultra-dense frequency-multiplexed fluorescence RF signals without the need for mechanical scanning of the focal spot. Furthermore, 2D imaging of fluorescence lifetime is demonstrated on the basis of parallel PM of fluorescence RF signals, which corresponds to parallelization of the PM method.

RESULTS

Experimental setup

f-DCM is based on a combination of the 2D spectral mapping and frequency multiplexing of dual-comb optical beats (see Fig. 2), which is described in Materials and Methods together with details on the experimental and analytical methodology used for the following measurements. Figure 3A shows a schematic diagram of the experimental setup for f-DCM to implement the 2D spectral mapping and frequency multiplexing of dual-comb optical beats. A pair of green OFCs were respectively generated by a custom-built femtosecond Er:fiber OFCs (OFC1 and OFC2; frep1 = 100,385,982 Hz; frep2 = frep1 + ∆frep = 100,386,960 Hz; ∆frep = 978 Hz) with a chirped periodically poled lithium niobate (PPLN) crystal. We obtained dual-comb optical beats around 520 nm by spatially overlapping the two green OFC beams (power = 11.1 and 9.7 mW) with a beam splitter (BS1, Thorlabs Inc., Newton, NJ, USA, CCM1-BS013/M, transmission = 50%, reflectance = 50%), as shown in the inset in Fig. 3A. The green dual-comb optical beats were fed into a 2D spectral disperser, composed of a VIPA (free spectral range = 14.8 GHz, finesse = 65) and a diffraction grating (groove density = 1200 grooves/mm, blaze wavelength = 500 nm). To measure amplitude and phase of the excitation light frequency-multiplexed in RF region, a portion of the green dual-comb optical beats was extracted and detected by a combination of another beam splitter (BS2, SIGMA KOKI Co., Tokyo, Japan, BS4-25.4C03-10-550, transmission = 95%, reflectance = 5%), an optical bandpass filter (BPF1, Semrock Inc., Lake Forest, IL, USA, FF01-531/40-25, passband = 511 to 551 nm), a lens, and a PMT (PMT1, Hamamatsu Photonics K.K., Hamamatsu, Japan, H11901-20, sensitive wavelength range = 230 to 920 nm). A 2D spectrograph of the green dual-comb optical beats was formed at the optical Fourier plane of the grating and was then relayed onto a sample as a 2D array of focal points of excitation light by a combination of lenses (L1 and L2), a dichroic mirror (DM, Semrock Inc., Lake Forest, IL, USA, FF562-Di03-25x36, transmission wavelength = 569.5 to 950 nm, reflection wavelength = 350 to 554.5 nm), and a dry-type objective lens (OL, Nikon Corp., Tokyo, Japan, CFI Plan Fluor 10×, numerical aperture = 0.30, working distance = 16 mm). Fluorescence from the sample passed through the DM was filtered by another optical band-pass filter (BPF2, Semrock Inc., Lake Forest, IL, USA, FF01-593/40-25, passband = 573 to 613 nm) and was then detected with a thermoelectric cooling PMT (PMT2, Hamamatsu Photonics K.K., Hamamatsu, Japan, H7844, sensitive wavelength range = 185 to 900 nm, quantum efficiency at 590 nm = 10%). The temporal waveform of the detected electrical signal was acquired by a digitizer (sampling rate = frep2 = 100,386,960 samples/s; number of sampling points = 102,644; resolution = 14 bit). The Fourier transform of the acquired temporal waveform gives the amplitude spectrum and phase spectrum of the fluorescence RF comb. Last, images of fluorescence amplitude and lifetime were obtained on the basis of the one-to-one correspondence between fluorescence RF comb modes and fluorescence image pixels and the parallelized PM method.

Fig. 2 Principle of operation for f-DCM.

(A) Dual-comb optical beating between dual OFCs. (B) 2D spectral mapping of dual OFCs. RF multiplexing of excitation light in (C) the frequency domain and (D) the image domain. RF multiplexing of fluorescence in (E) the image domain and (F) the frequency domain.

Fig. 3 Experimental setup and sample.

(A) Schematic drawing of the experimental setup. OFC1 and OFC2, dual OFCs; PPLNs, chirped periodically poled lithium niobate crystals; BS1 and BS2, beam splitters; L1 and L2, lenses; DM, dichroic mirror; OL, objective lens; BPF1 and BPF2, optical band-pass filters; and PMT1 and PMT2 PMTs. Inset shows optical spectra of OFC1 and OFC2 after converting into green light. a.u., arbitrary units. (B) Schematic drawing of the sample.

As a fluorescent sample, we combined a 1951 United States Air Force high-resolution target (negative type, size = 2 inch by 2 inch, minimum line spacing = 1.55 μm/line pairs) with a fluorescent solution. Figure 3B shows a schematic drawing of the sample. We used the target as a spatial mask and put a chamber containing a fluorescent solution behind the target. The 2D array of focal spots of dual-comb optical beats comes from the target side, passes through the transparent region of the target, and excites fluorescence in the chamber. The fluorescence radiates in all directions from the fluorescent solution, and its backpropagating component passes through the target and thus contains information about the target in the form of a target image. We prepared four kinds of fluorescent solutions with different lifetimes, as shown in Table 1.

Table 1 Characteristics of prepared fluorescent solutions (37, 38).

View this table:

Scan-less imaging of fluorescence intensity

We first demonstrated scan-less imaging of fluorescence intensity. A rhodamine 6G aqueous solution (molar concentration = 300 μM, excitation wavelength = 461 to 533 nm, fluorescence wavelength = 543 to 617 nm, and fluorescence lifetime = 4.08 ns) (37) was filled into the chamber to serve as the fluorescent sample. We measured the region around the group 6/element 1 pattern (line spacing = 39 μm/line pairs) of the target. Figure 4A shows a temporal waveform of fluorescence RF comb modes from the sample (sampling interval = 97 fs, number of accumulated signals = 100,000, data acquisition time = 102 s), in which the effective time and lab time are given as the lower (window size = 9.96 ns) and upper scales (window size = 1.02 ms). The effective time and lab time are related by a factor of frep1/∆frep (100,385,982/978 = 102,644). The temporal behavior of the fluorescence RF comb modes was observed as a center burst. The Fourier transform of the temporal waveform gives the amplitude and phase spectra of the fluorescence RF comb modes. Figure 4B shows the amplitude spectrum of the fluorescence RF comb modes. As many as 44,400 high-density distributed RF modes appeared, as shown by the filled region inside the spectrum. The spectral envelope reflects the 2D distribution of fluorescence intensity in the sample. Then, we reconstructed the fluorescence intensity image from the amplitude spectrum based on the one-to-one correspondence between RF comb modes and 2D image pixels. Figure 4C shows the reconstructed fluorescence amplitude image (image size = 234 μm by 79 μm, pixel size = 300 pixels by 148 pixels). The fluorescence amplitude image of the target pattern was confirmed to have a high image contrast. Spatial resolution of f-DCM depends on the spectral resolution of the 2D-SE optical system, composed of VIPA and diffraction grating, rather than the spectral resolution of DCS or the imaging performance of OL. The VIPA with the spectral resolution of 230 MHz determines the spatial resolution along the vertical axis to be 1.2 μm; on the other hand, the grating with the spectral resolution of 32 GHz determines the spatial resolution along the horizontal axis to be 1.7 μm.

Fig. 4 Scan-less imaging of fluorescence intensity.

(A) Temporal waveform and (B) amplitude spectrum of fluorescent RF comb modes. (C) Fluorescence intensity image reconstructed from the amplitude spectrum of fluorescent RF comb modes.

We next investigated the dependence of the signal-to-noise ratio (SNR) on the number of accumulated signals in the fluorescence intensity imaging. Figure 5 (A to D) shows fluorescence intensity images when the number of accumulated signals was set to 100 (data acquisition time = 102 ms), 1000 (data acquisition time = 1.02 s), 10,000 (data acquisition time = 10.2 s), and 100,000 (data acquisition time = 102 s), respectively. The image contrast changed depending on the number of accumulated signals. A little uneven distribution of background noise is due to the frequency-dependent electric noise in PMT and the following electronics. For a more quantitative analysis, we defined image SNR as the ratio of the mean to the SD of the fluorescence intensity for a region of interest in the image. Figure 5E shows the relation between the number of accumulated signals and image SNR. We also show the corresponding data acquisition time on the upper scale. The image SNR linearly increased depending on the square root of the number of accumulated signals or the data acquisition time, indicating that shot noise–limited measurement was achieved. We consider that the electrical noise of PMT is dominant rather than the laser noise.

Fig. 5 Fluorescence intensity image with respect to the number of accumulated signals.

(A) 100, (B) 1000, (C) 10,000, and (D) 100,000. (E) Relation between number of accumulated signals and image SNR.

Scan-less bimodal imaging of fluorescence lifetime and fluorescence intensity

We performed scan-less imaging of fluorescence lifetime in addition to fluorescence intensity for a different region of the same sample as in the experiment above. Figure 6A shows a temporal waveform of fluorescence RF comb modes from the sample (window size of effective time = 9.96 ns, sampling interval = 97 fs, number of accumulated signals = 100,000, and data acquisition time = 102 s). By performing Fourier transformation of the temporal waveform, we obtained the amplitude and phase spectra of fluorescence RF comb modes as shown in Fig. 6 (B and C). Then, we reconstructed two kinds of image, namely, in fluorescence intensity and fluorescence phase delay, based on the one-to-one correspondence between RF comb modes and 2D image pixels, as shown in Fig. 6 (D and E) (image size = 234 μm by 79 μm, pixel size = 300 pixels by 148 pixels). Last, we calculated the fluorescence lifetime image of the sample from the fluorescence phase delay image based on the PM method (see Materials and Methods), as shown in Fig. 6F. The image background was set to be gray color in Fig. 6E and black color in Fig. 6F depending on threshold of background noise in Fig. 6D. The test pattern was clearly confirmed in those images. However, note that the mechanism of the image contrast is different between them. The mean and SD of fluorescence lifetime in the image region were 4.1 and 0.7 ns, respectively. The mean fluorescence lifetime agrees well with the lifetime value reported in the literature for this fluorescence solution (4.08 ns) (37). The relatively large SD may be caused by the fact that the actual 2D phase distribution of the excitation light is uneven depending on the spherical aberration and/or spatial overlapping of dual-comb optical beats. Also, note the position dependence of time resolution in the fluorescence lifetime image. The time resolution of fluorescence lifetime measurement is determined by the phase resolution and RF frequency of dual-comb optical beats (see Materials and Methods). To evaluate the phase resolution of the present system, we repeated 100 acquisitions of fluorescence phase delay image without signal accumulation and then calculated the SD of the phase value at a certain pixel in 100 fluorescence phase delay images to provide the phase resolution. The resulting phase resolution was 0.13 rad, independent of sample position. On the other hand, RF beat frequency is different among sample positions within a frequency range of 5 to 50 MHz as shown in Fig. 6 (B and C). In other words, the obtained fluorescence lifetime image shows the position dependence of the time resolution: better time resolution for the sample position with higher RF beat frequency and worse time resolution for that with lower RF beat frequency. For example, the time resolution is estimated to be 4.1 ns for the 5-MHz beat frequency and 0.41 ns for the 50-MHz beat frequency. This position dependence of time resolution may be another reason for the relatively large SD of fluorescence lifetime. Fortunately, since the worse time resolution was adopted for the measurement around the edge of fluorescence images in Fig. 6 (E and F), it does not decrease quantitativeness of fluorescence lifetime measurement. Also, the moderate accumulation of multiple fluorescence phase delay images will improve the time resolution of the fluorescence lifetime.

Fig. 6 Scan-less bimodal imaging of fluorescence lifetime and fluorescence intensity.

(A) Temporal waveform, (B) amplitude spectrum, and (C) phase spectrum of fluorescent RF comb modes. (D) Fluorescence intensity image, (E) fluorescence phase delay image, and (F) fluorescence lifetime image.

To evaluate the quantitativeness of fluorescence lifetime imaging, we acquired fluorescence lifetime images of samples with different fluorescence lifetimes as shown in Table 1. The corresponding fluorescence lifetime images (image size = 234 μm by 79 μm, pixel size = 300 pixels by 148 pixels, number of accumulated signals = 100,000, and data acquisition time = 102 s) are shown in Fig. 7: (A) rhodamine 6G aqueous solution, (B) rhodamine B aqueous solution, (C) rhodamine B methanol solution, and (D) rhodamine B ethanol solution. While all images showed similar patterns of the test chart, the lifetime values were different among them: (A) 4.1 ± 0.7 ns, (B) 1.8 ± 0.4 ns, (C) 2.4 ± 0.6 ns, and (D) 3.0 ± 0.5 ns. Figure 7E shows a correlation between the literature values (37, 38) and the measured values of fluorescence lifetime. A high correlation value (0.96) was obtained between them. When the measurement accuracy is defined as a root mean square error between measured values and literature values, the measurement accuracy of 0.07 ns was achieved in this experiment, indicating the high quantitativeness of fluorescence lifetime imaging using f-DCM.

Fig. 7 Fluorescence lifetime images of samples with different fluorescent lifetimes.

(A) Rhodamine 6G aqueous solution, (B) rhodamine B aqueous solution, (C) rhodamine B methanol solution, and (D) rhodamine B ethanol solution. (E) Comparison of fluorescence lifetime between literature values and measured values.

DISCUSSION

We first discuss one-to-one correspondence between OFC modes and 2D image pixels in f-DCM. One may consider that the spectral resolution of VIPA is insufficient for the separation of each OFC mode. From a free spectral range of 14.8 GHz and a finesse of 65 in VIPA, the spectral resolution of VIPA is estimated to be 230 MHz, indicating that a few OFC modes are included in each spot of the 2D spectrograph corresponding to the image domain (see Fig. 2, D and E). However, in the frequency domain (see Fig. 2, C and F), Fourier transform of the temporal waveform with a full repetition period (see Fig. 4A or Fig. 6A) gives mode-resolved spectra of amplitude and phase (see Fig. 4B or Fig. 6, B and C) corresponding to pixel-resolved 2D image of fluorescence amplitude and phase. In other words, although it is difficult to separate each mode in the image domain, one can separate each mode in the frequency domain. Benefiting from the discrete spectrum of OFC, there is no ambiguity of one-to-one correspondence between OFC modes and 2D image pixels.

We next discuss the possibility of further increasing the image acquisition rate with the proposed f-DCM method. In principle, the maximum rate of image acquisition is equal to ∆frep, because the temporal waveform of image-encoded fluorescence RF comb modes is acquired at a rate of ∆frep. In the present setup, ∆frep was set to 978 Hz; in other words, single-shot imaging can be achieved at 978 Hz. However, the scan-less full-field fluorescence imaging demonstrated above needs a number of accumulated images to obtain a high-SNR image (see Fig. 5E). The reason why image accumulation is needed is the considerably weak excitation light, which is limited by the efficiency of nonlinear wavelength conversion (typically 10%) in the PPLN crystal and signal loss (typically 1%) in the VIPA. Considering the total power of excitation light (242 μW) at the focus and the number of 2D image pixels (149 × 200 = 29,800 pixels), the excitation power per pixel was only 8 nW, which is much lower than that in the conventional PM method (typically on the order of microwatts) (10, 11) or FIRE (2022). Since the image SNR in the proposed system was shot noise–limited, as shown in Fig. 5E, a higher total power of excitation light will efficiently contribute to improvement of the image SNR or reduction of the image acquisition time. The efficiency of nonlinear wavelength conversion can be increased by pulse shortening of the excitation light with precise compensation of linear and nonlinear chirp (39). On the other hand, the large signal loss in the VIPA is due to the use of zero-order dispersion mode with the highest spectral resolution. Use of higher-order dispersion mode will lead to the decreased signal loss at the expense of spectral resolution. From the viewpoint of signal detection, there is still space to increase the image SNR by use of an OL with higher numerical aperture and a PMT with higher quantum efficiency.

We also discuss the possibility of achieving confocality in the f-DCM. Confocal fluorescence microscopy is a powerful tool for low-invasive 2D optical sectioning or 3D imaging in life science, because it provides confocality, which enables depth selectivity on the order of micrometers, together with stray light elimination, in thick biological specimens. In usual confocal fluorescence microscopy, confocality is achieved by placing a detector pinhole at a conjugate position with respect to a focused excitation spot in a specimen. In the case of f-DCM, since a 2D array of focal spots is formed in the sample, one has to place a 2D array of pinholes at a conjugate position with respect to the 2D array of focal spots. One possible approach for implementing a 2D pinhole array with flexibility is to use a spatial light modulator (SLM). Work is in progress to install an SLM as a flexible 2D pinhole array into f-DCM.

Last, we compare f-DCM with the state-of-the-art TC-SPC or PM method. While the TC-SPC is confocal and benefits from a low dark noise in the photon-counting detection, the acquisition speed can be boosted up to about several tens of megahertz per channel. However, since the number of parallel channels is insufficient for full-field 2D image, the mechanical scanning of a focal spot or line is still required, hampering real-time FLIM imaging. On the other hand, the combination of the PM method with a modulated 2D camera allows a frame rate of 45 frames per second (fps) with 1 megapixel in higher-intensity FLIM imaging, benefitting from higher quantum efficiency and low read noise in the state-of-the-art complementary metal-oxide semiconductor camera. However, further increase of frame rate, acquisition of lower-intensity FLIM image, and/or giving of confocality will be challenging for this combination. f-DCM has a potential to increase a frame rate up to 1000 fps and an option of confocality discussed above. Tunability of dual-comb optical beats and corresponding 2D image pixels via frequency control of OFC1 and OFC2 will be useful for further increase of 2D image pixels via pixel interleaving and/or FLIM imaging of multiple fluorophores. The high photon rate in f-DCM does not allow for photon-counting detection and hence the noise is larger by the excess noise of PMT compared with TC-SPC. No use of photon-counting detection is an essential drawback of f-DCM for lower-intensity FLIM imaging similar to the camera-based PM method. However, 1-bit photon correlation technique (40) may be used for the acquisition of temporal waveform in fluorescent RF comb modes with photon-counting detection.

In summary, we demonstrated f-DCM for scan-less full-field fluorescence lifetime imaging and fluorescence intensity imaging. Use of dual-comb optical beats between dual OFCs enables us to expand the DCM (35, 36) to fluorescence microscopy. The scan-less full-field imaging capability of f-DCM was achieved by 2D spectral mapping and frequency multiplexing of dual-comb optical beats. The maximum acquisition rate of fluorescence lifetime imaging was 978 Hz, although image accumulation was required to obtain a high SNR. The measurement accuracy of 0.07 ns in fluorescence lifetime measurement was ensured in the scan-less imaging by using a parallelized PM method with high-density multiplexed RF comb modes. We discussed the possibility of further reducing the image acquisition time of f-DCM and the compatibility with confocal fluorescence microscopy. f-DCM will be a powerful tool for rapid quantitative fluorescence microscopy.

MATERIALS AND METHODS

PM method and its parallelization with spatially frequency-multiplexed excitation

Figure 1A shows a schematic drawing of the PM method when excitation light with modulation frequency b is incident on the sample. The resulting fluorescence is also modulated synchronously with the excitation light modulated at b; however, there are differences of amplitude and phase between the excitation light and the fluorescence, as shown in Fig. 1B. These differences are related to fluorescence intensity and lifetime. The fluorescence lifetime τ is calculated from the phase delay ϕ and the modulation frequency b byτ=tanϕ2πb(1)

We next consider parallelizing the PM method for scan-less full-field FLIM. Figure 1C shows a principle of parallelized PM method with spatially frequency-multiplexed excitation. 2D array of excitation spots pi is generated with different amplitude modulation frequencies bi to have one-to-one correspondence between pi and bi. When such 2D array is used for 2D fluorescence imaging, the excitation of PM methods is spatially parallelized with different modulation frequencies. The resulting 2D array of fluorescence spots pi has the same amplitude modulation frequencies bi as the 2D array of excitation spots pi; however, there are differences of amplitude and phase between the amplitude-modulated excitation light and the corresponding fluorescence based on the principle of the PM method (see Fig. 1B). When all of the fluorescence RF beat signals are detected collectively by a point photodetector, the fluorescence intensity image can be obtained by parallel measurement of amplitude values of the modulated fluorescence for all modulation frequencies corresponding to 2D image pixels. On the other hand, 2D image of fluorescence lifetime τi can be reconstructed from ϕi and bi based on one-to-one correspondence between pi and bi and Eq. 1.

2D spectral mapping of dual-comb optical beats

When an OFC (OFC1: frequency spacing = frep1, optical frequency = νi = ν1, ν2, ν3, ···, νn) beam is spatially overlapped with another OFC beam with a slightly different frequency spacing (OFC2: frequency spacing = frep2 = frep1 + ∆frep), a group of optical beats is generated as dual-comb optical beats (beat frequency = bi = ifrep = ∆frep, 2∆frep, 3∆frep, ···, nfrep) via multifrequency heterodyning interference between adjacent modes of OFC1 and OFC2, as shown in the upper part of Fig. 2A. In the RF region, such dual-comb optical beats show a number of frequency spikes with different frequencies bi (= ∆frep, 2∆frep, 3∆frep, ···, nfrep), namely, an RF comb, as shown in the lower part of Fig. 2A. Since the RF comb is regarded as a replica of OFC1 in which the frequency spacing is downscaled from frep1 to ∆frep, a one-to-one correspondence is established between OFC modes νi and RF comb modes bi. In other words, such dual-comb optical beating is equivalent to tagging each OFC mode νi with a unique RF beat frequency bi.

2D spectral mapping of OFC modes is achieved by a 2D spectral disperser, composed of a VIPA and a diffraction grating (35). The 2D spectral disperser enables us to spatially develop OFC modes as a 2D spectrograph of zigzag lines depending on the optical frequency or wavelength, as shown in Fig. 2B, enabling another one-to-one correspondence between 2D image pixels pi and OFC modes νi. When beams of both OFC1 and OFC2 are simultaneously fed into the 2D spectral disperser, each adjacent mode pair of dual-comb optical beats is spatially overlapped and forms a common focal point at different positions depending on their optical frequency.

Scan-less 2D imaging with RF-multiplexed fluorescence signals

2D spectral mapping of dual-comb optical beats enables us to combine (i) the one-to-one correspondence between OFC modes νi and RF comb modes bi with (ii) the one-to-one correspondence between 2D image pixels pi and OFC modes νi. This results in a one-to-one correspondence between 2D image pixels pi and RF comb modes bi at the fluorescence excitation side, as shown in Fig. 2 (C and D). The resulting dual-comb optical beats with high-density frequency multiplexing are suitable for a frequency-multiplexed excitation source in 2D-FIRE. When 2D focal points of dual-comb optical beats with different RF modulation frequencies excite a fluorescent sample, the fluorescence appears from the corresponding 2D spots with equivalent modulation frequencies as a fluorescence RF comb. At the fluorescence detection side, 2D image pixels pi have a one-to-one correspondence with fluorescence RF comb modes bi, as shown in Fig. 2 (E and F). Therefore, by detecting the frequency-multiplexed fluorescence signal with a point photodetector and reconstructing a 2D image based on the one-to-one correspondence between fluorescence 2D image pixels pi and fluorescence RF comb modes bi, a full-field 2D fluorescence image is acquired without the need for mechanical scanning of the focal spot.

Experimental setup

Figure 3A shows a schematic diagram of the experimental setup. We used a pair of custom-built femtosecond Er:fiber OFCs (OFC1: center wavelength = 1560 nm, spectral bandwidth = 15 nm, mean output power = 167 mW, fceo1 = 21.4 MHz, frep1 = 100,385,982 Hz; OFC2: center wavelength = 1560 nm, spectral bandwidth = 15 nm, mean output power = 121 mW, fceo2 = 21.4 MHz, frep2 = frep1 + ∆frep = 100,386,960 Hz, ∆frep = 978 Hz). Near-infrared light from OFC1 and OFC2 was converted into green light via a combined wavelength-conversion process of second harmonic generation and sum frequency generation in chirped PPLN crystals (HC Photonics Corp., Hsinchu, Taiwan, TY-0299, operating wavelength range = 1540 to 1570 nm). We obtained dual-comb optical beats around 520 nm by spatially overlapping the two green light beams (power = 11.1 and 9.7 mW) with a beam splitter (BS1, Thorlabs Inc., Newton, NJ, USA, CCM1-BS013/M, transmission = 50%, reflectance = 50%), as shown in the inset in Fig. 3A. Since the spectral overlapping of these green lights was not so good, we extracted the spectral region with better overlapping (≈522 to 526 nm) from them with a band-pass filter (not shown in Fig. 3A); this also suppresses aliasing in the RF region. The green dual-comb optical beats were fed into a 2D spectral disperser, composed of a VIPA (LightMachinery Inc., Nepean, Ontario, Canada, OP-6721-6743-2, free spectral range = 14.8 GHz, finesse = 65) and a diffraction grating (Thorlabs Inc., Newton, NJ, USA, GR25-1205, groove density = 1200 grooves/mm, blaze wavelength = 500 nm). To measure amplitude and phase of the excitation light frequency-multiplexed in RF region, a portion of the green dual-comb optical beats was extracted and detected by a combination of another beam splitter (BS2, SIGMA KOKI Co., Tokyo, Japan, BS4-25.4C03-10-550, transmission = 95%, reflectance = 5%), an optical band-pass filter (BPF1, Semrock Inc., Lake Forest, IL, USA, FF01-531/40-25, passband = 511 to 551 nm), a lens, and a PMT (PMT1, Hamamatsu Photonics K.K., Hamamatsu, Japan, H11901-20, sensitive wavelength range = 230 to 920 nm). A 2D spectrograph of the green dual-comb optical beats was formed at the optical Fourier plane of the grating and was then relayed onto a sample as a 2D array of focal points of excitation light by a combination of lenses (L1 and L2), a DM (Semrock Inc., Lake Forest, IL, USA, FF562-Di03-25x36, transmission wavelength = 569.5 to 950 nm, reflection wavelength = 350 to 554.5 nm), and a dry-type OL (Nikon Corp., Tokyo, Japan, CFI Plan Fluor 10×, numerical aperture = 0.30, working distance = 16 mm). Fluorescence from the sample passed through a DM was filtered by an optical band-pass filter (BPF2, Semrock Inc., Lake Forest, IL, USA, FF01-593/40-25, passband = 573 to 613 nm) and was then detected with a thermoelectric cooling PMT (PMT2, Hamamatsu Photonics K.K., Hamamatsu, Japan, H7844, sensitive wavelength range = 185 to 900 nm, quantum efficiency at 590 nm = 10%). The temporal waveform of the detected electrical signal was acquired by a digitizer (National Instruments Corp., Austin, TX, USA, NI PCI-5122, sampling rate = frep2 = 100,386,960 samples/s, number of sampling points = 102,644, resolution = 14 bit). The Fourier transform of the acquired temporal waveform gives the amplitude spectrum and phase spectrum of the fluorescence RF comb. Last, images of fluorescence amplitude and lifetime were obtained on the basis of the one-to-one correspondence between fluorescence RF comb modes and fluorescence image pixels and the parallelized PM method.

https://creativecommons.org/licenses/by-nc/4.0/

This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial license, which permits use, distribution, and reproduction in any medium, so long as the resultant use is not for commercial advantage and provided the original work is properly cited.

REFERENCES AND NOTES

Acknowledgments: Funding: The work was supported by grants for the Exploratory Research for Advanced Technology (ERATO), the Japan Science and Technology Agency (MINOSHIMA Intelligent Optical Synthesizer Project, JPMJER1304), the Japan Society for the Promotion of Science (18H01901, 18K13768, and 19H00871), the Cabinet Office, Government of Japan (Subsidy for Regional University and Regional Industry Creation), the Nakatani Foundation for Advancement of Measuring Technologies in Biomedical Engineering, and the Research Clusters Program of Tokushima University (1802003). Author contributions: T.Y. and H.Y. conceived the project. T. Miz. and E.H. performed the experiments and/or analyzed the data. T. Miz. and T.Y. wrote the manuscript. T. Min., Y.T., R.O., and H.K. discussed the results and commented on the manuscript. Competing interests: T.Y., T. Min, T. Miz., E.H., and H.Y. are inventors on a patent related to this work filed by the Japan Science and Technology Agency (no. 10837906, 17 November 2020). T.Y., T. Min., T. Miz., E.H., and H.Y. are inventors on a pending patent related to this work filed by the Japan Science and Technology Agency (no. WO2019/031584, 14 February 2019). The authors declare that they have no other competing interests. Data and materials availability: All data needed to evaluate the conclusions in the paper are present in the paper. Additional data related to this paper may be requested from the authors.

Stay Connected to Science Advances

Navigate This Article