Research ArticleAPPLIED SCIENCES AND ENGINEERING

Microscopic scan-free surface profiling over extended axial ranges by point-spread-function engineering

See allHide authors and affiliations

Science Advances  28 Oct 2020:
Vol. 6, no. 44, eabc0332
DOI: 10.1126/sciadv.abc0332

Abstract

The shape of a surface, i.e., its topography, influences many functional properties of a material; hence, characterization is critical in a wide variety of applications. Two notable challenges are profiling temporally changing structures, which requires high-speed acquisition, and capturing geometries with large axial steps. Here, we leverage point-spread-function engineering for scan-free, dynamic, microsurface profiling. The presented method is robust to axial steps and acquires full fields of view at camera-limited framerates. We present two approaches for implementation: fluorescence-based and label-free surface profiling, demonstrating the applicability to a variety of sample geometries and surface types.

INTRODUCTION

Surface profiling (profilometry) is used to characterize the topography of a surface. Because the functional properties of a material can be strongly influenced by the surface structure (13), profilometry has many important applications, e.g., in catalysis (4), biomechanics (5, 6), and pathology (7). In designing functional materials, profilometry is vital over a broad range of length scales, e.g., airfoils, 101 m (8); lenses, 10−2 m (9, 10); and thin-film structures and biocompatible materials, 10−6 to 10−9 m (11, 12), the latter scale being the primary subject of this work.

Profilometry techniques can be divided into two categories: contact (including pseudo-contact) and noncontact methods (13). Contact methods, namely, stylus profilers and scanning probe microscopy techniques, are based on moving a cantilever-mounted tip across the surface of a material. Surface topography induces tip displacements (14), deflections (15, 16), or conductance changes (17), which are then correlated with vertical displacement to produce topographic maps, with resolution down to the subnanometer scale in three dimensions (18). Contact techniques, however, have several limitations. First, close proximity to a specimen is not feasible for particularly rough surfaces, namely, surfaces with feature sizes on the scale of the probe. Second, measurements may impart detrimental effects on a sample, e.g., surface-texture deformation. Third, the temporal resolution for scanning dynamic samples is limited by the scan rate, often on the order of seconds or more depending on the field-of-view (FOV) size (19), although high-speed atomic force microscope can be used to substantially decrease the acquisition time (16). Noncontact methods are typically based on electron microscopy or optical imaging. Scanning electron microscopy can directly image an entire surface, providing topographic information at nanometer resolution, albeit with limited quantitative depth information (20). In addition, most instruments require a vacuum; thus, the conditions may not be suitable for a variety of samples. Furthermore, because surface charging produces image artifacts, it is sometimes necessary to deposit a thin metallic layer onto nonconductive sample, e.g., biological samples.

Optical methods have been widely applied to microprofilometry as well (21) and benefit from being nondestructive (for samples that are not light sensitive) and applicable to a broad range of sample sizes. Another key advantage of scan-free optical techniques is the high-speed acquisition rate, essential for observing fast-time dynamics in microscale objects, e.g., microelectromechanical system (MEMS) (22), deforming elastomers (23), and living specimen (5, 6).

The most commonly applied optical microprofilometry methods are based on interferometry. Briefly, an illumination path is split; one channel is reflected off a reference mirror, and the other off the sample, before recombining the paths. Phase-shifting interferometry (PSI) uses a spectrally narrow beam and extracts nanoscale axial information based on the optical path difference between the beams. The method relies on controlled phase shifts obtained by mechanically scanning the reference mirror (24). Problematically, because of the cyclic interference signal, there is a 2π-phase ambiguity that limits the measurements to relatively smooth surfaces, i.e., surfaces with axial discontinuities smaller than a quarter wavelength (25). To overcome the phase ambiguity, coherence scanning interferometry (CSI) uses a broadband light source, thus enabling the measurement of surfaces with larger step heights and discontinuities (26, 27). While the axial resolution of CSI is inferior with respect to PSI, nanoscale axial resolution can still be recovered using frequency domain analysis (28). Both PSI and CSI require axial scanning that fundamentally limits their temporal resolution.

Digital holography microscopy (DHM) is a scan-free interferometric technique that enables computational reconstruction of three-dimensional (3D) structures from a single image (29). The technique achieves subnanometer axial resolution and diffraction-limited lateral resolution (21). Similarly to PSI, there is phase ambiguity, requiring phase unwrapping. This is problematic for features with steep slopes or abrupt steps greater than a half wavelength, where the reconstruction can fail to recover the true topography (30). Resolving the phase ambiguity is possible by combining coherence interferometry principles with holography (31).

Another approach to measure the topography of a 3D surface is to apply 3D microscopy techniques, such as stereoscopic microscopy (e.g., by structured light projection) (32), multiplane imaging, or confocal microscopy (7, 33). Another technique is 3D localization microscopy using transiently bound fluorescent emitters to achieve a high spatial density, namely, 3D point accumulation for imaging in nanoscale topography (PAINT) (34). Very recently, it was suggested that optical aberrations of a microscope could be exploited to localize fluorescent particles attached to a MEMS surface, acquiring axial information at specific positions simultaneously on the order of the depth of field of the imaging system (35).

The standard point-spread-function (PSF) of an optical system is ill-suited for depth imaging, because (i) it is approximately symmetric above and below the focal plane and (ii) the precision to which the position of a point source can be determined (localized) degrades rapidly with defocus. To circumvent this limitation, PSF engineering can be used. PSF engineering is a method that enables extended 3D imaging by modifying the shape that a point source forms on the camera to encode depth information. The modification of the PSF shape can be achieved by adding a phase-shaping element to the optical system, e.g., a liquid crystal spatial-light modulator (LC-SLM) or a fabricated phase mask (3638). Using this method, the axial position of a point source can be encoded efficiently over ~40 times the depth of field of a standard microscope (39).

Here, we leverage PSF engineering to implement a noncontact microprofilometry technique that overcomes several key limitations of existing methods. The approach is scan-free, and therefore inherently fast, not restricted to smooth surfaces, that is, it can handle surfaces with substantial axial discontinuities or irregularities, such as axial steps. Furthermore, it can be implemented as an add-on to existing light microscopes. We demonstrate two modes of implementation (i) using fluorescent emitters scattered on a surface of a 3D dynamic object and (ii) a label-free method using reflected light from a projected pattern of illumination spots. For both implementations, we use the Tetrapod PSF, which has been optimized to contain the maximum amount of information on the 3D position of a point source (38, 39). PSF engineering–based profilometry is applicable to a variety of engineering fields, particularly those requiring dynamic measurements of large FOVs and axial ranges, such as MEMS, microfluidics, and soft actuators.

Principle of the method

The topography of a surface can be estimated using discretely measured positions at various points. Localization microscopy by PSF engineering relies on images of spatially separated point sources whose 3D positions are robustly encoded. We obtain the spatial separation, either by using scattered fluorescent emitters or by illuminating an array of spots along the surface, to reconstruct a topographic map (Fig. 1A).

Fig. 1 Localization-based profilometry.

(A) Point sources for 3D localization can be produced using illuminated fluorescent beads attached to the surface and wide-field illumination, or by projecting an array of spots using the DMD-based illumination module (illumination shown in green). In both modes, the light-collection path (red) is extended with a 4f system containing an SLM in the Fourier plane that bestows the 3D PSF. (B) Top: The Tetrapod phase mask, displayed on the SLM, is designed to encode an axial depth range of 600 μm (using a 4× objective, 0.13 NA). Bottom: Simulated and measured Tetrapod PSFs. Scale bar, 20 μm.

In our implementation, we use a standard fluorescence microscope with two modifications: (i) the illumination of the instrument is controlled by a digital micromirror device (DMD) to switch between wide-field illumination (for sparse fluorescent emitter samples) and patterned illumination, i.e., an array of spots. (ii) The light-collection side of the microscope is extended with a 4f optical processing system using two lenses, with a phase modulating device (i.e., LC-SLM) placed in the conjugate back focal plane (BFP) of the objective lens.

The SLM modulates the wavefront using the Tetrapod phase-mask pattern, and by this, the 3D position at each measurement point in the sample is inferred algorithmically. The phase mask is designed for a desired depth range and a given set of optical system parameters: numerical aperture (NA), magnification, emission wavelength, and SLM pixel size (39). Figure 1B shows the depth dependence of the simulated and experimentally measured Tetrapod PSF designed for a 600-μm range (40).

RESULTS

Fluorescence-based PSF profilometry

To demonstrate the applicability of PSF engineering for dynamic microprofilometry, we first characterized a deformable elastic membrane (Fig. 2). In brief, a thin polydimethylsiloxane (PDMS) sheet was bonded to a glass-backed 10-mm-thick PDMS slab, containing a 1.25-mm-diameter cylindrical cavity attached to a syringe (Fig. 2A). The air pressurize in the cavity was adjusted to inflate and deflate the membrane. For measuring the resulting deformation, the outer membrane was decorated with a low density of fluorescent beads and imaged using the optical setup depicted in Fig. 1A. The membrane was imaged continuously, for 10 s with 20-ms exposure time and a frame rate of 50 Hz, while being inflated and deflated; this caused each bead’s PSF shape to change dynamically with its axial location.

Fig. 2 Dynamic PSF profilometry based on scattered fluorescent beads.

(A) Fluorescent beads are sparsely immobilized on an elastic membrane, which is deformed by controlling its inner pressure. The membrane is imaged under the microscope at 50 Hz using a 4× 0.13 NA objective. (B) Top: Two captured fluorescence video frames (after subtracting the background). Top left: Membrane in its flat position showing that most of the fluorescent emitters are in focus. Top right: Deformed membrane showing that most of the emitters are defocused. Insets: Close-up images of a single scattered emitter. Scale bar, 200 μm. Bottom: Corresponding 3D reconstructions of the frames. The red dots represent the 60 3D localized beads; saturated and overlapping beads were excluded from the analysis. Color map represents the axial displacement of the membrane, based on a linear interpolation of 100 grid points. (C) DHM line reconstructions for 10× 0.3 NA objective. The curves show partial view of the flat, concave, and convex states of the membrane. For the convex state, at angles above 4.7°, the DHM method fails to reconstruct the surface.

To reconstruct the 3D dynamics of the membrane, we first localized the 3D positions of each bead per frame, using an interpolated calibration dictionary that was generated by scanning a single fixed bead (Fig. 1B; see Materials and Methods). Next, we interpolated the positions of 60 fluorescent beads to render the membrane in each frame (movie S1). Two example frames and associated surface reconstructions of the uninflated and inflated (convex) states are shown in Fig. 2B. The difference between the extreme concave and convex states was ~512 μm, corresponding to nearly 85% of the designed phase-mask axial range (Fig. 1B), for a 3.3 × 3.3 mm2 FOV. The maximum deformation of the membrane was calculated in terms of radius of curvature, by fitting a sphere to the reconstruction, to be 1 mm (the mean slope angle obtained for this case is ~25°, as calculated from the maximal axial depth and the membrane radius).

To estimate the resolution of our approach, we considered the lateral and axial sources of error separately. The lateral resolution is affected by two factors: the lateral localization precision of each point and the emitter density used to reconstruct the surface relative to its topography features. We measured the lateral localization precision to be 188 nm in x and 179 nm in y, 15 times smaller than the effective pixel size; this measurement was based on the localizations of all 60 beads (scattered on the membrane) during 120 video frames (fig. S1). The lateral sampling frequency is determined by the emitter density. Here, the beads were attached at low density, and only the 60 beads that their PSFs were not overlapping or saturated were used for the reconstruction, to simplify the analysis (mean lateral distance of ~143 μm, see note S1). Hence, according to Nyquist sampling, the maximal lateral frequency that can be resolved corresponds to a distance of ~286 μm, which is sufficient under the assumption regarding the membrane smoothness. For more complex samples, a substantial degree of emitter overlap can be handled using more sophisticated fitting algorithms (41).

The axial precision was determined to be 5.3 μm (fig. S1), seven times smaller than the depth of field, defined byDOF=λnNA2(1)where n is the refractive index of the objective’s immersion media (air), λ is the emission wavelength, and NA is the numerical aperture of the imaging system (42).

To compare our approach to the current state of the art, namely, holography, we performed an analogous measurement using a commercial, reflection-based DHM device (see Materials and Methods). The reflection profilometer we used is configured with a single laser source (666 nm), hence is limited to relatively smooth surfaces with shallow slopes.

To capture the full lateral extent of the membrane, we used a 2.5× 0.07 NA objective to view a 2.64 × 2.64 mm2 region. At the maximum deformation obtained in this DHM measurement, i.e., the largest deformation correctly reconstructed, the membrane’s radius of curvature was 19 mm (fig. S2), which is 19 times worse than using our PSF engineering approach (Fig. 2B) (the maximum mean slope angle that could be measured in this DHM measurement was ~1°). To measure larger slopes with the DHM device, we used a 10× 0.3 NA objective, where only a ~35% of the membrane could be visualized (0.66 × 0.66 mm2; Fig. 2C). For this measurement, the corresponding radius of curvature was 5.6 mm, 5.6 times worse than our full-membrane approach (the maximum mean slope angle obtained in this DHM measurement was ~4.7°). For both objective configurations, deformation beyond the maximal slope caused incorrectly reconstructed surfaces, exhibiting nonsmooth jumps (Fig. 2C, fig. S2, and movies S2 and S3). Figure 2C presents the deformations obtained for a 10× 0.3 NA objective, fig. S2 provides the deformations for a 2.5× 0.07 NA objective, and movies S2 and S3 show the corresponding dynamic DHM line reconstructions of the deformed membrane. One of the reasons for DHM technique failure in this case is its inherent 2π-phase ambiguity, noted in Introduction.

Label-free PSF profilometry using projected illumination points

In many cases, adhering fluorescent particles to the sample is not possible or ill-advised, e.g., when the sample must be kept clean. Here, we show that an engineered illumination path can be used to replace the fluorescent emitters with an array of illuminated points, projected onto a reflective sample. As illustrated in Fig. 1A, the illumination-engineering module consists of a DMD inserted into the illumination path of the microscope in a position conjugate to the sample plane. The pattern on the DMD, namely, an array of spots, is projected onto the sample (fig. S3) (43). Analogous to the fluorescent sample, light reflected from the sample is modulated in the light-collection path, where the PSF engineering is performed.

Tilted reflective plane. While fluorescent point sources emit light in all directions, reflective samples do not. To demonstrate that our technique is still applicable even for inclined planes, we used a tilted silicon wafer illuminated with a patterned 640-nm laser (Fig. 3A). For this measurement, we used a 100× 1.45 NA oil objective and a Tetrapod phase mask encoding an axial range of 4 μm. A single illuminated spot was used to build the PSF calibration dictionary (Fig. 3B and fig. S3). Figure 3C presents captured images of a 70-spot illumination array on the reflective surface for both the standard PSF (i.e., using the SLM without any mask) and the Tetrapod PSF. In the standard PSF image, the signal-to-noise ratio (SNR) of the spots varies substantially over the FOV, because the depth extends far beyond the objective’s depth of field. In contrast, the engineered Tetrapod PSFs are well observed far above and below focus, efficiently encoding the axial location in their shape.

Fig. 3 PSF profilometry based on a projected point array.

(A) Illustration of the experimental setup for profiling a tilted surface, imaged using 100× objective, 1.45 NA. The illumination array of spots is projected on the surface and reflected back. (B) Top: Tetrapod phase mask designed for a depth range of 4 μm. Bottom: Corresponding measured dictionary PSFs for different axial locations along the desired depth range. Scale bar, 2 μm. (C) Captured images (10-ms exposure time) of the reflected spots from the tilted surface. Right: Using the Tetrapod PSF, high SNR throughout the axial range is maintained, and the PSF shape corresponds to the axial location. Left: Standard PSF, when no mask is used on the SLM. The contrast of the image within the red rectangle is stretched to illustrate SNR degradation with defocus. Scale bar, 10 μm. (D) Reconstructed 3D tilted surface. The red dots represent the localized illumination spots. The color map represents the deviation from a fitted plane. (E) Illustration of the experimental setup for profiling a step in height, imaged under the microscope, 4× objective, 0.13 NA. (F) Top: Tetrapod phase mask for a depth range of 600 μm. Bottom: Corresponding measured dictionary PSFs. Scale bar, 20 μm. (G) Captured images (100-ms exposure time) of the reflected illumination grid from the two surfaces. Left: Obtained standard PSFs; the contrast of the image within the red rectangle is stretched. Right: Obtained Tetrapod PSFs (directly visible from both surfaces). Scale bar, 200 μm. (H) Reconstructed 3D step. The localized spots are represented as red dots. The color map indicates the axial (z) location, based on a linear interpolation of 250 grid points in x and 50 in y.

The reconstruction of the tilted planar surface was attained from the localized positions (Fig. 3D). The deviations from the planar fit at each measured position provides an estimate for the axial precision, which is represented by the color map of the surface. The standard deviation of the position differences from the fitted plane was 33 nm. To verify that these deviations from the plane represented the axial precision and not an underlying nonplanarity of the sample, we repeated the experiment under different illumination conditions and obtained uncorrelated error patterns (fig. S4). The lateral sampling frequency, i.e., the distance between two projected focused spots in the sample plane, was 8.32 μm; however, higher densities could be achieved with different illumination patterns and smaller engineered PSFs (41) or by sacrificing temporal resolution using time-varying illumination patterns.

Reflective stepped surface. The most challenging surfaces to profile are those with large axial steps. For contact methods, this is due to geometrical constraints; for interferometric optical methods, the step creates a redundancy in phase. A main advantage of our method is that it is applicable in the case of axial steps. To demonstrate our technique’s robustness to such geometries, we constructed a stepped metallic surface with a designed step height of 170 μm (Fig. 3, E to H). We imaged the reflective stepped surface using a 4× 0.13 NA objective and designed the corresponding phase mask for a range of 600 μm; the obtained dictionary of the engineered PSFs is shown in Fig. 3F.

Images of the illumination spots for both the standard PSF and the Tetrapod PSF are shown in Fig. 3G. For the standard PSF, the axial displacement only permits one surface at a time to be clearly visible. In contrast, using the Tetrapod, the PSFs on both surfaces are easily observed. The lateral sampling frequency, determined by the illumination pattern, was 208 μm. After localization, the step height was measured by subtracting the absolute mean of the axial positions on the left side of the step from that of the right. This measurement, 172 ± 13 μm, was in line with a physical measurement using calipers, 170 ± 10 μm (Fig. 3H). The obtained measurement uncertainty of our method in this case (±13 μm) is in the order of magnitude of the deformed-membrane experiment axial precision (5.3 μm), which was performed using the same objective (4× 0.13 NA). A discussion of localization precision is performed in note S2.

Using the DHM profilometer described above, with a 2.5× 0.07 NA objective, the height of the step could not be resolved. This is due to the inherent limitation of the profilometer to measure abrupt changes in height, as specified by the manufacturer to be a maximal 340-nm step height, when using a single wavelength, which is ~500 times smaller than our step.

Dynamically tilting mirror. To demonstrate the applicability of the presented label-free profilometry for dynamic samples, we constructed a tilting mirror setup as shown in Fig. 4A. Using our method, we were able to quantify the damped-oscillation mirror dynamics, as demonstrated by plotting the angle value of the fitted plane (based on our surface reconstructions), relative to the steady-state resting angle, for each video acquisition frame (Fig. 4D and movie S4). The setup consists of a mirror glued on a metal rod and both glued to a vertical card stock. The rod was placed on the microscope stage, and the mirror was imaged using a 20× 0.75 NA objective. By gently tilting the card stock paper and then releasing it, we caused the mirror to undergo damped oscillatory motion. The mirror was imaged continuously, for 2.2 s, with 20-ms exposure time and a frame rate of 50 Hz. The depth range of its movement spanned ~43 μm. As shown in movie S4, the mirror movement caused each illumination spot’s PSF shape to change dynamically with its axial location (we used the same phase mask presented in Fig. 1B and rescaled it to match the 20× 0.75 NA objective). Two example frames and associated mirror surface reconstructions of two mirror states are shown in Fig. 4B, where deviations from the planar fit at each measured position are represented by the color map of the surface. By computing the average of these deviations for the 112 video frames, we obtained the pattern presented in Fig. 4C. From the pattern, a minor field-dependent aberration is observed, which could be corrected by using a different PSF dictionary per FOV area.

Fig. 4 Dynamic PSF profilometry using a projected point array.

(A) Illustration of the experimental setup. The mirror (blue) is imaged at 50 Hz using a 20× objective, 0.75 NA. After tilting the mirror and releasing it, the mirror undergoes a damped oscillatory motion. (B) Top: Two captured video frames at the upper and lower positions. Bottom: Corresponding surface reconstructions. Red dots represent the localized illumination spots in 3D. The reconstruction is based on a linear interpolation of 50 grid points. Scale bar, 50 μm. (C) Averaged deviation from a fitted plane for 112 video frames. Scale bar, 50 μm. The color map represents the deviation from the fitted plane. (D) Experimentally derived angles of the fitted plane during the video acquisition time. The angles presented are relative to the steady-state resting angle (which was ~5° relative to the xy plane).

Scaling of the depth range

The applications demonstrated in this work span a depth range between several micrometers to about half a millimeter. To quantify the scalability of the approach in terms of depth range, we can characterize the applicable depth range of the PSFs as a function of the objective used.

Consider an objective imaging a point source. The phase shift in the BFP due to axial displacement relative to the focal plane can be approximated asΦ=2πnλz1(NAnρ)2Taylor2πnλz[112(NAnρ)2](2)where n is the refractive index of the objective’s immersion media (assuming no refractive index mismatch between the immersion media and the sample) (40). λ is the emission wavelength, z is the axial location of an emitter (zero at the focal plane), and 0 < ρ < 1 is the normalized polar coordinate in the BFP. In the Taylor series expansion presented in Eq. 2, orders beyond the second order were neglected. This approximation is formally correct only for NA values sufficiently smaller than 1 but, in practice, holds well even for larger NAs, as can be seen by the similarity of the two insets in Fig. 5, captured with different objectives.

Fig. 5 PSF depth range for air and oil objectives as a function of the NA.

The analytical expression for the depth range is given in Eq. 3. Insets: Experimentally measured Tetrapod PSFs obtained using the same phase mask presented in Fig. 1B (laterally rescaled to match the oil objective BFP size). The left inset is a measured PSF that corresponds to a 4×, 0.13 NA air objective at an axial location of 160 μm (300-ms exposure time, obtained from the dictionary presented in Fig. 1B). Scale bar, 20 μm. The right inset is a PSF that corresponds to a 100×, 1.45 NA oil objective at axial location of 2 μm (100-ms exposure time, obtained by imaging a 200-nm fluorescent bead on a coverslip). Scale bar, 2 μm.

For a specific axial position, up to a constant phase, the phase shift can be approximated asΦ2πnλz12(NAnρ)2=πλzNA2nρ2(3)

Hence, it is possible to obtain approximately the same phase shift in the BFP using objectives with different NAs (under normalized radial BFP units), as long as z is changed accordingly. This means that the same PSF shape (up to a lateral scaling by the objective magnification) can be obtained using the same phase-mask pattern, with appropriate lateral scaling (see insets in Fig. 5) (44).

DISCUSSION

We presented a method for contactless optical microprofilometry based on PSF engineering. Our approach addresses key limitations in microsurface profiling, thus enabling the characterization of challenging samples. Specifically, main advantages include high-speed, simultaneous acquisition of a large FOV (limited by detector size and acquisition rate), versatile depth ranges (by selecting the relevant phase mask), and complete robustness to axial steps. Importantly, the method can be used on dynamic or otherwise challenging samples using immobilized fluorescent emitters or in a label-free fashion by point-array illumination.

We have demonstrated the versatility of our profilometry method over a diverse set of samples and depth ranges: a dynamic inflatable membrane on the submillimeter range, a reflective stepped sample on the hundreds of micrometer range, and static and dynamically tilting reflective planes on the tens to single micrometer range. The applicable axial range can, in principle, be extended even further, as needed, practically constrained by the optical imaging setup and specifically by the objective lens. We found that the label-free approach works best for highly reflective samples. Samples with surface structures smaller than the size of the illumination spot pose a challenge because of local aberrations to the PSFs; however, these aberrations can be overcome to some extent by continuously shifting and averaging the sample.

Practically, the profilometry method presented in this work is simple, necessitating only an add-on to a standard optical microscope, and the PSF modulation can be achieved in a variety of ways, including an LC-SLM, a deformable mirror (45), or a dielectric mask (46).

One way to take advantage of our approach is to combine it with DHM technology to solve the inherent ambiguity for large phase jumps while simultaneously attaining nanoscale depth sensitivity. Last, the lateral resolution can be improved by algorithmic enhancements to allow dense emitter localizations in 3D (41) or for the engineered illumination application by temporally changing the illumination patterns to scan the array laterally.

MATERIALS AND METHODS

Experimental design

Optical setup. The experiments described throughout the paper were performed using the profilometry setup shown in Fig. 1A, which uses a customized inverted fluorescence microscope (Eclipse Ti, Nikon). For both the illumination-engineering module and the 4f system, achromatic lenses were used (Thorlabs) with the following focal lengths: L1: f = 30 mm, L2: f = 75 mm, L3: f = 150 mm, L4: f = 100 mm, L5: f = 150 mm, L6: f = 400 mm, L7: f = 200 mm, L8: f = 150 mm, L9: f = 150 mm. In the 4f system, we placed a linear polarizer (Thorlabs) to remove light polarized in the unmodulated direction of the SLM.

In the illumination-engineering module, the light exiting the laser fiber was focused on a rotating diffuser (1500-grit ground glass diffuser, Thorlabs) for beam decoherence. This light was then collimated by a 4× 0.13 NA air objective (Plan Fluor, Nikon) and expanded (by a factor of 2) to cover the full DMD. Because a DMD acts as a diffraction grating, the intensity of the diffracted light from the DMD is distributed between different diffraction orders. These orders were focused and then blocked by an iris to transmit only the order with the maximal intensity. The pattern was then focused by a Köhler lens to the BFP of the objective and projected onto the sample (note S3).

For the deformed-membrane experiment, fluorescent microspheres 542/612 nm, 2 μm in diameter (Fluoro-Max, Thermo Fisher Scientific), were diluted 1:1200 in deionized water and distributed randomly on the surface. The emitters were excited with 561-nm laser light (iChrome MLE, Toptica). The excitation light was reflected upward toward the sample through the 4× 0.13 NA air objective (Plan Fluor, Nikon) with a multibandpass dichroic mirror (TRF89902-EM-ET-405/488/561/647 nm Laser Quad Band Set, Chroma). Emission light was also filtered with a bandpass filter (FF02-617/73, Semrock). In the 4f system’s BFP, we placed a reflective LC-SLM for phase modulation (PLUTO VIS 020, Holoeye). For acquisition, we used an Scientific complementary metal–oxide–semiconductor (sCMOS) camera (Prime 95b, Photometrics).

For the projected point-array experiments, 640-nm laser light (iChrome MLE, Toptica) was projected onto the DMD (DLP LightCrafter 6500 Evaluation Module, Texas Instruments) as part of the illumination-engineering module. The engineered illumination pattern was reflected up with a multibandpass dichroic mirror (TRF89902-EM- ET-405/488/561/647 nm Laser Quad Band Set, Chroma) such that the measured signal was actually due to the nonzero transmittance of the filter at 640 nm (0.4%). The objectives used were 100× 1.45 NA oil objective (Plan Apo λ, Nikon) for the static tilted reflective sample, 4× 0.13 NA air objective (Plan Fluor, Nikon) for the reflective stepped surface, and 20× 0.75 NA air objective (Plan Apo λ, Nikon) for the dynamically tilting mirror. Phase modulation was performed using a reflective LC-SLM (high-speed 1920X1152, Meadowlark Optics, for the static reflective samples, and PLUTO VIS 020, Holoeye, for the dynamically tilting mirror). For the static reflective samples acquisition, we used an electron multiplying charge-coupled device (EMCCD) camera (iXon Ultra 897, Andor), and for the dynamically tilting mirror, we used an sCMOS camera (Prime 95b, Photometrics).

In the different experiments, two different LC-SLMs and cameras were used because of setup availability, which also demonstrates the flexibility of our technique. In addition, for the reflection case, the emission light does not fill the whole BFP (in contrast to the fluorescence case). Therefore, modifications to the phase-mask size and location in the BFP were applied to obtain an optimal PSF shape (in the image plane).

PDMS membrane apparatus. The apparatus consisted of three layers, the first layer was a thin (30 μm) PDMS (1:10 cross-linker to resin ratio, 184 silicone elastomer; Sylgard, USA) fabricated by spinning the polymer at two steps. The first step was a slow spinning (150 rpm for 40 s) process to uniformly spread the polymer on a smooth silicon wafer, and the second step consisted of spinning at high speed (1000 rpm for 175 s) and baking the polymer at 82°C for 2 hours. The second layer of the device was a thick (10 mm) PDMS mold with a tube insert diameter of 1/16 inch at the middle of the mold, which was then baked at 82°C for 2 hours. Next, we used a 1.25-mm biopsy punch to construct the chamber, designated for inserting air flow under the elastic membrane. The two layers were then connected by exposure to plasma using a corona discharge wand (BD-20 V, Electro-Technic Products, USA). The corona discharge wand was also used to attach the opposite side of the PDMS mold to a standard glass slide. To inflate and deflate the membrane, a 1/16-inch tube [perfluoroalkoxy (PFA) 1/16 inch diameter, Index Health and Science, USA] was inserted into the mold and connected to a standard syringe.

DHM profilometer. The dynamic inflatable membrane DHM reconstructions were obtained using a DHM profilometer from the reflection series (DHM R1003, Lyncée Tec) with a single 666-nm laser source and 80-ms exposure time. The objectives used were 2.5× 0.07 NA and 10× 0.3 NA (standard set, Lyncée Tec).

Tilted reflective plane. A 20 × 20 mm2 silicon wafer, 1 mm thick, was fixed to a glass slide and tilted by attaching magnets to one side of it (as shown in Fig. 3A).

Reflective stepped surface. A glass coverslip was coated with a 180-nm-thick chromium layer and cut in half to obtain two reflective surfaces. One surface was fixed directly on top of a glass slide, while the other was raised by inserting a 170-μm-thick coverslip (Marienfeld Superior) on top of the same glass slide (as shown in Fig. 3E).

Tilting mirror. A 12.7-mm-diameter mirror was glued to a 2-mm-diameter, 100-mm-long metal rod and a 56 × 26 mm2 card stock (0.5 mm width). The rod held the assembly over the microscope objective and was moved by pressing and then releasing the edge (Fig. 4A).

Statistical analysis

Calibration dictionary acquisition. For the 3D reconstructions, we used a calibration dictionary (as presented in Figs. 1 and 3), which relates the PSF of a single bead/spot to its precise z position. This is done by scanning the objective relative to the focal plane, in a defined depth range with constant steps. The dictionary z-stacks were acquired over a 600-μm range with 20-μm step size and 300-ms exposure time for the deformed-membrane experiment, 12-μm range with 0.2-μm steps and 500-ms exposure time for the tilted reflective plane, 600-μm range with 5-μm steps and 100-ms exposure time for the reflective stepped surface, and 100-μm range with 1-μm steps and 20-ms exposure time for the dynamically tilting mirror.

Reconstruction algorithm. The following MATLAB algorithm was applied on both inflatable membrane and the reflective surfaces frames. A preprocessing step was used for background subtraction to the acquired dictionary elements. The background level was determined on the basis of bead/spot-free regions. A correlation map was calculated by the maximal correlation value between the first frame and the dictionary elements, per pixel.

For each bead/spot, the following steps were performed:

1) Global lateral coordinate calculation—Defined as the point of the maximal value of the correlation map (which is updated by cropping the localized bead/spot in each iteration).

2) For each temporal frame, the following was applied:

2.1) Center of mass (CoM) lateral coordinates calculation—A region of interest (ROI) was defined around the global lateral coordinates and its CoM was computed. According to the CoM coordinates, the global lateral coordinates were updated.

2.2) Initial estimation of axial coordinate—Correlation between the defined ROI containing the current bead/spot and the dictionary elements was calculated. The bead’s axial position was defined as the known axial position of the dictionary element with the highest correlation value.

2.3) Maximum likelihood estimation (MLE) interpolation (40)—Refinement of the recovered 3D position was performed by an MLE interpolation algorithm. Signal and background level were also recovered by the algorithm.

The 3D dynamic reconstruction was obtained using the 3D localization matrix of the beads/spots throughout the frames. Where noted, a linear interpolation was used to reconstruct the surface.

SUPPLEMENTARY MATERIALS

Supplementary material for this article is available at http://advances.sciencemag.org/cgi/content/full/6/44/eabc0332/DC1

https://creativecommons.org/licenses/by-nc/4.0/

This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial license, which permits use, distribution, and reproduction in any medium, so long as the resultant use is not for commercial advantage and provided the original work is properly cited.

REFERENCES AND NOTES

Acknowledgments: We thank H. Feldman for discussions. Funding: This project has received funding from the European Research Council under the European Union’s Horizon 2020 Research and Innovation Programme, grant agreements 802567 (to Y.S.) and 678734 (to M.B.). L.E.W. and Y.S. were supported by the Zuckerman STEM Leadership Program and Y.S. by a Career Advancement Chair from the Technion’s Leaders in Science and Technology Program. In addition, this research was supported by the Israel Science Foundation (grant no. 450/18) and by the Israeli Innovation Authority (grant no. 880326). Author contributions: R.G.-S. and Y.S. designed the studies. R.G.-S., L.E.W., and R.E. performed the experiments. R.G.-S., B.F., and E.N. wrote the reconstruction algorithm and performed the simulations. R.G.-S., L.E.W., M.B., and Y.S. wrote the manuscript. Competing interests: The authors declare that they have no competing interests. Data and materials availability: All data needed to evaluate the conclusions in the paper are present in the paper and/or the Supplementary Materials. Additional data related to this paper may be requested from the authors.
View Abstract

Stay Connected to Science Advances

Navigate This Article