Research ArticleENGINEERING

Scalable tactile sensor arrays on flexible substrates with high spatiotemporal resolution enabling slip and grip for closed-loop robotics

See allHide authors and affiliations

Science Advances  13 Nov 2020:
Vol. 6, no. 46, eabd7795
DOI: 10.1126/sciadv.abd7795
  • Fig. 1 Active matrix ZnO piezoelectric TFT array for force sensing.

    (A) Schematic illustration of force sensing using a ZnO TFT. (B) Structured layers of the ZnO TFT. (C and D) Representative transfer (C) and output (D) characteristics of the fabricated TFT. (E) Temporal response of the TFT upon different external forces.

  • Fig. 2 DC characteristics of ZnO TFT with different O2/Ar flow rate ratios.

    (A to C) Transfer characteristics and (D to F) output characteristics of the fabricated TFT with oxygen flow rates of 20, 10, and 5% during the growth. (G to J) Gate bias–dependent physical properties of ZnO TFT, (G) sheet resistance, (H) contact resistance, (I) transfer length, and (J) contact resistivity curves of ZnO TFT with oxygen flow rates of 10% (black) and 5% (red).

  • Fig. 3 Force sensing characteristics of the fabricated TFT arrays for finger mounting.

    (A) Photograph of the fabricated active matrix TFT array on a 4-inch wafer. Inset is a magnified image of a single TFT. (B) Schematic diagram of the active matrix structured TFT array. amp, amplification; mux, multiplexing. (C) Element-wise response map of the 8 × 16 TFT array. (D) Cross-talk correlation matrix of the array. (E) Average current change curve under different applied force measured from 10 random areas in the array. Error bars are corresponding SDs. (F) Two-point discrimination study of the TFT array. Top: Illustrations of differently spaced tips used in the experiment. Middle: Response heatmaps under the force applied by the tips. Bottom: Plots of the current change along the dashed line in the heatmaps in the middle row. (G and H) Study on the temporal resolution of the TFT array. (G) Photograph of the experimental setup. (H) Corresponding response heatmap of the TFT when force is applied. (I) (Black dots) Overlapped average responses of the elements indicated by the red box in (H) and (blue curve) time versus z-position profile of the tip. Photo credit: Hongseok Oh, UC San Diego.

  • Fig. 4 Shear force measurement.

    (A) Strategy of measuring shear force applied on the TFT array using the 3D PDMS bump array. (B) Photograph of the 3D PDMS array placed on the TFT sensor array. Insets: Top view (top) and side view (bottom) of a single bump. (C) Calculation of shear force from the force applied to each sensor in the unit cell. (D) Photograph of the shear force measurement setup. Inset: A close-up view of the tip applying in-plane force to the bump. (E) Average shear force versus current change curve from six random unit cells. (F to H) Element-wise characterization of shear force response for the upward shear forces. (F) Schematic illustration of applying shear force along the y axis. (G) Quiver plot of the calculated force upon the uniformly applied upward force. (H) Correlation matrix of the y-axis component for the cross-talk analysis. (I to K) Same analysis for the shear force along the x axis. Photo credit: Hongseok Oh, UC San Diego.

  • Fig. 5 Demonstration of sensing multidimensional touch input using the TFT array.

    (A) Response heatmaps of TFT arrays with single- and double-touch input. (B) Quiver plots of shear force when a finger slides over the TFT array. The direction of sliding finger is indicated by red arrows. The entire demonstration can be found in movie S1.

  • Fig. 6 Application to the robotic gripper.

    (A) Schematic illustration of the robotic gripper experiment with feedback from the force sensor array. (B) Snapshots of video (movie S2) showing each stage for closed-loop controlled grip experiment. (C) Normal and (D) shear force distribution of each finger when holding an egg. (E) Overlapped snapshots of the gripper holding a baseball from multiple incidents, which is slipping at grip interfaces. (F) Schematic illustration of slip action at the interface. (G) Illustration of additional feedback loop to provide secure grip. (H) Snapshot of the gripper controlled by shear force feedback closed-loop showing secure grip of the baseball. (I) Algorithm diagrams of the shear force feedback closed-loop control system. (J) Stacked plot of the gripper under shear force feedback control. Gripper angle (top), calculated threshold (middle top), and current changes corresponding to the net normal force (middle bottom) and net shear force (bottom). <1> to <5> Important moments in the grip process. <1> Grip command issued. <2> Initial grip made. <3> Shear feedback turned on. <4> Lift-off started. <5> Grip secured. The entire demonstration of the slip-adjusted grip control can be found in movie S3. a.u., arbitrary unit. Photo credit: Hongseok Oh, UC San Diego.

  • Fig. 7 Scaling to a 100-μm spatial resolution.

    (A) Photo of the devices having 16 × 16 TFTs, with different pitch of (top) 500 μm, (bottom left) 250 μm, and (bottom right) 100 μm. For size comparison, a one-cent coin is shown together. (B to D) Microscope images of TFT array having pitch of (B) 500 μm, (C) 250 μm, and (D) 100 μm. Scale bars, 200 μm. (D) Inset: High-magnification image of a single TFT. (E and F) Normal force distribution of the 100-μm-pitch TFT array during the discrimination test. Response heatmap when pressed by a stamp having two extruded circles with the center-to-center distance of (E) 750 μm and (F) 500 μm. (G and H) Response heatmap plots of the 100-μm-pitch TFT array squeezed by a glass slide together with single hair, oriented toward (G) the diagonal direction and (H) the vertical direction. (I) Photograph of the 250-μm-pitch TFT array. Yellow circles indicate the location where controlled force is applied to investigate sensitivity distribution. (J) Response heatmaps of the TFT array when the same 25 mN is applied to the locations indicated by the yellow circles in (I). (K) Plot of average current change from nine areas as a function of the applied force. Error bars are corresponding SDs. (L) Photo of the 500-μm-pitch TFT array with 3D PDMS pillar array placed on top. Inset: Side-view photo of the 3D PDMS pillar array. Scale bar, 2 mm. (M) Snapshot of the video (movie S4) showing the transparent plastic rod rubbing the TFT array to the rightward direction. (N) Corresponding quiver plot showing the shear force distribution of the array. Photo credit: Hongseok Oh, UC San Diego.

  • Fig. 8 Device performance on curved surfaces and device durability under multiple bending cycles.

    (A) Photo of a 16 × 8 TFT array rolled conformally over a plastic rod having a bending radius of 5 mm. (B and C) Demonstration of normal and shear force sensing under highly bent conditions from the TFT array in (A). (B) Photos of the same TFT array on which small wooden stick applies normal force at the center left position of the array (top left) and the center right position of the array (top right), and the corresponding response heatmaps in the bottom left and right panels. (C) Photos of the TFT array on which transparent plastic rod applies a downward shear force (top left) and upward shear force (top right), and the corresponding shear force response plots in the bottom left and right panels. (D) Photos of the setup for measuring electrical characteristics of the device under different bending radii and applied forces. (E) Transfer curves of the single TFT under different bending radii. (F) Current change of the TFT with applied force, measured under different bending radii. (G) Photos of the bending cycle test, showing the device when released (top) and folded (bottom). (H) Response heatmap plots of the device under an applied force of 25 mN, measured before bending cycle test (top) and after 10,000 bending cycles (bottom). (I) Average current change of the force applied TFTs under different force values, measured before the bending cycle test and after 1000, 2000, 5000, and 10,000 bending cycles. Photo credit: Hongseok Oh, UC San Diego.

  • Fig. 9 Application of the TFT array.

    (A to D) Demonstration of the pulse sensing. (A) Photo of the wrist showing the placement of the 250-μm-pitch TFT array. (B) Video snapshot (movie S5) of the response heatmap of the sensor array. The element marked by the red box showed the most dynamic changes during the measurement. (C) Temporal response plot obtained from the TFT marked in (B). (D) Averaged current change obtained from 23 pulse signals in (C). (E to G) Demonstration of the waterproof operation. (E) Video snapshot (movie S5) of the sensor array submerged in the saline solution. Pipet applies water flow onto the TFT array without making physical contact with it. (F) Response heatmap plots of the array capturing the moment of water flow reaching the array, taken at four different frames. (G) Corresponding shear force distribution plot of (F) at 0.023 s [top right in (F)]. (H) Video snapshot (movie S6) of the robotic gripper holding an egg, with the 2-mm-pitch TFT array mounted on the curved surface. (I) Corresponding normal force distribution map. (J) Video snapshot of the robotic gripper holding a heavy ball and detecting shear force. (K) Corresponding distribution of normal force, shear force, and derivative of shear force on the TFT array. (L) Plot of average shear force of the array as a function of time. Photo credit: Hongseok Oh, UC San Diego.

  • Table 1 DC characteristics of the ZnO TFTs fabricated with different O2 flow rates.

    ParametersO2/Ar = 20%O2/Ar = 10%O2/Ar = 5%
    voltage Vth (V)
    slope (mV/dec)
    mobility μlin

Supplementary Materials

  • Supplementary Materials

    Scalable tactile sensor arrays on flexible substrates with high spatiotemporal resolution enabling slip and grip for closed-loop robotics

    Hongseok Oh, Gyu-Chul Yi, Michael Yip, Shadi A. Dayeh

    Download Supplement

    This PDF file includes:

    • Sections S1 to S16
    • Figs. S1 to S18
    • Tables S1 and S2
    • Movies S1 to S6

    Files in this Data Supplement:

Stay Connected to Science Advances

Navigate This Article