Research ArticleAPPLIED SCIENCE AND ENGINEERING

Self-powered user-interactive electronic skin for programmable touch operation platform

See allHide authors and affiliations

Science Advances  08 Jul 2020:
Vol. 6, no. 28, eaba4294
DOI: 10.1126/sciadv.aba4294

Abstract

User-interactive electronic skin is capable of spatially mapping touch via electric readout and providing visual output as a human-readable response. However, the high power consumption, complex structure, and high cost of user-interactive electronic skin are notable obstacles for practical application. Here, we report a self-powered, user-interactive electronic skin (SUE-skin), which is simple in structure and low in cost, based on a proposed triboelectric-optical model. The SUE-skin achieves the conversion of touch stimuli into electrical signal and instantaneous visible light at trigger pressure threshold as low as 20 kPa, without external power supply. By integrating the SUE-skin with a microcontroller, a programmable touch operation platform was built that can recognize more than 156 interaction logics for easy control of consumer electronics. This cost-effective technology has potential relevance to gesture control, augmented reality, and intelligent prosthesis applications.

INTRODUCTION

Human-machine interaction (HMI) is implemented by three parts (human, machine, and interactive medium) whose task is to complete two processes (instructing the machine and giving feedback to humans) (13). Since most devices are controlled by electrical signals, the process of giving control commands to the machine generally relies on electrical readouts, such as using a keyboard and mouse to operate the computer (49). On the basis of the fact that humans cannot directly obtain information in electrical signals, the process of giving feedback to human in HMI relies on the human-perceivable signals such as visible light, sound, and force (1013). Visible light is generally considered as the best candidate because of its higher spatial resolution and intuitiveness, such as using a display screen (14, 15). However, with the rapid development of wearable devices, artificial prosthetics, and robotics, the traditional interactive medium is difficult to meet the needs in terms of flexibility, portability, and low power consumption (1620).

As a network of flexible sensors, the electronic skin (e-skin) is becoming the primary interactive medium for next-generation HMI due to spatial mapping and amplitude quantifying of touch stimuli. Early works on e-skin focused on the optimization of pressure sensors for command delivery and lack of research on human-readable output (2127). Until Javey and colleagues (28) integrated thin-film transistors, pressure sensors, and organic light-emitting diode arrays to invent a device that provides an instant visual response to pressure, researchers opened a new era of user-interactive e-skin. After that, Bao and colleagues (29) reported a stretchable e-skin with interactive color-changing and tactile-sensing properties. Shepherd and colleagues (30) proposed an electroluminescent skin for optical signaling and tactile sensing. Despite these achievements, the above devices rely on the external power supply for both tactile sensing and touch visualization, which brings significant challenges to portability and integration. Although Wang and colleagues (31) invented a self-powered optical and electrical dual-mode pressure sensor based on the triboelectrification and mechanoluminescence, it requires up to several megapascals of pressure to produce visible light. Since the pressure threshold of mechanoluminescence is much higher than the strength of human touch (typically less than 100 kPa), this device cannot be used for user-interactive e-skin.

The present work introduces a self-powered, user-interactive e-skin (SUE-skin), which can be readily prepared by a cost-effective processing approach (Fig. 1). The realization of this concept is based on the proposed triboelectric-optical model. The SUE-skin is capable of simultaneously converting touch stimuli into electrical signals and real-time visible lights without relying on the external power supply. We explored a programmable touch operation platform for implementing interactive control of consumer electronics by combining the microcontroller. Furthermore, benefiting from the above characteristics, the SUE-skin may achieve high-robust touch track mapping by superimposing of the electrical and optical signals.

Fig. 1 Concept and processing approach of the SUE-skin.

(A) Schematic diagram of the SUE-skin, which can convert a touch stimulation into an optical signal visible to the human eye and an electrical signal for machine control simultaneously. IoT, Internet of things. (B) The multiple frames at different timestamps (1 to 7) and the time-lapse optical photo (8) of the visual output under the sliding touch. (C) The chromaticity diagram calculated from the emission spectrum of SUE-skin. (D) Stability and repeatability test of the SUE-skin’s luminescence feature by 1000 cycles. (E) The voltage output of the SUE-skin under sliding touch stimuli. (F) Process flow diagram for device fabrication. (G) The SUE-skin was subjected to different mechanical deformations sequentially, which was (i) original, (ii) bent, (iii) rolled, (iv) folded, and (v) released. Scale bars, 1 cm. a.u., arbitrary units. Photo credit for (B) and (G): Xuan Zhao, University of Science and Technology Beijing.

RESULTS

Concept and processing approach of the SUE-skin

Here, the SUE-skin, capable of simultaneously generating electrical and optical signals without external power supply under touch stimuli, is shown in Fig. 1A. The electrical signals generated when the finger touches the SUE-skin can be processed by the microcontroller unit (MCU) as control commands for various consumer electronic devices, and the optical signals generated can be directly observed by the human eyes. Figure 1B shows the optical photograph of the visual output during a sliding touch (movie S1, under dark conditions; fig. S1A and movie S2, under ambient conditions). One frame is taken every 0.5 s, and a total of seven frames are taken in 3 s. For comparison, the last picture was obtained by time-lapse photography. During photography, the exposure time of each frame is 1/10 s. Figure 1C shows the chromaticity diagram calculated from the emission spectrum acquired during the touch process, by the GoCIE software. The color coordinate of the yellow-green light emitted by the SUE-skin is (0.33, 0.60). Figure 1D shows the stability and repeatability test of the SUE-skin’s luminescence feature by 1000 cycles. Figure 1E shows the output voltage of the SUE-skin under sliding touch stimuli. The schematics for the fabrication procedure of the SUE-skin, as shown in Fig. 1F [note S1 and fig. S1 (B to D) for synthesis and characterization of ZnS (ZnS: Cu, Al) phosphor particles; Materials and Methods for details of the SUE-skin fabrication]. The as-fabricated SUE-skin consists of a phosphor layer, an electrode (Al), an insulating layer (Kapton), a shield layer (Al), and a substrate [polydimethylsiloxane (PDMS)]. The shield is grounded during operation to minimize external interference during electrical signal acquisition. Most of the traditional triboelectric-electroluminescent devices are based on ZnS: Cu (3235), and the ZnS: Cu, Al synthesized here has higher luminous efficiency than ZnS: Cu because it is doped with Al as a co-activator. As shown in fig. S1E, under the same touch stimuli, the luminous brightness of ZnS: Cu, Al–based SUE-skin is much higher than that of ZnS: Cu–based SUE-skin. Figure 1G shows that the SUE-skin can be bent, rolled, and folded without any mechanical failure, with a minimum bending radius of 2 mm and a maximum folding pressure of 2 MPa. As shown in fig. S1 (F to K), the luminous intensities have been recorded every 40 bending/rolling cycles, and the time-lapse optical photos and surface scanning electron microscopy (SEM) images of SUE-skin have been taken every 400 bending/rolling cycles. The luminous intensity and surface topography remain stable during and after 1200 bending/rolling cycles, which demonstrate that the SUE-skin has excellent mechanical stability.

The triboelectric-optical model

The SUE-skin’s ability to achieve touch visualization and mapping at low-pressure thresholds is based on our proposed triboelectric-optical model, which induces both electrostatic induction and electroluminescence through triboelectrification, as shown in Fig. 2A (contact-separation mode) and fig. S2 (sliding mode; note S2). The charge transfer behaviors of the processes of contact-separation mode and sliding mode are similar in the basic principle of the triboelectrification. Here, it is assumed that the dielectric binder has a larger surface work function than human skin.

Fig. 2 The triboelectric-optical model.

(A) Schematic diagram of the physical model of the triboelectric-optical effect in contact-separation mode. (B) Light intensity and output voltage of the SUE-skin under different pressures. (C) Light intensity and output voltage of the SUE-skin under different prestress. (D) The spectrogram of triboluminescence (TL) of the SUE-skin. (E) The spectrogram of electroluminescence (EL) of the synthesized ZnS: Cu, Al powder. (F) The spectrogram of room temperature photoluminescence (PL) of the synthesized ZnS: Cu, Al powder.

When the human skin is in full contact with the SUE-skin (Fig. 2Ai), the human skin and the dielectric adhesive acquire net positive and net negative triboelectric charges, respectively. When the human skin gradually separates from the SUE-skin (Fig. 2Aii), the top surface of the dielectric adhesive is negatively charged, and an induced current is generated in the external circuit. During this process, there is a varying electric field between the top surface of the dielectric binder and the electrode. The electrons produced within the lattice by the impact ionization move toward the bottom of the ZnS phosphor particle under the induction of the above varying electric field (36, 37). During the movement, some of the electrons affect the luminescence centers, thereby exciting the luminescence centers and causing electroluminescence. When the human skin is completely separated (the threshold distance for complete separation is about 1 to 2 mm) from the SUE-skin (Fig. 2Aiii), no current is generated in the external circuit. At the same time, the electrons have been concentrated at the bottom of the ZnS phosphor particle under the electric field induction and will not affect the luminescence centers, so the device does not emit light. When the human skin approaches the SUE-skin again (Fig. 2Aiv), an induced current is generated again in the external circuit. During this process, the electric field between the upper surface of the dielectric binder and the electrode gradually disappears. The electrons inside the ZnS that lose the effect of the electric field move in reverse and impact with the luminescence centers again to excite it and cause electroluminescence again. As described in the model, there are two luminescence phenomena in one contact-separation cycle (movie S3). Of course, for sliding touch, since the time from the start of the contact to the complete separation is short, and the electroluminescence of ZnS has an afterglow of tens to hundreds of milliseconds, it is generally difficult to distinguish the two luminescence phenomena during the period.

The higher the pressure during the contact-separation process, the greater the amplitude of the electrical and optical signals generated by the SUE-skin, as shown in Fig. 2B. However, if a prestress is applied in advance under the contact state, and then adding stress is added to the prestress so that the final pressure is equal to 300 kPa, as shown in Fig. 2C, then the amplitude of the output electrical and optical signals decreases as the prestress increases. The details of the prestress here are described in note S3 and fig. S3. Furthermore, eventually, the optical outputs tend to zero when the prestress is more than 160 kPa, and the electrical outputs tend to zero when the prestress is more than 240 kPa. The reason for this phenomenon is that the contact is not tight when the prestress is small, and triboelectricity is generated because the contact becomes tighter in the subsequent pressure increase. When the prestress is large, then the adding stress will not produce triboelectricity, and thus, there will be no electrical and optical output. From another point of view, although the sphalerite-ZnS having piezoelectricity is used here (fig. S1D), the outputs of the SUE-skin under touch stimuli are not caused by the piezoelectric effect. It also proves that the optical signal output generated by the SUE-skin here is not from the mechanoluminescence caused by lattice distortion or material cracking. The comparisons of Fig. 2 (D to F) reveal that the triboluminescence (TL) of the SUE-skin under touch stimuli is from electroluminescence (EL) rather than room temperature photoluminescence (PL). A detailed explanation is given in note S4 and fig. S4.

Besides, we have found that when the liquid flows through the surface of the SUE-skin, the output optical signals can also be generated (note S5, fig. S5, and movie S4). This previously unknown phenomenon can further prove the above working principle of the SUE-skin. In summary, the total intensity of the electrical output and light output of the SUE-skin depends on the strength of the triboelectrification, and the intensity of the two outputs is mutually constrained when the strength of the triboelectrification is constant.

Performance optimization of the SUE-skin

In the process of touching, the contact area and pressure of the human skin and the SUE-skin are difficult to precisely control. So, in all quantitative tests on the outputs of SUE-skin (fig. S6 for the schematic of the testing equipment), glass was used instead of human skin, which is close to the human skin in the triboelectric series (Fig. 3A). As mentioned above, the outputs of the SUE-skin, whether electrical or optical, are induced by the triboelectrification. Therefore, enhancing the strength of frictional electrification is the fundamental method to optimize the performance of the SUE-skin. Two of the most recognized methods for improving triboelectrification are the selection of suitable friction materials and surface treatment, such as the fabrication of microstructures to increase the surface area and change the excited state concentration of the surface (38, 39). As shown in Fig. 3B, the effect of various binders [ethyl cellulose (EC), methylcellulose (MC), polyvinyl alcohol (PVA), PDMS, and Ecoflex 00-20 (Ecoflex)] on device output was compared. The results are consistent with the triboelectric series, and the output performances are best when Ecoflex is used as the binder. As shown in Fig. 3C, after the surface treatment of Ecoflex by reactive ion etching (RIE), the output performance of the device is significantly improved. Characterization of Ecoflex surface morphology before and after RIE treatment is shown in fig. S7 (A to D). In the above comparison, the mass ratio of ZnS to Ecoflex is 1:1, and the thickness of the ZnS and Ecoflex composite is 500 μm. For practical interactive applications, the higher the intensity of the output optical signal, the easier it will be recognized by the human eye. Therefore, on the premise that the output voltage can be directly identified by the microcontroller (without filtering and amplification), we want the intensity of the output optical signal to be as high as possible. In other words, after increasing the total output of the SUE-skin by material selection and surface treatment, the strength of the electrical output and the light output has to be coordinated.

Fig. 3 Performance optimization of the SUE-skin.

(A) The position of different materials in the triboelectric series. (B) The output electrical and optical signals of the SUE-skin when different materials are used as binders. (C) The output electrical and optical signals of the SUE-skin before and after the reactive ion etching (RIE) process. (D) The output electrical and optical signals of the SUE-skin when the mixing ratio of ZnS and Ecoflex is different. (E) The output electrical and optical signals of the SUE-skin when the thickness of the composite is different.

The main factors affecting the relative strength of the two outputs are the mass ratio of ZnS to Ecoflex and the thickness of the composite, as shown in Fig. 3 (D and E). When the mass ratio of ZnS to Ecoflex is increased (as the thickness of the composite is 500 μm), the density of the luminescent center in the same rubbing region is increased, so the intensity of the output optical signal of the device is increased. At the same time, since ZnS is a semiconductor, it will have a shielding effect on the electrostatic induction process of the SUE-skin. Therefore, as the ratio of ZnS to Ecoflex increases, the output electrical signal strength of the device will decrease. When the thickness of the composite is increased (as the mass ratio of ZnS to Ecoflex is 1:1), the density of the luminescent center in the same friction region is also increased, so the intensity of the output optical signal of the device is also increased. Besides, when the mass ratio of ZnS to Ecoflex is more than 1:1, the composite film is easily cracked and has reduced elasticity (fig. S7, E to H). However, the higher the thickness, the longer the electrostatic induction distance for the SUE-skin, so the output electrical signal is smaller.

To realize that the electrical signal can be accurately recognized by the microcontroller, the electrical signal must be larger than a certain threshold to ensure sufficient anti-interference performance and smaller than the reference voltage inside the microcontroller to avoid breakdown. For our integrated control platform, when the analog input signal of the analog-to-digital conversion (ADC) is higher than 0.5 V, it can achieve very good interference immunity. Also, the analog voltage input range of the microcontroller is 0 to 3.3 V. In actual tests, when the output voltage of the SUE-skin is about 40 V, the corresponding divided voltage of the ADC module is about 2.7 V, which just meets the above requirements. So, the relevant parameters of the SUE-skin are determined as, friction material (binder), Ecoflex; surface treatment, RIE; the mass ratio of ZnS and Ecoflex, 1:1; and the thickness of the ZnS and Ecoflex composite, 500 μm. Movie S5 demonstrates the luminescence of the ZnS-Ecoflex composite film rubbed by hand.

Demonstration of the programmable touch operation platform

Because of its good intuitiveness, the interactive operation has essential application value in the fields of wearable electronic devices, industrial Internet, and smart home. We combined the SUE-skin with a microcontroller to build a programmable touch operation platform, which can recognize different touch tracks for easy control of consumer electronics. As shown in Fig. 4 (A to C), each channel from the SUE-skin is connected to the low-pass resistor-capacitance filter to decrease electrical noise. Using 10-megohm resistor and 1-nF capacitor, the corresponding cutoff frequency is 15.9 Hz. The electrical signals generated by the four-electrode SUE-skin are input to the MCU (STM32F407) and converted into digital signals by the built-in ADC of the MCU. The MCU is programmed in LabVIEW embedded by advanced reduced instruction set computing machine to recognize electrical signals of the SUE-skin and control external devices through drive circuits. The signal processing details and flowcharts are shown in fig. S8. First, the MCU collects input voltage signals of the SUE-skin by ADC from four pins (PA0, PA1, PA2, and PA3) and then checks if signals are above the threshold value. Once signals that exceed the threshold are acquired, the MCU records the pins that were triggered during a time period and converts them into a one-dimensional array and then checks if the array is a preset instruction. If yes, then it converts the resulting one-dimensional array to a control variable. Different control variables correspond to different output instructions to control the peripheral equipment. Because of the intuitive touch lighting and signal processing technology, this interactive programmable touch operation platform can support more than 156 touch interaction logics (see fig. S9).

Fig. 4 Demonstration of the programmable touch operation platform by tracks recognition.

(A) Schematic diagram of the four-electrode SUE-skin. (B) Optical photograph and electrodes schematic of the SUE-skin. (C) Schematic diagram of the programmable control module. (D) Touch operation demonstration for audio playback. (E) Touch operation demonstration for the characters displaying. Photo credit for (B), (D), and (E): Xuan Zhao, University of Science and Technology Beijing.

Figure 4 (D and E) is interactive operation demonstrations of the control of the external audio module and display module (movies S6 to S8). As shown in Fig. 4D, when the finger is swiped from the region 1 to 2, the audio is turned on for music (Hotel California) playback; when the finger is swiped from the region 1 to 3, the music is played at 2× speed; and when the finger is swiped from the region 3 to 1, the standard rate is resumed. When the finger is swiped from the region 2 to 4, the music is played at 0.5× speed; when the finger is swiped from the region 4 to 2, the standard rate is resumed; and when the finger is swiped from the region 2 to 1, the music is paused. The inset map in the lower-left corner of each process shows the illuminating tracks during the operation. Figure 4E is the photographs of the luminescence at different touches, the characters displayed on the external LCD screen, and the real-time electrical signals were acquired in the four channels. In addition, finger sliding velocity sensing may be useful for the application of HMI. Here, from the distance between the center points of different electrodes and the time difference between the electrical signals generated by different channels, the sliding speed of the fingertips can be calculated easily. Moreover, we explored the possibility of high-robust touch track monitoring with the superposition of electrical and optical signals, as shown in note S6 and fig. S10.

DISCUSSION

In this work, we demonstrated the concept of SUE-skin, which is capable of interactive luminescence and tactile-sensing properties without external power supply. The theoretical basis for realizing this concept is the triboelectric-optical model, which is effective in illustrating the mechanism of touch light emission at low trigger pressure threshold (20 kPa, only 1% compared to the traditional mechanoluminescence). On the basis of the triboelectric-optical model, a seamless integration of visual and electrical mapping of touch stimuli is achieved. The perception of touch stimuli can be achieved through the observation of human eyes, and the touch stimuli can be monitored through electrical readout. This work feature is not only the core of user-interactive e-skin but also a good solution to poor practicality caused by only optical signal output in the previous strain visualization. Moreover, by integrating with a MCU, the SUE-skin can also be used as a programmable touch operation platform that supports more than 156 interactive logics. Last, we explored the possibility of high-robust touch track monitoring with the superposition of electrical-optical dual signals. Future work will involve introducing other electroluminescent materials in a variety of colors and array designs to achieve higher luminous intensity, contrast, and resolution. This SUE-skin is suitable for applications in interactive wearable devices, military applications, artificial prosthetics, and intelligent robots.

MATERIALS AND METHODS

Preparation of composite film

The synthesized ZnS particles are mixed with the binder in proportion and stirred well. The mixture was then poured into a homemade mold. After the adhesive is cured in a vacuum environment, the composite film is obtained by demolding. The PDMS elastomer base (Sylgard 184, Dow Corning) was mixed with a 10:1 mixture of curing agents, and ZnS particles were added and stirred well. The mixture was then heated at 50°C for 12 hours to cure the PDMS. The components A and B of Ecoflex (00-20, Smooth-on) were mixed 1:1, and ZnS particles were added and stirred uniformly. The mixture of Ecoflex, EC, MC, PVA, and ZnS particles was allowed to stand at room temperature for 18 hours for curing. The top surface of the ZnS-Ecoflex composite film was etched (O2 plasma, 100 mTorr, 100 W) using a reactive ion etcher (Ke Mao, RIE-100).

Fabrication of the SUE-skin

First, an Al electrode (thickness, 300 nm) was evaporated at the bottom of the composite film, and the wires were taken out. Then, a Kapton film (thickness, 50 μm; DuPont) was pasted as an insulating layer. A layer of Al (thickness, 150 nm) is further evaporated as a shielding layer (the shielding layer is always grounded during the use of the SUE-skin). Last, a layer of PDMS having a thickness of about 0.5 mm was pasted as a substrate.

Device characterization

SEM images were characterized using a JEOL JSM-7001F FEG SEM operated at 10-kV beam voltage. The contact pressure was measured by a pressure sensor (SBT671), which is tied to the stepping motor. Spectrometers (NanoLOG-TCSPC, QE65pro Ocean Optics) in the range of 300 to 800 nm were carried out to analyze the spectral curve of the emission light in the touching process, and a multichannel measurement system (PXIe-4300, National Instruments Corporation) was used to collect the electric signal. Time-lapse optical photographs and videos of optical output under sliding touch were collected by Nikon 7000D (18 to 140 mm). The brightness of luminescence under touch stimuli was recorded using a spectroradiometer (PR-655).

SUPPLEMENTARY MATERIALS

Supplementary material for this article is available at http://advances.sciencemag.org/cgi/content/full/6/28/eaba4294/DC1

https://creativecommons.org/licenses/by-nc/4.0/

This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial license, which permits use, distribution, and reproduction in any medium, so long as the resultant use is not for commercial advantage and provided the original work is properly cited.

REFERENCES AND NOTES

Acknowledgments: Funding: This work was supported by the National Key Research and Development Program of China (nos. 2018YFA0703500 and 2016YFA0202701), National Natural Science Foundation of China (nos. 51991340, 51722203, 51672026, and 51527802), the Overseas Expertise Introduction Projects for Discipline Innovation (B14003), and the Fundamental Research Funds for Central Universities (FRF-TP-18-001C1). Author contributions: X.Z., Z.Z., Q.L., and Y.Z. conceived the concept, processing, and structure details. X.Z., Z.Z., and Q.L. prepared and purified the materials. X.X. and Z.K. assisted in carrying out the SUE-skin fabrication. X.X., F.G., and X.X. assisted in the SUE-skin performance measurements. X.Z. built the programmable touch operation platform. X.Z., Q.L., and Y.Z. co-wrote the paper. All authors discussed the results and commented on the manuscript. Competing interests: The authors declare that they have no competing interests. Data and materials availability: All data needed to evaluate the conclusions in the paper are present in the paper and/or the Supplementary Materials. Additional data related to this paper may be requested from the authors.
View Abstract

Stay Connected to Science Advances

Navigate This Article