Research ArticleAPPLIED SCIENCES AND ENGINEERING

Soft, wireless periocular wearable electronics for real-time detection of eye vergence in a virtual reality toward mobile eye therapies

See allHide authors and affiliations

Science Advances  13 Mar 2020:
Vol. 6, no. 11, eaay1729
DOI: 10.1126/sciadv.aay1729
  • Fig. 1 Overview of a soft, wireless periocular electronic system that integrates a VR environment for home-based therapeutics of eye disorders.

    (A) A portable, wearable system for vergence detection via skin-like electrodes and soft circuit and relevant therapeutics with a VR system. The zoomed inset details the magnetically integrated battery power source. (B) VR therapy environment that simulates continuous movements of multiple objects in three varying depths of near, intermediate, and distance, which correspond to 1°, 2°, and 3° of eye motions. (C) Zoomed-in view of a skin-like electrode that makes a conformal contact with the nose. (D) Highly flexible, soft electronic circuit mounted on the back of the neck. (E and F) X-ray images of the magnified mesh interconnects (E) and circuit flexion with a small bending radius (F). (G to I) Photos of an ultrathin, mesh-structured electrode, mounted near the eyebrow. (J and K) Two types of eye misalignment due to a disorder: esotropia (inward eye turning) (J) and exotropia (outward deviation) (K), which is detected by the wireless ocular wearable electronics. Photo credit: Saswat Mishra; photographer institution: Georgia Institute of Technology.

  • Fig. 2 AJP parametric study supplemented by the characterization of deposited NP ink.

    (A) AJP out of deposition head with a close-up of deposition. (B) Atomization begins in the ultrasonic vial and a carrier gas (N2) flows the ink droplets through tubing, the diffuser, and the deposition head where a sheath gas focuses the particles into a narrow stream with a diameter of ~5 μm. (C) Platen is heated to ensure evaporation of the solvents, but surface treatment with plasma cleaner enables clean lines (top). Otherwise, the traces tend to form into bubbles over a large area. (D) High focusing ratio with low width/thickness ratio enables lower resistivity. (E) Resistivity measurements from a four-point probe system was determined for various sintering temperatures and time. (F) Ultimately, the resistance of the electrodes needs to be low, so a two-point probe measurement demonstrates the resistance across a fractal electrode in series. (G) AgNPs capped with oleylamine fatty acid that prevents agglomeration end up breaking down after low- and high-temperature sintering. The grain size for small NPs increases at low temperatures, and then, that for bigger NPs increases at higher temperatures. (H) SEM images show the agglomeration and densification of the binder material after sintering. (I) X-ray diffraction analysis shows larger crystallization peaks after sintering and the recrystallization of peaks along (111), (200), and (311) crystal planes. The large peak between (220) and (311) represents Si substrate. (J) Raman spectroscopy demonstrates the loss of carbon and hydrogen peaks from the binder dissociation near wave number 2900 cm−1. (K) Cyclic stretching (50% strain) of the electrode. (L) Cyclic bending (up to 180°) of the electrode.

  • Fig. 3 Study of ocular vergence and classification accuracy based on sensor positions.

    (A) Converging and diverging ocular motions and the corresponding EOG signals with three different targets, placed at 40, 60, and 400 cm away. (B and C) Sensor positioning that resembles the conventional setting (two recording channels and one ground). (D and E) Alternate positions for improved detection of ocular vergence, showing an enhanced accuracy of 87%. (F and G) Finalized sensor positions (ocular vergence 2) that include three channels and one ground, showing the highest accuracy of eye vergence. Photo credit: Saswat Mishra; photographer institution: Georgia Institute of Technology.

  • Fig. 4 Optimization of vergence analysis via signal processing and feature extraction.

    (A) Raw eye vergence signals (top) acquired by the wireless ocular wearable electronics in real time and the corresponding derivative signals (bottom). a.u., arbitrary units. (B) Preprocessed data with a band-pass filter and the corresponding derivative, raised to the second power. (C) Further processed data with a 500-point median filter and the corresponding derivative, raised to the sixth power. A coefficient is multiplied to increase the amplitude of the second- and sixth-order differential filters. (D) SNR comparison between the second- and sixth-order differential filters, showing an increased range of the sixth-order data. The sixth-order data are used for thresholding the vergence signals for real-time classification of the dataset. (E) Data from the sliding window are added into the ensemble subspace classifier (shown by the decision boundaries of two dimensions of the feature set).

  • Fig. 5 Comparison of ocular classification accuracy between VR and physical apparatus.

    (A and B) Representative normalized data of eye vergence (red line, average; gray shadow, deviation), recorded with a VR system (A) and a physical apparatus (B). The VR device uses a higher signal gain than the physical setup. (C and D) Normalized peak velocities according to the normalized positions for both convergence and divergence in the VR (C) and the physical setup (D). (E and F) Summarized comparison of averaged classification accuracy (total of six human subjects) based on cross-validation with the VR-equipped soft ocular wearable electronics (91% accuracy) (E) and physical apparatus (80% accuracy) (F).

  • Fig. 6 VR-enabled vergence therapeutic system.

    (A) Example of ocular therapy programs in the VR system: Brock String, which is a string of 100 cm in length with three beads (top images) and measured eye vergence signals (bottom graph). (B) Program, named Eccentric Circle, that uses a set of two cards with concentric circles on each card (top images) and corresponding EOG signals (bottom graph). (C) Continuous use of the VR headset program, showing improved eye vergence, acquired from three human subjects with no strabismus issues from near point convergence. (D) Photos of strabismus exotropia, showing a subject who had difficulty holding the near point convergence during pencil push-ups. (E) Corresponding EOG signals upon convergence and divergence during the pencil push-ups; the left eye shows slower response at convergence, followed by an exotropic incidence after the blink at the near position. Photo credit: Saswat Mishra; photographer institution: Georgia Institute of Technology.

Supplementary Materials

  • Supplementary material for this article is available at http://advances.sciencemag.org/cgi/content/full/6/11/eaay1729/DC1

    Section S1. Conformal contact analysis for aerosol jet–printed electrodes

    Section S2. Methods for cross-validation

    Section S3. Fabrication and assembly process

    Section S4. Vergence physical apparatus and VR system

    Fig. S1. Apparatus for testing eye vergence motions.

    Fig. S2. Fabrication and assembly processes for the flexible device and the skin-like electrodes.

    Fig. S3. Circuit components, bending, and powering of the flexible device.

    Fig. S4. Design and characterization of the AgNP electrodes.

    Fig. S5. Stretching/bending properties of the skin-like electrodes fabricated by AJP.

    Fig. S6. Comparison between Ag/AgCl gel electrodes and aerosol jet–printed skin-like electrodes.

    Fig. S7. Electrode assessment for subjects 11 to 13.

    Fig. S8. Sensitivity of the periocular wearable electronics.

    Fig. S9. Performance differences between the periocular wearable electronics and the physical apparatus.

    Fig. S10. Comparison of average amplitudes from vergence training and a summary of classification accuracies.

    Table S1. Feature comparison between BioRadio and periocular wearable electronics.

    Table S2. OV2 cross-validation accuracies of subjects 1 to 5 using the physical apparatus.

    Table S3. OV1 cross-validation assessment of subjects 6 to 10 using the physical apparatus.

    Table S4. OV2 real-time classification of test subjects 6 to 10 using the physical apparatus.

    Table S5. OV2 real-time classification of test subjects 4, 8, and 9 using the physical apparatus.

    Movie S1. An example of a real-time vergence detection with a physical apparatus.

    Movie S2. An example of operation of a VR program—Brock String.

    Movie S3. An example of a real-time VR-based training apparatus.

  • Supplementary Materials

    The PDF file includes:

    • Section S1. Conformal contact analysis for aerosol jet–printed electrodes
    • Section S2. Methods for cross-validation
    • Section S3. Fabrication and assembly process
    • Section S4. Vergence physical apparatus and VR system
    • Fig. S1. Apparatus for testing eye vergence motions.
    • Fig. S2. Fabrication and assembly processes for the flexible device and the skin-like electrodes.
    • Fig. S3. Circuit components, bending, and powering of the flexible device.
    • Fig. S4. Design and characterization of the AgNP electrodes.
    • Fig. S5. Stretching/bending properties of the skin-like electrodes fabricated by AJP.
    • Fig. S6. Comparison between Ag/AgCl gel electrodes and aerosol jet–printed skin-like electrodes.
    • Fig. S7. Electrode assessment for subjects 11 to 13.
    • Fig. S8. Sensitivity of the periocular wearable electronics.
    • Fig. S9. Performance differences between the periocular wearable electronics and the physical apparatus.
    • Fig. S10. Comparison of average amplitudes from vergence training and a summary of classification accuracies.
    • Table S1. Feature comparison between BioRadio and periocular wearable electronics.
    • Table S2. OV2 cross-validation accuracies of subjects 1 to 5 using the physical apparatus.
    • Table S3. OV1 cross-validation assessment of subjects 6 to 10 using the physical apparatus.
    • Table S4. OV2 real-time classification of test subjects 6 to 10 using the physical apparatus.
    • Table S5. OV2 real-time classification of test subjects 4, 8, and 9 using the physical apparatus.

    Download PDF

    Other Supplementary Material for this manuscript includes the following:

    • Movie S1 (.mp4 format). An example of a real-time vergence detection with a physical apparatus.
    • Movie S2 (.mp4 format). An example of operation of a VR program—Brock String.
    • Movie S3 (.mp4 format). An example of a real-time VR-based training apparatus.

    Files in this Data Supplement:

Stay Connected to Science Advances

Navigate This Article