Compression of dynamic tactile information in the human hand

See allHide authors and affiliations

Science Advances  15 Apr 2020:
Vol. 6, no. 16, eaaz1158
DOI: 10.1126/sciadv.aaz1158
  • Fig. 1 Recording touch-elicited waves in the whole hand.

    (A) Sensor placements and (C) view of the 30 three-axis micro-electromechanical system accelerometers worn on the hand. (B) Representative patterns of propagating vibrations in the skin, drawn from the dataset. Each was elicited via hand-object contact during one of 13 manual gesture types (see Materials and Methods). (D) Example of 90 acceleration time series elicited by tapping Digit II. The signals rose quickly and faded within 30 ms after contact. (E) Skin-object contacts elicit vibrations that propagate throughout the hand as elastic waves.

  • Fig. 2 Optimal spatiotemporal primitives and encodings.

    (A) Each row represents a spatiotemporal basis pattern. Motifs were similar when the rank, M, was varied (Fig. 3). The tactile bases, displayed at 2-ms intervals in descending order of activation, reflect the individuation of digits and larger representations of commonly used digits II and III. (B) Analysis within different frequency bands revealed that different basis patterns captured distinct frequency content. (C) Activations, hi(t) (shown in grayscale, temporal resolution: 1 ms), produced by encoding the displayed tactile stimulus (see Fig. 1). Blue ticks show the time instant for each displayed stimulus frame. (D) Mean activations for stimuli elicited by each gesture class, averaged across all encoded trials.

  • Fig. 3 Learning and evaluating efficient tactile encodings.

    (A) Random sampling of 100 tactile stimuli drawn from the dataset illustrates its diversity (time averages shown). (B) Maximizing the efficiency with which the stimuli were encoded yielded primitive bases, wi(x, t) (time averages shown). Each row corresponds to an encoding of fixed rank, M, from M = 2 to 12 bases, arranged in order of increasing activation. Individuated digit representations were highly conserved. Higher-rank encodings included additional diffuse patterns. (C) The encoding residual decreased with the rank, while classification improved. (D) Results based on the neural simulations. As the number of spiking bases increased, the encoding residual decreased, and classification also improved. (E) Optimizing the encoding of spiking data from 773 simulated afferents yielded bases (M = 8 shown) that integrated activity in neural populations throughout the hand. They reflected the individuation of digits and denser activation of the fingertips, similar to results obtained with the mechanical data.

Supplementary Materials

  • Supplementary Materials

    Compression of dynamic tactile information in the human hand

    Yitian Shao, Vincent Hayward, Yon Visell

    Download Supplement

    The PDF file includes:

    • Figs. S1 to S9
    • Table S1
    • Sections S1 to S5
    • References

    Other Supplementary Material for this manuscript includes the following:

    Files in this Data Supplement:

Stay Connected to Science Advances

Navigate This Article