Research Article

Decoding crystallography from high-resolution electron imaging and diffraction datasets with deep learning

See allHide authors and affiliations

Science Advances  30 Oct 2019:
Vol. 5, no. 10, eaaw1949
DOI: 10.1126/sciadv.aaw1949
  • Fig. 1 Neural network data architecture and workflow for crystal space group determination from experimental high-resolution atomic images and diffraction profiles.

    (A) Any material in an STEM, in this case crystalline STO islands distributed on a rock salt MgO substrate, can be simultaneously imaged with (B) high-resolution atomic mass contrast STEM imaging and (C) decoupled with a selective area. (D) FFT to reveal the material’s structural details. (E) On the basis of either electron diffraction or FFT of an atomic image, a two-dimensional azimuthal integration translates this information into a relevant one-dimensional diffraction intensity profile from which the relative peak positions in reciprocal space can be indexed. arb. units, arbitrary units. (F) Seeding the prediction of crystallography is a hierarchical classification using a one-dimensional convolution neural network model replicated at (G), each layer from family to space group forming a nested architecture. On the basis of the derived peak positions in the azimuthal integration profile, the prediction on STO is reported in Table 2.

  • Fig. 2 Model evaluation from low- to high-symmetry materials.

    Given the 571,340 total structures, 136,534 randomly chosen structures were used to evaluate the model at the family level, finding a high concentration primarily along the 1:1 diagonal. At one level down to the genera level, we further compare the labeled best and worst confusion matrices from cubic and orthorhombic crystal families, highlighting the highest and lowest level of accuracies trained into the nested neural network architecture. See the Supplementary Materials for accompanying confusion matrices over all crystal families and genera.

  • Fig. 3 Optimizing the CNN and pipeline for real time.

    The number of peaks used per family benchmarks the model’s sensitivity and accuracy. On the basis of these confusion matrices, a strict threshold to select four peaks is used in the current implementation. The result is the highest level of accuracy, predictive speed, and compression of structural-based information from electron microscopy images and diffraction patterns.

  • Fig. 4 Evaluation and validation over low- to high-symmetry materials with either high-resolution electron imaging or diffraction.

    The model has been evaluated on (A) a cubic polycrystalline CeO2 using high-resolution imaging, (B) SAED of graphene at 60 kV, and (C) BSTS, a quantum-based topological insulator. (D) Rounding out the series is the orthorhombic α-phase uranium studied using selected area electron diffraction from four separate zone axes for the same material inside a high-resolution FEI Titan STEM. The predictions are reported in Table 2.

  • Table 1 Population statistics and training accuracy evaluated.

    Reported population for individual structures over family and genera constituting a database of 571,340 total structures. Note that N/A appears for those genera with a single class.

    Accuracy (%)Population
    Triclinic91.04%105,200
    PedialN/A16,740
    PinacoidalN/A88,460
    Monoclinic86.73%217,156
    Sphenoidal86.50%21,705
    Domatic74.95%14,997
    Prismatic90.14%180,454
    Orthorhombic75.75%104,526
    Rhombic-disphenoidal77.18%22,182
    Rhombic-pyramidal92.32%19,722
    Rhombic-dipyramidal67.42%62,622
    Tetragonal84.81%40,770
    Tetragonal-pyramidal65.80%1,081
    Tetragonal-disphenoidal76.46%1,437
    Tetragonal-dipyramidal96.23%5,112
    Tetragonal-trapezohedral82.99%2,373
    Ditetragonal-pyramidal88.16%1,450
    Tetragonal-scalenohedral84.22%4,309
    Ditetragonal-dipyramidal81.14%25,008
    Trigonal82.78%31,252
    Trigonal-pyramidal81.48%3,499
    Rhombohedral90.46%6,017
    Trigonal-trapezohedral94.13%2,321
    Ditrigonal-pyramidal82.90%5,831
    Ditrigonal-scalenohedral89.43%13,584
    Hexagonal86.07%24,147
    Hexagonal-pyramidal88.17%1,828
    Trigonal-dipyramidal90.14%453
    Hexagonal-dipyramidalN/A2,100
    Hexagonal-trapezohedral99.24%930
    Dihexagonal-pyramidal92.01%2,969
    Ditrigonal-dipyramidal95.36%3,230
    Dihexagonal-dipyramidal93.64%12,637
    Cubic95.47%48,289
    Tetartoidal96.75%1,842
    Diploidala90.56%1,389
    Gyroidal93.68%737
    Hextetrahedral97.14%6,475
    Hexoctahedral87.29%37,846
  • Table 2 Crystallographic prediction of experimental images and diffraction patterns.

    Comparing from high- to lower-symmetry materials, this shows the prediction for each material.

    MaterialExpectationFirst predictionSecond predictionThird prediction
    CeO2Cubic Fm3¯m225219221
    No. 22587.1%5.6%3.6%
    C grapheneHexagonal P63/mmc194173191
    No 19490.1%2.5%0.2%
    BSTSTrigonal R3¯m166163148
    No. 16626.2%25.7%3.48%
    α-Phase UOrthorhombic Cmcm741963
    No. 6334.7%33.7%15.9%

Supplementary Materials

  • Supplementary material for this article is available at http://advances.sciencemag.org/cgi/content/full/5/10/eaaw1949/DC1

    Fig. S1. Confusion matrix for monoclinic genera and space group.

    Fig. S2. Confusion matrix for orthorhombic genera and space group.

    Fig. S3. Confusion matrix for tetragonal family and genera 15.

    Fig. S4. Confusion matrix for trigonal family and genera 20.

    Fig. S5. Confusion matrix for hexagonal family and genus 25.

    Fig. S6. Confusion matrix for cubic family and genera 31.

    Fig. S7. Aggregate signals for triclinic, monoclinic, orthorhombic, and cubic families plotted against two theta values.

    Table S1. Comparing models for crystallographic determination.

  • Supplementary Materials

    This PDF file includes:

    • Fig. S1. Confusion matrix for monoclinic genera and space group.
    • Fig. S2. Confusion matrix for orthorhombic genera and space group.
    • Fig. S3. Confusion matrix for tetragonal family and genera 15.
    • Fig. S4. Confusion matrix for trigonal family and genera 20.
    • Fig. S5. Confusion matrix for hexagonal family and genus 25.
    • Fig. S6. Confusion matrix for cubic family and genera 31.
    • Fig. S7. Aggregate signals for triclinic, monoclinic, orthorhombic, and cubic families plotted against two theta values.
    • Table S1. Comparing models for crystallographic determination.

    Download PDF

    Files in this Data Supplement:

Stay Connected to Science Advances

Navigate This Article