Experimental evidence of hourglass fermion in the candidate nonsymmorphic topological insulator KHgSb

+ See all authors and affiliations

Science Advances  05 May 2017:
Vol. 3, no. 5, e1602415
DOI: 10.1126/sciadv.1602415


Topological insulators (TIs) host novel states of quantum matter characterized by nontrivial conducting boundary states connecting valence and conduction bulk bands. All TIs discovered experimentally so far rely on either time-reversal or mirror crystal symmorphic symmetry to protect massless Dirac-like boundary states. Several materials were recently proposed to be TIs with nonsymmorphic symmetry, where a glide mirror protects exotic surface fermions with hourglass-shaped dispersion. However, an experimental confirmation of this new fermion is missing. Using angle-resolved photoemission spectroscopy, we provide experimental evidence of hourglass fermions on the (010) surface of crystalline KHgSb, whereas the (001) surface has no boundary state, in agreement with first-principles calculations. Our study will stimulate further research activities of topological properties of nonsymmorphic materials.

  • nonsymmorphic materials
  • topological insulator
  • Hourglass fermion


The discovery of symmetry-protected states of matter in time-reversal topological insulators (TIs) and topological semimetals (19) has attracted tremendous interest in a wide range of areas from condensed matter physics to materials science and high-energy physics. The first discovered TI (10) has time-reversal symmetry with Kramers degeneracy protecting nontrivial conducting states on the boundary. Over the past decade, the classification of topological insulating phases has been theoretically extended to many other discrete symmetry classes (1118). Among these, we mention particle-hole symmetry for topological superconductors, certain magnetic translation symmetry for antiferromagnetic TIs, and crystal symmetries for topological crystalline insulators (TCIs). TCIs studied in the past rely on the presence of a symmorphic mirror plane. In all the previously known cases, time-reversal and/or crystal symmetry generically protects massless boundary states with the usual Dirac-like dispersion (10, 19). Among spatial symmetries, symmorphic symmetries preserve the origin, whereas nonsymmorphic symmetries unavoidably translate the origin by a rational fraction of the lattice vector. The fractional translation could, in principle, play an important role in the classification of band topologies without analog in symmorphic crystals (20, 21). Very recently, Wang et al. (22) and Alexandradinata et al. (23) proposed the first theory of nonsymmorphic time reversal–invariant insulators. In contrast to the surface Dirac fermions ubiquitous in symmorphic TIs, nonsymmorphic TIs exhibit exotic surface fermions with hourglass-shaped dispersion (Fig. 1A). Wang et al. (22) also proposed that this new kind of TI can be realized in the series of three materials KHgX (X = As, Sb, and Bi).

Fig. 1 Hourglass fermions predicted in KHgSb with nonsymmorphic spatial symmetry.

(A) Schematic surface state bands for materials with symmorphic and nonsymmorphic spatial symmetries. Panels (i) and (ii) present topologically trivial and nontrivial cases with symmorphic spatial symmetry, respectively. Panels (iii) and (iv) are the same as (i) and (ii), respectively, but with nonsymmorphic spatial symmetry showing hourglass-shaped surface band dispersion. (B) Crystal structure of double-layered KHgSb with space group P63/mmc (no. 194). The yellow plane indicates the glide mirror plane Embedded Image preserved on the (010) side surface. The arrows and coordinates indicate the operation of the glide reflection Embedded Image. (C) Three-dimensional (3D) bulk BZ of KHgSb as well as its projected (001) and (010) surface BZs. (D and E) Projections of calculated bulk and surface bands on the (001) and (010) surfaces of KHgSb, respectively. The calculations show hourglass-shaped surface bands along the glide-symmetric line Embedded Image on the (010) surface. The complex-conjugate Embedded Image eigenvalues are paired as ±i pairs at Embedded Image and +1 (or −1) pairs at Embedded Image. (F) Logarithmic plot of XRD data measured on the (001) plane of KHgSb single crystal. The peaks marked by stars are from the Sb flux. Inset: LEED pattern shows the hexagonal structure of the (001) surface. arb.u., arbitrary units. (G) Core-level photoemission spectrum showing characteristic peaks of K 3p, Hg 5d, and Sb 4d core levels.

We start with a brief review of the surface state properties predicted theoretically for KHgSb, which is measured experimentally here. KHgSb has a layered crystal structure constructed from alternately stacked honeycomb HgSb and trigonal K layers. Previous ab initio calculations have predicted that the single-layered KHgSb is a weak TI, whereas the double-layered KHgSb is a trivial insulator, in which only time-reversal symmetry was considered (24). The double-layered KHgSb has a nonsymmorphic space group P63/mmc (SG 194) generated by screw rotation and glide mirror symmetries. Wang et al. (22) pointed out that the glide reflection Embedded Image, indicated as yellow plane in Fig. 1B, is crucial to form the hourglass fermions on the (010) surface. As illustrated in Fig. 1E, because of time-reversal symmetry, the complex-conjugate Embedded Image eigenvalues are paired at time reversal–invariant momenta Embedded Image and Embedded Image as ±i and +1 (or −1) pairs, respectively. The degeneracies at Embedded Image and Embedded Image have to exchange glide partners along the glide-symmetric line Embedded Image, resulting in an hourglass-shaped dispersion along Embedded Image. The crossing of the hourglass is stabilized by glide symmetry.


The chemical composition of KHgSb single crystals is confirmed by core-level photoemission measurements. As shown in Fig. 1G, the characteristic peaks of K, Hg, and Sb elements are observed. Because the theoretical prediction relies heavily on the nonsymmorphic crystal structure, we have performed x-ray diffraction (XRD) measurements on both single crystal and powder of KHgSb. On the basis of the XRD data, we determine that KHgSb has a nonsymmorphic double-layered structure with space group P63/mmc or P63mc (SG 186) (see the Supplementary Materials for details on structural analysis). Because both variants have the same nontrivial topology defined by the glide symmetry Embedded Image, our discussion will be limited to space group P63/mmc, which has been assumed in the theoretical work of Wang et al. (22). The XRD data on the (001) plane of KHgSb single crystal exhibit sharp (00l) diffraction peaks (Fig. 1F), indicating that the samples are well crystallized. The low-energy electron diffraction (LEED) pattern on the (001) surface shows a hexagonal structure, consistent with the sixfold symmetry of the crystal structure (Fig. 1F, inset).

To search for hourglass fermions, we performed systematic angle-resolved photoemission spectroscopy (ARPES) measurements to investigate the electronic structure of KHgSb. We first summarize the ARPES results recorded on the (001) surface of KHgSb in Fig. 2. The chemical potential on the pristine surface lies in the bandgap between the valence and conduction bands (Fig. 2, C and D), demonstrating the insulating nature of the bulk states in this material. The measured band dispersions along the high-symmetry lines Embedded Image are in good agreement with the calculated valence band structure (Fig. 2, C and D). The valence bands form an “M”-like top along both Embedded Image and Embedded Image. The band top is close to Fermi energy (EF), resulting in enhanced spectral intensity in the vicinity of Embedded Image at EF in Fig. 2A.

Fig. 2 Electronic structure on the KHgSb (001) surface.

(A) 3D intensity plot of ARPES spectra showing the electronic structure of valence bands in the kx-ky plane. (B) ARPES intensity plot along Embedded Image after in situ K doping, showing the bottom of the conduction bands. (C) ARPES intensity plot showing band dispersions along Embedded Image on the pristine (001) surface. The dashed curves represent the calculated bulk bands at kz = 0. Because generalized gradient approximation (GGA) calculations usually underestimate the bandgap of semiconductors, we rigidly shifted the calculated conduction bands upward to have a 0.46-eV bandgap, which is the value determined experimentally in (B). For clarity, the chemical potential of the calculated band structure is chosen to have the best match with the measured valence band dispersions. (D) Corresponding curvature intensity plot of (C).

To observe the conduction bands, we deposited potassium atoms onto the (001) surface in situ, which dopes electrons into the surface layer and thus shifts the local chemical potential upward. The bottom of the conduction bands is observed around Embedded Image after deposition of potassium (Fig. 2B). No sign of surface state is observed within the bulk bandgap, in agreement with the theoretical prediction.

We then turn our focus to the ARPES results recorded on the (010) surface, where the hourglass-shaped surface state was predicted to exist. As illustrated in the top-down view of the lattice in Fig. 2B of Wang et al. (22), the crystal structure of KHgSb can also be described as a stacking of chemically equivalent monatomic layers along the (010) direction, but the interlayer coupling along (010) is much stronger than that along (001). Thus, it is more difficult to obtain a flat (010) surface by cleaving. After many attempts of in situ cleaving, we succeeded in obtaining a mirror-like (010) surface several times. Highly dispersive and well-defined bands are observed with good reproducibility. We performed photon energy dependence measurements along Embedded Image on the (010) surface. These measurements map the electronic structure in the Γ − MK − Γ plane of the bulk Brillouin zone (BZ). For the bulk states, the measurements on the (010) surface with varying photon energy should be analogous to the in-plane mapping on the (001) surface at a fixed photon energy. In Fig. 3A, the Fermi surface (FS) intensity map obtained by varying photon energy on the (010) surface exhibits enhanced spectral intensity near Γ, which is consistent with our observation on the (001) surface in Fig. 2A. In addition, the FS intensity map in Fig. 3A shows two straight lines at kx = 0 and Embedded Image along Γ − MK − Γ, which is perpendicular to the (010) surface.

Fig. 3 Band dispersions along Embedded Image on the KHgSb (010) surface.

(A) ARPES intensity plot at EF recorded along Embedded Image by varying the photon energy from 20 to 100 eV on the (010) surface, which maps the kx-ky plane at kz = 0 of the 3D bulk BZ. The overlaid hexagons indicate the BZ structure in the kx-ky plane. (B and C) Band dispersions along cuts 1 and 2 indicated in (A), recorded with photon energy hv = 22 and 56 eV, respectively. The solid curves represent the calculated bulk bands along Γ − M. Note that the chemical potential on the (010) surface is slightly different from that on the (001) surface, most likely because of different surface chemistry effects. (D) Band dispersions along kx at different ky positions recorded with different photon energies from 22 to 56 eV. The dashed curve indicates the dispersion of one representative bulk band along Γ − KM. (E) Calculated bulk bands along Γ − KM. The red curves indicate the calculated bulk bands that correspond to the experimental band dispersion indicated in (D). (F) Near-EF band dispersions along kx at different ky positions recorded with different photon energies from 56 to 78 eV, showing a Dirac-like surface state band along Embedded Image. The momentum locations are indicated in (A).

To illuminate the origin of the straight-line FSs, we investigate the band dispersions recorded with different photon energies. In Fig. 3 (B and C), we compare the band dispersions measured along Γ − M with the calculated bulk bands. Although most of the experimental band dispersions are in good agreement with the bulk band calculations, we observe an extra feature close to EF at kx = 0 and Embedded Image, corresponding to the two straight lines in the FS intensity map. Figure 3D illustrates the evolution of the band dispersions upon sliding along the ky direction. The valence band top at Γ gradually sinks upon sliding from Γ to K, as indicated in Fig. 3D, which is consistent with the calculated bulk bands highlighted by red color in Fig. 3E. In contrast, the extra band is observable near EF in all the cuts in Fig. 3D, although with weaker spectral intensity around M. Figure 3F displays the near-EF band dispersions obtained with different photon energies. The extra band shows a nearly linear dispersion with the crossing point close to EF. The nondispersive feature of the extra band along ky (the momentum perpendicular to the surface) indicates its surface origin.

The measurements along Embedded Image indicate the existence of surface states on the (010) surface, where the hourglass-shaped surface state was predicted to exist. To capture the hourglass fermions and their connectivity to other bands in different directions of the surface BZ, we systematically investigate the low-energy electronic structure of the surface states in the (010) surface BZ in Fig. 4. Let us first review the overall surface band structure in the (010) surface BZ predicted by the calculations. The calculations show two Dirac-like bands that are split in energy along Embedded Image (kz = 0), leading to two Kramers points with an energy separation of 50 meV at Embedded Image (Fig. 4A). These bands are doubly degenerate along Embedded Image (kz = π) but open a small gap of 10 meV between the upper and lower branches at Embedded Image (Fig. 4B). The crossing points at Embedded Image and Embedded Image are nothing else but the doubly degenerate end points of the theoretically calculated hourglass-shaped dispersion along Embedded Image (Fig. 4, C and D).

Fig. 4 Electronic structure of surface states on the KHgSb (010) surface.

(A to C) Calculated surface band dispersions along Embedded Image, Embedded Image, and Embedded Image, respectively. (D) 3D sketch of the band structure of hourglass fermions along high-symmetry lines. (E and G) ARPES intensity plots showing band dispersions along Embedded Image and Embedded Image, respectively. (F and H) Corresponding curvature intensity plots of momentum distribution curves of (E) and (G), respectively. (I and J) ARPES intensity plot and corresponding curvature intensity plot along Embedded Image, respectively. (K) FS intensity map recorded at hv = 70 eV. (L) Summary of the experimental results recorded on the (001) and (010) surfaces. Top: Curvature intensity plot of the band dispersions along Embedded Image on the (001) surface. Bottom: Surface band structure along Embedded Image on the (010) surface. The symbols represent the extracted bands from the experimental data in (E) to (J). Middle: 3D bulk BZ of KHgSb and its projected (001) and (010) surface BZs.

Along Embedded Image, we observe a band splitting from the ARPES intensity plot in Fig. 4E. The band splitting is more discernible in the curvature intensity plot in Fig. 4F. Along Embedded Image, we observe only one Dirac-like band in Fig. 4 (G and H). These observed band dispersions are generally consistent with the calculated band structures shown in Fig. 4 (A and B). However, some fine features in the calculations remain to be identified. For instance, the band splitting along Embedded Image is only observed below −0.2 eV, whereas the paired crossing points at the Embedded Image point are not resolved. Moreover, the calculations have predicted a small gap at the Embedded Image point between the upper and lower branches, which is not resolved experimentally because of an insufficient resolution.

Figure 4I displays the ARPES intensity plot along Embedded Image, where the hourglass fermions theoretically reside. A broad peak disperses from −150 meV at Embedded Image to −50 meV at Embedded Image, which is consistent with the lower branch of the calculated hourglass bands in Fig. 4C. However, the upper branch is difficult to identify in the ARPES intensity data. In Fig. 4J, the curvature intensity plot exhibits a cross-like dispersion, which is consistent with the characteristic of the calculated hourglass fermions in Fig. 4C. However, it should be noted that band dispersions near EF in the curvature plots suffer interference from the Fermi cutoff, which could lead to some artifacts. We summarize the measured surface band dispersions on the (010) surface in the bottom panel of Fig. 4L. The surface bands along Embedded Image are connected to the bulk states via Dirac-like bands along Embedded Image and Embedded Image, indicating the nontrivial topology of surface states. Although the predicted hourglass fermions remain to be completely resolved, the observed (010) surface states are consistent with the global feature of the calculated band structure, thus providing strong evidence for the existence of hourglass fermions.


By mapping a large set of high-symmetry manifolds in the surface BZ, we provided the first experimental evidence that a newly proposed type of TI, hosting a novel type of surface state, exists in crystals protected by nonsymmorphic symmetry. Most of the symmetry groups in nature are nonsymmorphic and can host interesting topological classes with their own unique surface state responses. Last, nonsymmorphic symmetries can protect bulk semimetals hosting exotic types of fermions (25). All these await experimental discovery.


Sample growth method

Single crystals of KHgSb were grown using self-flux methods. The starting materials K (99.95%; Alfa Aesar), Hg (99.9995%; Aladdin), and Sb (antimony shot, 99.999%; Alfa Aesar) were put in an Al2O3 capsule and sealed in a quartz tube. The operation was in a glove box filled with high-purity argon. We presintered the samples by heating at 200°C for 20 hours in the furnace. After that, the samples were sealed in a tantalum capsule and then in a quartz tube, followed by heating at 900°C for 10 hours and maintaining this temperature for 5 hours. Then, the samples were cooled down to 480°C at a rate of 3°C/hour and were held at this temperature for 5 to 7 days.

ARPES measurement technique

ARPES measurements were performed at the “Dreamline” beamline of the Shanghai Synchrotron Radiation Facility with a Scienta D80 analyzer. The energy and angular resolutions were set to 15 to 30 meV and 0.2°, respectively. The samples for ARPES measurements were mounted in a BIP argon (>99.9999%)–filled glove box, cleaved in situ, and measured at 30 K in a vacuum better than 5 × 10−11 torr.

Calculation method

The Vienna Ab initio Simulation Package (26) was used for first-principles calculations. The GGA of Perdew-Burke-Ernzerhof type (27) was used for the exchange-correlation potential. Spin-orbit coupling was self-consistently taken into account. The cutoff energy for plane-wave expansion is 500 eV, and the k-point sampling grids for different structures were tested to be dense enough. The atomic structure and the lattice constants a = b = 4.78(4) Å and c = 10.22(5) Å were adapted in our calculations (22), which are very close to those determined by our powder XRD data. The KHgSb layers were stacked along the c axis, which is consistent with our experimental observations. The surface Green’s function method was used to calculate the surface states based on the maximally localized Wannier functions (28) of Hg 6s and Sb 5p orbitals obtained from ab initio calculations.


Supplementary material for this article is available at

Crystal structure of KHgSb determined from powder XRD data

fig. S1. Refinements of powder XRD data of KHgSb.

fig. S2. Band inversion confirmed by HSE06.

References (29, 30)

This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial license, which permits use, distribution, and reproduction in any medium, so long as the resultant use is not for commercial advantage and provided the original work is properly cited.


Acknowledgments: We acknowledge W. Yi, S.-F. Jin, and Z.-L. Li for refinements of the powder XRD data, as well as C. Fang, X. Dai, T.-T. Zhang, and D. Chen for valuable discussions. Funding: This work was supported by the Ministry of Science and Technology of China (2016YFA0300600, 2016YFA0401000, 2015CB921300, and 2013CB921700), the National Natural Science Foundation of China (11622435, 11474340, 11422428, 11274367, 11474330, 11504117, 11234014, 11274362, and 11674371), and the Chinese Academy of Sciences (CAS) (XDB07000000). Z.W. and B.A.B. were also supported by the Department of Energy (DE-SC0016239), NSF Early Concept Grants for Exploratory Research Award (NOA-AWD1004957), Simons Investigator Award (ONR-N00014-14-1-0330), Army Research Office Multidisciplinary University Research Initiative (W911NF-12-1-0461), NSF Materials Research Science and Engineering Center (DMR-1420541), Packard Foundation, and Schmidt Fund for Innovative Research. Y.H. acknowledges funding from the CAS Pioneer Hundred Talents Program (type C). Author contributions: H.D. and T.Q. conceived the ARPES experiments. J.M., B.L., P.Z., and T.Q. performed the ARPES measurements with the assistance of L.K., K.Y., K.K., Y.H., and S.S. Z.W., S.N., and H.W. performed ab initio calculations. C.Y., Y.S., and L.W. synthesized the single crystals. J.M., T.Q., and H.D. analyzed the experimental data. J.M., B.L., Z.W., and T.Q. plotted the figures. T.Q., H.D., B.A.B., J.M., Z.W., and P.R. wrote the manuscript. Competing interests: The authors declare that they have no competing interests. Data and materials availability: All data needed to evaluate the conclusions in the paper are present in the paper and/or the Supplementary Materials. Additional data related to this paper may be requested from the authors.
View Abstract

Related Content

Navigate This Article