Magnetic Chern Insulators in a monolayer of Transition Metal Trichalcogenides.

A monolayer of transition metal trichalcogenides has received a lot of attention as potential two dimensional magnetic materials. The system has a honeycomb structure of transition metal ions, where both spin-orbit coupling and electron correlation effect play an important role. Here, motivated by these transition metal series with effective doping or mixed valence case, we propose the possible realization of magnetic Chern insulators at quarter filled honeycomb lattice. We show that the interplay of intrinsic spin-orbit coupling and electron correlation opens a wide region of ferromagnetic Chern insulating phases in between metals and normal insulators. Within the mean field approximation, we present the phase diagram of a quarter filled Kane-Mele Hubbard model and also discuss the effects of Rashba spin-orbit coupling and nearest neighbor interactions on it.

Introduction

The effect of spin-orbit coupling plays an important role in the electronic structures of solids. In particular, spin-orbit coupling is essential for the realization of topological insulators where the gapless edge states are protected by time reversal symmetry[1-10]. Both theoretical prediction[4,11,12] and experimental realization[13-18] of topological insulators in real materials have been extended to the study of topologically non-trivial phases. More recently, it is also pointed out that the electron interaction effect could induce exotic phases such as topological Mott insulators[19,20]. Thus, the interplay of spin-orbit coupling and electron interaction has garnered a lot of attention, leading to a new discovery of materials and theoretical studies[21-37].

Topologically distinct phases introduced by Haldane showed that the quantum Hall phenomena could occur purely from the band structure in the absence of any external magnetic field, as a realization of the parity anomaly in (2 + 1) dimensional relativistic field theory[38]. By introducing the staggered flux on a honeycomb lattice, the system becomes an insulator with a non-zero topological invariant, termed as a Chern insulator. Experimental realization of Haldane model has been recently proposed in ultra cold atom system[39], yet none of them has been reported in any solid state systems.

Here, we propose possible realization of Chern insulators in two dimensional van der Waals materials, especially in transition metal trichalcogenides. The van der Waals materials are characterized with layered crystals where individual layers are weakly coupled via van der Waals forces but with strong covalent bonding in the layer. Thus it is possible to peel away a single layer breaking the van der Waals bonds. One of the most well known examples is a single layer of graphene, peeled away from bulk graphite using scotch tape[40-42]. Using scotch tape technique, a variety of van der Waals materials have been successfully exfoliated into atomically thin layers. It turns out that pure two dimensional materials are not just limited to graphene, rather, 2D hexagonal boron nitride and the family of transition-metal chalcogenides are also present[43-52]. In particular, the transition-metal trichalcogenide (TMTC) series (with chemical formula TMBX 3 where TM represents transition metals, B = P, Si or Ge and X represents chalcogens) are recently receiving a great attention in both theoretical studies[45,46,53,54] and experiments[47-50,55-58].

In TMTCs, the transition metal ions form a layered honeycomb structure, thus, a single 2D unit consisting of these transition metal atoms has similar lattice structure as that of graphene. However, unlike the case of graphene which has a zero bandgap, TMTC series have a sizable variation of bandgap ranging from 0.5 eV to 3.5 eV depending on the transition metal atoms[59]. In addition, the transition metal compounds possess large spin-orbit coupling and strong electron correlations compared to the case of graphene. Hence, these monolayers of TMTC series open a whole zoo of new exotic phases in two dimensional honeycomb lattice allowing possible control of both electron interaction and spin-orbit coupling. So far, there have been many recent studies on TMTC materials especially with 3d transition metal ions but not much attention on TMTCs with 4d and 5d transition metal ions.

In this paper, we study the interplay of spin-orbit coupling and electron correlation motivated by TMTC materials with 4d and 5d transition metal ions. Especially, we study quarter- (or three-quarter) filled system with effective pseudospin-1/2 model. As a minimal model, we consider the Kane-Mele Hubbard model[21-37]. At quarter filling, we found several metallic and insulating phases within mean field approximation; ferromagnetic Chern metals, ferromagnetic Chern insulators and ferromagnetic normal insulators with broken time reversal and inversion symmetries. In particular, we point out that the magnetic Chern insulators could naturally arise when both spin-orbit coupling and electron interactions are present at quarter filling. In addition, we also found the possible transition from Chern insulator to normal insulator as originally proposed in the Haldane model[38]. We also investigate the stability of these phases in the presence of nearest-neighbor interaction and Rashba spin-orbit coupling.

In 4d or 5d TMTCs, the presence of strong spin-orbit coupling and crystal field splitting can split t2 orbitals of transition metals ions (octahedral sites) into lower quartet orbitals with the effective total angular momentum j = 3/2 and upper doublet with j = 1/2 in the atomic limit[60,61]. When there are 9 or 11 electrons per unit cell (two sites) of honeycomb lattice, the j = 3/2 orbitals are fully filled, while the j = 1/2 orbitals of two sites have one or three electrons in total, resulting in effective quarter- or three-quarter fillings with pseudospin-1/2 model. Such fillings that include odd number of electrons per unit cell, can be realized by combinations of the two transition metal ions in a unit cell. Among Mo, W, Ru, Os, Tc, Re ions, one can combine two ions which satisfy d4 (or d6) and d5 in each sublattice. It can also be realized by doping via gating or hydrogen substitution.

Before we study the quarter filled case, we briefly summarize the earlier work related to the Kane-Mele (Hubbard) model. The Kane-Mele model was first proposed to study the quantum spin Hall (QSH) effect in graphene, but, due to very small spin-orbit coupling, the topological properties were not clearly visible. Instead, the search was extended to real materials with strong spin-orbit coupling[19,62-66]. There were also studies of possible QSH phases due to the spontaneous spin SU(2) symmetry breaking, induced by electron interactions even in the absence of spin-orbit coupling[67,68]. Further related work on topological phase transitions in the presence or absence of spin-orbit coupling have been studied in other lattices like kagome, decorated honeycomb and diamond etc[69-72]. The electron correlation effects on the Kane-Mele model were also extensively studied at half filling[21-36]. Away from half filling, possible pairing mechanism of superconductivity has received attention which could occur at 3/8 or 5/8 filling near the Van-Hove singularity in doped Kane-Mele model[37,73,74]. However, few studies related to the interplay of strong intrinsic spin-orbit coupling and electron correlations have been discussed for the case of 1/4 or 3/4 filling[75] and there are no detailed study of the full phase diagram at these fillings.

Results

We start by introducing the Kane-Mele Hubbard model. The Hamiltonian is,where (c) is the electron creation (annihilation) operator at site i with spin α ∈ {↑, ↓} on a honeycomb lattice, is the number density operator, σ is a Pauli matrix, 〈ij〉 and 〈〈ij〉〉 denotes pairs of nearest-neighbor and next-nearest-neighbor sites respectively. t, U and λ are the nearest-neighbor hopping energy, the strength of the on-site Coulomb repulsion and the second-neighbor spin-orbit coupling strength respectively. Throughout this paper, we set the hopping amplitude t ≡ 1. ν = −ν = ±1, depending on whether the electron traversing from i to j makes a right (+1) or a left (−1) turn.

At quarter filling, the system remains metallic with or without spin-orbit coupling for the non-interacting case U = 0. At this filling, irrespective of the spin-orbit coupling strength, the perfect nesting wave vectors are absent at the Fermi surface. Thus, we neglect the instability of any charge density wave or spin density wave with finite momentum when the onsite repulsion U is turned on. The interaction term in the Hamiltonian Eq. (1) can be rewritten in terms of spin operator and the total number of electrons in the system is N; . The last term is a constant and just shifts the total energy of the system, thus can be ignored. We solve this interacting Hamiltonian using mean field approximation and the mean field Hamiltonian can be written as,where is the non-interacting part of the Hamiltonian and . There are two vector order parameters represented by where a labels the two sublattices (A, B) of honeycomb lattice. This mean-field Hamiltonian has the following form up to a constant termwhere c = (c(), c(), c(), c()) is the basis for honeycomb (A, B) sublattices with spins ↑, ↓. The Hamiltonian matrix h() can be represented in terms of two Pauli matrices  = (τ, τ, τ) and  = (σ, σ, σ) for A, B sublattices and spin ↑, ↓ respectively. d1() = (1 + cos k1 + cos k2), d2() = (sin k1 − sin k2) and d3() = 2λ(sin k1 + sin k2 − sin (k1 + k2)) with k1, k2 being the momentum components along the basis vectors and in a honeycomb lattice and , with μ = x, y, z.

Figure 1 is the phase diagram as a function of U and λ at quarter filling, based on solving the self consistency equation. The phases in the figure are classified in terms of the magnetization. The phase where both and are zero is the paramagnetic phase. While ≠ 0,  = 0 ⇒  =  phase corresponds to the ferromagnetic phase, M ≠ 0, m ≠ 0 ⇒  ≠  corresponds to the ferromagnetic phase with broken inversion symmetry. The phase diagram is explained in detail below.

Figure 1

Phase diagram for the Kane-Mele Hubbard model at quarter filling as functions of intrinsic spin-orbit coupling λ and onsite Coulomb repulsion U. Here t = 1. The inset shows enlarged view of dashed box in the main plot. PM: paramagnetic metal, FM: ferro-magnetic (FM) metal, FNI: FM normal insulator, FCM: FM Chern metal, FCI: FM Chern insulator. F and distinguish FM order with or without inversion symmetry and the subscripts xy or z represent the direction of magnetic order. The blue and yellow shaded regions show the magnetic Chern insulator phases. Detailed explanation of each phase is described in the main text.

In the absence of both spin-orbit coupling and onsite interaction (λ = U = 0), the system is in a metallic phase. With increasing U but λ = 0, the magnetic moment is being developed and the system goes into magnetically ordered phases. In the range 6.8 < U < 7.5, the ferromagnetic metal is stabilized where  =  ≠ 0 with broken time reversal symmetry. In this phase, h() is represented as two copies of graphene Hamiltonian with spin up and down, and their energies are separated proportional to the magnetization values  = . At quarter filling, hence, the lowest energy band remains gapless i.e. metallic. At U = U = 7.5, there is a second order phase transition into a ferromagnetic metal where the system starts developing magnetization  ≠  with broken inversion symmetry. In this case, both and in Eq. 3 are non-zero and thus, the lowest two energy bands are separated at every momentum value, but, the bands still cross the Fermi level. On further increasing U, the magnetization keep increasing opening a band gap between the lowest two bands and a ferromagnetic insulator with inversion symmetry broken is stabilized. All of these phases at λ = 0 are topologically trivial cases, thus we labeled these phases as paramagnetic metal ‘PM’, ferromagnetic metal with inversion symmetry ‘FM’, ferromagnetic metal with inversion symmetry broken ‘’ and ferromagnetic normal insulator with broken inversion symmetry ‘’ as shown in Fig. 1.

In the presence of spin-orbit coupling (λ ≠ 0), the non-interacting Kane-Mele model is just two copies of Haldane model with the phase factor ϕ = π/2 discussed in ref.[38] and opposite sign for spin up and down. Here, there is no extra mass term related to inversion symmetry breaking in the Hamiltonian and the system is metallic at quarter filling. With increasing interaction U, the magnetization is being developed along z direction i.e. above U which depends on the value of λ. In Eq. (3), this is equivalent to M ≠ 0 and M and are zero. The preference of magnetization along z direction can be easily understood by comparing the energies of single particle mean field Hamiltonian h() for two different cases, and . In the former case, the single particle energy of the lowest band is . In the latter case, the energy of the lowest band is . Near the second order transition point from paramagnetic metal to ferromagnetic metal, the magnetization value M is small and we see that the mean field solution with M ≠ 0 has lower energy than the case with M ≠ 0, thus magnetic order along z direction is favored. With a finite M, the degenerate bands are separated having non zero Chern number ±1. Although the lowest two bands are well separated at each momentum, the bands still cross the Fermi level at quarter filling, thus a ferromagnetic Chern metal phase, ‘FCM’, is stabilized.

With further increasing U, the lowest two bands are eventually separated, resulting in a second order phase transition from ‘FCM’ phase to ferromagnetic Chern insulating phase denoted as ‘FCI’ with M ≠ 0 (See Fig. 1). In this phase, quarter filling corresponds to filling the lowest band that has the Chern number +1, resulting in the Hall conductivity σ = e2/h. This Chern insulating phase can also be confirmed by the edge state calculation. We consider the honeycomb lattice with periodic boundary condition along one of the basis vector and zigzag edge along the direction of other basis vector. Based on the mean field Hamiltonian, we can plot the energy dispersion of the bands with edge states as shown in Fig. 2. Figure 2(a) shows the energy dispersion in FCI at U = 8 and λ = 0.3. We see that there are gapless edge states for the FCI phase along with the bulk energy gap thus indicating it to be a non-trivial phase. As shown in Fig. 1, it is remarkable that the presence of both spin-orbit coupling and Coulomb repulsion naturally opens a wide range of ferromagnetic Chern insulator at quarter filling. This is very distinct situation compared to the half filled case where the QSH phase exists even without electron correlation.

Figure 2

Energy spectra for system with zigzag edge for λ = 0.3 and (a) U = 8 which is in FCI region, (b) U = 13 which is in region.

For , there is another second order phase transition within mean field approximation. Increase of U leads to different magnetic phases where , thus both M and m are non zero in Eq.(3). In this phase, the system still has non-trivial topology and becomes ferromagnetic Chern insulator with broken inversion symmetry denoted as ‘’. With further increasing U, there exist a critical value of m where the lowest two bands cross and then the system goes into a ferromagnetic normal insulator with broken inversion symmetry denoted as ‘’. Such phase transitions between FCI or and are exactly consistent with the phase transition in the original Haldane model that is induced by inversion breaking mass term at the phase ϕ = π/2[38]. These phases and phase transitions are only stable for weak spin-orbit coupling. For λ < 0.06, there are two metallic phases with broken inversion symmetry: ferromagnetic Chern metal ‘’ and ferromagnetic normal metal ‘’ as shown in Fig. 1.

For very large values of U, the system stabilizes a ferromagnetic normal insulator with magnetization in xy plane but with broken inversion symmetry denoted as . (See Fig. 1) The phase can be understood by considering the spin-orbit coupling term as a perturbation in the large U limit. In the absence of spin-orbit coupling, the single particle mean-field energies are exactly the same irrespective of whether the magnetization is along z direction (case I) or xy plane (case II) due to SU(2) symmetry. When the spin-orbit coupling is small but finite, the second order correction in energy is for case I, whereas for case II, the correction is . In the ferromagnetic phase with broken inversion symmetry, increase of U leads to large M ≈ m. Hence, the second order correction for the case I is proportional to 1/M3 and for the case II the correction is proportional to 1/M. Therefore, at large U limit, when the spin-orbit coupling term is present, the magnetization along xy plane is preferred. For small , there is a first order phase transition from phase to , while for , first order phase transition is directly from FCI phase to phase. (See Fig. 1). Figure 2(b) shows the energy band for the honeycomb lattice with zigzag edge in phase, at U = 13 and λ = 0.3. There is no edge state crossing at quarter filling which indicates the system in a trivial insulating phase.

Finally, we also discuss the stability of the phase diagram in the presence of Rashba spin-orbit coupling and the nearest neighbor Coulomb interaction. The nearest-neighbor Coulomb interaction favors charge order and develops a mass term which breaks the inversion symmetry of the lattice. Thus, the phase with ferromagnetic order  ≠  is further stabilized and the area of inversion broken ferromagnetic normal insulator phase is increased in the phase diagram. When both intrinsic (λ) and Rashba (λ) spin-orbit couplings are present, the particle hole symmetry is broken. In this case, the energy dispersion for non-interacting Hamiltonian is independent of the sign of λ but depends on the sign of λ; n∈[1, 4] is the band index from the lowest to highest energy bands. Here, . This can be easily seen from the energy form of the single particle Hamiltonian with both λ and λ. We found that the phase diagram remains almost unchanged for λ ≤ |λ| but the magnetization value decreases with the increase of λ. For λ > |λ| and negative λ, the system in phase undergoes a first order phase transition to FCI phase and the critical λ value for this transition increases with increase in U. For λ > λ with positive λ, there is a first order transition from to phase. For the latter case, the system in FCI phase goes to under first order phase transition for intermediate U.

Conclusion

In conclusion, we have studied the interplay of intrinsic spin-orbit coupling and onsite Coulomb interactions at quarter filled honeycomb lattice and predicted possible phases realizable in series of transition metal trichalcogenides under optimal doping. Especially with 4d and 5d transition metal ions, we have focused on the Kane-Mele Hubbard model at quarter filling and have shown possible realization of magnetic Chern insulators due to the presence of both electron interaction and spin-orbit coupling, as shown in Fig. 1. Within mean field approximation, we found that the magnetic Chern insulating phase is naturally opened for a wide range of interaction strength and spin orbit coupling as shown in Fig. 1. Furthermore, it can also lead to the phase transition between Chern insulator and normal insulator that was originally proposed by Haldane[38], by stabilizing magnetic order even in the presence of Rashba spin-orbit coupling and nearest neighbor Coulomb interaction.

The realization of such topological phases in real materials will be very interesting as a future work. In addition, the effect of magnetic field on these systems with spin-orbit coupling and considerable electron correlation is also interesting which is beyond the scope of this paper. In the presence of finite temperature, the magnetic Chern insulating phase with magnetization along the z-direction are robust when both the interaction and the spin orbit coupling strength is large. On increasing the temperature, the fluctuation effect becomes more dominant and above a critical value, these phases may no longer remain robust. In the presence of finite temperature, the spontaneous symmetry breaking of the continuous symmetry is prohibited, hence, the phase transition to ferromagnetic phases with magnetization along x or y direction may not be a stable phase though quasiordering is possible.

Methods

We adopt the mean field approximation to solve the Kane-Mele Hubbard model Eq. (1), resulting in the phase diagram shown in Fig. 1. The mean field Hamiltonian is given in Eq. (2). Here the fluctuation term is neglected as in mean field theory and we assume that the deviation of the operator from its average value is very small. Except near the transition points, the fluctuations are very small and thus can be ignored. Within the mean field approximation, our order parameter is given as

In order to solve a self-consistent equation for , (i) start the iteration with a random initial guess for each components of , (ii) diagonalize H using and find the energy and eigenfunctions, H|ψ(k, M)〉 = ε(k, M)|ψ(k, M)〉, where k takes the value in the N × N Brillouin Zone mesh and n is the band index. Here we took the value of N upto 150 and checked the phases are robust against the change of N. (iii) Tune the chemical potential μ to quarter filling by fixing the number of particles, where n[ε(k, M), μ] is the Fermi distribution function. (iv) Using the eigenfunctions of the mean field Hamiltonian, calculate the expectation value of the spin vector on each site in the unit cell and compute the new values of . The whole process from step (ii) to (iv) is repeated until all the quantities converge. We repeat this process for various initial guesses and sometimes find different mean field solutions. Comparing the energies of these solutions, we pick up the lowest energy state as the ground state of the Hamiltonian.