Spin polarization of gapped Dirac surface states near the topological phase transition in TlBi(S(1-x)Se(x))2.

We performed systematic spin- and angle-resolved photoemission spectroscopy of TlBi(S(1-x)Se(x))(2) which undergoes a topological phase transition at x ~ 0.5. In TlBiSe(2) (x = 1.0), we revealed a helical spin texture of Dirac-cone surface states with an intrinsic in-plane spin polarization of ~0.8. The spin polarization still survives in the gapped surface states at x > 0.5, although it gradually weakens upon approaching x = 0.5 and vanishes in the nontopological phase. No evidence for the out-of-plane spin polarization was found, irrespective of x and momentum. The present results unambiguously indicate the topological origin of the gapped Dirac surface states, and also impose a constraint on models to explain the origin of mass acquisition of Dirac fermions.