How does habitat filtering affect the detection of conspecific and phylogenetic density dependence?

Conspecific negative density dependence (CNDD) has been recognized as a key mechanism underlying species coexistence, especially in tropical forests. Recently, some studies have reported that seedling survival is also negatively correlated with the phylogenetic relatedness between neighbors and focal individuals, termed phylogenetic negative density dependence (PNDD). In contrast to CNDD or PNDD, shared habitat requirements between closely related individuals are thought to be a cause of observed positive effects of closely related neighbors, which may affect the strength and detectability of CNDD or PNDD. In order to investigate the relative importance of these mechanisms for tropical tree seedling survival, we used generalized linear mixed models to analyze how the survival of more than 10 000 seedlings of woody plant species related to neighborhood and habitat variables in a tropical rainforest in southwest China. By comparing models with and without habitat variables, we tested how habitat filtering affected the detection of CNDD and PNDD. The best-fitting model suggested that CNDD and habitat filtering played key roles in seedling survival; but that, contrary to our expectations, phylogenetic positive density dependence (PPDD) had a distinct and important effect. While habitat filtering affected the detection of CNDD by decreasing its apparent strength, it did not explain the positive effects of closely related neighbors. Our results demonstrate that a failure to control for habitat variables and phylogenetic relationships may obscure the importance of conspecific and heterospecific neighbor densities for seedling survival.