Hierarchical deep reinforcement learning reveals novel mechanism of cell movement.

Time-lapse images of cells and tissues contain rich information of dynamic cell behaviors, which reflect the underlying processes of proliferation, differentiation and morphogenesis. However, we lack computational tools for effective inference. Here, we exploit Deep Reinforcement Learning (DRL) to infer cell-cell interactions and collective cell behaviors in tissue morphogenesis from 3D, time-lapse images. We used Hierarchical DRL (HDRL), known for multiscale learning and data efficiency, to examine cell migrations based on images with ubiquitous nuclear label and simple rules formulated from empirical statistics of the images. When applied to C. elegans embryogenesis, HDRL reveals a multi-phase, modular organization of cell movement. Imaging with additional cellular markers confirms the modular organization as a novel migration mechanism, which we term sequential rosettes. Furthermore, HDRL forms a transferable model that successfully differentiates sequential rosettes-based migration from others. Our study demonstrates a powerful approach to infer the underlying biology from time-lapse imaging without prior knowledge.