Bioimage Analysis and Cell Motility.

Bioimage analysis (BIA) has historically helped study how and why cells move; biological experiments evolved in intimate feedback with the most classical image processing techniques because they contribute objectivity and reproducibility to an eminently qualitative science. Cell segmentation, tracking, and morphology descriptors are all discussed here. Using ameboid motility as a case study, these methods help us illustrate how proper quantification can augment biological data, for example, by choosing mathematical representations that amplify initially subtle differences, by statistically uncovering general laws or by integrating physical insight. More recently, the non-invasive nature of quantitative imaging is fertilizing two blooming fields: mechanobiology, where many biophysical measurements remain inaccessible, and microenvironments, where the quest for physiological relevance has exploded data size. From relief to remedy, this trend indicates that BIA is to become a main vector of biological discovery as human visual analysis struggles against ever more complex data.

Introduction

Movement appears as a recurring advantage to life at any scale. At the smallest level, cells evolved a myriad of migration modes because it helped them thrive through natural selection: a longer range of action boosts their potential to colonize resources and cooperate. Whether physiological or pathological, many biological processes in multicellular organisms also rely on changes in cell position or location; for example, mounting an immune response relies on the ability of lymphocytes to circulate through blood and tissue to the target antigen, whereas infectious agents navigate within the organism to reach their niche. Accordingly, many potential therapeutic treatments aim at impairing the motility of pathogenic cells while sparing the host, although in some cases, such as the metastasis of cancer cells or infections by certain ameboid parasites, it can be difficult to be specific because parts of the protein repertoire regulating cell migration are conserved across the organisms., Therefore, studies on cell motility do not only constitute fundamental research but are also of use to therapeutic investigation. To deconstruct the molecular machinery underlying movement in sufficient detail, research on cell migration profits from the combination of (at least) three approaches: microscopy imaging, biophysical measurements, and the engineering of nature-like microenvironments. We argue that all of them benefit from bioimage analysis (BIA).

(1) The study of cell motility has been inseparable from microscopy. Advances in optical magnification have allowed resolving increasingly smaller features; the combination of fluorescence microscopy and tags has permitted identifying some of the prime proteins that regulate motion, and other technical improvements have facilitated high-speed or long-term observations both in vitro and in vivo; e.g., newer digital camera sensors such as charge-coupled device (CCD) or complementary metal oxide semiconductor (CMOS) require fewer photons, reducing acquisition time or phototoxicity. A quantitative analysis of image data has been essential to ensure the objectivity and reproducibility of the findings, but also to pick up on subtle or complex differences invisible to the naked eye. For instance, it was only the tracking and statistical analysis of fluctuations in speckle microscopy images that led to the realization that two actin networks coexisted within lamellipodia.

(2) To study the physical mechanisms driving cell movement, it is not enough to solely watch the molecules involved. Rather, their action has to be measured to determine whether they are able to generate the necessary forces and pressures. Physical measurements were introduced to biology experimentally: the first myosin forces were probed using an enhanced laser trap and wrinkles on deformable substrates were used as indicators of traction forces, but quickly moved into more theoretical and non-invasive grounds either by way of simulations or by integrating image data.

(3) Recent developments in tissue microfabrication have helped reproduce in vitro some of the key characteristics of a cell's natural environment, confirming their profound impact on cell behavior (including differentiation) and motility. An investment in image analysis is crucial to keep up with the emerging 3D image acquisition systems required to capture these more and more complex sceneries.

BIA techniques have evolved in parallel to these three approaches in order to tackle increasingly challenging image data by exploiting breakthroughs in algorithmic design and computing power. In this review, we summarize the evolution of the most common methods dedicated to studying cell motility (namely segmentation, tracking, feature extraction, and mechano-imaging methods) and point to related software resources. We will highlight that image quantification does not only bring logical rigor, but that it is itself a driver of discovery, notably when complexity increases. The aim is to unravel how BIA turns image data into biological insight to promote a critical eye for why and what to quantify.

Diverse Cell Motion Requires Diverse Image Quantification Strategies: A Few Motivating Examples

Keeping in movement is a relentless task for a cell. While a single stroke of a whale's fluke is enough to coast it for meters ahead, inertia plays a drastically different role at the cell scale. In fact, inertia is irrelevant: relatively small size and speed and high kinematic viscosity all add up to favor viscous forces over inertia∗ (definitions for terms with an asterisk can be found in Box 1). This means that past forces matter little, or that a cell needs to exert forces constantly in order to make its way through the surrounding media. To this end, other than swimming flagellates, nearly all migrating cells rely on the forces generated by their actomyosin cytoskeletons. This power can be harnessed in diverse ways, but one expects them all to induce cell shape changes in a polarized and cyclic way. First, most moving cells present a marked directional arrangement with two poles: the cell front, characterized by intense actin polymerization renewal and adhesion foci; and the cell rear, where actin is more stable, adhesions disassemble, and myosin gathers to generate contraction. Second, most moving cells also have three iterant phases in common: the cell makes a protrusion, it interacts with its surroundings (most likely through adhesions), and then it further translocates its center of mass while retracting its rear. The precise molecular repertoire of a cell, either as present constitutively or activated as a response to environmental cues, determines the mode of cell migration., In the literature, single-cell migration is loosely classified in a range between two diametric modes: mesenchymal and ameboid.,

Reynolds number: The three parameters diameter, speed and kinematic viscosity can be combined into Reynolds’ number (Re=d s/v) to characterize the relation between inertia and viscous forces. Cells live in low Re, where viscosity dominates over inertia.

Representation: (or feature extraction) reformulating the object of study in a way that highlights the characteristics most relevant to the problem at hand (selectivity), but is blind to those that are irrelevant (invariance). See an example in a change-of-basis.

Classification: finding a set of rules to categorize objects into sub-populations given their representation.

Change of basis: a mathematical concept whereby a given vector (e.g. (1,1)) that is usually represented as a sum of elementary vectors (basis), 1∗(1,0) + 1∗ (0,1), is rewritten as a sum of other elementary vectors (the other basis) 1∗(1,1) + 0∗(−1,1) that makes either the representation or the interpretation easier, but still describes the vector unequivocally and without redundance (i.e. as a bijection). For example, separating two red and blue objects of similar intensities is easier in RGB than in HSB. Another: imperial vs metric. Many mathematical problems such as classification become much easier when a good basis is chosen.

Brownian motion: a continuous, stochastic type of motion whose incremental movements are completely independent of any previous movement and are normally distributed; for example, that of an immersed particle as it is hit by the thermally agitated molecules of the surrounding liquid. It is the scaling limit of a discrete random walk.

Inverse problem: in a forward problem the cause is known and the effects have to be calculated, whereas an inverse problem consists in guessing the cause from the effects. The latter is more complicated because many causes can lead to the same effects and thus some additional a priori information is required. Where on earth did that earthquake come from? Given an epicenter it is much easier to compute the consequences of the tremor on the surface (forward) than it is to find out the source given the readings of a couple seismographs (inverse). What the hell is the earth’s core made of? Again, simulating wave propagation given the properties of the mantle is much better posed than figuring out the composition of the crust by studying how seismic waves propagate. Indeed, many combinations of material layers are possible given a limited set of data (the data might come from earthquakes themselves, which fortunately do not occur all that often).

In mesenchymal cells, the actin-rich cytoskeleton exerts constant forces by physically pushing on the cortex with the growing barbed end of microfilaments. The Arp2/3 protein complex promotes this process by nucleating new actin filaments as branches of the existing scaffold, which, braced by microtubules, give rise to very recognizable fan-like (lamellipodia) and spike-like structures (filopodia) at the leading edge. This way of crawling requires strong focal adhesions (mediated via integrin receptors) that provide traction with the extracellular matrix (ECM) and can also propel the cell forward under retrograde flows of actin. As the accumulation of all these structures stiffens the morphology of the cell, pericellular proteolysis becomes necessary to cleave obstructing fibers, especially in 3D environments. BIA has been instrumental in building this paradigm. For instance, tractions are probed non-invasively using image-based traction force microscopy, and retrograde flows would not have been identified were it not for multi-particle tracking. Another example involves Arp2/3: assessing spatiotemporal correlations was key to understanding its role at the forefront of lamellipodia in migrating epithelial cells, and measuring the spread of segmented osteosarcoma cells has allowed to reveal its differential functions in 2D and 3D environments.

In ameboid migration, actin dynamics delivers in a different way: adhesions are weaker and turn over faster; actomyosin contractility is enhanced, creating localized tensions and, when properly synchronized, whole-cell hydrostatic pressure gradients. In this way, cells become much more deformable and the need of actively lysing the ECM is practically eliminated in favor of squeezing their way through the interstices of the matrix. In this case, the size of the nucleus becomes a limiting factor that has been studied using segmentation, and deformation models., Although actin polymerization-driven gliding and dendrites are possible propulsion mechanisms, so-called blebs are most emblematic in ameboid cells. A bleb is a hemispheroidal protrusion that results from an expansion of the plasma membrane as a cellular hernia is filled by pressure-driven cytoplasm; a process so far only measured by image-based inverse problems. The initial breach opens when the cortical cytoskeleton breaks or detaches from the membrane as a result of weakened adhesions or stronger tensions. Once the pressure stabilizes, the actin cortex repolymerizes and the bleb normally becomes a pseudopod that will guide the cell onward.

The ameboid and mesenchymal migration modes are neither mutually exclusive nor necessarily discrete. Some cells can switch behavior in response to diverse factors such as the ECM architecture (e.g., density, stiffness, dimension, molecular composition). In fact, the ability of cancer cells to alternate between migration strategies during metastasis has encouraged motility studies, especially on the ameboid mode because it had been less investigated. The transition between modes has been described as a quasi-continuum of states laying in between these two apical modes, where some not-as-canonical hybrids such as lobopodial migration are especially interesting. Cell segmentation, together with shape and motion descriptors, have recently started being used to classify and rediscover migration modes with the help of unbiased statistical analyses.

BIA not only helps the community understand how cells move, but also why they move. Most cells can move randomly or in a specific direction, normally following physical or chemical cues, such as stiffness (duro-) or molecular (chemotaxis) gradients. However, the distinction between these incentives is sometimes not straightforward, partly due to the dominance of viscosity over inertia: another consequence of the low Re∗ regime is that cells drag a good part of their surrounding media with them. Therefore, short displacements, even when random, are little use to feed: cells have to move a certain distance before they can outrun nutrient diffusion, turning purposely random trajectories into zigzags. In fact, cells remain considerably persistent during non-chemotactic migration, possibly following intermittent optimal environment-searching strategies such as Lévy or gamma walks that combine local probing with long-range exploration and can be captured using cell tracking.

Entamoeba Histolytica to Exemplify BIA Techniques

We will illustrate the progressive repercussions of BIA on migration research with a case study: the amebic parasite Entamoeba histolytica. The reasons are two-fold. First, relevance: E. histolytica is the causative agent of human amebiasis, an enteropathic disease affecting millions worldwide and whose virulence is highly influenced by the parasite's motility., Second, quintessence: this ancient protozoan is an elementary example of migration as it only mobilizes highly conserved basic cytoskeletal elements where actin is central. However, the potential of E. histolytica as a migration model is limited from a biological standpoint because gene disruption is not yet available via homologous recombination.

Conveniently, E. histolytica is understood to stay mostly within the ameboid range of cell migration. To invade human tissue such as the intestine or the liver, E. histolytica needs to traverse mucus, epithelia, connective tissue, and eventually blood. A combination of in vitro (glass) and in vivo (colon explants or liver tissue) studies have shown that myosin contractility and transient adhesions are tuned according to the stage of infection. An initial exploratory stage is characterized by pseudorandom blebbing;, however, triggered by chemoattractant molecules such as ECM-derived proteins or proinflammatory cytokines,, the ameba soon polarizes, forming a pseudopod at the cell front and a uropod at the rear. Under this form, the parasite can penetrate into the colonic tissue using the inflammatory response of the host as positive feedback and hijacking human matrix-degrading metalloproteinases to ease their way in. This eagerness for secreted immune factors such as tumor necrosis factor (TNF) is well backed up by an effective phagocytic system and a solid evasion mechanism whereby the immunoglobulins that identify the ameba as a pathogen are packed into the uropod and ejected to (presumably) misdirect the immune response.

This description of the migratory behavior of E. histolytica has been built in intimate interaction with BIA and thus will serve as a unifying plot to guide the reader.

Detecting, Characterizing, and Following Cells in Microscopy Imaging

In its full breadth, BIA is a big-data problem that combines multiple abstraction layers to translate pixels into biological classification, be it according to shape, motility, or biophysics (Figure 1). Precisely, the concatenation of different methods into a pipeline enables high data throughputs and the analysis of complex multi-factorial interactions therein. For instance, the combination of automatic time-lapse imaging, cell segmentation, morphological descriptors, and statistical classification allowed identifying the human genes most relevant to cell division and motility by automatically screening the phenotypes of around 20 million HeLa cells. Mouse embryogenesis has been approached similarly, by converting long-term fluorescence imaging into tens of thousands of 4D multi-scale cellular trajectories that draw maps of cell fate and tissue morphodynamics during development. Well aware of this potential, the BIA community has organized into several software platforms from where (semi-)automatic workflows are built by connecting methods without compatibility issues (Table 1).

Figure 1

Classic Schematic of a Bioimaging Workflow

One or more cell conditions (or strains) are to be compared. Once the image data have been acquired, the first step is pre-processing (e.g., deconvolution or registration). Next, the cells can be segmented out using numerous methods; e.g., pixel based (here H-clustering) or contour based (here active contours) (images are 45 μm high). With this information, the morphology can be analyzed using descriptors or basis decomposition. The resulting shape description is used to differentiate cell populations (classify) and thus to identify and predict conditions, or sometimes can be revealed as a causal agent. If time information is available, the segmented cells can be tracked (image is 122 μm high). The time trajectories can explain the reason a cell is moving based on statistical measures: randomly, with a clear direction, or interacting with other cells. Biophysical measurements such as force estimates also constitute a great source of information, reporting directly on cell forces. Omics data can also be integrated in the analysis. All this input can be combined to uncover any correlations and (hopefully) any causalities, resulting in biological discoveries.

Table 1

Open Software Resources for BIA

	Segmentation							Tracking		Morphology		Biophysics
	Threshold	Cluster	Watershed	Wavelet	Variational	IML	DL	Manual/Auto	Motion Field	Descriptors	Base	Passive
ImageJ	Auto-Threshold	k-means	Classic watershed	Spot Caliper	Level Sets	Weka	DeepImageJ	TrackMate	Kymo/Proj/PIV	Shape descriptor	Fourier	TFM/FRET
Cell Profiler	Identify Primary/Secondary Objects				Active Contour	via Ilastik	ClassifyPixels-Unet	TrackObjects	Projection	ObjectSizeShape
Icy	Best Threshold	HK-Means	Watershed 3D	Spot Detector	Active Contours		DeepClas4Bio	SpotTrackingTrackManager	Kymo/Proj/PIV	ROI statistics	SH	BioFlow

Implementation of different BIA techniques in the three most popular BIA free open-source software platforms: ImageJ/Fiji, Cell Profiler, and Icy. The names listed refer to popular plug-ins in each category; this list is not exhaustive. ImageJ is oriented to single-image analysis with the biggest community and a wide range of tools; Cell Profiler is intended for batch processing and thus has fewer (but more curated) modules. Icy is the youngest of the three and therefore it has had the chance to incorporate a bit of both philosophies; it has less support but provides state-of-the art algorithms. There exist many other platforms, as well as stand-alone software packages; installation is typically easy so we recommend shopping around before committing.

While pipelining is predominantly a question of software, the choice of each individual method in the workflow is obscured by subtle assumptions underlying the corresponding algorithms. From the perspective of data science, most algorithms presented here are twofold: represent∗ and then classify∗; both parts introduce assumptions. Being more selective in the representation dilemma allows for simpler classifiers and results in methods with fewer adjustable parameters, but at the same time incorporates bias, limiting the applicability of the method. Images themselves are a good example: they can be represented in a variety of ways, enforcing different invariances; for instance, wavelets can capture the multiple scales within an image, making denoising easier. Other examples arise in methods such as segmentation or shape analysis. We show how each method alone can extract information from biological images, how different assumptions branch these methods out into as many niche algorithms, and how their combination can tackle big-data problems, completing the full cycle.

Cell Segmentation

Segmentation is a cornerstone of BIA; it is the initial step of many other image analysis techniques, such as cell shape description, cell tracking, or measuring certain biophysical quantities (Figure 1). The aim of cell segmentation is to partition an image into individual cells and background, or, put more abstractly, to group pixels that share some common characteristic.61, 62, 63 To this end, most segmentation methods in biology use either the spatial or frequency distribution of the image brightness (Figure 2).

Figure 2

Segmentation Methods from a Represent-and-Classify Perspective

(Column 1) Thresholding separates this epifluorescence image of HeLa cells into cells (without distinction) versus background. There is no need for an alternative representation because the intensity already characterizes the cells, as seen in the histogram. The simple threshold classifier takes one parameter when set manually, or zero if set with a thresholding algorithm. (Column 2) In this confocal image of mouse embryonic stem cells, the intensity differences are not so clear-cut. To identify the two cells in the image, we use a more complicated classifier (k-means) on the original representation. The mix of location and intensity data is enough to divide the pixels into three groups belonging to two different cells and the background; notice how the cell hole is also associated to the cell. (Column 3)Texture analysis aims at finding a measure of intensity patterns by considering groups of nearby pixels. In this brightfield image of E. histolytica, standard deviation (SD) is enough to characterize the inside of the ameba with respect to its environment. While the SD filter takes one parameter to specify the size of the neighborhood, the representation is good enough that one only needs a simpler classifier, such as a thresholder. (Column 4) Using the wavelet transform helps represent the original image in a basis that reflects a mix of spatial scales and frequency. Since the breast cancer cells in the image are all salient and of approximately the same size, the wavelet coefficients at the chosen scale (one parameter) can identify them. Conversely, the middle structure (smaller) and bottom (bigger) are fainter; they are present in other scales. In this representation, the classification can be as simple as using a threshold (one or zero parameters). (Column 5) IML combines several representations (or filters, or features) and learns how to classify from a set of training examples. This is advantageous when there is not a single feature that characterizes the cell; for example, these bacterial colonies in phase contrast respond to a combination of intensity, edge detection, and texture. (Column 6) Deep learning (DL) learns its own feature extraction, in addition to the classification. This involves adjusting many more parameters with respect to IML, which is done automatically by feeding the algorithm with many examples of segmented cells. This approach can tackle “harder” problems like the touching E. histolytica cells in this faint brightfield image. The adoption of more general methods often requires higher computational power. While the difference across the first columns is hardly perceptible inside a modern computer, IML and DL are more resource hungry in their quest to adjust to the training set. In particular, the huge number of parameters required to learn both representation and classification in DL calls for much bigger training sets and for much higher processing power, which is typically outsourced to graphical processing unit (GPU) farms in external servers. Conversely, if the images are suited for the assumptions made by a simpler method, a change in representation can be all it takes. Raw image data in columns 1 and 2 (threshold and cluster) were taken from the database in Maška et al. Images in rows 3 and 6 were acquired in-house. Images in rows 4 and 5 are from collaborators (see acknowledgments).

The most naive approach to cell segmentation is thresholding the image brightness; i.e., directly associating dark and bright pixels to the background and the cells (Figure 2; Table 1). The brightness value that separates these two categories is best chosen diligently; for example, by bisecting the frequency histogram according to its variance or its shape. However, this only works when cells are homogeneous and of similar brightness, and when image noise is low.

More realistically, the brightness values have to be divided into, not two, but multiple clusters corresponding to cells in different shades of gray, which ideally show up as distinct peaks in the histogram, (Figure 2; Table 1). Under this type of clustering algorithms, additional quantitative measures such as color, position (i.e., spatial distribution), or texture can also be considered alongside the brightness to define an alternative “ruler” that further evidences the distance between the clusters of pixels belonging to each cell.

Otherwise, if none of these quantities seem homogeneous within a given cell, it is advisable to focus instead on the cell contours. Accordingly, edge detection techniques such as Sobel, Laplacian, or steerable filters exploit the spatial derivatives of the image to look for significant changes in brightness, which are expected to correlate with a change of physical milieu.

Cell-cell interaction is another important challenge to segmentation. It is hard to assign a pixel to either of two cells in contact when their physiognomy is similar. Watershed methods look for the boundaries between adjacent cells in the ridgelines of the brightness mountains that cells feature over the darker background (Table 1)., Alternatively, variational methods allow readily combining information from the image (similarly to the methods above, e.g., uniform/non-uniform brightness separation or edge detection) with a priori knowledge (which can include constraints on cell shape and contour intersections) by customizing a so-called energy that is to be minimized. This flexibility comes at the price of additional burden: they require a “seed” (an initial prospective contour for each cell), as well as more complex numerical techniques. For example, active contours (or level sets) algorithms gradually deform the initial seed contour by exerting localized forces according to the criteria and constraints formulated in the energy. Their evolution finishes when the forces equilibrate and the curve is (ideally) fitted around each individual cell with no overlaps, resulting in a very accurate representation of the cell shape (Figure 1; Table 1). At the tissue scale, where cells migrate in crowds, graph-based approaches discriminate membrane-labeled tissue cells by building networks that connect the walls like a honeycomb.

Segmentation is resolved likewise at the intracellular scale and can be complemented with statistical colocalization methods that examine a protein's location relative to the also-segmented cell (e.g., myosin is found at the back of uroid-capping trophozoites), or to other proteic structures, (e.g., Arp2/3 colocalizes with phagocytic macropinosomes) in order to investigate potential molecular interactions.

We have found active contours the most convenient when describing the rapidly varying shape of ameboid cells during migration, but the approach is only optimal for fluorescence microscopy images; for example, of cytoplasmic or membrane labels. Opportunely, texture filters like SD translate texture, into intensity and make phase-contrast or brightfield acquisitions more amenable to this fluoro-friendly technique (Figure 2). Conversely, when the aim is to address the interactions within a sparse (but large) population of amebas, the parasites are imaged small enough to apply spot-detecting techniques such as multi-scale wavelets (Figure 2; Table 1).

In conclusion, cell segmentation is addressed on an ad hoc basis. Each acquisition technique and each tag has its exemplar image, but none of the features that set apart cell from backdrop seem to be general enough: there is a semantic gap between human heuristics and algorithms. This has given rise to many situational approaches that manage as many different assumptions; for example, leveraging the motion of the cell through matrix decomposition or imposing predetermined cell shapes (e.g., rods for bacilli). To make segmentation more automatic, a common solution is to combine several approaches. A rough seed can be obtained by one of the simpler thresholding methods and later refined into a smoother segmentation by a variational contour-based method. This strategy is especially fruitful if multiple color channels with different labels are available. For instance, the nucleus channel is usually specific enough to serve as a seed, while the membrane or cytoplasmic signal can drive the deforming contour from this seed to the cell boundary.

Admittedly, our ability to redesign the experiment in anticipation of image processing might be limited by practical constraints: some markers are costly or simply incompatible with other indispensable protein labels, for example in terms of emission wavelength; one has to conform to the biological question. Say calcium imaging or STAT3-GFP, both are essential to the analysis of cell migration but present several singularities that complicate cell segmentation. For instance, calcium suffers from big intensity fluctuations that are tackled by considering an entire time sequence instead of a single image; in this way, the video (a 2DxT matrix) can be decomposed into background and cells, for example via non-negative matrix factorization (NMF). On the other hand, although STAT3-GFP promptly relocalizes to the cell nucleus, morphological operations such as dilation can be used to sample the surrounding cytoplasm.

Label-free imaging such as brightfield or phase-contrast microscopy is even less invasive but introduces halo-like artifacts that hamper segmentation; in fact, no feature by itself seems characteristic enough of a cell observed under transmitted light. Challenges like this have popularized interactive machine learning (IML) algorithms (Figure 2; Table 1), which use non-linear classifiers (e.g., random forest) to select the best combination of features from a pre-set collection of filters (see above: e.g., edge, texture) by training on user-annotated data., On the other hand, rather than working with a predetermined set of features like IML, deep learning and artificial neural networks (ANNs) tailor their own filters to the training set from a rather general template that includes a range of non-linear mappings; that is, they look for good features automatically, but, in exchange, the underlying assumptions are practically inaccessible. In this way, the ad hoc customization necessary to segmentation algorithms can be fully delegated to the network if the results are “known.” Because they learn both representation and classification at a time, ANNs need bigger training sets that reflect the full heterogeneity of the data. The resulting demand in computational resources has kept widespread use at bay for a long time, but end-user solutions are starting to appear in bioimaging platforms (Figure 2; Table 1). Indeed, ANNs are now blooming in image segmentation; they are promising because they can be extrapolated almost exhaustively to all different imaging modalities and conditions, but they need to be trained case by case in a (progressively less) laborious process that assumes collected data can be correctly annotated and are representative of future data., This effort has been especially rewarding to traditionally hard problems, notably to the segmentation of brightfield images during cell-cell contacts.

Cell Shape Description

Morphology has historically been regarded as a purely predictive marker of biological response (think of cell fate or differentiation), but it has been revisited as part of a complex feedback loop that integrates mechanical and chemical signals. On one hand, changes in cell shape overlie the constant reorganization of the actin cytoskeleton as it adapts to specific whole-cell functions such as division in response to intracellular (IC) or extracellular (EC) cues. On the other, cell morphology might repercuss in signaling, and have other consequences stemming purely from geometric considerations.

In computers, segmented cells are typically represented as groups of pixels or as vertices of a contour. Even if visually appealing, the numerical size and conceptual complexity of these representations complicate the quantification of cell morphology. To establish comparisons between large populations of cells, it is recommended to simplify the description of cell boundaries using descriptors or a change of basis∗ (Figure 1; Table 1).

Morphology descriptors quantify potentially determinant shape traits such as cell size or roundness. The choice of descriptors can be purely heuristic (that is, by observing the cells), or educated with the help of data analysis techniques such as dimensionality reduction algorithms (e.g., NMF, principal component analysis [PCA], t-distributed stochastic neighbor embedding [t-SNE], or uniform manifold approximation and projection [UMAP]), which simplify visualization on the basis of statistical or topological principles., An extensive list of descriptors is available,, starting from very simple features (e.g., area, elongation) and covering a broad range of applications; for example, the ramification factor and branching points are adapted to approximately count filopodia and neuron dendrites, whereas principal axes have been used to quantify the relative orientation of dividing neuroblasts and growing bacteria.

However, shape changes can be too subtle to summarize in a couple of geometric measures. When descriptors become too simplistic or too manual, a change of basis is a good compromise between complexity and interpretability. Just like a color can be decomposed in a basis of red + green + blue or hue + saturation + brightness components, shapes can be mathematically described as a sum of deviations from the circle (2D) or the sphere (3D) (Figure 1). These deviations can be expressed in different bases that ease different interpretations of the boundaries. For example, Fourier series (2D) and spherical harmonics (SH; 3D) are better at capturing global cell shape because the periodic basis functions are rotationally invariant, whereas wavelets (2D) and spherical wavelets (SW; 3D) are best at localizing surface protrusions because of their smaller function support. Alternatively, decompositions can be applied on the motion of the cell contours to tackle shape changes directly; e.g., the Hilbert-Huang transform has been used to relate local movement profiles with different subcellular signaling regimes in fibroblasts, while the transported square-root vector field representation decomposes the evolution of the entire cell contour.

In the study of E. histolytica, we have used SW to automatically screen the position and number of blebs, a strong physiological indicator and a key driver of ameboid migration. Conversely, SH have allowed recognizing different migration patterns in the temporal deformation profile of the trophozoites, as well as inspecting whole-cell shape differences between populations; for example, when the adhesive properties of the parasite are reduced by signaling blockage of the Gal/GalNac lectin protein complex, the cells shrink and their surface coarsens/wrinkles significantly (high-frequency SH components are depleted/enhanced), which is consistent with membrane tension considerations. This strain can still navigate the ECM but can no longer cross the hepatic barrier,, delaying the inflammatory response.

Unfortunately, even when complexity is reduced through descriptors or changes of basis, it is common that results still lack human-ready interpretability. Unsupervised data analysis techniques can exploit the now-simpler representations to classify and discriminate populations (e.g., according to subtle migration changes) as well as to answer questions reflecting on the biological relevance of experimental conditions, but they cannot always give a user-friendly picture of the combination of features (say, SH components) that make the difference. For example, t-SNE and UMAP exploit non-linear dimensionality reduction in an attempt to reveal local data structures while preserving global-scale information;, the resulting manifolds facilitate cell type phenotyping via standard clustering methods in high-dimensional sets, but their focus on pairwise distances obscures any connection to the original features as compared with more classic linear approaches such as PCA or NMF. To improve the interpretability of the results, we expect future work to lean toward differential geometry because it is a natural and more illustrative way of representing manifolds such as the (2D) surface of a cell bending in (3D) space., Meanwhile, some recent efforts take a shortcut and directly train supervised machine learning algorithms using human input on simplified image data. This is only effective if the end result is tangible; for example, to detect and classify cell protrusions. In this way, it has been shown that some proteins in melanoma cells are differentially modulated by certain morphological motifs such as blebs or filopodia.

Cell Tracking and Velocimetry

Diffusive, confined, intermittent, and directed motions are four of many patterns of cell movement that can be inferred from the stochastic analysis of cell trajectories (Figure 1). Purely diffusive∗ motions are hardly expected at the scale of the cell because Re∗ is low, cells are self-propelled, and molecular polarization carries some inertia. Instead, cells display different degrees of persistence that reflect why or how they are moving.

Computationally, the problem of cell tracking is one of optimal mapping: already-segmented cells in consecutive frames are paired over time in order to build trajectories according to some criteria. Nearest neighbors—a criterion assigning a cell's center to the closest center in the next frame—is an effective choice when the temporal resolution is high relative to the number of cells in the field of view. This is the case with most essays in morphodynamics. At a lower spatiotemporal resolution, where cell and particle tracking are practically equivalent, there is a need for better-educated criteria that do not only privilege short distances but rather consider multiple hypotheses given the time course of the particles and some statistical priors., A collection of methods designed to deal with cluttered environments in which particles (here cells) “jump” across each other can be found in Chenouard et al. (Table 1). Once the cell trajectories are known, a range of statistical techniques are used to classify the observed motion into one of the many patterns of cell movement. For example, the mean squared displacement (MSD) has traditionally been used to discriminate between different diffusivity regimes because it measures the progressive deviation of the cell with respect to its initial position; it is typically reported by comparison with diffusive motion.

According to the MSD, the directionality of E. histolytica increases once in the liver, but randomly migrating trophozoites are still super-diffusive in the absence of stimuli. Correlative studies have shown that E. histolytica alternates between low-persistence exploration and a highly polarized and invasive mode induced by signals from the intestinal environment that include TNF, erythrocyte factors, bacterial lysates, fibronectin residues,, and, perhaps most interestingly, self-secreted proteins. The underlying motility switches involve the activation of different molecular pathways that regulate the cytoskeleton in response to external stimuli.

More recent statistical tests are able to associate a significance (p value) to the classification of cell trajectories, detect pattern changes within the same track, and account for any surface manifold to which the cells might be constrained within a 3D environment. Further classification of the tracks, as well as their relation with the different migration modes, can be approached with methods similar to those presented for cell shape: descriptors (e.g., net versus total displacement), changes of basis, machine learning, manifolds, etc.

The “social” behavior of cells is written down on their tracks too, and could be studied as is done in mice and flies. For example, cell-cell contact is a means of information exchange in multiple biological processes, including the immune response, wherein T cells need to recognize surface antigens. Engineered molecular labels that are enzymatically transferred upon contact have been used to monitor the dynamics of these so-called kiss-and-run interactions, but cell tracking could help unveil any underlying interaction network. Another remarkable instance of multicellular tracking concerns tracing cell lineages through development; for example, to study embryo morphogenesis (discussed later).

Yet another by-product of tracking is cell speed, which frequently doubles as a measure of motility; for example, to quantify the invasiveness of E. histolytica in biological environments such as (enterocytic-like) Caco-2 cell monolayers or hamster livers. Intracellular velocity fields, such as those describing the movement of the cytoplasm, are also interesting (Figure 3E.i; Videos S1, S2, and S3, Table 1); e.g., to study how active molecular transport administers the distribution of proteins when diffusion is not sufficient. The classic methods to compute intracellular speed are two: kymographs, which concentrate on the time evolution of a single cross-section of the image; and projections, where the brightness values across an image stack are cast onto a single image. Since these approaches discard plenty of information, nowadays image analysts resort instead to a combination of speckle microscopy and multiple-particle tracking, or to particle image velocimetry (PIV) techniques, which can pick up on the movement of pixel intensity directly on standard fluorescence microscopy by following pixels of constant intensity (optical flow) or by maximizing the correlation of successive image patches. The resulting intracellular velocity fields have been dense enough (Figure 3E.i) to show that bleb formation results primarily from a redistribution of cytoplasm in zebrafish primordial germ cells (Video S2), to determine the effect of cortical contractions on streaming in oocytes, and to allow tracking the movement of molecular regions that are too diffuse to be segmented, (Videos S3).

Figure 3

Classification of Methods to Measure Biophysical Quantities

Active:

(A) Atomic force microscopy to measure force or elasticity.

(B) Servo-null micropipette to measure pressure.

(C) Optical trap to measure forces or probe properties (e.g., rheology).

Passive: most passive methods require image analysis to some extent.

(D) TFM measures forces exerted on the substrate.

(E) Measurements of cytoplasmic streaming. (E.i) Left to right: IC velocity field (0.2–6.1 μm/s), IC velocity field corrected for bulk movement (0–4.8 μm/s), and IC pressure (0–6.7 Pa) in a migrating E. histolytica trophozoite imaged in two-dimensions with a confocal microscope. The images are 25 μm high. Lower values are in blue, higher in red. (E.ii) Arrows represent IC velocity, stream lines show cytoplasmic streaming, and surface colors display pressure. Migrating E. histolytica trophozoite imaged in three-dimensions with a confocal microscope.

(F) FRET or photo-quenching (see text).

While the first three methods require interacting actively with the cell (and possibly disturbing it), the last three use passive reporters that are captured by non-invasive imaging techniques.

Video S1. 3D Cytoplasmic Flow in E. histolytica In Vitro

The velocity arrows (blue low to red high) between two frames (gray/white meshes) shows a cytoplasmic redistribution toward the bleb, which is visualized with the help of streamlines. See Acknowledgments for original image data.

Videos S2. 3D Cytoplasmic Flow Inside a Zebrafish Primordial Germ Cell In Vivo

The velocity arrows (blue low to red high) shows the cytoplasmic streaming within the moving cell represented by the mesh. See acknowledgments for original image data.

Videos S3. 2D Movement of a Diffuse Protein Structure within a Zebrafish Primordial Germ Cell In Vivo

The trail arrows (color meaningless) show the trajectory followed by the diffuse actin structure within the moving cell in black. See acknowledgments for original image data.

Biophysical Measurements Using Image Data

While century-old studies had anticipated the importance of physical forces in shaping and modeling the organism, only the relatively recent development of force-probing techniques has shown their far-reaching implications in morphogenesis, differentiation, and cell migration. It is now established that physical forces can propagate information within (and between) cells, elicit mechanical and biochemical signals down to the metabolic, and transcriptional pathways, and regulate cell migration. Indeed, the mechanisms underlying cell shape and motility involve complex molecular machinery that senses and actuates both mechanical and chemical signals (internal and external), including the generation of endogenous forces by the contractile actomyosin network. Even though this ensemble of molecular motors contracts locally and independently, by exploiting the biophysical properties of the cell, their proper coordination translates local mechanical tension into whole-cell motion and, eventually, into global cell migration. Therefore, deciphering how cells deform and move requires a better understanding of the biophysical quantities that do not only drive but also reflect IC/EC dynamics, such as IC/EC forces and IC pressure (Figure 1). Unfortunately, many such measurements cannot be taken directly, especially at the IC level. Instead, current experimental methodology is either active or passive (Figure 3).

Some methods are classified as active because they apply exogenous forces (Figure 3, top). For example, using micropipette aspiration or atomic force microscopy to probe cortex tension or elasticity (Figure 3A), microchip injection or servo-null micropipette penetration to measure local IC pressure (Figure 3B), or magnetic and optical tweezers to estimate molecular-level IC forces (Figure 3C). As regards E. histolytica, micropipette aspiration has been used to study the role of the membrane-cortex pair during bleb-based motility. Magnetic tweezers have also been quite versatile. They have been used to study the response of the parasite to mechanical stimuli, whereby forces exerted on the cell rear are transduced and amplified into a biochemical signal that enhances the polarization of the cell, and also to determine the rheological properties of the amebic cytoplasm.

BIA has contributed methods that are more passive (Figure 3, bottom; Table 1) and less harmful to cells. These include any combination of image-based data extraction such as PIV with posterior simulations of theoretical models,, for example to study cytoplasmic streaming in early animal development; or with inverse problems∗, for instance traction force microscopy (TFM) estimates EC forces by watching cells interact with deformable substrates that have been filled with marked beads (Figure 3D). If the constitutive relation of the substrate material is known, the traction forces can then be “inverted” from the deformation observed under a fluorescent microscope. In a similar way, the study in Boquet-Pujadas et al. estimates several IC quantities inside migrating amebae by imaging the movement of their cytoplasm with standard fluorescence microscopy (Figure 3E) and modeling it as a viscous fluid as previously probed using magnetic rheology. By using a data assimilation framework to couple PIV techniques with physical models, this approach not only quantifies IC forces, stresses, and pressure gradients but also yields velocity estimates at unprecedented detail (Figure 3E.i and ii and Videos S1, S2, and S3). These measurements allowed corroborating experimentally some long-established hypotheses regarding the physical mechanisms driving ameboid migration in E. histolytica and unveiled a coupled periodicity underlying the parasite's protrusions.

Although the majority of passive methods described so far report macroscopic measurements, the different mechanisms of mechano-transduction and -sensation span several scales; accordingly, the measuring range of the techniques stretches from the tissue to the protein levels. At the tissue scale, techniques include micro-droplet embeddings and monolayer stress microscopy, while gauges at the protein level typically resort to genetic engineering. For example, biosensors based on fluorescence resonance energy transfer (FRET) or photo-quenching are conceived to shine in response to tension across a protein of interest (Figure 3F), resolving magnitudes down to the pN. However, their exact range of application is still controversial and the calibration process remains very delicate: it requires, first, controls for environmental independence and, later, extensive image analysis.

We remark that models are not a requirement exclusive to passive methods, and physical assumptions (not biological) underlie both active and passive approaches because forces can only be accessed indirectly (Newton's second law) through their effect on other quantities; most commonly, the deformation of well-described materials.

Even though we have introduced movement at the single-cell scale, let us point out that a good part of cells is gregarious. Collective cell migration and rearrangement are at the heart of fundamental biological processes, including morphogenesis and metastasis. However, the emerging mesoscale properties of collective cell organization, as seen for example in epithelial sheets, cannot necessarily be predicted efficiently from the behavior of individual cells. For instance, the fluid-to-solid-like phase transitions that have been characterized in cell monolayers can be inferred from a small set of physical properties such as cell shape and cell alignment. In the likes of schools or flocks, it is the local interaction between neighboring cells that creates this global cell behavior. Context is important.

Microenvironments, a Lens to the Future of BIA

Historically, the lack of technological solutions at the micro scale has found good rationale in reductionism to justify 2D (or glass) experiments. However conducive this bottom-up avenue has been to singling out the key players in morphodynamics, it has also demonstrated that cell behavior in vivo should not be (carelessly) extrapolated from studies performed outside the native context of the cell; for instance, new migration strategies are discovered as cells face increasingly complicated obstacles. Indeed, cells consider the most variate environmental stimuli. They take into account the composition and stiffness of the ECM; physical forces (e.g., shear flows and peristaltic compressions), which have been shown to influence differentiation and infection;, biochemical cues such as cytokines; and multicellular factors (e.g., E. histolytica interacts with the immune system and with the commensal microbiota in the colon). Consequently, to facilitate experimental design while retaining physiological relevance, great effort has been invested in engineering 3D microenvironments that integrate a select (e.g., in balance with the cost) subset of variables relevant to human pathologies.

The quest for increasingly relevant setups has put optical microscopy to a test: the resulting multicellular 3D structures require higher penetration depth and multiple wavelengths, and, if the system is to be imaged live, faster acquisition and reduced phototoxicity; even at the single-cell scale, a boost in speed is necessary to capture the molecular mechanisms behind cell protrusions, which occur at the 1 μm3 s scale. Lattice light-sheet microscopy (LLSM) has delivered and it is now open season for BIA. In its relatively short life, LLSM has already been key to elucidating cell fate during embryogenesis; for example, to understand how oriented cell division and migration shapes the early nervous system or how the spatial modulation of cell proliferation is necessary to limb formation. Instead of resorting to genetic markers, by exploiting the spatiotemporal resolution of LLSM, cells can be automatically followed all the way from gastrulation to the onset of organogenesis through cell division, differentiation, migration, and apoptosis in order to classify and map their fates. Although big-data management plays a big role in these projects, robust segmentation and a well-tailored set of mapping rules within the tracking algorithm are fundamental because small mistakes are quickly amplified through the lineage tree. For this reason, image analysts rely on highly specific nuclear labels and combine trained ANNs with wavelet representations to ensure cell divisions are properly detected., LLSM is but one example of an imaging revolution that is generating increasingly complex datasets. Under the new paradigm, biologists have to step out of their comfort zone: manual quantification is far-fetched and automatic image analysis becomes nigh imperative.

With the help of BIA, we have seen that the ameboid mode of E. histolytica has certain nuances that support the parasites' ability to penetrate complex environments such as intestinal or hepatic tissue. First, the migratory behavior of E. histolytica switches from exploratory to invasive upon detection of intestinal signals. Second, although this cell deforms considerably and its nucleus is particularly small, proteolytic activity appears necessary to invade the deep layers of the intestine. While maintaining ameboid motility, E. histolytica takes advantage of human cells because proteolysis results only from the combined activity of parasitic and human proteinases when faced with dense ECMs. To reproduce intestine-like conditions in a controlled environment, potential models have to mimic not only intercellular relations but also its crypt-villi topography and physical peristalsis., Applying BIA to these active 3D tissue models will be a requirement if we are to decipher how the parasite's movements occur in deep tissue layers and eventually understand the molecular basis of the infection; for example, to reconstruct 3D live images of the infection process, to investigate the motivations behind their tracks and register their interactions, to study morphological changes and their relation with protein location and intracellular redistribution, and to consider the role of physical forces in infection both at the cellular and tissue levels (e.g., does stressed tissue facilitate infection?). To this end, and to the study of cell motility in general, we count four main future challenges for BIA.

(1) The sheer size of the new volumetric datasets precludes manual analysis, but processing the throughput of modern microscopy is no cinch either. Admittedly, rather than a big conceptual leap, dealing with the 3D nature of images is more a question of reformulating algorithms to scale better with increasing image sizes. At the same time, integrating image processing directly into a smarter microscope might help discard irrelevant data from the get-go or adapt to multi-scale processes on-the-fly., We expect that reducing the computational burden will pave the way for a comprehensive analysis of biological processes that complements image analysis with all the omics approaches (Figure 1).

(2) Aside from economizing countless hours of labor for scientists to focus on higher-level hypotheses and complementing their intuition with advanced 3D visualization tools, quantitative BIA is to become the main vector driving biological discovery as systems grow increasingly complex and non-linear.

(3) To avoid compromising the integrity of contemporary microenvironments, biophysical gauges will have to continue their drift toward inverse problems. This is usual in medical imaging: tomographies, magnetic resonances, and elastographies are all based in the theory of inverse problems. The common theme is taking measurements indirectly. However, uncertainty is the “elephant in the room” of inverse problems in general, and of their bio-applications in particular. For example, TFM does not provide measurement errors. In the near future, any trustworthy measurement will have to take into account the uncertainty introduced by image noise, as well as the approximations inherent to the physical model.

(4) If the hope is to tip the balance from expensive laboratory equipment to algorithms, it is imperative to provide working software. Multi-purpose BIA platforms have now been offering open-source solutions for a decade or two (Table 1) and they are well prepared for the advent of 3D imaging, although not so much for that of big data. Having many algorithms within the same platform offers the opportunity to build automatic workflows (Figure 1: pre-processing, segmentation, tracking) that travel all the way from pixel information to biological discoveries.

Unfortunately, these pipelines are still not very robust, as many of the steps require extensive fine-tuning; the large collection of segmentation methods is good evidence of this. The boom of supervised deep learning, will help tackle the work-intensive fraction of BIA challenges by (quite literally) substituting human vision and memory with chips., Those include problems such as cell segmentation, where the outcome is “known” and thus the algorithms can replace hand-designed feature extraction with sufficient training. However, the same approach is unsuited for intuition-driven BIA challenges because supervised ANNs enforce preconceived ideas, missing subtle differences and eventually leading to a confirmation bias incompatible with new knowledge. Therefore, unsupervised algorithms, mathematical methods, and curated physical laws will play an increasingly large role in guiding biologists forward through the big data resulting from the quest for physiological relevance.