Multi-input convolutional network for ultrafast simulation of field evolvement.

There is a compelling need for the regression capability of mapping the initial field and applied conditions to the evolved field, e.g., given current flow field and fluid properties predicting next-step flow field. Such a capability can provide a maximum to full substitute of a physics-based model, enabling fast simulation of various field evolvements. We propose a conceptually simple, lightweight, but powerful multi-input convolutional network (ConvNet), yNet, that merges multi-input signals by manipulating high-level encodings of field/image input. yNet can significantly reduce the model size compared with its ConvNet counterpart (e.g., to only one-tenth for main architecture of 38-layer depth) and is as much as six orders of magnitude faster than a physics-based model. yNet is applied for data-driven modeling of fluid dynamics, porosity evolution in sintering, stress field development, and grain growth. It consistently shows great extrapolative prediction beyond training datasets in terms of temporal ranges, spatial domains, and geometrical shapes.

Introduction

A deep convolutional network (ConvNet) utilizes stacks of convolutional layers for automatic, hierarchical representation learning of high-dimensional data. The convolutional components of ConvNet bring great advantages when processing data in the form of multiple arrays, such as signals and sequences (1D), images or audio spectrograms (2D), and video or volumetric images (3D). Since its first remarkable application in digit recognition nearly 30 years ago, ConvNet has served as the backbone for performing various computer vision (CV) and image-centered tasks.3, 4, 5, 6 Upon realization of its distinct advantage in image modeling, there has been a surge of applications in different scientific domains where image-involved problems are ubiquitous. Some of these scientific applications include image classification with respect to scientific images,, microstructure characterization and reconstruction (MCR),9, 10, 11 and process-structure, and structure-property, relationship modeling in materials science and engineering. They all leverage ConvNets to explicitly process and “understand” scientific images free from hand-craft featurization and with minimal human intervention. For example, conventional structure-property modeling relies heavily on domain experts to design effective structure descriptors,, whereas ConvNets directly take raw structure information as input and can construct a high-fidelity relationship between image-based structure and property (Figures 1A and 1B).

Figure 1

Four different regression tasks with increasing complexity and multi-input ConvNet

(A) Parameter(s)-to-parameter(s) regression for conventional structure-property modeling, where the input and output are one or a few parameters; i.e., scalar quantities.

(B) Image-to-parameter(s) regression for high-fidelity structure-property modeling, where the input is an image and output is parameters.

(C) Image-to-image/field-to-field regression for semantic segmentation and fluid dynamics simulation, where the input and output are images.

(D) Multi-input field-to-field regression for condition-incorporated fluid dynamics simulation, where the mixed input is parameters of fluid properties and an image of the current flow field and the output is an image of the next-step flow field.

(E–G) The proposed y-shaped multi-input ConvNet for handling the multi-input field-to-field regression tasks (F) and comparison of the conventional flattening-based (E) and the proposed gating-based (G) signal merging strategy.

Another prominent application is ConvNet-based simulation of field evolvement and development. This also takes advantage of its superior image modeling capability because different types of physical fields, such as temperature, velocity, density fields, and structures described by various physical variables, can eventually be presented as images. In this regard, ConvNets treat field evolvement and development as a purely data-driven, image-to-image/field-to-field regression problem (Figure 1C). Some typical examples include prediction of transient fluid dynamics (i.e., current flow field to future flow field), microstructure evolution (i.e., current structure to future structure), and stress/strain field development (structure to stress/strain field,). ConvNets simply aim to build these field-to-field mappings from training datasets without using any physical differential equations. Therefore, the ConvNet, when trained, can be an inexpensive substitute for a physics-based model, enabling fast but realistic simulations of field evolvement and development.

However, most of the related research,21, 22, 23, 24, 25, 26, 27 just adopted the CV task-oriented ConvNets, especially those with a typical encoder-decoder architecture. Obviously, such an encoder-decoder model, originally used for semantic segmentation, is inadequate when it comes to modeling field evolvements. In semantic segmentation, the output image or segmentation result is uniquely determined by the input image to be segmented (Figure 1C). However, the evolvement and development of a physical field are usually not only decided by the initial state but also controlled by the applied conditions. For instance, a flow field developed at a future point is concerned with the current flow field as well as fluid properties (Figure 1D), and stress field development greatly depends on the condition of applied force. With a pure image-to-image regression tool, those existing research have to fix conditions, thus failing to incorporate condition-related parameters. The trained ConvNet is strictly applicable to the specific condition used for training. This is apparently contrary to the general fact of great variability in applied conditions and prevents wide use of the trained ConvNet. Developing a more general multi-input field-to-field regressor that can build a functional relationship between multi-input (i.e., input field and condition parameters) and output fields would be of huge significance. It will, for example, allow us to train one single ConvNet applicable to flow fields with distinct conditions (Figure 1D) instead of training separate ConvNets for each. Such a multi-input field-to-field/image-to-image regression capability with well-tested general usefulness is fundamentally important, as are the other three regression capabilities. It will solidly advance the border of our reachable regression tasks, as shown in Figures 1C and 1D.

We propose a multi-input deep ConvNet that naturally exhibits a signature Y shape as a multi-input field-to-field regressor (Figure 1F). We thus call it yNet, which conveys its most salient feature, efficient fusion of an additional input signal of condition-related parameters, in comparison with the pure image-to-image regression neural network (I-shaped) in the aforementioned CV and scientific applications (Figure 1C). The main architecture is basically composed of an encoder-decoder based on a deep ConvNet and a branch of multi-layer perceptron (MLP). The encoder essentially plays the role of non-linear dimension reduction. It decomposes the original high-dimensional pixel-based field into N2 information-rich n1×n2 feature maps (Figure 1G). MLP acts to expand the condition-related parameter(s) into N1 neurons that generate a 1×N1 embedding vector. In this way, the encoder and MLP would facilitate effective fusion of input signals at their ends, as described later. The decoder serves to correctly reconstruct the merged signal back to a meaningful and desired output field through the deep deconvolution process. In the deep deconvolution process, we also concatenate feature maps extracted during the early autoencoding process. This technique is commonly adopted in semantic segmentation networks, such as U-Net and FCN, with the aim of improving segmentation details. It is expected to compensate information loss induced by coarsening during max-pooling in the encoding process and improve information flow from (image) input to decoder.

After building the main architecture, merging signals of multiple inputs poses a key problem. In fact, such multiple mixed inputs (a high-dimensional image and scalar parameters) are not uncommon in various machine learning (ML) tasks, although the output is not necessarily an image. However, the merging strategies in existing multi-input ConvNets can suffer from various drawbacks associated with dense parameters. A common merging strategy30, 31, 32 is to flatten the image signal, here represented by its feature maps, into a long vector, making it compatible and, hence, concatenable to the vectorial output of MLP (Figure 1E). The flattening-based strategy is suitable and sometimes unavoidable for predicting a single label or numeric value but may not be optimal for ConvNets with an image-type output. The flattening operation would yield large, fully connected layers, as shown, that are known to be parameter intensive. The dense parameters can bring about drawbacks, including increased training complexity requiring sophisticated training schemes, more hardware resources demanded during training and inference, and slow inference speed. This is clearly shown, for example, in comparative studies between the light-weight SegNet and other semantic segmentation ConvNets.,, Apart from the parameter-efficiency issue, the flattening operation will interrupt the signal flow of 2D feature maps, which themselves are meaningful as spatial features in coarse form. The merged signal in those manners however becomes a skinny numerical vector, which is somewhat meaningless or less interpretable. After flattening-based merging, ConvNet would have to take further steps to force the long vector back to 2D useful maps for the decoding process. A flattening-based strategy, if workable for multi-input image-to-image regression,, might make the multi-input ConvNet more like a brute-force regressor.

To overcome the shortcomings of the flattening-based merging strategy, we propose to properly merge signals using a one-to-one connection via multiplication (Figure 1G), which is mathematically akin to the excitation/gating mechanism in SENet for improving image classification. The proposed gating-based signal merging strategy features stunning simplicity and would result in smooth signal flow of feature maps throughout the encoding-decoding process. In this case, MLP actually turns as a signal modulator of feature maps, which is anticipated to rearrange encoded high-level feature maps toward high-level representation of the output field. The effectiveness of the gating-based merging strategy lies in the fact that these neurons interact with respective feature maps in a direct, neat, and therefore strong manner. We posit that, through training, MLP can learn to precisely manipulate the initial field represented by high-level feature maps into the developed field (preliminarily in the form of gated feature maps) for a given condition input; the decoder then appropriately reconstructs the gated feature maps into the realistic evolved or developed field.

This research focuses on parameter-type conditions only. However, the core of yNet should shed light on efficiently incorporating other conditions in more complex forms, as discussed later. That said, the umbrella term “condition” in this paper refers to not only external conditions (e.g., ambient temperature and applied loading level) but also internal conditions (e.g., materials and physical properties concerned with evolving kinetics) as well as the time period of evolvement. As long as one has an association with the field evolvement and development, it can be seen as “condition,” falling within the interest of the proposed yNet.

Next, we present different applications of yNet in physical sciences and engineering, which collectively demonstrate the general effectiveness of yNet in handling many field evolvement and development problems. Following these diverse applications, we finally discuss its enormous potential for future applications as well as some limitations.

Results

Fluid dynamics

We start with data-driven modeling of flow fields over a cylinder using the proposed yNet. This fluid dynamics problem is commonly used as a benchmark in the data-driven modeling community., Specifically, a 2D, incompressible, viscous fluid is studied. In this case, the spatiotemporal evolution of a viscous fluid field is expressed in the non-dimensional form aswhere , and are vorticity, time, and velocity of fluid in scaled units with respect to unit length L and unit fluid velocity V. is the Reynolds number related to kinematic viscosity, unit length, and unit fluid velocity. As implied by Equation 1, the Reynolds number plays a dominant role in determining flow behavior and characterizing the flow pattern. Therefore, we will train a single yNet capable of simulating fluid flow evolution over a range of Reynolds numbers. The trained yNet will be tested on (1) predicting flow field development characterized by various Re unused in the training and validation datasets, (2) simulating dynamic flow field evolution via recursive predictions, (3) temporal extrapolation for predication (i.e., extending the dynamic simulation beyond the original time domain in training data), and (4) computational accuracy and efficiency in comparison with a conventional multi-input ConvNet.

Training and testing procedure

Figure 2A illustrates the overall procedure of training an yNet applicable to a range of Reynolds numbers. The transient vorticity field is selected as the flow field of interest and the Reynolds number as the condition parameter, but we can easily train yNet for the other flow fields in the same way. To achieve simulation of dynamic field evolution, the yNet is trained to predict a next-step flow field based on the current flow field and Re condition. Therefore, a long-term flow field evolution simulation can be performed by yNet through multi-step recursive predictions, as shown in Figure 2D.

Figure 2

Illustration of training and testing of yNet for fluid dynamics simulation

(A) Overall flowchart of training and testing yNet.

(B) To generate data, a physical fluid dynamics simulation using OpenFOAM is performed at each Reynolds number sampling point, followed by extraction of neighboring snapshots as data pairs.

(C) Each data point is thus a triplet, consisting of respective Reynolds number and flow fields at t and t (Δt = 0.1 s), where n = 1, 2, 3, …, 159.

(D) Correspondingly, the trained yNet can be used to predict the flow field for the next time step based on a given flow field and Reynolds number condition. Therefore, the trained yNet can also work in a recursive manner to predict long-term flow field evolution.

More condition parameters can be easily incorporated by simple adjustment of the input layer of MLP, as executed for the next application. To provide a dataset, physical simulations by solving differential equations, including Equation 1, are performed at 30 equally spaced Reynolds numbers (i.e., Re = 75, 78, 81, …, 159, 162), where 20 cases would be randomly chosen for training, 5 for validation, and the remaining 5 for testing (see data splitting in Figure 2A). For all physical simulations, extraction of neighboring snapshots is performed to generate one-step-distance data pairs, as illustrated in Figure 2B. A physical simulation will put out a total of 160 frames, indicative of 159 data pairs obtained at each Re sampling point or 4,770 for the entire dataset (Figure 2C). They are split into training, validation, and testing datasets based on Re as mentioned.

For yNet instantiated in this application, the detailed architecture is illustrated in Figure S1A and explained under Experimental procedures. We train yNet for 100 epochs with a mini-batch size of 2 by using Adam Optimizer with a learning rate of 0.001, β1 of 0.9, and β2 of 0.999.

Testing under new Re conditions

Figure 3 presents a set of testing results under four different Re. As indicated by the predictive error, yNet can accurately predict the next-step flow field in all cases, with a minor difference from the original physics-based simulation. For all 795 testing results, yNet achieves a negligibly small root-mean-square error (RMSE) of 0.426 s−1. It represents one of the first cases of data-driven modeling that directly incorporates the Reynolds number. Previous strategies for considering the Reynolds number are usually used to train a neural network that takes as input a series of historical flow fields,,, which, as a whole, embed corresponding Re information (see LSTM + ConvNet model in Figure S2B). Thus, the neural network can predict the next step based on the evolving trend of consecutive flow fields without being informed by any additional inputs like the Reynolds number. yNet, however, explicitly incorporates the important condition of the Reynolds number and takes as input only one image at the last step, tn. This is to some extent analogous to PDE-based temporal simulation, where the state at tn+1 is computed based on that at tn only instead of a series of historical states. yNet thus possesses two major advantages over previous LSTM + ConvNet strategies.

Figure 3

Testing results of yNet under different Re conditions

We randomly picked a set of testing results that show yNet in predicting next-step flow field development under different Re testing conditions.

First, yNet is as conceptually simple as a basic ConvNet without the complexity of integrating a sequence-modeling-purposed LSTM model., Second, yNet is widely applicable to general condition-dependent fluid flow simulation problems. One typical example is to predict the development of a flow field for different bluff body shapes., This is essentially a different mapping problem between distinct field quantities; i.e., from a structural field (bluff body structure) to a flow field (see comparison in Figure S3). For such regression tasks beyond standard field evolvement, the aforementioned LSTM + ConvNet model apparently will no longer work. We have to explicitly incorporate effects of condition parameters on flow field development by developing a general multi-input field-to-field regression capability.

Testing on dynamic fluid simulation and temporal extrapolation

Figure 4A shows a representative testing result of predicting multi-step evolution, under the testing condition of Re = 84, through recursive inference as depicted in Figure 2D. Noteworthy is that predictive errors can accumulate after many steps of recursive predictions. However, compared with the ground truth of physics-based simulation, yNet well reproduces the long-term dynamic flow field evolution, suggestive of a rather small error during recursive prediction at every step. The predictive error over multi-step predictions for different Re testing conditions is summarized in the left half of Figure 4C. The quantitative result reveals that predictive error does accumulate as recursive prediction proceeds but remains at a small level, (i.e., <10.0 s−1) for different testing conditions.

Figure 4

Testing results of yNet for dynamic fluid simulation

(A) Testing of yNet in predicting dynamic flow field evolution with time through recursive predictions. In the dynamic simulation, we start from step 128, which is around the wake of unstable flow showing significant evolution. We present snapshots at some time points to show the dynamic evolution process.

(B) Testing of yNet in predicting dynamic flow field evolution beyond the original time domain in training data.

(C) Predictive errors over the long-term simulation of fluid dynamics under five Re testing conditions.

Attempts have been made to predict fluid dynamics toward the distant future; i.e., beyond the time domain in training data (see the visualized example for Re = 84 in Figure 4B). Correspondingly, for validation purposes, we also extend the physical simulation for another 32 frames. Despite the big challenge associated with temporal extrapolation, predictive error generally continues to display gradual accumulation without a sudden rise, as shown in the right half of Figure 4C. Notably, predictive error can finally reach a maximum RMSE of 19.43 s−1 for Re = 150. However, such a predictive error is acceptable, still providing good prediction of the flow field, as clearly shown by the inset in Figure 4C. The above results indicate successful learning of the underlying evolution dynamics by yNet, which supports its temporal scalability to reasonably predict flow field evolvement into the distant future.

yNet versus conventional multi-input ConvNet

Figures 5A and 5B compare the computational accuracy and efficiency, respectively, of yNet and the conventional flattening-based multi-input ConvNet. For fair comparison, both networks are implemented with the exact same encoder and decoder parts (38 layers in total) but using gating- and flattening-based signal-merging strategies respectively. Also, because there are a few variants of the flattening-based strategy, we implement one by directly concatenating Re to the flattening result. The detailed architectures and configurations of the two implemented ConvNets are graphically compared in Figure S1. As the validation loss curves in Figure 5A indicate, after reaching convergence of training (>60 epochs), the two models exhibit the same level of validation accuracy. The testing result of the finally trained yNet (RMSE = 0.426 s−1) is also comparable with that (RMSE = 0.421 s−1) of the conventional multi-input ConvNet. The inset in Figure 5A shows almost the same prediction by the two trained models at a randomly selected testing data point. However, yNet has only one-tenth of the model size of the conventional multi-input ConvNet (yNet, 7,907,685; conventional, 74,900,837) and thus has various computational benefits compared with the conventional ConvNet, such as significantly reduced memory consumption, shorter training time, and faster inference speed (Figure 5B).

Figure 5

Comparison of yNet with conventional multi-input ConvNet

(A and B) Validation loss curves of yNet and conventional multi-input ConvNet (A) and comparison of model size (i.e., total number of parameters) (B) and three other performance metrics between yNet and conventional multi-input ConvNet. The performance is measured purposely based on a laptop (Intel Core i7-7500U CPU, NVIDIA GeForce GTX 950 M GPU, 16 G RAM). Memory use is measured with a single unit input instead of a batch.

These merits of yNet are attributed to the gating-based signal merging strategy, which permits effective interaction between two input channels while inducing only a minimal number of model parameters. The root reason for the effectiveness of directly manipulating feature maps lies in the high expressiveness and meaningfulness of encoded feature maps, which are widely recognized in ConvNet visualization and interpretation studies in the CV community. That is, a feature map would strongly correlate with a certain internal feature/pattern if one projected them back to a pixel-based image., Thus, direct manipulation of these feature maps will effectively generate newly weighted combinations of features toward the target image. This is, in spirit, not inconsistent with proper orthogonal decomposition (POD), by which different high-dimensional images can be expressed as different linear combinations of orthogonal modes; i.e., extracted features. However, POD relies on a linear-theory-based feature extraction, which may impair the expressiveness of POD modes., In addition, the deep decoder of yNet can contribute to accurate reconstruction of the target or output image through the non-linear and complex deconvolution process.

Porosity evolution in selective laser sintering

Here we apply yNet to simulation of porosity development in selective laser sintering (SLS) to demonstrate (1) the capability in spatially extrapolating prediction and (2) extreme computational efficiency over a physics-based model. SLS is one of the most popular additive manufacturing (AM) techniques, widely adopted for fabricating metal, ceramic, and polymer components in a layer-by-layer manner. Fast prediction of porosity formation in a realistically large area with multiple sintering layers is of huge practical significance. However, accurate simulation of porosity formation in SLS usually requires proper consideration of rigid-body translation and rotation of powder particles for particle calescence, grain growth by boundary migration through various diffusion mechanisms, and densification (Figure S4). Physics-based SLS simulation is thus usually restricted to a few layers with limited length, far from SLS practice. There is an urgent need for an inexpensive surrogate for the cumbersome physics-based SLS model.

A multi-physical SLS model is used to generate training, validation, and testing datasets. In current SLS modeling, the sintering effects (particle calescence, grain growth, and porosity evolution) are largely decided by the heat-affected zone (HAZ), which depends on the laser condition, and inter-layer interaction, which mainly includes re-sintering of previous layers during scanning of the newly added layer in the layer-by-layer fabrication. It is assumed that the sintering effect (i.e., sintering strength as a main function of laser conditions) and the layer interaction behavior can be transferred from a small patch to a long track as long as the size of the small patch is larger than the HAZ size. With this above assumption, we will train yNet based on dataset of small 128 × 128 patches (greater than the maximum HAZ depth of 117 pixels) and then demonstrate its spatial scalability to handling multiple layers of long tracks. Datasets of standard patches are obtained by cropping as-received simulation results every 10 pixels (Figure 6A). The detailed training and testing procedure is illustrated in and Figures S5 and S6 and is described in Note S1, and the detailed yNet architecture implemented in this application is illustrated in Figure S7.

Figure 6

Testing of yNet on small standard patches

(A) Pre-processing of raw simulation results for a dataset of standard patches.

(B) Testing results of yNet on standard patches. We randomly selected 5 of 43,500 testing results.

Figure 6A illustrates the data generation of standard patches for training and testing from raw simulation results. The generated dataset is split into training, validation, and testing parts based on laser power and scanning speed sample points; i.e., 75 [P, V] conditions randomly selected for training, 25 for validation, and the other 25 for testing (see detailed partition of [P, V] points in Figure S6), and Figure 6B shows 5 testing results on standard patches. yNet predictions closely resemble physics-based simulations, and only a minor morphological error (see last row of Figure 6B) can be observed through pixel-wise comparison of their phase variables. In addition to visual resemblance, we calculate the pixel-wise global accuracy (i.e., the percentage of correct pixels) for such two-phase, high-contrast fields. yNet achieves a similarity as high as 99.53% to ground truth on average for all 43,500 testing results.

Testing on long tracks and spatial extrapolation

The as-trained yNet is scalable to modeling multiple layers of long tracks. To test step by step, we first examine the spatial extrapolation of the trained yNet to handling a single-layer long track (Figure 7A, and 7B compares the accuracy of yNet in modeling small patches of original size and long tracks. For different laser power conditions, yNet consistently shows a comparably high accuracy when dealing with long tracks. On average, yNet achieves a pixel-wise accuracy of 99.04% in modeling long tracks, which is close to 99.24% in modeling small patches. The second step is testing of sintering multi-layer long tracks that involves inter-layer interaction. Figure S8 shows an example of a detailed simulation process for 3 layers. By adopting such a layer-wise simulation, we obtain the multi-layer sintering results by yNet and a physics-based model under different laser conditions (Figure 7C). By increasing laser power, we can see that the results by yNet and the physics-based model both show a transition from the layer-scale unsintered region (Figure 7C, top left inset) to small scattered unsintered areas (Figure 7C, bottom left inset) and, eventually, to a fully sintered state (Figure 7C, bottom right inset). The unsintered regions are located between layers because of insufficient sintering depth under small laser powers. The detailed labeling results of unsintered regions can be also found in Figure S9. The quantitative results of measured unsintered regions (Figure 7C, top right) further prove the good agreement between yNet-based and physical multi-layer simulations.

Figure 7

Testing of yNet on extrapolating prediction for long tracks

(A) Schematic of the as-trained yNet for predicting sintering of a long track.

(B) Testing results of yNet for predicting sintering of a long track under different laser conditions.

(C) Extension of yNet to predicting sintering of 3-layer long tracks under different laser conditions. By following the workflow in Figure S8, we perform sintering simulation of multiple layers under three laser power conditions (power = 20, 25, and 30 W). Other settings are the same as that used in Figure S8. The overall sintering effect of multiple layers (measured by the percentage of unsintered region) as a function of laser power is then obtained for yNet and the physics-based model, as shown.

The tested spatial scalability can help address the computationally daunting task of large-scale sintering simulation, even including a full-component SLS simulation. To demonstrate this, we perform an extreme-scale sintering simulation of a practical 315-layer SLS component with dimensions of 70.8 × 44.1 mm2 (or 3,5416 × 22,050 pixels), as illustrated in Figures S10 and S11 and described in Note S2.

yNet versus physics-based model

As discussed earlier, physics-based SLS simulation, especially at the component level, is notoriously computationally demanding. It is therefore an ideal case to demonstrate the computational superiority of yNet over a physics-based model. We look into the computation time of yNet and the original physics-based SLS model. Again, yNet and the physics-based model are intentionally tested on a laptop used in the benchmark study in application 1.

Figure 8 shows the computation time as a function of the length of the modeled powder bed. Simulation of small tracks, like the 640-pixel-long tracks in Figure 7A, is just a matter of tens of milliseconds by using yNet. In striking contrast, the original physics-based model used in this study will take a few hours to complete the same task in the same computational setting. The test for the physics-based model stops at a length of 2,048 pixels because it can be predicted that, by modeling an even longer track, the computation time will increase rapidly to days. For the 8 different lengths tested for both models, yNet is generally 5–6 orders of magnitude faster than the physics-based model; see Table S1 for a more detailed quantitative comparison. yNet can easily handle extremely large layers with a maximum length of ∼21,248 pixels with such a limited computational resource. Modeling even this large layer remains nearly instant for yNet. In this regarding, the component-level SLS simulation (i.e., hundreds of large layers), which previously was a formidable task even for high-end computing facilities, can now be accomplished in a few minutes with yNet by simply using a laptop.

Figure 8

Comparison of computation time of yNet and a physics-based model

yNet is not only computationally much faster than the physics-based model but also able to handle extremely long tracks with limited computational resources.

Stress field development

Here we present another application of yNet: predicting stress field development. The main objective of this application is to test the robustness of yNet in (1) handling an input and output field with striking differences and (2) extending the stress field prediction for structures beyond the specific type of shape used in training. Stress field development is a totally different type of field evolvement problem compared with the previous two. The task in this application is to construct mapping between fields of different physical meaning: a structural field and a stress field. Compared with standard field evolvement of a certain type of field, stress field development shows a radical difference between input and output images. The input structure is a simple high-contrast field, whereas the output stress field is characterized by complex texture and great variability. Therefore, this application may push the performance limit of the proposed yNet in the tough application scenario of strong “field evolvement.”

For illustration purposes, we trained yNet to predict stress field development in a typical perforation structure (i.e., a solid structure with a hole) subjected to compressive loading (Figure S13). Note S3 describes the detailed training and testing procedure. As briefly illustrated in Figure 9A, the basic idea is to train yNet with specific types of perforation structures that feature elliptic holes but with random combinations of hole orientation, size, and aspect ratio. When trained, yNet is expected to predict stress field development for perforation structures with holes of not only elliptic shape but also other geometries.

Figure 9

Testing of yNet on predicting stress development for various types of perforation structures

(A) yNet is trained based on a simple dataset containing elliptical holes only. The as-trained yNet can predict stress fields for other types of holes. All holes used in training, validation, and testing are randomly generated based on the descriptive methods as shown.

(B) Testing of yNet on different types of perforation structures. The perforation structures have completely different types of holes than that used in the training dataset. Five randomly generated rectangular, triangular, and polygonal holes are tested.

We first routinely test yNet for perforation structures with elliptical holes unused during training; see Figure S14 for more details. Briefly, the prediction by yNet overall agrees well with the finite element (FE)-based simulation. The RMSE for all 1,000 testing results is as small as 12.70 MPa. Therefore, although the input and output images are totally different and display huge changes, yNet can correctly predict these significant “evolvements.”

Testing on other types of perforation structures and shape extrapolation

Besides routine testing, we tested yNet for perforation structures with other categories of holes, including triangular, rectangular, and polygonal ones, as illustrated in Figure 9A. For the 5 random testing results shown in Figure 9B, the RMSEs are 20.97, 50.43, 49.66, 41.33, and 51.71 MPa, respectively, which are relatively higher than that of testing with elliptic holes. Although the predictive performance shows a slight decrease, from the comparison of detailed stress distribution, yNet still reasonably predicted stress field development. It is thus believed that yNet has learned the basic rule for stress field development. For example, stress concentration tends to take place during sharp geometrical transitions along the loading direction and also initiates from two bottom corners with increasing loading (associated with the fixed boundary at the bottom in the current modeling); the discontinuous geometrical changes transverse to the loading direction, however, do not serve as significant stress risers. These rules are fully contained in the training dataset and universally valid for perforation structures of any type. Learning these fundamental rules endows yNet with great extrapolation capability for coping with different perforation structures.

Discussion

The proposed yNet should be generally applicable to many condition-dependent field evolvement and development problems. Note S4 describes another interesting application of yNet for grain growth simulation; see Figure S15 for the detailed training and testing processes and Figure S16 for some testing results. In addition, we utilize the spatial and temporal scalability of yNet to enable long-term, large-scale grain growth simulation; see Video S1. yNet-based large-scale grain growth simulation (Δt = 1), Video S2. yNet-based large-scale grain growth simulation (Δt = 5), Video S3. yNet-based large-scale grain growth simulation (Δt = 30) for simulations performed at different time resolutions. The four presented applications (3 major and 1 supplemental) have helped us examine the general effectiveness of yNet in handling many distinct field evolvement and development problems. As a general-purpose regression tool, the well-tested yNet should have an enormous potential for future applications. Some promising applications would be ultrafast modeling of structural/morphological evolution in materials science and engineering,53, 54, 55 temperature field development in thermal design and control, and geometrical distortion in manufacturing, to name a few.

Physics-informed ML is now emerging as a revolutionary technique by which physics constraints can be imposed to learning processes by directly formulating physical differential equations into loss function or casting them into an energy minimization framework that naturally fits in the loss minimization task of ML training. In doing so, ML would be penalized for its violation of physical laws and forced to learn physical governing equations. Anitescu et al. adopted adaptive collocation to selectively refine training grids at regions with large residuals, which can significantly reduce errors of physics-informed ML. With the rapid advancement of physics-informed ML, the large datasets required for training yNet will be potentially alleviated or even completely eliminated with increased robustness and generalization.

In the future, the extension to real-world data is crucial to fully showcase the modeling capability and realize the practical significance of yNet. Nowadays, data-driven modeling of field evolvement and development still relies principally on simulation-derived datasets because performing highly controlled experiments and measuring 2D or even 3D physical fields with the desired resolution is not only costly but also often technically challenging. However, there have recently been advances in obtaining experimental data at high resolution and even in real time for some physical phenomena. For example, in the fluidics community, besides continued advancement of experimental imaging technologies,, data postprocessing techniques have been explored to obtain high-resolution flow fields, in time,, space,, or both, from as-measured imperfect data. In AM, X-ray computed tomography (XCT) has been increasingly used for characterizing 3D porosity.70, 71, 72 In experimental mechanics, the digital volume correlation (DVC) method permits characterization of 3D displacement, strain, and stress fields with ever-increasing spatiotemporal resolution because of parallel improvement of DVC algorithms and internal imaging instruments.73, 74, 75 It is thus expected that yNet will be readily applied to experimental data in these fields, with fast development of data acquisition techniques and upon availability of (sufficient) high-quality experimental data in the near future.

The main limitation of yNet is that the trained yNet is sometimes limited in scaling to fields of large and variable dimensions. In yNet-based component-level SLS simulation, the scalability to long tracks relies on the assumption of globally uniform evolving kinetics of porosity evolution in SLS. That is, the sintering effect depends on the applied laser condition and is consistent along the length of the sintered track for a given laser condition (see the illustration in Figure S12). The large-scale simulation of natural grain growth is also based on its completely uniform evolving kinetics over the modeling domain. This is, however, not always the case; e.g., for fluid dynamics simulation and stress field prediction where such simple scalability may not hold. Other solutions are needed for developing more advanced scientific ConvNets. However, the core of yNet is still expected to play a role whenever it comes to developing multi-input ConvNets and merging input signals.

Another limitation is that only parameter-type conditions are considered and demonstrated in the current study. There are, however, higher-dimensional and more complex conditions for various physical and engineering problems. For example, the applied loading in stress field modeling in Figure 9 can be non-uniform and, thus, not describable by a single scalar parameter. Also, besides natural grain growth (Note S4), grain growth in practice can be influenced by the temperature of the field because the preferred growth direction is along the local thermal gradient. These complex conditions, no matter how many dimensions they have, can be eventually converted to an embedding vector of the desired length. It would then allow adoption of gating-based signal merging for efficient incorporation of their effects on field evolvement.

We have proposed a multi-input ConvNet, yNet, with stunning simplicity and usefulness, which are concretely demonstrated through comprehensive applications and testing. It thus may solidly push our fundamental regression capability to multi-input field-to-field/image-to-image regression, which is broadly in demand in data-driven modeling across many physical and engineering domains. yNet merges input signals without flattening the field input into a large vector, instead directly manipulating high-level-encoded feature maps. It possesses a significant computational advantage over its ConvNet counterpart and extreme computational efficiency compared with the physics-based model. Based on its computational superiority, large-scale and/or long-term dynamic simulation of field evolvements can be easily performed with even limited computational resources. As indicated by the unprecedented wide and deep applications presented in this research, the proposed yNet may have enormous potential and a profound effect as a conceptually simple, lightweight, but powerful multi-input field-to-field regression tool.

Experimental procedures

Resource availability

Lead contact

Further information and requests for resources and reagents should be directed to and will be fulfilled by the lead contact, L.C. (leichn@umich.edu).

Materials availability

This study did not generate new unique materials or reagents.

Fluid dynamics model

A two-dimensional, incompressible, viscous model is used to study flow past a cylinder using the numerical method in OpenFOAM. The full governing equations in describing the flow arewhere is the velocity field, is the pressure field, and ν and are the kinematic viscosity and density of the fluid, respectively.

The discretized continuity and Navier-Stokes Equations 2 and 3 are solved using the pressure implicit with splitting of operator (PISO) algorithm on a mesh consisting of 85,720 cells. Specifically, the computational fluid dynamics (CFD) simulation is realized using icoFoam. The velocity field is computed into the vorticity field for flow visualization. It is important to transfer the Navier-Stokes equation into the non-dimensional form so that a dimensionless number is used to characterize the system. Hereby, the Reynolds number is introduced using dimensional analysis that represents the ratio of inertial forces to the viscous forces in the flow field, as shown in Equation 1. In this case, variables in Equations 2 and 3 can be rewritten in scaled units with respect to typical length L (diameter of the cylinder) and typical fluid velocity V (incoming flow velocity); namely, , , , . We focus on the moderate Reynolds number range where a wake is developed passing a cylinder and oscillating with a certain frequency. The flow behavior is identified as a von Karman vortex and is ideal to characterize the flow pattern with different Reynold numbers.

SLS model

In this study, we basically customize a phase-field-based sintering model for applicability in SLS, with incorporation of a heat transfer model and a Gaussian heat source model describing the effective heat input from a moving laser beam. The sintering model has properly taken into account multiple physical processes by first reformulating the effective diffusion coefficient aswhere, D, D, D, and D are temperature-dependent diffusion coefficients in solid volume, vapor, along the surface, and the grain boundary, respectively, which are thus dependent on fully coupled thermal modeling. η is phase variable that describes the αth particle. In addition, the non-conserved porosity evolution is partially governed by the rigid body motion-induced advection velocity field, which is a combination of the translation of rotation from all relevant particles, . The velocity fields of rigid-body translation, and rotation, r, of the αth particle are calculated bywhere m = 500 and m = 1. The force, , and torque, , acting on the powder particles are associated with the their real-time structural environment and materials density. Full mathematical details of our physical SLS model are reported in a separate paper on multi-physical SLS simulation.

Powder generation model

In addition to the sintering model for simulating porosity evolution, generation of a powder bed (i.e., the initial porous structure) is simulated using a “rain” model. The basic mechanism of a rain model is to add a powder particle to the lowest position on the surface of the current powder layer, update the powder layer, and repeat the previous two steps. This iterative process is terminated when the newly added particle reaches the user-defined layer height. The mean and standard deviation of the radius of deposited powders are 25 and 2.5 μm, respectively. It should be pointed out that yNet in this study will be trained to simulate porosity evolution and thus replace a physics-based sintering model only. The powder bed generation model is a separate model that provides an initial structure for physical sintering simulation and yNet-based sintering simulation.

Solid mechanics model

An FE-based solid mechanics model is used to generate stress field datasets. The modeled object is a perforation structure with dimensions of 0.1 × 0.1 m2. To generate random perforation structures, the elliptical hole at the center is defined by the length of a-semiaxis and b-semiaxis as well as rotation angle. The rectangular hole is also located at the center and defined by width, height, and rotation angle. The triangular hole and polygonal hole are specified by the coordinates of their respective vertices. Fixed constraint is applied at the bottom of the perforation structure and uniform compressive force at the top. This model is implemented in COMSOL Multiphysics 5.5.

Grain growth model

A phase-field grain growth model is used to perform natural grain growth simulation and generate grain structure datasets. In the phase-field model, each grain is described by a phase variable, φ. The grain growth is then described by the temporal and spatial evolution of the phase variable, which can be simulated by solving the time-dependent Ginzburg-Landau equationwhere L is the kinetic rate coefficient related to the grain boundary mobility and F is the total free energy of a polycrystalline system that drives the microstructure evolution. All phase-field grain growth simulations start with a random seeding of 150 grains of size r = 5. A periodic boundary condition is applied for the top (left) and bottom (right) boundary.

yNet

We use the yNet instantiated in fluid dynamics simulation as an illustrative example. The detailed architecture is shown in Figure S1A. The first layer of MLP is the condition input (i.e., Reynolds number), the hidden layer is a fully connected layer with 256 neurons, and the final layer has 512 neurons. For the encoder, the input is a vorticity flow field at the current step in a 256 × 64 image. Each green block represents a combination of Conv + Relu operations, and a light green block means max-pooling. In this manner, the encoder finally yields 512 8 × 2 feature maps, which are passed to a dropout layer with a rate of 0.5 before merging with the MLP signal. The decoder just has a somewhat mirrored topology of the encoder, with each blue block representing a combination of Conv + Relu operations and dark blue block up-sampling. At the end of the MLP and encoder, conventional approaches adopted a flattening-based strategy to merge multi-input signals. As illustrated in Figure 1E, the signal merging through a sequence of flattening, concatenating, compressing, and reshaping operations can be expressed aswhere n1×n2 and N2 are, respectively, the shape and number of encoded feature maps, at the end of encoder, N3 = n1×n2×N2 is the dimension of the vector, , by flattening feature maps, denotes concatenate operation, is the concatenation result, N1 is the dimension of the output vector of MLP, , represent the resulting vector by compressing , and is finally obtained feature maps through reshaping for decoding process.

For yNet, we propose to simply merge the signal using the gating-based strategy as depicted in Figure 1G,where denotes channel-wise multiplication, and is the multiplication results (gated feature maps) for the decoding process. To successfully perform channel-wise multiplication, the number of neurons in the last layer of MLP is always set equal to the number of feature maps at the end of encoder; i.e., N1 = N2 in Figure 1G.

For the other three applications, we basically use the yNet described above but with the potentially adjusted depth of encoder-decoder and width of MLP depending on the complexity of the modeling problem. Specifically, in data-driven modeling of stress field development, we use the same yNet as implemented in modeling fluid dynamics (Figure S1A). In data-driven modeling of porosity development in SLS and grain growth, we implement a slightly shallower yNet (Figure S7), considering the relative textural simplicity of the images to be processed. Correspondingly, MLP is properly adjusted to guarantee channel-wise multiplication at the end of MLP and ConvNet encoder. In all four applications, min-max normalization of datasets is performed for ease of training.

Two types of loss functions are used in this study. For an output field with strong texture (i.e., flow field and stress field), we use mean squared error (MSE) to evaluate loss,where N is the number of samples processed, is the condition-parameter input, X is the field input, is the evolved field by prediction, and is the evolved field of ground truth. For an output field that clearly displays two distinct phases (i.e., porosity structure and grain structure), we use binary cross-entropy loss to better penalize predictive errors:

In all four applications, yNet is trained end to end by using the training hyperparameters listed in Table S2. Learning curves are summarized in Figure S17.