{"id":9469,"date":"2022-05-11T13:39:30","date_gmt":"2022-05-11T04:39:30","guid":{"rendered":"https:\/\/www.ecomottblog.com\/?p=9469"},"modified":"2022-05-13T16:13:14","modified_gmt":"2022-05-13T07:13:14","slug":"introduction-to-physics-informed-neural-networks","status":"publish","type":"post","link":"https:\/\/test.ecomottblog.com\/?p=9469","title":{"rendered":"INTRODUCTION TO PHYSICS-INFORMED NEURAL NETWORKS"},"content":{"rendered":"

\u306f\u3058\u3081\u307e\u3057\u3066\u30d5\u30d0\u30c1\u3068\u7533\u3057\u307e\u3059\u3002\u79c1\u306f\u3001\u3082\u3068\u3082\u3068\u5316\u5b66\u30a8\u30f3\u30b8\u30cb\u30a2\u3067\u3001\u65e5\u672c\u306b\u6765\u308b\u524d\u306f\u3001\u30ef\u30eb\u30b7\u30e3\u30ef\u5de5\u79d1\u5927\u5b66\uff08\u30dd\u30fc\u30e9\u30f3\u30c9\uff09\u306720\u5e74\u307b\u3069\u50cd\u3044\u3066\u3044\u307e\u3057\u305f\u3002\u79c1\u306f2021\u5e745\u6708\u304b\u3089\u30a8\u30b3\u30e2\u30c3\u30c8\u306e\u30c7\u30fc\u30bf\u30a2\u30ca\u30ea\u30c6\u30a3\u30af\u30b9\u90e8\u306b\u52e4\u52d9\u3057\u3066\u3044\u307e\u3059\u3002\u30a8\u30b3\u30e2\u30c3\u30c8\u3067\u50cd\u304f\u3053\u3068\u3067\u3001\u7570\u5e38\u691c\u77e5\u3001\u70b9\u7fa4\u8d85\u89e3\u50cf\u3001\u753b\u50cf\u30ce\u30a4\u30ba\u9664\u53bb\u3068\u3044\u3063\u305f\u554f\u984c\u306b\u5bfe\u3059\u308bAI\u624b\u6cd5\u306e\u5fdc\u7528\u306b\u3064\u3044\u3066\u3001\u65b0\u305f\u306a\u7d4c\u9a13\u3092\u7a4d\u3080\u3053\u3068\u304c\u51fa\u6765\u307e\u3057\u305f\u3002\u3053\u308c\u3089\u306e\u554f\u984c\u306f\u3059\u3079\u3066\u3001\u79c1\u306b\u3068\u3063\u3066\u975e\u5e38\u306b\u8208\u5473\u6df1\u3044\u3082\u306e\u3067\u3057\u305f\u3002\u307e\u305f\u3001\u79c1\u81ea\u8eab\u3001\u5316\u5b66\u5de5\u5b66\u306e\u8ab2\u984c\u306bAI\u30c4\u30fc\u30eb\u3092\u5c0e\u5165\u3059\u308b\u3053\u3068\u306b\u3082\u8208\u5473\u304c\u3042\u308a\u3001\u6700\u8fd1\u3001\u7269\u7406\u60c5\u5831\u306b\u57fa\u3065\u304f\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\uff08PINN\uff09\u306b\u3064\u3044\u3066\u8abf\u3079\u3066\u3044\u307e\u3059\u3002
\n\u3053\u306e\u30d6\u30ed\u30b0\u3067\u306f\u3001\u79c1\u304c\u8abf\u67fb\u3057\u305fPINN\u306b\u3064\u3044\u3066\u3001\u7c21\u5358\u306a\u5b9f\u9a13\u3092\u4ea4\u3048\u3066\u7d39\u4ecb\u3057\u305f\u3044\u3068\u601d\u3044\u307e\u3059\u3002\u30c7\u30fc\u30bf\u5206\u6790\u3084AI\u6d3b\u7528\u306b\u8208\u5473\u304c\u3042\u308b\u65b9\u304b\u3089\u53cd\u5fdc\u3092\u9802\u3051\u308c\u3070\u5b09\u3057\u304f\u601d\u3044\u307e\u3059\u3002<\/p>\n

PINN\u306f\u3001\u7269\u8cea\u3001\u30a8\u30cd\u30eb\u30ae\u30fc\u3001\u6d41\u4f53\u306a\u3069\u306e\u8f38\u9001\u73fe\u8c61\u306e\u30b7\u30df\u30e5\u30ec\u30fc\u30b7\u30e7\u30f3\u3092\u53ef\u80fd\u306b\u3057\u307e\u3059\u3002\u3053\u306e\u3088\u3046\u306a\u30b7\u30df\u30e5\u30ec\u30fc\u30b7\u30e7\u30f3\u306f\u3001\u5316\u5b66\u53cd\u5fdc\u306b\u304a\u3051\u308b\u30d7\u30ed\u30bb\u30b9\u7406\u89e3\u306b\u306f\u6b20\u304b\u305b\u307e\u305b\u3093\u3002
\n<\/p>\n

\u7269\u7406\u60c5\u5831\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\uff08PINN\uff09<\/h3>\n

PINN\u306f\u3001\u79d1\u5b66\u7684\u554f\u984c\u3092\u89e3\u6c7a\u3059\u308b\u30c4\u30fc\u30eb\u3068\u3057\u3066Raissi et al.<\/em>(2017a), Raissi et al.<\/em>(2017b), Raissi et al.<\/em>(2019)\u306b\u3088\u3063\u3066\u7d39\u4ecb\u3055\u308c\u3066\u3044\u307e\u3059\u3002\u3053\u306e\u3088\u3046\u306a\u554f\u984c\u306f\u901a\u5e38\u3001\u504f\u5fae\u5206\u65b9\u7a0b\u5f0f\uff08PDE\uff09\u307e\u305f\u306f\u5e38\u5fae\u5206\u65b9\u7a0b\u5f0f\uff08ODE\uff09\u3092\u7528\u3044\u3066\u8a18\u8ff0\u3067\u304d\u308b\u7269\u7406\u6cd5\u5247\u306b\u3088\u3063\u3066\u652f\u914d\u3055\u308c\u3066\u3044\u308b\u3002\u305d\u306e\u305f\u3081\u3001PINN\u306e\u69cb\u9020\u306f\u3001\u6240\u671b\u306eODE\u307e\u305f\u306fPDE\u7cfb\u3092\u7d71\u5408\u3059\u308b\u3088\u3046\u306b\u8a2d\u8a08\u3055\u308c\u3066\u3044\u307e\u3059\u3002\u3053\u306e\u305f\u3081\u3001PINN\u306e\u5b66\u7fd2\u6642\u306b\u306f\u3001\u3055\u3089\u306a\u308b\u5236\u7d04\u304c\u8ab2\u3055\u308c\u307e\u3059\u3002\u305d\u308c\u3089\u306e\u5236\u7d04\u6761\u4ef6\u306f\u3001PDE\u3084ODE\u306e\u5f62\u5f0f\u3060\u3051\u3067\u306a\u304f\u3001\u5883\u754c\u6761\u4ef6\u3084\u521d\u671f\u6761\u4ef6\u304b\u3089\u3082\u751f\u3058\u307e\u3059\u3002\u3055\u3089\u306b\u3001\u3053\u308c\u3089\u306e\u5236\u7d04\u306f\u3001\u5b66\u7fd2\u3057\u305f\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306b\u65b0\u305f\u306a\u60c5\u5831\u3092\u63d0\u4f9b\u3057\u307e\u3059\u3002\u305d\u306e\u305f\u3081\u3001\u5b9f\u9a13\u30c7\u30fc\u30bf\u306e\u5fc5\u8981\u6027\u304c\u6e1b\u308a\u3001\u5883\u754c\u6761\u4ef6\u3084\u521d\u671f\u6761\u4ef6\u306e\u5024\u3060\u3051\u3067\u3001\u554f\u984c\u3092\u89e3\u304f\u3053\u3068\u3082\u53ef\u80fd\u306b\u306a\u308b\u306e\u3067\u3059\u3002<\/p>\n

\u4ee5\u4e0b\u3001\u6d41\u308c\u306e\u30b7\u30df\u30e5\u30ec\u30fc\u30b7\u30e7\u30f3\u306b\u95a2\u3059\u308b\u8abf\u67fb\u3068\u7c21\u5358\u306a\u5b9f\u9a13\u3092\u884c\u3044\u307e\u3057\u305f\u306e\u3067\u3001\u8aad\u3093\u3067\u3044\u305f\u3060\u3051\u308c\u3070\u5e78\u3044\u3067\u3059\u3002\uff08\u82f1\u8a9e\u3067\u7533\u3057\u8a33\u3042\u308a\u307e\u305b\u3093\u3002\uff09<\/p>\n

Application of PINN<\/strong><\/p>\n

    \n
  1. Simulation of the phenomenon (i.e., profiles prediction of velocity, temperature, component concentration etc. using calculations) (Raissi et al., 2017a). This kind of calculation can be performed using only boundary and initial conditions. However, the availability of experimental data will improve the accuracy of the simulation.<\/li>\n
  2. Use of PINN for inverse estimation of physical properties from measured experimental data (Raissi et al., 2017b).<\/li>\n
  3. Discovery of the form of governing partial differential equations from experimental data (Chen et al., 2021).<\/li>\n<\/ol>\n

    PINN for flow simulation<\/strong><\/p>\n

    In the literature it can be found at least two different approaches to flow simulation using PINN:<\/p>\n

    Approach 1: PINN for solving Navier-Stokes equation in the case of 2D incompressible fluid flow introduced by Raissi et al. (2017b). Let\u2019s call it traditional PINN. The authors applied stream function to automatically satisfy the continuity equation. Hence, in this approach neural networks were used for the estimation of the values of pressure and stream function\uff08p,\u03a8\uff09. Subsequently, velocity vector was calculated using the stream function.<\/p>\n

    Approach 2: Mix-variable PINN introduced by Rao et al. (2020). The authors wanted to avoid second order derivatives in PDE. This was achieved by introducing Cauchy stress tensor into their model. Consequently, neural networks were used for the estimation of the values of pressure, stream function and the Cauchy stress tensor\uff08p,\u03a8,\u03c3\uff09(Fig.2). The authors claimed that in this way the trainability and accuracy of the model was improved.<\/p>\n

    Application of PINN for the simulation of flow between two parallel plates<\/strong><\/p>\n

    \"\"Fig. 1. Flow between two plates: the geometry and the values of the parameters.<\/p>\n

    PINN was applied to simulate the fluid velocity during the steady and incompressible flow between two parallel and horizontal plates. The geometry of the flow and the values of the other operating parameters are shown in the Fig.1. As it can be seen in this figure flow was treated as 2D. This means that the changes in velocity along the second horizontal direction was neglected. It should be mentioned, that in this case the profile of the x component of velocity vector can be also obtained analytically if considering the region sufficiently far from the inlet.<\/p>\n

    \"\"Fig. 2. Traditional PINN for the simulation of flow between two plates.<\/p>\n

    The flow between two plates was simulated using both traditional and mix-variable PINNs of which architectures and the PDEs forms are shown in Fig. 2 and Fig. 3 respectively. As it can be seen in these figures mix-variable PINN model needed more equations than traditional PINN model. However, mix-variable PINN model did not contain second order derivatives. Additionally, both models were implemented using SciANN (Haghighat and Juanes, 2021) a Python package for PINN application, which was built based on Tensorflow and Keras. Thus, it inherited many of Kera\u2019s functionalities.<\/p>\n

    The calculation was conducted for 6 different cases of which details are shown in Table 1. Moreover, the default SciANN setting was kept for the other parameter not shown in this table (e.g. loss_func: “mse”, optimizer: “adam”).<\/p>\n

    In Table 1, there are also shown the total number of colocation points (i.e., the points for which PINN calculated flow parameters) and the number of those point in x and y directions (in the bracket). During these calculations, those points were regularly distributed in the space between two plates.<\/p>\n

    \"\"Fig. 3. Mix-variable PINN for the simulation of flow between two plates.<\/p>\n

    The example of simulation results is shown in Fig.4. The distribution of velocity and pressure along the space between two plates can be seen in Fig. 4a. On the other hand, the comparison of PINN calculation with analytical solution is given in Fig. 4b. These results indicated that velocity profile developed relatively quickly and after that there was a good agreement between PINN result and analytical solution. Similar results were obtained for all considered cases (see Table 1).<\/p>\n

    The additional results of simulation are shown in Table 2. This table contains the values of average axial velocities vx* , which were calculated using x velocity components along the axis y at the given distance from the inlet. Because of the inlet boundary condition (see Fig. 2 and 3), it was expected that vx*=1. Nevertheless, due to the computational error the values of vx* are smaller than one. This discrepancy between expected and calculated value was smaller in the case of mix-variable PINN model than in the case of traditional PINN model. Similarly, when the number of colocation points was increased vx* become closer to 1.<\/p>\n

    In Table 2 there are also shown the values of loss function for each of the studied cases. The loss function was smaller in the case of mix-variable PINN model and its value decreased when the number of colocation points increased.<\/p>\n

    To conclude, it seems that Rao et al. (2020) was right that the rejection of second derivatives from PDE improves trainability and accuracy of the model. Nevertheless, my results also indicated that performance of PINN depends on the number of colocation points.<\/p>\n

    Table 1. The PINN architecture, training pattern and the number of colocation points.\"\"<\/p>\n

    \"\"Fig. 4. The example of simulation results obtained using mix-variable PINN (case 6): a) distribution of velocity and pressure in the space between two plates; b) velocity profile at the given distance from the inlet.<\/p>\n

    Table 2. The results of simulation\"\"<\/p>\n

    Application of PINN for the simulation of flow between two concentric cylinders with rotating inner one<\/strong><\/p>\n

    \"\"<\/p>\n

    Fig. 5. Flow between two concentric cylinders with rotating inner one: a) reactor with such a flow; b) 2D approximation – black contours shows the space between cylinders where the fluid flows.<\/p>\n

    Flow between two concentric cylinders with rotating inner one (rotor) is illustrated in Fig. 5. This flow is 3D (Fig 5a), however, at sufficiently low rotor rotation its geometry can be simplified as shown in Fig. 5b. This simplification requires assumption that the values of all derivatives in angular direction are zeros (this means that pressure and velocity can change values only in x and r direction). Under such an assumption this flow can be approximated using axisymmetric swirl model (see for example ANSYS FLUENT 12.0\/12.1 Documentation). This model is not really 2D because although derivatives of pressure and velocity in angular direction disappear the angular component of velocity is still calculated.<\/p>\n

    Application of axisymmetric swirl model requires transformation of governing PDEs into cylindrical coordinate form. My previous attempts of simulation of the flow in a pipe using equations in cylindrical coordinates showed poor performance of traditional and mix-variable PINNs. Therefore, I decided to use neural networks for the estimation of the values of pressure and velocity components\uff08p, vx, vr, v\u03b8\uff09. As the result, the form of PDEs and PINN architecture used for these simulations were as illustrated in Fig. 6.<\/p>\n

    The calculation was conducted for 4 different cases of which details are shown in Table 3. Meanwhile, the values of axial and rotor angular velocity were chosen to be small enough (see Fig. 5 b) so that velocity components could be calculated analytically for the region sufficiently far from the inlet.<\/p>\n

    \"\"<\/p>\n

    Fig. 6. PINN for the simulation of flow between two concentric cylinders with rotating inner one.<\/p>\n

    The example of simulation results is shown in Fig.7. The distribution of velocity and pressure along the space between two cylinders can be seen in Fig. 7a. Meanwhile, in Fig. 7b and 7c the comparison of PINN calculations with analytical solutions are given. These results indicated that velocity profile developed relatively quickly in the case of axial velocity. On the contrary, in the case of angular velocity the discrepancy between simulation and analytical solution was significant for the wider distance from the inlet. Similar results were obtained for all considered cases (see Table 3).<\/p>\n

    The values of average axial velocities vx* and loss function are shown in Table 4. Similarity to the flow between two plates the discrepancy between expected (which should be also 1) and calculate vx* were noticed. This discrepancy as well as loss function became smaller when number of colocation points was increased.<\/p>\n

    Table 3. The neural network architecture, training pattern and the number of colocation points.\"\"<\/p>\n

     <\/p>\n

    Table 4. The results of simulation<\/p>\n

    \"\"<\/p>\n

    \"\"<\/p>\n

    Fig. 7. The example of simulation results (case 4): a) distribution of velocity and pressure in the space between two cylinders; b) axial velocity (vx) profile at the given distance from the inlet; c) angular velocity (v\u03b8) profile at the given distance from the inlet.<\/p>\n

    In conclusion, it seems that if the PDEs have cylindrical coordinate form better results could be obtained when velocity vector is estimated directly by neural networks (i.e., without stream function). Moreover, similarly to the flow between two plates the accuracy of the prediction increases with the increasing in the number of colocation points.<\/p>\n

     <\/p>\n

    Literature:<\/strong><\/p>\n

    Chen Z., Liu Y., Sun H. (2021): Physics-informed learning of governing equations from scarce data. Nat Commun 12, 6136. <\/p>\n

    Haghighat E, Juanes R. (2021): SciANN: A Keras\/Tensorflow wrapper for scientific computations and physics-informed deep learning using artificial neural networks. Computer Methods in Applied Mechanics and Engineering, 373.<\/p>\n

    Raissi, M., Perdikaris, P., and Karniadakis, G. E. (2017a): Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations. arXiv preprint arXiv:1711.10561.<\/p>\n

    Raissi, M., Perdikaris, P., and Karniadakis, G. E. (2017b):\u3000Physics informed deep learning (part ii): Data-driven discovery of nonlinear partial differential equations. arXiv preprint arXiv:1711.10566.<\/p>\n

    Raissi, M., Perdikaris, P., and Karniadakis, G. E. (2019): Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics 378, 686\u2013707<\/p>\n

    Rao C., Sun H., Liu Y., (2020): Physics-informed deep learning for incompressible laminar flows. Theoretical and Applied Mechanics Letters,vol. 10, 3, 207\u2013212.<\/p>\n","protected":false},"excerpt":{"rendered":"

    \u306f\u3058\u3081\u307e\u3057\u3066\u30d5\u30d0\u30c1\u3068\u7533\u3057\u307e\u3059\u3002\u79c1\u306f\u3001\u3082\u3068\u3082\u3068\u5316\u5b66\u30a8\u30f3\u30b8\u30cb\u30a2\u3067\u3001\u65e5\u672c\u306b\u6765\u308b\u524d\u306f\u3001\u30ef\u30eb\u30b7\u30e3\u30ef\u5de5\u79d1\u5927\u5b66\uff08\u30dd\u30fc\u30e9\u30f3\u30c9\uff09\u306720\u5e74\u307b\u3069\u50cd\u3044\u3066\u3044\u307e\u3057\u305f\u3002\u79c1\u306f2021\u5e745\u6708\u304b\u3089\u30a8\u30b3\u30e2\u30c3\u30c8\u306e\u30c7\u30fc\u30bf\u30a2\u30ca\u30ea\u30c6\u30a3\u30af\u30b9\u90e8\u306b\u52e4\u52d9\u3057\u3066\u3044\u307e\u3059\u3002\u30a8\u30b3\u30e2\u30c3 […]<\/p>\n","protected":false},"author":104,"featured_media":9519,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[27],"tags":[45,127,126,125],"_links":{"self":[{"href":"https:\/\/test.ecomottblog.com\/index.php?rest_route=\/wp\/v2\/posts\/9469"}],"collection":[{"href":"https:\/\/test.ecomottblog.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/test.ecomottblog.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/test.ecomottblog.com\/index.php?rest_route=\/wp\/v2\/users\/104"}],"replies":[{"embeddable":true,"href":"https:\/\/test.ecomottblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=9469"}],"version-history":[{"count":27,"href":"https:\/\/test.ecomottblog.com\/index.php?rest_route=\/wp\/v2\/posts\/9469\/revisions"}],"predecessor-version":[{"id":9526,"href":"https:\/\/test.ecomottblog.com\/index.php?rest_route=\/wp\/v2\/posts\/9469\/revisions\/9526"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/test.ecomottblog.com\/index.php?rest_route=\/wp\/v2\/media\/9519"}],"wp:attachment":[{"href":"https:\/\/test.ecomottblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=9469"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/test.ecomottblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=9469"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/test.ecomottblog.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=9469"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}