{"id":1691,"date":"2026-10-21T10:40:43","date_gmt":"2026-10-21T02:40:43","guid":{"rendered":"https:\/\/people.utm.my\/arazin\/?p=1691"},"modified":"2026-10-21T10:49:57","modified_gmt":"2026-10-21T02:49:57","slug":"from-fem-to-pinns-a-new-research-direction","status":"publish","type":"post","link":"https:\/\/people.utm.my\/arazin\/2026\/10\/21\/from-fem-to-pinns-a-new-research-direction\/","title":{"rendered":"From FEM to PINNs: A New Research Direction"},"content":{"rendered":"
\n

AI from First Principles · Week 06 Revisited<\/p>\n

I’ll be honest up front: this is the topic I’m most excited about in the whole course. Here’s why Week 06 matters, what the results tell us, what they don’t — and an open invitation at the end for students who want to work on it with me.<\/p>\n

Session date: 7 October 2026 · Week 06 on the MSNM course page<\/a><\/p>\n

From data-driven to physics-informed<\/h2>\n

For five weeks, every network we built learned the same way: give it inputs, give it the right answers, and let it close the gap. In Week 04<\/a>, for example, the CNN surrogate needed 5,000 FE solutions before it could predict anything.<\/p>\n

Week 06 flipped that around. A Physics-Informed Neural Network (PINN) doesn’t need a dataset of solutions at all. Instead, the governing equation, boundary conditions and initial conditions go straight into the loss function. The network is penalised whenever it breaks the physics<\/em>, not whenever it disagrees with a label.<\/p>\n

For anyone who has spent years writing FEM or meshfree codes, that should sound very familiar. And that’s exactly why I find it so interesting.<\/p>\n

\"Week
Tuesday night, 7 October — the PINN solution surface over the x<\/i>–t<\/i> domain, built entirely in NumPy.<\/figcaption><\/figure>\n

The problem<\/h2>\n

We used the classic one-dimensional wave equation, kept in dimensional (unscaled) form on purpose for Part A:<\/p>\n

∂²u<\/i>\/∂t<\/i>² − c<\/i>² ∂²u<\/i>\/∂x<\/i>² = 0<\/p>\n

with fixed ends, u<\/i>(0,t<\/i>) = u<\/i>(1,t<\/i>) = 0, and a string released from rest in its first mode, u<\/i>(x<\/i>,0) = sin(πx<\/i>) and ∂u<\/i>\/∂t<\/i>(x<\/i>,0) = 0. With L<\/i> = c<\/i> = A<\/i> = 1, the analytical solution is simply u<\/i> = sin(πx<\/i>) cos(πt<\/i>), so we always know exactly how wrong the network is.<\/p>\n

The network itself is as small as it gets: inputs x<\/i> and t<\/i>, one hidden layer of m<\/i> tanh neurons, and a single output:<\/p>\n

û<\/i>(x<\/i>,t<\/i>) = Σj<\/i>=1<\/sub>m<\/i><\/sup> v<\/i>j<\/i><\/sub> tanh(w<\/i>2j<\/i>−1<\/sub>x<\/i> + w<\/i>2j<\/i><\/sub>t<\/i> + b<\/i>j<\/i><\/sub>)<\/p>\n

What I really liked<\/h2>\n

1. Everything by hand. No autodiff, no PyTorch.<\/h3>\n

This is the part that makes the course unusual. Every derivative of the network, every residual, and all twelve loss gradients (four parameter types × three losses) were derived on paper first, then coded line by line in NumPy with the equation numbers written next to each line. No loss.backward()<\/code> hiding anything.<\/p>\n

It’s slow, and it’s meant to be. Once you’ve derived ∂L<\/i>D<\/sub>\/∂b<\/i>k<\/i><\/sub> yourself and seen the third<\/em> derivative of tanh appear in it, you understand why PINNs are expensive in a way no tutorial can teach you.<\/p>\n

2. “The derivative of a network is just another network”<\/h3>\n

This was my favourite moment. Dr. Airil drew the derivatives as four networks — A, B, C and D — sharing exactly the same inputs and hidden layer. The only things that change are the activation and the output weights:<\/p>\n

\n\n\n\n\n\n\n\n\n
Network<\/th>\nComputes<\/th>\nHidden activation<\/th>\nOutput weight<\/th>\n<\/tr>\n<\/thead>\n
A<\/td>\nû<\/i><\/td>\nσ<\/td>\nv<\/i>j<\/i><\/sub><\/td>\n<\/tr>\n
B<\/td>\n∂û<\/i>\/∂t<\/i><\/td>\nσ′<\/td>\nv<\/i>j<\/i><\/sub>w<\/i>2j<\/i><\/sub><\/td>\n<\/tr>\n
C<\/td>\n∂²û<\/i>\/∂x<\/i>²<\/td>\nσ″<\/td>\nv<\/i>j<\/i><\/sub>w<\/i>2j<\/i>−1<\/sub>²<\/td>\n<\/tr>\n
D<\/td>\n∂²û<\/i>\/∂t<\/i>²<\/td>\nσ″<\/td>\nv<\/i>j<\/i><\/sub>w<\/i>2j<\/i><\/sub>²<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

So the domain residual is just R<\/i>D<\/sub> = Network D − c<\/i>² × Network C, evaluated at a collocation point. Once you see it drawn this way, it’s hard to un-see.<\/p>\n

3. It looks a lot like our own methods<\/h3>\n

This is where my own research brain switched on. Put the PINN next to things we already know:<\/p>\n