Neural differential equation

In <a href="/facts/Machine_learning/e0w0XJTu">machine learning</a>, a neural differential equation is a <a href="/facts/Differential_equation/9MGbE3Ca">differential equation</a> whose right-hand side is parametrized by the weights θ of a <a href="/facts/Neural_network_(machine_learning)/6V1jMlkx">neural network</a>. In particular, a neural ordinary differential equation (neural ODE) is an <a href="/facts/Ordinary_differential_equation/ygKGQ8kD">ordinary differential equation</a> of the form

 
 
 
 
 
 
 
 d
 
 
 h
 
 (
 t
 )
 
 
 
 d
 
 t
 
 
 
 =
 
 f
 
 θ
 
 
 (
 
 h
 
 (
 t
 )
 ,
 t
 )
 .
 
 
 {\displaystyle {\frac {\mathrm {d} \mathbf {h} (t)}{\mathrm {d} t}}=f_{\theta }(\mathbf {h} (t),t).}
 
In classical neural networks, layers are arranged in a sequence indexed by natural numbers. In neural ODEs, however, layers form a continuous family indexed by positive real numbers. Specifically, the function 
 
 
 
 h
 :
 
 
 R
 
 
 ≥
 0
 
 
 →
 
 R
 
 
 
 {\displaystyle h:\mathbb {R} _{\geq 0}\to \mathbb {R} }
 
 maps each positive index t to a real value, representing the state of the neural network at that layer.
Neural ODEs can be understood as continuous-time <a href="/facts/Control_system/8CEjIDlS">control systems</a>, where their ability to interpolate data can be interpreted in terms of <a href="/facts/Controllability/p1TLxgLf">controllability</a>.

Neural differential equation open-in-new

Neural differential equation