Let me summarize the basic idea of the Leap-frog method here. The idea is from the famous Japanese text book on Bayesian modeling, so called another
"Green book".
Potential energy and Momentum energy
The potential energy \(U(\tau)\) is proportional to the height \(h\) (consider the height as negative log-likelihood!), and suppose \(h\) can be determined by where the parameter \(\theta\) is, then \(U(\tau)=mgh\left(\theta(\tau)\right)\), where \(m\) is mass and \(g\) is gravitational acceleration. The momentum energy \(K(\tau)\) is a function of the momentum \(p(\tau)\), i.e., \(K(\tau)=\frac{1}{2m}p(\tau)^2\). Assume \(m=g=1\), we have:
\begin{align}
U(\tau) & =h\left(\theta(\tau)\right)
\nonumber \\
K(\tau) & = \frac{1}{2}p(\tau)^2
\nonumber
\end{align}
Hamiltonian
Let \(H(\tau)\) be the Hamiltonian at time \(\tau\) which is the sum of the potential energy \(U(\tau)\) and the momentum energy \(K(\tau)\), i.e., \(H(\tau)=U(\tau)+K(\tau)\), and \(H(\tau)\) must be constant, i.e., \(\displaystyle \frac{d H(\tau)}{d \tau}=0 \Longleftrightarrow \frac{d K(\tau)}{d \tau}=-\frac{d U(\tau)}{d \tau}\). Consider the product rule of derivative, we get:
\begin{align}
\frac{d K(\tau)}{d \tau} &= -\frac{d U(\tau)}{d \tau}
\nonumber \\
\Leftrightarrow \frac{d K(\tau)}{d p(\tau)}\frac{d p(\tau)}{d \tau} &= -\frac{d U(\tau)}{d \theta{(\tau)}}\frac{d \theta{(\tau)}}{d \tau}.
\nonumber
\end{align}
We have the following simultaneous differential equation:
\begin{align}
\frac{d p(\tau)}{d \tau} &= -\frac{d U(\tau)}{d \theta(\tau)} = -\frac{d h(\tau)}{d \theta(\tau)}:= - h'(\theta(\tau))
\nonumber \\
\frac{d \theta(\tau)}{d \tau} &= \frac{d K(\tau)}{d p(\tau)} = p(\tau).
\nonumber
\end{align}
Euler's method
Let \(\epsilon\) be a small number for numerical calculation, and consider the time \(\tau + 1\), the simultaneous differential equation can be approximated as:
\begin{align}
p(\tau + 1) &= p(\tau ) - \epsilon h'(\theta(\tau))
\nonumber \\
\theta(\tau + 1) &= \theta(\tau) + \epsilon p(\tau).
\nonumber
\end{align}
However, the approximation is not efficient, there is better way to approximate.
Leap-frog method
Consider the time \(\tau + 1/2\), and introduce the following approximation:
\begin{align}
p\left(\tau + \frac{1}{2}\right) &= p(\tau ) - \frac{\epsilon}{2} h'(\theta(\tau))
\nonumber \\
\theta(\tau + 1) &= \theta(\tau) + \epsilon p\left(\tau + \frac{1}{2}\right).
\nonumber \\
p(\tau + 1) &= p\left(\tau + \frac{1}{2}\right) - \frac{\epsilon}{2} h'(\theta(\tau+1)).
\end{align}
Comments
Post a Comment