How can we jump by the Leap-frog method

Let me summarize the basic idea of the Leap-frog method here. The idea is from the famous Japanese text book on Bayesian modeling, so called another "Green book".

Potential energy and Momentum energy

The potential energy $U(\tau)$ is proportional to the height $h$ (consider the height as negative log-likelihood!), and suppose $h$ can be determined by where the parameter $\theta$ is, then $U(\tau)=mgh\left(\theta(\tau)\right)$, where $m$ is mass and $g$ is gravitational acceleration. The momentum energy $K(\tau)$ is a function of the momentum $p(\tau)$, i.e., $K(\tau)=\frac{1}{2m}p(\tau)^2$. Assume $m=g=1$, we have: $$\begin{align} U(\tau) & =h\left(\theta(\tau)\right) \nonumber \\ K(\tau) & = \frac{1}{2}p(\tau)^2 \nonumber \end{align}$$

Hamiltonian

Let $H(\tau)$ be the Hamiltonian at time $\tau$ which is the sum of the potential energy $U(\tau)$ and the momentum energy $K(\tau)$, i.e., $H(\tau)=U(\tau)+K(\tau)$, and $H(\tau)$ must be constant, i.e., $\displaystyle \frac{d H(\tau)}{d \tau}=0 \Longleftrightarrow \frac{d K(\tau)}{d \tau}=-\frac{d U(\tau)}{d \tau}$. Consider the product rule of derivative, we get: $$\begin{align} \frac{d K(\tau)}{d \tau} &= -\frac{d U(\tau)}{d \tau} \nonumber \\ \Leftrightarrow \frac{d K(\tau)}{d p(\tau)}\frac{d p(\tau)}{d \tau} &= -\frac{d U(\tau)}{d \theta{(\tau)}}\frac{d \theta{(\tau)}}{d \tau}. \nonumber \end{align}$$ We have the following simultaneous differential equation: $$\begin{align} \frac{d p(\tau)}{d \tau} &= -\frac{d U(\tau)}{d \theta(\tau)} = -\frac{d h(\tau)}{d \theta(\tau)}:= - h'(\theta(\tau)) \nonumber \\ \frac{d \theta(\tau)}{d \tau} &= \frac{d K(\tau)}{d p(\tau)} = p(\tau). \nonumber \end{align}$$

Euler's method

Let $\epsilon$ be a small number for numerical calculation, and consider the time $\tau + 1$, the simultaneous differential equation can be approximated as: $$\begin{align} p(\tau + 1) &= p(\tau ) - \epsilon h'(\theta(\tau)) \nonumber \\ \theta(\tau + 1) &= \theta(\tau) + \epsilon p(\tau). \nonumber \end{align}$$ However, the approximation is not efficient, there is better way to approximate.

Leap-frog method

Consider the time $\tau + 1/2$, and introduce the following approximation: $$\begin{align} p\left(\tau + \frac{1}{2}\right) &= p(\tau ) - \frac{\epsilon}{2} h'(\theta(\tau)) \nonumber \\ \theta(\tau + 1) &= \theta(\tau) + \epsilon p\left(\tau + \frac{1}{2}\right). \nonumber \\ p(\tau + 1) &= p\left(\tau + \frac{1}{2}\right) - \frac{\epsilon}{2} h'(\theta(\tau+1)). \end{align}$$