IEEETPAMI2021

Video Snapshot: Single Image Motion Expansion via Invertible Motion Embedding

Qianshu Zhu        Chu Han        Guoqiang Han        Tien-Tsin Wong        Shengfeng He       

IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 43, No. 12, December 2021,
pp. 4491-4504.


Abstract

In this paper, we aim to generate a video preview from a single image by proposing two cascaded networks, the Motion Embedding Network and the Motion Expansion Network. The Motion Embedding Network aims to embed the spatio-temporal information into an embedded image, called video snapshot. On the other end, the Motion Expansion Network is proposed to invert the video back from the input video snapshot. To hold the invertibility of motion embedding and expansion during training, we design four tailor-made losses and a motion attention module to make the network focus on the temporal information. In order to enhance the viewing experience, our expansion network involves an interpolation module to produce a longer video preview with a smooth transition. Extensive experiments demonstrate that our method can successfully embed the spatio-temporal information of a video into one live image, which can be converted back to a video preview. Quantitative and qualitative evaluations are conducted on a large number of videos to prove the effectiveness of our proposed method. In particular, statistics of PSNR and SSIM on a large number of videos show the proposed method is general, and it can generate a high-quality video from a single image.

Paper

(PDF, 77MB)

Supplementary Material

Source Code


Video

BibTex:

    @article{zhu-2021-video,
        author   = {Qianshu Zhu and Chu Han and Guoqiang Han and Tien-Tsin Wong and Shengfeng He},
        title    = {Video Snapshot: Single Image Motion Expansion via Invertible Motion Embedding},
        journal  = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
        month    = {December},
        year     = {2021},
        volume   = {43},
        number   = {12},
        pages    = {4491-4504}
    }