TY - JOUR
T1 - Temporally coherent superresolution of textured video via dynamic texture synthesis
AU - Hsu, Chih Chung
AU - Kang, Li Wei
AU - Lin, Chia Wen
N1 - Publisher Copyright:
© 1992-2012 IEEE.
PY - 2015/3/1
Y1 - 2015/3/1
N2 - This paper addresses the problem of hallucinating the missing high-resolution (HR) details of a low-resolution (LR) video while maintaining the temporal coherence of the reconstructed HR details using dynamic texture synthesis (DTS). Most existing multiframe-based video superresolution (SR) methods suffer from the problem of limited reconstructed visual quality due to inaccurate subpixel motion estimation between frames in an LR video. To achieve high-quality reconstruction of HR details for an LR video, we propose a texture-synthesis (TS)-based video SR method, in which a novel DTS scheme is proposed to render the reconstructed HR details in a temporally coherent way, which effectively addresses the temporal incoherence problem caused by traditional TS-based image SR methods. To further reduce the complexity of the proposed method, our method only performs the TS-based SR on a set of key frames, while the HR details of the remaining nonkey frames are simply predicted using the bidirectional overlapped block motion compensation. After all frames are upscaled, the proposed DTS-SR is applied to maintain the temporal coherence in the HR video. Experimental results demonstrate that the proposed method achieves significant subjective and objective visual quality improvement over state-of-the-art video SR methods.
AB - This paper addresses the problem of hallucinating the missing high-resolution (HR) details of a low-resolution (LR) video while maintaining the temporal coherence of the reconstructed HR details using dynamic texture synthesis (DTS). Most existing multiframe-based video superresolution (SR) methods suffer from the problem of limited reconstructed visual quality due to inaccurate subpixel motion estimation between frames in an LR video. To achieve high-quality reconstruction of HR details for an LR video, we propose a texture-synthesis (TS)-based video SR method, in which a novel DTS scheme is proposed to render the reconstructed HR details in a temporally coherent way, which effectively addresses the temporal incoherence problem caused by traditional TS-based image SR methods. To further reduce the complexity of the proposed method, our method only performs the TS-based SR on a set of key frames, while the HR details of the remaining nonkey frames are simply predicted using the bidirectional overlapped block motion compensation. After all frames are upscaled, the proposed DTS-SR is applied to maintain the temporal coherence in the HR video. Experimental results demonstrate that the proposed method achieves significant subjective and objective visual quality improvement over state-of-the-art video SR methods.
KW - Video super-resolution
KW - dynamic texture synthesis
KW - motion-compensated interpolation
KW - video hallucination
KW - video upscaling
UR - http://www.scopus.com/inward/record.url?scp=84921875963&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84921875963&partnerID=8YFLogxK
U2 - 10.1109/TIP.2014.2387416
DO - 10.1109/TIP.2014.2387416
M3 - Article
AN - SCOPUS:84921875963
SN - 1057-7149
VL - 24
SP - 919
EP - 931
JO - IEEE Transactions on Image Processing
JF - IEEE Transactions on Image Processing
IS - 3
M1 - 7001251
ER -