HEVC intra frame coding based on convolutional neural network

Chia Hung Yeh, Zheng Teng Zhang, Mei Juan Chen, Chih Yang Lin*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

18 Citations (Scopus)


The HEVC standard offers high performance with a lower bitrate for intra frame coding, but still requires many bits. An alternative intra frame coding framework based on convolutional neural network (CNN) is proposed in this paper. Two CNN models, simplified CNN (S-CNN) and complicated CNN (C-CNN), were designed and trained to improve the coding performance of HEVC. In intra frame coding, the trained CNN predicts the residual for reconstructed blocks to enhance visual quality. Due to the high computational complexity of CNN in HEVC encoding, we further explore the tradeoff between computational complexity and coding performance. For S-CNN, an early termination mechanism is proposed to further reduce the HEVC encoding complexity. With regard to C-CNN, a GPU-based heterogeneous architecture is proposed to accelerate CNN processing. Experimental results show that the proposed method with S-CNN achieves bitrate savings of 3.1% with a 37% increase in time cost, and 2.8% in bitrate savings with only a 23% increase in time cost when applying the early termination algorithm. In the case of C-CNN, the execution time of CNN can reach increases in speed of 8.8-17.8 × and a bitrate savings of up to 5.1% on average.

Original languageEnglish
Article number8447192
Pages (from-to)50087-50095
Number of pages9
JournalIEEE Access
Publication statusPublished - 2018 Aug 24


  • CNN
  • Convolutional neural network
  • HEVC
  • high efficiency video coding
  • intra frame coding

ASJC Scopus subject areas

  • General Computer Science
  • General Materials Science
  • General Engineering
  • Electrical and Electronic Engineering


Dive into the research topics of 'HEVC intra frame coding based on convolutional neural network'. Together they form a unique fingerprint.

Cite this