Download or read book Perceptual Optimization and Complexity Reduction for Real-time Video Encoding written by Madhukar Bhat and published by . This book was released on 2021 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: This thesis explores perceptual optimization and complexity reduction towards improving video compression schemes in a real-time encoding framework context. Different strategies are proposed to address perceptual enhancements and complexity reduction. The work is divided into three main parts that tackle several aspects to improve real-time video encoding. The first contribution introduces a perceptual pre-processing filter based on a model of the Human Visual System. This preprocessing filter can be tuned and has been optimized for multiple viewing conditions. An investigation of different objective quality metric's accuracy in measuring the performance of pre-processing is also reported. The second part of the thesis proposes a machine learning classification-based methodology to adaptively predict and select the best encoding resolution in a single-pass encoding scenario. For this purpose, three different classifiers have been considered: Support Vector Machine, Random Forest (RF), and Multi-Layer Perceptron. Several objective quality metrics have been characterized and benchmarked for various quality ranges concerning the uncertainty of subjective quality ground-truth to evaluate and drive the perceptual gain at a given bitrate. Also, a new, subjectively more accurate RFbased fusion metric has been introduced for ground truth dataset annotation of classifiers. The final part focuses on tackling the most challenging encoding part of the new Versatile Video Coding (VVC) standard: the Multi-Type Tree partitioning decision. A new ML-based partitioning decision criteria is proposed covering all standard partitioning possibilities for both intra and inter, especially suitable for realtime hardware encoder. A CNN-based approach was considered, offering classifiers for different coding unit depth, shapes, and picture types.