Teacher Student Network for Real-world Face Super-resolution with Progressive Embedding of Edge Information

[Audio] Good morning everyone. Today, my colleague Chenggong Zhang and I are pleased to present our study on how Teacher Student Network can enable real-world face super-resolution with progressive embedding of edge information. We have developed a new methodology combining teacher-student models and developed a novel CNN network, which can integrate facial super-resolution and edge information progressively. As a result, the super-resolution on real-world faces is improved dramatically in terms of image quality and computation efficiency. Our approach and results have been published in the IEEE ICIP 2023 Tutorial..

[Audio] The project I am about to present is Zhilei Liu and Chenggong Zhang's Teacher Student Network for Real-world Face Super-resolution with Progressive Embedding of Edge Information at IEEE ICIP 2023 Tutorial. The main objectives of this project are to enhance the accuracy and speed of real-world face super-resolution by proposing a new Teacher Student Network. To do this, our motivations will be introduced before proposing our method. Experiments and discussion will follow before concluding. I hope the information provided is beneficial for all here..

[Audio] We will be discussing an important paper from IEEE ICIP 2023 Tutorial on face analysis techniques. The paper is entitled 'Zhilei Liu and Chenggong Zhang presented their Teacher Student Network for Real-world Face Super-resolution with Progressive Embedding of Edge Information'. It has a focus on face analysis in the wild, where the resolution of the face images is often quite low. This presents a challenge as reliable results are hard to be obtained. To address this, the authors proposed a teacher-student network for real-world face super-resolution with progressive embedding of edge information. We will discuss this further as the presentation progresses..

[Audio] Zhilei Liu and Chenggong Zhang proposed the Teacher Student Network for Real-world Face Super-resolution with Progressive Embedding of Edge Information. The network combines low-level feature learning and high-level prior information for better recognition and restoration results. It takes a low-resolution face image and its low-level prior information, like edge maps, to reconstruct a detailed high-resolution face image. The progressive embedding of facial prior information allows the model to capture facial geometry and semantic features of the target domain. This approach is resistant to the gap between synthetic and real data and therefore suitable for real-world FSR applications..

[Audio] We will be discussing a new face super-resolution model, based on a teacher-student network, which embeds progressive edge information. This model employs pseudo-paired data generated using a degraded network to discover the real degradation process and train a teacher super-resolution network. The teacher-student framework generates pseudo high-resolution images for real low-resolution images to reconstruct the face structure whilst preserving the pixel-level accuracy of the student network. To enhance the restoration of the global shape and local details in the face image, the edge information is embedded progressively in the reconstruction process in combination with traditional image processing methods..

[Audio] Today, I'm going to explain the Teacher Student Network for Real-world Face Super-resolution with Progressive Embedding of Edge Information. Firstly, we use a Degradation modeling Network (DeNet) to generate an LR image that conforms to the LR real LR domain distribution as much as possible. Then, the generated LR-HR image pairs are used to train our Teacher super-resolution Network (TNet). To further reduce the domain gap between synthetic LR images and real LR images, we design a Student Network (SNet) that is specially used for super-resolution reconstruction in the real LR domain. Additionally, we use a Down-Sampling Network (DSNet) to downsample to the LR domain SR, as well as an adversarial learning manner to make the generated image more realistic. Therefore, by leveraging the edge embedding information and the domain knowledge of the real LR image, our proposed network can produce more real and accurate SR results..

[Audio] I'm going to discuss the Teacher-Student Network for Real-world Face Super-resolution with Progressive Embedding of Edge Information developed by Zhilei Liu and Chenggong Zhang. To ensure the generated low-resolution image is consistent with the high-resolution image, we use a content loss to restrict the difference between Igen and Ibic which is obtained by bicubic downsampling from the HR image. An adversarial loss is also imposed between Ireal and Igen to generate a realistic LR image. To further refine the results, the TNet applies a reconstruction loss. In addition, CycleGAN and DSNet introduce cycle consistency constraints, which further promote reconstruction of real LR images. In conclusion, the Content Loss and Adversarial Loss are used in the DeNet, the Reconstruction Loss is used in the SNet and the Cycle Consistency Constraints are used in the TNet to achieve the best possible results..

[Audio] Today, I would like to discuss the results of Zhilei Liu and Chenggong Zhang presented their Teacher Student Network for Real-world Face Super-resolution with Progressive Embedding of Edge Information at the IEEE ICIP 2023 Tutorial. This work explores the usage of a Teacher Student Network architecture and its application to real-world face super-resolution. It uses two sets of datasets to experiment, the FFHQ dataset containing 20000 high-quality, high-resolution face images and the Widerface dataset containing 4000 low-resolution face images for training. For testing, two synthetic datasets, i.e. LS3D-W balanced and FFHQ, and two real-world datasets – Widerface and Webface – are used. Four metrics – FID, LPIPS, PSNR, and SSIM – are utilized for quantitative evaluation for two real-world datasets with corresponding high-resolution reference images. Accuracy and F1-score are used to evaluate AU Detection. Results are encouraging, and this research serves as a significant contribution to the field of face super-resolution..

[Audio] Our proposed method shows excellent results when compared to other existing state-of-the-art methods. Our method was the most successful on two datasets, achieving the highest PSNR and SSIM values on both, as well as the lowest LPIPS value on the FFHQ dataset. On the two datasets, our FID was the second best and was very competitive with the SCGAN method..

[Audio] In the study of Teacher Student Network for Real-world Face Super-resolution with Progressive Embedding of Edge Information presented at IEEE ICIP 2023 Tutorial, Zhilei Liu and Chenggong Zhang achieved excellent results. As demonstrated in Table 2, their method successfully captured the distribution of real face images and natural images, leading to the lowest NIQE value and the second lowest FID value on the two datasets. It is worth noting that these 'perceptual' metrics may not always map onto human-opinion-scores on a finer scale..

[Audio] Research conducted by Zhilei Liu and Chenggong Zhang at the IEEE ICIP 2023 Tutorial investigated the use of Teacher Student Network for Real-world Face Super-resolution with Progressive Embedding of Edge Information. Qualitative results they showed in figure 3 showed the method was successful in capturing expressions (first column), skin color (second column), and face features and details. On real-world datasets, their method was seen to recover face structures such as contours, eyes, and mouth in a more natural way than other methods. Comparison of their results to those achieved with LR images and Bicubic showed their method was better than LRGAN and SCGAN in terms of skin color and overall color restoration, indicating the significance of their findings..

[Audio] We explore the effectiveness of each part of the proposed method by performing ablation studies on the FFHQ dataset and the Widerface dataset. Our baseline model does not incorporate Canny or Gaussian difference detail edges, however, as seen in the upper part of Table 3, both of these edges are effective in improving the metrics. The lower part of Table 3 shows the results for all variants, further confirming the contribution of each branch network..

[Audio] A new teacher-student network for face super-resolution with progressive embedding of edge information was presented in this paper. This method was shown to achieve superior results compared to existing approaches. The network was trained with synthetic low-resolution data, generated by the DeNet, and with explicit embedding of edge information. This technique enabled the model to focus more on recovering high-frequency details. Experiments confirmed its effectiveness on face-specific tasks. This research contributes to the field of face super-resolution and provides further insights for applications in real-world scenarios..

[Audio] I hope you found this presentation on Zhilei Liu and Chenggong Zhang's Teacher Student Network for Real-world Face Super-resolution with Progressive Embedding of Edge Information to be informative and insightful. This research was conducted at Tianjin University in China from October 8th to 11th of 2023 and presented at the IEEE ICIP 2023 Tutorial. The Teacher Student Network proposed can be used for real-world face super-resolution with the progressive embedding of edge information. Thank you for your attention..