question about the test_video.py

[CauchyHu]

I am using your code to reproduce the experimental results of the ESPCN paper, but I find that the results are quite different from those in the paper. Therefore, I speculate whether the input image only has y value and no CB and Cr value into the model, which will lead to the final image can not achieve the effect of the paper?
this is my results:

[windmaple]

I'm also trying to reproduce the results in the paper. I also had issue getting the same level of quality (PyTorch official example repo also has an implementation; quality isn't that great either). Were you able to get good quality for images?

But I think what you mentioned here is expected. If you read the paper closely, you will see it does only use Y channel. Cb/Cr are upsampled using bicubic interpolation.

Having said that, I think the main diff is that the original paper uses a very small patch size (17x17). I haven't been able to confirm this though.

Btw, this implementation uses diff activation functions as well, although that doesn't make much diff per my testing.

[CauchyHu]

I'm also trying to reproduce the results in the paper. I also had issue getting the same level of quality (PyTorch official example repo also has an implementation; quality isn't that great either). Were you able to get good quality for images?

But I think what you mentioned here is expected. If you read the paper closely, you will see it does only use Y channel. Cb/Cr are upsampled using bicubic interpolation.

Having said that, I think the main diff is that the original paper uses a very small patch size (17x17). I haven't been able to confirm this though.

Btw, this implementation uses diff activation functions as well, although that doesn't make much diff per my testing.

Do you have a good solution now? I have tried to remake the training samples, but it has not improved much. I doubt whether there is a problem with the training sample cropping method in data_utils.py?

[windmaple]

Unfortunately no. I have tried half a dozen TF/PyTorch implementation on GitHub. None of them could get even close to what was reported in the paper.

[543877815]

@windmaple @CauchyHu
I've done several experiments recently, I think the key points is the about:

1. Data augmentation method. It seems that the large size of patches is very hard to learn. Thus, cropping big images into the small size of patches is necessary.
2. The PSNR/SSIM/IFC evaluation in the paper is probably using MatLab in https://github.com/jbhuang0604/SelfExSR/blob/master/quant_eval/compute_difference.m. Btw, the method that converts yCrCb to RGB is quite different between Pillow in python and Matlab. The latter one will get higher performance

[543877815]

My result using data augmentation is as above. Please let me know if reproducing a better result :)

[windmaple]

直接用OpenCV好了，已经有训练好的ESPCN
https://docs.opencv.org/master/d9/de0/group__dnn__superres.html