Real-Time Image Super-Resolution Challenge

xindongzhang

New member
Your output is already uint8 by definition (since quantized model can output only these values), therefore no rescaling is needed here if quantization is done correctly.
Furthermore, I have test the quantize script you provided. If we did not dequantize the output of uint8-model, large numerical error of outputs from fp32-model and uint8-model were introduced. However when scaling the output of uint8-model by zero-point & scale, the result of uint8-model are quite close to that of fp32-model.
 

xindongzhang

New member
The quantized model without the following two lines loses small psnr, but if I uncomment them, psnr decreases from 29dB to 20dB. I want to know whether it is normal?
[1].converter.inference_input_type = tf.uint8
[2].converter.inference_output_type = tf.uint8
From my experiments, one main reason is that the output from uint8-model should be further de-quantized by "zero point & scale" of output node. If you do that, the loss of psnr should be acceptable。
 

JetDZC

New member
Thanks, I understand what you mean. But uint8 is only the bound of quantized Op, it is not the actually space of fp32-output. Only if we de-quantize the output of network, we then get the correct numerical range and result.
Yes, we exactly need to do de-quantization. How can we achieve this?
 

xindongzhang

New member
Yes, we exactly need to do de-quantization. How can we achieve this?
I still have no idea, just waiting for the official reply. As my experiment from the offical scripts, large quantized error to fp32-model was introduces if we didn't de-quantize the output node.
 

xindongzhang

New member
Your output is already uint8 by definition (since quantized model can output only these values), therefore no rescaling is needed here if quantization is done correctly.
Could you please check https://www.tensorflow.org/lite/performance/post_training_integer_quant. For the uint8-inference, the input still needs to be further quantized before fed into the uint8-network(If the mean and std of input hapends to be (0.0, 1,0), you could straightly feed it to the uint8-network). As for the output node of uint8-network, de-quantization step is still needed, otherwise it should be an imcomplete mapping. Accordingi to the implementation of tensorflow, the complete mapping should be <input ---> quantized to int8/uint8 ---> fully-quantized-network--->dequantized output ---> output>. Please give us a hint or conclusion, thanks a lot !
 

Andrey Ignatov

Administrator
Staff member
it is not the actually space of fp32-output

In terms of the outputs - the range is exactly the same: uint8, [0, 255] interval.

As for the output node of uint8-network, de-quantization step is still needed, otherwise it should be an imcomplete mapping
Yes, we exactly need to do de-quantization. How can we achieve this?

Dequantization is a floating-point op, you cannot run it on integer NPUs / DSPs. If you explicitly specify that the output should be int8 - this is taken into account during quantized aware training or post-training quantization, thus no dequantization is needed.

If you do that, the loss of psnr should be acceptable

This won't really change too much - the difference will mainly come from the fact that your outputs are floating-point, and not rounded to the closest integer values. All visual results will be basically the same.
 

xindongzhang

New member
In terms of the outputs - the range is exactly the same: uint8, [0, 255] interval.




Dequantization is a floating-point op, you cannot run it on integer NPUs / DSPs. If you explicitly specify that the output should be int8 - this is taken into account during quantized aware training or post-training quantization, thus no dequantization is needed.



This won't really change too much - the difference will mainly come from the fact that your outputs are floating-point, and not rounded to the closest integer values. All visual results will be basically the same.
Thanks for your reply. I have checked the "output scale and zero point" of the output node based your provided scripts? Both the input and output were in the uint8 type, it does change a a lot if you did not de-quantize the output node. Let's put it simply, fully-quantized network can make sure that the sub-graph (without quantization of input & de-quantization of output) can be deployed on-devices, it didn't mean that the quantization of input & de-quantization of output could be neglected.
 

Andrey Ignatov

Administrator
Staff member
it didn't mean that the quantization of input & de-quantization of output could be neglected.

You should probably check this section of the tutorial that you sent before, especially the last sentence about:

Now you have an integer quantized model that uses integer data for the model's input and output tensors, so it's compatible with integer-only hardware such as the Edge TPU.

In other words - your model is either fully quantized and executable on integer NPUs, or it uses dequant ops and is not runnable on them. When you specify that the inputs and outputs should be quantized, TFLite converter is adjusting the values of these tensors as well so that you don't need to apply any additional de-quant steps.
 

xindongzhang

New member
You should probably check this section of the tutorial that you sent before, especially the last sentence about:
Thanks for your reply. I have checked this before, the sample of this code is about classification. We don't need to decode the result, since linear transformation does not affect the argmax of the output and predicted class. However the SR task is quite different from classification problem, if we neglect the de-quantization stage, it indeed would introduce large impact on the quality of restored SR image.
 

xindongzhang

New member
If the mean and std of the output node is not close to (0.0, 1.0), simply restore results by uint8-output would make the SR image darker or brighter from visual perception.
 

JetDZC

New member
Thanks for your reply. I have checked this before, the sample of this code is about classification. We don't need to decode the result, since linear transformation does not affect the argmax of the output and predicted class. However the SR task is quite different from classification problem, if we neglect the de-quantization stage, it indeed would introduce large impact on the quality of restored SR image.
Couldn't agree with you more!
 

Andrey Ignatov

Administrator
Staff member
However the SR task is quite different from classification problem, if we neglect the de-quantization stage, it indeed would introduce large impact on the quality of restored SR image

That's correct, and it is great that you finally got the difficulty of the proposed task. However, this does not change the fact that 1) the model should be fully quantized in order to run on INT8 NPUs, 2) you were provided with two months to solve this task and all accompanying issues, 3) one can possibly change mean and std values in the majority of nodes, but this goes beyond using a simple post-training quantization script.

So, quantization is all about research and not just applying some standard built-in functions and getting perfect results. INT8 quantization for image-to-image mapping is even a more challenging problem, where no ideal solution exists yet. Hoping to solve it in the last 12 hours of the challenge is generally not a good idea, and therefore we were mentioning in all tutorials that you should always check the results obtained with your quantized model, and not just submitting the outputs of the floating-point network and hoping that everything works fine. The questions you are asking above refer to the nature of the problem itself rather than to some issues with quantization codes or tools.
 

xindongzhang

New member
That's correct, and it is great that you finally got the difficulty of the proposed task. However, this does not change the fact that 1) the model should be fully quantized in order to run on INT8 NPUs, 2) you were provided with two months to solve this task and all accompanying issues, 3) one can possibly change mean and std values in the majority of nodes, but this goes beyond using a simple post-training quantization script.

So, quantization is all about research and not just applying some standard built-in functions and getting perfect results. INT8 quantization for image-to-image mapping is even a more challenging problem, where no ideal solution exists yet. Hoping to solve it in the last 12 hours of the challenge is generally not a good idea, and therefore we were mentioning in all tutorials that you should always check the results obtained with your quantized model, and not just submitting the outputs of the floating-point network and hoping that everything works fine. The questions you are asking above refer to the nature of the problem itself rather than to some issues with quantization codes or tools.
Thanks for you reply, and the hints you provided may be very insightful and good research problems.
 
Last edited:

deepernewbie

New member
Have you tried de-quantizing the uint8-output node first?
I have solved my problem, if you need some sort of linear mapping include it at the end of the model as a linear operator and as a floating point operator assuimg ranges 0-1 then use the tflite quantization it takes care of everything and calculates quantizations differently approximates the floating point operation as a fixed point operation and outputs in 0-255 range.

and that's what the challenge requires from a submission (IMHO)

As a guidance to whom willing to keep working in this area, The problem in this challenge can be approched from many different aspects including

Quantization Robust SR Model Design (Smart Design)
Non-disturbing Quantization Method Design (Smart Quantization)
Pretrained Heavy model to Lite Model Design with Quantization in Mind (Smart Prunning/Sparsification)
Pretrained Heavy model guiding Lite Model Optimization (Smart Optimization)
...

regarding to quantization and my insight to the problem from the point of the view of optimization and why not every model is suitable for uint8 conversion in SR problem, I am planing to submit in a challenge paper if (BIG IF) I am invited to send one, so hope to see you in workshop proceedings :)
 

xindongzhang

New member
I have solved my problem, if you need some sort of linear mapping include it at the end of the model as a linear operator and as a floating point operator assuimg ranges 0-1 then use the tflite quantization it takes care of everything and calculates quantizations differently approximates the floating point operation as a fixed point operation and outputs in 0-255 range.

and that's what the challenge requires from a submission (IMHO)

As a guidance to whom willing to keep working in this area, The problem in this challenge can be approched from many different aspects including

Quantization Robust SR Model Design (Smart Design)
Non-disturbing Quantization Method Design (Smart Quantization)
Pretrained Heavy model to Lite Model Design with Quantization in Mind (Smart Prunning/Sparsification)
Pretrained Heavy model guiding Lite Model Optimization (Smart Optimization)
...

regarding to quantization and my insight to the problem from the point of the view of optimization and why not every model is suitable for uint8 conversion in SR problem, I am planing to submit in a challenge paper if (BIG IF) I am invited to send one, so hope to see you in workshop proceedings :)
Thanks for you sharing, I have solved this problems too in the last 10 hours with very simple revise of network, only just one line of code. I really appreciate it in the four days journey of this competition.
 

deepernewbie

New member
Hi when should we expect the preliminary results?

This is what the page states "03.23.2021 Preliminary test results release to the participants"

Is it going to be later today or any change in the initial plans?
 
Top