How does individual category store correlate with speed?

[Dwayne Duane]

Good morning,

Regarding this table https://ai-benchmark.com/ranking.html,
I am not sure I understand how to interpret score columns for individual categories. For example, for the Google Pixel 6, the score for Int 8/NNAPI 1.2 is 39049 while the score for FP16/NNAPI 1.2 is 72832. But if I click on the "View Detailed Results" button, I see that for the same device, the speed for int8 is consistently superior to that of FP16. For example, for MobileNet-V2, int8 takes 1.5ms while fp16 takes 2.1ms; for VGG19, int8 takes 13ms while fp16 takes 33ms, etc. I seem to be missing some part of this picture. Could you shed some light on it?


Thank you.

 

[Andrey Ignatov]

Hi @Dwayne Duane,

I see that for the same device, the speed for int8 is consistently superior to that of FP16

The scores for the FP16 and INT8 categories are not directly comparable to each other. The reason for this is that the INT8 inference is almost always faster, but it is running a different quantized model which accuracy is usually lower than the one of the FP16/32 network. Therefore, the direct comparison between these categories does not make much sense as they are dealing with different model types.