Benchmarking the A311D / VIM3 NPU

endian

New member
I have a vim3 which includes an amlogic A311D chip.

This chip has a npu built in but it does not seem to have been used when benchmarked currently, as the ai benchmark is the same as on the s922x chip which does not have an npu.

What is required to be able to use the npu when running this benchmark?

What would you imagine might be missing for the npu to be used?
 

Andrey Ignatov

Administrator
Staff member
Hi @endian,

What is required to be able to use the npu when running this benchmark?

What would you imagine might be missing for the npu to be used?

The situation with the A311D chipsest is quite complex. First of all, there is no way to access its NPU through Android: it doesn't support Android NN API (NN HAL is missing), there are no custom TensorFlow Lite delegates for this SoC as well as any proprietary SDKs.

Secondly, even when using Linux - you cannot run the standard TF / TFLite models on this platform: you need to compile them using Amlogic's NPU SDK provided upon a request. It also looks like this NPU is supporting a limited number of TFLite ops and can accelerate INT8 inference only, which means that just some standard quantized image classification models can be executed on it.
 

chro

New member
I find out that vendor of NPU (VeriSilicon) created custom TFLite Delegate on their github repository.
After running tflite benchmark I've got 6.5 msec on single thread MobileNet v2 on NPU delegate
 

Andrey Ignatov

Administrator
Staff member
Hi @chro,

Thanks for the info.

NPU (VeriSilicon) created custom TFLite Delegate on their github repository.

Yes, we have some internal plans for including this delegate to one of our next releases, though do not have a concrete timeline for this yet.

I've got 6.5 msec on single thread MobileNet v2 on NPU delegate

That looks reasonable, you can find the results of another board with VeriSilicon NPU (VideoSmart VS680) here: https://ai-benchmark.com/ranking_IoT
 
Top