What are Qualcomm QNN HTP/DSP Delegates?

For Snapdragon 888 based devices like Sony Xperia 1 III, and One Plus 9 Pro, the ranking table states that benchmark data were collected using the Qualcomm QNN HTP or DSP delegate. Is this different from the NNAPI delegate or Hexagon delegate? Where can I find documentation for the QNN delegate? How do I enable the QNN delegate in Tensorflow Lite?

Googling "tensorflow lite QNN delegate" yields nothing.


Thank you.

I am also interested in the topic. Is the QNN delegate a custom delegate developed around the SNPE SDK?

Hi @Dwayne Duane and @Hachimaki,

Hachimaki said:

Is the QNN delegate a custom delegate developed around the SNPE SDK?

Click to expand...

QNN is a new delegate developed by Qualcomm, it is targeted at the TFLite Hexagon replacement. There obviously exists some connection between the QNN and SNPE, but we cannot say much more than that now.

Dwayne Duane said:

Where can I find documentation for the QNN delegate?

Click to expand...

Unfortunately, there is still no public information or documentation for it. Nevertheless, we expect that it will become public at some point, but don't know the exact dates.

For now, you can use AI Benchmark's PRO mode to check its performance for custom NN architectures.

@Andrey Ignatov that is exciting news! Earlier this year, we had to abandon SNPE in favor of TFLite+NNAPI when we found out that having SNPE leverage the Hexagon DSP from an Android App with targetSDK > 28 is an intractable problem due to issues like:


Hopefully the QNN delegate manages to address these shortcomings of SNPE.

@Andrey Ignatov Qualcomm has finally revealed its distribution policy regarding QNN: https://www.qualcomm.com/support/so...82ec93b7-601d-4b88-a87e-edea93e00241#overview

It is only available to members of "verified companies". It's not clear yet which companies will be verified, but it's obvious that free and hobbyist developers are ruled out. Since most android apps originate from those, this will be very detrimental for mobile AI. Moreover, this means that you buy a powerful NPU for a lot of money and then are not allowed to use it (particularly since Qualcomm has obviously stopped to support NNAPI on the NPU in the 8gen2 chipset, see https://browser.geekbench.com/search?k=ml_inference&q=kalama , where NNAPI has roughly same performance as CPU and is much worse than on 8gen1 chipset).

For your benchmark this means, that results achieved with QNN suggest a performance, which will be only available to selected apps. So they are somewhat misleading. Therefore I think, it might be better not to use QNN in AI Benchmark. Considering the impact, that your benchmark has, this might lead to Qualcomm rethinking its current strategy. And this would be very beneficial to mobile AI.

Hi @Andreas Kirmse,

Andreas Kirmse said:

It is only available to members of "verified companies"

Click to expand...

As far as we know, you should be able to download QNN SDK from Qualcomm Createpoint after registering there. However, it might not yet be available for all users now.

Andreas Kirmse said:

particularly since Qualcomm has obviously stopped to support NNAPI on the NPU in the 8gen2 chipset

Click to expand...

Well, there is a good reason for this. NNAPI has lots of issues that were not solved by Google, and writing NNAPI drivers for newer NPUs is extremely complicated. You might notice that only MediaTek is still able to do this, while the majority of other vendors are no longer investing their efforts. For smaller companies the situation is even worse: they basically need to have a dedicated team only for NNAPI, which is not feasible. So, it's not surprising that everyone is now trying to focus on custom delegates that are much simpler to implement, easier to maintain, have a broader coverage, better functionality, and are independent of Google.