News, Releases and More

What's New:

• Updated TFLite GPU, Qualcomm QNN, MediaTek Neuron and Hexagon NN delegates.
• Updated in-app ranking table.
• Various bug fixes and performance improvements.

Download this release from the official website or from the Google Play store.

Feel free to discuss AI Benchmark results in this thread.

Any found issues can be reported here.

 

Detailed AI Benchmark V5 results were released for over 750 Android devices:

https://ai-benchmark.com/ranking
https://ai-benchmark.com/ranking_detailed

Historical results for previous AI Benchmark versions are still available in our archive section:

v4.0.3 (year 2021): https://ai-benchmark.com/archive/ranking_4_0_3
v3.0.2 (year 2020): https://ai-benchmark.com/archive/ranking_3_0_2
v2.1.2 (year 2019): https://ai-benchmark.com/archive/ranking_2_1_2
v2.0.0 (year 2018): https://ai-benchmark.com/archive/ranking_2_0_0

 

What's New:

• New models and tasks: 4K Video Super-Resolution, Image Denoising, Question Answering, Object Tracking, Depth Estimation, etc.
• Updated TFLite NNAPI, GPU, Hexagon NN and MediaTek Neuron delegates.
• Added Qualcomm QNN delegate for direct inference on Snapdragon DSPs, HTPs and GPUs.
• Added new inference options (Fast Single Answer, Sustained, Low Power).
• Extended accuracy measurements.
• Power consumption tests (PRO mode).
• The total number of tests increased to 78.

Download this release from the official website or from the Google Play / Huawei AppGallery stores.

Feel free to discuss AI Benchmark results in this thread.

Any found issues can be reported here.

 

Starting from version 4.0.4, we release AI Benchmark Nightly builds for ML developers every month.

Compared to the standard version, AI Benchmark Nightly:

• is built with the latest TensorFlow Lite nightly runtime,
• provides support for the full range of TensorFlow ops,
• contains the newest Android TFLite delegates: GPU, NNAPI, Hexagon, Eden and Neuron.

These builds allow ML developers to test their own TFLite models converted with TF-nightly or containing custom TF operators, making it possible to run nearly any TensorFlow model on any mobile device with this benchmark edition.

AI Benchmark Nightly is provided for R&D purpose only and should not be used for benchmarking mobile devices.

You can download the latest AI Benchmark Nightly build from the official website:
https://ai-benchmark.com/download

All questions related to machine learning development for mobile devices can be asked in this thread.

 

We are happy to announce that the Mobile AI Workshop organized in conjunction with the CVPR 2021 conference will be streamed live on YouTube and available for everyone!

During this event, you will see tutorials from all major mobile SoC vendors including MediaTek, Samsung, Qualcomm, Huawei, Imagination Technologies, OPPO, Synaptics and Google telling you how to efficiently deploy machine learning models on their hardware. An introductory talk from AI Benchmark will additionally introduce all basic concepts related to ML inference on smartphones, mobile deep learning libraries and SDKs, acceleration options, edge NPUs and their performance.

The event will start at 7am Pacific Time (4pm CET / 10pm China Time) on the 20th of June and will be streamed at:
https://ai-benchmark.com/workshops/mai/2021/#live

You can also ask the speakers your own questions - for this, please leave them on the forum in this thread.

 

The final results and solutions obtained in the Mobile AI 2021 Challenges are now available online:

Learned Smartphone ISP on Mobile NPUs:
https://arxiv.org/pdf/2105.07809.pdf

Fast Camera Image Denoising on Mobile GPUs:
https://arxiv.org/pdf/2105.08629.pdf

Real-Time Quantized Image Super-Resolution on Mobile NPUs:
https://arxiv.org/pdf/2105.07825.pdf

Real-Time Video Super-Resolution on Smartphones:
https://arxiv.org/pdf/2105.08826.pdf

Fast and Accurate Quantized Camera Scene Detection on Smartphones:
https://arxiv.org/pdf/2105.08819.pdf

Fast and Accurate Single-Image Depth Estimation on Mobile Devices:
https://arxiv.org/pdf/2105.08630.pdf

The live presentation of the challenges and top solutions will be given on the 20th of June at the CVPR 2021 Mobile AI Workshop.

 

Starting from version 4.0.4, we release AI Benchmark Nightly builds for ML developers every month.

Compared to the standard version, AI Benchmark Nightly:

• is built with the latest TensorFlow Lite nightly runtime,
• provides support for the full range of TensorFlow ops,
• contains the newest Android TFLite delegates: GPU, NNAPI, Hexagon, Eden and Neuron.

These builds allow ML developers to test their own TFLite models converted with TF-nightly or containing custom TF operators, making it possible to run nearly any TensorFlow model on any mobile device with this benchmark edition.

AI Benchmark Nightly is provided for R&D purpose only and should not be used for benchmarking mobile devices.

You can download the latest AI Benchmark Nightly build from the official website:
https://ai-benchmark.com/download

All questions related to machine learning development for mobile devices can be asked in this thread.

 

Starting from version 4.0.4, we release AI Benchmark Nightly builds for ML developers every month.

Compared to the standard version, AI Benchmark Nightly:

• is built with the latest TensorFlow Lite nightly runtime,
• provides support for the full range of TensorFlow ops,
• contains the newest Android TFLite delegates: GPU, NNAPI, Hexagon, Eden and Neuron.

These builds allow ML developers to test their own TFLite models converted with TF-nightly or containing custom TF operators, making it possible to run nearly any TensorFlow model on any mobile device with this benchmark edition.

AI Benchmark Nightly is provided for R&D purpose only and should not be used for benchmarking mobile devices.

You can download the latest AI Benchmark Nightly build from the official website:
https://ai-benchmark.com/download

All questions related to machine learning development for mobile devices can be asked in this thread.

 

What's New:

• TFLite NNAPI, GPU, XNNPACK and Hexagon NN delegates are updated to their latest builds.
• Samsung Eden Delegate is updated to version V2.
• Integrated new MediaTek Neuron Delegate.
• Various bug fixes and performance improvements.

Download this release from the official website or from the Google Play / Huawei AppGallery / Jolla stores.

Feel free to discuss AI Benchmark results in this thread.

Any found issues can be reported here.

 

What's New:

• TFLite XNNPACK delegate is now enabled by default for CPU inference.
• Updated Qualcomm Hexagon NN and Samsung Eden NN libraries.
• GPU acceleration is now available for quantized neural networks.
• Various enhancements and performance improvements.

Download this release from the Google Play: link

Feel free to discuss AI Benchmark results in this thread.

Any found issues can be reported here.