News, Releases and More

What's New:

• 1. New tasks and models: Vision Transformer (ViT) architectures, Large Language Models (LLMs), Stable Diffusion network, etc.
• 2. Added tests checking the performance of quantized INT16 inference.
• 3. LiteRT (TFLite) runtime updated to version 2.17.
• 4. Updated Qualcomm QNN, MediaTek Neuron, TFLite NNAPI, GPU and Hexagon NN delegates.
• 5. Added Arm NN delegate for AI inference acceleration on Mali GPUs.
• 6. The total number of tests increased to 83.

Download this release from the official website or from the Google Play store.

Feel free to discuss AI Benchmark results in this thread.

Any found issues can be reported here.

 

What's New:

• Updated Qualcomm QNN and MediaTek Neuron delegates.
• Enhanced stability and accuracy of the power consumption test.
• Various bug fixes and performance improvements.

Download this release from the official website or from the Google Play store.

Feel free to discuss AI Benchmark results in this thread.

Any found issues can be reported here.

 

Detailed AI Benchmark V5 results were released for over 50 IoT, smartTV and automotive platforms:

https://ai-benchmark.com/ranking_IoT
https://ai-benchmark.com/ranking_IoT_detailed

The results of the recently presented mobile chipsets including the Snapdragon 8 Gen 3, Dimensity 9300, Google Tensor G3 and Apple A17 Pro are available in the Mobile SoC ranking table:

https://ai-benchmark.com/ranking_processors
https://ai-benchmark.com/ranking_processors_detailed

 

What's New:

• Added new NPU power consumption test.
• Updated TFLite runtime.
• Updated TFLite GPU, NNAPI, Qualcomm QNN, Hexagon NN and Samsung ENN delegates.
• Updated in-app ranking table.
• Various bug fixes and performance improvements.

Download this release from the official website or from the Google Play store.

Feel free to discuss AI Benchmark results in this thread.

Any found issues can be reported here.

 

What's New:

• Updated TFLite GPU, Qualcomm QNN, MediaTek Neuron and Hexagon NN delegates.
• Updated in-app ranking table.
• Various bug fixes and performance improvements.

Download this release from the official website or from the Google Play store.

Feel free to discuss AI Benchmark results in this thread.

Any found issues can be reported here.

 

Detailed AI Benchmark V5 results were released for over 750 Android devices:

https://ai-benchmark.com/ranking
https://ai-benchmark.com/ranking_detailed

Historical results for previous AI Benchmark versions are still available in our archive section:

v4.0.3 (year 2021): https://ai-benchmark.com/archive/ranking_4_0_3
v3.0.2 (year 2020): https://ai-benchmark.com/archive/ranking_3_0_2
v2.1.2 (year 2019): https://ai-benchmark.com/archive/ranking_2_1_2
v2.0.0 (year 2018): https://ai-benchmark.com/archive/ranking_2_0_0

 

What's New:

• New models and tasks: 4K Video Super-Resolution, Image Denoising, Question Answering, Object Tracking, Depth Estimation, etc.
• Updated TFLite NNAPI, GPU, Hexagon NN and MediaTek Neuron delegates.
• Added Qualcomm QNN delegate for direct inference on Snapdragon DSPs, HTPs and GPUs.
• Added new inference options (Fast Single Answer, Sustained, Low Power).
• Extended accuracy measurements.
• Power consumption tests (PRO mode).
• The total number of tests increased to 78.

Download this release from the official website or from the Google Play / Huawei AppGallery stores.

Feel free to discuss AI Benchmark results in this thread.

Any found issues can be reported here.

 

Starting from version 4.0.4, we release AI Benchmark Nightly builds for ML developers every month.

Compared to the standard version, AI Benchmark Nightly:

• is built with the latest TensorFlow Lite nightly runtime,
• provides support for the full range of TensorFlow ops,
• contains the newest Android TFLite delegates: GPU, NNAPI, Hexagon, Eden and Neuron.

These builds allow ML developers to test their own TFLite models converted with TF-nightly or containing custom TF operators, making it possible to run nearly any TensorFlow model on any mobile device with this benchmark edition.

AI Benchmark Nightly is provided for R&D purpose only and should not be used for benchmarking mobile devices.

You can download the latest AI Benchmark Nightly build from the official website:
https://ai-benchmark.com/download

All questions related to machine learning development for mobile devices can be asked in this thread.

 

We are happy to announce that the Mobile AI Workshop organized in conjunction with the CVPR 2021 conference will be streamed live on YouTube and available for everyone!

During this event, you will see tutorials from all major mobile SoC vendors including MediaTek, Samsung, Qualcomm, Huawei, Imagination Technologies, OPPO, Synaptics and Google telling you how to efficiently deploy machine learning models on their hardware. An introductory talk from AI Benchmark will additionally introduce all basic concepts related to ML inference on smartphones, mobile deep learning libraries and SDKs, acceleration options, edge NPUs and their performance.

The event will start at 7am Pacific Time (4pm CET / 10pm China Time) on the 20th of June and will be streamed at:
https://ai-benchmark.com/workshops/mai/2021/#live

You can also ask the speakers your own questions - for this, please leave them on the forum in this thread.

 

The final results and solutions obtained in the Mobile AI 2021 Challenges are now available online:

Learned Smartphone ISP on Mobile NPUs:
https://arxiv.org/pdf/2105.07809.pdf

Fast Camera Image Denoising on Mobile GPUs:
https://arxiv.org/pdf/2105.08629.pdf

Real-Time Quantized Image Super-Resolution on Mobile NPUs:
https://arxiv.org/pdf/2105.07825.pdf

Real-Time Video Super-Resolution on Smartphones:
https://arxiv.org/pdf/2105.08826.pdf

Fast and Accurate Quantized Camera Scene Detection on Smartphones:
https://arxiv.org/pdf/2105.08819.pdf

Fast and Accurate Single-Image Depth Estimation on Mobile Devices:
https://arxiv.org/pdf/2105.08630.pdf

The live presentation of the challenges and top solutions will be given on the 20th of June at the CVPR 2021 Mobile AI Workshop.