Search results

The results for all challenges except for Stable Diffusion and Efficient LLMs have been released. For the latter two, the results would be finalized in the next few days.

All results should be released by the end of this week.

Not yet, the results are released in waves, right now they are already available for Quantized Super-Res, Floating-Point Super-Res and Image Denoising Challenges. For other competitions they would be released next days. Note that if you want to submit a paper tomorrow - you don't need to...

Sorry, yes, it died unfortunately with old Codalab storage. We are using the standard built-in TensorFlow PSNR/SSIM functions for computing metrics in all competitions: https://www.tensorflow.org/api_docs/python/tf/image/psnr https://www.tensorflow.org/api_docs/python/tf/image/ssim Input /...

Yes, that's an expected behavior. Note that only your last upload counts.

Yes, it's acceptable, the resolution is the same.

It's hard to say now if preliminary results would be released on the 21st of March for all competitions. However, that shouldn't be a blocker - if you have good results on the validation set, you can submit a paper without final numbers. Based on preliminary numbers, it's likely that challenge...

You can test your model on any machine with MacOS: CoreML GPU delegate supports both Apple and AMD/Nvidia silicon, so if your solutions works - then at least GPU-based inference would be available on the Apple M4.

Each team can make only one final submission per challenge. However, multiple solutions can be described in the challenge paper if you decide to submit one to the workshop.

If you are making the final submission - you can ignore all issues. For the main phase, Codabench is used only for computing numerical (PSNR/SSIM) results on the validation set, thus you need to follow the instructions on the corresponding webpage and upload your reconstructed images there.

Yes, that's the case, you can ignore Codabench issues as we are validating all submissions offline.

Yes, that's the case: we've evaluated all solutions submitted to ICCVW2025 for all challenges, but unfortunately there were not many major changes in the results / model performance compared to the previous CVPR edition. Therefore, there was not much sense in publishing challenge reports for...

Any technical questions and questions related to model conversion to TFLite / ONNX / etc. formats can be asked in this thread.

Any questions related to general MAI Workshop and Challenges organization can be asked in this thread.

No, this is just an opportunity for you to have a paper describing your solution being published in ICCV proceedings.

Please upload your executables to a separate shared storage platform and provide the corresponding link in the factsheet submitted to Codalab. Yes.

[0, 255] By default, we are using NHWC format. You can get a TFLite model with NHWC input format by using the following conversion option: sample_input = (torch.randn(1, heights, width, 3),) model = ai_edge_torch.to_channel_last_io(model, args=[0], outputs=[0]) edge_model =...

In this challenge, we compute PSNR and SSIM scores on a CPU, so the result you are getting with the TFLite model on your local machine = your actual PSNR and SSIM scores.

Yes. Please ignore all errors, Codalab is only used for uploading your zip archives in the final test phase. They are then downloaded and processed separately by us.

TFLite GPU delegate automatically casts FP32 models to FP16 format, therefore no additional conversion is needed, it won't bring any latency improvements. Average of two runtimes.

One prompt (string). With a batch size of 1. No, weights initialization time will not be counted. [Tokens / s] ~ [1 / runtime], so there are no issues with the formula: higher tokens/s rate leads to a higher score.

Yes, if you are sending us model converted from PyTorch.

Yes, all models not compatible with Dimensity's NPU would be evaluated on its GPU (using the TFLite GPU delegate).

There are no issues with the server, your submission format is incorrect: all 100 png images should lie in the root folder.

As is stated in the challenge description: Unfortunately, there are still no automatic tools allowing to provide you with a meaningful image perceptual score, therefore we have to rely on MOS results in this competition, making it impossible to perform any evaluation during the development...

Because these are two different challenges (quantized vs. floating-point) with different requirements (target platforms, min. required accuracy).

Please find this information using the following link: https://codalab.lisn.upsaclay.fr/competitions/21868#learn_the_details-evaluation The participants in this challenge are required to submit their final python codes that: 1) take the input text prompt, 2) generate text using this prompt as...

Thanks for noticing. Instead of the PUNET model, we are now providing a more efficient MicroISP baseline. The corresponding link was added to the challenge description.

Yes, in this challenge there is only one phase, where you can upload your final solution. You can make as many submissions as you want, but only your last one counts (will be validated). There is no automatic validation in Codalab - instead, we use it only for uploading solutions (thus the...

APK is just an archive, you can unpack it with 7z or any other file archiver.

The results were sent today by email. Yes, this will be done soon.

For the majority of challenges, the results would be sent this week. You need to follow the final submission instructions. All conditionally accepted papers would be checked before sending them to the publisher to see whether all issues are fixed. If this was not done, the paper would be...

100x100

The correct one is: dim_2 = 8 * torch.export.Dim("height", min=8, max=256) dim_3 = 8 * torch.export.Dim("width", min=8, max=256) edge_model = ai_edge_torch.convert(model.eval(), sample_input, dynamic_shapes=({2:dim_2, 3:dim_3},)) However, this option is broken in the latest PyTorch...

You don't need to upload the results on the test images in this challenge. That's another issue, we've mentioned during yesterday's Q&A session that for the denoising challenge the data would be provided later when this challenge transitions to ICCV.

First of all, this is not about 0.002dB accuracy difference - this is about some ops that are completely screwed up by the ai-edge-torch converter, so that you get total corruptions instead of real output. Secondly, this primarily refers to the challenges, where we have an unconstrained track...

If your submission exceeds 300Mbs, please upload your visual results or codes to a separate shared storage platform and provide the corresponding link in your factsheet. Yes, you can submit everything in one zip file.

It's not an issue with the input channels - it's a problem with a bilinear resize layer. The issue is self-explanatory: half_pixel_center and align_corner options cannot be used at the same time. Try changing the parameters of the resize layer in your model.

Are you using the ai_edge_torch plugin for model conversion like in this tutorial?

By default, all solutions will be evaluated using FP16 + TFLite GPU Delegate mode. However, CPU backend will be used for solutions not supporting this option. TFLite GPU delegate is the easiest mode in terms of model adaptation as it supports nearly all common ops. However, yes, there might be...

Using logarithm operator is not a very good option itself as it's a purely logical op, i.e., no acceleration by NPU/GPU if supported at all by the platform.

Yes, it is based solely on NPU runtime, CPU runtime is provided only for the reference. However, if the final submission won't run on NPU, its CPU runtime will be used instead.

SoC score is obtained in the same way as the Mobile score, but it does not take into account the results of the last memory test.

Yes, but an alternative implementation (not from Google).

Does this error also occur during CPU-based inference? One possible issue might be the dimensionality of your tensor: GPU delegate might be expecting a 4D tensor, but it received a 3D one. If you are still unable to solve this problem, you can send us your model by email.

ai-edge-torch plugin automatically converts PyTorch NCHW models to TFLite NHWC models: https://github.com/aiff22/MAI-2025-Workshop/blob/main/pytorch_to_tflite.py If you have any issues with it, please post in this thread, we will then check this separately.

We extended the number of submissions to 10 in the test phase in all challenges. Note, however, that only your last submission counts. Training code is also needed as we check some submissions for reproducibility.

Yes, there is indeed an issue with dynamic input size for PyTorch to TFLite conversion. We will accept this for models converted from PyTorch. However, be prepared that we might ask you to provide the same model with additional input sizes if automatic tensor resizing won't work for your model.