Any questions related to general MAI Workshop and Challenges organization can be asked in this thread.
Mobile AI Workshop General Organization Questions
- Thread starter Andrey Ignatov
- Start date
I would like to ask whether there are any plans to postpone the schedule for this MAI 2025 challenge.
As the submission deadline for the model file and factsheet is approaching, we have yet to receive updates regarding the inference speed and performance on the NPU.
I would like to know the exact deadline for MAI 2025 challenge
As the submission deadline for the model file and factsheet is approaching, we have yet to receive updates regarding the inference speed and performance on the NPU.
I would like to know the exact deadline for MAI 2025 challenge
We understand that the sRGB challenge has Final Submission Instructions for Get Data in Participate as well as for Evaluation in Learn the Details, which one of these instructions should be referred to for submissions during the Test phase?
We've just sent the final submission instructions for all challenges.
The deadline for the current competitions cannot be extended due to hard restrictions from CVPR main organizers. However, all current challenges would transition to ICCV challenges with a new deadline set in summer.
No, the deadline is on the 28th of March as was stated on the Codalab webpage.
As was stated in the latest instructions, you can send us up to three models this week for runtime validation on the target device.
The deadline for the current competitions cannot be extended due to hard restrictions from CVPR main organizers. However, all current challenges would transition to ICCV challenges with a new deadline set in summer.
Good point, we will fix this. In principle, both instructions tell you to send us the TFLite model, visual results (if test set provided in the challenge) and the factsheet. But we will clarify this.
I don't understand the mail exactly.
As I understand it, people who want to submit a TFLite model and submit a paper because they need to show a report paper in order to organize a CVPR workshop, are they submitting a paper?
Also, does this mean that the actual ranking of the challenge is done by ICCV?
Or is there a challenge winner in CVPR and a challenge winner in ICCV?
As I understand it, people who want to submit a TFLite model and submit a paper because they need to show a report paper in order to organize a CVPR workshop, are they submitting a paper?
Also, does this mean that the actual ranking of the challenge is done by ICCV?
Or is there a challenge winner in CVPR and a challenge winner in ICCV?
Thank you for your prompt reply, I also noticed a problem. I was about to submit information for the sRGB challenge and the system platform detected the need that I need to have 3145 png images. However, as far as I know, the sRGB track does not provide input images for testing in the current testing phase. I'm not sure if this is a bug in the sRGB challenge system platform, my attachment is two system platform logs.
These are separate things:
- To participate in the final competition phase, you need to submit your TFLite model and a factsheet describing your solution to Codalab.
- In addition to that, you can also submit a separate full-length paper to the CMT paper submission website. If the paper is well written, your solution is performant enough (you don't need to be the challenge winner, but the results should be clearly above some baseline model) or has some interesting ideas, then the paper will be accepted and published in CVPR Conference Workshop Proceedings.
Yes, there will be CVPR challenge winners and ICCV challenge winners. We therefore encourage you to submit your current solution now, and then continue with its improvement till the next deadline as you can potentially win both competitions / get two publications.
Yes, that's right, you can ignore this error. What we need is just a zip archive uploaded to Codalab with the requested TFLite model and a factsheet.
sr_enhance
New member
Before your notification, we had already exhausted the number of submissions on Codalab. Can we increase the number of submissions by 3 times or delete our previous submissions?
sr_enhance
New member
Only the TFLite model, visual results (if a test set is provided in the challenge), and situation sheet are needed. Do I need to submit training code？
Excuse me, can we use inference scripts to run the model for inference? For example, in PyTorch, our input is in NCHW format. Can we also use NCHW format to input the model for testing? If NHWC input format is mandatory, can we use inference scripts to convert it into NCHW as model input?
Yes, I've wondered that too. Since the first submission was Failed, I kept making corrections and submitting, and have submitted 4 times now. But I guess it might be possible that only the last submission is chosen as a valid submission, hopefully.
We extended the number of submissions to 10 in the test phase in all challenges. Note, however, that only your last submission counts.
Training code is also needed as we check some submissions for reproducibility.
ai-edge-torch plugin automatically converts PyTorch NCHW models to TFLite NHWC models:
https://github.com/aiff22/MAI-2025-Workshop/blob/main/pytorch_to_tflite.py
If you have any issues with it, please post in this thread, we will then check this separately.
Yes, it is based solely on NPU runtime, CPU runtime is provided only for the reference.
However, if the final submission won't run on NPU, its CPU runtime will be used instead.
Using logarithm operator is not a very good option itself as it's a purely logical op, i.e., no acceleration by NPU/GPU if supported at all by the platform.
If your submission exceeds 300Mbs, please upload your visual results or codes to a separate shared storage platform and provide the corresponding link in your factsheet.
Yes, you can submit everything in one zip file.
Dear organizers, regarding Q4 in the QA:
Q4: The results of my converted TFLite model do not correspond to the results of the original PyTorch model. What should I do?
A4: Add a note at the beginning of your factsheet about this case. We will then check your conversion codes and if the issue is indeed with the ai-edge-torch plugin, then we will use your original PyTorch model for generation of numerical and visual results.
I have some concerns about this. First, I also trained my model using PyTorch, and I manually built the TensorFlow computation graph and converted it into a quantized TFLite model. I believe there are no technical barriers in the conversion from PyTorch to TFLite. However, since the operators in PyTorch and TensorFlow are not perfectly aligned, there will inevitably be a slight accuracy loss during the conversion process (approximately 0.002dB). In your response to Q4, you mentioned using PyTorch for result generation, which I find to be an unfair comparison. If we generate results using PyTorch, what is the purpose of converting the model to TFLite? It seems to undermine the entire effort of converting to TFLite, as the goal is typically to deploy the model on edge devices where TFLite is optimized. Using the original PyTorch model for evaluation defeats the purpose of demonstrating the effectiveness of the conversion.
Q4: The results of my converted TFLite model do not correspond to the results of the original PyTorch model. What should I do?
A4: Add a note at the beginning of your factsheet about this case. We will then check your conversion codes and if the issue is indeed with the ai-edge-torch plugin, then we will use your original PyTorch model for generation of numerical and visual results.
I have some concerns about this. First, I also trained my model using PyTorch, and I manually built the TensorFlow computation graph and converted it into a quantized TFLite model. I believe there are no technical barriers in the conversion from PyTorch to TFLite. However, since the operators in PyTorch and TensorFlow are not perfectly aligned, there will inevitably be a slight accuracy loss during the conversion process (approximately 0.002dB). In your response to Q4, you mentioned using PyTorch for result generation, which I find to be an unfair comparison. If we generate results using PyTorch, what is the purpose of converting the model to TFLite? It seems to undermine the entire effort of converting to TFLite, as the goal is typically to deploy the model on edge devices where TFLite is optimized. Using the original PyTorch model for evaluation defeats the purpose of demonstrating the effectiveness of the conversion.
First of all, this is not about 0.002dB accuracy difference - this is about some ops that are completely screwed up by the ai-edge-torch converter, so that you get total corruptions instead of real output.
Secondly, this primarily refers to the challenges, where we have an unconstrained track but still require the participants to submit TFLite models. Some teams, e.g., were developing transformer models, and in this case neither TF re-implementation nor manual PyTorch-to-TF conversion is really possible.
Alright, I understand.
have you solved these problems？i've met the same
You don't need to upload the results on the test images in this challenge.
That's another issue, we've mentioned during yesterday's Q&A session that for the denoising challenge the data would be provided later when this challenge transitions to ICCV.
Dear organizers, how can i get the test datasets in SISR? The download link is not open.
codalab.lisn.upsaclay.fr
CodaLab - Competition
sr_enhance
New member
When calculating PSNR for sRGB image enhancement, is it calculated at a resolution of 100x100 or at the original image resolution?
100x100
Is the ‘ONE DAY WORKSHOP/TUTORIAL PASS’ sufficient to cover the registration requirement for publishing a paper in the MAI Workshop?
I noticed the registration webpage states: "Each paper (Main Conference AND Workshop) MUST be registered under an AUTHOR." and "Virtual registrations will not cover a paper submission - even workshop papers." My question is: If my paper is accepted by the MAI workshop, can I use the "ONE DAY WORKSHOP/TUTORIAL PASS" for registration?
I noticed the registration webpage states: "Each paper (Main Conference AND Workshop) MUST be registered under an AUTHOR." and "Virtual registrations will not cover a paper submission - even workshop papers." My question is: If my paper is accepted by the MAI workshop, can I use the "ONE DAY WORKSHOP/TUTORIAL PASS" for registration?
For the majority of challenges, the results would be sent this week.
You need to follow the final submission instructions. All conditionally accepted papers would be checked before sending them to the publisher to see whether all issues are fixed. If this was not done, the paper would be rejected.
No, you need to select the "Author" registration option even if you have only one workshop paper. That's not really fair, but this was always the rule at the CVPR.
sr_enhance
New member
Dear organizer, can the status of the papers submitted on cmt3 be turned over as soon as possible so that we can register for the CVPR conference before April 30?
chinatownwhite
New member
Will there be a report for the Bokeh Rendering track?
Dear Organizers, I have a few questions:
1. Are there any further actions that top-ranked participants need to take moving forward?
2. Will there be any certificates or similar materials (e.g., awards) provided to recognize the participants' achievements?
3. For those who did not submit a workshop paper, will it be necessary for them to attend the CVPR event in person?
1. Are there any further actions that top-ranked participants need to take moving forward?
2. Will there be any certificates or similar materials (e.g., awards) provided to recognize the participants' achievements?
3. For those who did not submit a workshop paper, will it be necessary for them to attend the CVPR event in person?