Real-Time Image Super-Resolution Challenge

mkudelski

New member
Yes, your TFLite files and your original codes for training the model.
And what about the case when an internal and proprietary company tool was used to train the model (e.g., an internal wrapper on top of TensorFlow)?

May I ask why the model accompanied with the inference code - which fully allows you to reproduce our results on the evaluation set - are not enough for a complete submission? Why do you need the training codes?
 

oasis

New member
Can the maximum submission total be increased? The runtime submission guidelines and final submission guidelines are not compatible. This invalidates the commits, so I'm wondering if it's possible to increase the maximum submission total. :)
 

mrblue3325

New member
Can the maximum submission total be increased? The runtime submission guidelines and final submission guidelines are not compatible. This invalidates the commits, so I'm wondering if it's possible to increase the maximum submission total. :)
what is the difference between the runtime submission guidelines and final submission guidelines?
 

oasis

New member
what is the difference between the runtime submission guidelines and final submission guidelines?
When I submit results following the final test phase submission guidelines, I get an error "No model files in submission". However, if I just submit the model.tflite and readme.txt files, I can successfully see the model run time, but also waste one commit (maximum commits are 3).
 

mrblue3325

New member
When I submit results following the final test phase submission guidelines, I get an error "No model files in submission". However, if I just submit the model.tflite and readme.txt files, I can successfully see the model run time, but also waste one commit (maximum commits are 3).
that mean no need to submit the factsheet and models folder?
 

Andrey Ignatov

Administrator
Staff member
what is the difference between the runtime submission guidelines and final submission guidelines?
Maybe the last submission should include all the files.

Test submissions are used just for uploading your final results (including fact sheets and executables), and only your last submission would be considered.

May I ask why the model accompanied with the inference code - which fully allows you to reproduce our results on the evaluation set - are not enough for a complete submission?

We need to ensure that the training method described in your fact sheet is actually real. Otherwise, you can write absolutely anything there, and the results would not be reproducible.
 

Lishen

New member
Test submissions are used just for uploading your final results (including fact sheets and executables), and only your last submission would be considered.



We need to ensure that the training method described in your fact sheet is actually real. Otherwise, you can write absolutely anything there, and the results would not be reproducible.
Hello,
I notice that someone‘s model may be very similar with ours according to the timing log. How will the organizers deal with such issues?
 

Andrey Ignatov

Administrator
Staff member
Does anyone know how to download test data? I can't click on the test data link on the page.

Please check the email that was sent to all challenge participants this Tuesday. It contains a detailed description of the final challenge submission format and data. It is very important that you follow the provided submission guideline precisely so that your submission is not disqualified.

I notice that someone‘s model may be very similar with ours according to the timing log. How will the organizers deal with such issues?

The final score of each submission would be computed according to this formula. If multiple submissions would get top scores, then they all would be considered as challenge winners.
 

NBCS

New member
I notice that the new timing log has two times: "inference_us" and "bylayer_sum_us", which running time will be used to compute the final score?
 

Andrey Ignatov

Administrator
Staff member
The fact sheet provided is AIM 2021's. Is there a 2022's version? Or just modify 2021 to 2022?
Yes, you can just modify the year.

Do you count only our last upload? or the best among our all uploads?
Only last attempt.

which running time will be used to compute the final score?
The total runtime per inference.

Can we know our ranking on August 2?
You should expect to get the results at the end of the next week - checking all submissions in 1 day is not really feasible.
 

CCjiahao

New member
Hello, thank you very much for holding this series of high-quality competitions. Now I have a question about the ranking and would like to ask you: Is there any news about the ranking now? When will the rankings be announced? And what method/platform will be used to publish the ranking?
 

ggoo

New member
Dear Organizers,

When will the final results be announced? And will there be an email notification to all participants or only top-ranked participants?

Best regards,
 

Lishen

New member
The mail mentions that most the teams in the final test phase were contacted by the organizers, Does anyone receive the result?
 

deepernewbie

New member
It means most of the challenges' preliminary results were published. Like "Compressed Image Single Image Superresolution" which I received the preliminary results. No news on this one yet. Either you rank 1st or the last if you have participated with a factsheet and required files. You will be informed about the results, this is for sure, I have participated 4-5 challenges so far. (I think even if you have just registered but did not send any files you may still be informed but I am not sure). Stay tuned, these guys ( Andrey, Radu and Synaptics Guys) are professional and will do their best as soon as possible as high quality as possible.

Good Luck
 

CCjiahao

New member
It's been 48 hours since I received the email, but I haven't received the result yet. Has anyone received the results yet?
 

CCjiahao

New member
Is that all the results now? What does "no report" mean? Is there no complete result yet? Or no report due to submission of missing files, such as training code or feet sheet?
 

maciejos

New member
Will be latencies measured once again, or the submission latency is the final one? I ask, because we observed that the reported times can vary significantly (~0.6 ms) between the submissions of the same model, and it seems, based on the metrics, submission latencies and provided final score equation, that the top ranked teams will be really close. In this scenario, such benchmarks variance can define the rank.
 

Ritsumio

New member
What does the result of the float model operation mean by NA? May I ask if the teams that submitted the report and the int model have valid scores, but the float model score is that of NA, will they participate in the final evaluation?
 

CCjiahao

New member
Dear Organizers,

I learned from the competition homepage that the top three would receive the award certificate, is that right? We are eager to get this award certificate.
 

Andrey Ignatov

Administrator
Staff member
I learned from the competition homepage that the top three would receive the award certificate, is that right?
Yes, you will receive them closer to the actual ECCV workshop event.

moreover, is it known when the final versions of the reports will be released?
We don't know yet when the publisher will release these papers, but we might also upload them to arXiv this months.

AIM'2022 Workshop during ECCV'22 is going to be a purely online event?
It will likely be a hybrid event, more information would be published soon on the website.
 
Top