Real-Time Camera Scene Detection Challenge
- Thread starter Andrey Ignatov
- Start date
You can use float input format when training the model, but the converted TFLite model should consist of INT8 ops only. All model conversion instructions were provided previously here.
This was done for the development phase.
In the final test phase, you need to submit your final results only as specified in the email sent yesterday.
Hello! The online validation server is running the following error with a new submission:
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
Traceback (most recent call last):
File "/tmp/codalab/tmpscXoW5/run/program/evaluation.py", line 37, in
accuracy = 100.0 * n_correct_prediction/len(label_list)
NameError: name 'label_list' is not defined
Earlier, the same model was running fine and I had an accuracy of 91.5. The model is running fine on the lightspeed and local phone.
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
Traceback (most recent call last):
File "/tmp/codalab/tmpscXoW5/run/program/evaluation.py", line 37, in
accuracy = 100.0 * n_correct_prediction/len(label_list)
NameError: name 'label_list' is not defined
Earlier, the same model was running fine and I had an accuracy of 91.5. The model is running fine on the lightspeed and local phone.
andrewBatutin
New member
Regarding
- fully-quantized INT8 model (with no FP32 / FP16 / INT16 ops);
Is it ok to use tensorflow Integer with float fallback option?
Or we need to make it strictly INT8 model with tf.lite.OpsSet.TFLITE_BUILTINS_INT8 ?
- fully-quantized INT8 model (with no FP32 / FP16 / INT16 ops);
Is it ok to use tensorflow Integer with float fallback option?
Or we need to make it strictly INT8 model with tf.lite.OpsSet.TFLITE_BUILTINS_INT8 ?
Yes.
I guess you are submitting your solution to the final test server. The validation server is running fine now.
andrewBatutin
New member
Hi folks,
What version of iOS TensorFlowLite do you use for lightspeed service and for final evaluation?
Can you publish the code of iOS app you use to test the model?
I'm having an issue with submitting my model to lightspeed
email - johnmeetup56@gmail.com
model - efficientnetb2_no_norm_quantized.tflite
And i had an issue with running my model on old iOS TensorFlowLite 2.0 lib version
After i've updated to iOS TensorFlowLite 2.4 - issue was fixed on device, and my model runs OK
Can it be the same issue with lightspeed service?
Thnx
What version of iOS TensorFlowLite do you use for lightspeed service and for final evaluation?
Can you publish the code of iOS app you use to test the model?
I'm having an issue with submitting my model to lightspeed
Code:
Status: Error
Inference time (sec): Unknownemail - johnmeetup56@gmail.com
model - efficientnetb2_no_norm_quantized.tflite
And i had an issue with running my model on old iOS TensorFlowLite 2.0 lib version
Code:
TensorFlow Lite Error: Didn't find op for builtin opcode 'RESIZE_BILINEAR' version '3'After i've updated to iOS TensorFlowLite 2.4 - issue was fixed on device, and my model runs OK
Can it be the same issue with lightspeed service?
Thnx
I would like to ask if the ZIP submitted during the test phase must contain sheets, Tflite models, checkpoint files and all codes?Or just two models. Tflite.
Can I see the results of every test I take?Are the three test opportunities to choose the best as the final result, or choose the latest as the final result?
What information is required for sheet files and what content is required for codes? I read the sheet. tex provided by the officia and feel very confused. I don't know what information to provide.
Can I see the results of every test I take?Are the three test opportunities to choose the best as the final result, or choose the latest as the final result?
What information is required for sheet files and what content is required for codes? I read the sheet. tex provided by the officia and feel very confused. I don't know what information to provide.
Yes.
Only the latest one will be used.
In the provided factsheet template, you can find all the information that you need to add there.
We used TFLite-2.4 nightly, updating it to the latest nightly build has probably solved the above issue. However, EfficientNets are anyway incompatible with TFLite CoreML and NNAPI delegates as they contain some dynamically sized tensors that are not yet implemented.
andrewBatutin
New member
So can we submit EfficientNet?
We are able to fully int8 quantize, and it seems to run on ios 14 device
If it runs successfully on the provided lightspeed server - then yes, you can submit it.
andrewBatutin
New member
Question about fact check doc
Is this like yes/no question? Will we submit or not?
thnx
What does this mean?
Is this like yes/no question? Will we submit or not?
thnx
andrewBatutin
New member
Hi folks, it's me again
Can't get runtime for model from lightspeed
Website constantly sends same request with no response
Is it up?
Can't get runtime for model from lightspeed
Website constantly sends same request with no response
Code:
curl 'http://lightspeed.difficu.lt:60002/status/null' \
-H 'Accept: */*' \
-H 'Referer: http://lightspeed.difficu.lt:60002/' \
-H 'DNT: 1' \
-H 'X-Requested-With: XMLHttpRequest' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36' \
--compressedIs it up?
Last edited:
andrewBatutin
New member
Hi!
When can we expect the final evaluation?
Thnx for the challenge!
When can we expect the final evaluation?
Thnx for the challenge!
The preliminary results of the Camera Scene Detection challenge:
docs.google.com
MAI 2021 Preliminary Results
saikatdutta
New member
Hi,
Is the challenge report already sent out?
Is the challenge report already sent out?
No, it will be sent soon.