Questions about the Usage of the Dataset for the Competition

Hi @gaozhanfire

Yes, you’re allowed to use the any information you can extract from the test data as well.

View post 0

Questions about the Usage of the Dataset for the Competition

Hi @gaozhanfire

Yes, you can use all the common training data for any of the tasks.

View post 0

Is there any problem in opening the submissions page for CDX track?

Hi @subatomicseer

This is resolved. You should be able to see it now.

View post 2

Question about submission

@mikhail_sukhovey Good to know. You should also be able to make normal submissions now.

View post 1

Question about submission

@mikhail_sukhovey

Can you please try now

View post 0

When will we begin testing on 20/30 songs of the hidden dataset

Hi @lyndon_reid

The current submissions are scored on Phase 1 and Phase 2 data combined, since March 7th.

View post 1

Reasons of submission failure

@alina_porechina

I’ve rerun the submissions after adding an extra buffer to the timeout. They’re scored now.

View post 1

Reasons of submission failure

I’ve made some changes to how the cloud instances are provisioned for inference, I think now the failures shouldn’t occur. However if you do observe it again please let me know.

View post 1

Reasons of submission failure

@alina_porechina , Unfortunately I haven’t found what the issue is on our side. I’m looking into it and it should be resolved soon.

Can you let me know if you expect your model to be consistent in the speed for every song or it can vary? Since the evaluator is checking if every prediction is above the speed constraint.

View post 0

Timeout submission logs

@v_mudryi both are timeouts.

View post 1

Timeout submission logs

@v_mudryi Can you please provide the submission ids you’d like me to check.

Also, regarding failed submissions, we count failed submissions as valid submission slots, since submissions are also a medium of private data probing. Hence the number has to be limited.

View post 0

Product Matching: Inference failed

@bartosz_ludwiczuk

They timed out.

View post 1

Product Matching: Inference failed

@youssef_nader3 I’ve added your error on gitlab comment.

View post 0

Product Matching: Inference failed

@strekalov , I’ve added your error on gitlab comment.

View post 0

Product Matching: Inference failed

I’ve added the error in the gitlab comments.

About the failed builds with no changes to dependencies. The only time this has occurred is that the repository has too many models which kills the Docker build, not sure if it’s also the case here, I’ll check further. If it is indeed the cause, unfortunately for now you’ll have to reduce the overall repository size somehow.

For the one with time-out but it didn’t start, can you give me the submission id?

View post 1

My submission failed

Hi everyone,

The logs only show for the validation step, which is just some small subset of the public data for you to debug. For obvious reasons we can’t share logs from runs on a private dataset.

View post 0

Product Matching: Inference failed

Hi @hca97

Thank you for the insightful and extensive testing.

Indeed the experiments are run with Kubernetes, however we explicitly provision on-demand AWS g4dn.xlarge (for the GPU nodes) for all the submissions. I understand that this may allow for variations in the quality of CPU or memory that AWS has attached to the nodes. However this is the only way we can run the competition.

As for the time limit, the Machine Can See organizers have explicitly asked for the 10 minute limit, as they want the models to fast. So participants have to add provisions in their code to track the time spent, or ensure that the code has enough buffer to allow for the allegedly slower runs to complete.

View post 0

Product Matching: Inference failed

@radhika_chhabra Can you check my comment on your latest submissions’s gitlab issue page.

View post 0

Use of previous frames?

@nippun_sharma The original plan for the challenge was to provide video clips instead of images, but the problem statement was changed. I think the organizers forgot to remove that clause in the rules by mistake. Please ignore it.

View post 0

External dataset

Publicly available data is allowed

View post 1