Submission failed during Inference

@hca97 No it is not the first prediction, I see many print statements before this with varying time values.

View post 0

Submission failed during Inference

@fkemeth Can you give me the submission ids where you get the failure and I’ll check if it’s due to timeout or something else.

View post 0

Submission failed during Inference

@hca97

The timeout on these submission does indeed cross 2 seconds for one instance. I believe these are your print statements?

Totla Time  2018.3565616607666 ms

Unfortunately we can’t relax the timing criteria, if possible try to reduce the time taken by your model.

View post 0

Submissions are quite unstable

@MPWARE @tfriedel the time is measured per image. Upon request from the organizer, we have increased this number to 2 seconds per image.

@MPWARE , would it be possible to give a link to your submissions where same image got different times, as its not expected to vary based on cluster workload, your submission container gets the full node.

View post 1

Error with submissions in MDX23

Okay it’s fixed and I’ve rerun it here, you should get the scores once its completed.

View post 0

Error with submissions in MDX23

Is it not this repo?

View post 0

Error with submissions in MDX23

Hi @ZFTurbo

The old base image seems to not be available anymore, can you change to this base image in the Dockerfile

FROM nvidia/cuda:11.7.1-cudnn8-runtime-ubuntu20.04

View post 0

Submission Times

@quickpepper947 We do not check for identical commits when counting submissions. I missed that the date you transitioned from debug false to true was not the date you got 6 submissions. In that case I’ll need to further dig into why you got 6 submissions.

I suspect that it may be related to the tags having identical timestamps as you mentioned, I guess further investigation is required.

View post 0

Submission Times

@XavierJ , yes, all teams had the possibility to use 6 submissions due to this issue. However, to my best of my knowledge no one has used it intentionally. I apologize again that your team ended up having fewer submissions due to this.

I also checked @quickpepper947’s extra submission and 2 out of the 6 made on 8th May are actually identical commits.

View post 0

💬 Feedback & Suggestions

@CPMP 5 submissions per team should be enforced by the UI on a per day basis (00:00 to 00:00 UTC fixed window). We have two settings that the organizers can choose from, 24 hour rolling window and 24 hour fixed window. For this challenge its set to fixed window.

Is your team able to make more than 5 submissions in the fixed window?

View post 0

💬 Feedback & Suggestions

On Kaggle a merge is possible only if the sum of submission counts does not exceed 5 times the number of days spent on the competition so far.

Apologies, I wasn’t aware of this.

I’ll mention that we generally do not run competitions without private test sets, which is where this is a much bigger issue. This was an oversight on our part.

I checked the leaderboards and see that there is one team that has managed to make a lot of submissions doing this. After the team merge deadline we’ll check for any more unfair merges, and let the Amazon team decide if they want to provide everyone else with additional submissions to keep things fair.

View post 0

💬 Feedback & Suggestions

Agreed, but this is the system we have, it applies to everyone. If I understand correctly it is also the system on some other platforms.

I understand it’s an outsized advantage for this challenge as it’s not a code based challenge and doesn’t have a private test set.

We’re open to alternative ideas for future challenges, please share your feedback here or in dm.

View post 0

💬 Feedback & Suggestions

Hi @BenediktSchifferer

Yes, every team is limited to 5 submissions per day. After merging teams, submissions limits will apply to the whole team.

View post 0

💬 Feedback & Suggestions

@ECNU_Wei_Zhang Please ask all you team members to dm me and confirm that they want to leave the team.

View post 0

💬 Feedback & Suggestions

@ECNU_Wei_Zhang Okay your team has been removed

View post 2

Clarification and More Information for Phase 2

@gaozhanfire Thanks for letting me know. Leaderboards should be displayed correctly now.

View post 0

Clarification and More Information for Phase 2

@tereka Yes, you can use the Phase 1 test set as you like.

View post 0

Clarification and More Information for Phase 2

@tereka The competition will be ranked based on the results of Phase 2. The deadline may be extended, that is upto Amazon, they haven’t notified us yet.

View post 0

Clarification and More Information for Phase 2

@BenediktSchifferer The only difference between the Phase 1 and Phase 2 is the test sets. All new submissions are scored on Phase 2 test sets only.

View post 0

Clarification and More Information for Phase 2

The Phase 2 datasets are available on the Resources page.

They have been available since 9 hours ago when the Phase 2 submissions opened.

View post 0