Question about submission

Hi everyone,

Anyone who got a " Submission failed : You have not qualified for this round" previously, please retry your submissions, they should work now.

View post 0

Question about submission

Hi @Tomasyu can you give the link to your submission please.

View post 0

Questions about submissions

Hi @subatomicseer

Leaderboard A and B submission for MDX track are now enabled. The instructions are added here. In short, the keys “labelnoise” and “bleeding” have to be added to aicrowd.json

View post 0

Aicrowd-bot adds people to my private submission repository for CDX track

Hi @subatomicseer

All the users added above are part of the organizing team. By default, we provide access to the user submitted code to the organizing team to help debug any issues participants face. However, the SDX organizers categorically requested that they not have access to any user submitted code, and we have enabled the same. The organizing team will be added to any new repositories that are used for submissions. We have also retrospectively ensured that no one from the organizing team has access to any of the repositories already submitted by the participants, including yours.

View post 0

💬 Feedback and suggestion

The baseline was trained on the training set shared, and the current dataset used for the leaderboard was used as validation. Hence, unfortunately until that set is made public (it will be a few weeks before the end of challenge when the actual test set is ready), its not possible to reproduce the results right now.

View post 1

Error in local evaluation

Hi @sagar_rathod

We’ve updated the file in the Resources section - suadd_23_semantic_annotations-v0.2.zip

Let me know if you still get the error with the updated dataset.

View post 0

💬 Feedback and suggestion

Hi @adithya_t_p

The validation set is used for the public leaderboard, this set will be made public at a later stage in the competition as the test set is still in development by the organizers. So for now you have to make your own dataset split.

View post 1

💬 Feedback and suggestion

@adithya_t_p

The baseline should work with this commit

Thanks for pointing it out, we’ll add a note in the baseline repo.

View post 0

How to install mmcv-full while submission

@xav_c @FY1 , Please resubmit any submissions you made for which the tag didn’t reflect on the issues page. It was a problem on our end, its fixed now.

View post 0

How to install mmcv-full while submission

Can you dm me the repository link @FY1

View post 0

Why my submission is failed?

Added error log on the gitlab comments

View post 0

Why my submission is failed?

Please ensure that your code has no hardcoded values, hardcoded dictionary keys, etc. Also make sure local evaluation works.

If you’re still unable to find the error, I can check the logs manually, but this will be slow as I might not be available always and we cannot show the logs any other way.

View post 0

Why my submission is failed?

Hello,

@strekalov @victorkras2008

The validation step is only a debugging dataset as we can’t give logs on the main dataset to prevent data leaks.

View post 0

Can't download the public test dataset

@tomtom , I’m still unable to replicate the problem as the link is working for me. I tried it with my personal (non-admin account) and it still works.

Would it be possible for you to try it with a different internet connection?

View post 0

Error on baseline submission

Hi @victorkras2008

Thanks for finding this, its fixed now.

View post 0

Can't download the public test dataset

Hi @tomtom

It’s working for me, could you share what error you’re getting?

View post 0

Update in mono-depth evaluation method

@xav_c

I did notice that, however your submission didn’t get NaN scores. I’ll need some time to investigate, should be resolved by tomorrow.

View post 1

Update in mono-depth evaluation method

Hi Victor,

Thanks for pointing this out, the SILog had gone to NaN because your predictions were too large, I think you probably output the depth values without the scaling factor as shown in local_evaluation.py? The submission is now marked as failed.

Please check this function for reading depth images. The values should be closer to ~1-50.

View post 0

Update in mono-depth evaluation method

Hi everyone,

We have made a change in the evaluation setup, to allow for models that predict relative depth maps from mono image inputs. We now apply an alignment step to the model to align a relative depth prediction to the ground truth. For more details please refer to this paper Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer.

Updated code for local evaluation of your models can be found in the starter-kit . All valid old submissions will be re-evaluated. All future submissions will be evaluated with these updated metrics.

View post 0

Update in segmentation scoring metrics

Hi everyone,

We have made a change in the way metrics are calculated, to better match the standard practice used in segmentation literature. The IoU and Dice scores are now calculated per class on the entire dataset and then averaged, instead of per image. As a result, most submissions are expected to get slightly higher scores.

Updated code for local evaluation of your models can be found in the starter-kit. All valid old submissions will be re-evaluated within the next few days. All future submissions will be evaluated with these updated metrics.

View post 0