IMPORTANT: Details about end of competition evaluations 🎯

Hi @chuifeng

There will be no live submissions with these changes. You will have to select 2 of your submissions from Round 2 for the final evaluations. Form and instructions selecting submissions will be provided before the competition ends.

View post 0

IMPORTANT: Details about end of competition evaluations 🎯

@ArtemVoronov Its intentional. Please select 4.

View post 0

Ideas for better sim2real transfer

Hi @azam_kamranian

This was your last successful submission on 24th November which is already updated since a few days. Failed submissions are not run on the real track. But seems like the your submission was consistently turning towards the right side so the score is lower than your previous submission.

Please check all your other successful submissions
after 20th November and let me know in case any are missing, we were running the best submission per user per day until recently. We’ll run any older ones in case any are missing.

Please let me know if I can help with anything else, I’ll try my best but might be bit delayed since I’m travelling today.

View post 0

Regarding evaluation criteria for online submissions

Hi @unnikrishnan.r

Sorry I missed reply to this this post earlier.

The reward function for both the rounds is staying close to the center, this is a proxy reward but its good enough as the action space does not allow stopping. The score on the simulation leaderboard is indeed the mean cumulative reward over multiple episodes. The maximum possible reward in both rounds may vary as the tracks are different.

But the main idea of the competition is the sim2real transfer and the formula for the real track is based on waypoints crossed and time. The final prizes are based on the real track leaderboard and a small weightage on the simulation scores. I’ll make a separate post to clarify the formula for the real track.

View post 1

Round 2 is live!

Hi Everyone,

The much awaited Round 2 of the Deepracer Challenge is now live. Submissions will now be run on a real deepracer track everyday until November 30th.

The submission flow will be a follows:

Each team is allowed upto 5 successful submissions everyday, these will run on the Round 2 simulator.

The top scoring submission from each team each day will be run on the real track.

Since the time remaining for Round 2 is short, we encourage everyone to train their models on the Round 2 simulator as soon as possible.

There will be no obstacles on the track, but sim2real transfer will not be a trivial task. Be on the lookout for on board videos from your submissions to help you fine tune your solutions. All the best!!

View post 0

⏰ Round 1 extended till Nov. 10th!

We’re in the process of setting up the track, thanks for the patience.

IMG-20211111-WA0001569×573 29.5 KB

View post 1

Updates for starter kit

Hi @asche_thor

This used to be an older issue but was fixed, are you on the latest version of the env docker release

Can you please pull the latest docker and check once.

View post 0

Some questions ahout the agents' speed

Seems I’d missed answering the question on varying speeds, thanks @sa7show for the answer. Indeed you’re right the speeds for each train are decided at the start of the episode and stays fixed throughout the episode.

View post 1

How to revise “Select Submissions” form?

@fxofxo You can make 3 new submissions in round 2 all of which will be considered for final evaluation, if this is not sufficient let us know we can arrange for changing old ones.

View post 0

Example of using non-neural model

Hi @pajansen

The env returns the observations that are already separated and encoded into a dictionary for convenience, this has nothing to do with neural net. Is there a particular format you need?

View post 0

Colab Notebook including submission and working ttyrec recording

Hi @charel_van_hoof, could you try once with git bash on Windows. I’ll try to check but I think nle is currently not support on windows. But just the ttyplay2 script might work.

View post 0

Submission Error : Inference Failed

Hi @shreesha_g_bhat_cs18

Seems like the error is due to RAM usage of your code. The evaluations run with a limit of 4 GB RAM, please make sure to run within this limit.

Some garbage collection code in python might help.

View post 0

Submission limit

Hi @arnav_anil_mhaske_cs

You should be able to test everything locally then submit.

Can you please share why the extra submissions are needed?

View post 0

Episode Count in Runner

Hi @s_tarun_prasad_me17b

BSuite gives the limit of 10000 as a fair amount episodes needed to converge.

Generally speaking, to have a level playing field among all competitors, while also not making it too easy, some constraint has to he applied. In this case, the number of episodes serves that role.

I know the number 10000 may be arbitrary, so if enough students feel it should be increased, we’ll do it. For now please try to improve your algorithm to get the highest score with 10000 episodes.

Best Regards,
Dipam

View post 1

Welcome to your 2nd Assignment!

Hi @s_tarun_prasad_me17b

Unfortunately the choice of giving extra test cases is not upto me, please talk to the TAs.

In my opinion the test cases provided are sufficient, please review your code properly.

View post 1

Welcome to your 2nd Assignment!

Hi @richa_verma_cs20d020

Can you check if your local scoring cell is working without any error.

In case your local scoring cell gives an error its probably because the output format is wrong, check the targets file for the output format.

If your local scoring cell gives no error please let me know.

View post 0

Welcome to your 2nd Assignment!

Hi @narra_jeevan_reddy_e,

Sounds like a formatting issue with the code on your end, since its not a general issue that affects all students I encourage you to find the bug on your own. With correct format you should get decimals for all algorithms in the local scoring code. Look at the targets for example of the format.

Do let me know in case the problem still persists after you’ve checked it thoroughly.

View post 0

Welcome to your 2nd Assignment!

Hi @jaswanthi_mandalapu

It looks like the error says you need to accept the challenge rules. Can you please accept the rules then try submitting again.

View post 0

Unable to submit post 20th april

Hi @sreenadhuni_kishan_r

We recently fixed this, can you close the notebook and run again from start, that should fix it.

View post 0

Can we use training data from task 1 or 2 for task 3?

Hi @sungbinchoi

Yes, you can use training data of Task 1 for 2 and 3. Feel free to use all the data at your disposal.

You can even use some unsupervised learning on the test sequences.

View post 1