Activity
Ratings Progression
Challenge Categories
Challenges Entered
Evaluate Natural Conversations
Latest submissions
See Allfailed | 246107 | ||
submitted | 246104 | ||
submitted | 246102 |
Multi-Agent Dynamics & Mixed-Motive Cooperation
Latest submissions
See Allgraded | 244563 | ||
graded | 243604 | ||
graded | 237014 |
Advanced Building Control & Grid-Resilience
Latest submissions
See Allfailed | 238874 | ||
failed | 238665 | ||
failed | 238660 |
Specialize and Bargain in Brave New Worlds
Latest submissions
See Allfailed | 237925 | ||
failed | 237924 |
Small Object Detection and Classification
Latest submissions
See Allgraded | 237160 | ||
failed | 237147 | ||
graded | 236860 |
Understand semantic segmentation and monocular depth estimation from downward-facing drone images
Latest submissions
See Allgraded | 214611 | ||
failed | 214573 | ||
failed | 214570 |
Audio Source Separation using AI
Latest submissions
See Allgraded | 236127 | ||
graded | 236125 | ||
failed | 236121 |
Identify user photos in the marketplace
Latest submissions
See Allgraded | 209775 | ||
failed | 209210 | ||
graded | 208987 |
A benchmark for image-based food recognition
Latest submissions
Using AI For Buildingβs Energy Management
Latest submissions
See Allfailed | 235438 | ||
failed | 235434 | ||
failed | 205233 |
Learning From Human-Feedback
Latest submissions
What data should you label to get the most value for your money?
Latest submissions
Interactive embodied agents for Human-AI collaboration
Latest submissions
See Allgraded | 199453 | ||
graded | 199452 | ||
graded | 198521 |
Specialize and Bargain in Brave New Worlds
Latest submissions
Amazon KDD Cup 2022
Latest submissions
Behavioral Representation Learning from Animal Poses.
Latest submissions
See Allgraded | 198630 | ||
graded | 197504 | ||
graded | 197503 |
Airborne Object Tracking Challenge
Latest submissions
ASCII-rendered single-player dungeon crawl game
Latest submissions
See Allgraded | 158823 | ||
failed | 158209 | ||
failed | 158208 |
Latest submissions
Training sample-efficient agents in Minecraft
Latest submissions
Machine Learning for detection of early onset of Alzheimers
Latest submissions
Measure sample efficiency and generalization in reinforcement learning using procedurally generated environments
Latest submissions
Self-driving RL on DeepRacer cars - From simulation to real world
Latest submissions
See Allgraded | 165209 | ||
failed | 165208 | ||
failed | 165206 |
Robustness and teamwork in a massively multiagent environment
Latest submissions
Latest submissions
5 Puzzles 21 Days. Can you solve it all?
Latest submissions
Multi-Agent Reinforcement Learning on Trains
Latest submissions
Latest submissions
See Allgraded | 143804 | ||
graded | 125756 | ||
graded | 125751 |
5 Problems 15 Days. Can you solve it all?
Latest submissions
Learn to Recognise New Behaviors from limited training examples.
Latest submissions
See Allgraded | 125756 | ||
graded | 125589 |
Reinforcement Learning, IIT-M, assignment 1
Latest submissions
See Allgraded | 125767 | ||
submitted | 125747 | ||
graded | 125006 |
IIT-M, Reinforcement Learning, DP, Taxi Problem
Latest submissions
See Allgraded | 125767 | ||
graded | 125006 | ||
graded | 124921 |
Latest submissions
See Allgraded | 128400 | ||
submitted | 128365 |
Latest submissions
See Allfailed | 131869 | ||
graded | 130090 | ||
graded | 128401 |
Latest submissions
See Allfailed | 131869 | ||
graded | 130090 | ||
graded | 128401 |
Latest submissions
See Allgraded | 135842 | ||
graded | 130545 |
Round 1 - Completed
Latest submissions
Round 1 - Completed
Latest submissions
Identify Words from silent video inputs.
Latest submissions
Round 2 - Active | Claim AWS Credits by beating the baseline
Latest submissions
See Allgraded | 198630 | ||
graded | 182252 | ||
graded | 178951 |
Round 2 - Active | Claim AWS Credits by beating the baseline
Latest submissions
See Allgraded | 197504 | ||
graded | 197503 | ||
graded | 182254 |
Use an RL agent to build a structure with natural language inputs
Latest submissions
See Allgraded | 199453 | ||
graded | 199452 | ||
graded | 198521 |
Language assisted Human - AI Collaboration
Latest submissions
See Allgraded | 196399 | ||
graded | 196379 | ||
failed | 196363 |
Latest submissions
See Allsubmitted | 245759 | ||
submitted | 245757 | ||
graded | 200962 |
Estimate depth in aerial images from monocular downward-facing drone
Latest submissions
See Allgraded | 214522 | ||
graded | 214521 | ||
failed | 214517 |
Perform semantic segmentation on aerial images from monocular downward-facing drone
Latest submissions
See Allgraded | 214611 | ||
failed | 214573 | ||
failed | 214570 |
Music source separation of an audio signal into separate tracks for vocals, bass, drums, and other
Latest submissions
See Allgraded | 236127 | ||
graded | 236125 | ||
failed | 236121 |
Source separation of a cinematic audio track into dialogue, sound-effects and misc.
Latest submissions
See Allfailed | 236027 | ||
failed | 236026 | ||
failed | 236024 |
Latest submissions
See Allgraded | 206453 | ||
submitted | 206452 | ||
submitted | 206337 |
Latest submissions
See Allgraded | 223574 | ||
graded | 223573 | ||
graded | 223572 |
Participant | Rating |
---|---|
![]() |
0 |
![]() |
0 |
![]() |
0 |
![]() |
0 |
![]() |
0 |
![]() |
0 |
Participant | Rating |
---|
-
Random-walk Airborne Object Tracking ChallengeView
-
R2D2 NeurIPS 2022: CityLearn ChallengeView
-
dipam_chakraborty Testing NetView
MeltingPot Challenge 2023-8a7d12
MosquitoAlert Challenge 2023
Select submissions for final leaderboard
About 1 month agoHi Everyone,
All teams need to select upto 3 submissions for their final leaderboard entries using the form on this page.
Git Clone Runs infinitely
About 2 months agoHi Everyone,
Weβre looking into whatβs wrong with the submission script. In the meantime, please use the advice from @hca97. The same is also described in the starter kit. Please note the you need to push to the correct remote repo. I think for most participants, this would be set to aicrowd
instead of origin
, please check the name of your remote repo using git remote -v
before trying to push the tag.
Submission failed during Inference
About 2 months ago@saidinesh_pola @gavinwangshuo , both of these failed due to timeout.
Submission failed during Inference
2 months ago@hca97 No it is not the first prediction, I see many print statements before this with varying time values.
Submission failed during Inference
2 months ago@fkemeth Can you give me the submission ids where you get the failure and Iβll check if itβs due to timeout or something else.
Submission failed during Inference
2 months agoThe timeout on these submission does indeed cross 2 seconds for one instance. I believe these are your print statements?
Totla Time 2018.3565616607666 ms
Unfortunately we canβt relax the timing criteria, if possible try to reduce the time taken by your model.
Submissions are quite unstable
3 months ago@MPWARE @tfriedel the time is measured per image. Upon request from the organizer, we have increased this number to 2 seconds per image.
@MPWARE , would it be possible to give a link to your submissions where same image got different times, as its not expected to vary based on cluster workload, your submission container gets the full node.
NeurIPS 2023 Citylearn Challenge
Support for central_agent=False
2 months agoHi Everyone,
We now support setting central_agent=False in the Citylearn Environment. Since the creation of the environment is not directly controlled by participants. You need to provide a separate key in the aicrowd.json file in your starter kits, to let us know to use non central agents.
Things to keep in mind:
- The actions you provide will be passed to the environment as is, so make sure to use the correct action format when using non central agent.
- Make sure the aicrowd.json is formatted correctly as a valid json file.
- Make sure to use βFalseβ in quotes when setting central agent to False in the json file.
All the best for the challenge.
Control Track: CityLearn Challenge
How to load a trained model during AIcrowd evaluation?
2 months ago@skywuuuu You an use git-lfs using the instructions provided in the starter kit.
Sound Demixing Challenge 2023
Error with submissions in MDX23
5 months agoOkay itβs fixed and Iβve rerun it here, you should get the scores once its completed.
Error with submissions in MDX23
5 months agoHi @ZFTurbo
The old base image seems to not be available anymore, can you change to this base image in the Dockerfile
FROM nvidia/cuda:11.7.1-cudnn8-runtime-ubuntu20.04
Amazon KDD Cup '23: Multilingual Recommendation Ch
π¬ Feedback & Suggestions
6 months ago@CPMP 5 submissions per team should be enforced by the UI on a per day basis (00:00 to 00:00 UTC fixed window). We have two settings that the organizers can choose from, 24 hour rolling window and 24 hour fixed window. For this challenge its set to fixed window.
Is your team able to make more than 5 submissions in the fixed window?
π¬ Feedback & Suggestions
6 months agoOn Kaggle a merge is possible only if the sum of submission counts does not exceed 5 times the number of days spent on the competition so far.
Apologies, I wasnβt aware of this.
Iβll mention that we generally do not run competitions without private test sets, which is where this is a much bigger issue. This was an oversight on our part.
I checked the leaderboards and see that there is one team that has managed to make a lot of submissions doing this. After the team merge deadline weβll check for any more unfair merges, and let the Amazon team decide if they want to provide everyone else with additional submissions to keep things fair.
π¬ Feedback & Suggestions
6 months agoAgreed, but this is the system we have, it applies to everyone. If I understand correctly it is also the system on some other platforms.
I understand itβs an outsized advantage for this challenge as itβs not a code based challenge and doesnβt have a private test set.
Weβre open to alternative ideas for future challenges, please share your feedback here or in dm.
π¬ Feedback & Suggestions
6 months agoYes, every team is limited to 5 submissions per day. After merging teams, submissions limits will apply to the whole team.
π¬ Feedback & Suggestions
6 months ago@ECNU_Wei_Zhang Please ask all you team members to dm me and confirm that they want to leave the team.
Clarification and More Information for Phase 2
7 months ago@ZhipengZhang Thanks for letting me know. Leaderboards should be displayed correctly now.
Notebooks
-
MosquitoAlert - YoloV5 Baseline Submission MosquitoAlert - YoloV5 Baseline SubmissiondipamΒ· 5 months ago
-
[Getting Started] ETH PSC Summer School Hackathon This is a Baseline Code to get you started with the challenge.dipamΒ· About 1 year ago
-
Baseline - BERT Classifier - BM25 Ranker Official baseline that uses BERT based classifier and BM25 rankerdipamΒ· Over 1 year ago
-
Unsupervised model - SimCLR - Ant-Beetles Video Data Unsupervised model training using contrastive learning with modified SimCLRdipamΒ· Over 1 year ago
-
Unsupervised model - SimCLR - Mouse Video Data Unsupervised model training using contrastive learning with modified SimCLRdipamΒ· Over 1 year ago
-
Getting Started - Mouse-Triplets Video Data Initial data exploration and a basic embedding using a vision modeldipamΒ· Over 1 year ago
-
Getting Started - Ant-Beetles Video Data Initial data exploration and a basic embedding using a vision modeldipamΒ· Over 1 year ago
-
BSuite Challenge Starter Kit IITM RL Final Project Bsuite starter kit with random baselinedipamΒ· Over 2 years ago
-
Solution for submission 128367 A detailed solution for submission 128367 submitted for challenge IIT-M RL-ASSIGNMENT-2-GRIDWORLDdipamΒ· Over 2 years ago
-
Solution for submission 130090 A detailed solution for submission 130090 submitted for challenge IIT-M RL-ASSIGNMENT-2-GRIDWORLDdipamΒ· Over 2 years ago
-
Solution for submission 128401 A detailed solution for submission 128401 submitted for challenge IIT-M RL-ASSIGNMENT-2-GRIDWORLDdipamΒ· Over 2 years ago
-
Solution for submission 128400 A detailed solution for submission 128400 submitted for challenge IIT-M RL-ASSIGNMENT-2-TAXIdipamΒ· Over 2 years ago
-
Taxi Notebook IITM RL Assignment 2 Notebook to be filled for IITM RL Assingnment 2 TaxidipamΒ· Over 2 years ago
-
Gridworld Notebook IITM RL Assignment 2 Notebook to be filled for IITM RL Assingnment 2 GridworlddipamΒ· Over 2 years ago
Evaluation Score Update
About 1 month agoTL;DR: We are re-calculating scores using the correct normalization. We donβt expect any significant changes in the ranking, only on the numerical values.
Dear Participants - We are making an amendment to the evaluation score shown on the leaderboard. Unfortunately we noticed that score computation had a normalisation discrepancy with that of Melting Pot 2.0. We donβt expect the ranking of any of the submissions to change, only the numerical score. The Melting Pot 2.0 tech report defines a normalization of return for a scenario to be zero for the baseline with the lowest focal per-capita return, and one for the highest one. The normalization currently reported used the exploiter return from Melting Point 2.0 as the upper bound and random performance as lower bound. For most scenarios these are the same, but there are some for which this is not the case.
Going forward we are updating the leaderboard score to use the normalization from Melting Pot 2.0. We will use this updated normalization for the generalization score to be displayed from Nov 1 too.
Please note: This change will only reflect as a scale factor on overall score where in your performance on some scenarios may slightly decrease by the same scale for all teams and hence we do not expect any significant change in the current leaderboard ranking due to this. We will be using the new score moving forward for both development and generalization phase and also final evaluation.