Loading
15 Follower
29 Following
shivam
Shivam Khandelwal

Organization

AIcrowd

Location

Gurgaon, IN

Badges

11
7
4

Connect

Activity

May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
Jan
Feb
Mar
Apr
May
Mon
Wed
Fri

Ratings Progression

Loading...

Challenge Categories

Loading...

Challenges Entered

Latest submissions

See All
graded 178904
graded 178898

Behavioral Representation Learning from Animal Poses.

Latest submissions

See All
graded 179081
graded 173187
graded 173085

Latest submissions

See All
graded 180080
graded 180079

A benchmark for image-based food recognition

Latest submissions

See All
failed 177330
graded 177329
graded 177328

What data should you label to get the most value for your money?

Latest submissions

See All
graded 177046
failed 176542
failed 176404

Improving the HTR output of Greek papyri and Byzantine manuscripts

Latest submissions

No submissions made in this challenge.

ASCII-rendered single-player dungeon crawl game

Latest submissions

See All
submitted 158196
failed 158192
failed 158191

Latest submissions

See All
graded 158774
graded 147099
graded 141760

Airborne Object Tracking Challenge

Latest submissions

See All
graded 150618
graded 148372
graded 145797

Latest submissions

See All
graded 165005
graded 165004
graded 164936

5 Puzzles 21 Days. Can you solve it all?

Latest submissions

No submissions made in this challenge.

Machine Learning for detection of early onset of Alzheimers

Latest submissions

No submissions made in this challenge.

Sample Efficient Reinforcement Learning in Minecraft

Latest submissions

See All
graded 150880
failed 149894
failed 149002

Latest submissions

See All
graded 167201

Measure sample efficiency and generalization in reinforcement learning using procedurally generated environments

Latest submissions

No submissions made in this challenge.

5 Puzzles 21 Days. Can you solve it all?

Latest submissions

No submissions made in this challenge.

Self-driving RL on DeepRacer cars - From simulation to real world

Latest submissions

No submissions made in this challenge.

Robustness and teamwork in a massively multiagent environment

Latest submissions

No submissions made in this challenge.

3D Seismic Image Interpretation by Machine Learning

Latest submissions

See All
graded 82746
graded 82738
graded 82737

5 Puzzles 21 Days. Can you solve it all?

Latest submissions

No submissions made in this challenge.

Latest submissions

See All
graded 149881
submitted 149879
failed 144887

Play in a realistic insurance market, compete for profit!

Latest submissions

See All
graded 113613
graded 110174
graded 110172

5 Puzzles 21 Days. Can you solve it all?

Latest submissions

No submissions made in this challenge.

5 Puzzles 21 Days. Can you solve it all?

Latest submissions

No submissions made in this challenge.

Multi-Agent Reinforcement Learning on Trains

Latest submissions

See All
failed 84106
failed 84105
failed 84104

A dataset and open-ended challenge for music recommendation research

Latest submissions

No submissions made in this challenge.

A benchmark for image-based food recognition

Latest submissions

See All
graded 59791
failed 59371
failed 31084

Latest submissions

See All
graded 139559
graded 139558
graded 139557

Sample-efficient reinforcement learning in Minecraft

Latest submissions

See All
graded 120070
graded 117123
graded 113473

Latest submissions

See All
graded 125603
graded 125602
graded 125601

5 Puzzles, 3 Weeks. Can you solve them all? 😉

Latest submissions

No submissions made in this challenge.

Multi-agent RL in game environment. Train your Derklings, creatures with a neural network brain, to fight for you!

Latest submissions

No submissions made in this challenge.

Predicting smell of molecular compounds

Latest submissions

See All
failed 114224
failed 114223
failed 114222

Latest submissions

See All
graded 60335
graded 60334

Latest submissions

See All
failed 68839
failed 68599

5 Problems 21 Days. Can you solve it all?

Latest submissions

See All
failed 122822

5 Puzzles 21 Days. Can you solve it all?

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Robots that learn to interact with the environment autonomously

Latest submissions

See All
graded 99115
graded 99114
failed 99109

5 Puzzles, 3 Weeks | Can you solve them all?

Latest submissions

See All
graded 115551
graded 115537
graded 115536

5 PROBLEMS 3 WEEKS. CAN YOU SOLVE THEM ALL?

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Grouping/Sorting players into their respective teams

Latest submissions

No submissions made in this challenge.

5 Problems 15 Days. Can you solve it all?

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Sample-efficient reinforcement learning in Minecraft

Latest submissions

No submissions made in this challenge.

Multi Agent Reinforcement Learning on Trains.

Latest submissions

No submissions made in this challenge.

Recognise Handwritten Digits

Latest submissions

See All
submitted 136812
failed 132176
graded 63126

Online News Prediction

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

5 Problems 15 Days. Can you solve it all?

Latest submissions

See All
graded 67402
graded 66492

Project 2: Road extraction from satellite images

Latest submissions

No submissions made in this challenge.

Project 2: build our own recommender system, and test its performance

Latest submissions

No submissions made in this challenge.

Project 2: build our own text classifier system, and test its performance.

Latest submissions

No submissions made in this challenge.

Help improve humanitarian crisis response through better NLP modeling

Latest submissions

See All
failed 32245

Reincarnation of personal data entities in unstructured data sets

Latest submissions

No submissions made in this challenge.

Robots that learn to interact with the environment autonomously

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

A new benchmark for Artificial Intelligence (AI) research in Reinforcement Learning

Latest submissions

No submissions made in this challenge.

Predict if users will skip or listen to the music they're streamed

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

PlantVillage is built on the premise that all knowledge that helps people grow food should be openly accessible to anyone on the planet.

Latest submissions

No submissions made in this challenge.

Predict Labor Class

Latest submissions

See All
graded 68597

5 PROBLEMS 3 WEEKS. CAN YOU SOLVE THEM ALL?

Latest submissions

See All
failed 85240

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Real Time Mask Detection

Latest submissions

No submissions made in this challenge.

Latest submissions

See All
graded 80051

Predict if users will skip or listen to the music they're streamed

Latest submissions

No submissions made in this challenge.

Recognizing bird sounds in monophone soundscapes

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Predict Wine Quality

Latest submissions

No submissions made in this challenge.

Predict if an online advertisement will be clicked or not

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Detect water bodies from satellite Imagery

Latest submissions

See All
failed 85240

DA Final Project challenges for Monsoon 2020

Latest submissions

No submissions made in this challenge.

ACL-BioNLP Shared Task

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Identify which side has least players

Latest submissions

No submissions made in this challenge.

Predict viewer reactions from a large-scale video dataset!

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Detect Multi-Animal Behaviors from a large, hand-labeled dataset.

Latest submissions

See All
failed 125036

Reinforcement Learning, IIT-M, assignment 1

Latest submissions

No submissions made in this challenge.

Latest submissions

See All
graded 177209

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

See All
failed 149894
failed 149002

Can you dock a spacecraft to ISS ?

Latest submissions

No submissions made in this challenge.

Multi-Agent Reinforcement Learning on Trains

Latest submissions

See All
graded 152700

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Image-based plant identification at global scale

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Round 1 - Completed

Latest submissions

See All
graded 173085

Localization, SLAM, Place Recognition, Visual Navigation, Loop Closure Detection

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Predict Age From A Picture

Latest submissions

See All
failed 182084

Identify Words from silent video inputs.

Latest submissions

See All
graded 182202
failed 182087

Round 2 - Active | Claim AWS Credits by beating the baseline

Latest submissions

See All
graded 179081

A Challenge on Continual Learning using Real-World Imagery

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.
Participant Rating
krishna_kaushik 0
cadabullos 0
jerome_patel 0
amitabh 0
singstad90 0
vrv
jyotish
deepesh_grover 0
leocd 0
ashivani
ayush_upadhyay 0
sanjaypokkali 221
rohitmidha23 265
anssi 221
thomas_cloarec 0
Participant Rating
jyotish
michael_schmuker 0
dmytro_poplavskiy 0
michael_bordeleau 0
leocd 0
ashivani
vrv
shubhankar.sb 0
kartik.gupta0204 0
shubham_sharma 0
rohitmidha23 265
nickinack 0
sanjaypokkali 221
aditya_morolia 0
shraddhaa_mohan 272
hagrid67 103
yoogottamk
piotrekpasciak 0
akhilesh 0
akshatcx -39
pulkit_gera 0
sauravkar 0
nikhil_rayaprolu 190
user18081971 0
marcel
mohanty
MasterScrat 229
anssi 221
aicrowd-bot

Spotify Million Playlist Dataset Challenge

Which is the deadline?

7 days ago

Hi, there is no deadline.
It’s an always running benchmark.

422 Error when accepting challenge rules

About 1 month ago

Hi @kim1,

The issue has been fixed and was happening with selected accounts (2 to be exact) on AIcrowd due to conflicting username.

Sorry about the trouble, and wishing you best with Spotify dataset download and submissions! :rocket:

Any way to get access to this dataset for teaching?

About 2 months ago

Hi @nathan_carter,

Please reach out to the Spotify team for your use case.

The emails along with the research paper are available here:
Recsys Challenge 2018: Automatic Music Playlist Continuation : Spotify Research.

ImageCLEF 2022 Caption - Caption Prediction

Submission deadline

22 days ago

It ends on 06 May 23:59 UTC.

Food Recognition Benchmark 2022

Local Run Produces Different AP

23 days ago

Hi all,

Thanks for opening this discussion, and it is indeed an important one.
I am happy that we all can brainstorm and improve the pipeline for all the participants together. :muscle:

The scores calculated right now we’re using the complete annotation files as @Mykola_Lavreniuk suspected, while the predictions made by your codebases get added for the calculation purposes on the go. The intention has always been to provide real-time feedback to you to show the approximate scores as soon as possible.


Issue #1: Score gradually increases

The flaw
We run COCOEval using complete annotations BUT with the subset of images (predictions generated ones), due to which the score starts with a much lower value and grows to your final scores.

Future update
We will be making an update in the pipeline based on this thread to create a subset of annotations file dynamically, calculate the scores with that for live reporting.


Issue #2: Validation scores don’t make sense

The flaw
We were using only 100 images in the validation phase because it worked as an integration test. We missed providing a proper scores table for the validation phase that you can use to compare local v/s online performance – which is our bad. The scores get calculated with annotation files for the complete validation subset, but only 100 images are present in the validation phase (as in Issue #1), causing much lower scores.

Future update
We will continue use a tiny subset to limit the compute used in validation phase and believe the score on 100 images should be reliable enough to compare local v/s online scoring. The list of these ~100 images will be shared in the resources section in future so you can reproduce the scores locally. (meanwhile the current scores were quite low, example 0.09AP while full evaluation for a lot more, it was happening due to Issue #1)

End of Round 2⏱️

23 days ago

Round 2 of Food Recognition Benchmark has ended !! :rocket:

This round saw a lot more activity and a lot more rank swaps on the leaderboard.
Thanks to you all for making it a success. :raised_hands:

While a few of submissions are being re-evaluated at the moment, the public leaderboard looks like this right now:

The number of participants who crossed the official baseline has increased from 12 to 15.
At the same time, participants having score >= 0.200 AP is increased from 6 to 11 as well! :partying_face:

We can’t wait to complete the due diligence before making the winners’ announcement.
Stay tuned for it! :loudspeaker:


Why Food Recognition Benchmark matters?

It’s a good time to re-share about exciting updates on how the community collaboration has worked so far and what it’s enabling! [Blog Post]

The Food Recognition Benchmark is not a competition, but an ongoing collaboration with a community of inspiring individuals like yourself, to deliver the future of diet tracking in research cohorts around the world.


What can you do meanwhile?

:handshake: Start some interesting discussions with other challenge participants’ here on the Forums or on Discord Chat.
:wink: Share what worked and what didn’t during the competition. And maybe some interesting approaches you may want to try out in future?
:ear: Share any feedbacks you have for future Rounds either in this thread of here.
:speak_no_evil: Guess the winners in this thread

Gitlab seems to be down

24 days ago

Hi, it will be back up within 5 minutes.

Error when building Docker image in active submission

25 days ago

I assume you might be trying to make debug submissions.
Please use debug: false in your aicrowd.json.

Error when building Docker image in active submission

25 days ago

Hi everyone, @unnikrishnan.r, @jesusmolrdv,

The active submission pipeline is back and we have re-evaluated any failed submissions during this duration. :rocket:

What was the issue?
We were caught off-guard when nvidia key rotation happened, and the packages on the upstream used different key causing the error you shared above.

Why it impact your submissions?
It impacted the pipeline due to couple of reasons:

  1. Image Build Failures
    We use nvidia docker image as the base image, which haven’t been re-created recently. Due to this they contain older keys, causing apt to fail on apt -qq update. (and hence your image building process). We have fixed it in our internal Dockerfile to take care of this issue in all the future builds.
    Meanwhile, if you are using your own Dockerfile, please add following command before apt -qq update (or similar)
RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub
  1. GPU infrastructure (causing Preparing the cluster for you error)
    We internally use GPU machines on Kubernetes during the evaluation phase, which was impacted as well. The docker images have been updated and the pipeline is back now.

Wishing everyone all the best ! :wave:
Please let us know in case you face any issue with your future submissions OR your past one hasn’t started again yet
.

Extension of the round?

29 days ago

Hi @Mykola_Lavreniuk,

Given the challenge started on 3rd March (instead of 1st March), the deadline of the challenge was 3rd May as well (not 1st May) from the start. The date available on the challenge overview was correct but the countdown was wrong.

We just fixed the counter on the challenge page to fix this ambiguity.
Let us know in case it clarifies the confusion.

Wishing everyone all the best! :checkered_flag:

Dockerfile Doesn't Exist

About 2 months ago

Hi @lapp09 , GPUs are present only at runtime, not the build.

Dockerfile Doesn't Exist

About 2 months ago

In case you want a different base image or CUDA version due to any reason, you will have to use the Dockerfile to specify your environment. :slightly_frowning_face:
But the good part is that you seems to be comfortable with it. :smile:

Here is one which you can build on top of:
https://github.com/AIcrowd/food-recognition-benchmark-starter-kit/blob/1a806413a3bbb092f682209a69ee908e3a5b3f0d/Dockerfile

Dockerfile Doesn't Exist

About 2 months ago

Hi,

We use nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04 as base image which takes care of it.

Example from one of your submission’s log:
https://gitlab.aicrowd.com/lapp09/food-recognition-challenge-starter-kit/snippets/60580/raw

ESCI Challenge for Improving Product Search

When and How do I submit my code and model?

About 1 month ago

Hi @Alex830,

We will be releasing the code starter kit on 20th May (or a few days before that) along with code submission instructions.

You can expect it to be a git-based interface like in our other code submission challenges.
We will also provide a notebook-based interface for easier implementations (which can help you in making git-based submissions).

The deadline of the challenge is confusing

About 2 months ago

Hi @tsotfsk,

Predictions Submission
We will allow them till the end of the challenge, i.e., July 15 (and not only till May 20 – updated the deadline in the banner).

Code Submission
They will start on May 20 and have the same deadline, i.e., July 15.

Yes, you can update your model in the middle of these two dates.


NOTE: You have to make valid codebase submission before the deadline to be eligible for the prizes, and the scores generated by your code on the AIcrowd evaluation servers will be counted for the prizes leaderboard.

Please let us know in case you have any follow up questions.

Always upload failed

About 2 months ago

Hi, can you share output of command I shared in the last message? It will help us to understand the problem and share solution accordingly.

Always upload failed

About 2 months ago

Hi @doubleQ @BT_Batman ,

Can you try out following commands in your local and share the whole output?
We can debug further using it.

aicrowd -vvvvv submission create -c esci-challenge-for-improving-product-search -f sample_submission.csv track=task1

Always upload failed

About 2 months ago

Based on the error, is it possible that AWS S3 is blocked on your ISP by any chance?

Always upload failed

About 2 months ago

Can you check browser’s console in case it shows any error?

Otherwise you can try making submission using aicrowd-cli as well.

aicrowd submission create -c esci-challenge-for-improving-product-search -f sample_submission.csv track=task1

Data Purchasing Challenge 2022

:rotating_light: Select submissions for final evaluation

About 2 months ago

Hi, we are working on the final evaluations and you can expect the results by 14th April.

Meanwhile the submissions selection is still open and you can select your submissions [if not done already] here:
https://www.aicrowd.com/challenges/data-purchasing-challenge-2022/dynamic_contents

shivam has not provided any information yet.

Notebooks

Create Notebook

Notebooks

Create Notebook