Round 1: Completed Round 2: Completed Post-Challenge Round: Completed

MEDIQA 2019 - Natural Language Inference (NLI)

ACL-BioNLP Shared Task


The MEDIQA challenge is an ACL-BioNLP 2019 shared task aiming to attract further research efforts in Natural Language Inference (NLI), Recognizing Question Entailment (RQE), and their applications in medical Question Answering (QA).

Natural Language Inference Task (NLI)

The objective of this task is to identify three inference relations between two sentences: Entailment, Neutral and Contradiction.


Training set: https://jgc128.github.io/mednli/ Participants will have to obtain access to MIMIC in order to access MedNLI and the test set.


  • April 15, 2019: Release of the test sets for the 3 tasks.
  • April 30, 2019: Run submission deadline. Participants’ results will be available on AIcrowd.
  • May 15, 2019: Paper submission deadline.
  • August 1, 2019: BioNLP workshop, ACL 2019, Florence, Italy.

Evaluation criteria

For the NLI and RQE tasks:

  • The evaluation will be based on Accuracy.
  • For the result submission file, we expect a csv file with the following header: pair_id,label (cf. ValSet_Baseline.csv of the RQE task)


1) Each team is allowed to submit a maximum of 5 runs.

2) Please choose a username that represents your team, and update your profile with the following information: First name, Last nam, Affiliation, Address, City, Country.

3) For each run submission, it is mandatory to fill in the submission description field of the submission form with a short description of the methods, tools and resources used for that run.

4) The final results will not be considered official until a working notes paper with the full description of the methods is submitted.

Contact us



01 varun764 0.988
02 arjun_nemani 0.714
03 hmehta4 0.002
03 mlu13 0.002