Challenge Rules

​​ RAG Benchmark Competition Official Rules

NO PURCHASE NECESSARY TO PARTICIPATE IN THIS COMPETITION. A PURCHASE WILL NOT INCREASE YOUR CHANCES OF WINNING. PARTICIPATION DOES NOT CONSTITUTE AN OFFER TO BUY SPONSOR'S (AS DEFINED BELOW) PRODUCTS OR SERVICES. VOID WHERE PROHIBITED BY LAW.

ACCESS TO INTERNET AND A PERSONAL COMPUTER, EMAIL ADDRESS, AND ACCOUNT WITH AICROWD (AS DEFINED BELOW) ARE REQUIRED TO PARTICIPATE.

IMPORTANT: PLEASE READ THESE OFFICIAL RULES, WHICH ARE A CONTRACT, CAREFULLY. WITHOUT LIMITATION, THIS CONTRACT INCLUDES INDEMNITIES TO SPONSOR FROM YOU AND A LIMITATION OF YOUR RIGHTS AND REMEDIES. BY PARTICIPATING, YOU AGREE TO BE BOUND BY THESE OFFICIAL RULES AND REPRESENT THAT YOU SATISFY ALL OF THE ELIGIBILITY REQUIREMENTS.

(1) OVERVIEW: The RAG Benchmark Competition ("Competition") will take place from 3/20/2024 at 10:00:01 AM Pacific Time ("PT") to 6/20/2024 at 23:59:59 PM PT ("Competition Period"). Following KDD Cup tradition, Participants may register at any time within the Competition Period. The earlier Participants begin, the more time they have to work on their Prototypes (as defined below).Qualifying individuals must register at: https://www.aicrowd.com/challenges ("Competition Site", which is incorporated into these Rules through this reference) during the Competition Period. Registration may require agreement to terms (see https://www.aicrowd.com/participation_terms and https://www.aicrowd.com/terms). These Official Rules ("Rules") govern the participation in the Competition. This is only a high-level summary of the Competition; read through these entire Rules and the Competition Site to make sure you understand the details.

Competition Timeline:

Date	Event
4/1/2024	Phase I begins
5/22/2024	Phase II begins
6/20/2024	Competition ends
7/15/2024	Results Announced
7/31/2024	Paper Due
8/26/2024	Winners Announced in KDD

(2) SPONSOR: If you participate as a resident of, or from within, the United States, the sponsoring entity of the Competition is Meta Platforms, Inc., 1 Hacker Way, Menlo Park, CA 94025. For all other participants, the sponsoring entity is Meta Platforms Ireland Limited, 4 Grand Canal Square, Dublin 2, Ireland. Either or both Meta entities will be referred to as "Sponsor."

(3) ELIGIBILITY: This Competition is open only to individuals who, as of the start of the Competition Period and through the awarding of any prizes are at least eighteen (18) years of age and the age of majority in their jurisdiction of residence and possess a valid email address through the completion date of the Competition. You are not eligible to participate if you (i) reside in a country or jurisdiction that is the target of U.S., EU, United Nations, or UK comprehensive trade sanctions (e.g., Crimea, Donetsk, and Luhansk regions of Ukraine, Cuba, North Korea, Iran, and Syria, as such list may be amended); or (ii) are the target of any trade sanctions or export controls administered or enforced by the U.S., EU, United Nations, or UK or are acting on behalf of parties that are the target of such trade sanctions or export controls. Participants (as defined below) must have experience with coding.

Employees, officers, directors, members, managers, agents, and representatives of Sponsor, and its parent and subsidiary companies, affiliates, divisions, representatives, consultants, sub-contractors, suppliers, distributors, legal counsel, prize providers, administrators, advertising, public relations, promotional, fulfillment and marketing agencies, and The Association for Computing Machinery ("ACM") (collectively, the "Released Parties") and members of their immediate families (defined for these purposes as including spouse, domestic partner, parents, legal guardian, legal ward, children, and siblings and each of their respective spouses) and individuals living in the same household as such individuals, are not eligible to participate. Participation in this Competition constitutes Participants' full and unconditional agreement to and acceptance of these Rules and the decisions of Sponsor.

(4) HOW TO ENTER: During the Competition Period, eligible individuals may complete the registration form available at the Competition Site, including all required information, such as name and email address. Registration requires acceptance of the Competition Rules.

To register as a Team (as defined below): The Team Representative (as defined below) must create a Team. Then invite up to four (4) other eligible individuals to join their Team. Those invited individuals can accept or deny the invitation. A Team is only valid for this Competition.

To create a Team: Register to participate in the Competition and select the "Create Team" button to create a Team. Choose a name for your Team. Please note that the name cannot be changed and must adhere to Content guidelines below.

To invite Team members: Select "Invite Members" to add up to four (4) potential teammates and enter their email or AIcrowd username. Upon accepting an invitation, that individual becomes a member of the specified Team for the duration of the Competition. Each eligible individual may only join one Team for the Competition, and once an invitation is accepted, a Participant cannot join any other Team for the duration of the Competition. Any additional Team invitations received after joining a Team will be automatically canceled. If an individual creates another Team with no other members, it will be deleted upon accepting an invitation. All pending invitations for the previous Team will also be automatically canceled. Once an invitation is accepted, you cannot cancel your Team members for the duration of the Competition.

Limit one (1) registration per Participant (either individually or as a Team). Each Participant (either individuals or Teams) can choose to complete in 1, 2, or 3 tasks. Individuals may only participate as an individual Participant OR a part of a Team. Individuals can only be a part of one (1) Team. Registration must indicate whether the Participant will participate individually or as a member of a Team of up to five (5) eligible individuals (each, a "Team"). Where distinction is not necessary individual Participants and Teams will be referred to as "Participants" in these Rules. When prompted, each Team must appoint and authorize one individual (the " TeamRepresentative")to represent, act, and enter, on their behalf. All members of a Team must meet the eligibility requirements listed above.

Attempts made by an individual to participate in excess of any stated limit are void and persons engaging in such conduct may, in Sponsor's sole discretion, be disqualified. Registration must be submitted by a Participant to Sponsor during the Competition Period in strict accordance with the instructions and restrictions in these Rules and any other instructions that Sponsor may provide through its authorized representatives or otherwise. Registrations that are not complete when submitted or received by Sponsor and its representatives during the Competition Period will not be considered for the Competition. Purported proof of registration (such as, without limitation, a screenshot of your purported registration) does not constitute proof of actual registration for purposes of this Competition. Purported registration that is incomplete, received outside the Competition Period, or contains obscene, offensive, or any other language or imagery communicating messages inconsistent with the positive images with which Sponsor wishes to associate itself (all as determined by Sponsor in its sole discretion) will be void. Those who do not abide by these Rules and the instructions of Sponsor and its representatives and provide all required information may, in Sponsor's sole discretion, be disqualified and any purported entry by such person void. Entry that is fraudulent, forged, altered, incomplete, lost, late, misdirected, mutilated, illegitimate, incomprehensible, garbled, or generated by a macro, bot, or other automated means will not be accepted and will be void. Entry made by someone on behalf of any other individual or any entity or group, made by another on your behalf, or originating at any online service other than as specifically described above (including, without limitation, through a commercial promotion subscription, notification, or entering service) will be declared invalid and disqualified for this Competition. As a condition of participating in the Competition, without limiting any other provision in these Rules, each Participant gives consent for Sponsor and its agents to obtain and deliver their name and other information and content to third parties, specifically including ACM for the purpose of administering this Competition and complying with applicable laws, regulations, and rules, and potential participation in ACM KDD 2024 ("Convention"). YOU SHOULD RETAIN A COPY OF YOUR REGISTRATION, CONTENT SUBMITTED DURING THE COMPETITION, AND ALL RECORDS OF YOUR PARTICIPATION. SPONSOR IS NOT RESPONSIBLE FOR PROVIDING A COPY OR REPORT OF ANY ELEMENT OF PARTICIPATION.

(5) COMPETITION TASKS: There will be three (3) tasks for the Competition. Participants can partake in 1, 2, or all 3 tasks. Participants will be given benchmarking datasets and will be instructed to create a system that uses different sources of ground truth knowledge for each one. The final deliverables for the Competition will be system prototypes for each task ("Prototype"). LLaMA-2 is required to submit Prototypes. Creation of a LLaMA-2 account requires agreement to terms, available here:https://www.facebook.com/policies_center/ and here: llama.meta.com/llama-downloads, which are hereby incorporated into these Rules. Prototypes will be scored as described below. Submission limits will be communicated on the Competition Page during the competition. All Team members can make submissions for the task, but submissions count against the daily limit.

Validation and public test sets will be shared at the beginning of the Competition. The private test set will be held out for the final evaluation. For validation purposes, the winners must submit their code and a written technical report describing their model and solution to Sponsor for a final review. Formatting instructions, instructions to use the mock API, validation and public test sets will be provided. Participants may use external data to train their system but must declare the external data sources in their submission.

Use of external resource s: Participants are encouraged to exploit public resources to build their solutions. Please ensure that the used datasets or models are publicly available and equally accessible to use by all Participants. Participants may re-formulate existing datasets (e.g., adding additional data/labels manually or with Llama models), but winners are required to make them publicly available after the Competition.

Phase I: Participants can submit their solutions to be evaluated by Auto-eval (as defined below). Participants will see their scores and rank on the public test set from the leaderboard. Participants can submit up to the submission limit.

Phase II: Participants only have two (2) chances to submit a final solution. The highest score between the submissions on the private test set will determine the winners of the Competition.

Scoring: Each perfect, acceptable, missing, or incorrect answer will be scored Sp, Sa, Sm, and Sin, respectively. For example, Sp = 1, Sa = 0.5, Sm = 0, and Sin = -1. Hallucinated answers will be penalized. Preference will be given to missing answers over incorrect answers.

An average score will be computed for each domain. Macro average across all domains will be taken (i.e., giving the same weight to all domains). Within each domain, the questions will be clustered into sub-domains, and each sub-domain will be weighted according to distribution observed in real traffic. The weighted average within each domain will be computed, but subdomain weights will not be released.

Evaluation Method: Both model-based automatic evaluation ("Auto-eval") and human evaluation ("Human-eval") will be employed. First, Auto-eval will be used to select the top five to ten (5-10) Prototypes. Then, Human-eval will be used to decide the final top three (3) Prototypes for each task. A time-out is applied after the first token was generated. Only Llama-2 models (downloaded from either http://ai.meta.com/, https://llama.meta.com/llama-downloads/ or https://huggingface.co/TheBloke/Llama-2-70B-GGML will be supported for this Competition. Other base models are not allowed on the leaderboard or for the prizes.

Automatic Evaluation: In Auto-eval only three scores are considered: correct (merging perfect and acceptable), missing, and incorrect, and use score 1, 0, -1 for them. The resulting score is effectively Accuracy – Hallucination where Accuracy, Hallucination and Missing are the percentage of correct, incorrect, and missing answers in the test set. These score choices penalize incorrect answers, while awarding correct answers only.

Test Sets: Test data is split into three (3) sets with similar distribution: validation, public test, and private test.

Any and all aspects of participation in the Competition, including without limitation, any Competition-related material or information submitted by a Participant, and Prototypes (collectively, "Content") must meet all of the following requirements, as determined by Sponsor in its sole discretion, or the associated participation may be disqualified:

• Content must be original to you (or in the public domain) and may not infringe, misappropriate, or violate any rights of any third party including, without limitation, copyright (including moral rights), trademark, trade secret, or rights of privacy or publicity.
• Content must not include any malware, spyware, viruses, or similar harmful code or content.
• Content must not include information or content that is false, fraudulent, deceptive, misleading, defamatory, libelous (including trade libel), disparaging, harassing, threatening, profane, obscene, pornographic or otherwise adult-oriented, hateful, indecent, inappropriate, or injurious to any Released Party or any other party.
• Content must not contain or describe any harmful or illegal activity or content or in any way violate any laws, rules, or regulations.
• You agree that your participation in this Competition and agreement to these Rules will not violate any agreement to which you are a signatory or party and that no such agreement otherwise limits your ability to participate in this Competition or grant the rights granted by you in these Rules.

Sponsor reserves the right in its sole discretion to disqualify you from the Competition if in Sponsor's sole discretion your Content does not comply with these Content requirements or if you do not comply with any other requirement of these Rules.

(6) WINNER SELECTION/NOTIFICATION: The tasks completed during the Competition will be judged based by Sponsor -selected ("Judge(s)") as described below. Prototypes will be ranked according to total points awarded by the Judges. The Prototypes with the three (3) highest total scores will be the winners, subject to verification. In the event of a tie in the selection of the winner of a prize, the Judges will name the potential winner for that prize based on the system that was built on a smaller model or uses less resources, subject to verification.

The winner determination will be made during the Competition. Winners will be announced at the end of the Competition. Potential winners will be required to complete, sign and return a form presented by Sponsor and other documents (if necessary, which largely affirm the rights granted and obligations contained in these Rules) (collectively, the "Prize Winner Documents") according to Sponsor's instructions. Failure to comply with these requirements, Sponsor's or its representative's instructions, or these Rules may, in Sponsor's sole discretion – having due regard of the interests of the potential winner, result in disqualification from the Competition and forfeiture of any prize potentially won. If a Team wins any prize(s), the Team Representative will be responsible for equitably distributing the prizes to all members of the Team without any involvement of Sponsor. Sponsor shall have no liability for the prize after it has been distributed to the Team Representative and is not responsible for the Team Representative's actions with respect to the prize distribution. The Prize Winner Documents are each subject to verification by Sponsor and may require the Participant to provide a copy of government-issued identification card or number therefrom. Sponsor will instruct Team Representatives how to handle Prize Winner Documents among a Team. If any prize, prize notification, or other Competition-related communication is returned as undeliverable or if a selected potential winner cannot be reached or does not respond as instructed after Sponsor has attempted to notify that potential winner, that selected winner may – after the sponsor has taken reasonable efforts to reach the selected winner – be disqualified and an alternate winner may be selected (time permitting and in Sponsor's sole discretion). The prize claim and Prize Winner Documents are subject to verification by Sponsor. The prizes, if legitimately claimed, will be awarded. Sponsor will not be obligated to pursue more than two (2) alternate winners (time permitting) for any reason.

(7) PRIZES, QUANTITY, & APPROXIMATE RETAIL VALUE ("ARV"): The Participants whose Prototypes receive the ten (10) highest scores will receive the following prizes:

• First prize (highest score): $4,000 USD
• Second prize (second-highest score): $2,000 USD
• Third prize (third-highest score): and $1,000 USD
• Fourth to Tenth place: $500, each

Total ARV of all prizes: $31,500 USD

Winners may be invited to present at the Convention (ACM KDD 2024 Sunday 25 August 2024 through Thursday 29 August 2024 in Barcelona). Winners who are invited to present at the Convention are responsible for all costs to attend the Convention, including Convention registration fees and any travel costs, if applicable. Sponsor will not cover any Convention or travel-related costs. Attendance at the Convention is not required to be named a winner. Monetary prizes will be paid out by AICrowd SA, EPFL Innovation Park, Bâtiment C, c/o Fondation EPFL Innovation Park, 1015 Lausanne ("AICrowd").

Limit three (3) prizes per Participant, (one per track). The Released Parties are not responsible for, and winner will not receive, the difference between the actual value of the prize at the time of award and the stated ARV in these Rules or in any Competition-related correspondence or materials. Sponsor is not responsible for a potential winner's inability to accept or use the prize for any reason. Prize details will be decided at Sponsor's sole discretion. Prizes will be shipped after the Competition ends. Prizes are distributed based on participation in the Competition and are not limited to or conditioned upon the purchase of Sponsor's products or services. If a winner cannot receive a prize from Sponsor per such winner's employer's policies, such winner will forfeit the prize won and Sponsor will have no further obligation to that winner.

Any taxes and other costs and expenses associated with prize acceptance or use and not specified in these Rules as being part of the prize will be the sole responsibility of the winners. In the event the prize is awarded to a Team, any taxes and other costs and expenses will be the sole responsibility of the Team Representative. Winner may be issued a tax form for the actual value of the prize. No more than the stated prizes will be awarded. Sponsor will not replace any lost, mutilated, or stolen prizes or prize elements or any prizes that are undeliverable or do not reach the winner because of an incorrect or changed contact information. If a winner does not accept or use the entire prize, the unaccepted or unused part of the prize will be forfeited, and Sponsor will have no further obligation with respect to that prize or portion of the prize. No transfers, prize substitutions, or cash redemptions will be made, except at Sponsor's sole discretion. Winner is strictly prohibited from selling, auctioning, trading, or otherwise transferring any part of the prize, except with Sponsor's permission, which may be granted or withheld for any reason in its sole discretion. Sponsor reserves the right to substitute any prize or portion thereof with another prize or portion thereof of equal or greater value for any reason, including unavailability of the stated prize. Each Participant waives the right to assert as a cost of winning any prize any and all costs of verification and redemption or and any liability and publicity which might arise from claiming or seeking to claim said prize.

(8) LICENSE: Each Participant will retain ownership of, and all intellectual and industrial property rights to, their Prototype and all Content submitted; provided that, as a condition of participation, Participants hereby grant Sponsor and any recipients of software distributed by Sponsor: (i) a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute their Prototype(s) and related Content and derivative works thereof, and (ii) a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Prototype. In addition, Participants agree that Sponsor shall have a perpetual, irrevocable, world-wide, royalty-free, transferable, sublicenseable right to use, copy, distribute, modify and make publicly available Participants' Content for the purposes of the operation, conduct, administration, and promotion of the Competition, for internal research and development purposes, and for any marketing or promotional purposes, without further review, notice, approval, consideration, or compensation beyond the opportunity to win a prize in this Competition. You agree to sign any necessary documentation that may be required by Sponsor or its designees to make use of the rights you granted herein. Nothing in these Rules shall be construed as granting you any right or license under any intellectual property right of Sponsor.

By entering the Competition (except where prohibited by law), each Participant grants the Released Parties the irrevocable, sublicensable, free-of-charge, absolute right and permission to use, publish, post or display their name, photograph, likeness, voice, biographical information, any quotes attributable to them, and any other indicia of persona (regardless of whether altered, changed, modified, edited, used alone, or used with other material in the Released Parties' sole discretion) for advertising, trade, promotional and publicity purposes without further obligation or compensation of any kind to them, anywhere worldwide, in any medium now known or hereafter discovered or devised (including, without limitation, on the Internet) without any limitation of time and without notice, review or approval, and each such person releases all Released Parties from any and all liability related to such authorized uses. Nothing contained in these Rules obligates Sponsor to make use of any of the rights granted herein and each natural person granting publicity rights under this provision waives any right to inspect or approve any such use.

(9) LIMITATION OF LIABILITY: Each Participant hereby acknowledges and agrees that the relationship between the Participant and each of the Released Parties is not a confidential, fiduciary, or other special relationship, and that the Participant decision to participate for purposes of the Competition does not place any of the Released Parties in a position that is any different from the position held by members of the general public with regard to elements of the Content, other than as set forth in these Rules. Each Participant understands and acknowledges that the Released Parties have wide access to ideas, text, images, code, registrations, software, and other creative materials. Each Participant also acknowledges that many ideas for products, services, and advertising may be competitive with, similar to, or identical to their Content and/or each other in idea, function, components, format, or other respects. Each Participant acknowledges and agrees that such Participant will not be entitled to any compensation as a result of any Released Party's use of any such similar or identical material that has or may come to such Released Party from other sources. Each Participant acknowledges and agrees that Sponsor does not now and will not have in the future any duty or liability (direct or indirect; vicarious, contributory, or otherwise) with respect to the infringement or protection of the Participant's or any third party's patent, copyright or other proprietary rights in and to their Content.

TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, EACH PARTICIPANT AGREES TO RELEASE, HOLD HARMLESS, AND INDEMNIFY EACH OF THE RELEASED PARTIES FROM AND AGAINST ANY LIABILITY WHATSOEVER FOR INJURIES OR DAMAGES OF ANY KIND SUSTAINED IN CONNECTION WITH THE ACCEPTANCE, USE, MISUSE, OR AWARDING OF A PRIZE OR WHILE PREPARING FOR, OR PARTICIPATING IN ANY PRIZE- OR COMPETITION-RELATED ACTIVITY INCLUDING, WITHOUT LIMITATION, ANY INJURY, DAMAGE, LOSS, DEATH OR ACCIDENT TO OR OF PERSON OR PROPERTY. THE PRIOR LIMITATION ON DAMAGES IS NOT INTENDED TO LIMIT THE RELEASED PARTIES' OBLIGATION (IF ANY) TO PAY PREVAILING PARTY COSTS OR FEES IF RECOVERABLE PURSUANT TO APPLICABLE LAW. THE LIMITATIONS SET FORTH IN THIS SECTION WILL NOT LIMIT OR EXCLUDE THE RELEASED PARTIES' LIABILITY FOR PERSONAL INJURY OR TANGIBLE PROPERTY DAMAGE CAUSED BY THE RELEASED PARTIES, OR FOR THE RELEASED PARTIES' GROSS NEGLIGENCE, FRAUD, OR INTENTIONAL, WILLFUL, MALICIOUS, OR RECKLESS MISCONDUCT. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, EACH WINNER AGREES THAT THE PRIZES ARE PROVIDED AS-IS WITHOUT ANY WARRANTY, REPRESENTATION, OR GUARANTEE, EXPRESS OR IMPLIED, IN FACT OR IN LAW, WHETHER NOW KNOWN OR HEREINAFTER ENACTED, RELATIVE TO THE USE OR ENJOYMENT OF THE PRIZE, INCLUDING, WITHOUT LIMITATION, THEIR QUALITY, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.

(10) DISPUTES/GOVERNING LAW: EXCEPT WHERE PROHIBITED, AS A CONDITION OF PARTICIPATING IN THIS COMPETITION, EACH PARTICIPANT AGREES THAT ANY AND ALL DISPUTES THAT CANNOT BE RESOLVED BETWEEN THE PARTICIPANT AND ANY RELEASED PARTY, CLAIMS AND CAUSES OF ACTION ARISING OUT OF OR CONNECTED WITH THE SURVEY, THIS COMPETITION, OR THE PRIZE AWARDED, OR THE DETERMINATION OF A WINNER MUST BE RESOLVED INDIVIDUALLY, WITHOUT RESORT TO ANY FORM OF CLASS ACTION.

THIS COMPETITION, THESE RULES, AND ANY DISPUTE ARISING UNDER OR RELATED TO THIS COMPETITION AND/OR RULES (WHETHER FOR BREACH OF CONTRACT, TORTIOUS CONDUCT OR OTHERWISE) WILL BE GOVERNED, CONSTRUED, AND INTERPRETED UNDER THE INTERNAL LAWS OF THE STATE OF CALIFORNIA, USA, WITHOUT REFERENCE OR GIVING EFFECT TO ITS CONFLICTS OF LAW PRINCIPLES OR RULES THAT WOULD CAUSE THE REGISTRATION OF ANY OTHER STATE'S LAWS AND, IF THAT IS NOT POSSIBLE, THEN IF THAT IS NOT POSSIBLE, THEN THE LAWS OF THE UNITED KINGDOM. ANY LEGAL ACTIONS, SUITS, OR PROCEEDINGS RELATED TO THIS COMPETITION (WHETHER FOR BREACH OF CONTRACT, TORTIOUS CONDUCT, OR OTHERWISE) WILL BE BROUGHT EXCLUSIVELY IN THE STATE OR FEDERAL COURTS LOCATED IN OR HAVING JURISDICTION OVER SAN MATEO COUNTY, CALIFORNIA, USA AND EACH PARTICIPANT IRREVOCABLY ACCEPTS, SUBMITS, AND CONSENTS TO THE EXCLUSIVE JURISDICTION AND VENUE OF THESE COURTS WITH RESPECT TO ANY LEGAL ACTIONS, SUITS, OR PROCEEDINGS ARISING OUT OF OR RELATED TO THIS COMPETITION, AND IF THAT IS NOT POSSIBLE, THE SUCH ACTIONS, SUITS OR PROCEEDINGS WILL BE BROUGHT IN THE COURTS HAVING JURISDICTION OVER LONDON, UNITED KINGDOM. UNLESS PROHIBITED BY APPLICABLE LAW, YOU WAIVE ANY AND ALL OBJECTIONS TO JURISDICTION AND VENUE IN THESE COURTS AND HEREBY SUBMIT TO THE JURISDICTION OF THOSE COURTS. IF REQUIRED UNDER APPLICABLE LAW, NOTHING HEREIN WILL LIMIT THE RIGHT OF ANY PARTICIPANT TO BRING PROCEEDINGS (INCLUDING THIRD PARTY PROCEEDINGS) AGAINST SPONSOR IN A COURT OF COMPETENT JURISDICTION LOCATED WITHIN THE PARTICIPANT'S JURISDICTION (AS APPLICABLE). IF RESIDENTS OF GERMANY ARE PARTICIPATING: FOR GERMAN RESIDENTS, DER RECHTSWEG IST AUSGESCHLOSSEN.

(11) ADDITIONAL DISCLAIMERS: The Released Parties are not responsible and/or liable for any of the following, whether caused by a Released Party, the Participant, or by human error: participation submitted by illegitimate means (such as, without limitation, by an automated computer program) or participation in excess of any stated limit; any lost, late, incomplete, illegible, unintelligible, garbled, mutilated, or misdirected participation, email, or Competition-related correspondence or materials; any error, omission, interruption, defect or delay in transmission or communication; viruses or technical or mechanical malfunctions; interrupted or unavailable cable or satellite systems; errors, typos, or misprints in these Rules, any Competition-related advertisements, or other materials; failures of electronic equipment, computer hardware, or software; lost or unavailable network connections or failed, incorrect, incomplete, inaccurate, garbled or delayed electronic communications or participation information. Released Parties are not responsible for electronic communications that are undeliverable or do not reach a Participant as a result of any form of active or passive filtering of any kind or insufficient space in a potential winner's email inbox to receive email messages. Released Parties are not responsible, and may disqualify you, if your email address or other contact information does not work or is changed without prior written notice to Sponsor. Without limiting any other provision in these Rules, the Released Parties are not responsible or liable to any Participant or winner (or any person claiming through such Participant or winner) for failure to supply the prize or any part thereof in the event that any of the Competition activities or Released Parties' operations or activities are affected by any cause or event beyond the sole and reasonable control of the applicable Released Party (as determined by Sponsor in its sole discretion), including, without limitation, by reason of any force majeure event, equipment failure, threatened or actual terrorist acts, air raid, act of public enemy, war (declared or undeclared), civil disturbance, insurrection, riot, epidemic, pandemic, fire, explosion, earthquake, flood, hurricane, unusually severe weather, blackout, embargo, labor dispute or strike (whether legal or illegal), labor or material shortage, transportation interruption of any kind, work slow-down, any law, rule, regulation, action, order, or request adopted, taken, or made by any governmental or quasi-governmental entity (whether or not such governmental act proves to be invalid), or any other cause, whether or not specifically mentioned above. This Competition is not intended to advertise Sponsor's products or services or induce the consumption of Sponsor's products or services.

(12) GENERAL RULES: By entering the Competition, each Participant agrees to maintain his/her behavior in accordance with all applicable laws and generally accepted social practices in connection with participation in any Competition- or prize- related activity. Each Participant hereby agrees that the Released Parties have the right, in their sole discretion, to disqualify and remove any Participant from any activity at any time if his/her behavior is disruptive or may, or does, cause damage to any person, property or the reputation of the Released Parties or otherwise interrupts the orchestration of the Competition.

Sponsor's decisions will be final in all matters relating to this Competition, including interpretation of these Rules, selection of the winners, and awarding of the prizes. All Participants, as a condition of participating, agree to be bound by these Rules and the decisions of Sponsor. Participants further agree to not damage or cause interruption of the Competition and/or prevent others from participating in or engaging with the Competition. Sponsor reserves the right to restrict or void participation from any IP address or other identifiable source if any suspicious participation is detected. Sponsor reserves the right, in its sole discretion, to void the participation of any Participant who Sponsor believes has attempted to tamper with or impair the administration, security, fairness, or proper play of this Competition. Sponsor's failure or decision not to enforce any provision in these Rules will not constitute a waiver of that or any other provision. In the event there is an alleged or actual ambiguity, discrepancy, or inconsistency between disclosures or other statements contained in any Competition-related materials and/or these Rules (including any alleged ambiguity, discrepancy, or inconsistency within these Rules), it will be resolved by Sponsor in its sole discretion. Participants waive any right to claim ambiguity in these Rules. If Sponsor determines at any time in its sole discretion that a winner or potential winner is disqualified, ineligible, in violation of these Rules, or engaging in behavior that Sponsor deems obnoxious, inappropriate, threatening, illegal or that is intended to annoy, abuse, or harass any other person, Sponsor reserves the right to disqualify that winner or potential winner, even if the disqualified winner or potential winner may have been notified or displayed or announced anywhere. The invalidity or unenforceability of any provision of these Rules will not affect the validity or enforceability of any other provision. If any provision is determined to be invalid or otherwise unenforceable or illegal, these Rules will otherwise remain in effect and will be construed in accordance with their terms as if the invalid or illegal provision were not contained herein. If the Competition is not capable of running as planned for any reason, Sponsor reserves the right, in its sole discretion, to cancel, modify, or suspend the Competition and award the prizes from eligible, non-suspect participation prior to cancellation, modification, or suspension or as otherwise deemed fair and appropriate by Sponsor. If any person supplies false information, participates by fraudulent means, or is otherwise determined to be in violation of these Rules in an attempt to obtain a prize, Sponsor may disqualify that person and seek damages from them, and that person may be prosecuted to the full extent of the law. In the event of a dispute concerning the identity of a Participant, the dispute must be resolved to Sponsor's satisfaction or the related participation will be disqualified. Any Participant may be required to provide Sponsor with proof of eligibility in the form requested. CAUTION: ANY ATTEMPT TO DAMAGE ANY ONLINE SERVICE OR WEBSITE OR UNDERMINE THE LEGITIMATE OPERATION OF THE COMPETITION MAY VIOLATE CRIMINAL AND CIVIL LAWS. IF SUCH AN ATTEMPT IS MADE, SPONSOR MAY DISQUALIFY ANY PARTICIPANT MAKING SUCH ATTEMPT AND MAY SEEK DAMAGES TO THE FULLEST EXTENT PERMITTED BY LAW.

(13) PRIVACY: By participating in the Competition, you agree to the collection, storage, processing, and transmission of your submitted personal data by Meta Platforms, Inc. and its affiliated companies and representatives for the purposes of conducting this Competition, evaluating registrations, contacting participants and potential participants, and/or to conducting screenings to determine eligibility and suitability for the Competition. The personal data collected is subject to applicable data protection laws and Sponsor's Data Policy (https://www.facebook.com/about/privacy).

(14) WINNERS' LIST/OFFICIAL RULES: To find out who won, send an email to CRAGKDDCup2024@meta.com and in the subject line writeRAG Benchmark CompetitionWinners. Only one (1) request per person or person will be fulfilled. Requests for winner information must be received no later than three (3) months following the end of the Competition. For a copy of these Rules, print out these pages or send an email during the Competition to CRAGKDDCup2024@meta.com and in the subject line write: RAG Benchmark Competition Rules Request.

Meta Platforms, Inc. Individual Contributor License Agreement ("Agreement")

You accept and agree to the following terms and conditions for Your Contributions submitted in connection with the KDD Cup 2024 to Meta Platforms, Inc. ("Meta") or its representative. Except for the license granted herein to Meta and recipients of software distributed by Meta, You reserve all right, title, and interest in and to Your Contributions.

1. Definitions.

"You" (or "Your") shall mean the copyright owner or legal entity authorized by the copyright owner that is making this Agreement with Meta. For legal entities, the entity making a Contribution and all other entities that control, are controlled by, or are under common control with that entity are considered to be a single Contributor. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.

"Contribution" shall mean any original work of authorship, including any modifications or additions to an existing work, that is intentionally submitted by You to Meta or Meta's representative in connection with the KDD Cup 2024 (the "Work"). For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to Meta or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, Meta for the purpose of discussing and improving the Work.

2. Grant of Copyright License. Subject to the terms and conditions of this Agreement, You hereby grant to Meta and to recipients of software distributed by Meta a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute Your Contributions and such derivative works.

3. Grant of Patent License. Subject to the terms and conditions of this Agreement, You hereby grant to Meta and to recipients of software distributed by Meta a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by You that are necessarily infringed by Your Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was submitted. If any entity institutes patent litigation against You or any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes direct or contributory patent infringement, then any patent licenses granted to that entity under this Agreement for that Contribution or Work shall terminate as of the date such litigation is filed.

4. You represent that you are legally entitled to grant the above licenses. If your employer(s) has rights to intellectual property that you create that includes your Contributions, you represent that you have received permission to make Contributions on behalf of that employer, that your employer has waived such rights for your Contributions to Meta, or that your employer has executed a separate Corporate CLA with Meta.

5. You represent that each of Your Contributions is Your original creation (see section 7 for submissions on behalf of others). You represent that Your Contribution submissions include complete details of any third-party license or other restriction (including, but not limited to, related patents and trademarks) of which you are personally aware and which are associated with any part of Your Contributions.

6. You are not expected to provide support for Your Contributions, except to the extent You desire to provide support. You may provide support for free, for a fee, or not at all. Unless required by applicable law or agreed to in writing, You provide Your Contributions on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON- INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE.

7. Should You wish to submit work that is not Your original creation, You may submit it to Meta separately from any Contribution, identifying the complete details of its source and of any license or other restriction (including, but not limited to, related patents, trademarks, and license agreements) of which you are personally aware, and conspicuously marking the work as "Submitted on behalf of a third-party: [named here]".

8. You agree to notify Meta of any facts or circumstances of which you become aware that would make these representations inaccurate in any respect.