Second Second NLG Challenge on Generating Instructions in Virtual Environments (GIVE-2.5)
Endorsed by SIGGEN and SIGSEM
The Second Second GIVE Challenge (GIVE-2.5) took place in 2011-2012. For this edition of GIVE, we partly reworked the software infrastructure while keeping the task essentially identical to the one of GIVE-2. This gave systems the opportunity to be improved based on experiences from the GIVE-2 evaluation and allowed more teams to participate in the same task.
Eight teams participated in GIVE-2.5, i.e. one more than in GIVE-2 and three more than in GIVE-1. The evaluation results, as presented at the ENLG 2011, are based on 536 valid interactions with users which were collected until August 2011. The systems represented a broader variety of approaches to NLG than in any previous edition of GIVE and generated instructions of high quality. We concluded the data collection in March 2012.