GIVE Generating Instructions in Virtual Environments

GIVE-2.5: Results

The following eight NLG systems participated in GIVE-2.5:

The systems were evaluated with respect to several different metrics: raw objective metrics (e.g. task success and duration), normalized objective metrics (e.g. error rate and speed), as well as subjective metrics, which were collected by asking users to fill in a post-task questionnaire. The analysis is based on a snapshot of the results containing 536 valid game runs.

The results were presented at the ENLG 2011 and are available in the official GIVE-2.5 report.