Abstract
The Third WEPIR 2020 workshop builds on the success of the first two WEPIR meetings held at CHIIR 2018 and CHIIR 2019. WEPIR 2020 again brings together researchers from different backgrounds interested in continuing to explore and advance the evaluation of personalisation in information retrieval. Similar to the first two workshops, WEPIR 2020 has a strong emphasis on active participation by workshop attendees. This was very successfully achieved in the first two workshops by the use of workshop breakout groups exploring topics related to personalisation and information retrieval, and the evaluation of personalisation in information retrieval in general, with subsequent report back to the workshop as a whole. However, a key difference for WEPIR 2020 is that while the first two workshops focused on developing and articulating principles and ideas relating general topics relating to these topics, identified as interesting and important by the attendees at the workshops, WEPIR 2020 focuses breakout discussion on a number of relevant specific use cases of the evaluation of personalisation in information retrieval. A use case is assigned to each breakout group with the plan being to have more than one group work on each use case. The task for each group is to identify specific relevant factors relating to the use case in terms of user activities, data to be collected, ethical issues, and evaluation metrics. Groups will make reports of their discussions to the assembled workshop in the final session with discussion of the alternative solutions relating to the same use case, and contrasting the issues raised by the different use cases. The overall goal of the workshop is to work towards developing a general set of principles and guidelines for addressing the evaluation of specific instances of the use of personalisation in information retrieval tasks. This is consistent with the activities and goals of the first two workshops, but represents a significant progression of the activities towards concrete outcomes of benefit to those exploring personalisation in search, the issues arising in its evaluation, and how researchers might go about tackling this in specific situations.