ADM+S student awarded Best Oral Presentation at Information Access Evaluation Conference
Author Natalie Campbell
Date 4 January 2024
ADM+S student Sachin Pathiyan Cherumanal from RMIT has been awarded Best Oral Presentation at the 17th Conference on Evaluation of Information Access Technologies (NTCIR-17), which took place in Japan on 12-15 December 2023.
Sachin represented a group of RMIT information retrieval researchers, including ADM+S members Kaixin Ji, Dr Danula Hettiachchi, Prof. Falk Scholer, and Dr Damiano Spina.
The ‘RMIT_IR’ team participated in the FairWeb-1 task, which required groups to investigate the relation between fairness and diversity in rankings using a systematic evaluation.
The FairWeb-1 task focused on three distinct entity types: researchers, movies, and YouTube contents. Each entity type is associated with one or two attribute sets, containing either nominal or ordinal groupings designed to ensure group fairness, and a target distribution is provided for each attribute set.
Groups were asked to submit results that not only included relevant documents at the top rank but also exhibited group fairness in alignment with the attributes specified for each entity type.
The RMIT_IR report details the team’s approach, exploring the role of explicit search result diversification (SRD) and ranking fusion to generate fair rankings considering multiple fairness attributes.
Sachin explains, “the report also considers the use of a linear combination-based technique (LC) which would take into consideration the relevance while re-ranking. Researchers compared results from five submitted runs, and the retrieval baselines along each topic type separately (i.e., Researcher, Movie, YouTube).”
Situated within the ADM+S project, Quantifying and Measuring Bias and Engagement, this work contributes to Sachin’s PhD thesis on fairness-aware question answering.
Each groups’ spokesperson presented a 10-minute overview of their system, for which Sachin was awarded Best Oral Presentation.
Since 1997, the NTCIR project has promoted research efforts for enhancing Information Access (IA) technologies such as Information Retrieval (IR), Text Summarization, Information Extraction (IE), and Question Answering (QA) techniques.
Dr Damiano Spina said, ‘by participating in NTCIR-17, ADM+S members contribute to the ongoing discussion around creating research infrastructure that allows large-scale evaluation of information access technologies.”
