Researchers investigate LLMs for search systems during Amsterdam research visit
Author
Date 8 October 2025
In early July, ADM+S researchers Nuha Abu Onq and Chenglong Ma visited the Information Retrieval Lab (IRLab) at the University of Amsterdam, Netherlands, organised by ADM+S Partner Investigator Prof Maarten de Rijke. Nuha and Chenglong attended a series of research conferences and collaborative meetings, creating a valuable opportunity for cross-institutional exchange.
On 11 July, both Nuha and Chenglong gave invited talks at IRLab:
- Chenglong Ma presented “PUB: An LLM-Enhanced Personality-Driven User Behaviour Simulator for Recommender System Evaluation,” introducing a simulator that infers personality traits from user behaviour logs and uses those to produce synthetic interaction data that better mirrors real user diversity.
- Nuha Abu Onq presented “Classifying Term Variants in Query Formulation,” analysing how users formulate diverse search queries, especially how cognitive complexity of underlying information needs affects query variation and the strategies people employ.
During the visit, Nuha and Chenglong had productive discussions with other researchers about topics like Large Language Models (LLM) for Evaluation in IR. They both attended the SIGIR (Special Interest Group on Information Retrieval) 2025 conference, including participating in the LLM4Eval workshop.
“At SIGIR’25, we considered several approaches for designing prompts to apply LLMs to categorisation tasks, aiming both to simplify future research and to support the training of models for automated categorisation,” Nuha said.
“Additionally, we discussed extending our work on personality traits to investigate how these traits might influence variations in user search behaviour.”
Nuha and Chenglong mention that one of the key takeaways was exploring the value of open, reproducible and user-centred research practices. The IRLab team’s emphasis on making code and data publicly available and combining technical methods with user studies provided important insight.
Chenglong and Nuha have plans to apply these approaches in their own work. Smaller, well-designed user studies were shown to be highly valuable for informing the development of trustworthy AI systems.
“Carefully designed small-scale user studies can provide valuable insights for future LLM-based search systems, as they can be validated against real user search interactions.” Nuha said.
Nuha and Chenglong recognised the need to bridge academic research with real-world applications, especially when it comes to fairness and evaluation in commercial search and recommendation systems.
This visit was funded by the ARC Centre of Excellence for Automated Decision-Making and Societies’ Research Training Grant,
