Researchers present work via LLM-based user behaviour simulation game
Author ADM+S Centre
Date 7 September
ADM+S researchers Chenglong Ma, Danula Hettiachchi and Xinye Wanyan recently presented their research on LLM-based user behaviour simulation at the RMIT University Open Day.
Their showcase, “User Behaviour Simulation with LLMs”, explores how Large Language Models (LLMs) can be used to simulate user behaviour, in the context of recommender and ranking systems.
The showcase featured an interactive online game, created by Chenglong with input from Danula and Xinye. The game invited visitors to take on the role of a recommender system, giving them a hands-on way to explore how systems make decisions about what content users see.
The game is designed for visitors to experience one of the main challenges of user modelling: determining which pieces of information about a person should influence a recommendation.
Participants were given a user profile and with a scenario and context, they were then asked to recommend an item to that user. Recommendations included movies, books, music and products.The game provided feedback on how well each recommendation matched the user’s interests and current needs.
“By asking participants to make recommendations from user profiles and contexts themselves, the activity encouraged them to think about the same questions our research examines: which user attributes genuinely matter, how context changes decisions, and when personalisation may cross over into stereotyping,” Chenglong Ma said.
Chenglong said they wanted visitors to understand that, “user simulation is not simply about asking an LLM agent to randomly perform an action,” and that simulation needs to account for a user’s preference distribution, behavioural patterns and current context when modelling decisions.
Danula Hettiachchi coordinated and organised the activity while contributing ideas and feedback during the development of the interactive game. Xinye Wanyan played a major role in engaging with students, parents and other visitors, explaining the research through accessible examples and guiding participants through the game.
“The Open Day showcase translated our broader research problem into an accessible experience.”
This research links to the ADM+S Signature Project, “Evaluating Automated Cultural Curating and Ranking Systems with Synthetic Data”, which investigates how synthetic data can be used to study and evaluate automated ranking and recommendation systems, including questions of diversity, fairness and inclusion.
