Multi-Turn Multi-Modal Question Clarification for Enhanced Conversational Understanding
PDFAbout Me
I am a postdoctoral researcher at Centrum Wiskunde & Informatica (CWI), working on bias and fairness in AI systems. I obtained my Ph.D. degree from the University of Amsterdam (UvA), supervised by Maarten de Rijke and Mohammad Alian Nejadi. I completed my MSc degree at the African Institute for Mathematical Sciences – AMMI (Rwanda), supervised by Siva Reddy from McGill University, and my BSc at Jomo Kenyatta University.
My research interests focus on user-centric evaluation of conversational systems and leveraging large language models for evaluation. I am also a member of Masakhane, contributing to NLP research for low-resource African languages.
I successfully defended my Ph.D. thesis titled Rethinking the Human-Centered Evaluation of Conversational Systems on 8th July 2025.
Research Interests
- User-centric evaluation of conversational systems
- Evaluation of and with large language models
- NLP for low-resource African languages
Publications
Preprints & Under Submission
Judging the Judges: A Collection of LLM-Generated Relevance Judgements
PDFLlmjudge: Llms for relevance judgments
PDF5th Workshop on African Natural Language Processing (AfricaNLP 2024)
PDFPublished Works
LLM4Eval: Large Language Model for Evaluation in IR
PDFDo Images Clarify? A Study on the Effect of Images on Clarifying Questions in Conversational Search
PDFLLM4Eval@ WSDM 2025: Large Language Model for Evaluation in Information Retrieval
PDFRethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs
PDFLlm4eval: Large language model for evaluation in IR
PDFAsking Multimodal Clarifying Questions in Mixed-Initiative Conversational Search
PDFAfrimte and africomet: Empowering comet to embrace under-resourced African languages
PDFCross-lingual Open-Retrieval Question Answering for African Languages
PDFUnderstanding and Predicting User Satisfaction with Conversational Recommender Systems
PDFAFRICANEWS: News Topic Classification for African News
PDFEvaluating the Robustness of Machine Reading Comprehension Models to Low Resource Entity Renaming
PDFBeyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
PDFUnderstanding User Satisfaction with Task-oriented Dialogue Systems
PDFMasakhaNER: Named entity recognition for African languages
PDFNews and Activities
- July 2025 - I will be co-organizing the AfricaNLP workshop at ACL,2025.
- July 2025 – Successfully defended my PhD thesis.
- July 2025 – Organized the LLM4Eval workshop at SIGIR 2025.
- May 2025 – Began my postdoctoral research position at Centrum Wiskunde & Informatica (CWI).
- December 2024 – 1 paper accepted for presentation at CHIIR 2025.
- November 2024 – Visiting researcher at the Machine Learning group, University of Copenhagen.
- September 2024 – Delivered an invited talk at Glasgow IRTalks on “Rethinking Human-Centered Evaluation of Conversational Systems.”
- July 2024 – Organized the LLM4Eval workshop at SIGIR 2024.
- May 2024 – Co-organized AfricaNLP workshop at ICLR 2024.
- May 2023 – Co-organized AfricaNLP workshop at ICLR 2023.
- September 2022 – Invited guest speaker for the Master AI program at the University of Amsterdam, presenting on evaluation of dialogue systems with users.
- 2022 to present – Mentor for Master AI students at the University of Amsterdam.
- 2019 to present – Active member of MasakhaneNLP, a collaborative research community focused on NLP for African languages.
Contact
Email: c.n.siro[at]cwi.nl