About me ...

I am a third-year PhD candidate at the University of Amsterdam, supervised by Maarten de Rijke and Mohammad Alian Nejadi. My research focus is on the evaluation of conversational systems, specifically from the user's perspective. I aim to improve user experience in their interaction with AI systems. I am also interested in how to leverage LLMs for evaluation and studying how factors affecting humans in system evaluation, affect LLMs when used as evaluators. Apart from that, I am an active member of Masakhane where I take part in promoting NLP for Low-resource African languages.

I completed my MSc degree from the African Institute for Mathematical Sciences - AMMI (Rwanda) supervised by Siva Reddy from McGill University. I received my BSc from Jomo Kenyatta University, Kenya.

Research Interests

User-centric evaluation of conversational systems

Evaluation of and with large language models

NLP for low-resource African languages

Publications

Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs

SIGIR'24 conference

Clemencia Siro, Mohammad Aliannejadi, Maarten de Rijke

PDF

Context Does Matter: Implications for Crowdsourced Evaluation Labels in Task-Oriented Dialogue Systems

NAACL'24 conference

Clemencia Siro, Mohammad Aliannejadi, Maarten de Rijke

PDF

Asking Multimodal Clarifying Questions in Mixed-Initiative Conversational Search

WWW'24 conference

Yifei Yuan, Clemencia Siro, Mohammad Aliannejadi, Maarten de Rijke

PDF

Cross-lingual Open-Retrieval Question Answering for African Languages

EMNLP'23 conference

Ogundepo Odunayo, ..., Clemencia Siro, et al.

PDF

Understanding and Predicting User Satisfaction with Conversational Recommender Systems

ACM TOIS 2023 journal

Clemencia Siro, Mohammad Aliannejadi, Maarten de Rijke

PDF

AFRICANEWS: News Topic Classification for African News

AACL'23 conference (Area Chair Award)

Adelani, David Ifeoluwa, ..., Clemencia Siro, et al.

PDF

Evaluating the Robustness of Machine Reading Comprehension Models to Low Resource Entity Renaming

AfricaNLP Workshop at ICLR'23 workshop

Clemencia Siro, Tunde Ajayi

PDF

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

TMLR 2023 journal

Srivastava, Aarohi,..., Clemencia Siro, et al.

PDF

Understanding User Satisfaction with Task-oriented Dialogue Systems

SIGIR'22 conference

Clemencia Siro, Mohammad Aliannejadi, Maarten de Rijke

PDF

MasakhaNER: Named entity recognition for African languages

TACL 2021 journal

Adelani, David Ifeoluwa, ..., Clemencia Siro, et al.

PDF

Activities

2024 - LLM4IREval workshop organizer at SIGIR'24

2023 - AfricaNLP workshop organizer at ICLR'23 and ICLR'24

2022: Guest speaker - Master AI University of Amsterdam: Evaluation of Dialogue System with Users

2022 - Present: Mentor for Master AI students at the University of Amsterdam

2019 - Present: Member of MasakhaneNLP, an NLP group for African languages