It is relevant, but is it useful?
- Chakrabarti, Hrishita Delft University of Technology, Delft, Netherlands
- Tobia, Diletta Micol ORCID Facoltà di scienze informatiche, Università della Svizzera italiana, Svizzera
- Allen, Garrett Delft University of Technology, Delft, Netherlands
- Landoni, Monica ORCID Facoltà di scienze informatiche, Università della Svizzera italiana, Svizzera
- Pera, Maria Soledad ORCID Delft University of Technology, Delft, Netherlands
- 2026
Published in:
- ACM Conference on User Modeling, Adaptation and Personalization (UMAP '26). - 2026, p. 392-397
English
The traditional Information Retrieval (IR) evaluation framework—anchored in topical relevance and relevance‑based metrics—reflects a system-centred perspective. Yet for specific user groups, relevance alone is insufficient; benchmarking that relies exclusively on conventional metrics overlooks qualities intrinsic to the users IR approaches are meant to serve. Here, we draw attention to Children IR and examine the value of extending traditional evaluation with a human-centred perspective that accounts for how children interpret and evaluate information to more authentically capture performance and better reflect how well an approach truly meets children’s needs. Our empirical exploration using a child‑focused dataset, multiple ranking strategies, and traditional and extended frameworks reveals not only the limitations of relevance-based assessments but also the advantages of employing frameworks that are tailored to reflect the needs of child users, paving the way for more inclusive and effective evaluation frameworks.
- Collections
- Language
-
- English
- Classification
- Computer science and technology
- Notes
-
- UMAP '26: 34th ACM Conference on User Modeling, Adaptation and Personalization
- Gothenburg , Sweden
- June 2026
- License
-
CC BY
- Open access status
- gold
- Identifiers
-
- DOI 10.1145/3774935.3806167
- RICERCO 67882
- ARK ark:/12658/srd1335782
- Persistent URL
- https://n2t.net/ark:/12658/srd1335782
Statistics
Document views: 0
File downloads:
- Landoni et al_2026_ACM_3774935.3806167: 0