In collaboration with ALT-EDIC and the European Language Data Space, the European Commission is organising an expert workshop on the linguistic and cultural evaluation of large language models (LLMs), with the purpose of charting current activities in Europe and working towards a common approach. In this workshop, the participating research or industrial groups present their activities, challenges and datasets, focussing on how they would imagine a joint approach.
If you work on relevant topics and want to share your insights, you are welcome to join!
CNECT-LANGTECH
ec [dot] europa [dot] eu (E-mail us to register)
- large language model
- Wednesday 11 December 2024, 09:00 - 13:00 (CET)
- Online only
Practical information
- When
- Wednesday 11 December 2024, 09:00 - 13:00 (CET)
- Where
- Online only
- Languages
- English
- Organisers
- European Commission | ALT-EDIC | European Language Data Space
- Part of
Description
Given ALT-EDIC’s upcoming focus on LLM evaluation and benchmark data, the workshop aims to kickstart ALT-EDIC’s work in this area by gathering stakeholders currently involved across the EU Member States.
After a brief introduction by the European Commission, the research and industrial groups participating in the workshop will have the opportunity to briefly present their topic of interest, sharing experiences and insights on
- identifying, creating and collecting language- and culture-specific LLM benchmarks within each Member State
- categorising LLM evaluation data
- developing a transparent LLM evaluation process
- establishing common guidelines for LLM evaluation
- exploring cross-language and cross-cultural evaluations and benchmarks across Europe
The presentations will be followed by a general discussion, in which suggestions for a joint European approach will be proposed.
