SS23: What do pre-trained language models really understand?

Course description

Large language models such as GPT-3 and ChatGPT have led to great advances in the field of natural language processing, and in many cases they provide responses to prompts that suggest that possess non-trivial abilities to understand language. At the same time, however, these models are primarily trained on text and have no explicit connection with the real world, whereas humans learn language by interacting with other humans and forming associations between words and phrases and entities and events in the world.

In this seminar, we will focus on the question to what extent large language models understand language. We’ll cover different philosophical schools of what it means to understand language, and then focus on a series of recent empirical papers that aim to evaluate different aspects of language understanding in models.