University of Potsdam
Here are the detailed guidelines for this seminar. Please read them carefully.
On our first meeting on Nov 4, I will introduce and motivate the topic in a live session and we’ll get to know each other. Then we’ll have two parts of 6 meetings each.
A pre-recorded video lecture (around 20 minutes) will be uploaded in the Moodle page every weekend with an overview of the main concepts. You can watch it anytime you want before our official slot. There will also be additional reading material with more details for deeper understanding.
On Wednesdays at 12:15, we’ll meet online for 60 minutes. We’ll use the first 15 minutes for questions and comments about the week’s content, then you’ll work in small groups to discuss the current week’s worksheet. Come to class prepared, i.e., watch the video lecture and read the material before our meeting.
Anytime before the following online session, you have to upload your solutions in Moodle (there will be a link for that each week). You can work in groups, but everyone must submit their own answers. Late submissions are not allowed unless you have a medical reason.
The worksheets will be mostly composed argumentative exercises (no programming). The answers will not be graded but a submission will be considered valid if, and only if, your solutions are coherent and complete. Remember that the valid submission of at least 70% of the worksheets is a prerequisite to pass the course.
After the Christmas break, we’ll have student presentations about evaluation in specific NLP tasks. More details about the plan and the format will be added here once I have an idea of the number of students taking the course.
You can choose between one of the following options:
EvalNews: feeling creative? You can be the editor-in-chief of the next edition of the EvalNLP Gazette. Summarize the content we covered in the course in the format of a small newspaper or magazine (like those distributed by students in campus or neighborhood news). Think of the many strategies that small newspapers use: interviews (you can interview real people of create a fictitious character), reports, comics, crosswords, trivia, ads, poetry, tips… use your imagination.
Real evaluation: want some hands-on experience? You can choose an NLP algorithm (either one you implement or someone else’s) and write a comprehensive evaluation report about it.
Seminar paper: prefer tradition? Then write a critical essay on the development and current state of evaluation procedure in NLP, discussing the positive and negative aspects and where to go from here. It has to be argumentative and use scientific literature to support your views.
Guidelines about the format, length, etc of each option will be posted soon.
There will be a link in Moodle for you to submit a pdf file.
Submission deadline for the final project: March 31, 2021, 23:59.