Here are the detailed guidelines for this seminar. Please read them carefully. If you face any personal problems during the semester that are affecting your participation, contact me asap before it’s too late to find a solution.
On our first meeting on Oct 19, I will introduce and motivate the topic and we’ll get to know each other. Then the course will be split into two parts.
A reading list will be uploaded in the Moodle page every Wednesday after class. You must go through it and fill in the short documentation form on Moodle anytime you want until the following Wednesday at 10.
On Wednesdays at 12:15, we’ll meet at room 2.14.0.32. We’ll use the first minutes for questions and feedback about the last exercises. Then I’ll usually give a short lecture to extend the reading. ON the second half of the class, use you’ll work in small groups to discuss the current week’s worksheet. Come to class prepared, i.e. read the material and fill in the documentation before our meeting.
By the following Monday at 12, you have to upload your worksheet answers in Moodle (there will be a link for that each week). You can work in groups, but each member must submit their own answers. Late submissions are not allowed unless you have a good reason.
The weekly worksheets will be mostly composed of argumentative exercises (no programming). The answers will not be graded but a submission will be considered valid if, and only if, your solutions are coherent and complete.
After the Christmas break, we’ll have student presentations (as prerecorded podcasts) about evaluation in specific NLP tasks as well as ethical and responsible evaluation followed by guided group discussions. More details about the plan and the format will be published once I have an idea of the number of students taking the course.
On two classes, one in the first part and one in the second part, you will work in groups to solve programming assignments, focussing on Python libraries that contain useful implementations of evaluation metrics and algorithms.
This is an individual effort. You can choose between one of the following options:
Hands-on evaluation: You can choose an NLP algorithm (either one you implement or someone else’s) and write a comprehensive evaluation report about it.
Seminar paper: Write a critical essay on the development and current state of evaluation procedure in NLP, discussing the positive and negative aspects and where to go from here. It has to be argumentative and use scientific literature to support your views.
Guidelines about the format, length, etc of each option will be posted on Moodle after the Christmas break.
There will be a link in Moodle for you to submit a pdf file.
Submission deadline for the final project: March 31, 2022, 23:59.