Natural Language Processing

NUS SoC, 2021/2022, Semester II, Time and Venue: L1: Fridays, 09:00–12:00 @ i3 Aud (hybrid). L2: Mondays, 18:30–21:30 (online only). Sessions will be recorded where our technology behaves appropriately and shared on LumiNUS.

Last updated: 27 May 2022 (Point to new semester, not on this website.)

This module deals with computer processing of human languages, including the use of neural networks and deep learning in natural language processing. The topics covered include: regular expressions, words and edit distance, n-grams, part-of-speech tagging, feed-forward neural networks, neural network training, word embedding, convolutional neural networks, recurrent neural networks, sequence-to-sequence models with attention, transformers, context-free grammars, syntactic parsing, semantics, and discourse.

We will be using the LumiNUS 🔒 Learning Management System for the administration of this course.

N.B. We will be teaching and using the Python programming language throughout this class and Jupyter Notebook via Google Colab. We will using Python 3.x, and largely the SciKitLearn and PyTorch libraries.

Tutorial Sessions

There will be tutorials for this class starting in Week 03. As the class is flipped, these sessions will be the primary means by which we touch base with you and get to know you personally. Tutorials will be held roughly bi-weekly (once every two weeks). Please do attend these session (physically or via Zoom), as they will not be webcasted (although tutorial solutions will be distributed, you should come to the sessions to get the complete picture, and to be a part of the class).

These tutorial session timings still subject to change. Please see NUSMods for the most up-to-date details. As an enrolled student, you are entitled to one tutorial placement, and need to attend that slot even if not optimal for you. Note that some of the tutorials are physical ones, that take place in-person at the School, while others are e-learning only. Check with your tutorial leader for more details.

Tutorial Sessions:

1. T01. Tuesdays, 13:00-14:00 (Seminar Room 2; SR2, COM1 02-04)
2. T02. Tuesdays, 14:00-15:00 (Seminar Room 2; SR2, COM1 02-04)
3. T03. Wednesdays, 13:00-14:00 (Seminar Room 2; SR2, COM1 02-04)
4. T04. Wednesdays, 14:00-15:00 (Seminar Room 2; SR2, COM1 02-04)
5. T05. Wednesdays, 15:00-16:00 (Seminar Room 2; SR2, COM1 02-04)
6. T06. Tuesdays, 13:00-14:00 (E-Learn)
7. T07. Tuesdays, 14:00-15:00 (E-Learn)
8. T08. Wednesdays, 13:00-14:00 (E-Learn)
9. T09. Wednesdays, 14:00-15:00 (E-Learn)
10. T10. Wednesdays, 15:00-16:00 (E-Learn)
11. T11. Tuesdays, 19:00-20:00 (E-Learn)
12. T12. Tuesdays, 20:00-21:00 (E-Learn)
13. T13. Wednesdays, 19:00-20:00 (E-Learn)
14. T14. Wednesdays, 20:00-21:00 (E-Learn)

Course Characteristics

Modular Credits: 4.

Prerequisites (each bullet is required):

• (CS3243 Introduction to Artificial Intelligence or CS3245 Information Retrieval), and
• (MA1102R Calculus (Deprecated) or MA2002 Calculus or MA1505 Mathematics I or MA1507 Advanced Calculus (Deprecated) or MA1521 Calculus for Computing or:
  • (both MA1511 Engineeering Calculus and MA1512 Differential Equations for Engineering)), and
• (EE2012/A Analytical Methods in Electrical and Computer Engineering, or MA2216 Probability, or ST2131 Probability, or ST2334 Probability and Statistics).

Questions about prerequisities and waivers are handled centrally by the department. For this semester (Semester 2120), Please contact cs-curriculum@comp.nus.edu.sg to describe your case and seek any waivers regarding prerequisities.

Workload

(3-0.5-0-3-3.5): 10 hours per week

Translation:

• 3 lecture hours per week.
• 0.5 hours of tutorials per week (biweekly).
• 3 hours for projects, assignments, fieldwork, etc., per week.
• 3.5 hours for preparatory work by a student per week.

Instructors

Min-Yen KAN
kanmy@comp.nus.edu.sg
AS6 05-12
Office hours are held after class, but more commonly by appointment. Emails and @Direct Mentions in Slack to me as a default are assumed to be public, and my replies and your anonymized email will likely be posted to Slack. Please let me know if you do not want the contents of your communication posted; I will be happy to honor your requests.

Christian VON DER WETH
chris@comp.nus.edu.sg

Teaching Assistants

MIAO Yisong
miaoyisong@gmail.com

Muhammad Reza QORIB
mrqorib@u.nus.edu

WU Jiaying
jiayingwu@u.nus.edu

XIE Yuxi
xieyuxi@u.nus.edu

XUE Fuzhao
f.xue@u.nus.edu

Frequently Asked Questions

1. What is the format of this course? This course’s administration is currently in flux due to higher demand for the subject. We have opened two sections of the course with different learning modes. L1 taught by Kan Min-Yen on Friday AMs will feature hybrid lecture (lecture both via synchronous Zoom online and physical lecture at LT15 i3 Auditorium); L2 taught by Christian von der Weth on Monday evenings will be online lecture only.

2. Will the L1 and L2 sections of the course work the same? Generally, yes. We will be offering synchronized lectures such that the L1 section on Friday starts the topic and the subsequent L2 section on Monday covers the same topic. This means that L2 will have no first lecture (no class on 10 Jan) and L1 will have no lecture on the last Friday (no class on 15 Apr; Good Friday holiday anyways).

3. Does the L1 section thus compel students to come in for physical lecture? No it does not. We believe there are certain students that learn better in a physical, less distracting environment. If you are one of them and enrolled for L1, please join us in the physical environment. Historically, we find that engaged learners tend to fare better with the course materials and with their marks when they are able to interact with our instruction staff, and this is easier to facilitate in a physical setting.

4. How about tutorials? Tutorials will be offered every other week after Week 03. The teaching mode for tutorials has yet to be decided. We have added evening tutorials (now reflected in LumiNUS timetable) to help cope with the load. Some tutorials are e-learning format only, others may be hybrid or physical tutorials in SR2 (COM1 02-04) only.

5. I’m doing an ATAP, SIP, FYP concerning NLP. Can I get an exception to enter the course? Generally no. Our course (even with the 2x increase in instructors) is still heavily oversubscribed. Therefore, we suggest that you come back to campus to take the course. If you absolutely need to get access to the course, make an official appeal and if it can be considered, we’ll hear about it from ModReg directly. You can check with us but we may not have the capacity to answer you directly.

6. Can you add me as a guest student and if so, what are my responsibilities? We do accept guest student enrolment in this course. You may write to either lecturer to gain access to the course notes and lecture webcasts. We do not accept auditing students (see NUS Registrar for the formal difference; but basically it appears on your transcript with an ‘Audit’ grade), as per NUS SoC policy. However, we have a quota cap for a reason, and that is to best serve the fully enrolled students. Thus, if our teaching staff lack bandwidth, we may not be able to answer your questions and concerns. Please do understand.

7. I’m having problems enrolling in a tutorial. How to get help? Each module has a Tutorial Registration Coordinator (TRC) assigned to them. Our TRC for CS4248 is JIANG Liu. Please contact Jiang for help in registering for tutorial if you cannot secure a slot. Our instructors will not be able to help you, as tutorial registration is centrally controlled.

8. Will the module be webcasted (recorded or broadcasted live)? Yes, barring technical difficulties, we plan to record the course lectures and make them available on LumiNUS. That is, we will make our best effort to do this, but do not guarantee quality of service for webcasted or remote learning. We will attempt to make the lectures available on simulcast on Zoom, links to be available to course students only (you will need to be authenticated to your NUS-linked Zoom account to connect to the lectures).

   Recorded lectures will be put into the Web Lectures folder on the LumiNUS platform.

9. I’d like to take this module but couldn’t secure it during ModReg because the quota is full. Will it be possible to make an exception? See below answers to similar questions. Generally the answer is no, but we welcome students to audit the class, or take the class in subsequent semesters (CS4248 is offered in both Sem 1 and 2 now). We know the demand is high (we heard 160 students for 100 slots), but we need to keep the student cap at a reasonable number this semester.

   If you’d like to audit, please send a mail to cs4248@comp.nus.edu.sg or a DM to @Min with your name and NUS LumiNUS-registered email address.

10. The course is stated as having a physical class for the L1 section. Is that correct?

    Yes. Lecture will be on Fridays physically at LT15 i3 Auditorium. i3 Auditorium has a physical safe distancing limit of 50 students, so the first 50 students who take the respective poll are welcomed at the auditorium. We will implement the NUS Classroom Attendance System to ensure that those who wish to attend can. If coming to the class, please make it to the venue by 8:55 so that class can start on-time.

    A fixed Zoom session e-learning option will be given out students to attend class if not attending physically (see first QA item).

1. I do not fulfill the exact prerequisite requirements for this module. Can you grant me this exception?

   Module coordinators do not approve prerequisite waivers in our School. You need to approach our School’s curriculum coordinator for permission. Generally, the answer is no, except in extenuating circumstances. E-mail cs-curriculum@comp.nus.edu.sg and apply for the waiver on ModRec.

2. I need this course in order to graduate this coming semester. Can you grant me this exception?

   Please ask your academic advisor to explain the situation to us. We will see what we can do and perhaps help your academic advisor help plan future students’ plans accordingly.