COMS 4995 Advanced Systems Programming

Spring 2024 Course Overview

W 4:10pm-6:40pm, 451 Computer Science Building
- CourseWorks open to whole Columbia first few weeks - search on Vergil
Two synchronous and in-person exams:
- Midterm: Wednesday March 6, 4:20pm
- Final: Wednesday May 8, 4:20pm
There are no make-up or alternate exams. If you cannot make any of those exams, please take the course next semester.
Instructors: Hans Montero and Jae Woo Lee
Office hours: Jae / Hans & TAs
REQUIRED READING: ACADEMIC HONESTY POLICIES
Anonymous feedback form
4157 Dev Guides

HW Due Dates

Date	Assignment
F 1/19	HW0 due
R 2/8	HW1 due
F 3/1	HW2 & HW3 due
W 4/3	HW4 due
W 4/17	HW5 due
~~M 4/29~~ F 5/3	HW6 due (hard deadline)

Course Schedule

1/17

Course overview and logistics

Lecture:
- L00-overview
Assignments:
1. Subscribe to 4995 ListServ – see L00 above for detail
2. Read the following two documents:
  - http://www.cs.columbia.edu/education/honesty
  - http://www.cs.columbia.edu/~jae/honesty.html
3. HW0 (50 points, individual assignment) – due Fri 9/6, 11:59pm
  
  Send an email to Jae and Hans containing:
  - To: jae@cs.columbia.edu, hans@cs.columbia.edu
  - Subject: “[4995] hw0-UNI” – without quotes, sole space before hw0, UNI replaced with your actual UNI in lowercase
  - Your name, major & school program, year – ex) Jae Woo Lee, Physics, Columbia College, class of 1994
  - Your pledge – see http://www.cs.columbia.edu/~jae/honesty.html
  - CS classes taken and/or other programming background
  - Optionally anything else you want to let us know
  - Optionally attach a picture of you, but please reduce image file size to about 100KB
4. Start forming your group of three people

Git Overview and Internals

Reading (optional)
- Pro Git, entirely free online
  - Ch. 2: Git Basics
  - Ch. 3: Git Branching
  - Ch. 10: Git Internals
Lecture:
- L01-git-version-control

1/24

GitHub

Reading (optional)
- Pro Git, entirely free online
  - Ch. 6: GitHub
Lecture (continued from last time)
- L01-git-version-control

Introduction to Memory Allocators

Reading
- CSAPP 9.1, 9.9 (skip 9.9.12)
Lecture:
- L02-memory-1

HW1: malloctopus

100 points, due Thu 2/8
- malloctopus

1/31

Caching, Memory Mapping, and Advanced Allocators

Reading
- Caching: CSAPP 6.1 (optional), 6.2, 6.3, 6.4 (optional), 6.5, 6.6 (skim 6.6.1, read 6.6.2-3)
- Memory mapping: APUE 14.8: read page 525-527 (skim the rest), 15.9: page 576-578
- Allocators: CSAPP 9.9.13-14, APUE 7.8 “Alternative Memory Allocators”
Lecture
- L02-memory-1 (continued from last time)
- L03-memory-2
Source code
- Makefile
- sum1.c, sum2.c
- mult-ijk.c, mult-kji.c, mult-ikj.c

Processes

Reading
- APUE 8.1-8.10 (8.4, 8.7, 8.8 optional)
Lecture:
- L04-process-control
Source code
- Makefile
- shell1.c

2/7

Processes (continued)

Lecture:
- L04-process-control (amended from last week with remote SSH login material)

Signals

Reading:
- APUE 9.4 - 9.6: skim, just to get a sense of process group, session, and controlling terminal
- APUE 9.7: skim pages 299-301, skip rest – just to get a sense of jobs running in foreground and background
- APUE 10.1 - 10.6 (skim signal descriptions in 10.2)
- APUE 10.8 - 10.9
- APUE 10.10: you can skip pages 340-341 & 343 (but interesting read if you know setjmp() / longjmp() or willing to find out)
- APUE 10.11 - 10.13
- APUE 10.14: skip page pages 352-353
Lecture:
- L05-signals
Source code:

File I/O

Reading:
- APUE 3.1 - 3.13
- APUE chapter 5 (skim)
Lecture:
- L06-file-io
Source code:

HW2: greptile

100 points, due Fri 3/1
- greptile

HW3: cowchat

100 points, due Fri 3/1
- cowchat

2/14

POSIX threads

Reading:
- APUE 11.1 - 11.4
- APUE 11.5: read page 388-393; skip the rest of 11.5
- APUE 11.6: read up to (and including) 11.6.6
Lecture:
- L07-thread
Source code:
- Makefile
- bank0.c
- bank1.c

Synchronization

Reading:
- CSAPP 12.5.2-4
- APUE 15.10
Lecture:
- L08-synchro
Source code:

2/21

Interprocess Communication in UNIX

Reading:
- Unnamed/named pipe: APUE 15.2, 15.5
- Memory mapping (previously assigned): APUE 14.8: read page 525-527 (skim the rest), 15.9: page 576-578
- Semaphore (previously assigned): CSAPP 12.5.2-4, APUE 15.10
Lecture:
- L09-ipc
Source code:

Nonblocking I/O and I/O multiplexing

Reading:
- APUE 14.2
- APUE 14.4.0 (i.e., page 500-501)
- APUE 14.4.1
Lecture:
- L10-adv-io
Source code:
- Makefile
- setfl.c
- clrfl.c
- nonblockw.c
- logger.c

2/28

UNIX domain sockets

Reading:
- APUE 17.1
- APUE 17.2: pages 631-633 are optional, but recommended
- APUE 17.2.1
- APUE 17.3: pages 637-641 are optional, but recommended
- APUE 17.4: read 642-643, skip the rest
Lecture:
- L11-dom-sock
Source code:

Threads 2

Reading:
- Thread safety & reentrancy: APUE 12.5, CSAPP 12.7.1-2
- Threads & signals: APUE 12.8
- Thread-specific data: skim APUE 12.6 just to get the idea
Lecture:
- L12-thread-2
Source code:
- Makefile
- sig1.c
- sig2.c
- tls.c
- pthread.c
- fork.c
- mm-test.c

3/6

Midterm

Coverage
- HW1, HW2, and HW3 (parts 1-4)
- All lecture materials, L01 through L12 (up to and including thread-local storage)
- All ANN emails
- All assigned textbook readings

3/13

Spring Break – no class

3/20

Threads 2 (continued)

Lecture
- L12-thread-2
  - clone(), false sharing, malloc() thread considerations

x86-64 Assembly

Reading
- CSAPP 3.0-3.1: Overview & History (optional)
- CSAPP 3.2: Program Encodings (skim)
- CSAPP 3.3: Data Formats
- CSAPP 3.4: Accessing Information
- CSAPP 3.5: Arithmetic & Logial Operation
  - 3.5.5 optional
- CSAPP 3.6: Control
  - 3.6.6 and 3.6.8 optional
- CSAPP 3.7: Procedures
- CSAPP 3.8: Arrays
  - 3.8.3 is previously assigned
  - Skip 3.8.4-5
- CSAPP 3.9: Heterogenous Data Structures
  - Everything except 3.9.1 is previously assigned
- CSAPP 3.10: Combining Control and Data
  - 3.10.1, 3.10.3-4 optional
  - Skim 3.10.5
Lecture
- L13-x86-assembly

HW4: ladebug

100 points, due Wed 4/3
- ladebug

3/27

Interrupts

Reading
- CSAPP 8.1, 8.2.4, 8.2.5
Lecture
- L14-interrupts

Introduction to ELF

Reading
- CSAPP 7.1 - 7.5, 7.8 - 7.9
Lecture
- L15-elf-intro

HW5: seald

100 points, due Wed 4/17
- seald

4/3

Linking

Reading
- CSAPP 7.6 - 7.7
Lecture
- L16-linking
  - Handwritten diagram & notes
Source code:
- echo1
- echo2
- hello
- sum

4/10

Linking 2

Reading
- CSAPP 7.10 - 7.14
Lecture
- L17-linking-2
Source code

UNIX Filesystem

Reading
- APUE 4.14, 4.15
Lecture
- L18-unix-fs

Intro to Linux Containers

Lecture
- L19-container-intro

HW6: zookeeper

100 points, due Fri 5/3 (Hard Deadline)
- zookeeper

4/17

Intro to Linux Containers (continued)

Lecture
- L19-container-intro

HW6 Overview

Course Wrap-up

Lecture
- L20-wrap-up

4/24

The XZ Utils Backdoor

Guest lecture presented by Denzel Farmer, current ASP TA

On March 29th, a developer from Microsoft published that he had discovered a backdoor built into XZ Utils, a compression package included with nearly every major Linux distribution. If gone unnoticed, this backdoor could have provided its authors with root-level access to millions servers across the internet. Interestingly, the core mechanism the backdoor uses to compromise host machines is something we just finished studying — dynamic linking and loading of ELF objects. This lecture will explore implementation details of the XZ Utils backdoor and describe the novel multi-year effort to put it in place–along with its consequences for the larger world of open source software development.

L21-xz-utils

Last updated: 2024-04-24