Malware Reverse Engineering

Instructor: Wu-chang Feng
Class: (Zoom link)
Contact and discussion:
  • DM @wuchang
  • Class channel #cs492_592_malware on
  • In-class questions/feedback anonymously
Office hours: here (10am on Mondays) on the course Zoom link.
TA: Allison Naaktgeboren (naak at pdx)
TA Office hours: Mondays 12:30pm-2pm on Zoom | In-person by arrangement
Required textbook


Week Topic Slides Labs Homework
1 Introduction
Basic Analysis
Chapter 1: Basic Static Techniques
Chapter 2: Malware Analysis in VMs
Chapter 3: Basic Dynamic Analysis


Install VM (video)
1-1, 1-2
3-2, 3-4
Cyberwar talk (edited)
Ch01, Ch03
Chapter 3: Basic Dynamic Analysis
Advanced Static Analysis
Chapter 4: x86 Assembly
Chapter 5: IDA Pro
Chapter 6: C code in Assembly
5-1, 6-1, 6-2 Ch04, Ch06
Chapter 7: Malicious Windows Programs
Chapter 8: Debugging
Chapter 9: OllyDbg
. Advanced Dynamic Analysis
Malware Functionality
Chapter 11: Malware Behavior
Chapter 12: Covert Launching
Chapter 13: Data Encoding
Chapter 14: Network Signatures
Chapter 15: Anti-Disassembly
  Software Armoring
Chapter 16: Anti-Debugging
Chapter 17: Anti-VM Techniques
Chapter 18: Packers and Unpacking
Special Topics
Chapter 19: Shellcode Analysis
Chapter 20: C++ Analysis
Chapter 21: 64-bit Malware
Final project
Symbolic execution/analysis


Talks #1,#2 | Slides #1,#2

MetaCTF level src

12-1, 12-3
13-1, 14-1
15-1, 15-2
18-1, 19-2, 19-3

AFL labs
angr labs/CTF
Ch11, Ch12
Ch13, Ch15
Ch18, Ch 21

00, 01, 02 03, 04, 05, 06, 07
08, 09, 10, 11, 12, 13, 14 15, 16


Labs and notebook
Lab assignments will be given each class covering the course material. You will perform each one, while maintaining a lab notebook in a Google Doc that documents your progress via screenshots with your OdinID in them. The notebook should also include answers to any questions in the labs.  Notebooks should be exported as a PDF file and include a table of contents generated by Google Docs. Submission will be done via adding, committing and pushing the file to your private git repository. Use the following naming convention to submit your notebooks.
  • notebooks/<week_number>.pdf e.g. notebooks/Week1.pdf
The notebook will be graded based upon the following rubric:
  • Neatness and organization
  • Completeness
  • Inclusion of OdinID or project identifier in screenshots
Homework (MetaCTF)
For homework, we will be applying the concepts learned in the labs to Linux binaries. Assignments are to be done individually at the following sites: and Binaries are unique to each student and no collaboration is allowed. The binaries implement a set of capture-the-flag challenges that require you to reverse engineer a set of binary executables. Each binary asks for a password that will unlock it and print "Good Job". While you will be running binaries on your own machine, answers to each should be submitted at the above site.

To download binaries on a linuxlab machine, do the following:
mkdir metactf; cd metactf; virtualenv -p python3 env
source env/bin/activate
pip install requests bs4
python username password
If you wish, instead, to run the binaries on a Ubuntu VM of your own, directions for doing so are here. Note that you must ensure that radare2 and the 32-bit libraries are installed on the VM by performing
sudo apt-get install gcc-multilib radare2

Course objectives

  • Understand the underlying mechanisms used by malware on compromised systems.
  • Understand counter-measures that detect malware.
  • Understand techniques malware uses to circumvent and evade detection and analysis
  • Develop skills to monitor the behavior of malware safely.
  • Develop skills to reverse-engineer malware


Attendance 5%
Homeworks 50%
Lab notebook 45%
Academic misconduct
  • Includes allowing another student to copy your work unless specifically allowed by the instructor.
  • Results in a grade of 0 for the assignment or exam.
  • Results in the initiation of disciplinary action at the university level.