Malware Reverse Engineering

Instructor: Wu-chang Feng
Class: T/Th 2:00pm-3:50pm EB 325
Contact and discussion: Office hours: here at Code Party
Required textbook
Resources

Schedule

Week Topic Slides Labs Homework
1 Introduction
Motivation
Basic Analysis
Chapter 1: Basic Static Techniques
00_Intro

01_BasicTechniquesTools

Install VM (video)
1-1, 1-2
Malware CTF site
Ch01, Ch03
2
Chapter 2: Malware Analysis in VMs
Chapter 3: Basic Dynamic Analysis
Advanced Static Analysis
Chapter 4: x86 Assembly
Chapter 5: IDA Pro
Chapter 6: C code in Assembly
02_C_x86_Windows
3-2, 3-4

5-1, 6-1, 6-2
Ch04, Ch06
3
Chapter 7: Malicious Windows Programs
Chapter 8: Debugging
Chapter 9: OllyDbg
03_Debugging
7-2
9-2
Ch08
4 Advanced Dynamic Analysis
Malware Functionality
Chapter 11: Malware Behavior (1-5)
Chapter 11 (6-7), Chapter 12: Covert Launching
04_Functionality
11-1
12-1, 12-3
Ch11, Ch12
5
Chapter 13: Data Encoding
Chapter 14: Network Signatures
Anti-Reverse-Engineering
Chapter 15: Anti-Disassembly
  Software Armoring


05_AntiReverse
talk
13-1, 14-1
15-1, 15-2
Ch13, Ch15
6
Chapter 16: Anti-Debugging
Chapter 17: Anti-VM Techniques
  BluePill

Talks #1,#2 | Slides #1,#2
16-1
17-1
Ch16
7
Chapter 18: Packers and Unpacking
Special Topics
Chapter 19: Shellcode Analysis
Chapter 20: C++ Analysis
Chapter 21: 64-bit Malware
Final project

06_Special
18-1, 19-2, 19-3

20-1
Final project
Ch18, Ch 21
8
Fuzzing
Symbolic execution/analysis
07_Fuzzing_SymbolicExecution

AFL labs
angr CTF Setup lab
angr CTF site
00, 01, 02
9
Symbolic execution/analysis
  angr CTF 03, 05, 06
08, 10, 11, 12, 13, 14
10
Symbolic execution/analysis
  angr CTF 15, 16
17
Lab notebook due in Canvas Friday @11:59pm (3/15)
Finals Final project screencast in MediaSpace, Final project screencast URL in Canvas, All CTF levels due Friday @11:59 (3/22)

Assignments

Lab notebook
Throughout the quarter, you will be maintaining a lab notebook that records your progress on lab assignments using any word document application that you are comfortable with such as a Google Doc (recommended), a Microsoft Word document, or a LibreOffice document. The lab notebook that will contain your write-ups of each lab. The write-up should only include answers to any questions and screenshots of the completed work (via gnome-screenshot, gimp, Print Screen, etc.) as listed in the bulleted lists associated for each lab on the lab site. You do not need to include those from the textbook's lab walkthroughs. The notebook will be graded using the following rubric:
  • Thoroughness and clarity of the write-ups.
  • Inclusion of your OdinId in all screenshots that are requested
  • Inclusion of a table of contents linking to individual labs completed
While you are encouraged to work together on labs, each student must submit an individual notebook at the end of the quarter. Any notebooks found with shared screenshots will result in a 0 for both notebooks.

Homework (MetaCTF)
For homework, we will be applying the concepts learned in the labs to Linux binaries. Assignments are to be done individually at the following sites: cs492.oregonctf.org and angr.oregonctf.org. Binaries are unique to each student and no collaboration is allowed. The binaries implement a set of capture-the-flag challenges that require you to reverse engineer a set of binary executables. Each binary asks for a password that will unlock it and print "Good Job". While you will be running binaries on your own machine, answers to each should be submitted at the above site. To download binaries on a Linux x86 machine, do the following:
mkdir metactf; cd metactf; virtualenv -p python3 env
source env/bin/activate
pip install requests bs4
wget http://thefengs.com/wuchang/courses/cs492/meta_dl.py
python meta_dl.py cs492.oregonctf.org username password
If you wish, instead, to run the binaries on a Ubuntu VM of your own, directions for doing so are here. Note that you must ensure that radare2 and the 32-bit libraries are installed on the VM by performing
sudo apt-get install gcc-multilib radare2

Course objectives

  • Understand the underlying mechanisms used by malware on compromised systems.
  • Understand counter-measures that detect malware.
  • Understand techniques malware uses to circumvent and evade detection and analysis
  • Develop skills to monitor the behavior of malware safely.
  • Develop skills to reverse-engineer malware

Policies

Grading
Attendance 5%
Homeworks 30%
Lab notebook 35%
Final project 30%
Academic misconduct
  • Includes allowing another student to copy your work unless specifically allowed by the instructor.
  • Results in a grade of 0 for the assignment or exam.
  • Results in the initiation of disciplinary action at the university level.