Automating Reverse Engineering with Machine Learning and Binary Analysis

Reverse engineering (RE) applications (e.g. malware and vulnerability analysis) have historically been a manual and time-intensive process performed by skilled practitioners. In this course, we will introduce, discuss, and demonstrate (via labs) how Binary Analysis and Machine Learning (ML) techniques can be leveraged to address automation and scaling challenges with respect to reverse engineering. In particular, we will introduce students to several prominent intermediate representation (IR) languages (i.e. VEX LLVM, and p-code) and show how the IR can be utilized to perform advanced static and dynamic analysis of desired firmware. Since advanced binary analysis can generates valuable meta-data about a target binary (or a collection of binaries), we will discuss and demonstrate how meta-data generated from a vast collection of binaries can be analyzed via Machine Learning techniques (e.g. clustering and classifying) to discover patterns and insights that inform and guide the automated reverse engineering process.