FIRMADYNE is an automated and scalable system for performing emulation and dynamic analysis of Linux-based embedded firmware. It incl 8000 udes the following components:
- modified kernels (MIPS: v2.6.32, ARM: v4.1, v3.10) for instrumentation of firmware execution;
- a userspace NVRAM library to emulate a hardware NVRAM peripheral;
- an extractor to extract a filesystem and kernel from downloaded firmware;
- a small console application to spawn an additional shell for debugging;
- and a scraper to download firmware from 42+ different vendors.
We have also written the following three basic automated analyses using the FIRMADYNE system.
- Accessible Webpages: This script iterates through each file within the filesystem of a firmware image that appears to be served by a webserver, and aggregates the results based on whether they appear to required authentication.
- SNMP Information: This script dumps the contents of the
publicandprivateSNMP v2c communities to disk using no credentials. - Vulnerability Check: This script tests for the presence of 60 known vulnerabilities using exploits from Metasploit. In addition, it also checks for 14 previously-unknown vulnerbailities that we discovered. For more information, including affected products and CVE's, refer to analyses/README.md.
In our 2016 Network and Distributed System Security Symposium (NDSS) paper, titled Towards Automated Dynamic Analysis for Linux-based Embedded Firmware, we evaluated the FIRMADYNE system over a dataset of 23,035 firmware images, of which we were able to extract 9,486. Using 60 exploits from the Metasploit Framework, and 14 previously-unknown vulnerabilities that we discovered, we showed that 846 out of 1,971 (43%) firmware images were vulnerable to at least one exploit, which we estimate to affect 89+ different products. For more details, refer to our paper linked above.
Note: This project is a research tool, and is currently not production ready. In particular, some components are quite immature and rough. We suggest running the system within a virtual machine. No support is offered, but pull requests are greatly appreciated, whether for documentation, tests, or code!
The following has been tested on a Ubuntu 14.04 machine. Other Debian-based systems should also be compatible.
First, clone this repository recursively and install its dependencies.
sudo apt-get install busybox-static fakeroot git kpartx netcat-openbsd nmap python-psycopg2 python3-psycopg2 snmp uml-utilities util-linux vlangit clone --recursive https://github.com/firmadyne/firmadyne.git
The extractor depends on the binwalk tool, so we need to install that and its dependencies.
git clone https://github.com/devttys0/binwalk.gitsudo ./binwalk/deps.shsudo python ./binwalk/setup.py install
- For Python 2.x,
sudo apt-get install python-lzma
sudo -H pip install git+https://github.com/ahupp/python-magic- Instead of upstream jefferson, it is recommended to install our jefferson fork, which supports extraction of additional file and compression types.
- Optionally, instead of upstream sasquatch, our sasquatch fork can be used to prevent false positives by making errors fatal.
Next, install, set up, and configure the database.
sudo apt-get install postgresqlsudo -u postgres createuser -P firmadyne, with passwordfirmadynesudo -u postgres createdb -O firmadyne firmwaresudo -u postgres psql -d firmware < ./firmadyne/database/schema
To download our pre-built binaries for all components, run the following script:
cd ./firmadyne; ./download.sh
Alternatively, refer to the instructions below to compile from source.
To use QEMU provided by your distribution:
sudo apt-get install qemu-system-arm qemu-system-mips qemu-system-x86 qemu-utils
Note that emulation of x86-based firmware is not currently supported, but installing
qemu-system-x86 resolves a packaging issue on certain Debian-based distributions.
Alternatively, use our modified version
of qemu-linaro for certain
firmware with an alphafs webserver that assumes a fixed memory mapping (not
recommended), or upstream qemu.
- Set
FIRMWARE_DIRinfirmadyne.configto point to the root of this repository. - Download a firmware image, e.g. v2.0.3 for Netgear WNAP320.
wget http://www.downloads.netgear.com/files/GDC/WNAP320/WNAP320%20Firmware%20Version%202.0.3.zip
- Use the extractor to recover only the filesystem, no kernel (
-nk), no parallel operation (-np), populating theimagetable in the SQL server at127.0.0.1(-sql) with theNetgearbrand (-b), and storing the tarball inimages../sources/extractor/extractor.py -b Netgear -sql 127.0.0.1 -np -nk "WNAP320 Firmware Version 2.0.3.zip" images
- Identify the architecture of firmware
1and store the result in theimagetable of the database../scripts/getArch.sh ./images/1.tar.gz
- Load the contents of the filesystem for firmware
1into the database, populating theobjectandobject_to_imagetables../scripts/tar2db.py -i 1 -f ./images/1.tar.gz
- Create the QEMU disk image for firmware
1.sudo ./scripts/makeImage.sh 1
- Infer the network configuration for firmware
1. Kernel messages are logged to./scratch/1/qemu.initial.serial.log../scripts/inferNetwork.sh 1
- Emulate firmware
1with the inferred network configuration. This will modify the configuration of the host system by creating a TAP device and adding a route../scratch/1/run.sh
- The system should be available over the network, and is ready for analysis. Kernel messages are logged to
./scratch/1/qemu.final.serial.log../analyses/snmpwalk.sh 192.168.0.100./analyses/webAccess.py 1 192.168.0.100 log.txtmkdir exploits; ./analyses/runExploits.py -t 192.168.0.100 -o exploits/exploit -e x(requires Metasploit Framework)sudo nmap -O -sV 192.168.0.100
- To access a console in the firmware, use a presupplied debug run script to access the default console (no network access), modify the network-enabled
run.shscript to provide console access, or use the second console provided by the framework.
./scripts/run-debug.sh 1nc -U /tmp/qemu.1.S1
- The following scripts can be used to mount/unmount the filesystem of firmware
1. Ensure that the emulated firmware is not running, and remember to unmount before performing any other operations.
sudo ./scripts/mount.sh 1sudo ./scripts/umount.sh 1
If you would like to compile the entire FIRMADYNE system from scratch without using our pre-built binaries, please follow the steps below.
In order to build any of the binaries used by FIRMADYNE, you will need three cross-compilation toolchains for the following architecture triples. Use only musl libc as the C runtime library for the toolchain; others have not been tested.
- arm-linux-musleabi
- mipseb-linux-musl
- mipsel-linux-musl
To simplify the process of building cross-compilation toolchains with musl, we recommend using the musl-cross project. Follow the below steps to build these toolchains from source, or alternatively click here to download our pre-built toolchains.
-
git clone https://github.com/GregorR/musl-cross.git -
Modify or set the following variables in
defs.shBINUTILS_URL=http://ftp.gnu.org/gnu/binutils/binutils-2.25.1.tar.bz2GCC_VERSION=5.3.0GMP_VERSION=6.0.0aMPC_VERSION=1.0.2MPFR_VERSION=3.1.3LIBELF_VERSION=masterMUSL_DEFAULT_VERSION=1.1.12MUSL_GIT_VERSION=615629bd6fcd6ddb69ad762e679f088c7bd878e2LANG_CXX=no
-
Modify or set the following variables in
config.shCFLAGS="-fPIC"
-
For little-endian MIPS, perform the following:
- set
TRIPLE=mipsel-linux-muslinconfig.sh - set
LINUX_HEADERS_URL=https://kernel.org/pub/linux/kernel/v2.6/longterm/v2.6.32/linux-2.6.32.70.tar.xzindefs.sh - run
./clean.shto clean out any previous builds - run
./build.shto build and install the toolchain into/opt/cross
- set
-
For big-endian MIPS, perform the following:
- set
TRIPLE=mipseb-linux-muslinconfig.sh - set
LINUX_HEADERS_URL=https://kernel.org/pub/linux/kernel/v2.6/longterm/v2.6.32/linux-2.6.32.70.tar.xzindefs.sh - run
./clean.shto clean out any previous builds - run
./build.shto build and install the toolchain into/opt/cross
- set
-
For little-endian ARM, perform the following:
- set
TRIPLE=arm-linux-musleabi,GCC_BOOTSTRAP_CONFFLAGS="--with-arch=armv6 --with-float=softfp", andGCC_CONFFLAGS="--with-arch=armv6 --with-float=softfp"inconfig.sh - set
LINUX_HEADERS_URL=https://kernel.org/pub/linux/kernel/v4.x/linux-4.1.17.tar.xzindefs.sh - run
./clean.shto clean out any previous builds - run
./build.shto build and install the toolchain into/opt/cross
- set
-
You should have the following directories, or wherever you installed the toolchains:
/opt/cross/arm-linux-musleabi/opt/cross/mipseb-linux-musl/opt/cross/mipsel-linux-musl
cd ./firmadyne/sources/consolemake clean && CC=/opt/cross/arm-linux-musleabi/bin/arm-linux-musleabi-gcc make && mv console ../../binaries/console.armelmake clean && CC=/opt/cross/mipseb-linux-musl/bin/mipseb-linux-musl-gcc make && mv console ../../binaries/console.mipsebmake clean && CC=/opt/cross/mipsel-linux-musl/bin/mipsel-linux-musl-gcc make && mv console ../../binaries/console.mipsel
cd ./firmadyne/sources/libnvrammake clean && CC=/opt/cross/arm-linux-musleabi/bin/arm-linux-musleabi-gcc make && mv libnvram.so ../../binaries/libnvram.so.armelmake clean && CC=/opt/cross/mipseb-linux-musl/bin/mipseb-linux-musl-gcc make && mv libnvram.so ../../binaries/libnvram.so.mipsebmake clean && CC=/opt/cross/mipsel-linux-musl/bin/mipsel-linux-musl-gcc make && mv libnvram.so ../../binaries/libnvram.so.mipsel
git clone https://github.com/firmadyne/kernel-v4.1.git && cd kernel-v4.1mkdir -p build/armelcp config.armel build/armel/.configmake ARCH=arm CROSS_COMPILE=/opt/cross/arm-linux-musleabi/bin/arm-linux-musleabi- O=./build/armel zImage -j8cp build/armel/arch/arm/boot/zImage ../firmadyne/binaries/zImage.armel
-
git clone https://github.com/firmadyne/kernel-v2.6.32.git && cd kernel-v2.6.32 -
For big-endian MIPS, perform the following:
mkdir -p build/mipsebcp config.mipseb build/mipseb/.configmake ARCH=mips CROSS_COMPILE=/opt/cross/mipseb-linux-musl/bin/mipseb-linux-musl- O=./build/mipseb -j8cp build/mipseb/vmlinux ../firmadyne/binaries/vmlinux.mipseb
-
For little-endian MIPS, perform the following:
mkdir -p build/mipselcp config.mipsel build/mipsel/.configmake ARCH=mips CROSS_COMPILE=/opt/cross/mipsel-linux-musl/bin/mipsel-linux-musl- O=./build/mipsel -j8cp build/mipsel/vmlinux ../firmadyne/binaries/vmlinux.mipsel
During development, the database was stored on a PostgreSQL server.
Although we cannot redistribute binary firmware, the data used for our experiments is available here.
Below are descriptions of tables in the schema.
brand: Stores brand names for each vendor.
| Column | Description |
|---|---|
| id | Primary key |
| name | Brand name |
image: Stores information about each firmware image.
| Column | Description |
|---|---|
| id | Primary key |
| filename | File name |
| brand_id | Foreign key to brand |
| hash | MD5 |
| rootfs_extracted | Whether the primary filesystem was extracted |
| kernel_extracted | Whether the kernel was extracted |
| arch | Hardware architecture |
| kernel_version | Version of the extracted kernel |
object: Stores information about each file in a filesystem.
| Column | Description |
|---|---|
| id | Primary key |
| hash | MD5 |
object_to_image: Maps unique files to their firmware images.
| Column | Description |
|---|---|
| id | Primary key |
| oid | Foreign key to object |
| iid | Foreign key to image |
| filename | Full path to the file |
| regular_file | Whether the file is regular |
| permissions | File permissions in octal |
| uid | Owner's user ID |
| gid | Group's group ID |
product
| Column | Description |
|---|---|
| id | Primary key |
| iid | Foreign key to image |
| url | Download URL |
| mib_filename | Filename of the SNMP MIB |
| mib_hash | MD5 of the SNP MIB |
| mib_url | Download URL of the SNMP MIB |
| sdk_filename | Filename of the source SDK |
| sdk_hash | MD5 of the source SDK |
| sdk_url | Download URL of the source SDK |
| product | Product name |
| version | Version string |
| build | Build string |
| date | Release date |
The results discussed in our paper were produced using pre-release versions of the following:
- toolchains:
BINUTILS_URL=http://ftp.gnu.org/gnu/binutils/binutils-2.25.1.tar.bz2,GCC_VERSION=4.9.3,GMP_VERSION=6.0.0a,MPC_VERSION=1.0.2,MPFR_VERSION=3.1.3,LIBELF_VERSION=71bf774909fd654d8167a475333fa8f37fbbcb5d,MUSL_DEFAULT_VERSION=1.1.10,MUSL_GIT_VERSION=996d148bf14b477b07fa3691bffeb930c67b2b62,LANG_CXX=no- ARM:
LINUX_HEADERS_URL=https://kernel.org/pub/linux/kernel/v3.x/linux-3.10.84.tar.xz - MIPS:
LINUX_HEADERS_URL=https://kernel.org/pub/linux/kernel/v2.6/longterm/v2.6.32/linux-2.6.32.67.tar.xz
- kernels:
- ARM: firmadyne-v3.10.92
- MIPS: firmadyne-v2.6.32.68 without
e2b9f315547ea50a65baad4899a4780078ab273eand26bb3636c987fc7e145af73ddea6c10fa93bdae9
- console:
c36ae8553fa4e9c82e8a65752906641d81c2360c - extractor:
5520c64bfa8554c5c17ab671aaed0fdeec91bf19 - libnvram:
b60e7d4d576b39dd46107058adb635d43e80e00d - qemu-linaro:
4753f5e8126a00cc0a8559bfd9b47d6340903323 - binwalk:
f2ce2992695fae5477c46980148c89e9c91a5cce- jefferson:
090a33be0be4aac8eee8d825447c0eb18dc8b51a - sasquatch:
287e4a8e059d3ee7a5f643211fcf00c292cd6f4d
- jefferson: