Tuesday, March 14, 2017

Python new file notification

This post will be short. It just explains how to get notified about new file being created.

I want to be notified when I downloaded malware. In the last post, I discussed gathering malware using maltrieve. This is not the only way to get samples. You can have something like BroIDS running on a network tap and use it to extract PE files or whatever files you want.

Besides just collecting samples, I want notifications in Python so I can do additional processing. For PE files, you can start doing static analysis, feature extraction for machine learning, or whatever else you want.

I could edit BroIDS file extraction script or maltrieve code to do additional processing but I like my solution better.

The solution is to use Python inotify library.

To install the inotify library, just run ‘pip install inotify’
Here’s my code:

from inotify import adapters
i = adapters.Inotify()
i.add_watch('/root/inotifytest/') #directory

for event in i.event_gen():
   if event is not None:
       (header, type_names, watch_path, filename) = event
       #according to man page "File opened for writing was closed."
       #We get notified after the file is downloaded.
       if type_names == ['IN_CLOSE_WRITE']:
           print "New file downloaded: " + watch_path + filename

Original code is from this post: https://dustinoprea.com/2015/04/24/using-inotify-to-watch-for-directory-changes-from-python/ I just modified it so I could watch for IN_CLOSE_WRITE.

In the screenshot below, you can see the script printing when new files are added. I was running maltrieve in another window.

Sunday, March 12, 2017

BSidesIndy 2017 CTF - Solving TREMENDOUS/starter

I went to BSidesIndy without the intention of participating in the CTF so I didn’t take my computer with me. I saw some of the people I knew playing so I decided to try to help but without my computer, I wasn’t much of a help.

I did this at home…

This is a reversing challenge.

I ran strings and got this.

Of course, that flag does not work…

We can move on to gdb and Hopper at this point.

Let’s start by figuring out when WRONG is used. We’ll call the block that print WRONG “wrong_func” for the sake of simplicity.

Below you can see that the JMP instruction jumps wrong_func.

Before the jump, you can strlen then comparison. If the comparison results in true, then we jump to 0x40066b, instead of going to wrong_func.

This is good. We can put a breakpoint at 0x400652 (je instruction) and examine the registers in gdb.

RAX is 0x20 (32) and RBX is 0x5 (5).
We know that we put in HELLO, which is 5 chars long. We need our input to be 32 char long for RAX to equal RBX.


Look at that. CMP results in true. The program will now jump to 0x40066b instead of going to wrong_func.

Even if we supply 32 char input, the program still go to wrong_func.

Let’s examine 0x40066b now.

It does bunch of stuff but I am not going to spend time trying to figure it out. I can just look at the call graph instead.

Notice the CMP under 0x400690. If CMP does not result in true, JNE 0x400656 is executed. That goes to nop, which then goes to wrong_func. Now we have to just avoid JNE from executing.

I do want to point out that we’re trying to get to 0x4006eb, which prints TREMENDOUS!. It will happen when we have the correct flag.

Back in GDB, we can put a breakpoint on JNE 0x400656 instruction.

In the picture above, you can see that RAX contains our input (A) and RDX contains expected input (I), since the CMP failed, we jump to nop and wrong_func is executed.

We can change our input to be IAAABBBBCCCCDDDDAAAABBBBCCCCDDDD and keep using the break point to figure out what our input should be but that’s not fun.

We’ll use Angr (angr.io) instead.

So far we know that input should be 32 char long. We also know that we want to AVOID going to 0x400656 (nop -> wrong_func) and 0x400657 (wrong_func) and we want to FIND a path to 0x4006eb, which prints “TREMENDOUS!”

I don’t know how to deal with args in Angr. Lintile(twitter.com/lintile), the CTF organizer, told me to look at some Angr CTF examples so I did. I found one that I could modify and get the flag with. Here is it is: https://github.com/angr/angr-doc/blob/master/examples/securityfest_fairlight/solve.py

This is the code that I used:

Argv1 is 32 char long.
I told Angr what I wanted to find and what I wanted to avoid.
After the path is found, print the argv1 required.

I ran the script and found the flag!

And yes, the flag does work.

Collecting malware using Maltrieve

I have a Raspberry Pi 2 at home, which has been sitting and collecting malware using open source tools. In this post, I will be using a virtual machine on ESXi instead and setting up the collection.

Main purpose for now is to just document how to setup collection server.

I will be using Ubuntu 16.04 Server and ESXi for this project.
I have a NAS containing all my malware, however, you don’t need to setup a dedicated NAS, if you don’t want to.
I will also be utilizing Tor. I want all my requests to go through Tor. It’s to protect me and to avoid alerts for the people monitoring my infrastructure.

VM Specs:
1 CPU core (I got Xeon E5507 @ 2.27GHz)
RAM: 1024MB (overkill but I plan to do more with this VM later)
Storage space: 8GB, again, I’m using a NAS for storage.

  1. Install Ubuntu server. I didn’t do anything special. I have two network interfaces. One on LAN (for internet connection) and one connected to directly to my NAS.
    1. Assign static IP to your interfaces, if you need to.
    2. Setup automounting NFS share (NAS) in fstab, if you need to.
  1. Run ‘apt-get update’
  2. Run ‘apt-get install python2.7 python2.7-dev python-pip’
    1. This will install required python packages.
    1. This is to clone the project we will be using to collect malware
    2. The repo was forked from https://github.com/krmaxwell/maltrieve
  3. Cd into maltrieve
  4. Run ‘pip install -e .’
  5. Edit maltrieve.cfg file and comment out cuckoo_dist, unless you’re going to use it.
Tor and Privoxy
  1. Run ‘apt-get install tor privoxy’
    1. By default privoxy runs HTTP proxy on port 8118 but you still need to configure it to forward traffic to tor
    2. Tor runs socks proxy on port 9050.
  2. To configure privoxy, edit /etc/privoxy/config
    1. Locate the line ‘# forward-socks5t / .’ then remove ‘#’ from the front
    2. Locate ‘listen-address localhost:8118’ and change it to ‘listen-address’
      1. For some reason, privoxy was only binding ipv6 address and not and maltrieve didn’t like that.
    3. Save the file
  3. Run ‘service tor start’
  4. Run ‘service privoxy start’

  1. Now test out the proxy by running the following commands
    1. curl ifconfig.co
    2. curl -x ifconfig.co
  2. For the results for above commands, you should get different IP addresses.

Now we have everything almost ready to go.
I am saving my malware data to /malware/samples and saving my log to /malware/mal.log.
Cron job will be used to download samples daily.

Edit your crontab file and add something similar to this:
00 03 * * * cd /home/malware/maltrieve && ./maltrieve.py -l "/malware/mal.log" -d "/malware/samples/" -p -s

This will run 3 AM every morning. It will also take all of the samples and put them in samples folder and organize them by MIME type.

You can test to see if this command works or not by running the command we put in crontab.

It seems to be downloading files. You can check the samples folder as well.

I have been running Maltrieve on my Pi for couple of months now. I'll find a place to share the samples soon. 

Wednesday, March 1, 2017

ClamAV & detection of a pcap file

I had bunch of pcap files that I created last year. I gave digital forensics students an assignment based on the data contained in the pcaps. ClamAV flagged a file as malicious.

The lab had ClamAV setup with schedule scanning enabled. It scans at night. The lab machines were also used to do the forensics exercise. I woke up the next day and got bunch of alerts from ClamAV (ClamWin to be specific) about one of my pcap file. I knew I had used metasploit when generating the pcap file but I wanted to know why it got detected.

This post investigates that. None of the techniques used in the post are new. AV signature creation about ClamAV nicely documented.

My setup: I am using Kali linux with ClamAV installed.

My first step is to figure out what malware ClamAV marks this file as.
I updated my ClamAV on Kali then scanned the file.
It’s labeled as Win.Exploit.Fnstenv_mov-1. This has to do with metasploit payload. I used metasploit during the creation of this pcap.

Now I need to look at the signature for Win.Exploit.Fnstenv_mov-1.
ClamAV stores signatures in /var/lib/clamav. The files have extension of .cvd.
I copied main.cvd and used the sigtool (this tool has been documented here https://linux.die.net/man/1/sigtool) to unpack it then looked for Win.Exploit.Fnstenv_mov-1.

To understand what “Win.Exploit.Fnstenv_mov-1:0:*:d9eed97424f45b817313{4}83ebfce2f4” means, I looked up how ClamAV signatures are created. I came across this blog post: "Create Your Own Anti-Virus Signatures with ClamAV" http://blog.adamsweet.org/?p=250

This is the format according to the blog post: “Name:Type:Offset:malware hex output”
Name: Win.Exploit.Fnstenv_mov-1
Type: 0 = Any file
Offset: * = Any
Hex: d9eed97424f45b817313

Next step was to search for the hex in Wireshark.

I don’t know how to get the payload. If anyone knows plz help me learn. I think I would need to look at the vulnerability again and figure out how metasploit sent the payload.

After googling D9EED97424F45B817313 I found this https://github.com/dzzie/pdfstreamdumper/blob/master/libemu/encoders.txt.
It’s part of encoding routine that metasploit uses (?) I popped it into ODA (https://www.onlinedisassembler.com) and it showed the assembly instructions used

If there are errors in this post, let me know. I should be sleeping right now.